Wednesday, 2022-08-03

corvusfyi my reply in gerrit just took like 30-60 seconds.00:50
ianwhrm, just saw this; cpu and load average graphs look normal02:08
ianwperhaps there was a bit of a traffic spike around then02:09
ianwquite likely i guess03:52
opendevreviewMerged openstack/project-config master: Add Grafana dashboard for the Tobiko CI
*** ysandeep|out is now known as ysandeep05:58
opendevreviewIan Wienand proposed openstack/project-config master: project-config-grafyaml: drop job
opendevreviewIan Wienand proposed openstack/project-config master: grafana: helper script updates
opendevreviewwangxiyuan proposed openstack/project-config master: Bump openEuler to 22.03 LTS
*** jpena|off is now known as jpena07:14
fricklerI'm seeing query errors for all the graphs on the AFS page while the timer panels seem to work fine
*** ysandeep is now known as ysandeep|mtg07:57
*** anbanerj is now known as frenzyfriday|rover07:59
*** ysandeep|mtg is now known as ysandeep08:37
*** tosky_ is now known as tosky08:47
*** ysandeep is now known as ysandeep|lunch09:04
*** ysandeep|lunch is now known as ysandeep10:00
*** rlandy|out is now known as rlandy10:37
opendevreviewjayaditya gupta proposed openstack/diskimage-builder master: Closes-Bug: 1974350
*** bhagyashris is now known as bhagyashris|ruck11:11
*** dviroel|afk is now known as dviroel11:23
*** ysandeep is now known as ysandeep|afk12:25
*** ysandeep|afk is now known as ysandeep12:25
*** dasm|off is now known as dasm13:18
*** pojadhav is now known as pojadhav|afk13:31
*** frenzyfriday|rover is now known as frenzyfriday|rover|lunch13:36
*** frenzyfriday|rover|lunch is now known as frenzyfriday|rover14:28
clarkbfrickler: 'Query error: -1 Request was aborted' is reported by the grafana page. If I look at the graphite metrics browser both metrics are there. I suspect the problem is with the construction of the relative percentage calculation as a result14:43
clarkbmaybe we can't refer to things using #A as a refid anymore?14:44
*** dviroel is now known as dviroel|lunch14:48
fricklerclarkb: I noticed that other pages seem broken in the same way, did we update grafana recently?14:48
clarkbimage: <- we may haev14:49
fricklerREPOSITORY            TAG            IMAGE ID       CREATED         SIZE14:49
fricklergrafana/grafana-oss   latest         1e5c4e689218   28 hours ago    296MB14:49
fricklerseems like it14:49
fricklerdocker log has a lot of: level=warn msg="Request Origin is not authorized"14:52
fricklerbut not sure if related14:52
fricklerdo we want to try and revert to an older container for now? seems the one on the server got pruned, though14:52
Clark[m]All of the state is in graphite not grafana so in theory that is a safe thing to try14:54
Clark[m]Might be worth root causing the actual failure first if we can. I've got errands I have to do today unfortunately though14:54
*** ysandeep is now known as ysandeep|out15:07
odyssey4mehey folks - not sure who reviews JJB things any more, but there's a patch up which could do with some eyes:
fricklerthe latest CI snapshots in ianw's patch look broken the same way
fricklerhumm, I tried running with the most recent stable container (9.0.6) while our :latest seems to match 9.1.0-beta1 on , now all dashboards are gone, even after running with :latest again15:23
fricklerI'm hoping they'll get restored with the next hourly deployment15:23
fricklerif not, I'll need to dig further or maybe wait for ianw to come to the rescue15:24
Clark[m]Wow they tag beta images with latest? That is not ideal15:27
fricklerthat's surprising, yes, but you can see it in the docker log, too
fricklerthe last working run of that job from a week ago has 9.0.415:30
fungiodyssey4me: they have their own channel, #openstack-jjb (not sure if any of the maintainers pay attention in there either though)15:32
odyssey4mefungi right, thanks - will go poking there - thanks!15:34
fricklero.k., restored the dashboard with the command used to import them in the test job. seems to be working fine now with 9.0.4, I assume we will be able to debug further in CI. I'll make a system-config patch to pin the version for now15:46
opendevreviewMerged openstack/project-config master: Add OpenStack K8S charms
fungithanks frickler!15:48
opendevreviewDr. Jens Harbott proposed opendev/system-config master: Pin version of grafana-oss container
fricklerfungi: clarkb: ianw: ^^ I'll stop at that for today, I can continue tomorrow if needed15:54
*** marios is now known as marios|out15:57
*** dviroel|lunch is now known as dviroel16:13
*** jpena is now known as jpena|off16:24
*** frenzyfriday|rover is now known as frenzyfriday|rover|afk16:47
hjensasianw: since tripleo is using separate /boot partition in images. We got this bug reported, - for some reason the label is "mkfs_boot" and not just "boot". if I'm not mistaken this is the code to look at
hjensas59 ? so I would expect label == name unless label is set?16:49
opendevreviewMerged opendev/system-config master: Pin version of grafana-oss container
*** frenzyfriday|rover|afk is now known as frenzyfriday|rover18:58
*** rlandy is now known as rlandy|brb19:24
fungipip 22.2.2 just went out
fungilooks like it's all bug fixes for problems i haven't encountered though19:38
*** rlandy|brb is now known as rlandy20:07
*** dviroel is now known as dviroel|afk20:34
*** timburke__ is now known as timburke20:46
*** rlandy is now known as rlandy|out21:32
ianwfrickler/fungi: thanks for looking at it :)  not sure i have any rescue steps :/  I tried running grafyaml directly against python3.10 late yesterday (usually i use the container) and unfortunately voluptuous doesn't work with it, and is pinned to an ancient version, and unpinning it doesn't work and it's not an obvious fix22:09
ianwso i mean it's fine in it's 3.8 container, but it's not going to magically get better22:10
ianwyou know i have some vague memory from looking at things that the datasource setup we use is deprecated for some reason, i wonder if it finally got pulled out22:11
clarkbfungi: do you know the answer for the https question on ?22:50
clarkbI guess I can update it and see if it works22:55
opendevreviewClark Boylan proposed opendev/system-config master: Run our base playbook on jammy
opendevreviewClark Boylan proposed opendev/system-config master: Ensure /etc/apt/sources.list.d exists
opendevreviewClark Boylan proposed opendev/system-config master: WIP Add a mailman3 list server
ianwsorry just had to stop the v6 dhcp client, reset the duid and restart, twice, to get ipv6 back.  just totally normal things that everyone should be able to do23:10
ianwright, so 23:12
ianwDeprecation Notice23:12
ianwThis data source uses browser access mode. This mode is deprecated and will be removed in the future. Please use server access mode instead.23:12
ianwis what i was thinking of23:12
ianwre grafana23:12
ianwi think the difference is that the browser doesn't talk to graphite, it gets proxied through grafana23:13
ianwthat seems unlikley to be it;
ianwstill shows the settings23:18
*** dasm is now known as dasm|off23:26
*** undefined is now known as Guest704723:34
ianw9.0.6 works, so it is something to do with the beta23:53
fungiclarkb: we probably can use https there, i'm not opposed (but also it provides little real benefit since the package manager is going to verify checksums from a list signed by a key in its trusted keyring, so at most it provides a questionable amount of privacy)23:58

Generated by 2.17.3 by Marius Gedminas - find it at!