| slaweq | dansmith hi, fedora job passed again today. I checked what rpm packages are installed in the job which passed and one which failed (from Wednesday). The only difference is osinfo-db-20220214-1.fc35.noarch vs. osinfo-db-20220516-1.fc35.noarch | 07:26 |
|---|---|---|
| knikolla | i don't think i'll be making it to this summit afterall :( my partner tested positive to covid and i'm also starting to feel some symptoms today. | 13:07 |
| fungi | oof, sorry to hear that! i hope you both feel better soon | 13:08 |
| knikolla | thank you! such terrible timing. | 13:08 |
| fungi | i'm sure you won't be the only one in that situation, unfortunately | 13:09 |
| rosmaita | knikolla: sorry to hear that, hope you & your partner feel better soon | 13:12 |
| fungi | better to discover it now rather than when you get there (and then being stuck for weeks before you can come home) | 13:12 |
| dansmith | slaweq: well, osinfo is related to libvirt but shouldn't be to qemu, but.. interesting | 13:29 |
| dansmith | slaweq: I guess the best thing to do is formalize my dump collector so we're ready if it crops up again | 13:29 |
| slaweq | dansmith yes, and sorry for the noise yesterday but it was failing every day for at least a week :) | 13:32 |
| dansmith | slaweq: no worries :) | 13:34 |
| gmann | knikolla: sorry to hear that, please take care of you and your partner. your health is more important. | 13:55 |
| slaweq | knikolla sorry to hear that, I wish You both all the best | 14:53 |
| rosmaita | gmann: yesterday you mentioned that stable/wallaby grenade job failing for "ERROR: Links are not allowed as constraints" ... does anyone have an idea of what's going on with that? | 15:40 |
| gmann | rosmaita: yeah, i rebased the backport and that is fixing this issue https://review.opendev.org/c/openstack/devstack/+/805008 | 15:41 |
| gmann | rosmaita: but seems ceph job failed there in gate | 15:41 |
| rosmaita | gmann: thanks! | 15:41 |
| rosmaita | gmann: looks like that failure is oom_reaper killing mysqld | 15:45 |
| gmann | ok | 16:23 |
| dansmith | rosmaita: mysql is generally not the problem, it's just the biggest single user (naturally) so the oom killer picks it over anything else | 16:42 |
| dansmith | no performance.json in that job for some reason :/ | 16:43 |
| fungi | did you manage to get stats on aggregate apache/httpd memory allocations? | 16:44 |
| dansmith | fungi: waiting for review https://review.opendev.org/c/openstack/devstack/+/843491 | 16:45 |
| fungi | ahh, not for that job anyway | 16:45 |
| dansmith | on a normal system I'd say we should set mysql | 16:45 |
| dansmith | 's oom score lower so it doesn't get picked | 16:45 |
| dansmith | doesn't help much on a CI job, although it might make it more obvious if the oom killer picks the next-biggest thing | 16:45 |
| fungi | +1 on the devstack change to collect apache stats, i'm not a core reviewer tho | 16:45 |
| dansmith | fungi: thanks.. I'm not sure why that ceph job didn't get stats run though, maybe it overrides the roles or something | 16:46 |
| clarkb | +2 from me | 16:49 |
| opendevreview | Merged openstack/governance master: Add project stats check tool https://review.opendev.org/c/openstack/governance/+/810037 | 17:28 |
| opendevreview | Merged openstack/governance master: Add resolution for OpenStack release identification schema https://review.opendev.org/c/openstack/governance/+/843214 | 17:29 |
| opendevreview | Merged openstack/governance master: Documenting the OpenStack release new identification process https://review.opendev.org/c/openstack/governance/+/841800 | 17:33 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!