Friday, 2023-02-17

*** soniya29 is now known as soniya29|ruck03:36
*** soniya29|ruck is now known as soniya29|ruck|lunch07:57
*** jpena|off is now known as jpena08:36
*** soniya29|ruck|lunch is now known as soniya29|ruck08:54
opendevreviewyatin proposed openstack/tempest master: Use assertLessEqual for downtime validation  https://review.opendev.org/c/openstack/tempest/+/87423212:03
ykareldansmith, when you get chance can you check comment https://bugs.launchpad.net/glance/+bug/199980012:17
*** artom_ is now known as artom12:31
dansmithykarel: ah yep, I never saw it fail there so it wasn't on my radar, but makes sense14:33
dansmithykarel: did you already start another patch?14:33
opendevreviewDan Smith proposed openstack/tempest master: Add retries to more glance location tests  https://review.opendev.org/c/openstack/tempest/+/87426014:57
dansmithykarel: ^14:57
dansmithhopefully that mean the retry loop is working for the other failures and just put pressure on these to fail14:58
ykareldansmith, Thanks and nope i didn't start any patch :)15:13
dansmithykarel: ack15:13
opendevreviewMerged openstack/devstack master: Added recursive for deletion of $OVN_RUNDIR  https://review.opendev.org/c/openstack/devstack/+/83720716:30
clarkbre gate troubles, I figured I'd remind ya'll that we can stick gate bugfixes at the beginning of the gate to reduce the thrash if they are stuck behind changes that are less likely to pass16:50
opendevreviewDan Smith proposed openstack/tempest master: Add retries to more glance location tests  https://review.opendev.org/c/openstack/tempest/+/87426016:52
clarkbdoing a reshuffle like that kicks out the currently running gate jobs, reorders git repo commits, then starts again. Which means if we can batch them up that is ideal. Also if things are stale enough with it then not doing it is also good16:53
dansmithclarkb: you're saying just promote recently-approved fixes to the front of the gate?16:54
dansmithI thought that was generally frowned-upon except for emergencies, but if not, that seems like a good strategy to me16:54
clarkbits frowned upon if the cost of evicting all running jobs and restarting everything is higher than just letting things run. I don't know where we are on that cost benefit right now.16:55
clarkbgate thrashing tends to have a high impact though and if reviewers aren't prioritizing fixes then we can prioritzie for them :)16:56
dansmithack16:56
dansmithI guess I'm more thinking of promoting not-yet-passing-tests changes as the real risk16:57
dansmithI think we're well past the point of worrying about thrashing the gate by promoting fixes to the front of the queue16:57
dansmithmeaning, I definitely think it'd be worth doing that16:57
dansmiththis past weekend a bunch of us spent all (friggin) weekend trying to land one fix that was 100% blocking several projects because it would keep failing something unrelated in check, gate, etc16:58
dansmithcontinually promoting that to the front of the queue would have no doubt made it much quicker16:58
*** jpena is now known as jpena|off17:01
opendevreviewFrank Ritchie proposed openstack/tempest master: Allow capital letters in mac address regex  https://review.opendev.org/c/openstack/tempest/+/87397317:12
gmanndansmith: mysql thigns, you can try enabling in tempet periodic jobs https://github.com/openstack/tempest/blob/b915c7f35ff105d8ac613cc171cd9235a96eac8d/zuul.d/project.yaml#L18617:59
dansmithokay, how often are those run?18:00
gmanni think every night18:01
dansmithoh okay18:01
gmannand they are in experimental also so on demand run18:01
dansmithbut .. we'd have to merge the devstack patch first yeah?18:01
dansmithdo you want me to remove it from devstack-multinode and only do the periodic for now?18:02
gmannyes, i have approved that18:02
gmanndevstack-multonode is ok to verify it in devstack  gate18:02
dansmithoh okay18:04
dansmithcool, I'll make that tempest change18:04
gmannthanks18:06
dansmithgmann: so those jobs listed in periodic aren't run on demand anywhere else?18:07
dansmithand are you thinking *all* of them or just a few for sniffing? maybe -all  -slow-parallel -full-parallel?18:08
gmanndansmith: i do not think they run anywhere else than tempest. yes in  -all  -slow-parallel -full-parallel only will be good18:09
dansmithgmann: cool18:11
opendevreviewDan Smith proposed openstack/tempest master: Use MYSQL_REDUCE_MEMORY in some periodic jobs  https://review.opendev.org/c/openstack/tempest/+/87365318:15
dansmithgmann: this passed the (relevant portion) of ceph-multistore: https://review.opendev.org/c/openstack/tempest/+/87426019:00
dansmithtl;dr: there are a couple more places where we can hit that glance problem, and now that we're retrying the two I changed, we're hitting the others (which I assume means the retries are working)19:01
gmannack, will check it19:02
opendevreviewArtom Lifshitz proposed openstack/hacking master: Bump flake8 to 5.0 to pick up fixes  https://review.opendev.org/c/openstack/hacking/+/87373819:07
dansmithdude, +1 from zuul first try.. that has to be some sort of record lately19:49
* dansmith buys a lottery ticket19:49
gmann:)20:02
gmanndansmith: is this blocking gate ? else I can leave kopecmartin to +W https://review.opendev.org/c/openstack/tempest/+/87426021:08
dansmithgmann: not blocking, this just reduces some more occasional fails21:09
gmanndansmith: ack21:09
opendevreviewFrank Ritchie proposed openstack/tempest master: Allow capital letters in mac address regex  https://review.opendev.org/c/openstack/tempest/+/87397321:19

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!