| *** ykarel__ is now known as ykarel | 04:28 | |
| opendevreview | Joan Gilabert proposed openstack/releases master: Release Watcher stable/2025.2 for several bug fixes https://review.opendev.org/c/openstack/releases/+/976720 | 08:45 |
|---|---|---|
| frickler | release-team: we keep on getting job timeouts on https://review.opendev.org/c/openstack/releases/+/976622 , I'm assuming this is due to general opendev.org slowness, not sure what we can do about it? ask stephenfin to split this into pieces? or override somehow? | 13:23 |
| elodilles | i guess the easiest workaround is to split the patch to multiple release patches (probably no need for a separate patch for every single xstatix-* deliverable, though) | 13:32 |
| elodilles | the question is whether this is a new slowness that would impact our coordinated release on April 1st | 13:33 |
| elodilles | checking on a random job that is not the usual 2-4 mins log, but 17 mins, i see these steps took extra long: https://paste.opendev.org/show/bkmbqLra5QPX3DbZuC1k/ | 13:39 |
| elodilles | so yeah, it seems that sometimes there are ~2 mins lags at some points during the job run | 13:43 |
| ttx | o/ | 14:00 |
| ttx | #startmeeting releaseteam | 14:00 |
| opendevmeet | Meeting started Fri Feb 13 14:00:26 2026 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'releaseteam' | 14:00 |
| ttx | Ping list: release-team elod | 14:00 |
| ttx | Our agenda is at https://etherpad.opendev.org/p/gazpacho-relmgt-tracking#L270 | 14:01 |
| elodilles | o/ | 14:01 |
| ttx | Hi elodilles ! | 14:01 |
| ttx | #topic Review R-7 week task completion | 14:01 |
| ttx | - Notify the Infrastructure team to generate an artifact signing key (but not replace the current one yet), and begin the attestation process (ttx) | 14:01 |
| ttx | fungi: consider yourself notified! | 14:02 |
| ttx | - Check with the Technical Committee to make sure Python runtimes have been determined for the next development cycle (elod) | 14:03 |
| elodilles | (elod) i've pinged TC about this via their IRC channel | 14:03 |
| elodilles | in fact, | 14:03 |
| elodilles | the patch is up for reviews: | 14:03 |
| elodilles | https://review.opendev.org/c/openstack/governance/+/976691 | 14:03 |
| elodilles | though usually it takes some time and discussions until it gets settled | 14:04 |
| fungi | thanks ttx | 14:04 |
| frickler | \o | 14:04 |
| fungi | planning to work on it today actually | 14:05 |
| ttx | - Propose DNM changes on repositories where no patches merged recently to check that tests are still passing with the current set of dependencies (libraries, client libraries) (elod) | 14:05 |
| elodilles | (elod) due to the current broken gates (setuptools 82.0.0 release) there are probably many broken gates. i've only restored a couple of health check patches to see the state there: https://review.opendev.org/q/topic:release-health-check-gazpacho+is:open | 14:05 |
| elodilles | and actually it doesn't look that bad | 14:05 |
| ttx | good timing :) | 14:06 |
| fungi | it could have been waaaay worse, but we were extremely well-prepared for that actually | 14:06 |
| ttx | OK let's keep track of those | 14:06 |
| elodilles | we might need to re-iterate when all the known fixes landed | 14:06 |
| ttx | right | 14:06 |
| ttx | - Send weekly email (ttx) | 14:06 |
| ttx | Will do shortly after the meeting | 14:07 |
| ttx | #topic Assign R-6 week tasks | 14:07 |
| elodilles | since you'll be away, i'm taking both meeting chair and other tasks :) | 14:08 |
| opendevreview | Stephen Finucane proposed openstack/reno master: scanner: Decode sha before printing https://review.opendev.org/c/openstack/reno/+/976737 | 14:08 |
| ttx | Thanks to my great skills at vacation timing I won't be able to take tasks in the next two weeks | 14:08 |
| opendevreview | Stephen Finucane proposed openstack/reno master: Handle missing versions https://review.opendev.org/c/openstack/reno/+/976738 | 14:08 |
| ttx | sorry about that :) | 14:08 |
| elodilles | no problem :] | 14:08 |
| elodilles | vacations are essential ;) | 14:08 |
| ttx | #topic Review weekly countdown email | 14:09 |
| ttx | #link https://etherpad.opendev.org/p/relmgmt-weekly-emails | 14:09 |
| elodilles | LGTM | 14:11 |
| ttx | alright will send shortly | 14:11 |
| frickler | I edited the last line to make it consistent | 14:11 |
| ttx | #topic Open Discussion | 14:11 |
| ttx | Nothing from me except the aforementioned impeding vacation | 14:12 |
| elodilles | i think we need to mention here a couple of things | 14:12 |
| fungi | hope you get rested and recharged | 14:12 |
| ttx | thanks frickler | 14:12 |
| ttx | I might never come back | 14:12 |
| elodilles | so there are the problems around the new setuptools release | 14:13 |
| elodilles | for now (for us) the master branch issues are the most problematic ones | 14:13 |
| elodilles | though a lot of them got fixed meanwhile | 14:13 |
| frickler | reqs bumps are still blocked though | 14:14 |
| elodilles | one that still lingers is regarding horizon, which pulls in xstatic-* deliverables and those are broken as they're missing pkg_resources module :/ | 14:14 |
| frickler | also cinder with os_win | 14:14 |
| elodilles | and that ^^^ | 14:15 |
| elodilles | so the xstatic-* deliverables have been fixed (thanks to stephenfin !), but they need to be released | 14:15 |
| elodilles | and the release patches gets time out on the list-changes job :/ | 14:16 |
| fungi | some of them only just got imported into gerrit yesterday | 14:16 |
| frickler | iiuc the first set of 28 or so repos is only the "old" ones | 14:16 |
| elodilles | as i wrote before the meeting: | 14:17 |
| frickler | we could maybe also just bump the timeout for this job from 1h to like 3h and see if that is enough? | 14:17 |
| elodilles | the question is whether this is a new slowness that would impact our coordinated release on April 1st | 14:17 |
| ttx | why is it taking so long? infinite number of changes? | 14:17 |
| elodilles | checking on a random job that is not the usual 2-4 mins log, but 17 mins, i see these steps took extra long: https://paste.opendev.org/show/bkmbqLra5QPX3DbZuC1k/ | 14:17 |
| elodilles | so yeah, it seems that sometimes there are ~2 mins lags at some points during the job run | 14:18 |
| elodilles | so for now, probably increasing the job timeout would do the trick | 14:18 |
| frickler | it pretty likely IMO that the reason for this is opendev.org getting AIdosed | 14:19 |
| elodilles | but i think if this issue will be present around release day, then we might not be able to keep our time schedule there | 14:19 |
| fungi | actually, it looks like https isn't making it to vexxhost's sjc1 region from some parts of the internet | 14:20 |
| fungi | so that's probably ipv6 timing out and then getting automatically retried over ipv4 | 14:20 |
| frickler | oh, so that would depend on which cloud the job is running on | 14:21 |
| fungi | tcp in general, in fact. ssh times out too | 14:21 |
| stephenfin | I was going to say, this sounds familiar | 14:21 |
| fungi | well, no, vexxhost sjc1 is where our giteaz servers are | 14:21 |
| * stephenfin also can't access opendev.org again today | 14:22 | |
| fungi | icmp6 is also not making it through | 14:22 |
| ttx | ok so we can hope this issue will be transient and fixed by release time | 14:24 |
| fungi | seems to be from everywhere on the internet i'm testing this time | 14:24 |
| ttx | elodilles: anything else you wanted to flag? | 14:24 |
| elodilles | one more thing maybe | 14:25 |
| elodilles | that i'd appreciate some reviews on these: https://review.opendev.org/q/topic:2026.1-runtimes o:) | 14:26 |
| elodilles | and that's all from me (at least that was all that came into my mind) | 14:26 |
| elodilles | oh, and maybe another note | 14:27 |
| elodilles | that requirements' gate is also broken (still, if i'm not mistaken) | 14:27 |
| elodilles | and some of the cross-<project>-jobs as well | 14:27 |
| ttx | ...ok | 14:27 |
| elodilles | so we can prepare for longer delays with upper-constraint bumps if those don't get sorted out soon | 14:28 |
| ttx | Ok, noted! | 14:29 |
| elodilles | and that's really all that i had in my mind | 14:29 |
| ttx | Anything else, anyone? | 14:29 |
| ttx | alright then | 14:31 |
| ttx | #endmeeting | 14:31 |
| opendevmeet | Meeting ended Fri Feb 13 14:31:05 2026 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:31 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/releaseteam/2026/releaseteam.2026-02-13-14.00.html | 14:31 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/releaseteam/2026/releaseteam.2026-02-13-14.00.txt | 14:31 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/releaseteam/2026/releaseteam.2026-02-13-14.00.log.html | 14:31 |
| ttx | Thanks everyone! | 14:31 |
| elodilles | thanks too o/ | 14:31 |
| opendevreview | Elod Illes proposed openstack/releases master: [CI] Temporary increased timeout for list-changes https://review.opendev.org/c/openstack/releases/+/976741 | 14:43 |
| elodilles | frickler: ^^^ i don't like this, but if we don't have better solution, then so be it... :/ | 14:44 |
| elodilles | (as i saw the job timed out after 11th deliverable, so we definitely need the 3hrs timeout as 2hrs won't be enough for the 27 deliverables... :S) | 14:45 |
| frickler | elodilles: well the other option would be to split the release patch into three parts with at most 10 deliverables each. or maybe 4/8 to be a bit more on the safer side? | 14:51 |
| frickler | but let's try the timeout bump first I'd say | 14:53 |
| elodilles | +1 | 14:53 |
| opendevreview | Merged openstack/releases master: [CI] Temporary increased timeout for list-changes https://review.opendev.org/c/openstack/releases/+/976741 | 15:42 |
| fungi | the vexxhost ipv6 routing should be back to normal now | 15:57 |
| opendevreview | Takashi Kajinami proposed openstack/releases master: Release tooz 8.0.0 https://review.opendev.org/c/openstack/releases/+/976761 | 16:04 |
| opendevreview | Stephen Finucane proposed openstack/releases master: Revert "[CI] Temporary increased timeout for list-changes" https://review.opendev.org/c/openstack/releases/+/976763 | 16:13 |
| stephenfin | fungi: gtk. I've proposed a revert of elodilles change on the assumption that we probably want to know when issues like this happen again | 16:14 |
| gmaan | elodilles: frickler: ttx can you please check this, https://review.opendev.org/c/openstack/releases/+/975702 | 16:31 |
| frickler | release-team: https://review.opendev.org/c/openstack/releases/+/976622 is also green now | 16:36 |
| opendevreview | Merged openstack/releases master: Release oslo.service 4.5.0 https://review.opendev.org/c/openstack/releases/+/975702 | 17:20 |
| opendevreview | Merged openstack/releases master: Bulk release xstatic packages https://review.opendev.org/c/openstack/releases/+/976622 | 17:29 |
| elodilles | stephenfin gmaan frickler : done o:) | 17:51 |
| opendevreview | Carlos Eduardo proposed openstack/releases master: [manila] Release 2025.2 stable branch https://review.opendev.org/c/openstack/releases/+/976787 | 17:56 |
| opendevreview | Stephen Finucane proposed openstack/releases master: Bulk release newly onboarded xstatic packages https://review.opendev.org/c/openstack/releases/+/976623 | 18:00 |
| opendevreview | Stephen Finucane proposed openstack/releases master: Bulk release newly onboarded xstatic packages https://review.opendev.org/c/openstack/releases/+/976623 | 18:00 |
| opendevreview | Carlos Eduardo proposed openstack/releases master: [manila] New 2025.1 Epoxy release https://review.opendev.org/c/openstack/releases/+/976799 | 18:01 |
| opendevreview | Carlos Eduardo proposed openstack/releases master: [manila] New 2025.1 Epoxy release https://review.opendev.org/c/openstack/releases/+/976799 | 18:01 |
| opendevreview | Carlos Eduardo proposed openstack/releases master: [manila] New 2024.2 Dalmatian release https://review.opendev.org/c/openstack/releases/+/976801 | 18:07 |
| opendevreview | Carlos Eduardo proposed openstack/releases master: [manila] New 2025.1 Epoxy release https://review.opendev.org/c/openstack/releases/+/976799 | 18:09 |
| opendevreview | Stephen Finucane proposed openstack/releases master: Normalize xstatic package configurations https://review.opendev.org/c/openstack/releases/+/976625 | 18:47 |
| opendevreview | Stephen Finucane proposed openstack/releases master: Bulk release xstatic packages (pt 2.) https://review.opendev.org/c/openstack/releases/+/976669 | 18:47 |
| opendevreview | Merged openstack/releases master: Release tooz 8.0.0 https://review.opendev.org/c/openstack/releases/+/976761 | 19:23 |
| gmaan | elodilles: frickler thanks | 19:48 |
| frickler | great, now we have 28 xstatic reqs bumps that all need to be squashed together and even then won't pass CI :( | 20:09 |
| clarkb | is that a force merge scenario? Just thinking they largely impact horizon which is already broken. So any failures caused by force merging probably wont' make things worse? | 20:53 |
| *** haleyb is now known as haleyb|out | 23:42 | |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!