hberaud | o/ | 05:54 |
---|---|---|
ttx | happy release day everyone! | 06:53 |
elodilles | \o/ | 07:25 |
elodilles | diablo_rojo: thanks! o/ | 07:25 |
elodilles | fungi: cool, thanks! o/ | 07:26 |
elodilles | as far as i see there's no newly merged reno-2023.2 patch since i've generated the last reno adding patch: https://review.opendev.org/c/openstack/releases/+/897146 | 07:31 |
elodilles | so we are good to go with this one ^^^ | 07:31 |
opendevreview | Elod Illes proposed openstack/releases master: Mark 2023.2 Bobcat as released https://review.opendev.org/c/openstack/releases/+/897302 | 07:40 |
hberaud | +W'd | 07:45 |
elodilles | hberaud: thanks o/ i've added PTL-Approved+1 | 07:46 |
hberaud | ack | 07:46 |
opendevreview | Merged openstack/releases master: Add release note links for 2023.2 Bobcat (#4) https://review.opendev.org/c/openstack/releases/+/897146 | 08:09 |
ttx | Booked our usual meeting slot on the PTG schedule | 09:17 |
elodilles | ttx: thx \o/ | 09:23 |
hberaud | thx | 09:23 |
elodilles | i've reviewed the patch again ( https://review.opendev.org/c/openstack/releases/+/896971/3 ), PS3 looks OK to me. we are @ T-36mins to start :) | 09:24 |
ttx | Infra looks good | 09:29 |
elodilles | ++ | 09:34 |
fungi | heyhey stackers! | 09:46 |
elodilles | good morning fungi o/ | 09:47 |
* hberaud school-run, back in minutes | 09:49 | |
fungi | showtime! | 10:00 |
elodilles | infra still looks good, status.python.org, too | 10:00 |
elodilles | +2'd the patch | 10:00 |
elodilles | ttx: your turn! :) | 10:00 |
fungi | https://grafana.opendev.org/d/21a6e53ea4/zuul-status?orgId=1 shows a surprising amount of activity for this time of day, but looks healthy enough | 10:00 |
elodilles | yeah, sime teams are surprisingly active on final release day :) | 10:01 |
elodilles | ttx hberaud : if you are around, it's time to start the machinery o:) https://review.opendev.org/c/openstack/releases/+/896971/3 | 10:10 |
ttx | oops sorry | 10:11 |
ttx | hberaud: around, or should I +a? | 10:12 |
ttx | I guess i should since he authored the patch | 10:12 |
elodilles | he mentioned he has to run to school for his kids, so +A please :) | 10:13 |
ttx | done | 10:13 |
elodilles | \o/ | 10:13 |
hberaud | back | 10:14 |
elodilles | jobs are queued | 10:14 |
elodilles | validation and pep8 have started already | 10:15 |
elodilles | validation will take like 30+ mins | 10:15 |
zigo | FYI, Debian Unstable already has Python 3.12 ... | 10:16 |
zigo | So, as far as I'm concerned, OpenStack is effectively exactly 1 year late regarding this, as some projects don't even have voting gate on 3.11. | 10:19 |
elodilles | zigo: sounds cool, especially given that 3.12 is released 2 days ago! :) | 10:19 |
elodilles | zigo: Caracal has python 3.11 is amongst supported versions, so every projects will have py311 voting job in their gate | 10:20 |
elodilles | zigo: https://governance.openstack.org/tc/reference/runtimes/2024.1.html | 10:21 |
zigo | Well, the Python transitions are done in Debian Unstable, not Ubuntu, so we really should be using Debian Testing moving forward, IMO. | 10:21 |
zigo | Either that, or Ubuntu with a custom interpreter. | 10:22 |
fungi | in the middle of making the release is hardly the time to debate dependency choices (nor is this the channel for it anyway), but we normally start testing on new python versions once they're available in ubuntu lts. normally the x.y.1 python releases get backported, which will likely happen around january, in time for us to start testing it at the beginning of our next development | 10:23 |
fungi | cycle after 2024.1/caracal | 10:23 |
zigo | fungi: Right, sorry... :) | 10:23 |
fungi | we've had rather lengthy debates on testing platform choices too, debian/stable we can do. unstable/rolling distributions are pretty impossible for us (we've had to drop opensuse tumbleweed, fedora, non-lts ubuntu, etc) | 10:24 |
fungi | too much changing in the distro means not enough stability for our own testing, and people spend more time trying to fight through transient problems in the platform rather than fixing bugs in the software | 10:25 |
fungi | we even mostly stopped running centos jobs for the same reason | 10:25 |
fungi | the developers want test platform choices to be made by the beginning of a new development cycle, and then have those platforms remain mostly unchanged (aside from minor bug fixes) until the release is done ~6 months later | 10:27 |
fungi | testing on a 6-month-old frozen version of debian/unstable probably wouldn't help anyone | 10:27 |
fungi | (which, ironically, is exactly what the non-lts ubuntu releases are too) | 10:32 |
ttx | breaking for quick lunch, brb | 10:42 |
elodilles | bon appétit! | 10:45 |
elodilles | the validation job is still running. (it could run 40+ mins long) | 10:47 |
fungi | yeah, i think for antelope it took just under an hour? anyway, looks like it's on solum now | 10:48 |
elodilles | yes, but with a minor change in releases repo it became way faster | 10:49 |
fungi | oh, cool i missed/forgot about that | 10:49 |
elodilles | well, it was a side effect of changing from 'python3 setup.py sdist' to 'python3 -m build --sdist --wheel' in requirements checking part of validation :) | 10:52 |
elodilles | anyway, here we can see the previous runs of the validator job on this patch: https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-validate&change=896971&skip=0 | 10:53 |
fungi | we are still calling setup.py --name though, i see constant warnings about it in the output | 10:53 |
fungi | maybe we should extend the pbr entrypoint to be able to output the project name | 10:54 |
elodilles | hmmm, i thought we replaced all occurances by now :-o | 10:55 |
fungi | validation completed, job is wrapping up now | 10:55 |
elodilles | fingers crossed | 10:55 |
opendevreview | Merged openstack/releases master: Bobcat final rc https://review.opendev.org/c/openstack/releases/+/896971 | 10:55 |
fungi | and now for the fun part | 10:55 |
elodilles | there it is! ^^^ \o/ | 10:55 |
fungi | tag-releases is already running | 10:56 |
elodilles | yeah, tagging & publishing the packages & docs | 10:56 |
fungi | openstack/adjutant-ui got tagged | 10:58 |
elodilles | so far so good | 10:59 |
fungi | and more tags are apearing, looks good so far | 10:59 |
fungi | release-openstack-python is underway for adjutant now | 10:59 |
fungi | python-adjutant 7.0.0 was just uploaded https://pypi.org/project/python-adjutant/#files | 11:01 |
fungi | we've got multiple releasenotes jobs running in parallel, so the semaphore removal worked | 11:04 |
fungi | first release announcement has gone out to the release-announce ml (adjutant) | 11:05 |
elodilles | yepp, and the 2nd mail has come, too: barbican -- https://lists.openstack.org/pipermail/release-announce/2023-October/thread.html | 11:07 |
fungi | we've had a spike in node requests for the burst of job activity, but plenty of capacity to satisfy those | 11:08 |
elodilles | ++ | 11:09 |
elodilles | yeah, 57 items in the release pipeline & 32 in the tag pipeline | 11:10 |
elodilles | there will be some release job failure as i see... monasca. the usual ones i guess... | 11:11 |
elodilles | (docker image publishing jobs) | 11:11 |
fungi | looks like tag-releases is cloning everything over the network... "WARNING: Cache directory /opt/git/openstack/nova does not exist, ignoring." | 11:13 |
fungi | something to look into | 11:13 |
hberaud | dad taxi again, brb | 11:17 |
elodilles | hberaud: ACK, o/ | 11:18 |
elodilles | release pipeline already shrinking | 11:19 |
fungi | yep, tag-releases finished | 11:19 |
fungi | (successfully) | 11:19 |
elodilles | \o/ | 11:20 |
fungi | trove-dashboard releasenotes failed to build | 11:24 |
elodilles | :S | 11:24 |
fungi | looks like a broken sphinx config | 11:25 |
elodilles | yeah, it failed at rc1 as well | 11:25 |
fungi | okay, so known issue | 11:25 |
elodilles | according to job history | 11:26 |
fungi | i wonder if it would make sense to find a way to batch up the constraints updates from coordinated releases | 11:28 |
fungi | each one starts about 40 new jobs in check | 11:28 |
elodilles | i guess frickler had a similar idea, discussed with the requirements team | 11:30 |
elodilles | s/guess/think/ | 11:30 |
fungi | luckily, most of these aren't resulting in constraints updates | 11:30 |
fungi | so maybe it's not all that bad | 11:31 |
elodilles | yeah, just wanted to write this ^^^ | 11:31 |
elodilles | services are not part of the uppder-constraints.txt in general | 11:31 |
fungi | right, so far it's been just ceilometer and tap-as-a-service | 11:32 |
hberaud | back | 11:32 |
frickler | well the idea I had was more related to multiple u-c updates conflicting each other | 11:36 |
frickler | not about batching multiple ones into a single commit | 11:36 |
fungi | yeah, for a moment i thought there were a lot more than there turned out to be | 11:37 |
fungi | so far the only job failures seem to be known ones for monasca-* image publication and trove-dashboard's broken sphinx config | 11:42 |
elodilles | yepp | 11:43 |
fungi | the tag pipeline is finally empty (all releasenotes besides trove-dashboard completed successfully), just waiting on the last few in the release pipeline to clear now | 11:48 |
fungi | the last release-openstack-python build (skyline-apiserver) just started | 11:50 |
fungi | and there it is! https://pypi.org/project/skyline-apiserver/#files | 11:52 |
elodilles | \o/ | 11:52 |
fungi | on track to have this phase done before 12:00z | 11:53 |
elodilles | 2 jobs are still in the queue for the last item (skyline-apiserver) in the release pipeline | 11:56 |
elodilles | though those are the announce-release & the propose-update-constraints jobs, | 11:57 |
elodilles | so i think the script that checks for missing releases can be started | 11:57 |
elodilles | ttx: fyi ^^^ | 11:57 |
ttx | on it | 11:58 |
fungi | yes, everything that's going to get uploaded and published has beem | 11:58 |
fungi | er, been | 11:58 |
fungi | the straggler jobs are finally running | 11:59 |
elodilles | so we misses 12UTC with some seconds then :D | 12:00 |
elodilles | (well, zuul says 2 mins :)) | 12:01 |
fungi | but we're also going at this way earlier than last time, so plenty of breathing room | 12:01 |
elodilles | yeah, we started earlier + jobs were faster, so far so good | 12:02 |
fungi | and the release pipeline is finally clear! | 12:02 |
elodilles | \o/ | 12:03 |
fungi | we started a little later than planned but still through this phase at approximately 2 hours in | 12:03 |
elodilles | yepp | 12:03 |
elodilles | i guess best is to wait until we see that there's no missing releases and then start with the next task: merging '[www] Set 2023.2 Bobcat as released' patch | 12:05 |
ttx | script is running | 12:05 |
elodilles | ++ | 12:08 |
ttx | successfully completed | 12:20 |
elodilles | \o/ | 12:20 |
fungi | awesome | 12:21 |
ttx | I'd say you can proceed with '[www] Set 2023.2 Bobcat as released' patch | 12:21 |
elodilles | ttx: if i remember well you had +W right for this repo, so could you please add a 2nd +W (so that we don't need to recheck :)) >>> https://review.opendev.org/c/openstack/openstack-manuals/+/896402/2 | 12:22 |
ttx | yes I do | 12:23 |
ttx | done | 12:23 |
elodilles | patch is on the gate! thanks! | 12:24 |
fungi | and all builds running, zuul estimates 11 minutes to completion | 12:25 |
elodilles | ++ | 12:25 |
fungi | though this took closer to 40 minutes for antelope, so the estimate may be skewed | 12:25 |
elodilles | hmmm, strange, i wouldn't expect a longer run than zuul estimates for this patch | 12:27 |
fungi | oh, because we also needed time for the promote pipeline jobs after it merges | 12:28 |
fungi | https://meetings.opendev.org/irclogs/%23openstack-release/%23openstack-release.2023-03-22.log.html#t2023-03-22T13:31:17 | 12:28 |
Clark[m] | Estimates now are only for running jobs so aren't accurate until every job is running. | 12:29 |
fungi | well, every build is in progress, but my recollection included the actual publication from the promote pipeline jobs | 12:29 |
Clark[m] | In the past it wouldn't estimate until all jobs were running. I forget why this changed. Ah | 12:29 |
elodilles | yeah, i guess in the 40 mins we have more than the jobs finishing times. so the estimate should be precise :) | 12:30 |
fungi | it was approximately 40 minyres from when we approved the change to when the content showed up on the site | 12:31 |
elodilles | ~5 mins left | 12:31 |
fungi | gate+promote+afs | 12:31 |
elodilles | ++ | 12:31 |
elodilles | hence we started the release process ~1hr earlier now :) | 12:32 |
elodilles | zuul says 3 mins, but the job is ahead, so it should be less i think | 12:33 |
fungi | yep, merged just now | 12:34 |
elodilles | yepp, it's ready \o/ | 12:34 |
fungi | promote jobs running, estimate is 24 minutes | 12:34 |
elodilles | so now we have to wait until docs.o.o refreshes, right? | 12:34 |
elodilles | yepp, both jobs are running | 12:35 |
fungi | these are the jobs that actually push the updated content into the afs writeable volume, and then once they complete we wait for afs to replicate the writeable volume to the read-only volumes. that kicks off every 5 minutes, but can take time to complete depending on how much data needs to be transferred between the fileservers | 12:36 |
elodilles | ++ | 12:36 |
* fungi takes this opportunity to put the kettle on for another cup | 12:38 | |
elodilles | enjoy your tea! :) | 12:40 |
fungi | thanks! | 12:40 |
fungi | promote jobs finished | 12:45 |
fungi | nevermind, they haven't yet | 12:46 |
fungi | zuul just got impatient because of the lack of output from the current upload task | 12:46 |
elodilles | :-o | 12:46 |
fungi | (if console output is silent for too long, the console stream gets terminated and reconnects) | 12:46 |
fungi | or rather the browser closes the websocket and reestablishes it | 12:47 |
elodilles | ah, i see | 12:52 |
fungi | content upload into afs has finished, so now we're waiting on the 13:10 vos release cronjob to start replicating it | 13:07 |
elodilles | \o/ | 13:08 |
hberaud | \o/ | 13:08 |
fungi | underway | 13:10 |
fungi | replication of the docs volume completed at 13:11 | 13:11 |
fungi | https://docs.openstack.org/ redirects to 2023.2 and says it's the currently supported release | 13:11 |
elodilles | \o/ | 13:12 |
elodilles | cool! | 13:12 |
fungi | and there's a drop-down option for "2024.1 (in development)" | 13:12 |
elodilles | i think we can proceed with the next patch then: https://review.opendev.org/c/openstack/releases/+/897302 | 13:14 |
hberaud | +1 | 13:14 |
elodilles | it needs a +W then :] | 13:14 |
hberaud | done | 13:15 |
elodilles | \o/ | 13:16 |
elodilles | let's wait until that merges | 13:16 |
fungi | i would have expected to see zuul react to that approval vote by now | 13:18 |
ttx | was missing PTL-Approved | 13:19 |
ttx | I just pushed it | 13:19 |
fungi | d'oh! | 13:19 |
fungi | now it's going | 13:20 |
hberaud | thx | 13:21 |
elodilles | :) | 13:22 |
fungi | estimated time to merge is 16 minutes | 13:26 |
fungi | then promote job to upload into afs, and vos release cronjob | 13:26 |
elodilles | it should be less than 16 mins, i guess the 'Final release' patch "confused" a bit the estimation :) | 13:27 |
Clark[m] | Is this the last change that needs processing to complete the release? | 13:28 |
elodilles | Clark[m]: yes, this should be the last one | 13:28 |
fungi | yeah, and then the announcement e-mail gets sent | 13:28 |
elodilles | then we only have 3 tasks that are: 1) inform foundation staff 2) send mail to openstack-announce 3) send mail to openstack-discuss | 13:29 |
fungi | Clark[m]: also i've un-wipped https://review.opendev.org/896949 to put the release notes semaphore back, if you want to approve it when you've got access (there's no rush) | 13:30 |
opendevreview | Merged openstack/releases master: Mark 2023.2 Bobcat as released https://review.opendev.org/c/openstack/releases/+/897302 | 13:30 |
fungi | woo! | 13:30 |
fungi | i guess publish-tox-docs-releases runs in release-post not promote | 13:31 |
elodilles | hberaud: for 2 and 3 if you want we can do the review part in advance at the usual place: https://etherpad.opendev.org/p/relmgmt-weekly-emails | 13:31 |
hberaud | the etherpad is already up-to-date | 13:32 |
elodilles | hberaud: ACK, reviewing in a minute! | 13:34 |
hberaud | I just added the openstack-discuss part at the end of the etherpad | 13:35 |
elodilles | should we wait with the mails ~5 o'clock-ish? | 13:35 |
fungi | should that be "Let me join *in* thanks"? | 13:35 |
ttx | elodilles: ideally sometimes between 14 and 15utc | 13:36 |
ttx | so that we beat the press releases | 13:36 |
fungi | the publish job for the releases site hasn't started yet anyway | 13:36 |
hberaud | let send it in 25 min, at 2pm | 13:36 |
fungi | so it may not actually be updated until after 14:00 utc depending on how long the remaining bits take to happen | 13:36 |
hberaud | that's work for you? | 13:37 |
fungi | publish-tox-docs-releases is finally in progress now | 13:37 |
hberaud | ack | 13:38 |
fungi | zuul estimates 7 minutes to completion | 13:38 |
fungi | so we may make the 13:45 vos release pulse, more likely 13:50 | 13:38 |
elodilles | "We will meet *at* the Virtual..." ? (i'm not natives speaker, so just asking o:)) | 13:38 |
elodilles | ttx: ACK, thanks | 13:38 |
fungi | fwiw, the foundation event organizers are no longer referring to it as "the virtual ptg" (or vptg), they just note that it's happening remotely, on-line | 13:39 |
elodilles | fungi: oh, thanks for the information. so that means that sentence can be rewritten a bit more than that | 13:40 |
fungi | though messaging it still rather split. ptg2023.openinfra.dev does say "virtual project teams gathering" up top | 13:41 |
fungi | i'll remind them that needs updating ;) | 13:41 |
elodilles | about the 2nd mail, it could be more like a reference to the announcement mail, like we did in the past: https://lists.openstack.org/pipermail/openstack-discuss/2022-October/030735.html | 13:42 |
elodilles | what do you think? | 13:42 |
hberaud | wfm | 13:43 |
elodilles | hmmm, maybe it's good as it is | 13:43 |
elodilles | never mind | 13:43 |
elodilles | (sorry o:)) | 13:44 |
fungi | okay, releases site contents have been uploaded into afs, in time for the 13:45 vos release pulse | 13:44 |
fungi | which is running now | 13:45 |
fungi | and https://releases.openstack.org/ shows bobcat as maintained rather than in development | 13:45 |
hberaud | elodilles: I updated the etherpad with your suggestion | 13:45 |
elodilles | hberaud: thx, both mails look good to me o:) | 13:46 |
hberaud | thx | 13:46 |
elodilles | cool, so now https://releases.openstack.org is also up to date \o/ | 13:47 |
elodilles | no we have to wait a bit until the mails can be sent out :] | 13:51 |
elodilles | s/no /now / | 13:51 |
ttx | Yeah, feel free to send once the clock hits 1400utc... I'll be in a meeting | 13:51 |
elodilles | hberaud: ^^^ | 13:52 |
hberaud | ok, will send it in ~8min | 13:52 |
elodilles | +1 | 13:52 |
fungi | i'm around to approve the announcement in moderation | 13:52 |
zigo | echo "Mail -f announcement.txt" | at "14:00" | 13:53 |
zigo | :) | 13:53 |
fungi | it's time! | 14:00 |
* fungi makes alarm clock noises | 14:00 | |
fungi | i've got the openstack-announce moderation queue up and waiting | 14:01 |
hberaud | sent | 14:04 |
fungi | and accepted | 14:05 |
elodilles | there we have it: https://lists.openstack.org/pipermail/openstack-announce/2023-October/002073.html | 14:07 |
elodilles | nice! | 14:07 |
hberaud | and sent on openstack-discuss | 14:07 |
elodilles | yepp: https://lists.openstack.org/pipermail/openstack-discuss/2023-October/035266.html | 14:08 |
elodilles | thanks \o/ | 14:08 |
fungi | i've notified the foundation marketing team, they're doing their stuff now | 14:08 |
elodilles | \o/ | 14:09 |
fungi | https://social.openinfra.dev/@OpenStack/111177133689009654 | 14:09 |
fungi | https://www.linkedin.com/feed/update/urn:li:activity:7115336803246567424 | 14:09 |
ttx | \o/ | 14:10 |
fungi | https://www.facebook.com/photo?fbid=706738644834152&set=a.624784356362915 (doesn't seem to be visible without a facebook account, but i'll take their word it's there) | 14:10 |
elodilles | :) | 14:11 |
hberaud | \o/ | 14:11 |
fungi | https://twitter.com/OpenStack/status/1709571368004567410 | 14:11 |
frickler | shouldn't that be x.com now? also one should seriously discuss whether to continue using it | 14:13 |
fungi | yeah, well mastodon was the first place the marketing folks posted to | 14:14 |
fungi | twitter was the last ;) | 14:14 |
fungi | i expect they'll continue to notify twitter followers about things until that's no longer viable | 14:14 |
fungi | but it hasn't stopped them from also adopting newer channels | 14:15 |
frickler | IMNSHO it hasn't been viable for quite some time, if one takes any form of openness serious | 14:16 |
frickler | but apparently also zoom continues to count as a viable option, so what do I know? | 14:18 |
fungi | is them posting announcements to twitter any worse than the announcements they post to facebook and linkedin? none of those platforms are anything i feel safe using, but reaching people who do use them still seems to have some utility | 14:42 |
fungi | they also send press releases to news sites, almost none of which are open platforms (and some of which are paywalled most likely) | 14:44 |
frickler | facebook and linkedin afaict at least still try to combat hatred and racism, yes. doing anything that helps people to continue using x is being actively malicious IMO | 14:44 |
fungi | i see. my distaste for proprieary platforms that snoop on conversations and try to profit off them means i have no idea which ones are less ethical than which other ones | 14:46 |
zigo | frickler: The line between "combat hatred and racism" and censorship is kind of blured, it's a long debate we don't want to enter ... The only thing we can agree on, is what fungi wrote (that it's all proprieary platform and that we should move to a distributed system...). | 15:03 |
fungi | we're getting closer to convincing people to use meetpad, at least. for example the foundation calls with community event organizers are using it now | 15:04 |
fungi | it came up in the tc meeting yesterday (in zoom, ironically) and there were concerns raised about sound quality of jitsi's local audio recording option, inability to dedicate channels to specific owners so that they can ban abusive participants from calls, lack of international dial-in trunks, no transcript feature... the proponents of proprietary teleconferencing platforms refer to | 15:08 |
fungi | those as "accessibility" features, and are opposed to moving to "less accessible" (but open) platforms | 15:08 |
zigo | I never had any issue with the sound quality of Jitsi, nor the recording during Debconf online events ... | 15:09 |
fungi | well, i don't think the people complaining have really tried it in a year or two, and we're constantly upgrading to the latest version | 15:13 |
fungi | "accessibility" has become the new word for "convenience" and there are definitely a lot of community participants who value convenience over free/libre open source principles. they're happy to use open source when it works better for them than any of the proprietary alternatives, but don't want to compromise on features in order to do so | 15:15 |
clarkb | zigo: gmann tested the recording feature a while back and reported there was an echo. Since then there have been many jitsi updates so it is worth testing again. As for general issues we've found that it consumes resources on the local side (not the server) in relation to the number of participants. Having a beefier machine helps. But also that limit seems to be above ~30 | 15:19 |
clarkb | at this point which covers the vast majority of ptg use cases | 15:19 |
fungi | fwiw, i've been on scs community jitsi sessions with well over 30 people most of whom have their cameras on. granted, i do adjust my browser to stop receiving video streams on those calls, but the workstation it's running on is pretty heavily loaded with other activity | 15:24 |
fungi | reviews on https://review.opendev.org/896943 appreciated, i'll un-wip on monday and we can merge then | 20:14 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!