14:00:26 <hberaud> #startmeeting releaseteam
14:00:26 <opendevmeet> Meeting started Fri Mar 17 14:00:26 2023 UTC and is due to finish in 60 minutes.  The chair is hberaud. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:26 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:26 <opendevmeet> The meeting name has been set to 'releaseteam'
14:00:26 <opendevreview> Boxiang Zhu proposed openstack/releases master: Release skyline-apiserver RC2 for Antelope  https://review.opendev.org/c/openstack/releases/+/877791
14:00:34 <hberaud> Ping list: hberaud armstrong elodilles
14:00:43 <hberaud> https://etherpad.opendev.org/p/antelope-relmgt-tracking
14:00:46 <elodilles> o/
14:00:52 <hberaud> See you at line 431
14:01:06 <elodilles> I'm there already ~o~
14:01:12 <hberaud> \o/
14:01:18 <hberaud> so lets start
14:01:23 <armstrong> o/
14:01:29 <hberaud> #topic Review task completion
14:01:36 <hberaud> o/ armstrong
14:01:51 <elodilles> hi armstrong o/
14:01:54 <hberaud> so, 1. Process any remaining stable branching exception
14:02:23 <hberaud> AFAIK we hadn't exceptions, right?
14:02:40 <elodilles> yes, but we have one patch that is still pending, let me check
14:03:01 <elodilles> grenade has not branched yet: https://review.opendev.org/c/openstack/releases/+/877125
14:03:04 <armstrong> Hi elodilles
14:03:09 <armstrong> Hello everyone
14:03:27 <elodilles> though it just need an upgrade from +1 to +2 from you hberaud as I see o:)
14:03:32 <hberaud> +2'd
14:03:44 <hberaud> and +W'd
14:03:50 <elodilles> ++
14:04:00 <hberaud> next one
14:04:10 <elodilles> then I think everything has branched
14:04:20 <hberaud> \o/
14:04:28 <hberaud> 2. Notify the TC that it should be safe to apply the process to create the new release series landing pages for docs.openstack.org
14:04:48 <hberaud> https://review.opendev.org/q/topic:www-antelope-final
14:04:59 <hberaud> https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/877550
14:05:02 <hberaud> thanks elodilles
14:05:06 <elodilles> np
14:05:12 <elodilles> also pinged TC as well
14:05:16 <hberaud> do you want to add something?
14:05:26 <elodilles> but I think we are covered
14:05:36 <hberaud> cool
14:05:50 <elodilles> Ian proposed the above zuul job as well
14:05:58 <elodilles> so we should be OK
14:06:17 <hberaud> nice
14:06:26 <hberaud> then next task
14:06:39 <hberaud> 3. On the day before the deadline for final release candidates, propose last-minute RCs where needed
14:06:46 <hberaud> On the day before the deadline for final release candidates, propose last-minute RCs where needed
14:06:57 <hberaud> sorry bad paste
14:06:59 <hberaud> https://review.opendev.org/q/topic:antelope-final-rc
14:07:08 <elodilles> yepp
14:07:09 <elodilles> so
14:07:18 <hberaud> I see some of them are -1
14:07:32 <hberaud> and the others are without response
14:07:40 <elodilles> I proposed it yesterday only, as one day before the deadline (as discussed on a previous meeting, Friday)
14:07:47 <elodilles> and I think it was a bit late :/
14:07:55 <hberaud> no timing LGTM
14:07:57 <elodilles> should have been proposed on Wednesday
14:08:03 <hberaud> ok
14:08:11 <elodilles> *i think*
14:08:25 <elodilles> so that we could have given more time to teams
14:08:30 <elodilles> anyway
14:08:30 <hberaud> I'd argue that those without response should be abandoned
14:08:35 <elodilles> half of them have merged
14:08:53 <elodilles> cinder and neutron have 3 patches still -1'd as you wrote
14:09:21 <elodilles> whoami-rajat ralonsoh : any update about the -1'd rc2 patches?
14:10:00 <elodilles> (we can continue the meeting, we might get info later)
14:10:08 <hberaud> ok
14:10:17 <opendevreview> Boxiang Zhu proposed openstack/releases master: Release skyline-console RC2 for Antelope  https://review.opendev.org/c/openstack/releases/+/877793
14:10:19 <elodilles> so I think the final patch can be generated,
14:10:33 <ralonsoh> elodilles, I'm updating it now
14:10:35 <elodilles> and it will need updates if any of the above 3 merges
14:10:35 <hberaud> which one?
14:10:55 <elodilles> hberaud: THE release patch :)
14:11:13 <elodilles> ralonsoh: ack, thanks, we will review after the meeting then!
14:11:52 <hberaud> ah sorry I misread, I seen merged instead generated
14:12:05 <elodilles> :)
14:12:06 <hberaud> s/instead of
14:12:38 <hberaud> so the next two task we be discussed later in the meeting
14:13:20 <hberaud> #topic Assign next week tasks
14:14:01 <elodilles> as I remember most of the tasks are mainly 'guidelines' and to 'all'
14:14:13 <hberaud> I wonder if it is a good idea to assign tasks to different people for the final day
14:14:15 <elodilles> but I'm maybe wrong :)
14:14:24 <hberaud> yeah
14:15:19 <elodilles> I'll be here all day on Wednesday, so we can do the tasks on-the-fly, if something needs specific person to do, the we can discuss then
14:15:39 <hberaud> So I think we can skip this topic and lets Thierry handle the mail sending as he is the chair this week
14:15:49 <hberaud> same thing here
14:15:57 <elodilles> hberaud: sounds good to me!
14:16:01 <hberaud> sold
14:16:03 <hberaud> thanks
14:16:14 <hberaud> #topic Review countdown email
14:16:21 <hberaud> https://etherpad.opendev.org/p/relmgmt-weekly-emails
14:16:26 <opendevreview> Merged openstack/releases master: Create stable/2023.1 for grenade  https://review.opendev.org/c/openstack/releases/+/877125
14:16:42 <elodilles> LGTM, I've changed a date in it :)
14:16:53 <elodilles> 22nd instead of 16th
14:17:01 <hberaud> lGTM
14:17:18 <hberaud> then I'm going to send it
14:18:18 <opendevreview> Rodolfo Alonso proposed openstack/releases master: Final RC patch for ovn-octavia-provider  https://review.opendev.org/c/openstack/releases/+/877658
14:19:17 <hberaud> sent
14:19:20 <elodilles> ++
14:19:54 <hberaud> #topic propose-final-releases is broken
14:20:18 <hberaud> so as I said just before the meeting this tool seems broken
14:20:23 <opendevreview> Rodolfo Alonso proposed openstack/releases master: Final RC patch for neutron  https://review.opendev.org/c/openstack/releases/+/877654
14:20:28 <whoami-rajat> elodilles, the 2 patches we want to get in are approved in master and backports being proposed, we will merge the backport and update the release patch
14:20:30 <hberaud> it doesn't generate something
14:21:32 <elodilles> whoami-rajat: ack, thanks for the heads up! note that we should have been in the pre-release freeze already
14:21:51 <hberaud> I think the problem is around https://opendev.org/openstack/releases/src/branch/master/openstack_releases/cmds/propose_final_releases.py#L183
14:21:52 <elodilles> whoami-rajat: sorry for proposing the rc2 patch only yesterday :/
14:22:17 <elodilles> hberaud: seems quite likely
14:22:59 <elodilles> hberaud: yet another place to add the 'get_release_id' magic function? :)
14:23:01 <hberaud> I'll try to fix it after the meeting, however, as this task should be down after the meeting I'll need you to review my changes asap
14:23:12 <hberaud> yes surely
14:23:17 <elodilles> hberaud: i'll be here
14:23:24 <hberaud> thnks
14:23:27 <hberaud> tanks
14:23:28 <elodilles> no problem :)
14:23:29 <hberaud> thanks
14:23:44 <elodilles> we can start the refactoring after the release :)
14:23:54 <hberaud> yeah
14:23:59 <hberaud> lets move on
14:24:02 <elodilles> ++
14:24:05 <hberaud> #topic Open Discussion
14:24:15 <fungi> related, i have a pair of changes currently flagged wip to temporarily remove and then readd the semaphores for the publish-openstack-releasenotes-python3 and publish-tox-docs-releases jobs so they'll run in parallel on wednesday, if people are keen to try that. it's my recollection that we only have those to avoid races between builds for multiple releases of the same project getting
14:24:17 <fungi> tagged at the same time, but when we're only tagging one release per project it should be safe:
14:24:19 <fungi> #link https://review.opendev.org/877552 Temporarily remove release docs semaphores
14:24:21 <fungi> #link https://review.opendev.org/877553 Revert "Temporarily remove release docs semaphores"
14:25:01 <hberaud> fungi yeah you remember correctly
14:25:02 <elodilles> fungi: \o/
14:25:13 <hberaud> thanks fungi
14:25:13 <fungi> the idea is to avoid the 8-10 hour wait for completion, and having outdated release notes initially when we make the announcement
14:25:37 <elodilles> since our guideline says "don't approve other releases on release day" that should be safe
14:25:47 <fungi> i can un-wip the first one on tuesday and we can merge it then, if no unrelated releases are going to get approved
14:26:01 <hberaud> wfm
14:26:27 <elodilles> Tuesday (your) EOB sounds good to me!
14:26:34 <fungi> cool, added to my schedule
14:26:44 <elodilles> \o/
14:27:01 <hberaud> oslo.log 5.0.0 remains as upper constraint in Antelope? https://review.opendev.org/c/openstack/requirements/+/873390
14:27:10 <fungi> and what time on wednesday are things going to start? i'll try to wake up early
14:27:27 <elodilles> fungi: as far as I remember mostly 11 UTC
14:27:39 <fungi> that's easy for me
14:27:49 <armstrong> For the final release?
14:27:53 <hberaud> yeah around 11utc
14:27:58 <whoami-rajat> elodilles, yeah sorry about that, I had to propose a devstack patch and merge it in stable branches till xena which took some time, so cinder depended on tempest depended on 5 devstack patches
14:28:07 <elodilles> so that we should be ready for 15 UTC
14:28:17 <whoami-rajat> i know things should've been on time but yeah got caught with a lot of mess
14:29:05 <elodilles> whoami-rajat: uhh :S I see, thanks for working on it
14:30:25 <hberaud> back to the oslo topic
14:30:35 <elodilles> so about oslo.log 5.0.0 -> I haven't got there to look what we could do to nova to pass with 5.1.0, so in upper-constraint we still have 5.0.0 :/
14:31:19 <hberaud> same thing here, I commented and asked some questions but I didn't get replies
14:31:19 <elodilles> even if the above 'requirements' patch merges, we need to backport it to requirements stable/2023.1
14:32:11 <hberaud> as we are really close to the deadline I'd prefer to keep 5.0.0 and backport a fix during bobcat
14:32:30 <elodilles> so as I see we have to release with oslo.log 5.0.0 and when there is a fix, then the req bump patch can be backported *after* the release. does this look feasible?
14:32:43 <elodilles> hberaud: ++
14:32:45 <fungi> what are the drawbacks to keeping with 5.0.0?
14:32:49 <hberaud> that wfm
14:33:37 <fungi> i guess the release page gets confusing if we say 5.1.0 is the version for the coordinated release but then we pin to 5.0.0 everywhere
14:33:37 <hberaud> apparently it contains a bug but the proposed fix seems to broken devstack
14:34:12 <hberaud> 5.1.0's requirements patch is still unmerged
14:34:29 <hberaud> and same thing for 5.2.0
14:34:49 <opendevreview> Rodolfo Alonso proposed openstack/releases master: Final RC patch for neutron  https://review.opendev.org/c/openstack/releases/+/877654
14:35:08 <elodilles> you mean 5.0.2, i think
14:35:29 <whoami-rajat> elodilles, np, thanks for being patient with our releases :)
14:35:32 <elodilles> the previous release, which wasn't added to upper constraint either
14:35:58 <fungi> yeah, my point is https://releases.openstack.org/antelope/index.html#library-projects says oslo.log 5.1.0 is the version for openstack 2023.1, but we're only using oslo.log 5.0.0, that seems like a problematic contradiction
14:35:59 <hberaud> elodilles: yeah sorry 5.0.2
14:36:23 <elodilles> fungi: yes, that is the main concern :/
14:37:42 <fungi> distro packagers are likely to believe the releases list, but do we want them packaging 5.1.0 which doesn't work in our integration tests or 5.0.0 which we're actually testing the release with?
14:38:28 <fungi> that seems like the main question to answer
14:38:52 <hberaud> yeah
14:38:53 <elodilles> i don't believe a fix would arrive (in nova or oslo.log?), so probably 5.0.0 should go out as a release
14:39:09 <elodilles> we are late with that already
14:39:23 <hberaud> +1 for 5.0.0
14:39:27 <fungi> in which case how do we go about correcting the releases page... or can we even do anything about that?
14:39:59 <hberaud> I don't know if it can be fixed manually
14:40:05 <hberaud> (the releases page)
14:40:22 <fungi> well, "manually" is probably a bit of a misnomer
14:40:43 <hberaud> yeah
14:41:02 <fungi> we could re-tag oslo.log 5.0.0 as 5.2.0 but that will probably significantly confuse the release notes
14:41:11 <hberaud> or by adding a blacklist mechanism if requirements re not aligned
14:41:32 <hberaud> yeah that could be a solution
14:41:54 <hberaud> and that would introduce other issues
14:41:55 <fungi> right, some filtering mechanism in the generator for the release site pages could work
14:42:27 <fungi> so we could flag 5.1.0 as skipped somehow
14:43:00 <fungi> so that the next most recent one would appear in the list of releases for 2023.1
14:43:02 <hberaud> especially if someone try to fix a bug and he believe that this version is higher in sha than the previous one (that will be the case)
14:43:25 <fungi> however the oslo.log release notes are still going to list 5.1.0 as a later release for the 2023.1 series
14:44:19 <fungi> a series of reverts could be merged to oslo.log along with a release note about everything that is no longer relevant from the earlier release notes, and then tag the result as 5.2.0
14:44:48 <hberaud> yeah
14:44:53 <fungi> that's probably the only way to unwind it so that the documentation/release notes and site are coherent
14:45:09 <hberaud> indeed
14:45:55 <fungi> it's at least the way that is least like trying to rewrite history
14:46:59 <elodilles> since so far everything was tested with oslo.log 5.0.0 that sounds to be the least worse case, yes :/
14:47:31 <elodilles> by reverting the patches (around 4 patch afair), we get back to 5.0.0 basically
14:47:59 <elodilles> not that nice, but sounds acceptable to me :/
14:48:06 <hberaud> My main concern is that it will require several patches in several repos and the deadline is in 5 days
14:48:30 <hberaud> (with backport included)
14:48:38 <hberaud> *backports
14:48:39 <elodilles> hberaud: it only needs the backport in oslo.log, doesn't it?
14:48:50 <elodilles> s/backport/revert
14:49:04 <hberaud> and a release and requirements patches
14:49:32 <elodilles> as I understand fungi want a 5.2.0 release, with the reverts,
14:49:44 <hberaud> yes
14:49:44 <fungi> and could in theory happen directly in stable/2023.1 yeah?
14:50:05 <elodilles> and yes, we need an upper constraint bump patch in requirements / stable/2023.1
14:50:06 <fungi> don't have to merge those to master and backport
14:50:20 <hberaud> indeed
14:50:25 <elodilles> fungi: yes
14:51:37 <hberaud> will try to see I found the bandwith to propose that next monday
14:52:11 <fungi> i suppose it could be done post-release too if it's only happening in the stable branch
14:52:30 <hberaud> yes it could be
14:52:46 <fungi> doesn't stop us from temporary version confusion for people on release day, but would still correct it pretty quickly thereafter
14:53:10 <hberaud> yes
14:53:34 <fungi> it would be slightly more of a stable branch management exception if it happened post-release, but justifiable
14:54:07 <hberaud> +1
14:54:21 <elodilles> so 1) these needs to be reverted on oslo.log's stable/2023.1: https://paste.opendev.org/show/buPjsOLKMr0JKqqfkhGp/ 2) propose a release patch oslo.log 5.2.0 for stable/2023.1 3) bump upper constraint of oslo.log to 5.2.0 on requirements' stable/2023.1. and that's all
14:54:53 <opendevreview> Tobias Urdin proposed openstack/releases master: [yoga] Release designate 14.0.2  https://review.opendev.org/c/openstack/releases/+/877806
14:55:34 <hberaud> yes
14:56:07 <elodilles> I can help proposing these 5 patches if that helps hberaud
14:56:23 <fungi> i would add a 1.5 to merge a change with a release note about any backed-out things that had release notes, if there are
14:57:04 <hberaud> elodilles: as you want, that will allow to review them and +2 them quickly
14:57:14 <hberaud> s/allow me
14:57:50 <elodilles> fungi: yes, we have one release note, so probably a clarification is needed - https://docs.openstack.org/releasenotes/oslo.log/2023.1.html
14:58:40 <hberaud> One last thing before ending this meeting
14:58:45 <hberaud> heads up: gerrit UI change around our PTL-Approved flag, see discussion from infra channel
14:59:16 <hberaud> https://meetings.opendev.org/irclogs/%23openstack-infra/%23openstack-infra.2023-03-16.log.html#t2023-03-16T15:28:28
14:59:34 <elodilles> yepp, I've added it here, just as a heads up
14:59:44 <hberaud> thanks
15:00:12 <elodilles> as now PTL-Approved column does not work as we want it to work
15:00:21 <elodilles> (but the label works)
15:00:31 <hberaud> I see
15:00:44 <elodilles> anyway, we can discuss this later, maybe even at our PTG session
15:00:53 <hberaud> yes
15:01:16 <hberaud> Could be added to the things to changes
15:01:24 <hberaud> at the end of our etherpad
15:01:52 <hberaud> I also added the oslo topic into this section just in case, to not forget it
15:02:08 <hberaud> Well, thanks everyone for joining this meeting
15:02:21 <elodilles> thanks hberaud o/
15:02:25 <armstrong> thanks hberaud
15:02:34 <elodilles> (i've added it to our TODO list)
15:02:43 <hberaud> tahnks
15:02:45 <elodilles> (the PTL-Approved flag issue)
15:02:55 <hberaud> Let's wraps up
15:03:00 <hberaud> #endmeeting