14:00:06 #startmeeting releaseteam 14:00:06 Meeting started Fri Sep 17 14:00:06 2021 UTC and is due to finish in 60 minutes. The chair is hberaud. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:06 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:06 The meeting name has been set to 'releaseteam' 14:00:11 Ping list: elodilles armstrong 14:00:14 o/ 14:00:16 #link https://etherpad.opendev.org/p/xena-relmgt-tracking 14:00:21 We're way down on line 349 now. 14:00:27 Will just wait a couple minutes for folks. 14:01:14 o/ 14:01:21 o/ 14:02:50 ok let's go 14:02:52 #topic Review task completion 14:03:41 The first task was "Process any remaining library branching exception". IIRC I didn't see exception 14:03:55 2) On the Monday, generate release requests for all deliverables that have do not have a suitable candidate yet. 14:04:11 this was done with 2 series of patches 14:04:57 the first one was for CWI projects => https://review.opendev.org/c/openstack/releases/+/808782 14:05:14 sorry wrong link https://review.opendev.org/q/topic:%22xena-c-a%22+(status:open%20OR%20status:merged) 14:05:33 only one patch remain unmerged yet, I'll approve it now 14:06:00 thanks! 14:06:17 Thanks to you elodilles 14:06:48 and the second series was for RC deliverables => https://review.opendev.org/q/topic:%22xena-rc1-deadline%22 14:07:59 deadline is over and almost all PTLs approved their part so I think that we are good to go with the patches which remain opened 14:08:18 ttx, elodilles feel free to validate these ones 14:08:27 I'll continue to review these patches after the meeting, 14:08:38 +1 thanks 14:08:53 now that the gate is working :) 14:09:21 We should only notice that even if the PTL validation badge is not present those patch are almost validated by current PTL 14:09:54 s/are all almost/ 14:10:34 Merged openstack/releases master: Proposing Xena RC1 for freezer-web-ui https://review.opendev.org/c/openstack/releases/+/808618 14:10:36 Merged openstack/releases master: Proposing Xena RC1 for adjutant https://review.opendev.org/c/openstack/releases/+/808593 14:10:38 And that's all for task completion 14:10:46 will also do reviews after meeting 14:10:53 thanks 14:10:57 #topic Assign R-2 tasks 14:11:23 Any volunteer for "After all the projects enabled in devstack by default have been branched, we can engage with the QA, I18n and Requirements PTLs to finalize the stable branch setup" ? 14:11:25 Merged openstack/releases master: Proposing Xena RC1 for freezer https://review.opendev.org/c/openstack/releases/+/808619 14:11:28 Merged openstack/releases master: Proposing Xena RC1 for murano-agent https://review.opendev.org/c/openstack/releases/+/808674 14:11:30 Merged openstack/releases master: Proposing Xena RC1 for murano-dashboard https://review.opendev.org/c/openstack/releases/+/808677 14:11:32 Merged openstack/releases master: Proposing Xena RC1 for murano https://review.opendev.org/c/openstack/releases/+/808680 14:11:42 and "Ensure that all projects that are publishing release notes have the notes link included in their deliverable file." 14:12:03 i can help with those though i might need help 14:12:07 I took that one 14:12:21 ping me once branches are ready and QA will take care of devstack/grenade things 14:12:34 then let me put your name on engaging QA, i18n etc... 14:12:41 sure, +1 14:13:00 gmann: I mean elodilles's name :) 14:13:15 ok :) either is ok 14:13:21 elodilles: this one is easy 14:13:28 and I'll let you know gmann when we are there :) 14:13:33 do not hesitate to ping me if needed 14:13:39 elodilles: ack. 14:13:39 hberaud: +1 14:13:47 excelllent 14:14:18 and that's all 14:14:32 #topic Review countdown email contents 14:14:40 #link https://etherpad.opendev.org/p/relmgmt-weekly-emails#L10 14:14:53 let me know if it LGTY 14:16:30 lgtm 14:16:41 only fixed a Xena capitalization 14:17:03 we all use the same color :) 14:17:09 thanks 14:17:23 boom 14:17:34 ahah 14:18:22 * yoctozepto has to go, will read later the discussion on patching the relationship with governance repo 14:18:29 yoctozepto: o/ 14:18:46 yoctozepto: i've followed up about it on the ml anyway, we can summarize further there 14:19:54 Merged openstack/releases master: Proposing Xena RC1 for nova https://review.opendev.org/c/openstack/releases/+/808706 14:19:58 Merged openstack/releases master: Proposing Xena RC1 for senlin-dashboard https://review.opendev.org/c/openstack/releases/+/808727 14:20:01 Merged openstack/releases master: Proposing Xena RC1 for solum-dashboard https://review.opendev.org/c/openstack/releases/+/808729 14:20:39 (email lgtm, too) 14:20:44 thanks 14:20:47 #topic PTG time slot 14:20:52 We need to schedule our TZ for the PTG, so which will fit well for you? 14:21:24 As it will be a 1 hour PTG for us, I was thinking reusing our meeting time, is it ok for you all? 14:21:56 it could work :) 14:22:10 hmmm sure 14:22:18 Ideally we should avoid conficting with TC sessions 14:22:24 since that gets horizontal attention 14:22:38 oh and that's also usually friday 14:22:56 how about we wait to see where they place themselves 14:23:05 usually relmgmt was on Tuesdays, wasn't it? 14:23:30 i would not say that was a tradition 14:23:33 ttx: Ashlee and diablo_rojo_phone wait for our reply today (and of the week) 14:23:46 elodilles: yes it was 14:24:02 maybe we can answer that we are flexible and want to avoid TC slots 14:24:19 actually I didn't give any project that we shouldn't collide in the survey :X 14:24:38 Concerning myself I don't have a great preference 14:24:40 if i remember well 14:24:57 elodilles: do you want to handle that point? 14:25:17 and reply about our flexibility as suggest by ttx 14:25:29 fungi: thank you for that followup email I was very confused as to why siblings wasn't doign that already 14:25:35 ok /me needs to do morning errands now 14:26:07 ok, sure, then i will propose preferably Tuesday but avoid TC 14:26:17 sold 14:26:26 thanks 14:26:54 #topic PTG etherpad 14:27:06 #link https://etherpad.opendev.org/p/relmgmt-yoga-ptg 14:27:14 i've just added a draft 14:27:21 no content yet 14:27:28 but feel free to add any topic 14:27:35 thanks I'll 14:27:53 maybe the "things to change" from the end of the tracking etherpad? 14:28:06 yes it's a starting point 14:28:30 Usually we did that 14:28:49 Merged openstack/releases master: Release glance-tempest-plugin for Xena https://review.opendev.org/c/openstack/releases/+/808782 14:29:12 #topic double check unreleased client-libraries 14:29:19 #link https://review.opendev.org/q/topic:%22xena-mileston-3%22 14:29:45 So I missed to propose these patch during R-5 14:30:12 These projects are quiet 14:30:29 is that change topic intentionally misspelled? 14:30:49 nope 14:30:56 It's a nit 14:30:59 good catch 14:31:19 no worries, i was trying to work out if the url was wrong or the topic was just set with a typo 14:31:20 I'll update it all over the patches 14:31:29 typo only :) 14:31:39 pro tip: gertty can batch set change topics 14:31:58 Oh!?!? How to do that? 14:32:39 query it for a list of changes with the old topic, then process mark the changes listed (% key by default) and then set the topic while you're on the change list 14:32:48 ah sorry I misread gertty 14:33:18 I see, thanks for the tips 14:34:21 so these deliverables don't have a lot of changes to publish but it could worth to reserve a range of version for them for stable xena 14:34:30 this is the reason behind this batch 14:35:20 I don't think that we will receive validations from PTL/liaison for these deliverables and the opened patches, so I'll force them after the meeting, is it ok for you? 14:35:37 these are what you discussed with ttx, that despite they don't have release content we should release to avoid confusions? 14:35:46 yes 14:36:06 ok 14:36:16 I can help with the review here as well 14:36:19 if they need to fix something at some point during Xena it will be possible 14:36:29 elodilles: thanks 14:36:33 np 14:36:39 much appreciated 14:37:02 #topic pydot2 and the governance repo 14:37:15 So now we are here :) 14:38:29 updated the topic on those 11 changes to xena-milestone-3 (with an e) now 14:38:31 As I said in my email I would avoid to rely on pypi for our own deliverables in the requirements of openstack/release to avoid future catch-22 situations 14:38:37 fungi: thanks 14:39:28 for now we fixed the problem but fungi proposed a solution which seems to be better 14:39:45 +1, I think we should stop releasing governance itself 14:39:58 I will check election scripts/deps which i know use it 14:40:23 ttx: fungi do you know if any one else, foundation site etc need governance in PyPI ? 14:40:27 +1 14:40:45 the only reason we tagged the governance repo originally was to mark the point in time where the state of the projects list was used in determining contributors for the electorate, yes 14:41:04 fungi: ok, and we can use hash for that also 14:41:09 it could be done with an arbitrary commit id instead (and i've even done it that way for special elections) 14:41:11 for local use/run pypi is more convenient, isn't it? 14:41:17 but anyways do not want to hijack this meeting on this, will discuss in TC meeting or so 14:41:37 fungi: +1 14:43:00 my point is if we just run 'tox -e validate' for example, locally, then it will use the old openstack-governance 0.10.0 from pypi and similarly could fail 14:43:42 or do i miss something? 14:44:01 elodilles: i suppose you could include some helper which grabs the necessary content from governance if it's not available already 14:44:04 Do we have any volunteer to handle that point and propose the changes? yoctozepto maybe? 14:44:35 hberaud: what we need now? 14:45:03 the way projects handle obtaining constraints files from the requirements repo for local tox invocation is an example 14:45:21 gmann: at least "declare openstack/governance in" 14:45:22 the required-projects lists for the jobs which are using it, and 14:45:24 then the tox role from zuul-jobs will know to consume it from 14:45:26 locally supplied source instead of fetching it from PyPI 14:45:49 I suppose it will require update in project-config 14:46:02 at least 14:46:18 ah, I thought that is done. I can help 14:46:23 yes, i think the key release jobs which use the files from governance are probably defined in project-config 14:46:51 gmann: much appreciated 14:46:54 best to scope that addition just to the jobs which actually use the governance files though 14:47:11 I added this topic to our PTG agenda to ensure that we don't forget it too 14:47:24 so that we're not unnecessarily installing openstack/governance in every single release-related job whether it's needed or not 14:47:31 yeah 14:48:07 i have a feeling only the reference/projects.yaml file from governance is used, right? 14:48:24 yes 14:48:31 at least 14:48:50 so just jobs which iterate over or do lookups in that file should need governance listed as a required project for the job 14:49:29 and then it can be removed from the (test-?)requirements.txt file 14:49:41 gmann: ^ 14:49:55 ack. thanks 14:50:16 let us know if you need help 14:51:28 #topic Open Discussion 14:51:47 Anything else to discuss today? 14:51:59 looks like we import from governance in doc/source/_exts/deliverables.py so it might still need to be included in doc/requirements.txt 14:53:00 ack 14:54:10 also anything which uses find_gerrit_acl_issues.py, get_contacts.py, list_changes.py, validate.py, deliverable.py, aclissues.py... 14:54:32 Yeah I was thinking about acl 14:54:43 so it's not consuming/parsing the projects.yaml directly, it's using the public api provided by the openstack_governance python module 14:54:55 yep 14:55:45 continuing to release openstack_governance would therefore make sense, so if the concern is just with tagging releases of the governance repo itself i suppose that module could be split into its own separate governance-tools repo or something 14:56:14 * yoctozepto is back 14:56:43 so, what's the consensus on governance-in-releases fix? 14:56:54 gmann: in summary, openstack/governance currently contains a public api in the form of a python package, which is consumed by other projects. treating it as a proper library would make sense because that's how it's already consumed 14:57:03 for now follow fungi's proposition on the ML thread 14:57:19 I must say I'm unsure we can easily patch the tox jobs as they rely on global definitions 14:57:27 so we would switch it for all the projects 14:57:33 and also ignore local git checkouts 14:57:50 fungi: humm, if those are used in release or election then we can ask to use directly from source otherwise lib make sense 14:58:29 as far as I understand both, governance and releases repos only really make sense when considered from the very tip 14:58:31 yoctozepto: which tox jobs specifically are you concerned about? you can add required-projects similar to how we do for, e.g., tox jobs run by horizon plugins which need horizon installed from source 14:58:57 fungi: all jobs were tox and all were failing, so e.g. pep8, py36, py38, cover, whatever 14:59:08 we would need to inherit all 14:59:15 and control which python vers to test 14:59:27 in project-config, I only find this job using the releases/tools/check_approval.py (which is governance/project.yaml) and it has governance in required_projects - / https://opendev.org/openstack/project-config/src/branch/master/zuul.d/jobs.yaml#L1078 14:59:28 which might not be *that* bad considering the repo's purpose 15:00:22 mayhaps it's just an overkill that it's included in requirements.txt and should be moved to tox's definition - for endusers rely on git path and zuul could substitute its own one? wdyt? 15:01:08 gmann: this one was the only one who works previously, I think because "required-project" is used 15:01:23 gmann: the problem comes from things which call modules in the openstack_releases package or scripts in openstack/releases tools directory, since they import from the openstack_governance package which must be installed 15:01:39 I see many files use openstack_governance 15:01:43 yeah 15:02:34 so openstack_releases has numerous places where openstack_governance is treated as a python library and needs it installed via pip from somewhere 15:02:44 mhm, precisely 15:02:55 agreed, the problem was that we use governance as an API ans so we need to install and pull its dependencies and this is where we faced the pydot2 problem 15:02:56 and we want that to be direct from git 15:03:10 the current solution is only incapable to do depends-on 15:03:24 and slightly less efficient 15:03:35 than with required-projects 15:03:54 yoctozepto: well, also it results in running git clone from opendev every time the project is installed, that's inefficient and fragile, so will lead to a lot of new random job failures 15:04:18 fungi: yeah, that was included in "slightly less efficient" 15:04:29 I do wonder how bad that would be but yeah 15:04:36 for releases we should strive for best stability 15:04:44 so perhaps really control all the jobs locally? 15:04:49 releases-tox etc. 15:04:55 fungi: however I'm not sure to see how splitting the python module part of the governance repo will protect us from similar situation? 15:04:58 releases don't have to gate on the same python versions tbh 15:05:02 unfortunately, even a modest increase in random failures for release jobs means me or another opendev admin getting involved to reenqueue failures because many of these jobs aren't idempotent 15:05:17 fungi: yeah, I feel you totally 15:05:35 hberaud: yeah not sure that will solve these kind of issues 15:05:43 ok, so how about this 15:05:51 hberaud: splitting the python module part of the governance repo to a separate repo means that the tc can stop worring about releasing the governance repo and can only have to release the governance-tools repo 15:06:03 separating the software from the data 15:06:07 1) releases control their jobs by themselves 15:06:25 2) they add required-projects to the base tox job 15:06:43 fungi: yeah but those tools still face same issue as currently we are facing, its tool broke us not data 15:06:54 3) governance also adds gating on a relevant test job 15:06:55 yes 15:07:06 the problem come from the env 15:07:13 then we have more control AND more reliability 15:07:21 hberaud: gmann: i suggested splitting the library out of the governance repo as an alternative so that people can still continue to consume it from pypi for running locally 15:07:21 yoctozepto: on 2nd, base tox job you mean install gov for everyone ? 15:07:36 gmann: no, I mean custom base tox job for releases 15:07:41 +1 15:08:11 how do horizon plugins do things like pep8 since they need horizon installed from source? 15:08:15 yoctozepto: and thing is if governance ship any tool, it should test in their gate which is missing key here 15:08:36 yoctozepto: seems a good idea 15:08:39 and if no one use those tool like universal* then just delete 15:08:40 gmann: we test it; we had our gate broken 15:08:43 fungi: good question 15:08:55 fungi: I bet they just use release lol 15:09:08 but let me check something sensible 15:11:23 yeah, they seem to be installing latest release 15:11:29 which is clumsy 15:11:41 (to avoid saying it's utterly wrong in general) 15:11:58 but it seems we just lost track of the needs of tox jobs 15:12:16 I am unsure if we want to try to develop a generic approach 15:12:38 clarkb: ^ do you know of a general approach to adding required-projects for every single job run for a particular project? 15:13:06 anyhow, releases have an outdated set of template jobs: "openstack-python3-victoria-jobs" 15:13:09 the idea is that any time openstack_releases is installed, openstack_governance needs to be installed but from source not from a release on pypi 15:13:18 thus it makes sense to change the process 15:14:10 it looks like we would probably have to use child jobs of all the generic ones and then add required-projects to each one that way 15:14:11 meh, and the jobs are overridden in the pipeline already lol 15:14:23 so it can just be added to these overrides 15:14:32 there is a lot of duplication, ok 15:15:42 it's possible we can set required-projects on variants instead of using job inheritance 15:15:47 yeah 15:16:05 how many jobs we have ? may be let's go with like check-release-approval 15:16:10 simple way 15:16:20 it just means the project-templates are no longer useful 15:17:32 gmann's proposition could be a first step 15:17:53 and continue releasing governance for its public APIs for local use or so 15:18:22 nothing prevents us from improving it behind 15:19:54 WFY? 15:20:23 loving that keystone also stayed on testing victoria https://opendev.org/openstack/keystone/src/branch/master/.zuul.yaml 15:20:23 meh 15:20:33 I'll have to grab kids at school 15:20:45 hberaud: make sure to take only yours 15:21:00 will try :) 15:21:02 otherwise they may report a KIDnapping 15:21:16 :) 15:21:40 I'll propose something and let's see how it's going to work 15:22:01 +1 15:22:03 We could continue the discussion on the thread 15:22:13 I'll close the meeting 15:22:20 or just in this channel ;-) 15:22:32 As you want :) 15:23:39 Thank you everyone for joining. I added a record of this discussion to our PTG etherpad 15:24:04 #closemeeting 15:24:19 #endmeeting