Tuesday, 2021-09-21

opendevreviewEric Xie proposed openstack/os-vif master: Fix typos  https://review.opendev.org/c/openstack/os-vif/+/81013701:34
opendevreviewBalazs Gibizer proposed openstack/nova stable/xena: [stable-only]Update .gitreview for stable/xena  https://review.opendev.org/c/openstack/nova/+/80975907:15
opendevreviewBalazs Gibizer proposed openstack/nova stable/xena: [stable-only]Update TOX_CONSTRAINTS_FILE for stable/xena  https://review.opendev.org/c/openstack/nova/+/80976007:16
opendevreviewBalazs Gibizer proposed openstack/nova stable/xena: [stable-only]Update TOX_CONSTRAINTS_FILE for stable/xena  https://review.opendev.org/c/openstack/nova/+/80976007:16
gibizigo, artom, clarkb: I've updated the stable/xena setup patches 07:17
gibiI think in general the backport check helps enforceing that we only merge patches in a good branch order07:17
gibinew stable branch setup seems to be an edge case 07:18
bauzasgood morning folks07:19
bauzasgibi: hmmmm07:20
gibibauzas: good morning07:21
bauzasgibi: modifying the commit msg would mean that the CI job would accept it ?07:21
bauzas"Stable branch requires either cherry-pick -x headers or [stable-only] tag!"07:21
gibibauzas: yes, either the patch needs to have a cherry-picked from line where the hash points to a merged change or the patch needs to state that it is stable only07:22
bauzassaw in https://zuul.opendev.org/t/openstack/build/97c7712edfe34c5386c0cc2fc37ef46607:22
bauzasgibi: kk, +2 then07:22
*** rpittau|afk is now known as rpittau07:24
bauzasgibi: looks like we need to do this also for all of the changes https://review.opendev.org/q/owner:infra-root%2540openstack.org+(project:openstack/nova+OR+project:openstack/placement)+is:open07:24
bauzasah no07:28
bauzasjust rechecks because of timeouts in placement07:29
gibiI think we dont have the backport check in placement07:34
gibithe placement lower constraint timeout needs https://review.opendev.org/c/openstack/placement/+/810001 to merge first07:35
bauzasgibi: this is correct, hence my "ah no" :)07:48
bauzas(placement doesn't verify the backports)07:48
bauzasgibi: and shit, you're right07:49
bauzaswe need a second core07:49
bauzaslyarwood: I know your placement knowledge is the same than old greek, but we'd appreciate a +W for a dependency bump https://review.opendev.org/c/openstack/placement/+/810001 in order to fix the gate07:51
gibibauzas: also the stable team needs to make a short term decision what to do with the lower constraint bump on stable/xean07:56
bauzaswe have an open change about it07:56
gibiit is not really allowed, but we can push an RC2 to be in the xena release 07:56
gibiyeah that also possible07:57
gibiturning off the job07:57
bauzashonestly, I have no opinion and this could be discussed at the PTG07:57
bauzasbut we need to consider the bump, agreed07:58
gibiso 1) bump lower constarint and release RC2 so we are not breaking stable policy abour reqs 2) try to pin virtualenv during tox installation to avoid unpinned setuptools on stable (this is the long term solution) 3) turn off the lower job07:58
gibiand the problem is not placement specific but hitting a lot of projects 07:59
gibinova is not impacted07:59
gibiso a common solution would be good07:59
bauzasgibi: let's discuss this during our meeting tonight08:20
gibibauzas: btw would you like to take over charing of the meeting?08:20
bauzasgibi: I was thinking about it08:20
bauzasI can do it :)08:20
gibielodilles: will you be available on todays meeting to talk about the stable req bump problem?08:20
gibibauzas: then I officially give you the baton :)08:21
gibibauzas: https://review.opendev.org/c/opendev/irc-meetings/+/81016508:24
bauzasgibi: thanks, was looking on it...after finding the Simpson video :)08:25
* bauzas should start modifying a few things https://docs.openstack.org/nova/latest/contributor/ptl-guide.html#new-ptl08:26
gibibauzas: on the today meeting we should also talk about the release liaison role08:26
bauzasgibi: yeah, I was about to ask for it08:27
bauzasalso, we have other liaisons fwiw08:27
gibifor me only the release one was visible08:28
bauzastechnically we have other roles in our team :) https://wiki.openstack.org/wiki/Nova#People08:28
bauzasbut meh08:28
bauzasit's not like we have 50 engineers wondering how to help our community :)08:29
lyarwood\o mornig08:31
* lyarwood reads08:31
elodillesgibi: yes, i'll be there on the meeting :)08:35
gibielodilles: do you already see a common solution forming from stable perspective?08:38
gibilyarwood: o/08:42
elodillesgibi: btw, I agree with your option 2 but needs to be checked with infra I think08:46
gibielodilles: yeah, I can accept that option 2 needs more effort probaly to make it right so if we need a short term solution then I'm fine with either 1 or 3. 1 has the limit that we need to cut RC this week. 3 has the effect that we will never turn the job back again08:48
gibihm feels like the gate is clogged08:50
bauzasgibi: elodilles: fwiw, I'm in favor of bumping the dep now, which is option 1 and revisit the issue at the PTG09:14
bauzasshort-term, we need to addree the fire09:15
bauzaslong-term, we could draft a plan on how to avoid fire next time09:15
gibibauzas: I was cheated a bit with 1). on stable/xena this is viable as we can have RC2. But on older stable branches we only have the option of 2) or 3) (or going against the stable req policy)09:28
bauzasgibi: what policy issue do you see ? we can bump a .y release which can need to upgrade a dependency, right?09:37
alexe9191good day everyone :) after restarting the scheduler after an OpenStack upgrades all of the schedulers are running 100% cpu and are taking a lot of time building the cache. Scheduling is not working of course as the schedulers are busy creating this cache.09:41
alexe9191I am wondering if adding more schedulers would solve this problem ? current resources allocated to each scheduler is about 8G of memory and 8 CPUS09:41
alexe9191but I do not think that the # of cpu matters as it seems that the python process is single threaded. I also did not find any "workers" configuration for the scheduler in the newton version.09:42
gibibauzas: on stable we should not bump major version of a dependency I think09:44
bauzasfor a .z release, yes09:44
gibican we do that for a .y. release?09:44
bauzaslemme try to look at the Openstack semver rules09:45
bauzasthat's not really a "stable policy" AFAIK09:45
bauzasat least, I can't find anything in https://docs.openstack.org/project-team-guide/stable-branches.html09:45
bauzasthere it is https://docs.openstack.org/pbr/latest/user/semver.html#semantic-versioning-specification-semver09:46
bauzasalas, can't find anything specific09:48
bauzaswe won't technically change placement API if we bump the minimum09:49
bauzasgibi: I guess we should raise this question to the wider community, maybe09:49
bauzasand see whether they freak out about upgrading our deps in a .y release09:50
alexe9191anyone on the scheduling issue :) ? 09:50
opendevreviewMerged openstack/placement master: Bump min decorator to 4.0.0  https://review.opendev.org/c/openstack/placement/+/81000109:54
bauzasalexe9191: we superseded the idea to have multiple workers by telling we should rather use Placement09:54
bauzasalexe9191: for the CPU usage, try to get a GMR 09:54
bauzasalexe9191: https://docs.openstack.org/nova/latest/reference/gmr.html09:55
bauzasso you should see the greenlets and greenthreads concurrently running09:55
alexe9191I am still on the way to upgrade to rocky and to use the placement fully. I can not drop the sceduler in newton as far as I know09:56
bauzasnova-scheduler will continue to exist09:56
bauzasit's just that scheduler will call out placement for getting a list of candidates before running 09:56
bauzaswhich reduces the amount of complexity by adding a single-lock mechanism09:57
alexe9191We will start using the placement UI as recommended. However we need to upgrade first. We are coming from Kilo and need to transit first at newton before we continue the migrations. 09:58
alexe9191The scheduler is running at 100% cpu for about an hour now and updating host stats/aggregates etc as I can see in the log files. And as I understand, this needs to be done first before the scheduler starts scheduling anything by either using placement or using the filters. 09:59
alexe9191So my question is, would adding more workers speed up the process of building that cache? or it's just a matter of waiting 09:59
gibibauzas: good idea to start a discussion around this in the wider community10:02
bauzasalexe9191: honestly, updating the cache shouldn't take one hour10:14
bauzasyou have another issue10:14
alexe9191What could that be?10:15
bauzasI dunno, get the GMR10:15
alexe9191I see a ton of those messages `Update host state with aggregates`10:15
bauzashow many aggregates do you have ?10:15
alexe9191We also have about 900 hosts and more than 10K vms10:15
alexe9191one moment10:16
alexe9191I also see a lot of those "Update host state with service dict"10:16
bauzas9 can't be an issue10:22
bauzasagain, please do a GMR and see what threads run10:23
alexe9191I am honestly not sure what you mean when you say do a GMR? :) 10:23
alexe9191ah `kill -USR2 8675 `10:24
alexe9191that wouldn't terminate the process right ?10:24
alexe9191Nova did not generate any report 10:37
sean-k-mooneyalexe9191: it will more or less kille the process10:42
sean-k-mooneythe report is stored in the nova log10:42
sean-k-mooneywhere ever you redirect that too10:42
sean-k-mooneyto fix the process you will need to restart it10:42
alexe9191I did send the USR2 signal but the logs where empty10:43
alexe9191well not empty I did not see any GMR logs10:44
sean-k-mooneywhat process did you send it to10:44
alexe9191nova-scheduler 10:44
alexe9191and also nova-api as a test 10:44
sean-k-mooneynova-api wont work if its running under appache or uwsgi10:44
alexe9191I am actually wondering if having debug enabled is causing an overhead?10:44
sean-k-mooneybut the shcheduler should10:44
sean-k-mooneydebug logging10:44
alexe9191it's not it's running under dumb-init. All of the processes are running in containers. 10:44
sean-k-mooneyam if you have slow disks then it could10:45
sean-k-mooneyoh this is kolla10:45
alexe9191Yeah more or less, kolla containers 10:45
sean-k-mooneyso dumb-init is likely not passing the signal10:45
alexe9191hmmmm 10:45
sean-k-mooneythat said if you used the pid of the python process not dumb-init it should work10:45
alexe9191Let me try it out again10:46
alexe9191Nothing 10:50
alexe9191I do however see a lot of those in the logs "Lock "host_instance" acquired by "nova.scheduler.host_manager.sync_instance_info""10:50
opendevreviewVlad Gusev proposed openstack/nova stable/stein: Add regression test for bug #1908075  https://review.opendev.org/c/openstack/nova/+/81019110:51
alexe9191and now after about 90 minutes it's done 11:13
gibibauzas: could you please triage this vgpu bug https://bugs.launchpad.net/nova/+bug/1943933 ?11:30
sean-k-mooneyi think the way our config is ment to work is we list the adress of the parent11:33
sean-k-mooneyand then report the quantity of the mdev type that can be created for that device11:33
gibiohh so the reported wants to partially report the possible mdevs from a physical device11:34
sean-k-mooneyi think so11:34
sean-k-mooneywhich i dont think we support11:34
sean-k-mooneythey basically want to treat it like we do for sriov i think where you can enable indiviugual vfs11:34
gibiso this would be a new feature11:35
sean-k-mooneyi could be reading into the report too much but i think that is what they were expecting11:35
sean-k-mooneygibi: ya basically a host_reserved_mdev parmater or somehting11:35
gibiOK. I let bauzas respond but I think I see what you see in the report11:36
sean-k-mooneye.g. enable all mdevs of this type form this parent but reserve x for host use11:36
gibibauzas: also ther is another vgpu bug to triage https://bugs.launchpad.net/nova/+bug/194403111:36
sean-k-mooneythe second one likely is correct but never personally used vgpu so not sure11:37
gibiyeah bauzas has an environment to reproduce :)11:37
gibihence my ping11:37
bauzasgibi: sean-k-mooney: ack, will look at both12:03
gibibauzas: thanks12:04
gibisean-k-mooney: this feels like a networking / neutron bug but I'm not certain. cloud you check it please https://bugs.launchpad.net/nova/+bug/1944083 ?12:17
sean-k-mooneygibi: the only /32 that i can think of that we install is the one for the metadata service12:27
sean-k-mooneygibi: but ya i think this is all contoled more or less on the neutron side in combindation with cloud-init12:28
sean-k-mooneywe might store some to the network info in the metadata which will be used by cloud-init12:28
sean-k-mooneybut i think this is more or less out of our contol12:29
sean-k-mooneylets add neutron and see what they think12:30
gibisean-k-mooney: thanks for the analysis12:31
gibiI agree to involve neutron12:31
sean-k-mooneythe closet thing i can tink of is https://github.com/openstack/nova/blob/master/nova/virt/interfaces.template12:32
sean-k-mooneywe do list the dhcp server per interface https://github.com/openstack/nova/blob/master/nova/virt/interfaces.template#L2112:33
sean-k-mooneybut we are not adding any routes here12:33
sean-k-mooneywe do set the default route via the gateway in the metadata https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/nova/virt/netutils.py#L326-L34812:39
sean-k-mooneyand populate any addtional neutron routes https://github.com/openstack/nova/blob/50fdbc752a9ca9c31488140ef2997ed59d861a41/nova/virt/netutils.py#L103-L12112:39
sean-k-mooneybut ya as far as i can see there is noting on the nova side that would install /32 routes to the dns servers12:40
opendevreviewnorman shen proposed openstack/nova master: Recreate mdev devices according to placement  https://review.opendev.org/c/openstack/nova/+/81022012:40
opendevreviewMerged openstack/nova master: Add missing __init__.py in nova/db/api  https://review.opendev.org/c/openstack/nova/+/80998012:54
opendevreviewMerged openstack/nova stable/xena: [stable-only]Update .gitreview for stable/xena  https://review.opendev.org/c/openstack/nova/+/80975912:54
opendevreviewMerged openstack/nova stable/xena: [stable-only]Update TOX_CONSTRAINTS_FILE for stable/xena  https://review.opendev.org/c/openstack/nova/+/80976012:54
opendevreviewBalazs Gibizer proposed openstack/nova stable/xena: Add missing __init__.py in nova/db/api  https://review.opendev.org/c/openstack/nova/+/81019213:14
gibielodilles, lyarwood, bauzas: we need this in xena for RC2 ^^13:15
gibithis was fast :)13:17
lyarwoodI'm going to be AFK for the meeting btw, dental checkup for the first time in a long time.13:17
gibilyarwood: ack, thanks for the headsup13:18
gibiand I hope the dentis will be painless13:18
opendevreviewLee Yarwood proposed openstack/nova-specs master: Store and allow libvirt instance device buses and models to be updated  https://review.opendev.org/c/openstack/nova-specs/+/81023513:21
lyarwoodI'm British so the normal and in my case correct stereotypes apply, it's not going to be fun.13:22
bauzaslyarwood: we need a release liaison that I'll ask in today's meeting13:24
bauzaslyarwood: if you are interested in this role, tell me 13:24
gibithis reminds me to book a dentis checkup too13:24
bauzasgibi: booked since 6 months13:25
gibiI'm not rushing :) 13:25
kashyapgibi: Yikes; I've been avoiding it13:25
* bauzas loves doctor's appointments in my area, especially eye specialists and dentists13:25
kashyapI know I should13:25
bauzas"I have a tooth rage", "sure, we have a slot in Feb 2022, works for you ?"13:26
bauzaslonger than delivering a car13:26
gibias far as I see I can book a dentist slot for this thursday morning 13:36
gibithat feels too close :/13:37
kashyapgibi: Don't you prefer your band-aid to be quickly removed?13:40
gibithat one way to look at it 13:40
artom_bauzas, obviously the solution is to drive a new car over your tooth13:41
artom_No tooth, no pain13:41
*** artom_ is now known as artom13:41
gibiartom: :D13:42
*** abhishekk is now known as akekane|home13:45
*** akekane|home is now known as abhishekk13:45
bauzasgibi: agenda is updated https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting feel free to add your points13:52
bauzasgibi: will you propose a backport to stable/xena for the decorator bump ?13:52
gibibauzas: ack will check the agenda13:52
gibibauzas: I can propose a backport then we can drop it if we choose other option to go forward13:53
opendevreviewBalazs Gibizer proposed openstack/placement stable/xena: Bump min decorator to 4.0.0  https://review.opendev.org/c/openstack/placement/+/81019313:56
gibibauzas: done13:56
bauzaswe could discuss this during the meeting then13:56
gibiadded two topic to the end of the agenda14:02
opendevreviewThomas Goirand proposed openstack/nova stable/xena: Add missing __init__.py in nova/db/api  https://review.opendev.org/c/openstack/nova/+/81019214:20
zigogibi: bauzas: My bad, I shouldn't have pushed an update, now this patch needs another +2W: https://review.opendev.org/c/openstack/nova/+/81019214:21
zigoI was waiting for the .gitreview, and thought I should be doing it...14:21
opendevreviewElod Illes proposed openstack/nova stable/xena: Add missing __init__.py in nova/db/api  https://review.opendev.org/c/openstack/nova/+/81019214:33
*** akekane_ is now known as abhishekk14:34
elodilleszigo: i've uploaded the original patch again14:34
zigoelodilles: Thanks.14:35
elodilleszigo: and +2+W'd as the content is now the very same as PS1 had14:39
clarkbgibi: thats kind of my point though updating things like tox.ini or .gitreview or translations or other mechanical infrastructure bits don't need that and aren't really a corner case. The lint rules are just too aggressive and you can trust reviewers more imo14:48
clarkbIt is ok to update the branch so that it functions14:48
clarkbbut if you insist on having those rules please update the bots so that they don't run afoul of the linter14:49
gibiclarkb: I'm not the right person to convince as I wasnt the one pushed for the stable backport linter. lyarwood, elodilles: what is your view? 14:52
gibiclarkb: I do think that having couple of paches needing a manual touch every six months is not a big deal. But I also see the point that we could update the bot to use a specific commit message in nova stable setup patches14:53
clarkbI probably feel more strongly about this than most because 90% of the time if I'm pushing to stable branches it is to fix a stable branch specific issue without a backport and I always run into problems like this and it is frustrating that no one seems to accept it is ok to fix a branch on its own14:53
bauzasgibi: oh shit, I wanted to discuss about the release liaison role but I forgot to add it in the meeting notes :facepalm:14:54
gibibauzas: I added it :D14:54
bauzasI sqaw14:54
clarkbif the branch is broken and someone pushes a fix reviewers should have the discretion to land the fix14:54
gibiclarkb: thank you for taking case of the stable branches. I hope elodilles and lyarwood can join. 14:55
gibi... and add their views14:55
gibiclarkb: most of the linters are to help the reviews not missing things. in this case this linter helps the reviewer to check that the fix is already landed on master and newer stable branches. 14:58
bauzashoneslty I tend to prefer trusting reviews rather than using linters14:58
gibiwhich is the rule, with exceptions14:58
bauzasbut I think I already said that a lot (context: mypy)14:58
elodilleswell, the point with that extra check is to ensure we have the *correct* patch backported14:59
bauzasif linters block us instead of helping us, then it's a problem14:59
elodillesas it sometimes happened that a patch was backported to several branch at the same time and later on modified (a middle patch) and those changes were not backported to older branches15:00
gibibauzas: what if linters both help us but sometimes blocking us15:00
elodillesbauzas: don't look at that linter as a blocker but as a helper to catch human failure :)15:01
elodillesabout the .gitreview patch needed manual change: IIRC we had a prompt discussion in the past and decided that it's not a big deal to update those 2 patches every six month15:04
clarkbelodilles: right I get that, the issue is occasionally (and usually when I'm touching stable) we have to modify the mechanics of the branch directly without a backport15:07
clarkbthis is an example of that case. I think reviewers can see that an update to .gitreview to set the correct branch or update a git review option is fine without a backport15:07
elodillesclarkb: actually i also do stable fixes that are stable only and for those cases we (nova (and some other teams)) use 'stable-only' tags in commit message15:10
clarkbelodilles: yes but that isn't consistent and you never updated the bot to do that (which is really my complaint)15:11
clarkbyou should disable the bot on your repos or update the bot15:12
elodillesclarkb: ok, i see your point15:12
clarkbpersonally I'd remove the lint rule entirely because the only way to discover the rule is to push code and have it fail first15:12
elodillesclarkb: I'll update the bot15:12
elodillesclarkb: well, in our team stable cores decided that this linter is useful for us (especially for reviewers). I understand your point, but i tend to look it from another perspective :)15:14
clarkbok. In general I think commit message linters are a bad idea. Users don't know they will fail until after they have pushed (because typical workflow is make changes, run tox, commit, push)15:17
sean-k-mooneyclarkb: we tried to strick a blance by having it non votign in check to give early feedback and block it only in gate15:19
elodillesclarkb: without the linter we did the very same: if a stable reviewer saw that something is not complete in the commit message (or a wrong patch was backported) then we asked the author to change the patch15:19
sean-k-mooneythey can also run the lint check locally15:19
gibiclarkb: I think it is not impossible to add the current check as a tox target too (or part of pep8) so the extra push can be avioded15:19
clarkbelodilles: right but in the case of fixing the mechanics of a stable branch you wouldn't do that because it is fine as is15:19
elodillesclarkb: and that linter (script) can be also run locally15:19
sean-k-mooneymost user do not backport15:19
clarkbsean-k-mooney: they can but literally no one expects they have to run the linter after committing15:19
clarkbthis is why we removed the '.' at end of subject lines rule from hacking15:20
sean-k-mooneyclarkb: well we have been trying to encurage people to use precommit more by the way15:20
clarkboh wow. I would -2 that too :P15:20
elodillesgibi: we have it as an extra tox target now as stephenfin proposed it a couple of months ago15:20
clarkb(it doesn't handle dependencies properly and you'll completely bypass mirrors etc)15:20
sean-k-mooneyclarkb: pre comiit is not a repleace fdor any of our ci15:21
clarkbsean-k-mooney: right but people run it locally and may have proxies or mirrors etc15:21
sean-k-mooneyit just elimnates the excuse fo forgetting to run expected checks15:21
clarkbsean-k-mooney: the pre commit hook won't catch the commit issue errors though because it happens too soon right?15:21
sean-k-mooneyin this case yes15:22
sean-k-mooneybut it will run flake8 exctra15:22
sean-k-mooneyit is too early for this backport check15:22
sean-k-mooneyalthogh technially15:22
sean-k-mooneyyou can configure it to run on push15:22
sean-k-mooneynot on commit locally15:23
clarkbit would be better to run it after the commit like the gerrit chaneg id hook15:23
clarkbthen people won't ask me why push is so slow to gerrit15:23
sean-k-mooneyyes so if you configure it to run pre-commit on push instead of commit then you get that effect15:23
sean-k-mooneybut its not the default15:23
sean-k-mooneyand i dont think they have a post commit option15:24
sean-k-mooneyclarkb: being able to run it on push instead of commit i think is relitively new to the tool15:24
sean-k-mooneyactully you might be able to add addtion stages https://pre-commit.com/#config-stages15:27
sean-k-mooneythis is beyond my basic use of the tool to date15:27
bauzassean-k-mooney: I'm not super happy either with pre-commit hooks15:28
bauzassometimes I just wanna push15:28
sean-k-mooneybauzas: it removes a lot of the teedium with doing dev15:28
bauzasand I don't want it to be hold per the sake of something I don't care15:28
sean-k-mooneybauzas: you can bypass it15:28
sean-k-mooneyyou just set an env var on the push line15:28
sean-k-mooneyor commit line15:29
bauzassean-k-mooney: I told about *pre-commit*15:29
bauzasnot pre-gerrit ;)15:29
bauzasand sorry the push was meaningless15:29
bauzasI meant commit15:29
sean-k-mooneyyes you can overrid it if you just want to skip the checks15:29
clarkbwhat is the issue with running `tox` ?15:30
sean-k-mooneyits slower in some cases and pre-commit is not just about running tests15:31
sean-k-mooneyit can fix things like mixed tabs/spaces ectra15:31
sean-k-mooneyye are both going to "love" https://review.opendev.org/c/openstack/nova/+/806182 by the way15:32
sean-k-mooneybased on your reaction to date :)15:32
stephenfinbauzas: git commit -n15:32
bauzasreminder : nova meeting in 28 mins-ish15:32
bauzasclarkb: yeah that's one of my concerns15:33
bauzaswe push more and more checks into commit stage15:33
bauzasand we add more linters15:33
sean-k-mooneyand we improve the quaity of the code and our developemt workflow as a result15:33
stephenfindon't make humans do things that machines are good at15:34
clarkbif tox and pre commit run the same checks they should run in about the same amount of time15:34
stephenfinyeah, pre-commit and 'tox -e fast8' will run in ~ the same time, but you need to remember to do the latter 15:35
sean-k-mooneyclarkb: in some cases yes15:35
bauzasstephenfin: I don't disagree, I'm just on clarkb's side on the fact that could be left to tox15:35
clarkbstephenfin: the problem is discoverablity. Humans don't know what all the checks are and some of them don't run until you get to zuul (commit message linting). Also some of it is severe nitpicking like having a '.' in the subject line or not15:35
bauzasand linter jobs could be non-voting15:35
stephenfinwe have commit message linting?15:35
bauzaswe do15:35
clarkbstephenfin: yes that is what started this entire conversation15:36
bauzasfor backport checks15:36
sean-k-mooneybauzas: they could be but unless we are going to do that with pep8 im really not ok with that15:36
stephenfinoutside of the cherry-picked from line?15:36
clarkbstephenfin: the bot that fixes your .gitreview file when you make a new branch got -1'd because its commit message wasn't good neough15:36
sean-k-mooneyclarkb: yep15:36
sean-k-mooneythe both should have had a stable only statement in that commit15:36
clarkbthe bot predates the check :P15:36
sean-k-mooneyi know but its not the first time we have modfied the bots patch15:37
stephenfinclarkb: which change is this, specifically?15:37
sean-k-mooneystephenfin: the one for the .gittreview file for the stable branch15:37
stephenfinfor nova?15:37
stephenfinokay, that's what I was expecting to see. Where's the check for the '.' in the subject line?15:38
stephenfinOr is that another project?15:38
clarkbstephenfin: that was an old checker for hacking. I bring it up because after much fighting we finally conceded that any checks on commit messages are a bad idea because nothing checks them until it is too late15:39
stephenfinAh, okay, thanks. I'm caught up now :)15:39
clarkbbasically any linter checks for commit messages are doomed to fail15:39
clarkbbecause nothing runs the linters post commit15:39
clarkb(except for zuul)15:39
sean-k-mooneyclarkb: by the way if i am runging tox its often after i commit15:40
clarkbok I don't know anyone else that does that unless they are bisecting to find issues in already merged code15:40
stephenfinI agree that we should carve out an exception for these bot generated changes to stable15:40
sean-k-mooneyor just fix them15:41
clarkbthere is a semi related issue here with PBRs support for encoding semver requirements in commit messages15:41
clarkbthat causes a ton of problems too because commit messages will be merged and pbr will disagree with the version someone tagged by hand and then you end up i na really weird spot15:41
clarkb(basically you need to be flexible with commit messages because you creat headaches when you try to enforce too many rules around them)15:42
stephenfinsean-k-mooney: they've always been that way though so it seems weird to let our custom script dictate what the commits for every other project looks like15:42
* lyarwood jumps back on after the dentist 15:42
sean-k-mooneyi semi agree but on the ohter hand hacking allows things i hate and blocks things i like yet we allow it to keep the porject semi consitent15:43
lyarwoodI've likely missed something here but why does this matter if a core still needs to ACK the bot proposed patches?15:43
lyarwoodand in ACK'ing these bot changes can add the [stable-only] tag?15:43
sean-k-mooneystephenfin: when we added the script we were hoping ti would gain adoption in other porjects too15:43
lyarwoodit's not like these changes are merge automagically right?15:43
sean-k-mooneyi dont see why the stable team cant fix the bot patches when they approve15:44
clarkbthe whole point of having the bot is to reduce human overhead15:44
stephenfinclarkb: Agreed in general. This feels slightly different though. It was put in because dansmith was bothered by me proposing backports to multiple branches at once, in the fear that they'd merge in the wrong order or a patch higher up would be modified and the changes wouldn't be captured in later patches. Those still seems like sensible concerns that'd be easily missed by reviewers15:44
clarkbessentially we've got two bots that are in theory supposed to reduce overhead but in reality dobule it15:44
lyarwoodwell that's on the stable team who want the second bot surely?15:45
lyarwoodI don't get the argument here tbh15:45
lyarwoodwe want the cherry-pick job and are happy with the overhead15:45
lyarwoodsurely that's enough?15:45
clarkbI've got two main concerns. The first is that after many yaers we still act like proposing a non backport change to stable branches is immediately wrong. This is frustraing for people like me who basically only do that when touching stable branches15:45
clarkbthe second issue is that any linter applied to commit messages is problematic because you don't typically run linting post commit15:46
sean-k-mooneyim more or less of the opipion that we keep the cherry pick lines and enforce them in a job or we done enforce them in a job and dont require them any more15:46
sean-k-mooneyclarkb: well actully having the bot tell you its wrong to propose directly to stable was one of the goals of the script15:46
sean-k-mooneyclarkb: its not that uncommon to have one off patches propsoed directly to stable15:47
sean-k-mooneyi have -1'd patches a cople of time for that and explained that the issue need to be resolved in master first15:48
elodillesclarkb: just for the record, this cherry-pick-check is only in nova repository (well, and afaik cyborg adopted it as well), all the other repositories are not having it15:48
lyarwoodclarkb:  sorry back, I appreciate that the initial -1 is slightly off putting but again if the stable team looking after the repo are happy to do the work to fix this while they approve then I can't see an issue here.15:53
opendevreviewStephen Finucane proposed openstack/nova master: tools: Ignore bot-generated branch creation patches  https://review.opendev.org/c/openstack/nova/+/81028515:53
stephenfinlyarwood, sean-k-mooney, elodilles: eh? ^15:54
gibistephenfin: thanks!15:54
stephenfinno need to backport that, obv. It'll just avoid this discussion come stable/yoga creation15:54
dansmithstephenfin: can we not just agree on some flag for bot-generated things?15:54
lyarwoodha hackaroundFinucane has entered the chat15:54
stephenfindansmith: Sure. I just don't want to fix the bot :)15:54
dansmithchecking for those strings seems fragile, but I'll also say I agree with lyarwood and do not understand why humans can't just handle this15:55
dansmithare the bots always the same userid?15:56
dansmithi.e. checking for owner might just be easier15:56
clarkbI think there are a couple of bots but it is a small number15:56
bauzasreminder : 3 mins before the nova meeting15:57
bauzashonestly, I stopped discussing this 15:58
dansmithI also think that assuming everything proposed against stable is wrong until proven right is the correct approach, just FYI15:58
clarkbok I won't worry about the bot being told it is wrong then15:59
opendevreviewStephen Finucane proposed openstack/nova master: tools: Ignore bot-generated branch creation patches  https://review.opendev.org/c/openstack/nova/+/81028515:59
stephenfindansmith: Good idea ^15:59
dansmithstephenfin: ++16:00
bauzas#startmeeting nova16:00
opendevmeetMeeting started Tue Sep 21 16:00:51 2021 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
opendevmeetThe meeting name has been set to 'nova'16:00
bauzas#link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting16:01
bauzassorry for interupting you, folks :)16:01
bauzaswho's around ?16:01
* stephenfin lurks16:01
bauzaswe have some large discussions today, let's start quickly16:02
bauzas#topic Bugs (stuck/critical) 16:03
bauzasNo Critical bug16:03
bauzas #link 13 new untriaged bugs (-0 since the last meeting): #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New16:03
bauzas no open bugs marked with xena-rc-potential tag #link https://bugs.launchpad.net/nova/+bugs?field.tag=xena-rc-potential16:03
bauzas please start marking release critical bugs with xena-rc-potential tag16:03
bauzasreminder, we are in RC phase16:03
bauzasmeaning that we should focus on regression bugs16:03
bauzasyoga is now the master branch16:04
bauzasany bug to discuss ?16:04
bauzas #topic Gate status 16:04
bauzas Nova gate bugs #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure16:04
bauzaswe have a long list16:04
bauzas Placement periodic job status #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly16:04
bauzaswe now we failed because of a dependency issue16:05
bauzasknow* we 16:05
bauzasbut now https://review.opendev.org/c/openstack/placement/+/810001 is merged16:05
bauzasso we can recheck 16:05
bauzasany concern so far ?16:05
bauzasmoving on16:06
bauzas Please look at the gate failures, file a bug, and add an  elastic-recheck signature in the opendev/elastic-recheck repo (example:  #link https://review.opendev.org/#/c/759967)16:06
bauzas #topic Release Planning 16:06
bauzas(we'll discuss the placement dependency bump later)16:06
bauzas Release tracking etherpad #link https://etherpad.opendev.org/p/nova-xena-rc-potential16:06
bauzasas you can see, nothing fancy to tell16:06
gibiwe need https://review.opendev.org/c/openstack/nova/+/810192 then an RC2 for nova16:06
bauzasmost of our efforts should go to merging bugfixes16:07
bauzasgibi: right, I forgot about it16:07
bauzasstrangely, LP says Fix Released for Xena https://bugs.launchpad.net/nova/+bug/194411116:08
gibifor some reason the bug is not showing up in the rc critical query 16:08
gibifeels like an LP bug around branching16:08
bauzaswill add it in the etherpad for tracking16:09
gibialready done :)16:09
bauzasnaïce ;)16:09
bauzasok, let's wait a bit for releasing RC2 16:09
bauzas We now have RC1 releases for both Nova and Placement #link https://review.opendev.org/c/openstack/releases/+/808706 and https://review.opendev.org/c/openstack/releases/+/80871316:10
bauzas We will need a Placement RC2 release due to #link https://review.opendev.org/c/openstack/placement/+/81000116:10
bauzashttps://review.opendev.org/c/openstack/placement/+/810193 needs special treatme16:10
bauzas+2d now16:10
gibiwe have the last RC deadline at 1st of Oct afaik16:11
gibiso yes, we can hold up RC2 a bit to see if anything else pops up16:11
bauzashence me saying let's hold a bit for a RC2 proposal16:11
bauzas Remember to propose regression bugfixes for a new RC with nova-xena-rc-potential16:11
bauzas #topic Review priorities 16:12
bauzas https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement)+label:Review-Priority%252B116:12
bauzasthis is absolutely empty16:12
bauzascores, please enjoy this new button until we discuss other ways to use it at the PTG16:12
bauzasis there anyone wanting to have their changes be a priority for reviews ?16:13
bauzas(that's your time, folks)16:13
artomWait, only cores can set the priority label?16:13
bauzaswith the current implementation, yes16:14
bauzasI'm about to propose other possibilities at the PTG16:14
bauzas(which reminds me I have to write something in the etherpad)16:14
artomCan a non-core propose the label, or is the whole thing owned entirely by cores?16:14
bauzasartom: dude, that's the whole discussion we had in the related doc change :)16:15
gibiartom: this is the currently agreed process https://review.opendev.org/c/openstack/nova/+/792357 but sure we can discuss it on the PTG16:15
dansmithsurely it will be totally useless if not curated by a smaller group right?16:15
artomSorry, not opening pandora's box again, just catching up16:15
bauzasI'm happy to see people offering alternatives :)16:15
dansmithin bug trackers where everyone can set priority, people set their bugs to be "urgent" all the time16:15
opendevreviewTakashi Natsume proposed openstack/nova master: Update min supported service version for Yoga  https://review.opendev.org/c/openstack/nova/+/80993216:15
dansmithI can't imagine it will be useful otherwise16:15
opendevreviewTakashi Natsume proposed openstack/nova master: Update min supported service version for Yoga  https://review.opendev.org/c/openstack/nova/+/80993216:15
bauzasdansmith: I had an alternative proposal I left in the comments but we can discuss this at the PTG (and I'll propose a documentation change)16:16
artomEssentially, for my own selfish needs, all I can do with the Priority label is query it, and look at those reviews in case my opinion on them is of any use to anyone, right?16:16
bauzasartom: for the moment, the idea is that cores make some review vows when setting the label16:16
gibiartom: if you help merging reviews in priority then cores will have more bandwidth to look at other reviews including yours :D16:17
bauzasmeaning they engage themselves to review this patch in question16:17
bauzasbut I'm not sure we should continue this conversation now, we have some agenda left16:17
bauzas(and a stretched time)16:17
bauzasmoving on16:18
bauzas #topic PTG Planning 16:18
bauzas every info is in the PTG etherpad #link https://etherpad.opendev.org/p/nova-yoga-ptg16:19
bauzas If you see a need for a specific cross project section then please let me know (gibi or bauzas)16:19
bauzasgibi: I jinxed you :p16:19
bauzasat least we can see regular updates from sean-k-mooney :)16:19
gibiI expect more and more topics as we close to the PTG date16:20
gibiI think the current list is normal given we have a month still16:20
bauzasabsolutely right16:20
gibi(a bit less of a month16:20
bauzasI'm not afraid of having extra time16:20
gibime neither16:20
bauzasbut I'd say, take this month as an opportunity to start designing your features, so you can raise your questions at the PTG16:21
bauzasanyway, moving on16:21
bauzas #topic Stable Branches 16:21
bauzas(that could last a bit)16:21
bauzas nova's stable/ussuri and stable/train are blocked (due to latest virtualenv uses latest setuptools which removed use_2to3) 16:22
bauzasthe future proof solution would be to pin virtualenv during tox install  for stable branches, otherwise new errors can appear with every new  release of setuptools, virtualenv, etc16:22
bauzas until we decide about the right solution we can maybe set lower-constraints job as non-voting ( https://review.opendev.org/809955 )16:22
bauzas probably placement branches have the same errors16:22
bauzaselodilles: floor is yours16:22
gibibauzas: yes, this is the same error16:22
opendevreviewMerged openstack/nova stable/xena: Add missing __init__.py in nova/db/api  https://review.opendev.org/c/openstack/nova/+/81019216:22
gibiit affects a lot of projects that still uses decorator 3.4 as dep16:22
gibinova affected from ussuri backwards16:22
gibiplacement affected all the way to master16:23
gibi(master fix landed)16:23
lyarwoodah I missed this review sorry16:23
* lyarwood opens16:23
bauzassuper awesome16:23
sean-k-mooneywe may want to consider droping that dep at some point16:23
bauzaswe actually have a longer explanation in the open discussion section16:23
bauzaslet's just hold this discussion until that point16:23
sean-k-mooneyalthough we are unlikely to hit the same issue again16:24
bauzas(which we will have 30 mins for)16:24
bauzas#topic Sub/related team Highlights16:24
bauzasLibvirt (bauzas)16:24
lyarwoodbauzas: happy to take over the libvirt part now you're PTL btw16:24
bauzaslyarwood: awesome16:24
bauzasI was asking for it16:24
lyarwoodbauzas: not that I have anything for today but we have a few things this cycle16:25
bauzasany stuff to raise ?16:25
bauzas#info lyarwood to chair the libvirt subteam by now16:25
bauzaslyarwood: thanks for offering your name16:25
*** rpittau is now known as rpittau|afk16:25
artomDon't take it in vain now16:25
bauzas#topic Open discussion16:26
bauzasone last paperwork bit16:26
bauzas (gibi): release liaison role16:26
gibiso 16:26
gibibauzas: is now the PTL and the release liaison as well16:26
gibiwhich is suboptimal16:26
bauzaswhich means I'm also a bottleneck now16:26
gibido we have a volunteer to take that role over?16:26
gibiit is not hard, you will get cc-d to release proposal patches to check and approve16:27
bauzastbc, the release liaison role is about reviewing patches from the release team16:27
bauzasuntil either the PTL or the release liaison approves the patch, it can't land16:27
sean-k-mooneyi mean i can do it if no one else wants too i really dont mind16:28
bauzasexample https://review.opendev.org/c/openstack/releases/+/80870616:28
sean-k-mooneyi keep an eye on the os-vif ones anyway but someone on the stable team might make more sense16:28
bauzassean-k-mooney: appreciated16:28
bauzasI'll make the changes in the appropriate repo so you'd get automatically CC'd16:29
gibisean-k-mooney: thank you16:29
sean-k-mooneyno worries16:29
* bauzas just has to figure out which specific file to update in which specific repo :D16:29
elodilleswell, as I also a release core and also propose nova releases lately it would be weird to propose and approve my own patches and then +W o:)16:29
bauzaselodilles: heh, depending whether you're schizophrenic, this could work16:30
bauzasok, moving on16:31
bauzassean-k-mooney: thanks again16:31
bauzasnow the big discussion16:31
bauzaspasting the whole section16:31
bauzas (gibi): gathering opinions about the current lower-constraints failure.  16:31
bauzasbottom line: on stable branches we are installing tox which installs  virtualenv which bundles setuptools unconstrained. This now leads to  that we cannot install decorator 3.4.0 on stable any more as it depends  on "user_2to3" from setuptools but the recent setuptools 58.0 removed  support for that.16:31
bauzas  Options to resolve the situation  bump decorator major version from 3.4.0 to 4.0.0 on stable branches. Does it against stable policy?  pin virtualenv version on stable during tox install  disable lower-constraints testing16:32
bauzasshit, I pasted wrong16:32
bauzasgibi: your turn 16:32
bauzasexplain the 3 options16:32
gibilet me untangle that16:32
gibiso option 1) bump major version of decorator from 3.4 to 4.0 on stable. Is it allowed on stable?16:32
gibioption 2) pin virtualenv during tox install16:33
gibioption 3) disable lower-constraints job16:33
gibiI personally think that using unconstrained setuptools on stable is dangerous16:33
gibiso I would go with 2) long turn16:33
artomYeah, seems like 2 is the safest... what's the danger with it? Is there a catch?16:33
lyarwoodAgreed, 2 would be my choice 16:33
bauzasgibi: let's explain which stable branches again are impacted16:33
bauzasyou already told but this doesn't harm to tell again16:34
gibiso in nova we are impacted stable/ussuri and older16:34
sean-k-mooneygibi: technially i dont think we are allowe to bump miniums on stable on the other hand we already did it a while ago for the inital lower constraitnts fix16:34
gibion placement we are impacted on master but the fix has been landed16:34
gibion stable/xena we could release RC2 with the bump 16:34
elodillesI also like option 2, and actually found some (abandoned) trial from the past that was similar https://review.opendev.org/q/topic:constrain-tox-install16:34
gibibut on stable/wallaby and back the same quertion applies16:34
sean-k-mooneyfrom a disto point of view 2 is not great16:35
sean-k-mooneysince they cannot pin them the same way 16:35
bauzasoption 1 has a question I can't answer16:35
sean-k-mooneybut most distros also dont use lower constratints16:35
gibidoes LTS distros use unconstrained setuptools as well?16:35
bauzascan we deliver a .y release by bumping the decorator dependency without breaking the semver rules ?16:35
sean-k-mooneyrhel ignored the upper and lower constraitnts16:36
sean-k-mooneydebian used to look at lower16:36
sean-k-mooneyubunut used to look at upper16:36
sean-k-mooneywe can try 2 but i feel like we are going to end up with 316:36
gibisean-k-mooney: sure we could end up with 3 for different reasons :)16:37
artomSo 2 for distros means "maintain this old version of virtualenv in your repos", right?16:37
elodillesbauzas: I think we can, but it's probably not fortunate. semver says that req changes needs MINOR / .y version bump16:37
artomBut... don't we already have upper-constraints that does the exact same thing?16:37
gibiartom: yes, as virtualenv bundles setuptools16:37
bauzaselodilles: so option 1 doesn't break stable policy per se16:37
gibiartom: this is tricky as the tox install in our CI does not use constraint file16:38
gibiartom: we install tox, then with tox we install deps16:38
gibiartom: but when we install tox it pulls in setuptools16:38
artomOh, so this is before any -constraints is applied16:38
bauzasthe problem is with the venv, not the dependencies16:39
bauzasartom: right16:39
bauzasbecause of the bundle16:39
elodillesbauzas: I also haven't found that written, but I remember that we tried to avoid req changes16:39
bauzasbut, there are ways to create venvs without bundling setuptools, right?16:39
elodillesbauzas: and the problem is that we will face the same issue whenever something new things comes in16:40
artomWait, I though the decorator==3.4.0 install happened *inside* the venv, as a dependency?16:40
bauzaselodilles: agreed, option 1 only fixes the decorator issue and doesn't resolve any other issue16:40
bauzasI'm also in favor of option 216:40
gibiartom: to have a venv where you can install deps, you have to have the venv package installed16:41
bauzasbut I wonder whether we could just tell tox to create venvs without bundling setuptools16:41
stephenfinjust weighing in, 3 isn't an ideal solution either16:41
gibibauzas: I think if you have a venv without setuptools then you dont have pip in the venv either16:41
gibibauzas: so you cannot install additional things with pip16:41
bauzasgibi: are you sure ?16:41
gibibauzas: 75%16:42
bauzasI think it does install pip 16:42
gibiI think pip depends on setuptools16:42
stephenfinin this case it's the lower-constraint that has broken, however, it's conceivable that the upper constraint for a particularly old release could also become incompatible with setuptools16:42
gibibut I could be wrong16:42
gibistephenfin: good point16:42
artomgibi, right, so we install tox on the "host", which pulls in unconstrained setuptools, which then break installing decorator==3.4.0, my question is, aren't we installing decorator==3.4.0 inside the venv?16:42
gibiartom: hm,16:42
artomSo it should be independent of what's on the "host"?16:42
gibiartom: you have a point16:42
gibiartom: for some reason our ci install tox already in a venv but I lost following that track 16:43
stephenfinartom: it installs setuptools in the venv also, right?16:43
artomDon't know... but then it'd be a separate venv for the actual nova install that pulls in decorator...16:44
gibiI agree that something is missing from our understanding about venvs and tox install16:44
stephenfin>>> setuptools.__file__16:45
sean-k-mooneyif we need to pin it we can pin the setuptools and pip version with requires16:45
bauzasI think it's doable16:45
clarkbtox pulls in virtualenv, virtualenv bundles setuptools for the new virtualenvs it makes16:46
gibisean-k-mooney: good finding16:46
sean-k-mooneyit is but fungi and clarkb  advised against doing that 16:46
elodillesin my abandoned trial in the past i tried to use upper-constraints.txt for tox install, it probably could solve the issue if that works (i don't remember whether it worked)16:46
clarkbvirtualenv has an escape hatch for that but I'm not sure how tox exposes that (if at all)16:46
clarkbyou need to pin virtualenv is the tldr if you are going to pin something16:46
clarkbbut you have to do it when installing tox16:46
bauzasyou can create a venv without setuptools and pip it later16:46
clarkbright but does tox support any of that?16:46
clarkbas it is tox making the virtualenv automatically16:47
bauzastox uses the venv module, right?N16:47
clarkbnot by default I think it can16:47
artomclarkb, gibi, ah, that's the missing understanding, when virtualenv creates a venv, it automatically stuffs the setuptools version it came with in that venv... right?16:47
clarkbbasically oyu have to see how much of this is configurable on the tox side to do what you want16:47
clarkbartom: yes exactly16:47
opendevreviewStephen Finucane proposed openstack/nova master: db: Add migration to resolve shadow table discrepancies  https://review.opendev.org/c/openstack/nova/+/80573816:47
opendevreviewStephen Finucane proposed openstack/nova master: tests: Walk database migrations in correct order  https://review.opendev.org/c/openstack/nova/+/81029116:47
bauzasclarkb: you're right, I need to dig into tox16:48
stephenfinThis looks like good background reading https://tox.readthedocs.io/en/latest/example/package.html16:48
gibiclarkb: thanks16:48
artomSounds like we have no choice but to pin virtualenv then16:48
stephenfinSounds like the escape hatches we need might be there16:48
artomUnless there's a way to tell virtualenv which setuptools version to install in the venvs in creates16:48
fungiif you want tox to use venv as the virtualenv backend, these days the trick is to export VIRTUALENV_CREATOR = venv in the inherited setenv in your tox.ini16:48
fungithere used to be a tox-venv plugin, but that was deprecated/abandoned once virtualenv grew the option to call out to the venv module itself16:49
bauzascan we use https://tox.readthedocs.io/en/latest/config.html#conf-requires ?16:49
fungiso it's not tox calling venv directly, but rather tox calling virtualenv and then virtualenv being told to use the venv module to create the env rather than its internal implementation16:49
sean-k-mooneyi feel like the complexity of maintianing this pining outways the usefullness of the job16:50
bauzassean-k-mooney: that's the problem with option 216:50
bauzasideally, I'd like us to maitain setuptools in our reqs16:50
bauzasmaintain* even16:50
bauzasbut this doesn't sound easy16:50
artomSo the value of the job is what - guarantee that will still work with the oldest feasible versions of things?16:51
artom(lower-constraints I mean)16:51
stephenfinsean-k-mooney: We will always have pinning in the form of upper-constraints though16:51
sean-k-mooneyyes basically to make sure when we backport we dont raise the requiremetns as a result16:51
clarkbthe correct way to maintain setuptools in your requirements is via the pyproject file16:51
sean-k-mooneystephenfin: right but for stable we are not allowed to raise miniums16:51
clarkbnot requirements16:51
stephenfinand? we can't raise maximums either16:52
clarkbthe new pyproject file allows you to specify build time deps (possibly even tools other than setuptools)16:52
clarkbI don't know what would be involved to make all of that work with pbr though16:52
bauzasclarkb: how other projects consider this issue ? do they want to pin setuptools as well ?16:53
bauzasor do they just fix the decorator issue ?16:54
bauzas(and backport without specific care)16:54
bauzasI guess we're not alone in the dark16:54
clarkbmsot of the fixes I've seen have been to fix the dependencies16:54
bauzasand about backports ?16:54
clarkbopenstack governance had to switch pydot2 to pydot for example16:54
clarkbzuul-jobs dropped support for old python3.5 which allowed it to use a newer version of a lib16:55
clarkbI don't know how backports or stable issues have been handled16:55
bauzasgibi: I guess you have to write something and bubble it to the community for cross-project thinking16:56
gibibauzas: I can write up a summary from today on the ML16:57
bauzasthat could become an epic saga, but it's worth starting it16:57
bauzasgibi: appreciated, here lemme summarize for the audience16:57
bauzasoption 2 seems to be the better, but we struggle finding a good way to write it16:57
artomStupid question, but could we not just install virtualenv==<version we want> first, and *then* tox?16:58
bauzasoption 1 seems to be the alternative, option 3 seems not accepted16:58
artomOr would tox just upgrade?16:58
gibiartom: that could be also something to try16:58
bauzasartom: virtualenv pulls the latest setuptools IIRC16:58
bauzasbut that's a flag16:59
gibibauzas: old virtualenv will pull old setuptools I guess16:59
bauzasagain, the problem is how to trigger this flag thru tox16:59
bauzasgibi: I remember having played with it before and you're right16:59
bauzasbut you can explicitely specific to pull the latest setuptools17:00
bauzaseither way, we're at time17:00
bauzasgibi: thanks for writing the summary and raising it to the community17:00
bauzasfolks, wrapping up17:00
opendevmeetMeeting ended Tue Sep 21 17:00:40 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:00
opendevmeetMinutes:        https://meetings.opendev.org/meetings/nova/2021/nova.2021-09-21-16.00.html17:00
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/nova/2021/nova.2021-09-21-16.00.txt17:00
opendevmeetLog:            https://meetings.opendev.org/meetings/nova/2021/nova.2021-09-21-16.00.log.html17:00
gibibauzas: thanks17:00
elodillesthanks bauzas o/17:01
stephenfinI've already tagged people, but there are some necessary fixes for the DB migration tests here that we should probably merge before I forget about them https://review.opendev.org/c/openstack/nova/+/81029117:02
stephenfinThe alternative is to rediscover this stuff once someone adds a DB migration in Yoga or later17:02
stephenfin(https://review.opendev.org/c/openstack/nova/+/805738 proves that they work)17:02
bauzasstephenfin: I just made use of the review-priority label for your change :)17:03
gibistephenfin: thanks, I will try to get to them17:03
opendevreviewStephen Finucane proposed openstack/nova master: tools: Ignore bot-generated branch creation patches  https://review.opendev.org/c/openstack/nova/+/81028517:04
stephenfinlyarwood: sorry for the dumb typo in that fixed. Bash is the worst. Fixed ^17:05
opendevreviewBalazs Gibizer proposed openstack/placement stable/xena: [WIP] try to pin setuptools via  tox.ini  https://review.opendev.org/c/openstack/placement/+/81029317:05
gibitrying [tox]/requires ^^ 17:07
* gibi ends his day17:12
bauzasgibi: me too17:12
gibiit seems [tox]requires does not help us pinning setuptools17:53
fungijust to clarify, very old virtualenv pulls old setuptools, but a while back virtualenv switched to pulling whatever the most recent setuptools available is unless you can tell it not to, so just pinning to a slightly older virtualenv doesn't help as it will still insist on grabbing the latest available setuptools18:46
fungialso there's the fact that this isn't purely a ci side problem, you can't really know or control what version of setuptools a user will have18:47
fungibecause it's considered part of the build environment not part of the runtime environment, so python packaging doesn't handle it the same way with the same expectations18:48
fungithe idea historically was that there were mechanisms to insist on a minimum acceptable version of build dependendencies and raise errors if they weren't new enough, but not really any reliable way to indicate maximum acceptable versions18:51
fungiof course, that was also predicated on build dependencies maintaining backward compatibility, and setuptools removing a feature is what's put us in this situation18:52
clarkbupstream pypa seems to say everyone should use pyproject.toml to address this19:00
clarkbwhich is why it is safe for them to drop functionality like that19:00
*** hemna7 is now known as hemna20:24

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!