Thursday, 2021-09-16

opendevreviewMerged openstack/nova stable/victoria: address open redirect with 3 forward slashes  https://review.opendev.org/c/openstack/nova/+/80662600:21
opendevreviewmelanie witt proposed openstack/nova master: Add stub unified limits driver  https://review.opendev.org/c/openstack/nova/+/71213703:40
opendevreviewmelanie witt proposed openstack/nova master: Assert quota related API behavior when noop  https://review.opendev.org/c/openstack/nova/+/71214003:40
opendevreviewmelanie witt proposed openstack/nova master: Make unified limits APIs return reserved of 0  https://review.opendev.org/c/openstack/nova/+/71214103:40
opendevreviewmelanie witt proposed openstack/nova master: Add logic to enforce local api and db limits  https://review.opendev.org/c/openstack/nova/+/71213903:40
opendevreviewmelanie witt proposed openstack/nova master: Enforce api and db limits  https://review.opendev.org/c/openstack/nova/+/71214203:40
opendevreviewmelanie witt proposed openstack/nova master: Update quota_class APIs for db and api limits  https://review.opendev.org/c/openstack/nova/+/71214303:40
opendevreviewmelanie witt proposed openstack/nova master: Update limit APIs  https://review.opendev.org/c/openstack/nova/+/71270703:40
opendevreviewmelanie witt proposed openstack/nova master: Update quota sets APIs  https://review.opendev.org/c/openstack/nova/+/71274903:40
opendevreviewmelanie witt proposed openstack/nova master: Tell oslo.limit how to count nova resources  https://review.opendev.org/c/openstack/nova/+/71330103:40
opendevreviewmelanie witt proposed openstack/nova master: Enforce resource limits using oslo.limit  https://review.opendev.org/c/openstack/nova/+/61518003:40
opendevreviewmelanie witt proposed openstack/nova master: Add legacy limits and usage to placement unified limits  https://review.opendev.org/c/openstack/nova/+/71349803:40
opendevreviewmelanie witt proposed openstack/nova master: Update quota apis with keystone limits and usage  https://review.opendev.org/c/openstack/nova/+/71349903:40
opendevreviewmelanie witt proposed openstack/nova master: Add reno for unified limits  https://review.opendev.org/c/openstack/nova/+/71527103:40
*** bhagyashris|off is now known as bhagyashris05:34
lpetrutHi, I have a question about keypairs. Nova allows a single keypair to be associated with a vm, yet in some cases we must inject multiple keys. We're using the k8s CAPO provider so we can't really use the userdata directly to inject additional keys. Now, apparently it's possible to bundle multiple ssh keys with the same keypair. Can we rely on this behavior to remain available? Fwiw, when bundling multiple keypairs, apparently each keypair must 07:54
lpetruthave a comment, otherwise nova will fail to generate a fingerprint and reject it.07:54
bauzasgood morning Nova07:58
bauzasfor the first time during this week, I eventually have a bit time for going upstream...07:58
gibibauzas: o/ could you please check the comments on the prelude 07:59
bauzaslpetrut: the API doesn't look it supports more than one public key for a keypair07:59
bauzaslpetrut: https://docs.openstack.org/api-ref/compute/?expanded=create-or-import-keypair-detail07:59
bauzasgibi: sure, will look07:59
gibithank you08:00
bauzasI also want to work for the vgpu documentation08:00
gibialso would be nice to land this doc https://review.opendev.org/c/openstack/nova/+/809161 and link it to the prelude08:00
gibisure, if you push vgpu doc ping me and I will prioritize it08:00
bauzasgibi: ack, will look at it today08:01
lpetrutbauzas: we're passing multiple ssh keys separated by newline. apparently other people rely on it as well: https://help.switch.ch/engines/faq/how-to-use-multiple-ssh-keys/08:01
lpetrutit's an ugly workaround, but it would be nice if we could continue to allow it until nova gets to support associating multiple keypairs08:11
bauzaslpetrut: well,08:13
bauzasif this works, fine08:13
bauzasbut we can't say we would continue to support, given our API doesn't say this08:13
lpetrutmakes sense08:13
bauzasbut we could create a API microversion for supporting multiple public keys per keypair08:14
bauzasthat said, I'm not sure what could be stopping to have public keys08:15
bauzasgiven we ask for a string08:15
bauzasunless we verify this string08:15
lpetrutthere's some validation going on when the fingerprint gets generated08:16
lpetrutbut if the first key has a comment, the rest of the payload seems to be treated as a comment and ignored08:16
bauzashah08:23
bauzasI see08:23
bauzaswell, unless the input validation changes...08:23
bauzaslpetrut: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/keypairs.py#L4708:25
bauzaswell, this is treated from the API as a full string08:27
bauzasso...08:27
bauzasdon't be that afraid08:27
bauzaslpetrut: anyway, changing this contract would require a microversion so in case you think you're trampled, you could use an old microversion for importing your multiple-pub keypair08:28
lpetrutgreat, thanks. just wanted to be sure that others are aware of this situation as well, hopefully we'll be able to improve the API eventually.08:28
bauzaslpetrut: well, an opensource project can't be "aware" of how people use it08:28
lpetrutabout the api change, wondering which should be the best option: multiple keys bundled by a single keypair, or multiple keypairs associated with a single vm08:28
bauzaswe try to remember exotic usages, but for best effort, we always say that things that aren't tested in CI are unsupported08:29
lpetrutyep, definitely08:29
bauzasas we could break things 08:29
bauzaslpetrut: good question about the draft, I'd say this would be discussed in a spec08:30
lpetrutthis might require some cloud-init changes as well08:30
bauzasgibi: i'm tempted to rebase the prelude above lyarwood's doc change, thoughts on it ?09:16
gibibauzas: works for me09:16
bauzasok, working on it09:16
bauzasanway, needs to provide a new rev for the prelude09:17
opendevreviewSylvain Bauza proposed openstack/nova master: Add the Xena prelude section  https://review.opendev.org/c/openstack/nova/+/80778609:25
opendevreviewMerged openstack/nova master: docs: Add nova-volume volume_attachment refresh admin workflow  https://review.opendev.org/c/openstack/nova/+/80916109:34
bauzaswoah, the gate is quiet for a RC1 day09:43
bauzasgibi: working now on sean-k-mooney's doc change https://review.opendev.org/c/openstack/nova/+/806412 09:45
bauzaswe could merge it soon09:45
bauzasand before the prelude so we could add it in the prelude09:45
sean-k-mooneyam i dont know if we need to mention it in the prelude10:07
sean-k-mooneybauzas: have we not already mentioned the mdevs10:07
bauzassean-k-mooney: yes we told about them in the prelude10:08
bauzashttps://review.opendev.org/c/openstack/nova/+/80778610:08
sean-k-mooneybauzas: so we can proceed with the docs change but i dont think we need to update the prelude for it10:10
bauzassean-k-mooney: do you want to work on the doc change or do you let me fixing the nits ? 10:10
sean-k-mooneyill leave it to you10:11
bauzasok, will work on it later after lunch10:20
bauzasour lovely customer leaves us quiet for the moment :)10:20
sean-k-mooneyoh dont jinx us like that10:20
kashyapHeh10:25
opendevreviewOpenStack Release Bot proposed openstack/placement stable/xena: Update .gitreview for stable/xena  https://review.opendev.org/c/openstack/placement/+/80936310:36
opendevreviewOpenStack Release Bot proposed openstack/placement stable/xena: Update TOX_CONSTRAINTS_FILE for stable/xena  https://review.opendev.org/c/openstack/placement/+/80936410:36
opendevreviewOpenStack Release Bot proposed openstack/placement master: Update master for stable/xena  https://review.opendev.org/c/openstack/placement/+/80936510:36
opendevreviewOpenStack Release Bot proposed openstack/placement master: Add Python3 yoga unit tests  https://review.opendev.org/c/openstack/placement/+/80936610:36
*** slaweq__ is now known as slaweq13:23
*** abhishekk is now known as akekane|home13:24
*** akekane|home is now known as abhishekk13:25
opendevreviewArtom Lifshitz proposed openstack/nova stable/victoria: Update SRIOV port pci_slot when unshelving  https://review.opendev.org/c/openstack/nova/+/79690913:52
melwittgibi: apologies if I missed it but I was wondering what are we doing about placement release note prelude and release? I thought about it since we have a couple of new features this time15:05
gibimelwitt: in the last couple of release we had no placemen release prelude written15:06
melwittack15:06
gibiinterrestingly https://docs.openstack.org/releasenotes/placement/unreleased.html this is open15:06
gibis/open/empty/15:06
gibithat feels bad15:06
melwitthm... that's weird, I thought we had added renos15:07
gibiyeah I do remember we added15:07
gibirunning tox locally now15:07
melwittyeah, just checked and consumer types definitely had a reno15:08
gibihm, locally I get a proper releasenotes generated15:09
gibiwe have renos for 1.37 and 1.3815:09
gibithat is the two feature we added in Xena15:10
melwitthm.. ok I'll try to figure out what's wrong with it. I don't remember off the top of my head how the doc publishing works but I think I could find it15:10
gibithanks15:11
sean-k-mooneythat i think is based on tags15:11
sean-k-mooneyor something in the comit15:12
sean-k-mooneythat marks the start/end of a releas15:12
melwittwell they are supposed to show up in the "unreleased" area soon after merge15:13
melwittthe creation of the release page like "Xena" is a different thing15:13
sean-k-mooneyyes but they might be showing up in a xena section15:13
melwittthere is no xena section. the notes are nowhere15:14
bauzasplacement should have an unreleased.rst file15:14
bauzasif not, it's something we could do15:14
melwittI get why we don't have a xena section, we probably didn't do a release yet. but the notes should be in the unreleased area and there's nothing there15:15
gibimaybe we need this patch https://review.opendev.org/c/openstack/placement/+/80936515:15
bauzasgibi: no this is the xena stable one15:15
bauzasonce we branch with RC115:15
bauzasoh wait15:15
bauzaswe did merge RC115:15
melwittyeah, I'm saying that normally notes go live on the doc page in "unreleased" soon after merge15:15
bauzashence the fact you no longer see them15:15
gibiyepp placement has RC1 I think15:15
bauzasgibi: I approved it sooner15:16
bauzashence why they disappeared15:16
bauzasokay, +2ing the placement stable/xena reno one15:16
melwittoh.. ok15:16
melwittthanks for explaining that bauzas 15:16
bauzasmelwitt: sorry, my fault15:16
bauzasI approved RC1 cut15:16
bauzasbut I should have thought about the reno patches15:17
bauzasideally the release team could make them dependent15:17
bauzasbetween the release proposal and the reno update15:17
gibibauzas: this is not a fault. first we need to approve the release then the tooling proposes the stable branch creation and then the stable branch setup patches15:17
bauzasthat also explains why gibi was getting the notes15:18
bauzashe didn't cut yet I guess :p15:18
gibiI did not pull from remote15:18
gibiso yes15:18
bauzaset voila15:18
bauzaseven, et voilà15:18
bauzas(with an accent)15:18
melwittfahncy15:18
bauzasgibi: thanks for explaining15:19
bauzasso the patches only appear once we branch ?15:19
gibiI hope I'm correct :D15:19
gibibauzas: it cannot appeare before as there is no stable/xena to propose them against15:19
bauzasthat's an explanation :D15:20
* bauzas facepalms15:20
* bauzas rolls eyes at the guy who was release liaison since Liberty15:20
bauzasthat now said, gibi, I didn't had time to work on sean-k-mooney's doc change15:21
bauzas:/15:21
bauzasI guess we should approve the prelude and cut RC115:21
gibibauzas: then we go with the current reno15:21
bauzasyeah15:21
gibithen doc can be backported15:21
bauzaswe could update the prelude later15:21
bauzasthat works15:22
gibiok15:22
bauzasit's just, we need a prelude section before RC1 in order to see it for 24.0.0 15:22
bauzasor the prelude would only appear in 24.0.1 15:22
gibiI see15:22
gibicool15:22
bauzasbut once we merge stuff for 24.0.0, touching the prelude would also update the 24.0.0 (if I'm not wrong)15:22
bauzasthat's one of the reno odds15:23
bauzasdansmith has a point, revising the prelude15:24
gibiok, then merge the prelude15:24
bauzasgibi: hold your approval15:24
gibiohh, sure15:24
bauzasI mean, don't explicitely approve :p15:24
opendevreviewSylvain Bauza proposed openstack/nova master: Add the Xena prelude section  https://review.opendev.org/c/openstack/nova/+/80778615:25
bauzasgibi: dansmith: melwitt: any other around ^15:26
dansmithway ahead of you15:26
gibidone15:26
bauzasdansmith: man, you're way too much productive compared to me15:26
dansmithI was just waiting for it after the mention above ;)15:27
melwittseeya soon, highlights o/15:27
bauzas++15:27
dansmithoh man, an early morning melwitt spotting!15:27
dansmithwell, "early"15:27
melwittuh huh15:28
dansmith:P15:29
fungielodilles: i noticed yesterday you were pushing forward on some nova stable branch changes (thanks!) and wanted to mention that the vmt is waiting for the rest of https://review.opendev.org/q/I95f68be76330ff09e5eabb5ef8dd9a18f5547866 to merge so we can publish errata for ossa-2021-00216:05
fungisean-k-mooney: on that note, the failures on 806629 imply that fix may not be backportable as-is to train?16:06
fungilooks like it could be as simple as a couple of missing imports though16:07
melwittfungi: hm, looks like the original fix didn't merge yet on train as well16:10
fungimelwitt: oh, interesting yeah16:10
fungithat being 79180716:11
melwittfungi: I'll help review the ones that I didn't propose, I didn't realize we needed to wait for ussuri and train, I had thought the original fix notice went out after the victoria change merged16:11
melwittyeah16:12
fungimelwitt: we need stable/ussuri merged first ideally in this case, since it's still in a maintained state (projected transition to extended maintenance is november)16:13
melwittfungi: ack, will prioritize16:13
fungii didn't realize the original train fix never got approved either16:13
fungibut you're right that's less important, i can in theory link to the unmerged patch for train though i'm hesitant to do so unless it's actually passing ci jobs16:14
fungias i could be inadvertently encouraging someone to merge a change which just causes nova to start crashing16:15
fungier, to import a change into their deployment i mean16:15
opendevreviewElod Illes proposed openstack/nova stable/ussuri: address open redirect with 3 forward slashes  https://review.opendev.org/c/openstack/nova/+/80662816:17
melwittfungi: yeah makes sense. the train change (the original) is a combo of two patches squashed together bc a test change was needed in order for the test to run on python < 3.6. and the followup should be rebased over it16:17
sean-k-mooneyso presumable if we rebase the train one on top of the first patch that might fix it?16:18
fungiaha, that 'splains the errors. i can push that16:18
fungiunless someone else is already on it16:18
fungiahh, though there are merge conflicts between them as well, so i'll defer to someone familiar with doing nova backports16:19
fungii don't want to get the conflicts stuff in the commit message wrong16:19
sean-k-mooneyim on a call but if melwitt  does not get to it i can try and stack them16:19
fungithanks!16:19
elodillesfungi melwitt : I've updated the commit message of the ussuri patch ^^^16:20
melwittthanks elodilles 16:20
fungithanks elodilles!!!16:20
elodillesso that it's nice and clean (& ready to merge)16:21
fungitrying to make sure this ossa update doesn't fall through the cracks16:21
melwitt++ thanks fungi 16:23
melwittsean-k-mooney: I've got it16:32
opendevreviewMerged openstack/placement master: Update master for stable/xena  https://review.opendev.org/c/openstack/placement/+/80936516:32
opendevreviewmelanie witt proposed openstack/nova stable/train: address open redirect with 3 forward slashes  https://review.opendev.org/c/openstack/nova/+/80662916:34
clarkbHello, I've got a cloud (the inmotion cloud you might see some of your CI jobs running on) that nodepool is failing to boot instances on.16:34
clarkbthe compute log says things like Instance f98ce366-90b1-43ba-8513-bf2ea559c931 has allocations against this compute host but is not found in the database.16:35
clarkband the conductor log says things like nova.exception_Remote.NoValidHost_Remote: No valid host was found.16:35
clarkb`openstackclient limits show --absolute` shows that we aren't using any quota.16:35
clarkbBut it appears that nova is essentially saying the compute hosts are full up and it can't schedule more work?16:36
melwittclarkb: orphaned allocations in placement :(  are making the hosts appear to be using resources when they shouldn't be16:36
clarkbIs there a way to list those without digging into a database? osc server list is empty too16:37
melwittthis is the doc for dealing with that https://docs.openstack.org/nova/latest/admin/troubleshooting/orphaned-allocations.html probably just need to run 'nova-manage placement heal_allocations' tool16:37
clarkbthank you16:37
fungiremember to lay your hands on the server when you chant "heal_allocations" too16:39
fungiit doesn't really make the command work any better, but it looks cool if anyone happens to be walking by16:40
melwitt😂16:40
melwittthis is just another reminder to prioritize the healing service (that would run heal_allocations periodically on its own, among other things) that's been in the backlog that I haven't had time to work on16:42
opendevreviewMerged openstack/nova master: Add the Xena prelude section  https://review.opendev.org/c/openstack/nova/+/80778616:58
melwittelodilles: oh no, l-c job failure again https://review.opendev.org/c/openstack/nova/+/80662817:40
sean-k-mooney    The user requested decorator>=3.4.017:47
sean-k-mooney    The user requested (constraint) decorator==3.4.017:47
sean-k-mooney....17:47
sean-k-mooneythat does not feel like it should be a conflict17:47
sean-k-mooneysince 3.4.0==3.4.0 is true and 3.4.0>=3.4.0 is also true17:48
melwittyeah17:48
melwittthere's also this error "error in decorator setup command: use_2to3 is invalid."17:48
clarkbI think fungi said those issues may come up when pypi serves us stale indexes17:48
melwittI don't know what that means17:48
clarkbthe use_2to3 error is due to a new setuptools I think they removed that flag17:48
sean-k-mooneyoh     error in decorator setup command: use_2to3 is invalid.17:48
sean-k-mooneyso this is not deps related17:49
sean-k-mooneyya17:49
melwitthttps://zuul.opendev.org/t/openstack/build/4290861d5a464d099ad38165c99b647d/log/job-output.txt#79217:49
sean-k-mooneythe deps are fine but its unhappy with setuptools17:49
clarkbbasically python software with modern pypa tools are expected to be python3 and not converted17:49
sean-k-mooneyclarkb: is there a compat flag we can enable17:50
sean-k-mooneyor jus tmove this to python 317:50
melwittaside: the jump to link doesn't seem to be working for me lately when I link to a line number in a zuul output17:50
sean-k-mooneyya that only worked for me if the file is small17:51
sean-k-mooneyit highlights it17:51
sean-k-mooneybut does not move to it if its not loaded quick enough17:51
melwittyeah. hrm. I wonder if something changed. the above ^ link doesn't jump me to the line and it loads really fast17:51
clarkbsean-k-mooney: I think pypa isn't interested in having compat flags. Updating or replacing deps is probably necessary17:52
sean-k-mooneymelwitt: if i open it https://zuul.opendev.org/t/openstack/build/4290861d5a464d099ad38165c99b647d/log/job-output.txt#792 then change it to https://zuul.opendev.org/t/openstack/build/4290861d5a464d099ad38165c99b647d/log/job-output.txt#79117:52
sean-k-mooneyit works fine17:52
sean-k-mooneyclarkb: ya or droping lower constraints. i just checked and its on py317:53
sean-k-mooneywe have https://github.com/openstack/nova/blob/stable/train/tox.ini#L1017:54
sean-k-mooneyand we do not override it for lower constraints17:54
sean-k-mooneyclarkb: the other option we have woudl be to downgrade setuptools17:55
sean-k-mooneypin it in the tox env to one that works17:55
sean-k-mooneyperhaps usign requires https://tox.readthedocs.io/en/latest/config.html#conf-requires17:55
sean-k-mooneythat would be a stable only change i guess if we did that.17:56
sean-k-mooneyclarkb: do you know if this is affecting anyone else17:56
sean-k-mooneyi know a lot of project just deleted there old lower constriants jobs17:57
clarkbthe governance repo had problems with pydot2 which hasn't been maintained for years17:58
clarkbI don't think we want ot downgrade setuptools if we can avoid it. We can't really control what version of setuptools others use effectively and wide compatibility is desireable17:59
clarkb(note with the whole pyproject.toml stuff you do get a bit more control but openstack hasn'tdone any of that)17:59
sean-k-mooneyya its more we have 3 options delete the job, update the dep or hack around to make it work18:00
sean-k-mooneypinnign setup tools on an em branch is just that18:00
clarkbcan you bump the dep version up such that it works?18:00
sean-k-mooneya hackaround to make it work18:00
clarkbthat is all lower constraints is supposed to track iirc. The oldest version that works18:00
sean-k-mooneyclarkb: well we are technialy not allow to bump min verison on stable right18:01
sean-k-mooneybut pratically speaking yes we could18:01
sean-k-mooneyand have to keeep it working already18:01
clarkbya for stable maybe removing the job entirely makes sense18:02
funginote the most recent setuptools release was a week ago, not sure if that lines up with when these failures started18:02
fungihowever tox updated today18:03
fungiso it may have started vendoring a newer setuptools18:03
fungithis is one of the reasons "pinning" setuptools is complicated18:03
sean-k-mooneyya18:05
sean-k-mooneylooking at the releases https://pypi.org/project/decorator/#history18:05
sean-k-mooneywe are using 3.4.0 now18:05
sean-k-mooneyfrom october 201218:05
fungiwe, actually it's virtualenv which vendors setuptools, and that was also released today18:05
sean-k-mooneythe next release 3.4.1 is 201518:05
fungiv20.8.0 (2021-09-16): upgrade embedded setuptools to 58.0.4 from 57.4.0 and pip to 21.2.4 from 21.2.318:06
fungiso my guess is this is setuptools 58.x.x behavior brought in by tox 20.818:06
fungiyou could try manually downgrading setuptools<58 in the created env to find out, but i don't recommend that as a fix18:07
sean-k-mooneyya im just trying to repodcue it localy now18:09
fungiv58.0.0 Breaking Changes: Removed support for 2to3 during builds. Projects should port to a unified codebase or pin to an older version of Setuptools using PEP 518 build-requires.18:09
sean-k-mooneyGLOB sdist-make: /home/sean/repos/nova/setup.py18:09
sean-k-mooneythat new to me ^18:09
sean-k-mooneyi have not seen a GLOB sdist-make line before18:09
sean-k-mooneyThe conflict is caused by:18:10
sean-k-mooney    jinja2 2.10 depends on MarkupSafe>=0.2318:10
sean-k-mooney    The user requested (constraint) markupsafe==1.018:10
sean-k-mooneyso we have other conflict too aparently18:10
fungiand earlier i said tox 20.8 but meant virtualenv 20.818:10
sean-k-mooneyImportError: cannot import name 'Feature' from 'setuptools' 18:11
fungisean-k-mooney: yes, this is one of the reasons i contended calculating and maintaining a lower constraints list would be unworkable long term18:11
sean-k-mooneythe import error was from markupsafe with tox 3.20.118:12
sean-k-mooneyalthoguh ok i ghet the same on ewith latest tox18:12
sean-k-mooney3.24.4 18:12
fungiyeah, downgrading tox probably won't help. you more likely need to downgrade setuptools, maybe virtualenv, and possibly pip18:13
clarkband rebuild the venv18:13
sean-k-mooneyyeah am elodilles melwitt  how would you feel about makeing it non voting of killing it on stable/train18:14
sean-k-mooneyclarkb: yep used -r to rebuild the venv18:14
fungiright. the chain of problems is that these days tox *always* wants to use the latest available version of virtualenv, virtualenv regularly vendors in its own copy of latest setuptools, and setuptools occasionally drops features used by very old packages18:14
sean-k-mooneyi could try messing with the other packages but realisticlly we agree that pinning is not what we want to do18:14
melwittsean-k-mooney: this sounds like a case for removing the l-c job. also I saw this on stable/ussuri (I guess that means it's also happening on stable/train?)18:15
sean-k-mooneyoh i assume train train is broke with a diffent error18:15
fungialmost certainly18:15
sean-k-mooneybut still broken18:15
dansmithI'm just jumping in here, but yes, whatever $reason is a good case to drop the l-c job :P18:15
fungier, train almost certainly would suffer from this if it also has a l-c job i mean18:15
sean-k-mooneyfungi: https://github.com/openstack/nova/blob/stable/train/.zuul.yaml#L37518:16
sean-k-mooneyso yes18:16
sean-k-mooneywe still have the template18:17
melwitttomorrow we can ask elodilles what he thinks. I think he's likely done for today 18:17
sean-k-mooneywe fixed it the last time https://github.com/openstack/nova/commit/b2037fc4e356b55949339a1358c16431a9ab893018:17
fungiright, i personally never had high hopes for the l-c experiment (mainly because of the forward flow of time and packages from a decade ago not being able to reasonably predict modern systems), but especially not for stable branches. i would cut my losses on l-c if it were me, but it's not my call18:17
dansmithwe had some resistance to removing it in glance,18:17
dansmithbut then we got to a point where we couldn't add a new dep version or pip would run off into lala land, which helped seal the deal :)18:18
sean-k-mooneyit would resolve the disccion im haveing with stephenfin too18:18
sean-k-mooneywhere he wanted to have it pinend to the oldest python we support for a release18:18
sean-k-mooneywhich i disagreed with18:18
sean-k-mooneysince that was not realistic to how distors worked18:19
fungiwell, basically if you head down that road, using old dependencies is not isolated to the python deps you have, those don't exist in a vaccuum and they make assumptions about contemporary systems as a whole18:19
sean-k-mooneyspeaking of being done for today i likely should finish up soon18:20
fungii agree it's not a very good way to try to mimic what stable distros experience, because 1. they pin a lot more about the entire environment than we reasonably can, and 2. they backport fixes to things which wouldn't be reflected with this test methodology anyway18:20
sean-k-mooneysince this is now efffectily blocking the backport of a cve fix however i think we likely should move forward with droping it so we can do the release18:21
fungianother alternative is early transition out of maintenance mode for stable/ussuri, but that seems like the most drastic choice18:21
sean-k-mooneyfungi: well it was more the version of the packate and the version of python are often uncoupled18:22
fungiyes, that too18:22
fungianyway, lower constraints jobs are not mandated by the pti, so dropping them doesn't affect the maintenance state for the branch18:22
fungiat least not in any official sense18:22
opendevreviewDmitrii Shcherbakov proposed openstack/nova master: [PoC][yoga] Off-path Networking Backends Support  https://review.opendev.org/c/openstack/nova/+/80819919:08
gibibauzas: I've update the nova rc1 patch with the prelude hash https://review.opendev.org/c/openstack/releases/+/808706 I think it is good to go now 19:48
admin1hi @all .. how does a nova snapshot work when both glance and nova are backed by ceph ? does it still create a local disk on the hypervisor and uploads ? 20:34
admin1or does everything happen entirely on the ceph side ? 20:35
admin1my snapshot is kind of stuck .. even for a small instance .. and i don't see anything in the hypervisor  ( files, or uploads or disk waiting ) so i think its entirely on the ceph side .. just wanted to validate 20:39
dansmithadmin1: if you have them share a pool, nova just asks ceph to do the snapshot in-place and then nova "tells" glance about it.. no bits fly around like normal20:52
dansmith("them" meaning glance and nova)20:52
opendevreviewmelanie witt proposed openstack/nova master: Add section for 'nova-manage placement audit' tool  https://review.opendev.org/c/openstack/nova/+/80947920:56
admin1nova is on vms pool and glance is on images pool .. so 2 diff pools 20:56
dansmithadmin1: okay so I think that means it's going to download and then re-upload it, but not totally positive20:58
* dansmith has to duck out for a bit20:58
melwittadmin1: if you have configured your deployment like this https://docs.ceph.com/en/mimic/rbd/rbd-openstack/#configure-openstack-to-use-ceph it should be able to do the fast snapshot20:59
admin1melwitt , refering to enabling copy-on-write cloning of images ? 21:00
melwittyes21:00
admin1its not enabled .. 21:01
admin1what does exactly "Note that this exposes the back end location via Glance’s API, so the endpoint with this option enabled should not be publicly accessible." mean ? 21:01
admin1i will try to find more 21:01
admin1melwitt, in absense of this setting, how does the nova snapshot work ? 21:02
melwittanother thing that can cause it to not do the CoW is if the image is not in RAW format, but I think glance might automatically convert to RAW now, depending on what release you have21:02
admin1all images are in raw format 21:02
admin1i converted them before uploading to glance 21:02
melwittthat is a warning that the backend url will be shown to users querying glance API so it should not be a url that is publicly accessible (for security reason)21:02
melwittok cool. if you don't set up as described in that ceph doc, nova will download the image from glance first and then boot the instance21:03
melwittif the image is large, it can be slow21:04
melwittthere's some glance setting around chunk size which you might want to adjust to get better throughput, I think it's this? https://docs.openstack.org/glance/latest/configuration/glance_api.html#glance.store.rbd.store.rbd_store_chunk_size21:08
melwitt*settings21:09
opendevreviewmelanie witt proposed openstack/nova master: Add section for 'nova-manage placement audit' tool  https://review.opendev.org/c/openstack/nova/+/80947921:18
admin1melwitt, when you meant backend url, is that the url of the ceph ? 21:26
admin1i mean its osd url 21:27
admin1melwitt, what glance command can i use to see the url being exposed .. 21:29
opendevreviewmelanie witt proposed openstack/nova master: Add section for 'nova-manage placement audit' tool  https://review.opendev.org/c/openstack/nova/+/80947922:07
melwittadmin1: yeah ceph url I think. you can see it by doing an 'image show' if you have enabled showing it https://docs.openstack.org/api-ref/image/v2/index.html?expanded=show-image-detail#show-image22:19
melwitthttps://docs.openstack.org/python-openstackclient/latest/cli/command-objects/image-v2.html#image-show22:20
dansmithadmin1: that location thing means glance will show things like rbd://mycephserver:port/path/to/thing, and so you likely want a different endpoint with that enabled that regular untrusted users can't see22:45
dansmithadmin1: without the fast cow snapshot, nova will pause the VM, download its disk, flatten it out, upload it to glance, and then unpause the VM22:46
dansmithadmin1: it's far more efficient to just let ceph snapshot it for you (pretty much instantaneously) and then just tell glance it exists22:46

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!