Friday, 2020-05-15

*** tetsuro has joined #openstack-nova00:04
openstackgerritmelanie witt proposed openstack/nova master: Configure latex_engine = 'xelatex' for pdf docs build  https://review.opendev.org/72789800:05
openstackgerritmelanie witt proposed openstack/nova master: Fix pep8 errors uncovered by flake8 3.8.1  https://review.opendev.org/72831200:09
*** tbachman_ has joined #openstack-nova00:13
*** lbragstad has joined #openstack-nova00:15
*** tbachman has quit IRC00:16
*** tbachman_ is now known as tbachman00:16
*** JamesBenson has joined #openstack-nova00:34
openstackgerritBrin Zhang proposed openstack/nova master: Fix flake8 error from hacking <3.0.1  https://review.opendev.org/72758900:37
gmannmelwitt: we need this to be in to have right cap for hacking and flake8 - https://review.opendev.org/#/c/727347/100:44
gmanni thought it was merged00:44
gmannbut is doc job still failing ?00:44
melwittgmann: rechecking not gonna help, docs job still broken00:45
melwittit did merge but didn't stop the failure, I replied on the ML post00:46
gmanni thought openstackdocstheme 2.1.0 fixed that on gate00:46
melwitt2.1.1 was intended to fix it but it did not work00:47
melwittmy nova side fix is still proposed but now we have a chicken and egg with the pep8 job00:47
melwittI sent a ML post about that too a little while ago00:47
gmannah just saw, i think we can squash it with 72789800:48
brinzhanggmann: the hacking verion should change to 3.1.0?00:49
brinzhangright?00:49
gmannbrinzhang: yeah in your patch yes as it will fix the code together00:50
brinzhanggmann: ok, update soon00:50
melwittI wonder if alex_xu is around00:51
gmannmelwitt: this should avoid doc failure - https://review.opendev.org/#/c/727943/400:56
melwittgmann: it doesn't because 2.1.1 also fails00:56
melwittif you look in the docs job build logs, 2.1.1 is installed now00:57
openstackgerritBrin Zhang proposed openstack/nova master: Bump hacking min version to 3.1.0  https://review.opendev.org/72758900:58
gmannmelwitt: not sure why not complete revert that instead if backlist. PS1 was right revert - https://review.opendev.org/#/c/727943/1/upper-constraints.txt00:58
brinzhanggmann: updated, I just only keep the hacking min version to 3.1.0, i.e. hacking>=3.1.0, please check01:00
*** ttsiouts has joined #openstack-nova01:00
*** ttsiouts has quit IRC01:05
*** Liang__ has joined #openstack-nova01:09
openstackgerritBrin Zhang proposed openstack/nova master: Bump hacking min version to 3.1.0  https://review.opendev.org/72758901:18
brinzhanggmann: do we need to continue merged hacking 3.0.1 https://review.opendev.org/#/c/727347/?01:22
*** brinzhang_ has joined #openstack-nova01:31
*** brinzhang has quit IRC01:34
openstackgerritGhanshyam Mann proposed openstack/nova master: Bump hacking min version to 3.0.1  https://review.opendev.org/72734701:48
melwittgmann: lol, what a day huh!01:51
melwitt(gate stuff)01:51
gmann:). I think we made little hurry to release new hacking.01:54
gmannI vote is to move back openstackdocstheme to 2.0.2 as nova, cinder are hitting doc + pep8 and may be more projects but not checked01:55
gmann*my vote01:55
gmannasked in requirement channel, let's see if we can get any other fast way to unblock01:56
melwittyeah, I lean that way too01:56
openstackgerritBrin Zhang proposed openstack/nova master: Bump hacking min version to 3.1.0  https://review.opendev.org/72758901:57
gmannbecause hacking is blacklist req so it cannot be controlled on single place (g-r side).01:58
melwittI see01:58
*** sapd__x has joined #openstack-nova02:01
openstackgerritBrin Zhang proposed openstack/nova master: Bump hacking min version to 3.1.0  https://review.opendev.org/72758902:08
*** gyee has quit IRC02:18
*** icarusfactor has joined #openstack-nova02:25
*** factor has quit IRC02:26
melwittgmann: do you plan to depends-on the roll back patch or did you want to try it out some other way?02:35
gmanni can do that.02:36
melwittor could do a dummy patch02:36
openstackgerritGhanshyam Mann proposed openstack/nova master: Bump hacking min version to 3.0.1  https://review.opendev.org/72734702:37
gmannmelwitt: as it needs to re-run anyways. done ^^02:37
melwittyeah, I saw it had already failed the docs job02:37
melwittcool I'll keep an eye on it02:37
*** icarusfactor has quit IRC02:51
*** icarusfactor has joined #openstack-nova02:52
*** ttsiouts has joined #openstack-nova03:01
*** ociuhandu has joined #openstack-nova03:05
*** ociuhandu has quit IRC03:09
*** xiaolin has quit IRC03:11
openstackgerritmelanie witt proposed openstack/nova master: Configure latex_engine = 'xelatex' for pdf docs build  https://review.opendev.org/72789803:13
melwittgmann: oh no, depends-on doesn't work for requirements changes??03:15
melwittstill failing03:15
melwittand pulling in 2.1.103:15
*** xiaolin has joined #openstack-nova03:15
gmannmelwitt: yeah, we need to get requirement revert merge first03:19
melwittgmann: oh really? depends-on known not to work in this case? I didn't know that, guh03:19
melwittI thought we were going to be able to test if job passes with the requirement revert03:20
*** tetsuro has quit IRC03:21
*** ttsiouts has quit IRC03:35
*** kevinz has quit IRC03:37
*** psachin has joined #openstack-nova03:39
gmannmelwitt: left note on this to recheck after req patch merge - https://review.opendev.org/#/c/727347/303:43
melwittgmann: cool, I've re-approved it so it's ready to go03:43
gmannbauzas: stephenfin gibi ^^ whoever wake up early. this is req patch to merge first - https://review.opendev.org/#/c/728335/03:44
gmannmelwitt: thanks.03:44
gmanngoing to bed...03:45
melwittk seeya03:45
*** factor has joined #openstack-nova03:45
*** kevinz has joined #openstack-nova03:46
*** icarusfactor has quit IRC03:47
*** xiaolin has quit IRC03:51
*** xiaolin has joined #openstack-nova03:51
*** factor has quit IRC03:51
*** factor has joined #openstack-nova03:52
*** xiaolin has quit IRC04:01
*** ratailor has joined #openstack-nova04:02
*** zzzeek has quit IRC04:23
*** zzzeek has joined #openstack-nova04:24
*** evrardjp has quit IRC04:33
*** ociuhandu has joined #openstack-nova04:33
*** evrardjp has joined #openstack-nova04:33
*** ociuhandu has quit IRC04:43
*** ociuhandu has joined #openstack-nova04:44
*** ociuhandu has quit IRC04:50
*** vishalmanchanda has joined #openstack-nova04:55
*** tetsuro has joined #openstack-nova05:11
*** tetsuro has quit IRC05:11
*** ttsiouts has joined #openstack-nova05:11
*** ttsiouts has quit IRC05:16
*** tetsuro has joined #openstack-nova05:17
*** artom has quit IRC05:25
*** udesale has joined #openstack-nova05:25
*** ociuhandu has joined #openstack-nova05:52
*** ccamacho has joined #openstack-nova05:56
*** dpawlik has joined #openstack-nova05:58
*** ociuhandu has quit IRC05:59
*** links has joined #openstack-nova05:59
*** xiaolin has joined #openstack-nova06:00
*** xiaolin has quit IRC06:26
*** dklyle has quit IRC06:27
*** xek_ has joined #openstack-nova06:30
*** xek__ has quit IRC06:32
*** tbachman has quit IRC06:37
*** nightmare_unreal has joined #openstack-nova06:39
*** jaosorior has joined #openstack-nova06:43
*** ociuhandu has joined #openstack-nova06:45
*** tesseract has joined #openstack-nova06:55
openstackgerritAndreas Jaeger proposed openstack/nova master: DNM: Test openstackdocstheme master  https://review.opendev.org/72837206:57
gibigood morning nova07:01
gibigmann: I see melwitt rechecked it07:03
melwittgibi: yeah rechecked it just a little while ago, once that lands, our gate should be unblocked. fingers crossed nothing goes wrong07:03
gibigmann: I'm approving https://blueprints.launchpad.net/nova/+spec/policy-defaults-refresh-deprecated-apis as we agreed about it on yesterday's team meeting07:03
*** maciejjozefczyk has joined #openstack-nova07:09
*** ralonsoh has joined #openstack-nova07:12
*** rpittau|afk is now known as rpittau07:12
bauzasgood Friday Nova07:18
* bauzas wonders whether he should reuse good Friday habits07:24
*** bauzas is now known as bauwser07:24
bauwserthere it is07:24
*** ttsiouts has joined #openstack-nova07:31
*** jraju__ has joined #openstack-nova07:33
*** links has quit IRC07:33
*** tosky has joined #openstack-nova07:35
*** xek__ has joined #openstack-nova07:41
*** slaweq has joined #openstack-nova07:41
*** xek_ has quit IRC07:43
*** ociuhandu has quit IRC07:51
*** ttsiouts has quit IRC07:55
*** ttsiouts has joined #openstack-nova08:01
*** jsuchome has joined #openstack-nova08:02
kashyapbauwser: /me waves08:05
*** tkajinam has quit IRC08:14
*** dtantsur|afk is now known as dtantsur08:17
*** ttsiouts has quit IRC08:19
*** avolkov has joined #openstack-nova08:30
*** ttsiouts has joined #openstack-nova08:31
*** salmankhan has joined #openstack-nova08:32
*** salmankhan has quit IRC08:36
*** salmankhan has joined #openstack-nova08:37
*** martinkennelly has joined #openstack-nova08:48
*** ociuhandu has joined #openstack-nova08:49
*** ttsiouts has quit IRC08:52
*** ttsiouts has joined #openstack-nova08:53
openstackgerritAndreas Jaeger proposed openstack/nova master: DNM: Test openstackdocstheme master  https://review.opendev.org/72837208:56
*** tetsuro has quit IRC09:09
*** tbachman has joined #openstack-nova09:10
*** jraju__ has quit IRC09:11
*** links has joined #openstack-nova09:12
*** ttsiouts has quit IRC09:16
lyarwoodstephenfin: https://review.opendev.org/#/c/676208/ - I'm onboard with using mypy just not adding it to the pep8 while it's still optional and not used outside of the libvirt driver.09:17
lyarwoodpep8 env*09:17
*** ttsiouts has joined #openstack-nova09:18
stephenfinlyarwood: I originally had a separate mypy zuul job and tox target but when I proposed said zuul job against openstack-zuul-jobs, mordred noted that zuul simply integrated it into their pep8 target (or linters, I think they call it)09:19
stephenfinhence why I did it that way09:19
stephenfinwhere would be better?09:19
*** links has quit IRC09:19
*** brinzhang_ has quit IRC09:24
lyarwoodstephenfin: okay it might not be worth splitting hairs over this but given it's not widely used in openstack/nova yet it seems like overkill to run it everytime we run pep809:25
lyarwoodstephenfin: I guess zuul is using it as standard across their codebase and that's why they've merged things09:26
*** links has joined #openstack-nova09:26
lyarwoodstephenfin: if gibi and others are cool with it being in the pep8 env then I'll shut up and let it merge :)09:27
gibistephenfin, lyarwood: is it something that makes tox -e fast8 target significantly slower?09:29
stephenfinit doesn't affect fast8, no09:29
stephenfinand even for pep8, it takes seconds to run09:29
lyarwoodwith a single file at the moment ;)09:29
lyarwoodbut yeah it's little to no overhead09:30
gibipep8 takes minutes so additional seconds does not cause any pain for me09:30
gibiso I'm OK to integrate it to the pep8 target09:30
lyarwoodack thank s gibi09:30
*** brinzhang has joined #openstack-nova09:30
stephenfinthree files two commits later!09:30
gibi:D09:30
lyarwoodlol, we can always break it out again if it does become an issue09:31
gibiagree ^^09:31
stephenfincool09:31
brinzhangstephenfin: do I need to remove this sentence from the commit message? https://review.opendev.org/#/c/727589/8//COMMIT_MSG09:31
stephenfinalso, is it just me, or is Gerrit really slow this week?09:31
lyarwoodstephenfin: just you09:31
stephenfinlike, 'git review -d foo' takes minutes09:31
stephenfindamn09:31
*** ircuser-1 has quit IRC09:32
* lyarwood hugs his 1Gbps line09:32
stephenfinthe web UI is a-ok, but pulling stuff is taking an age09:32
brinzhanggit review -d, I always need minutes09:32
lyarwoodI've really not noticed anything tbh09:32
* gibi does not use git review09:33
stephenfinbrinzhang: you don't need to exclude F811 now, do you?09:33
stephenfingibi: :O09:33
stephenfinso you manually copy download links?09:33
stephenfin*checkout links09:33
gibistephenfin: yepp, and I use git push origin HEAD:refs/for/master/my-topic to push things up09:33
brinzhangstephenfin: you mean I should remove F811 from flake8 ignore in tox.ini?09:34
stephenfinbrinzhang: I assume so, since you added noqa lines to the "duplicated" functions/methods?09:34
stephenfinThat is what F811 is for, right?09:34
brinzhangyes09:34
* gibi is an old school git guy09:34
brinzhangin the future, if someone need to add one function name as the same as before, they also need to add noqa.09:35
brinzhangok, I will remove F811 from tox09:35
* stephenfin thinks gibi's just bit masochistic09:36
brinzhanggibi: IMO, you can give a guide to use link to submit code, and update code.09:36
gibistephenfin: I have full control this way09:36
brinzhangI always used 'git review ', it seems slowly always09:36
gibistephenfin: e.g. I can push patches from a series during interactive rebase09:37
stephenfingibi: you mean half way through the rebase?09:37
gibistephenfin: yepp09:37
gibistephenfin: for long chains it helps09:38
stephenfinFair. I'd usually just download patch N locally and rebase all of that09:38
gibistephenfin: yeah that works too09:38
openstackgerritBrin Zhang proposed openstack/nova master: Bump hacking min version to 3.1.0  https://review.opendev.org/72758909:39
gibiI don't see what really git review automates that I cannot do manually with git with significant extra burden09:39
gibiI mean plain git seems as easy as git review to me09:39
stephenfinCan't argue with that09:41
* stephenfin hugs git-review tightly though09:41
*** brinzhang has quit IRC09:47
gibi:)09:48
*** Liang__ has quit IRC09:54
openstackgerritHarshavardhan Metla proposed openstack/nova master: [Nova] Add reference to Placement installation guide  https://review.opendev.org/72693610:05
bauwserfwiw, git-review does more than just uploading your series10:06
bauwserat least automatically rebasing on top of the remote10:08
*** rpittau is now known as rpittau|bbl10:09
*** ttsiouts has quit IRC10:19
*** songwenping_ has quit IRC10:20
openstackgerritJiri Suchomel proposed openstack/nova master: Adapt the admin guide to describe the direct RBD export  https://review.opendev.org/72841110:22
*** ttsiouts has joined #openstack-nova10:25
*** dpawlik has quit IRC10:27
*** dpawlik has joined #openstack-nova10:28
*** happyhemant has joined #openstack-nova10:28
*** breizhkoala has joined #openstack-nova10:29
*** breizhkoala has quit IRC10:37
*** breizhkoala has joined #openstack-nova10:37
*** derekh has joined #openstack-nova10:43
*** ttsiouts has quit IRC10:59
*** ociuhandu has quit IRC11:01
*** ociuhandu has joined #openstack-nova11:02
*** ociuhandu has quit IRC11:02
*** ociuhandu has joined #openstack-nova11:03
*** ociuhandu has quit IRC11:13
*** ociuhandu has joined #openstack-nova11:14
*** ociuhandu has quit IRC11:14
*** ociuhandu has joined #openstack-nova11:15
*** ttsiouts has joined #openstack-nova11:18
*** ociuhandu has quit IRC11:25
*** ttsiouts has quit IRC11:33
*** threestrands has quit IRC11:46
sean-k-mooneystephenfin: add mypy to the linters target i was suggesting lyarwood add bashate too and make it non voteing for now?11:57
*** efried has quit IRC11:57
*** efried has joined #openstack-nova11:58
sean-k-mooneystephenfin: then we can either move it to pep8 when we want to make it required or we can make the linters target voting to make both voting11:58
lyarwoodsean-k-mooney: I didn't even think about actually using a linters target12:01
sean-k-mooneylyarwood: yep you should. i left a comment to that effect on your patch but it took me like an hour to realise that is where you should put the bashate command too12:03
sean-k-mooneygiven the linter tox enve and job are a semi standard now, they are just not required in the pti12:04
sean-k-mooneyat least not yet anyway12:04
*** ociuhandu has joined #openstack-nova12:05
*** rpittau|bbl is now known as rpittau12:06
lyarwoodsean-k-mooney: yeah I've just seen your reviews on that bashate change, missed them earlier.12:07
lyarwoodsean-k-mooney: I've already ack'd stephenfin's initial change but we can follow up and move it into a linters target later12:08
sean-k-mooneywhere is his mypy change landing currently?12:09
lyarwoodsean-k-mooney: https://review.opendev.org/#/c/676208/12:09
sean-k-mooneyto be clear i was pretty ok with it landing in pep8 straight away but i dont think it deserves its own job or tox target12:10
*** udesale_ has joined #openstack-nova12:10
sean-k-mooneyok i see12:10
lyarwoodsean-k-mooney: kk12:10
sean-k-mooneyits just a seperate target but not added to any job12:10
sean-k-mooneythat works too12:10
sean-k-mooneybut ya linter i think is the way to go12:11
sean-k-mooneystephenfin: by the way if you want to add mypy to os vif i wont say no :)12:11
*** ociuhandu has quit IRC12:13
*** udesale has quit IRC12:13
*** raildo has joined #openstack-nova12:15
jsuchomehey ... I do not know what are you discussing, is it related to the fact that pep8 builds are failing with errors unrelated to the patch?12:16
sean-k-mooneyjsuchome: no12:16
sean-k-mooneyjsuchome: mypy is a checker for python type annotations12:16
sean-k-mooneyjsuchome: we are slowly starting to adopt it now that we are python3 only and can do so more easily12:17
lyarwoodjsuchome: https://review.opendev.org/727347 should fix the pep8 errors12:17
jsuchomeI see, thanks12:18
*** ttsiouts has joined #openstack-nova12:25
*** artom has joined #openstack-nova12:32
*** ratailor has quit IRC12:38
openstackgerritAndreas Jaeger proposed openstack/python-novaclient master: Switch to newer openstackdocstheme and reno versions  https://review.opendev.org/72827012:39
openstackgerritAndreas Jaeger proposed openstack/nova master: Switch to newer openstackdocstheme and reno versions  https://review.opendev.org/72826312:39
*** ociuhandu has joined #openstack-nova12:39
*** ttsiouts has quit IRC12:51
*** lbragstad_ has joined #openstack-nova12:54
*** lbragstad_ has quit IRC12:54
*** jangutter has joined #openstack-nova12:55
*** lbragstad has quit IRC12:56
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Update and correct typing information  https://review.opendev.org/71469412:58
stephenfinlyarwood: Addressed your comments on ^12:59
*** lbragstad has joined #openstack-nova13:01
*** sapd__x has quit IRC13:04
lyarwoodstephenfin: ack thanks13:07
openstackgerritBalazs Gibizer proposed openstack/nova master: Reserve DISK_GB resource for the image cache  https://review.opendev.org/72843913:08
gibidansmith: hi! fyi there is the first version of disk reservation patch for the image cache ^^13:09
*** nweinber has joined #openstack-nova13:10
gmanngibi: thanks13:10
francoisplyarwood, bauzas hello, when you get a chance: https://review.opendev.org/#/c/726508/ -thank you!13:11
lyarwoodfrancoisp: ack sorry done13:13
francoispthanks lyarwood13:15
openstackgerritMerged openstack/nova master: Bump hacking min version to 3.0.1  https://review.opendev.org/72734713:25
openstackgerritJiri Suchomel proposed openstack/nova master: Bring back allowed_direct_url_schemes in support of RBD image download  https://review.opendev.org/72809513:33
*** Sundar has joined #openstack-nova13:34
gmannfinally merged. gate should be green now13:35
*** ttsiouts has joined #openstack-nova13:38
openstackgerritJiri Suchomel proposed openstack/nova master: Adapt the admin guide to describe the direct RBD export  https://review.opendev.org/72841113:41
*** ttsiouts has quit IRC13:43
dansmithgibi: you decided on reserved instead of a new allocation?13:44
gibidansmith: yes, reservation is the current working assumption13:44
dansmithgibi: I really wasn't arguing that reserved was better (and I think I said that) I just wanted us to consider it13:44
dansmithgibi: it certainly looks simpler than a new allocation though. I was going to look for code that deleted the allocation if they moved the cache from shared to separate filesystem, but this would auto correct itself13:45
gibidansmith: I realized that we recalculate reservation at every update_provider_tree() call anyhow13:46
dansmithyeah13:46
gibiso I plugged my calculation into that loop13:46
dansmithyeah, even though this is less granular, it feels like a much smaller and less scary change, even something we could backport13:47
gibithere is one downside, placement allow bumping the reservation over the total - usage13:47
dansmiththe other downside is this lags the actual usage13:48
dansmithbut, very simple13:48
gibiwhen I pushed the patch I added a TODO to bump the reservation before we download an image but since then I lost in the imagebackend code again to find the place where we download image to the cache and it is faaar from easy13:49
dansmithyeah, no13:50
*** kevinz has quit IRC13:50
gibiI start thinking that from the reservation handling perspective this is what we can do. This is far from perfect but it has a good roi13:52
gibidansmith: would it be OK with you if we stop in the virt driver at this point? I will still look into adding the pre-filter part separately13:53
gibibut I'm scared from the imagebackend13:53
dansmithwhat do you mean "stop in the virt driver"?13:54
gibiI mean, not trying to bump the reservation before we download a new image to the cache13:55
dansmithoh yeah, I don't think that's the way to go.. we'd have to update inventory before each boot13:57
dansmithI think if we go the reservation approach, we accept the imperfectness of it, in exchange for its simplicity13:57
*** ratailor has joined #openstack-nova13:57
dansmithif it turns out to not help often enough, then we can move to the allocation method or something else13:58
dansmitha pre-filter will help avoid that13:58
dansmithI dumped some comments about the approaches on the patch just now13:58
gibilooking13:58
*** breizhkoala has quit IRC13:59
*** mlavalle has joined #openstack-nova13:59
openstackgerritLee Yarwood proposed openstack/nova master: compute: Allow snapshots to be created from PAUSED volume backed instances  https://review.opendev.org/72801113:59
openstackgerritLee Yarwood proposed openstack/nova master: compute: Remove snapshot quiesce tests for STOPPED and SUSPENDED instances  https://review.opendev.org/72801213:59
gibiOK so we agree that this is a good compromise as is. I will move the config to [workaround] for now as that is safer place.14:02
gibian sure I will add test and documentation to the patch14:02
*** icarusfactor has joined #openstack-nova14:02
dansmithcool14:03
gibidansmith: thank you14:03
dansmithgibi: you're doing the work, but... sure :)14:03
gibidansmith: talking about it helps :)14:04
dansmithgibi: my couch is always open14:04
gibi:D14:05
*** factor has quit IRC14:05
* dansmith strokes his (non-existent) beard14:05
aarentsgibi: dansmith thanks for handling this reservation stuff, I will backport this once finished for sure :)14:09
gibiaarents: I have a mandate to backport it to Train at least due to the downstream project I'm supporintg14:09
dansmithespecially if it's in [workarounds] I have no problem supporting it going back to train upstream, given it's something we broke and there are reports in the wild14:11
aarentsgibi Ok I will need it for stein, I will backport it to stein if possible14:12
gibiaarents: cool14:12
dansmithI think at some point in the lineage the provider tree stuff will differ enough that it will stop being trivial14:14
dansmithaarents: any chance you'd take an unmerged version of this, apply to one host and let us know how it works for you?14:17
dansmithit would suck to do all this, backport it way back and then find out the small race windows still let too much stuff through14:17
aarentsdansmith: yes will try to do it next week14:20
dansmithcool14:20
gibiaarents: I try to add some test to the patch still today14:23
aarentsok14:26
*** bnemec is now known as beekneemech14:27
*** kevinz has joined #openstack-nova14:29
*** links has quit IRC14:33
*** dklyle has joined #openstack-nova14:41
*** munimeha1 has joined #openstack-nova14:46
*** ttsiouts has joined #openstack-nova14:48
*** Sundar has quit IRC14:48
jsuchomedansmith: once you have time ... I addressed your comments in https://review.opendev.org/#/c/574301/14:51
dansmithyep, in the queue14:52
jsuchomegreat14:52
*** ttsiouts has quit IRC14:52
openstackgerritAndreas Jaeger proposed openstack/python-novaclient master: Switch to newer openstackdocstheme and reno versions  https://review.opendev.org/72827014:57
dansmithjsuchome: btw, the spec really needs to be merged before we move too far on the code15:00
openstackgerritAndreas Jaeger proposed openstack/nova master: Switch to newer openstackdocstheme and reno versions  https://review.opendev.org/72826315:02
*** mriedem has joined #openstack-nova15:07
*** udesale_ has quit IRC15:10
*** ratailor has quit IRC15:17
jsuchomewell, I just thought I start with something... nothing I can do with the spec right now15:38
openstackgerritStephen Finucane proposed openstack/nova master: tox: Integrate mypy  https://review.opendev.org/67620815:39
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Update and correct typing information  https://review.opendev.org/71469415:39
openstackgerritStephen Finucane proposed openstack/nova master: libvirt: Add typing information  https://review.opendev.org/71469515:39
openstackgerritStephen Finucane proposed openstack/nova master: tests: Split instance NUMA object tests  https://review.opendev.org/71469615:39
openstackgerritStephen Finucane proposed openstack/nova master: objects: Replace 'cpu_pinning_requested' helper  https://review.opendev.org/71469715:39
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Don't consider overhead CPUs for unpinned instances  https://review.opendev.org/71469815:39
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Remove handling of pre-Train compute nodes  https://review.opendev.org/71469915:39
*** maciejjozefczyk has quit IRC15:39
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Add validation for 'cpu_realtime_mask'  https://review.opendev.org/46820315:39
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Tweak the 'cpu_realtime_mask' handling slightly  https://review.opendev.org/46145615:39
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Rework 'get_realtime_constraint'  https://review.opendev.org/71470015:39
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Invert order of NUMA topology generation  https://review.opendev.org/71470115:39
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Remove '_numa_fit_instance_cell_with_pinning'  https://review.opendev.org/71470315:39
*** dpawlik has quit IRC15:41
*** ociuhandu has quit IRC15:47
openstackgerritHuaqiang Wang proposed openstack/nova master: Introduce 'pcpuset' field for InstanceNUMACell  https://review.opendev.org/71465815:57
openstackgerritHuaqiang Wang proposed openstack/nova master: Introduce the 'CPUAllocationPolicy.MIXED' enum  https://review.opendev.org/71626715:57
openstackgerritHuaqiang Wang proposed openstack/nova master: Calculate the CPU usage for mixed instance  https://review.opendev.org/71626915:57
openstackgerritHuaqiang Wang proposed openstack/nova master: online data migration for cpuset of dedicated instance  https://review.opendev.org/71849315:57
openstackgerritHuaqiang Wang proposed openstack/nova master: libvirt/hardware: refactor numa_get_constraints  https://review.opendev.org/71829415:57
openstackgerritHuaqiang Wang proposed openstack/nova master: Setup 'mixed' instance through 'PCPU' and 'VCPU' resource  https://review.opendev.org/71335515:57
openstackgerritHuaqiang Wang proposed openstack/nova master: metadata: export the vCPU IDs that are pinning on the host CPUs  https://review.opendev.org/68893615:57
openstackgerritHuaqiang Wang proposed openstack/nova master: notification: add the pcpuset info to instance numa cell payload  https://review.opendev.org/72848015:57
openstackgerritHuaqiang Wang proposed openstack/nova master: libvirt: apply mixed instance CPU policy  https://review.opendev.org/72848115:57
openstackgerritHuaqiang Wang proposed openstack/nova master: scheduler: mixed instance CPU resource translating  https://review.opendev.org/72848215:57
openstackgerritHuaqiang Wang proposed openstack/nova master: flavor: Enable 'hw:cpu_dedicated_mask' for creating a mixed instance  https://review.opendev.org/72848315:57
openstackgerritHuaqiang Wang proposed openstack/nova master: compute: bump nova-compute service version for mixed instance  https://review.opendev.org/72848415:57
openstackgerritHuaqiang Wang proposed openstack/nova master: hardware: create 'mixed' instance for realtime and shared CPUs  https://review.opendev.org/72848515:57
*** jaosorior has quit IRC16:03
openstackgerritStephen Finucane proposed openstack/nova master: Add '__version__' attribute  https://review.opendev.org/72849016:05
*** jmlowe has quit IRC16:07
*** jangutter_ has joined #openstack-nova16:07
*** nweinber has quit IRC16:09
*** jangutter has quit IRC16:10
*** gyee has joined #openstack-nova16:13
openstackgerritAndreas Jaeger proposed openstack/nova master: Switch to newer openstackdocstheme and reno versions  https://review.opendev.org/72826316:14
openstackgerritBalazs Gibizer proposed openstack/nova master: Reserve DISK_GB resource for the image cache  https://review.opendev.org/72843916:17
gibiaarents, dansmith: this time with tests ^^16:18
gibiand that marks the end of my work week. Have a nice weekend o/16:19
*** nweinber has joined #openstack-nova16:23
*** salmankhan has quit IRC16:24
*** kberger_ has joined #openstack-nova16:31
*** KeithMnemonic has quit IRC16:31
*** kberger_ has quit IRC16:32
*** kberger_ has joined #openstack-nova16:32
*** evrardjp has quit IRC16:33
*** evrardjp has joined #openstack-nova16:33
*** ociuhandu has joined #openstack-nova16:36
openstackgerritHuaqiang Wang proposed openstack/nova master: Introduce 'pcpuset' field for InstanceNUMACell  https://review.opendev.org/71465816:40
openstackgerritHuaqiang Wang proposed openstack/nova master: notification: add the pcpuset info to instance numa cell payload  https://review.opendev.org/72848016:40
openstackgerritHuaqiang Wang proposed openstack/nova master: Introduce the 'CPUAllocationPolicy.MIXED' enum  https://review.opendev.org/71626716:40
openstackgerritHuaqiang Wang proposed openstack/nova master: libvirt: apply mixed instance CPU policy  https://review.opendev.org/72848116:40
openstackgerritHuaqiang Wang proposed openstack/nova master: computer: validate cpu pinning configuration for mixed instance  https://review.opendev.org/71626816:40
openstackgerritHuaqiang Wang proposed openstack/nova master: Calculate the CPU usage for mixed instance  https://review.opendev.org/71626916:40
openstackgerritHuaqiang Wang proposed openstack/nova master: online data migration for cpuset of dedicated instance  https://review.opendev.org/71849316:40
openstackgerritHuaqiang Wang proposed openstack/nova master: scheduler: mixed instance CPU resource translating  https://review.opendev.org/72848216:40
openstackgerritHuaqiang Wang proposed openstack/nova master: flavor: Enable 'hw:cpu_dedicated_mask' for creating a mixed instance  https://review.opendev.org/71470616:40
openstackgerritHuaqiang Wang proposed openstack/nova master: compute: bump nova-compute service version for mixed instance  https://review.opendev.org/71627116:40
openstackgerritHuaqiang Wang proposed openstack/nova master: hardware: create 'mixed' instance for realtime and shared CPUs  https://review.opendev.org/72848516:40
openstackgerritHuaqiang Wang proposed openstack/nova master: libvirt/hardware: refactor numa_get_constraints  https://review.opendev.org/71829416:40
openstackgerritHuaqiang Wang proposed openstack/nova master: Setup 'mixed' instance through 'PCPU' and 'VCPU' resource  https://review.opendev.org/71335516:40
openstackgerritHuaqiang Wang proposed openstack/nova master: metadata: export the vCPU IDs that are pinning on the host CPUs  https://review.opendev.org/68893616:40
*** ociuhandu has quit IRC16:41
*** ociuhandu has joined #openstack-nova16:42
*** derekh has quit IRC16:47
openstackgerritStephen Finucane proposed openstack/nova master: WIP: Add emulated TPM support to Nova  https://review.opendev.org/63136316:49
openstackgerritStephen Finucane proposed openstack/nova master: WIP: Add support for resize and cold migration of emulated TPM files  https://review.opendev.org/63993416:49
openstackgerritStephen Finucane proposed openstack/nova-specs master: Encrypted Emulated Virtual TPM  https://review.opendev.org/72850516:56
*** jangutter_ has quit IRC16:58
*** psachin has quit IRC17:02
*** mlavalle has quit IRC17:05
*** mlavalle has joined #openstack-nova17:06
*** dtantsur is now known as dtantsur|afk17:08
*** rpittau is now known as rpittau|afk17:11
*** nightmare_unreal has quit IRC17:18
*** tesseract has quit IRC17:21
*** ociuhandu has quit IRC17:30
*** ociuhandu has joined #openstack-nova17:32
*** avolkov has quit IRC17:39
*** ociuhandu has quit IRC17:42
*** ociuhandu has joined #openstack-nova17:44
*** ociuhandu has quit IRC17:54
*** ociuhandu has joined #openstack-nova17:54
*** ociuhandu has quit IRC17:55
*** ociuhandu has joined #openstack-nova17:55
*** ralonsoh has quit IRC17:55
*** ociuhandu has quit IRC18:05
*** ociuhandu has joined #openstack-nova18:06
*** ociuhandu has quit IRC18:06
*** ociuhandu has joined #openstack-nova18:06
*** slaweq has quit IRC18:12
*** ociuhandu has quit IRC18:16
*** ociuhandu has joined #openstack-nova18:17
*** slaweq has joined #openstack-nova18:23
*** jsuchome has quit IRC18:27
*** slaweq has quit IRC18:28
*** nweinber has quit IRC18:33
*** iurygregory has quit IRC18:35
*** iurygregory has joined #openstack-nova18:38
*** slaweq has joined #openstack-nova18:38
mnaserhmm18:42
mnaseri have a query that's absolutely destroying my db server and it seems pretty silly looking at it18:42
mnaserhttps://www.irccloud.com/pastebin/RZ58u5w0/18:42
*** slaweq has quit IRC18:43
mnaseri literally have 2 cells inside cell_mappings, yet im having to scan all of the instance mappings18:43
mnaserexplain in my case shows scanning 1.2m rows.  yes, we could use an archive, but..18:44
mnaserhttps://www.irccloud.com/pastebin/vmPO2rNN/18:44
*** iurygregory has quit IRC18:45
mnaserit sounds like this is the root: https://github.com/openstack/nova/blob/master/nova/objects/cell_mapping.py#L275-L28518:45
mnaserwhich i guess comes from https://github.com/openstack/nova/blob/master/nova/compute/instance_list.py#L150-L15318:46
*** icarusfactor has quit IRC18:47
mnaserwhich leads to more interesting things.  instance_mappings seem to be not soft deleted yet i have 2586748 records...18:52
*** nweinber has joined #openstack-nova18:53
*** ociuhandu has quit IRC18:57
*** ociuhandu has joined #openstack-nova18:58
*** ociuhandu has quit IRC18:59
*** ociuhandu has joined #openstack-nova18:59
*** ociuhandu has quit IRC19:10
*** ociuhandu has joined #openstack-nova19:12
*** ociuhandu has quit IRC19:17
*** JamesBenson has quit IRC19:18
dansmithmnaser: instance mappings are not soft deleted,19:23
dansmithand are only ever removed when you archive19:23
mnaserdansmith: ah.  i'll try and add archive to happen more often because i think teh annoying thing is once a cloud grows big enough, the archive really hurt in terms of performance19:24
dansmithyeah, it's not a very lightweight thing19:25
dansmithmnaser: I guess we need to filter that instance mappings query by deleted eh?19:25
dansmither, no we can't19:25
dansmithduh19:25
mnaseryeah :(19:25
mnaseri mean there's the queued_for_delete, i saw a lot of records with that19:26
dansmithyou'd think it would be pretty fast though19:26
dansmithbecase it's just distinct cell_id, and then filtered by project19:26
dansmithmaybe we're missing an index19:26
mnaserthe select distinct is def what hurts, im trying to run it manually and see how long it takes..19:27
mnaser16.02 seconds19:27
dansmiththere is an index on project_id, but not on cell_id19:27
dansmithso maybe that is making the distinct take a long time?19:27
mnaserdansmith: i think the other problem is that t his code is also used for quota checks, so thats ~16s per *single* request to boot an instance19:27
mnaserwhen we'd see a big surge of instance creations, that's 16s*20 or so and it starts getting worse19:28
mnaserspecifically here https://github.com/openstack/nova/blob/575a91ff5be79ac35aef4b61d84c78c693693304/nova/quota.py#L1194-L119519:28
dansmithapparently mysql always indexes foreign key columns, so cell_id should be indexed19:28
mnaserhttps://www.irccloud.com/pastebin/MftvYe3g/19:29
mnaserthats the EXPLAIN19:29
dansmithmnaser: you specifically have the subset conf turned on, is that intentional?19:29
mnaserdansmith: sorry, nto following, subset conf?19:29
dansmithmnaser: see the comment just above the code in instance_list ?19:30
dansmith    query_cell_subset = CONF.api.instance_list_per_project_cells19:30
dansmithif that is off then it won't do this19:30
dansmithdefaults to false19:31
dansmithheh, the conf text (written by me I'm sure) says turning that on "causes one more (fast) query to the API database"19:31
mnaserheh, let me check if its enabled19:31
dansmithmaybe I should edit that comment or say "assuming you're archiving regularly" :)19:31
mnaserinstance_list_per_project_cells is not set to anything so its probably set to default False19:32
dansmithmnaser: okay well you see the code, not sure how you could be runningthat19:32
mnaserdansmith: https://github.com/openstack/nova/blob/575a91ff5be79ac35aef4b61d84c78c693693304/nova/quota.py#L1194-L1195 ?19:32
mnaserthat doesnt seem to respect it19:32
dansmithmnaser: no, the instance list one19:33
dansmithmaybe you're not actually hitting the instance_list code, just the quota one?19:33
mnaserdansmith: yes, sorry, should have clarified, i think that's the running theory19:33
dansmithokay I thought you assumed both19:33
dansmithI wrote instance_list so was focusing there19:33
mnaseri'm looking at logs and os_volume-boot is taking some ~45-60s-ish to respond19:33
mnaseryeah, initially that was my two findings, but i agree, it seems quota code might be what is hurting19:33
dansmithgotcha19:34
dansmithmelwitt: ^19:34
dansmithmnaser: maybe we could make the quota code check that same conf, as for <=1 real cell, it's probably quicker not to do the query anywa19:34
mnaserdansmith: yes, given we want to backport though, i'd be tempted to actually change get_by_project_id behaviour to respect that config value so its not possible for someone to miss it later19:35
dansmithno, we definitely don't want that19:35
mnaser(for the future, but backport just fixing the behaviour in quota.py)19:35
mnaserim sure there's details im missing why its not a good idea :)19:36
dansmithif a caller is looking for a set of instance mappings, you want it to get what it asked for, not altered by some random config flag19:36
mnaseryep makes sense19:37
mnaserthe patch to fix this seems trivial, so ill push something up19:38
dansmiththat's a very low-level database query routine.. the "should we optimize for lots of cells or not" is a very high-level decision19:38
dansmithmnaser: cool, add me and melwitt, but the latter is the important review I thnk19:38
mnaserdansmith: will do.  i'll write up a bug right now too19:39
dansmiththanks19:40
mnaserdansmith: i could also workaronud this by enabling quota from placement, but yeah19:40
mnaserlet's get it fixed19:40
dansmithmnaser: you could also archive your damned database now and then :D19:40
mnaserdansmith: haha, always me bringing up those massive db issues :p19:40
dansmithI guess the benefit of not archiving is giving us some data on these things, so ... your choice :D19:40
*** jamesdenton has quit IRC19:41
*** jamesden_ has joined #openstack-nova19:41
*** nweinber has quit IRC19:45
*** kberger_ has quit IRC19:48
*** KeithMnemonic has joined #openstack-nova19:48
mnaserit's friday and i did a bad job at writing this probably: https://bugs.launchpad.net/nova/+bug/187897919:53
openstackLaunchpad bug 1878979 in OpenStack Compute (nova) "Quota code does not respect [api]/instance_list_per_project_cells" [Undecided,New]19:53
*** jamesden_ has quit IRC19:53
*** jamesdenton has joined #openstack-nova19:53
*** jamesdenton has quit IRC19:59
*** jamesden_ has joined #openstack-nova19:59
*** jamesden_ has quit IRC20:08
mriedemmnaser: related https://bugs.launchpad.net/nova/+bug/185730620:08
openstackLaunchpad bug 1857306 in OpenStack Compute (nova) "_bury_in_cell0 could not handle instance duplicate exception" [Low,Triaged]20:08
mriedemwait a minute, that's not right20:09
mriedemhttps://review.opendev.org/#/c/700456/20:09
*** jamesdenton has joined #openstack-nova20:10
mnasermriedem: ehhhh, that's a lot to think about for friday :(20:12
*** jamesden_ has joined #openstack-nova20:13
mriedemyou're stuck inside, what else do you have to do20:13
*** jamesdenton has quit IRC20:15
*** jamesdenton has joined #openstack-nova20:21
*** jamesden_ has quit IRC20:23
mnaserwelp, _instances_cores_ram_count_legacy has zero tests20:37
*** jamesdenton has quit IRC20:38
*** iurygregory has joined #openstack-nova20:42
*** ociuhandu has joined #openstack-nova20:44
*** ccamacho has quit IRC20:45
mriedemi think those are covered with functional tests20:47
mriedemdepending on if the placement flag is configured to use placement for quota or not20:47
mnasermriedem: yeah i found some unit test ones but they mostly mock that function out20:47
mriedemyeah it's functional tests i'm pretty sure20:47
mriedemmore reliable for those kind of tests hitting the dbs20:47
mnasermriedem: ok thanks ill go dig there20:47
mnasermriedem: my test is mostly "if flag X is enabled then make sure you call with all cells otherwise make sure you retrieve list of cells20:48
*** ociuhandu has quit IRC20:49
openstackgerritMohammed Naser proposed openstack/nova master: WIP: Make quotas respect instance_list_per_project_cells  https://review.opendev.org/72857520:49
mnasermriedem: ^ for context20:49
mnaseryeah that code is broken now that i look at it again20:49
mnaserah no nvm, python lets vars defined in if get accessed outside20:50
openstackgerritLin Yang proposed openstack/os-traits master: CPU: add traits for new AVX512 support  https://review.opendev.org/72857620:52
*** munimeha1 has quit IRC21:06
*** raildo_ has joined #openstack-nova21:06
*** raildo has quit IRC21:08
openstackgerritLin Yang proposed openstack/os-traits master: CPU: add traits for new AVX512 support  https://review.opendev.org/72857621:15
*** martinkennelly has quit IRC21:21
openstackgerritMohammed Naser proposed openstack/nova master: Make quotas respect instance_list_per_project_cells  https://review.opendev.org/72857521:24
mnaserdansmith, melwitt: ^ cooked up and tested locally :) thanks for the hints21:24
openstackgerritGage Hugo proposed openstack/nova stable/stein: [DNM] - testing numa placement  https://review.opendev.org/72857921:28
*** ociuhandu has joined #openstack-nova21:32
*** ociuhandu has quit IRC21:40
*** tbachman has quit IRC21:40
*** bauwser is now known as bauzas21:41
*** KeithMnemonic has quit IRC21:47
*** JamesBenson has joined #openstack-nova21:49
*** raildo_ has quit IRC21:57
melwittmnaser: here's the func test for cores ram count https://github.com/openstack/nova/blob/master/nova/tests/functional/db/test_quota.py#L14222:32
melwittoh you found it. nevermind22:33
mnasermelwitt: :D22:35
mnaserI tried to make it as backportable as possible22:36
melwittgood call. this doesn't look too bad at all, so thank you for that22:37
melwittmnaser: I assume you tried out this patch? what was the perf result, just curious22:38
iklaI'm trying to get pci_passthrough to work and followed the documents. I keep seeing Pci claimed failed. Anyway to drill this down. Config looks perfect.22:39
mnasermelwitt: I have not yet actually!  It’s Friday and the load dropped so it wasn’t hitting as hard22:40
mnaserI can try on Monday though but I imagine it’ll drop scanning instance mappings and I only have 2 cells really (0 and 1)22:41
melwittmnaser: I guess, this seems like the opposite of what you wrote in the bug report. it's always limiting to project_id and you're adding some that would look at all cells?22:41
*** mgariepy has quit IRC22:42
mnasermelwitt: sorry I’ve been pretty slow today. But yes before it was limited to project ID and this just doesn’t limit and checks all cells anywyas22:42
melwittlike before your patch it's always only looking at a particular project_id and quota check is always scoped to a project, so there's no need to look at all cells22:42
melwittmnaser: if that's the case, that would mean there's a bug in objects.CellMappingList.get_by_project_id if it's not actually scoping down to a project22:43
mnasermelwitt: that function does actually scope by project. The only problem is that to do that, it needs to scan all instance mappings to find all cells for that project which is where I took the performance hit22:44
mnaserAfter my patch, nova checks all cells, regardless if a user has instances in it or not22:44
melwittand no worry, I've been slow today too. spent all day/night yesterday dealing with fallout from a nova gate breakage and trying to get it fixed. so exhausted22:45
mnaserSorry if my commit message or bug aren’t clear, I’m just feeling bleeeh and dazed all day lol22:45
melwittmnaser: sorry this just doesn't seem to make sense. before your patch, it's always limiting to one project cells. after you patch, it's capable of not limiting. I don't see how that would help with any instance mapping scanning?22:47
melwittoh hang on22:48
mnasermelwitt: I think we may be talking about two scans. One for the quota calculation and one to find which cells the user has instances in22:49
melwittok so you're saying when you can look at all cells, you actually go _faster_ because you don't do the instance mapping thing? wow ok22:49
melwittyeah ok it's a tradeoff, slowly remembering now22:49
mnasermelwitt: https://github.com/openstack/nova/blob/master/nova/objects/cell_mapping.py#L275-L28522:49
melwittthe project limiting behavior is actually worse for single cell-ers22:49
melwittyeah, I see now22:50
melwittsorry22:50
mnaserAll good!  Yeah, it hurt me more in this case22:50
melwittyeah, I see22:50
melwittnice catch22:50
mnasernot the catch I wanted on a Friday but it is what it is :P22:50
melwitthaha right22:50
melwittok, I'll review this properly a bit later. thanks for opening the bug and doing the patch22:51
mnaserNo worries22:55
iklaTimed out waiting for nova-conductor.  Is it running? Or did this service start before nova-conductor?23:05
iklaany ideas?23:05
iklaor normal23:06
openstackgerritMerged openstack/nova master: docs: Add evacuation pre-conditions around the src host  https://review.opendev.org/72385323:07
melwittikla: nova-conductor is a required piece of nova, it needs to be running so if it isn't, that would be a problem23:10
*** JamesBenson has quit IRC23:23
*** mlavalle has quit IRC23:31
openstackgerritMerged openstack/nova master: Reproduce bug 1869050  https://review.opendev.org/71499723:31
openstackbug 1869050 in OpenStack Compute (nova) "migration of anti-affinity server fails due to stale scheduler instance info" [Low,In progress] https://launchpad.net/bugs/1869050 - Assigned to Balazs Gibizer (balazs-gibizer)23:31
iklaInsufficient compute resources: Claim pci failed. - is there anyway to see why it failed?23:32
*** tosky has quit IRC23:39
*** mriedem has left #openstack-nova23:45

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!