Thursday, 2019-06-13

*** markvoelker has quit IRC00:09
*** liuyulong has quit IRC00:31
*** bhagyashris has joined #openstack-nova01:03
*** markvoelker has joined #openstack-nova01:05
*** lbragstad has quit IRC01:10
*** lifeless has joined #openstack-nova01:14
*** _erlon_ has quit IRC01:17
*** markvoelker has quit IRC01:39
*** boxiang has joined #openstack-nova01:45
*** boxiang_ has quit IRC01:47
openstackgerritDustin Cowles proposed openstack/nova master: Introduces SDK to IronicDriver and uses for node.get
openstackgerritDustin Cowles proposed openstack/nova master: Use SDK instead of ironicclient for node.list
openstackgerritDustin Cowles proposed openstack/nova master: WIP: Use SDK instead of ironicclient for validating instance and node
openstackgerritDustin Cowles proposed openstack/nova master: Use SDK instead of ironicclient for setting instance id
openstackgerritDustin Cowles proposed openstack/nova master: WIP: Use SDK instead of ironicclient for add/remove instance info from node
*** gyee has quit IRC02:11
*** licanwei has joined #openstack-nova02:13
*** whoami-rajat has joined #openstack-nova02:14
*** brinzhang has joined #openstack-nova02:17
*** BjoernT_ has quit IRC02:24
*** markvoelker has joined #openstack-nova02:36
*** guozijn has joined #openstack-nova02:38
*** lifeless has quit IRC02:38
*** boxiang has quit IRC02:39
*** boxiang has joined #openstack-nova02:39
*** lifeless has joined #openstack-nova02:44
*** cfriesen has quit IRC03:00
*** guozijn has quit IRC03:02
*** BjoernT has joined #openstack-nova03:06
*** JamesBenson has joined #openstack-nova03:09
*** markvoelker has quit IRC03:09
*** BjoernT has quit IRC03:35
*** ricolin has joined #openstack-nova03:50
*** dave-mccowan has quit IRC03:54
*** boxiang has quit IRC04:00
*** kaisers has quit IRC04:06
*** markvoelker has joined #openstack-nova04:06
*** igordc has joined #openstack-nova04:07
*** kaisers has joined #openstack-nova04:21
*** whoami-rajat has quit IRC04:23
*** JamesBenson has quit IRC04:24
*** markvoelker has quit IRC04:40
*** udesale has joined #openstack-nova04:41
*** janki has joined #openstack-nova04:52
*** ratailor has joined #openstack-nova05:00
*** pcaruana|afk| has joined #openstack-nova05:01
*** pcaruana|afk| has quit IRC05:04
*** pcaruana has joined #openstack-nova05:04
*** maciejjozefczyk has joined #openstack-nova05:10
*** ivve has joined #openstack-nova05:10
*** Luzi has joined #openstack-nova05:20
*** guozijn has joined #openstack-nova05:27
openstackgerritEric Fried proposed openstack/nova master: Make RequestContext(instance_lock_checked) fail
*** ricolin has quit IRC05:36
*** markvoelker has joined #openstack-nova05:37
*** takamatsu has joined #openstack-nova05:42
*** luksky has joined #openstack-nova05:45
*** jaypipes has quit IRC05:47
*** guozijn has quit IRC05:49
*** guozijn has joined #openstack-nova05:52 proposed openstack/nova-specs master: Add spec for expose-auto-converge-post-copy
openstackgerritMerged openstack/nova master: Remove cells v1 parameter from 'ComputeTaskAPI.resize_instance'
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix a warning about flags in an expression string
openstackgerritMerged openstack/nova master: libvirt: Use SATA bus for cdrom devices when using Q35 machine type
*** guozijn has quit IRC06:07
*** spsurya has joined #openstack-nova06:07
*** markvoelker has quit IRC06:10
*** zbr|flow is now known as zbr|ooo06:11
*** jaypipes has joined #openstack-nova06:11
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/ (19)
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/ (20)
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/ (21)
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove mox in unit/network/ (22)
*** brinzh has joined #openstack-nova06:23
*** brinzhang has quit IRC06:26
*** guozijn has joined #openstack-nova06:27
*** hamdyk has joined #openstack-nova06:28
*** vsk_17 has joined #openstack-nova06:49
*** xek_ has joined #openstack-nova06:53
*** rpittau|afk is now known as rpittau07:00
*** markvoelker has joined #openstack-nova07:08
*** dtantsur|afk is now known as dtantsur07:10
*** slaweq has joined #openstack-nova07:12
*** markvoelker has quit IRC07:13
*** tssurya has joined #openstack-nova07:13
*** igordc has quit IRC07:15
*** ccamacho has quit IRC07:15
openstackgerritBoxiang Zhu proposed openstack/nova master: Validate requested host/node during servers create
openstackgerritBoxiang Zhu proposed openstack/nova master: Add host and hypervisor_hostname flag to create server
*** tesseract has joined #openstack-nova07:21
bhagyashrissean-k-mooney: Hi,07:22
*** boxiang has joined #openstack-nova07:26
openstackgerritBoxiang Zhu proposed openstack/nova master: Make evacuation respects anti-affinity rule
*** jangutter has joined #openstack-nova07:31
*** helenafm has joined #openstack-nova07:31
*** trident has quit IRC07:31
*** takamatsu has quit IRC07:32
*** trident has joined #openstack-nova07:34
openstackgerritLee Yarwood proposed openstack/nova stable/stein: libvirt: Use SATA bus for cdrom devices when using Q35 machine type
openstackgerritBoxiang Zhu proposed openstack/nova master: Fix live migration break group policy simultaneously
*** mkrai has joined #openstack-nova07:40
*** guozijn has quit IRC07:40
*** mkrai_ has joined #openstack-nova07:41
mkrai_Hi Nova Developers, I request you to provide your input on
*** awalende has joined #openstack-nova07:43
openstackgerritBrin Zhang proposed openstack/nova master: WIP: Specify availability_zone to unshelve
yaawangsean-k-mooney: johnthetubaguy Hi, I've updated this spec, please take a look if you have time.
openstackgerritBrin Zhang proposed openstack/nova master: WIP: Specify availability_zone to unshelve
*** guozijn has joined #openstack-nova08:09
*** ttsiouts has joined #openstack-nova08:09
*** markvoelker has joined #openstack-nova08:10
*** whoami-rajat has joined #openstack-nova08:11
*** sridharg has joined #openstack-nova08:13
*** markvoelker has quit IRC08:14
*** ralonsoh has joined #openstack-nova08:15
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix cleaning up console tokens
openstackgerritTakashi NATSUME proposed openstack/nova master: Add a live migration regression test
*** takamatsu has joined #openstack-nova08:19
*** mdbooth has joined #openstack-nova08:20
*** ttsiouts has quit IRC08:20
*** ttsiouts has joined #openstack-nova08:21
*** rcernin has quit IRC08:23
*** jcosmao has left #openstack-nova08:24
*** jcosmao has joined #openstack-nova08:24
*** ttsiouts has quit IRC08:25 proposed openstack/nova-specs master: Re-propose the spec to allow specifying a list of CPU models
*** ccamacho has joined #openstack-nova08:27
*** priteau has joined #openstack-nova08:28
*** cdent has joined #openstack-nova08:28
*** jaosorior has joined #openstack-nova08:29
*** ociuhandu has joined #openstack-nova08:31
*** ccamacho has quit IRC08:32
*** ccamacho has joined #openstack-nova08:33
*** ttsiouts has joined #openstack-nova08:34
*** ociuhandu has quit IRC08:36
*** imacdonn has quit IRC08:38
*** imacdonn has joined #openstack-nova08:39
*** luksky has quit IRC08:39
*** dpawlik has joined #openstack-nova08:40
*** ociuhandu has joined #openstack-nova08:42
*** mdbooth has quit IRC08:47
*** mdbooth has joined #openstack-nova08:48
*** mdbooth_ has joined #openstack-nova08:48
*** tkajinam has quit IRC08:51
*** mdbooth has quit IRC08:53
hjensasHi Nova, anyone who can review (my first?) ever nova change -
*** brinzhang has joined #openstack-nova09:04
*** brinzh has quit IRC09:06
*** mdbooth has joined #openstack-nova09:06
*** helenafm has quit IRC09:07
*** mdbooth has quit IRC09:09
*** boxiang has quit IRC09:10
*** markvoelker has joined #openstack-nova09:10
*** markvoelker has quit IRC09:15
*** edleafe has quit IRC09:22
*** luksky has joined #openstack-nova09:22
*** mdbooth has joined #openstack-nova09:25
*** mdbooth_ has quit IRC09:27
*** derekh has joined #openstack-nova09:31
*** takashin has left #openstack-nova09:33
*** gmann has quit IRC09:33
*** mkrai_ has quit IRC09:37
*** slaweq has quit IRC09:45
*** slaweq has joined #openstack-nova09:46
*** helenafm has joined #openstack-nova09:49
*** udesale has quit IRC09:59
*** udesale has joined #openstack-nova09:59
*** ttsiouts has quit IRC10:00
*** ttsiouts has joined #openstack-nova10:00
*** maciejjozefczyk has quit IRC10:00
*** maciejjozefczyk has joined #openstack-nova10:01
*** bhagyashris has quit IRC10:04
*** ttsiouts has quit IRC10:05
*** hamdyk has quit IRC10:05
*** hamdyk has joined #openstack-nova10:05
*** markvoelker has joined #openstack-nova10:11
*** markvoelker has quit IRC10:15
*** mdbooth has quit IRC10:25
gibihjensas: I'm on it. thanks for working on it10:31
*** maciejjozefczyk has quit IRC10:33
*** tssurya has quit IRC10:37
*** ttsiouts has joined #openstack-nova10:42
*** guozijn has quit IRC10:44
*** bbowen has quit IRC10:49
*** awalende has quit IRC10:49
*** awalende has joined #openstack-nova10:50
*** whoami-rajat has quit IRC10:50
*** davidsha has joined #openstack-nova10:51
*** priteau has quit IRC10:54
*** _erlon_ has joined #openstack-nova11:05
*** maciejjozefczyk has joined #openstack-nova11:06
*** gmann has joined #openstack-nova11:09
*** maciejjozefczyk has quit IRC11:09
*** mdbooth has joined #openstack-nova11:10
*** mdbooth_ has joined #openstack-nova11:11
gibihjensas: I've replied to you in
*** mdbooth has quit IRC11:15
*** nicolasbock has joined #openstack-nova11:21
*** guozijn has joined #openstack-nova11:26
hjensasgibi: thanks, I'll take look.11:38
*** cdent has quit IRC11:41
*** bhagyashris has joined #openstack-nova11:49
*** bbowen has joined #openstack-nova11:53
jangutterjaypipes: Did you know about this? If PEP 594 gets adopted you won't be able to do 'import pipes' in Python? THEY'RE REMOVING YOU!11:55
jaypipesjangutter: what the actual f!?11:55
jaypipesit's a conspiracy!11:55
jangutterjaypipes: I guess they'll have to adopt the much superior 'import gutter' module.11:56
*** ohwhyosa has joined #openstack-nova11:57
*** ttsiouts has quit IRC11:57
fricklerwoa, being called a dead battery sounds really harsh to me11:58
*** ttsiouts has joined #openstack-nova11:58
*** awalende has quit IRC12:02
*** ttsiouts has quit IRC12:02
*** awalende has joined #openstack-nova12:03
openstackgerritHarald Jensås proposed openstack/nova master: cell_mapping - wrap IPv6 address in format_url
ohwhyosaHiya there!12:06
ohwhyosaI was advised to come here to ask about an issue, I'll try to summarize it as well as I can!12:06
ohwhyosaSo, I have a 3-nodes openstack-ansible deployment12:07
*** awalende has quit IRC12:07
ohwhyosainfra1 ( compute1( and storage1(
ohwhyosaIt was kinda working on the first deployment, using spice (though it worked horribly) and without multidomain suport12:08
ohwhyosaSo I nuked the lxc containers, deployed again with novnc and multidomain support12:08
ohwhyosaI nuked them using lxc-containers-destroy.yml12:08
ohwhyosawhen it was all up again, I couldn't create instances (it complained no hosts were available, though that was a lie, there were plenty resources)12:10
ohwhyosaLooking at the logs, placement said it couldn't assign resources to that node beacsue "hostname" already existed12:11
ohwhyosaI changed the hostname, rebooted everything, and now I can create instances12:11
sean-k-mooneyohwhyosa: so i think i know what the issue is12:11
sean-k-mooneylet me see if i can find the bug12:12
ohwhyosaBut when I try to access the console, I get "Invalid token, expired or invalid token"... I've checked the logs, and even took the liberty of modifying the python file to provide more logging information...and the token string is empty (not when it hashes it, though, because it does still hash an empty string)12:12
sean-k-mooneybasically im guessing teh lxv-contaiern-destroy destored the compute node containers but not placment right?12:12
*** markvoelker has joined #openstack-nova12:13
ohwhyosaIt should have destroyed everything, I think I only have containers on infra112:13
ohwhyosabut could be, that would explain why the hostname was already registered12:13
sean-k-mooneyis infra1 where your contol plain services run12:13
sean-k-mooneywhat i think happend is that we destroyted the compute agent and the compute service entry for that agent12:14
sean-k-mooneybut im assuming we could not destroy the plamcenet resouce providers because there were allcoation against them12:14
ohwhyosaI'm checking, and all the console_auth_tokens are on the nova database, cell0 has that table empty, in case that is of importance12:14
sean-k-mooneyso when you redeployed a new compute service was created12:15
ohwhyosaHmmm makes sense as a protective measure12:15
sean-k-mooneyand that had a new uuid and when it tried to create a new inventory in placment it failed because the old RP still existd wiht a differnet uuid but the same name12:15
ohwhyosaYup, it did complain about uuid (though that problem "vanishged" after changing hostname...which I now realize might not have been a great idea)12:16
ohwhyosaBut didn't lxc-containers-destroy nuke the DB as well?12:16
*** awalende has joined #openstack-nova12:16
sean-k-mooneyohwhyosa: if you check placement for a resouce provider with the onld hostname it should have allocation against it12:16
sean-k-mooneyif that is the case its the same issue we are currently trying to fix12:17
*** tssurya has joined #openstack-nova12:17
*** markvoelker has quit IRC12:17
*** spsurya has quit IRC12:18
ohwhyosaCommand would be openstack allocation candidate list?12:18
sean-k-mooneyam im not sure if we can do this vai the openstack cli but no i dont think so12:19
ohwhyosasean-k-mooney: How should I check?12:20
sean-k-mooneyyou need to hit this endpoint
sean-k-mooneybut ill see if there is a cli you can use one second12:20
ohwhyosaThanks! brb, will check12:20
sean-k-mooneyohwhyosa: this is the mailing list post related to this issue by the way
sean-k-mooneyohwhyosa: openstack resource provider allocation show12:22
*** ratailor has quit IRC12:22
sean-k-mooneyactully no that is not per RP its per consumer which is not what you want12:23
ohwhyosasean-k-mooney: that command returns "is not an openstack command"12:24
ohwhyosasean-k-mooney: It recommends resource member list, service provider list and service list12:24
sean-k-mooneyyou would need a new enough version of osc-placmenet for it to be there12:24
sean-k-mooneybut it also is incorrect for what you want to do12:25
*** takashin has joined #openstack-nova12:25
*** janki has quit IRC12:26
ohwhyosaHmmm according to openstack endpoints list tells me that placement api is at
ohwhyosaBut nmap tells me that port is closed and thus curl connectionr efused12:28
sean-k-mooneythat is strange12:28
sean-k-mooneydo you need to run the command within one of the lxc continers12:29
ohwhyosathe openstack one?12:29
sean-k-mooneyim not sure how osa setup your networking12:30
ohwhyosaYup, I use it from the utility container12:30
ohwhyosaOkay, now I have access to the api12:31
ohwhyosaI was using the external_lb_vip u.u12:31
sean-k-mooneywell i would normally suggest doing12:31
sean-k-mooneycurl -s  -H "X-Auth-Token: ${OS_TOKEN}" "${uuid}/allocations" | python -mjson.tool12:32
sean-k-mooneyassuming you have a token and know the uuid of the RP with the old host name12:32
sean-k-mooneyyou can get a token with "OS_TOKEN=$(openstack token issue -c id -f value)"12:33
sean-k-mooneyyou can get teh uuid with "uuid=$(resource provider list --name ${old_hostname} -c uuid -f value)"12:35
*** edleafe has joined #openstack-nova12:36
sean-k-mooney sorry  "uuid=$(openstack resource provider list --name ${old_hostname} -c uuid -f value)"12:37
*** lbragstad has joined #openstack-nova12:37
*** tssurya has quit IRC12:38
ohwhyosaresource member or service provider?12:39
ohwhyosaresource provider complains it doesn't exists12:39
ohwhyosaand thanks a lot, I'm learning a ton12:39
sean-k-mooneydo you have osc-placement installed? and if so what version12:40
ohwhyosaI believe I don't, I don't seem to be able to use any of its commands12:41
sean-k-mooneyok you could install it but since we are using curl anyway this will fined the RP for you12:42
sean-k-mooneycurl -s  -H "X-Auth-Token: ${OS_TOKEN}" "${old_hostname}" | python -mjson.tool12:42
sean-k-mooneyyou need to replace with your placemetn endpoint12:43
ohwhyosaNot found (neither with the new nor the old hostname)12:45
sean-k-mooneyya i had a typo12:45
sean-k-mooneycurl -s  -H "X-Auth-Token: ${OS_TOKEN}" "${old_hostname}" | python -mjson.tool12:45
sean-k-mooneyyou do not want the / at the end of the url i think that was the error12:45
ohwhyosaIndeed I have both the old name and the new one12:46
sean-k-mooneycool so if you grab the uuid for the old name and append <uuid>/allocations to that url12:47
sean-k-mooneythen it should not be empty on the old host name correct?12:47
sean-k-mooneye.g. curl -s  -H "X-Auth-Token: ${OS_TOKEN}" "${uuid}/allocations" | python -mjson.tool12:47
ohwhyosa  "resource_provider_generation": 2,12:48
ohwhyosa  "allocations": {}12:48
sean-k-mooneyhuh i was expecting that to be non empty12:48
sean-k-mooneywell the issue is that that RP was not deleted12:48
*** mkrai_ has joined #openstack-nova12:49
sean-k-mooneyi guess the lxc container destory playbook may not have actully deleted it properly12:49
ohwhyosaIt has self link, inventories link and usages link when getting the uuid12:49
ohwhyosaHmmm could be, should I just delete it?12:50
ohwhyosaAnd should that fix the novnc console also?12:51
sean-k-mooneyyou have changed the hostname so it should not be an issue.12:51
sean-k-mooneyi think the novnc console issue is seperate12:51
sean-k-mooneythe fact the RP exsited is why you could not originally recreate the invenoties when the compute agent started up12:52
ohwhyosaIt does have a populated resource inventory though12:52
ohwhyosaOooooh get it!12:52
ohwhyosaThanks a ton12:52
sean-k-mooneyya the invetory and RP are just left over form the last time the compute node ran update_provider_tree before you redeploy12:53
sean-k-mooneyyou should clean up the RPs so that the schduler does not try to select them but the nova filters shoudl eliminate them anyway12:54
sean-k-mooneybefore you try to delete them can you first check if there is a compute service with the old name still12:54
ohwhyosasorry, i deleted it before reading this12:54
ohwhyosaHow can I check if there is a service with the old name?12:55
sean-k-mooneyopenstack compute service list --host ${old_hostname} --service nova-compute12:56
ohwhyosaYep, there is one! enabled and down12:56
sean-k-mooneyya its down because it has not recived a heartbeat since you redeployed.12:56
ohwhyosaI should disable and delete it right?12:57
ohwhyosaSo, only two issues left with nova for now hahaha the novnc proxy and the err 32 broken pipe12:57
sean-k-mooneyif you do openstack compute service delete <service uuid> it will clean up the old placmenet RP too12:57
ohwhyosaOh! So it does already what I did via the api!12:58
sean-k-mooneythe playbook you ran jsut nuked the containers12:58
sean-k-mooneyit did not also clean up the compute resouces in nova or placmenet12:58
sean-k-mooneynormally if it reused the same host name it should have reused the same compute service but im guessing somthing went wrong12:59
*** awalende has quit IRC12:59
*** awalende has joined #openstack-nova13:00
ohwhyosaGood, good, note to self then, delete all the resources before nuking containers13:00
*** maciejjozefczyk has joined #openstack-nova13:00
*** pcaruana has quit IRC13:01
sean-k-mooneywell i would check with the openstack-ansible people on how they expect it to be done13:01
sean-k-mooneye.g. im sure they have a playbook or docs for reporviioning node somewhere13:01
sean-k-mooneyor they will be able to tell you what the workflow shoud be with there tooling13:02
sean-k-mooneyif they dont you can file a bug and im sure they would be happy to add some13:02
*** awalende_ has joined #openstack-nova13:02
ohwhyosaYup! I'll ask them, to be fair the whole #openstack-* ircs are full of helpful people, thanks a couple tons13:03
sean-k-mooneyohwhyosa: mnaser  is really approachable although he is also quite busy but im sure he and the other osa folk will be able to help too13:03
mnaseryeah i don't think we have a 'off-board compute' playbook13:03
mnaserwhichgoes and cleans up placement and what not13:03
sean-k-mooneymnaser: do you have a playbook that will reconfigure exisitng nodes?13:04
*** awalende has quit IRC13:04
ohwhyosaThanks! mnaser so you recommend deleting the service via cli?13:04
mnaseri mean usually the idea is that we run things and they automatically converge again.. in theory13:05
*** awalende_ has quit IRC13:07
*** mriedem has joined #openstack-nova13:10
*** jaosorior has quit IRC13:11
ohwhyosamnaser: Hmmm converge as in work again right? So, even if I hadn't changed the hostnamed, should it have started working again?13:13
*** markvoelker has joined #openstack-nova13:14
ohwhyosaAlso,  --> This is the broken pipe error, in case you want to take a look, and this13:15
ohwhyosaAnd this is the noVNC issue13:18
*** markvoelker has quit IRC13:18
ohwhyosaThe problem, though, is that the "token" var is an empty string13:18
mriedemamorin: do you plan on backporting this?
ohwhyosaI don't really know where is it querying from, it does mention something in the docs about the token being in the cookies13:19
ohwhyosaBut it is inthe url13:19
mriedemneed some stable cores on these +2ed rocky changes
lyarwoodack, I can look shortly13:20
lyarwoodmriedem: would you mind looking at that in return13:20
amorinmriedem: yes13:22
*** udesale has quit IRC13:22
*** udesale has joined #openstack-nova13:24
amorinmriedem: however, I think I will abandon this one:
*** mdbooth_ is now known as mdbooth13:25
amorinwe dont need to force refresh, nova DB is always correct in my tests13:25
amorinwe dont need to ask neutron13:25
amorinwhat do you think?13:25
mriedemi'd say we don't need it until we do :)13:26
mriedemthe force refresh is more for when the cache gets corrupted13:26
*** udesale has quit IRC13:29
openstackgerritBrin Zhang proposed openstack/python-novaclient master: Microversion 2.74: Support Specifying AZ to unshelve
*** awalende has joined #openstack-nova13:30
openstackgerritBrin Zhang proposed openstack/python-novaclient master: Microversion 2.74: Support Specifying AZ to unshelve
openstackgerritBrin Zhang proposed openstack/nova master: Specify availability_zone to unshelve
openstackgerritLeehom Li proposed openstack/nova master: Make sure update resource usage when revert_resize
*** awalende has quit IRC13:35
*** BjoernT has joined #openstack-nova13:37
*** brinzhang has quit IRC13:39
mnaserohwhyosa: better to move the convo over to #openstack-ansible to avoid causing noise here :)13:40
*** dave-mccowan has joined #openstack-nova13:40
openstackgerritMerged openstack/nova-specs master: Spec to pre-filter disabled computes with placement
ohwhyosamnaser:Nice! Going there, I though since it was nova related here would be better!13:43
sean-k-mooneyohwhyosa: it depends on the issue. it sounds liek this might be more related to the confgiration of the service rather then the code13:44
sean-k-mooneyohwhyosa: so #openstack-ansible might be able to help more13:44
*** ricolin has joined #openstack-nova13:45
ohwhyosaGreat! And thanks for all the help, really, I feel like an openstack jedi already (not even a padawan really, but the feeling is there)13:47
openstackgerritsean mooney proposed openstack/nova-specs master: Libvirt: add vPMU spec for train
sean-k-mooneynow ^ in the correct folder and some typos fixed :)13:53
*** hamdyk has quit IRC13:55
*** ttsiouts has joined #openstack-nova13:55
*** artom has joined #openstack-nova13:57
sean-k-mooneystephenfin: can you take a look at these quickly
efriednova meeting now in #openstack-meeting14:00
bhagyashrisstephenfin, sean-k-mooney: Hi, I just want your opinion about
sean-k-mooneybhagyashris i replied to both this morning14:01
*** Luzi has quit IRC14:02
*** BjoernT_ has joined #openstack-nova14:02
sean-k-mooneybhagyashris: and
bhagyashrissean-k-mooney;  Thank you! i would also like more opinions about the same so that i can proceed14:02
*** BjoernT has quit IRC14:03
sean-k-mooneythe important thing to remember is vcpu_pin_set does not and has never ment that the host was use for pinnned instances14:03
sean-k-mooneyits related but it not a direct mapping14:04
*** igordc has joined #openstack-nova14:05
*** szaher has quit IRC14:05
openstackgerritArnaud Morin proposed openstack/nova stable/stein: Refresh instance network info on deletion
*** JamesBenson has joined #openstack-nova14:08
amorinmriedem and others see ^, cherry pick for stein14:08
openstackgerritArnaud Morin proposed openstack/nova stable/rocky: Refresh instance network info on deletion
openstackgerritArnaud Morin proposed openstack/nova stable/queens: Refresh instance network info on deletion
amorinmriedem: are pike and ocata still cherry-pickable also?14:09
amorinor are they freezed?14:10
mriedemthey're open, we just don't release them anymore14:10
mriedemthey are in extended maintenance mode14:10
*** mkrai_ has quit IRC14:11
*** mkrai__ has joined #openstack-nova14:11
*** guozijn has quit IRC14:12
*** mlavalle has joined #openstack-nova14:12
*** JamesBenson has quit IRC14:12
*** xek_ has quit IRC14:13
*** markvoelker has joined #openstack-nova14:14
*** amodi has quit IRC14:16
openstackgerritStephen Finucane proposed openstack/nova master: Follow-up for I6a777b4b7a5729488f939df8c40e49bd40aec3dd
openstackgerritStephen Finucane proposed openstack/nova master: Remove 'MultiattachSupportNotYetAvailable' exception
openstackgerritStephen Finucane proposed openstack/nova master: WIP: conductor: Remove cells v1 workarounds
openstackgerritStephen Finucane proposed openstack/nova master: WIP: Start rewriting all these tests
*** szaher has joined #openstack-nova14:19
*** markvoelker has quit IRC14:19
openstackgerritStephen Finucane proposed openstack/nova stable/stein: Fix double word hacking test
*** ricolin has quit IRC14:21
stephenfinmriedem: As promised. lyarwood might want to take a look too ^14:21
lyarwoodstephenfin: ah cool, I don't think the change to disable this has landed in stein yet14:22
stephenfinnope, just left a comment on that backport of yours14:22
openstackgerritJohn Garbutt proposed openstack/nova-specs master: Add Unified Limits Spec
*** luksky has quit IRC14:29
jangutterCould I ask someone more familiar with tox and multiple python versions to check sean-k-mooney's one-liner here:
stephenfinThe 'build_instances' function is so fricking complex. It's almost impossible figure out what most of these tests are supposed to be testing :(
stephenfinjangutter: I've already hit it. It's correct14:33
* stephenfin added that feature to tox14:34
sean-k-mooneystephenfin: yep thats why i pinged you to review :)14:34
sean-k-mooneyand you know its os-vif14:34
openstackgerritStephen Finucane proposed openstack/nova master: Recalculate 'RequestSpec.numa_topology' on resize
openstackgerritStephen Finucane proposed openstack/nova master: tests: Cleanup of '_test_resize' helper test
openstackgerritStephen Finucane proposed openstack/nova master: tests: Add '_setup_compute_services' helper
jangutterstephenfin: thanks, I suspected as much, but if it summons Cthulhu, I'm pointing him to you.14:34
*** mkrai__ has quit IRC14:36
stephenfin#success Cells v1 has finally be removed from nova. Next up, nova-network14:36
openstackstatusstephenfin: Added success to Success page (
*** JamesBenson has joined #openstack-nova14:36
stephenfins/be/been/ Damn it14:36
*** jaosorior has joined #openstack-nova14:37
*** JamesBenson has quit IRC14:41
*** igordc has quit IRC14:42
*** jaosorior has quit IRC14:45
*** bhagyashris has quit IRC14:48
*** cfriesen has joined #openstack-nova14:55
*** artom has quit IRC15:00
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve E731 (do not assign a lambda expression, use a def)
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve W503 (line break occurred before a binary operator)
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve E741 (ambiguous variable name)
openstackgerritStephen Finucane proposed openstack/nova master: hacking: Resolve W605 (invalid escape sequence)
*** artom has joined #openstack-nova15:01
*** ratailor has joined #openstack-nova15:01
*** BjoernT has joined #openstack-nova15:02
sean-k-mooneybefore i forget my alternitive proposal for the trait -> aggrage mapping would be as follows. when we set tratis in nova host_aggrages we create a resouceless shareing RP in placmeent and add the traits to that15:02
mriedemmgoddard: you might want to read through first15:02
sean-k-mooneyand dont modify the host at all15:02
sean-k-mooneythen the sync is simple15:02
sean-k-mooneythat would require resouceless RPs for it to work however15:02
mgoddardmriedem: I'll add it to my reading list15:03
sean-k-mooneyefried: ^ was for you and i guess anyone that is interested15:03
openstackgerritStephen Finucane proposed openstack/nova master: docs: Remove references to nova-consoleauth
openstackgerritStephen Finucane proposed openstack/nova master: tests: Stop starting consoleauth in functional tests
openstackgerritStephen Finucane proposed openstack/nova master: xvp: Start using consoleauth tokens
openstackgerritStephen Finucane proposed openstack/nova master: nova-status: Remove consoleauth workaround check
openstackgerritStephen Finucane proposed openstack/nova master: Remove nova-consoleauth
efriedsean-k-mooney: That could work, but sounds like a pretty big hammer.15:03
openstackgerritStephen Finucane proposed openstack/nova master: objects: Remove ConsoleAuthToken.to_dict
openstackgerritStephen Finucane proposed openstack/nova master: docs: Rework nova console diagram
sean-k-mooneyefried: it might be but it makes the nova and placmenet logic a lot simpler15:03
sean-k-mooneybut it also intoduce a depency on resouceles providers15:04
*** takashin has quit IRC15:04
mriedemmgoddard: the things you'd have to be aware of is the allow_resize_to_same_host config option in the compute api - you'd have to ignore that for baremetal instances, and figuring out if the instance is baremetal to begin with to change how scheduling is going to work etc - and then it's implementing some virt driver methods15:04
sean-k-mooneymaybe something to consider after train?15:04
mriedemsean-k-mooney: or, or, we just say nova doesn't orchestrate that and if you want it, do it external to nova15:04
*** BjoernT_ has quit IRC15:04
mriedemb/c christ almighty we're already bad at orchestrating external resources,15:04
sean-k-mooneymriedem: yes which is definetly what i think we shoudl do for train15:04
mriedemcase in point, see my ML thread about deleting a compute service15:05
sean-k-mooney:) fair point15:05
sean-k-mooneywe are also terrible at math15:05
efriedmriedem, mgoddard: ironic resize: Is the consensus that someone should produce a spec (either revive the one linked above or write a new one)? But generally speaking there's no major objection to doing some form of "same host resize" for ironic?15:05
sean-k-mooneyefried: you would proably need a prefilter or somthing that could understand its a baremetal instance and force the same host15:06
efriedyes, mriedem mentioned that above15:06
sean-k-mooneythe allow same hsot resize jsut allows it rather then forces it15:06
efriedyes, mriedem mentioned that above15:06
sean-k-mooneythe question i have however is do we really need to force that15:07
*** takashin has joined #openstack-nova15:07
sean-k-mooneyi mean ironic could support resize in general15:07
efriedare you suggesting allowing ironic resize to a new node?15:07
efriedbaby steps, sean-k-mooney. Baby steps.15:07
aspierssean-k-mooney: turns out meetbot will recognise anyone issuing an #action15:08
sean-k-mooneyfor bfv guest there is nothitng ot copy and for non-bfv the ironic agent could be extended to dd the data form one to anohter15:08
aspierswhich matches my memory15:08
sean-k-mooneyaspiers: cool good to know15:08
efriedsean-k-mooney: Yes, but let's address the case we know we want to solve first, before enabling the general case and all the test surface that goes with it.15:08
sean-k-mooneyefried: sure although im pretty sure the general case had a spec form a few cylces ago which is why i brought it up15:09
artommriedem, efried, and
sean-k-mooneyefried: but yes i think that the simpler case of same host could be a stepping stone15:10
efriedthanks artom15:11
*** takashin has left #openstack-nova15:19
mriedemartom: i'll update 522084 underneath the scheduler hints change15:21
artommriedem, ah, thanks, didn't notice it was a 2-patch series15:22
*** igordc has joined #openstack-nova15:23
openstackgerritMerged openstack/os-vif master: Fix mock of built in "open" function in unit tests
openstackgerritMerged openstack/os-vif master: OVS DPDK port representors support
openstackgerritStephen Finucane proposed openstack/nova stable/queens: Fail to live migration if instance has a NUMA topology
openstackgerritStephen Finucane proposed openstack/nova stable/queens: fix up numa-topology live migration hypervisor check
stephenfinlyarwood: Could you take a look at when you have a chance?15:29
*** dpawlik has quit IRC15:35
*** gyee has joined #openstack-nova15:35
lyarwoodstephenfin: queued15:35
openstackgerritEric Fried proposed openstack/nova master: Make RequestContext(instance_lock_checked) fail
*** ttsiouts has quit IRC15:37
*** ttsiouts has joined #openstack-nova15:37
*** ivve has quit IRC15:40
*** ttsiouts has quit IRC15:42
*** sridharg has quit IRC15:42
openstackgerritMatt Riedemann proposed openstack/nova master: Api-guide: Add Block Device Mapping
mriedemartom: rebase on ^ - stephenfin - easy docs change ^15:43
artommriedem, ack15:43
*** mkrai_ has joined #openstack-nova15:47
openstackgerritBalazs Gibizer proposed openstack/nova master: allow getting resource request of every bound ports of an instance
openstackgerritBalazs Gibizer proposed openstack/nova master: Pass network API to the conducor's MigrationTask
openstackgerritBalazs Gibizer proposed openstack/nova master: Add request_spec to server move RPC calls
openstackgerritBalazs Gibizer proposed openstack/nova master: re-calculate provider mapping during migration
openstackgerritBalazs Gibizer proposed openstack/nova master: update allocation in binding profile during migrate
openstackgerritBalazs Gibizer proposed openstack/nova master: Extend NeutronFixture to handle migrations
openstackgerritBalazs Gibizer proposed openstack/nova master: prepare func test env for moving servers with bandwidth
openstackgerritBalazs Gibizer proposed openstack/nova master: func test for migrate server with ports having resource request
stephenfinmriedem: Could you or artom s/.. code::/.. code-block:: json/ in that? +W if so
mriedemstephenfin: nope15:52
mriedemyou can't b/c of the ...15:52
*** helenafm has quit IRC15:52
stephenfinI noted that. Can we drop it like we did elsewhere in the file?15:52
mriedemdrop what? the ... ?15:52
mriedemi don't really want to put a full server create request in there15:52
mriedemsince those are big15:52
stephenfinsee line 810 in there15:52
stephenfinI know, but I imagine people can guess it's not a full response15:53
stephenfinas with 810, 829, etc.15:53
*** jangutter has quit IRC15:54
mriedemffs zuul restart15:54
openstackgerritHarald Jensås proposed openstack/nova master: cell_mapping - wrap IPv6 address in format_url
openstackgerritLee Yarwood proposed openstack/nova stable/stein: libvirt: Use SATA bus for cdrom devices when using Q35 machine type
mriedemlyarwood: why did you need to rebase ^?15:57
mriedemoh i see15:57
lyarwoodmriedem: yeeeeeeah, I didn't notice the zuul restart, I wonder if that's going to get picked up.15:58
openstackgerritMatt Riedemann proposed openstack/nova master: Api-guide: Add Block Device Mapping
mriedemstephenfin: done15:58
mriedemlyarwood: yeah idk, was just rechecking a bunch of other things too15:59
lyarwoodah, so that change was picked up15:59
stephenfinmriedem: +2. Happy for you to +W or I can15:59
stephenfinthanks :)15:59
mriedemi got it16:01
*** damien_r has quit IRC16:02
*** mkrai_ has quit IRC16:02
*** mkrai_ has joined #openstack-nova16:03
mriedemlooks like all my rechecks were dropped16:03
*** ccamacho has quit IRC16:08
*** ratailor has quit IRC16:10
*** mkrai_ has quit IRC16:13
*** rpittau is now known as rpittau|afk16:13
*** markvoelker has joined #openstack-nova16:16
sean-k-mooneyartom: i noticed you updated your numa migration stuff yesterday16:19
sean-k-mooneyartom: have you more changes pending or shoudl i plan to review it/test it next week16:20
openstackgerritStephen Finucane proposed openstack/nova master: docs: Rework all things metadata'y
openstackgerritStephen Finucane proposed openstack/nova master: conf: Rename 'configuration drive' to 'config drive'
*** markvoelker has quit IRC16:20
stephenfinefried: Think I asked you this before, but what's the chances of getting blindly approved?16:21
stephenfinIt's docs only and I think it's a huge improvement, but I've no idea how to break it up more than I have without rewriting the whole darn thing16:21
mriedemstephenfin: maybe ask dansmith to review it again first :)16:22
stephenfinI think I've done that a few times too :(16:22
mriedemsince he did the most thorough review16:22
*** dtantsur is now known as dtantsur|afk16:22
dansmithstephenfin: done what a few times?16:23
stephenfindansmith: Have a look at that ridiculously big metadata doc rework patch of mine16:23
artomsean-k-mooney, just a rebase for now, bottomg patch is WIP16:23
artomTo indicate that the whole series isn't ready yet16:23
dansmithstephenfin: just wanted to make sure you weren't implying you had asked me to revisit a few times16:24
sean-k-mooneyartom: ok so saving the updated xml after the migration is still pending16:24
sean-k-mooneyartom: that was the main thing that needed to be fixed and the rest is cleanup right16:24
stephenfinI think I may have and I get why you haven't, if so. Not intending to point fingers at all though. I'd just like to be done with it16:26
artomsean-k-mooney, the XML is fine, IIRC, it's the database numa_topology that wasn't saved16:28
dansmithstephenfin: point being, I'm quite sure I didn't hear anything after that review, but I'm looking at it right now16:28
sean-k-mooneyartom: maybe i just recall we didnt save somthing16:28
stephenfindansmith: Ah, sorry. That's my mistake so16:29
stephenfinAnd thank you16:29
*** READ10 has joined #openstack-nova16:29
*** mdbooth_ has joined #openstack-nova16:31
*** mdbooth has quit IRC16:34
dansmithmriedem: for the tenth and last time, can you hit this?
sean-k-mooneystephenfin:  i personlay prefer the ... then assuming people will guess its not a full respocne but its proably fine eitehr way16:36
stephenfinsean-k-mooney: Too late. mriedem already did the deed16:36
sean-k-mooneystephenfin: that was regarding mriedem change but he has updated it already16:37
sean-k-mooneyyep i know16:37
stephenfinNo take backsies16:37
mriedemdansmith: lucky for y'all i just got a fresh pack of red markers16:37
sean-k-mooneystephenfin: damn i have not heard that in a while16:37
*** mdbooth_ has quit IRC16:38
*** gibi has quit IRC16:41
efriedstephenfin: looking now.16:43
aspiersefried: oslo.log guidelines feedback addressed, when you are looking for some light relief16:46
*** cdent has joined #openstack-nova16:56
*** JamesBenson has joined #openstack-nova16:57
*** JamesBenson has quit IRC17:00
*** cdent has quit IRC17:01
*** derekh has quit IRC17:03
*** mdbooth has joined #openstack-nova17:04
*** davidsha has quit IRC17:05
*** mdbooth has quit IRC17:07
*** mdbooth has joined #openstack-nova17:07
*** ohwhyosa has quit IRC17:08
*** pcaruana has joined #openstack-nova17:09
*** nicolasbock has quit IRC17:10
*** cdent has joined #openstack-nova17:11
*** ociuhandu has quit IRC17:15
openstackgerritmelanie witt proposed openstack/nova master: Warn for duplicate host mappings during discover_hosts
melwittdansmith: ^ updated17:19
dansmithyou're supposed to wait a month between all updates17:23
*** panda has quit IRC17:24
efriedstephenfin: I'm going to wait until that sucker renders so I can at least feel I've done due diligence on the links and stuff.17:24
*** ralonsoh has quit IRC17:26
*** panda has joined #openstack-nova17:27
*** whoami-rajat has joined #openstack-nova17:38
*** JamesBenson has joined #openstack-nova17:41
*** hongbin has joined #openstack-nova17:43
*** JamesBenson has quit IRC17:45
*** JamesBenson has joined #openstack-nova17:50
artommriedem, so can be abandoned17:50
*** damien_r has joined #openstack-nova17:52
mriedemartom: not really17:53
mriedem isn't really describing scheduler hints in detail,17:53
mriedemit's just describing scheduler hints vs flavor extra specs17:53
mriedemwhich is more of an admin/operator thing than end user17:53
artommriedem, are we reading the same first paragraph of
mriedemi agree the in-tree hints are described now in the api-ref17:53
mriedemartom: i wrote
*** JamesBenson has quit IRC17:54
artommriedem, I know - it's almost word for word as what's in, no?17:54
artomOK, maybe not word for word17:55
mriedemi think the api-guide would benefit from (1) a short description of what are scheduler hints, (2) an example api request, and then (3) links to the thing above (hints vs extra specs) and the server create api reference for the description of the available in-tree hints17:55
mriedemour api-ref doesn't have a sample with a scheduler hint in17:56
*** ivve has joined #openstack-nova17:56
mriedemso every time i write a test that uses one, i have to look this up17:56
mriedemb/c 1. there are 2 param names accepted for hints and 2. it's at the top level of the request rather than within the server object17:56
openstackgerritMerged openstack/os-vif master: set ignore_basepython_conflict = True in tox.ini
artomWell, I signed up for this, might as well finish it17:57
artommriedem, same patch? Smells like a new change17:57
artomCan I achieve peak arrogance by trading you reviews on
artomI guess same change to preserve review history17:59
mriedemi would do the same patch for review history yes18:01
mriedemadd yourself as co-author18:01
mriedemand because you asked for yet again i'm going to say i'll never review it again now18:01
mriedemyou're welcome18:01
*** JamesBenson has joined #openstack-nova18:02
openstackgerritsean mooney proposed openstack/os-vif stable/stein: set ignore_basepython_conflict = True in tox.ini
*** gyee has quit IRC18:04
openstackgerritsean mooney proposed openstack/os-vif stable/stein: Fix mock of built in "open" function in unit tests
mriedemartom: before you raise HR alarms downstream, i was joking18:08
melwittmriedem: this is a bit old and has a +2 from dansmith now, you're probably the best person to review since it's about nova-manage cells stuff
artommriedem, you haven't touched me inappropriately (yet?), so no HR alarms18:08
mriedemdansmith: you know how we have this warning about stopping nova-compute before deleting it? otherwise dragons? it strikes me now that the delete compute service API should just be checking if the service is up and if so, fail the request with a 40918:09
sean-k-mooneyartom: a +1 from zuul, how did you apise the ci gods to make that happen18:10
dansmithmriedem: maybe, unless there's a reason to delete and have it recreate or something18:10
artomsean-k-mooney, which patch?18:10
dansmithmriedem: the other thing would be that unless you also force down, you can't stop and delete inside of a minute or whatever the service timeout is18:10
sean-k-mooneyartom: you have more then one with a +1 :P  the hybrid_plug events one18:11
mriedemyeah you'd have to force down18:11
mriedemdansmith: the problem with the recreate is the new provider it will create in placement with no allocations for the instances on that host, which will f up scheduling18:11
artomsean-k-mooney, except for the double words thing (that was fixed) I'm not aware of any Zuul issues18:11
mriedemyou can heal that with the heal_allocations CLI of course...18:11
mriedemwell, for non-nested / sharing allocations18:12
*** cdent has quit IRC18:12
dansmithmriedem: sure18:12
sean-k-mooneyartom: it was more the fact its taken a few hours for zuul to notice and run on your patch for the last few days then it being flaky18:12
artomsean-k-mooney, oh, yeah, it's slow18:13
mriedemi'm just down a hole on how we're orphaning providers today and how best to shore that up18:13
sean-k-mooneyim following the thread on the ml bug trying to stay out of it to a degree to leave ops speak up18:16
sean-k-mooneyalthough it looks like the last few mails are back and forth between you erric and chris18:16
*** markvoelker has joined #openstack-nova18:17
*** gyee has joined #openstack-nova18:19
*** markvoelker has quit IRC18:22
mriedemi think likely the best thing to do is a phased conservative approach,18:24
mriedem1. land my change that fixes it to delete all providers, not just the first (the ironic case),18:24
mriedem2. if deleting any provider fails, don't delete the compute service and bail (option 1 in the email),18:24
mriedem3. check for and cleanup allocations for evacuations - something we can be pretty sure we can get rid of safely18:25
mriedemthat leaves in-progress or unconfirmed migrations, and in that case i think we likely just should be saying "you can't delete the compute service until these migrations are done"18:25
sean-k-mooneyya that seam sane. the issue that im not sure how to solve/if we shoul solv eis if you had in progeress migration and the host failed/went offline18:26
sean-k-mooneycan you evaucate an instance that was migrating18:27
sean-k-mooneywhen the host went off line18:27
bnemecaspiers: +2. Thanks for taking care of this!18:27
aspiersbnemec: yw18:28
mriedemsean-k-mooney: don't think so18:28
mriedemunless you reset the instance state to ERROR18:28
sean-k-mooneyim fine to leave that up to the operator to clean up too, but im just wondering about the case where the reason their are pending migration or evacuation is because of a host failure18:28
sean-k-mooneyor reset it ot active18:28
sean-k-mooneybut ok18:28
sean-k-mooneyif it was in resize_verify or whatever the state is when we havent confirmed yet but was successafaly migrated we would proably want to reset it to active18:29
mriedemif you're in the middle of a migration and the source host went offline, i don't see why you'd delete the service - just bring the service back up if possible18:30
sean-k-mooneyya im somewhat unclear why the compute service is being deleted with outstanding migrations too18:31
sean-k-mooneyalso reset state to active instead of confirming might not be right as it likely would not have updated the db correctly for the new host18:32
sean-k-mooneyso you would want to confirm. let it error out and then rest to active/hardreboot to fix it right18:32
mriedemwell the good news is a contracted google recruiter has contacted me about opportunities in the bay area of new york so if i want to avoid this problem i can just follow that lead and up-root my entire life18:33
mriedemdelete compute service problems be damned!18:34
sean-k-mooneyhaha well that is one way to solve it yes then you can work on kubernetes problems18:34
sean-k-mooneyits really a seperate issue but should we allow you to confim a resize/migration if the souce host is down without erroring out?18:35
mriedemthen after 6 months you get there and they're like, "ok we have this resource leak issue when you delete a pod"18:35
mriedemsean-k-mooney: i'm not sure what happens in that case, i haven't tried it, but i'd think the cast to the source compute would blow up and you'd get a 500 response from the confirmResize API18:36
sean-k-mooneywell if its a cast it a fire and forget if its a call then yes18:37
sean-k-mooneydo we start the confrim form the souce node so the api will jsut time out the request18:37
sean-k-mooneye.g. because notheing handels it.18:37
melwittefried: do you want a follow up patch for your comments on
efriedmelwitt: meh, they're so minor.18:38
efriedmelwitt: If you want to spin a new PS I can fast approve18:38
sean-k-mooneyi need to get a better undestand of how to create nova functional test to test this kind fo stuff without actully depolying devstack18:38
efriedor we can just ignore18:38
efriedmriedem: Was about to start composing the email whining about bug queue. Unless you want to do it... <bats eyes>18:40
melwittefried: ok. I can respin, that's easy enough. mostly I'm thinking of the doc rendering where literal would help18:40
efriedmelwitt: There's probably opportunity to add literals throughout that doc - I didn't check. If so, maybe we could scoop it all up in a fup18:40
efriedanyway, I'm around for at least another 3-4h to push whatever18:41
melwittok. yeah, I see most seem literal'd but some others aren't so could be done all together18:42
mriedemefried: i do not, writing a reply to this compute service delete thread18:43
efriedmriedem: ack. Okay if I run it by you before sending?18:44
sean-k-mooneymriedem: i might steal your test code form and try and create a similar test for that migrate confim case while im waiting for the pizza to arrive18:47
dansmithmmm pizza18:48
* dansmith waits for it18:48
openstackgerritArtom Lifshitz proposed openstack/nova master: api-guide: better explain scheduler hints.
sean-k-mooneyhehe one of irelands suckiest inventions is slow "fast food"18:48
sean-k-mooneyall the quality of fast food with the slowness of going to the store and cooking it your self.18:49
sean-k-mooneygiven i havent ordered it yet i might actully do that18:49
melwittdid somebody say pizza18:51
mriedemdansmith: i was just thinking about that yesterday for some reason, probably b/c i made a pizza for lunch18:52
*** dpawlik has joined #openstack-nova18:54
*** hoonetorg has quit IRC18:55
*** hoonetorg has joined #openstack-nova19:00
dansmithmriedem: I don't even remember what the gag was.. trying to explain to sdague how to use punctuation or something19:03
openstackgerritmelanie witt proposed openstack/nova master: Literalize CLI options in docs
mriedemi think it was related to commit message titles19:05
mriedemsomething like that19:05
mriedemSomething like that.19:05
mriedemone of the annual flare ups in the ML19:05
mriedemthe good old days19:05
*** bnemec has quit IRC19:06
mriedemi think we were shitting on his commit message which was all lower case with sparse amount of periods19:06
dansmithsounds right19:06
mriedemmostly to poke the bear and air grievances19:06
mriedemit wasn't even festivus19:06
*** damien_r has quit IRC19:06
mriedemand now look at him, in front of congress and all19:07
mriedemwe really showed him19:07
dansmithbecause congress has such high standads19:07
mriedemlike garth brooks, sean has friends in low places19:08
melwittremember chris gaines19:09
mriedemhow could i forget19:10
openstackgerritMatt Riedemann proposed openstack/nova master: Delete resource providers for all nodes when deleting compute service
openstackgerritMatt Riedemann proposed openstack/nova master: Avoid unnecessary joins in delete_resource_provider
openstackgerritMatt Riedemann proposed openstack/nova master: Optimize SchedulerReportClient.delete_resource_provider
dansmithman.. bad hair day19:12
melwittthe intro made it look like the panpipe sound was coming from the tugboat lol19:12
mriedemmaya got her first exposure to that in the car last night, with me singing along to it19:12
melwittperuvian panpipes19:12
mriedemand laura telling me to shut up19:13
melwittdid you dance too? I remember you said that's a favorite for car trips with the family19:13
mriedemi did the shoulder shake sure19:14
mriedemit cannot be stopped19:14
melwittyeah,  that's what I meant. shoulder shake was the thing they are a fan of19:16
*** markvoelker has joined #openstack-nova19:18
efriednot sure what else to say19:20
mriedemefried: looks fine to me19:21
efriedight, thx19:21
*** markvoelker has quit IRC19:23
*** maciejjozefczyk has quit IRC19:25
*** mdbooth has quit IRC19:27
mriedemlyarwood: efried: did anyone talk about case here?
*** takamatsu has quit IRC19:32
efriedmriedem: doesn't look like it's ever uppercase, but I really don't know.19:34
*** mdbooth has joined #openstack-nova19:34
mriedemyeah the image meta props don't define an enum19:35
mriedemlooks like libvirt / qemu uses q35 though
openstackgerritMerged openstack/nova stable/rocky: Move get_pci_mapping_for_migration to MigrationContext
openstackgerritMerged openstack/nova stable/rocky: Allow driver to properly unplug VIFs on destination on confirm resize
openstackgerritMerged openstack/nova stable/rocky: Fixes multi-registry config in Quobyte driver
*** maciejjozefczyk has joined #openstack-nova19:38
openstackgerritMerged openstack/nova stable/rocky: tox: Use basepython of python3.5
openstackgerritMerged openstack/nova stable/rocky: docs: Don't version links to reno docs
openstackgerritMerged openstack/nova stable/rocky: Include all network devices in nova diagnostics
*** maciejjozefczyk_ has joined #openstack-nova19:41
*** maciejjozefczyk has quit IRC19:45
*** mdbooth has quit IRC19:45
*** maciejjozefczyk has joined #openstack-nova19:46
*** maciejjozefczyk_ has quit IRC19:47
*** whoami-rajat has quit IRC19:48
*** xek has joined #openstack-nova19:50
*** bbowen has quit IRC19:56
openstackgerritMerged openstack/nova master: Api-guide: Add Block Device Mapping
*** eharney has quit IRC20:01
*** spatel has joined #openstack-nova20:07
spatelsean-k-mooney: Question20:07
spatelbuilding new cloud so should i go with linuxbridge or OVS ( any performance difference here? other than features)20:08
mriedemefried: fwiw i'm going through stephen's metadata docs change now20:15
efriedmriedem: cool, thanks for that.20:15
efriedmriedem: It still hasn't rendered, right?20:15
efriedI guess I could pull it down and build it myself...20:15
mriedemthat's what i did20:15
mriedemthe docs are failing to build locally for me20:17
*** markvoelker has joined #openstack-nova20:19
*** markvoelker has quit IRC20:24
*** artom has quit IRC20:28
*** READ10 has quit IRC20:31
mriedemi see why now20:32
*** pcaruana has quit IRC20:36
*** panda has quit IRC20:49
*** panda has joined #openstack-nova20:51
*** bnemec has joined #openstack-nova20:52
mriedemgdi i've only made it 10 of 22 files in this change20:52
*** xek has quit IRC20:57
*** dpawlik has quit IRC21:01
sean-k-mooneyspatel: linuxbirdge is not that well tested i personally am much more comfortable with ovs however some large public cloud have use linux birdge like rackspace21:02
sean-k-mooneyspatel: in generall linuxbrige will have fewer fancy networking feature and tends not to scale beyond 10G21:03
sean-k-mooneybut its also simpler21:03
sean-k-mooneyso depending on your netwroking backgorund and the team that will be maintaining the clould you could be trading performace for lower operationgs costs21:04
spatelsean-k-mooney: thank you, i am already using LinuxBridge in my cloud but yes its very simple, first time i am going to play with OVS so not sure what and where i am going to face complexity21:04
*** JamesBenson has quit IRC21:05
spatelI may planning to use DPDK so i have to learn OVS21:05
*** oomichi_ has joined #openstack-nova21:05
sean-k-mooneyif your starting out with ovs its simpler to start with kernel ovs and learn how to use ovs-dpdk after you are comfortable with openflow and ovs in general21:06
*** BjoernT has quit IRC21:06
sean-k-mooneyit just helps not to add dpdk to the learning curve21:06
sean-k-mooneyovs is the default netrok backend for like 80% of all deployments in one form or another21:06
*** kaisers1 has joined #openstack-nova21:07
spatelsean-k-mooney: hmm21:07
*** mchlumsky has quit IRC21:07
spatelIs it possible i deploy ovs with general configuration and then slowly rollout ovs-dpdk compute by compute21:08
*** kaisers has quit IRC21:09
spatelCan i have mix environment like currently i have SR-IOV with general compute21:09
sean-k-mooneyspatel: yes you can mix ovs and ovs-dpdk in the same cloud and you can also have sr-iov in that cloud21:10
sean-k-mooneygenerally you will use a different host aggage to seperate your dpdk nodes that need hugepages form the rest that dont21:11
spatelI think i should go with OVS so i can have all kind of function and feature available..21:11
spatelwhat are the basic requirement for dpdk?21:12
mriedemefried: please don't fix the docs build break21:12
mriedemi just dumped 42 comments on that change21:12
efriedmriedem: okay21:12
spatelNIC support, hugepages etc..21:12
sean-k-mooneymost nicst that support sriov support dpdk but there is a list here21:13
*** takamatsu has joined #openstack-nova21:13
sean-k-mooneythat is more a list of driver but you likely have a nic that is supported21:13
sean-k-mooneyhugepages are needed for the guest and for the vswtich21:14
sean-k-mooneyother then that there are no special requiremetns21:14
*** hongbin has quit IRC21:14
mriedemwith that i think i'm done for the day21:14
spatelI have - 04:00.0 Ethernet controller: Intel Corporation 82599 10 Gigabit Dual Port Backplane Connection (rev 01)21:14
spatelHow much hugepage required for vswitch?21:15
sean-k-mooneyspatel: yep they use the ixgbe dirver and are supported21:15
spatelMy all compute nodes are 32G memory ( some 64G)21:15
sean-k-mooneydpending on your workload 2-4 G if you are using jumbo frams 1-2G is fine for standard 1500b packets21:16
sean-k-mooneyi have tunned it to less in the past21:16
sean-k-mooneyit jsut depends on your data rates and how long your queuse are going to be21:16
*** luksky has joined #openstack-nova21:17
spatelWhen you saying hugepage for vswitch does that means i have to give some dedicated hugepage to vSwitch right?21:17
sean-k-mooneyspatel: yes21:17
spatelNice doc..21:18
sean-k-mooneyovs-dpdk uses hugepage memroy to process packets so that it can DMA transfer packets directly to and from the nic bypassing the kernel21:18
spatelsean-k-mooney: do i also need to give some dedicated CPU to vSwitch?21:18
*** mriedem has quit IRC21:19
sean-k-mooneyyes ideally 1 PMD(poll mode driver) core per numa node21:19
*** markvoelker has joined #openstack-nova21:20
sean-k-mooneyovs-dpdk scalse effectivly liniarly with the number of cpu cores you give it so you can tune it up or down depending on your data rates21:20
spatelI have two numa zone so i have to give 2 CPU core right?  is that CPU core will be pin with vSwitch?21:20
sean-k-mooneyyes dpdk will run in a bussy loop waiting for packets and will consume the full core21:21
sean-k-mooneyeven when idel21:21
*** JamesBenson has joined #openstack-nova21:22
sean-k-mooneyit does that to miniumese latency but it basically means you cant run other thing on the core at the same time21:22
sean-k-mooneyper packet ovs-dpdk is much more efficent then kernel ovs however kernel ovs use kernel thread to process the packet and does not consume cpus cycles when idel21:23
sean-k-mooneyso its a trade off21:23
sean-k-mooneyovs-dpdk cpu usage will be fixed based on the cores you allcoate it21:23
spatelcore allocation process is automatic or part of configuration?21:24
sean-k-mooneykernel ovs will use less or more cpus as needed includeing stealing cpu cycles form the guests if you do not tune the kernel to prevent that21:24
sean-k-mooneyspatel: part of the configuration21:24
spatelah! ok..21:24
*** markvoelker has quit IRC21:24
sean-k-mooneyyou set a bit mask in the ovs db to choose what core dpdk is allows to use21:24
spatelDo i need to create neutron port for dpdk (like i am create currently for SR-IOV)?21:25
sean-k-mooneyyou can but no. it used vnic_type=normal which is the default21:25
sean-k-mooneyso you dont need to precreate them21:25
sean-k-mooneyform a user point of view its identiacl to linux bridge or kernel ovs21:25
sean-k-mooneyother then the fact you need a flaovr with hugepages21:26
spatelPerfect!! i love that part, currently in SR-IOV i have to create port in advance to map with vms21:26
sean-k-mooneywhich is annoying but there are reason why we require that21:26
spatelanyway my all vms running with hugepages at moment21:26
spatelQuestion why does vms required hugepage for dpdk?21:27
sean-k-mooneyit has to do with how the packeets are transfered between the vswitch and the vm21:27
sean-k-mooneywe use a technology called vhost-user21:27
sean-k-mooneywhich allows the vswitch to mapp the virtio-rings( tx and rx queues) of the guest nic into the memory of the vswitch21:28
sean-k-mooneyto do that we need to use gest memroy that is mmap shared  with an open file discripto, is contiguaon and preallcoated21:29
sean-k-mooneyhugepages do all of the above21:29
sean-k-mooneytechnically you can acive the same wiht out hugepages21:29
sean-k-mooneyits just slower, more work and not supported by openstack by defualt21:29
spatelGot it.. so hugepage has big role here21:30
sean-k-mooneyif you are using a new enought ovs-dpdk you can actully get 0 copy packet transmistion betwwen the guest and the nic21:30
openstackgerritMerged openstack/nova master: update constraints url
sean-k-mooneye.g. when the guest kernel rights the packet to the tx queue it is nver copyied and is drieectly trasmited by the nic similar to sriov21:31
sean-k-mooney0 copy recive i think is also posibel but experimental21:31
sean-k-mooneythat obviorsly depend on if you are using vxlan or vlans feature that may or may not be supported in your nic hardware21:32
spatelah! ok.. in short it bypass the kernel21:32
sean-k-mooneyyep and does everything in userspacce or in hardaware21:32
spatelDo you guys running dpdk in production?21:33
spatelvery few people i found using dpdk in prod cloud21:33
sean-k-mooneywell i dont run a production cloud. i just work on upstream stuff21:33
sean-k-mooneyspatel: but redhat has customer using it in production yest21:33
sean-k-mooneyit is much less common then kernel ovs or sriov21:34
spatelLet see how successful i can run in my production.21:34
sean-k-mooneyif you want the full intel pitch then
spatelThanks for that link..21:35
sean-k-mooneyhehe when i was working at intel i help some of the yahoo folk tuen there deployment alittle but after it was working i never heard form them how it worked out long term21:36
spatelsean-k-mooney: you are smarty pants :)21:37
spatelsean-k-mooney: hope dpdk support bonding too21:39
sean-k-mooneyit does21:39
sean-k-mooneyyou can have dpdk do it(not recommended) or use ovs bonding of dpdk ports21:39
sean-k-mooneyovs bonding give you more contol and can be modifed while the switch is running21:40
sean-k-mooneydpdk bonding required the switch to be restareted, is less configurable but used to be slight faster at failover and throuput21:40
sean-k-mooneyhonestly however dpdk bonding not via ovs is an operations pain that is best avoid21:41
sean-k-mooneyyou should just use ovs bonding21:41
sean-k-mooneyanyway im gong to call it a night o/21:41
*** slaweq has quit IRC21:43
*** luksky has quit IRC21:43
spatelsean-k-mooney: Thank you for wonderful tips21:43
spatelGood night21:43
*** slaweq has joined #openstack-nova21:44
*** mdbooth has joined #openstack-nova21:46
*** slaweq has quit IRC21:49
*** spatel has quit IRC22:03
*** bnemec has quit IRC22:05
*** mdbooth has quit IRC22:10
*** JamesBenson has quit IRC22:13
*** takamatsu has quit IRC22:25
openstackgerritEric Fried proposed openstack/nova master: Functional recreate for revert_resize
*** dave-mccowan has quit IRC22:26
*** slaweq has joined #openstack-nova22:36
*** oomichi_ has quit IRC22:40
*** bbowen has joined #openstack-nova22:40
*** oomichi_ has joined #openstack-nova22:40
*** slaweq has quit IRC22:41
*** rcernin has joined #openstack-nova22:45
openstackgerritMerged openstack/nova master: update comment on ignore_basepython_conflict
*** mlavalle has quit IRC22:47
*** tesseract has quit IRC22:48
*** tkajinam has joined #openstack-nova22:51
*** ivve has quit IRC22:57
*** _erlon_ has quit IRC23:12
*** slaweq has joined #openstack-nova23:19
*** markvoelker has joined #openstack-nova23:22
*** markvoelker has quit IRC23:26
*** dave-mccowan has joined #openstack-nova23:40
*** artom has joined #openstack-nova23:40
*** JamesBenson has joined #openstack-nova23:47
*** JamesBenson has quit IRC23:51
*** slaweq has quit IRC23:56

Generated by 2.15.3 by Marius Gedminas - find it at!