Friday, 2019-03-08

openstackgerritTakashi NATSUME proposed openstack/nova master: Add description about sort order in API ref guideline  https://review.openstack.org/62728200:00
*** takashin has joined #openstack-nova00:01
*** wolverineav has quit IRC00:02
dansmithefried: I meant os-brick is under cinder, afaik.. os-vif is under nova00:02
*** wolverineav has joined #openstack-nova00:02
efrieddansmith: Yes, confirmed. I was confused about that aspect. Anyway, the release is proposed and approved, merging. You may go about your business.00:04
efried(why is os-vif under nova and not neutron if os-brick is under cinder and not nova?)00:04
*** sapd1 has quit IRC00:16
jungleboyjefried:  I will request a requirements freeze exception here.00:18
*** IvensZambrano has quit IRC00:19
*** snevi has joined #openstack-nova00:19
dansmithefried: neutron doesn't use os-vif00:23
dansmithor hasn't previously, even though that was the evntual goal00:24
*** hongbin has quit IRC00:24
dansmiththey might use some of it now, I forget, but it's not central to their stuff like it is to ours (and os-brick is to cinder)00:24
*** snevi has quit IRC00:26
*** owalsh has joined #openstack-nova00:27
*** owalsh_ has quit IRC00:27
*** ttsiouts has quit IRC00:28
*** ttsiouts has joined #openstack-nova00:29
*** ttsiouts has quit IRC00:33
openstackgerritTakashi NATSUME proposed openstack/nova master: doc: mark the max microversion for stein  https://review.openstack.org/64188200:36
*** eharney has quit IRC00:37
*** wolverineav has quit IRC00:49
*** markvoelker has joined #openstack-nova00:50
*** sapd1 has joined #openstack-nova00:50
*** mriedem_away is now known as mriedem00:54
mriedemefried: os-brick is cinder's thing00:57
mriedemand a non-client lib so it should have been released last week00:57
mriedemnvm i see you've sorted it out00:58
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix cleaning up console tokens  https://review.openstack.org/63771601:01
*** wolverineav has joined #openstack-nova01:03
*** mriedem has quit IRC01:04
*** artom has joined #openstack-nova01:11
*** gyee has quit IRC01:15
*** igordc has quit IRC01:19
*** whoami-rajat has joined #openstack-nova01:23
*** markvoelker has quit IRC01:23
openstackgerritGhanshyam Mann proposed openstack/nova master: DNM: Testing nova legacy jobs on bionic  https://review.openstack.org/63901701:23
*** imacdonn has quit IRC01:27
openstackgerritMerged openstack/nova master: Remove cells v1 (for the most part) from the docs  https://review.openstack.org/63671801:29
*** Sundar has quit IRC01:38
openstackgerritBoxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible  https://review.openstack.org/64027101:39
*** wolverineav has quit IRC01:39
*** wolverineav has joined #openstack-nova01:40
*** wolverineav has quit IRC01:44
openstackgerritxulei proposed openstack/nova master: Check compute_node existence in when nova-compute reports info to placement  https://review.openstack.org/64189901:50
openstackgerritxulei proposed openstack/nova master: Check compute_node existence in when nova-compute reports info to placement  https://review.openstack.org/64189901:52
*** igordc has joined #openstack-nova02:04
*** tbachman has quit IRC02:18
*** markvoelker has joined #openstack-nova02:20
*** sunnaichuan has joined #openstack-nova02:27
*** mlavalle has quit IRC02:28
openstackgerritBoxiang Zhu proposed openstack/nova master: Raise error when force_raw_images and images_type are incompatible  https://review.openstack.org/64027102:37
*** hongbin has joined #openstack-nova02:42
*** jhesketh has quit IRC02:45
*** tbachman has joined #openstack-nova02:47
*** jhesketh has joined #openstack-nova02:47
*** markvoelker has quit IRC02:53
*** tbachman has quit IRC02:56
openstackgerritMerged openstack/nova master: Add user_id column to the instance_mappings table  https://review.openstack.org/63334903:06
openstackgerritMerged openstack/nova master: doc: mark the max microversion for stein  https://review.openstack.org/64188203:06
openstackgerritZhenyu Zheng proposed openstack/nova master: Should not skip volume_size check for bdm.image_id == image_ref case  https://review.openstack.org/64128203:11
openstackgerritBoxiang Zhu proposed openstack/nova master: Force to schedule even has force_hosts or force_nodes  https://review.openstack.org/64190803:15
*** ianw_pto has quit IRC03:23
*** ianw has joined #openstack-nova03:23
*** rmk has quit IRC03:25
*** mgariepy has quit IRC03:25
*** mgariepy has joined #openstack-nova03:26
*** yaawang has quit IRC03:31
*** lbragstad has quit IRC03:32
*** zhubx007 has quit IRC03:35
*** zhubx007 has joined #openstack-nova03:35
*** yaawang has joined #openstack-nova03:36
*** markvoelker has joined #openstack-nova03:50
*** BjoernT has joined #openstack-nova03:58
*** mdbooth_ has joined #openstack-nova04:22
*** markvoelker has quit IRC04:23
*** udesale has joined #openstack-nova04:24
*** mdbooth has quit IRC04:25
*** abhishekk has joined #openstack-nova04:30
*** whoami-rajat has quit IRC04:33
*** mdbooth has joined #openstack-nova04:35
*** mdbooth_ has quit IRC04:37
*** whoami-rajat has joined #openstack-nova04:53
*** lbragstad has joined #openstack-nova05:15
openstackgerritMichael Still proposed openstack/nova master: Move slight bonkers IP management to privsep.  https://review.openstack.org/62459205:17
openstackgerritMichael Still proposed openstack/nova master: Privsepify ipv4 forwarding enablement.  https://review.openstack.org/63543105:17
openstackgerritMichael Still proposed openstack/nova master: Remove unused FP device creation and deletion methods.  https://review.openstack.org/63543305:17
openstackgerritMichael Still proposed openstack/nova master: Privsep the ebtables modification code.  https://review.openstack.org/63543505:17
openstackgerritMichael Still proposed openstack/nova master: Move adding vlans to interfaces to privsep.  https://review.openstack.org/63543605:17
openstackgerritMichael Still proposed openstack/nova master: Move iptables rule fetching and setting to privsep.  https://review.openstack.org/63650805:17
openstackgerritMichael Still proposed openstack/nova master: Move dnsmasq restarts to privsep.  https://review.openstack.org/63928005:17
openstackgerritMichael Still proposed openstack/nova master: Move router advertisement daemon restarts to privsep.  https://review.openstack.org/63928105:17
openstackgerritMichael Still proposed openstack/nova master: Move calls to ovs-vsctl to privsep.  https://review.openstack.org/63928205:17
openstackgerritMichael Still proposed openstack/nova master: Move setting of device trust to privsep.  https://review.openstack.org/63928305:17
openstackgerritMichael Still proposed openstack/nova master: Move final bridge commands to privsep.  https://review.openstack.org/63958005:17
openstackgerritMichael Still proposed openstack/nova master: Cleanup the _execute shim in nova/network.  https://review.openstack.org/63958105:17
openstackgerritMichael Still proposed openstack/nova master: We no longer need rootwrap.  https://review.openstack.org/55443805:17
openstackgerritMichael Still proposed openstack/nova master: Cleanup no longer required filters and add a release note.  https://review.openstack.org/63982605:17
*** markvoelker has joined #openstack-nova05:20
*** Dinesh_Bhor has quit IRC05:29
*** takashin has left #openstack-nova05:30
*** Dinesh_Bhor has joined #openstack-nova05:33
*** janki has joined #openstack-nova05:34
*** tkajinam has quit IRC05:38
*** tkajinam has joined #openstack-nova05:39
*** hongbin has quit IRC05:42
*** sapd1 has quit IRC05:42
*** sapd1 has joined #openstack-nova05:52
*** markvoelker has quit IRC05:53
*** mikal has quit IRC05:55
*** mikal has joined #openstack-nova05:55
*** zhubx007 has quit IRC05:57
*** zhubx007 has joined #openstack-nova05:58
*** zhubx007 has quit IRC06:08
*** zhubx007 has joined #openstack-nova06:08
*** ratailor has joined #openstack-nova06:09
*** zhubx007 has quit IRC06:10
*** zhubx007 has joined #openstack-nova06:10
*** marst has joined #openstack-nova06:12
openstackgerritZhenyu Zheng proposed openstack/nova master: Should not skip volume_size check for bdm.image_id == image_ref case  https://review.openstack.org/64128206:13
openstackgerritChris Friesen proposed openstack/nova master: Split up _is_storage_shared_with() in libvirt driver  https://review.openstack.org/63993306:33
openstackgerritChris Friesen proposed openstack/nova master: Add emulated TPM support to Nova  https://review.openstack.org/63136306:33
cfriesenthese are for testing, pls ignore: ^06:34
openstackgerritChris Friesen proposed openstack/nova master: Add emulated TPM support to Nova  https://review.openstack.org/63136306:36
openstackgerritChris Friesen proposed openstack/nova master: Add emulated TPM support to Nova -- testcase test only  https://review.openstack.org/64193206:37
openstackgerritChris Friesen proposed openstack/nova master: Split up _is_storage_shared_with() in libvirt driver  https://review.openstack.org/63993306:44
openstackgerritChris Friesen proposed openstack/nova master: Add emulated TPM support to Nova  https://review.openstack.org/63136306:44
openstackgerritChris Friesen proposed openstack/nova master: Add support for resize and cold migration of emulated TPM files  https://review.openstack.org/63993406:44
*** sapd1 has quit IRC06:44
*** ivve has joined #openstack-nova06:49
*** markvoelker has joined #openstack-nova06:51
*** Bhujay has joined #openstack-nova06:52
*** Luzi has joined #openstack-nova06:53
*** sridharg has joined #openstack-nova07:06
*** wolverineav has joined #openstack-nova07:09
*** luksky has joined #openstack-nova07:17
*** markvoelker has quit IRC07:23
*** pcaruana has joined #openstack-nova07:26
*** lbragstad has quit IRC07:28
*** phasespace has quit IRC07:44
*** Bhujay has quit IRC07:50
*** Bhujay has joined #openstack-nova07:57
*** BjoernT has quit IRC07:57
*** tkajinam has quit IRC08:00
*** awalende has joined #openstack-nova08:03
*** wolverineav has quit IRC08:04
*** igordc has quit IRC08:04
*** wolverineav has joined #openstack-nova08:05
*** rpittau|afk is now known as rpittau08:06
*** tesseract has joined #openstack-nova08:10
*** markvoelker has joined #openstack-nova08:20
*** jlejeune_ has joined #openstack-nova08:22
*** jlejeune_ is now known as jlejeune08:22
*** mikal has quit IRC08:26
*** xek_ has joined #openstack-nova08:26
*** xek_ has quit IRC08:27
*** tssurya has joined #openstack-nova08:28
*** xek has joined #openstack-nova08:28
*** Bhujay has quit IRC08:31
*** ttsiouts has joined #openstack-nova08:32
*** ttsiouts has quit IRC08:37
*** phasespace has joined #openstack-nova08:42
*** ttsiouts has joined #openstack-nova08:42
*** liuyulong has joined #openstack-nova08:45
*** wolverineav has quit IRC08:46
*** marst has quit IRC08:51
*** markvoelker has quit IRC08:53
*** mcgiggler has joined #openstack-nova09:03
openstackgerritBalazs Gibizer proposed openstack/nova master: Use Selection object to fill request group mapping  https://review.openstack.org/63871109:11
*** pcaruana has quit IRC09:12
*** ccamacho has joined #openstack-nova09:16
*** pcaruana has joined #openstack-nova09:27
*** ralonsoh has joined #openstack-nova09:35
openstackgerritBalazs Gibizer proposed openstack/nova master: Use Selection object to fill request group mapping  https://review.openstack.org/63871109:43
aspierskashyap: I'm intrigued in which ways is the term "domain" Xen-specific?09:45
aspiersAFAICS libvirt uses it everywhere09:45
kashyapaspiers: libvirt's terminology of "domain" for a "guest" is a hold-over from Xen times.09:45
kashyapaspiers: Can you explain what is a "host domain"? :-)09:46
kashyapI mean, not being flippant.  Just trying to put myself in an operator's shoes reading the log09:46
aspiersYour point being that KVM doesn't have a dom0?09:46
kashyapYep09:46
kashyap"host hypervisor" is more generic09:46
kashyapNot sure you agree :-)09:47
aspiersNo, it's a good point that operators read logs and are less likely to understand arcane libvirt / Xen terminology09:47
aspiersI do agree09:47
*** rcernin has quit IRC09:47
aspiersI was trying to adhere to libvirt terminology since that's what the code aligns with, but that's not as good for operators so I'll change it.09:48
*** IvensZambrano has joined #openstack-nova09:49
kashyapaspiers: Nod, I did notice you were being careful to use the correct terminology09:49
*** markvoelker has joined #openstack-nova09:51
*** wolverineav has joined #openstack-nova09:57
*** wolverineav has quit IRC10:02
*** IvensZambrano is now known as snevi10:09
*** mikal has joined #openstack-nova10:12
*** amodi has quit IRC10:15
*** mikal has quit IRC10:17
openstackgerritAdam Spiers proposed openstack/nova master: Add detection of SEV support from QEMU/AMD-SP/libvirt on AMD hosts  https://review.openstack.org/63385510:20
*** jistr is now known as jistr|biab10:21
*** ccamacho has quit IRC10:21
*** markvoelker has quit IRC10:23
aspierskashyap: https://review.openstack.org/#/c/633855/10..11/nova/virt/libvirt/host.py10:25
kashyapThank you10:25
kashyapaspiers: Also, once again kudos for the very nice doc string10:26
aspiersthanks :)10:26
*** cdent has joined #openstack-nova10:30
*** ccamacho has joined #openstack-nova10:30
*** abhishekk has quit IRC10:32
openstackgerritKashyap Chamarthy proposed openstack/nova master: qemu: Make disk image conversion dramatically faster  https://review.openstack.org/64078110:37
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Use 'writeback' QEMU cache mode for Nova instances  https://review.openstack.org/64198110:37
kashyapstephenfin: Commented here: https://review.openstack.org/#/c/640866/; it's a simple phrasing fix that removes any ambiguity whatsover, let's please do it?10:48
*** jistr|biab has quit IRC10:51
*** jistr has joined #openstack-nova10:54
*** udesale has quit IRC11:03
*** udesale has joined #openstack-nova11:04
*** luksky has quit IRC11:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Use Selection object to fill request group mapping  https://review.openstack.org/63871111:08
*** dtantsur|afk is now known as dtantsur11:08
*** ratailor has quit IRC11:17
*** markvoelker has joined #openstack-nova11:20
*** jaosorior has quit IRC11:20
sean-k-mooneydid we kill force evacuate yesterday when we released python-novaclient11:28
sean-k-mooneyhttp://logs.openstack.org/65/607365/24/gate/nova-live-migration/0e40d39/job-output.txt.gz#_2019-03-08_07_40_10_65356611:28
sean-k-mooneylooks like we broke the nova-live-migration job11:30
sean-k-mooneylets see if i can figure out where to fix that11:31
*** jaosorior has joined #openstack-nova11:32
*** cdent has quit IRC11:33
*** liuyulong_zzZZZ is now known as liuyulong_11:34
*** luksky has joined #openstack-nova11:36
bauzassean-k-mooney: that's why we provided a major version11:37
sean-k-mooney ya11:37
sean-k-mooneyi found the issue11:37
sean-k-mooneyhttps://github.com/openstack/nova/blob/1554d35834a474514f827449bd7d4f1d2f0af1d6/gate/test_evacuate.sh#L64-L9011:37
sean-k-mooneynot really sure if i should just remove that or what11:37
bauzasthe force flag is no longer accepted11:37
sean-k-mooneyi know11:38
bauzasunless you provide a specific microversion11:38
sean-k-mooneywe removed the api11:38
*** erlon has joined #openstack-nova11:38
sean-k-mooneyha i could use the old microverion yes11:38
bauzasAFAIR, we still need to use the force flag or just using a specific aggregate for forcing to use a compute11:39
sean-k-mooneyya ok ill do that and ill see if we can de evacuate with osc while ime at it11:39
bauzassean-k-mooney: thanks11:39
* bauzas needs to go off 11:39
openstackgerritAdam Spiers proposed openstack/nova master: Use assertXmlEqual() helper for all XML comparison tests  https://review.openstack.org/64185211:43
aspiersstephenfin: https://review.openstack.org/#/c/641852/2..3/nova/test.py11:44
sean-k-mooneydumb question but --os-compute-api-version is used to set the microverion right11:48
sean-k-mooneywew dont use the term micro version in the help text11:48
openstackgerritsean mooney proposed openstack/nova master: update gate test for removal of force evacuate  https://review.openstack.org/64198611:52
*** markvoelker has quit IRC11:54
*** mikal has joined #openstack-nova11:54
*** panda|rover is now known as panda|rover|lunc11:54
*** mikal has quit IRC12:00
*** tosky has joined #openstack-nova12:12
*** _pewp_ has quit IRC12:12
*** _pewp_ has joined #openstack-nova12:13
*** mcgiggler has quit IRC12:18
*** ttsiouts has quit IRC12:19
*** ttsiouts has joined #openstack-nova12:20
kashyapmdbooth: On the off-chance you have time today: https://review.openstack.org/#/c/641981/12:20
kashyapFWIW, the code comment I wrote in there, I got it reviewed by Kevin Wolf of QEMU.12:20
*** yaawang has quit IRC12:21
*** cdent has joined #openstack-nova12:21
*** yaawang has joined #openstack-nova12:22
*** shilpasd has joined #openstack-nova12:22
*** ttsiouts has quit IRC12:24
shilpasdsmcginnis: Hi, need review for openstack/releases patch for masakariclient https://review.openstack.org/#/c/641988/112:25
shilpasddtantsur: Hi, need review for openstack/releases patch for masakariclient https://review.openstack.org/#/c/641988/112:25
dtantsurshilpasd: hi, why do you want a review from me on that? I have no relationship to either releases or masakari (or nova)12:28
shilpasddtantsur: ohh, thanks for confirmation, please ignore the request12:30
openstackgerritAdam Spiers proposed openstack/nova-specs master: Re-approve AMD SEV support for Train  https://review.openstack.org/64199412:31
kashyapaspiers: s/Stein/Train/ in the commit message :-)12:33
bauzassean-k-mooney: yup, that's the field to use12:35
aspierskashyap: doh :)12:39
openstackgerritAdam Spiers proposed openstack/nova-specs master: Re-approve AMD SEV support for Train  https://review.openstack.org/64199412:39
*** hemna has joined #openstack-nova12:42
*** ttsiouts has joined #openstack-nova12:44
shilpasdsmcginnis: Thank you for workflow +1 for masakariclient https://review.openstack.org/#/c/641988/112:48
smcginnisshilpasd: No problem. Probably best not to use the nova channel to discuss release activities for masakari though. ;)12:51
shilpasdsmcginnis: ok, will take a note of it12:52
*** sridharg has quit IRC12:56
*** snevi has quit IRC12:58
*** sridharg has joined #openstack-nova12:59
*** snevi has joined #openstack-nova13:03
*** janki has quit IRC13:07
*** ivve has quit IRC13:08
cdentgibi: is the rest of the bandwidth resource provider expected to land or is it being kicked to train?13:11
sean-k-mooneycdent: on monday i think there were only 3 patches left for stien13:12
sean-k-mooneythe rest was going to be in train13:12
sean-k-mooneyi dont know if we got those 3 in13:12
sean-k-mooneyhttps://review.openstack.org/#/c/636360/13:13
cdentthanks sean-k-mooney. trying to make some notes about it in the pupdate13:13
sean-k-mooneycdent: ^ that was the patch that enabled the feature i think]13:13
sean-k-mooneyso if i was following correctly we have landed enough for spwaning vms with bandwith request13:14
sean-k-mooneyall move opertaions are puntted to train13:15
gibicdent: the heal_allocation part is train (but will be backportad as that is a nova-manage extension). starts here: https://review.openstack.org/#/c/63795313:16
cdentthanks gibi13:16
sean-k-mooneygibi: we are going to make this as compelte right https://blueprints.launchpad.net/nova/+spec/bandwidth-resource-provider13:16
gibicdent: there is two bug like things https://review.openstack.org/#/c/639608/13:16
gibicdent: https://review.openstack.org/#/c/638711/13:16
sean-k-mooneyand add a new spec/blueprint for the final bits for train?13:16
gibisean-k-mooney: I'm not sure, the original spec talked about serve move operations which was not finished (not even started) in Stein13:17
*** eharney has joined #openstack-nova13:17
gibisean-k-mooney: I have to talk to melwitt and mriedem about the administration13:17
sean-k-mooneysure13:17
*** frickler has quit IRC13:17
gibicdent: and there is test coverage extension and refactor here https://review.openstack.org/#/c/641679/13:17
*** frickler has joined #openstack-nova13:18
sean-k-mooneybut enough has lannded that wyou can boot a vm with a min bandwith request now?13:18
gibisean-k-mooney: I'm OK in both ways13:18
gibisean-k-mooney: yeas, boot, delete, and detach port works13:18
cdent\o/13:18
sean-k-mooneycool :) i need to go test that i guess but that is good to see13:19
*** agopi has joined #openstack-nova13:19
sean-k-mooneygibi: we dont have any tempest tests for this right?13:20
sean-k-mooneywe can praobly add them to the numa/nfv test discussion at the ptg13:20
*** panda|rover|lunc is now known as panda|rover13:25
openstackgerritSurya Seetharaman proposed openstack/nova master: Delete orphaned RS and IM records of archived instances  https://review.openstack.org/56004213:26
*** pcaruana has quit IRC13:30
openstackgerritKashyap Chamarthy proposed openstack/nova master: libvirt: Rework 'EBUSY' (SIGKILL) error handling code path  https://review.openstack.org/63909113:30
gibisean-k-mooney: there is a tempest test proposed https://review.openstack.org/#/c/629253/13:34
gibisean-k-mooney: today I finally able too make an env with real SRIOV so I tested that the SRIOV part of the feature works13:35
gibiso far so good13:35
*** mriedem has joined #openstack-nova13:38
mriedemi heard the gate is broken?13:39
*** janki has joined #openstack-nova13:40
mriedemah shit i see13:42
sean-k-mooneymriedem: i proposed a patch for the --force issue13:42
mriedemlink?13:42
mriedembauzas: i bit my own ass http://logs.openstack.org/50/633350/16/check/nova-live-migration/86f6551/job-output.txt.gz#_2019-03-08_09_20_42_23764413:42
sean-k-mooneyhttps://review.openstack.org/#/c/641986/113:42
sean-k-mooneywrong link13:42
sean-k-mooneyoh no never mind13:42
sean-k-mooney ya that was the failure13:43
bauzasmriedem: yup, sean-k-mooney discovered it13:43
mriedemis there a bug?13:43
sean-k-mooneywell technically one of the sriov live migration ptachs discovered it when it was running its gate job13:44
sean-k-mooneyno but i can open one quickly13:44
sean-k-mooneythen respin the patch13:44
*** lbragstad has joined #openstack-nova13:44
mriedemi'm opening one13:45
mriedemjust for e-r13:45
sean-k-mooneyok ill stop creating one so.13:45
sean-k-mooneyi also looked and osc still does not have evacuate support13:46
sean-k-mooneydo we want to go add it?13:46
openstackgerritMatt Riedemann proposed openstack/nova master: update gate test for removal of force evacuate  https://review.openstack.org/64198613:46
*** zhubx007 has quit IRC13:46
*** zhubx007 has joined #openstack-nova13:47
mriedemyes osc should have evacuate support at some point13:47
sean-k-mooneye.g. i will be on a plane for a few hours tomorrow should i go play with trying to add it13:47
sean-k-mooneyill pull down osc anyway just to have it. does anyone know if they swapped to useing the sdk yet?13:49
*** jiteka has joined #openstack-nova13:51
mriedemnot for nova13:53
mriedembut some things yes, like neutron i think13:53
*** derekh has joined #openstack-nova13:53
sean-k-mooneymriedem: by the way what does e-r stand for.13:57
sean-k-mooneyoh elastic recheck13:57
mriedemyeah13:57
*** mlavalle has joined #openstack-nova14:00
*** ccamacho has quit IRC14:01
*** marst has joined #openstack-nova14:03
*** snevi has quit IRC14:04
*** cdent has quit IRC14:06
*** BjoernT has joined #openstack-nova14:08
sean-k-mooneyout of interest if i was to  add evacuate to osc, would people object to me calling it "recreate" e.g. openstack server recreate or somethign closer to what it actully does?14:08
edleafesean-k-mooney: generally the osc names are supposed to be saner than the names in the python-*client libraries14:09
sean-k-mooneyya what i would personally prefer is 3 commands14:10
sean-k-mooneyopenstack server recreate e.g. nova evacuate14:10
sean-k-mooneyopenstack host evacuate e.g. nova host-evecuate14:11
sean-k-mooneyand opentack host evacuate --live fo nova host-evacuate-live14:11
*** BjoernT_ has joined #openstack-nova14:11
sean-k-mooneyor actully it could be openstack host migrate and openstack host migrate --live14:12
*** awalende has quit IRC14:13
*** BjoernT has quit IRC14:13
*** awalende has joined #openstack-nova14:13
sean-k-mooneyin anycase i have never modifed osc before so ill jsut start with porting nova evacuate and see how things go14:13
bauzassean-k-mooney: openstack host evacuate doesn't exist from an API PoV14:14
bauzasit's just a client thingy14:14
edleafeyeah, I'm not sure which terms would make the most sense. But yes, names that actually convey what the call does are preferred. :)14:14
sean-k-mooneybauzas: yes i know14:14
bauzasso, no to be an OSC CLI command14:14
sean-k-mooneybauzas: is that a rule that osc cant have things that are not directly supported by the api14:15
bauzasgood question14:15
sean-k-mooneybut ok i was going to look at those after so no worries ill just focus on nova evacuate first ot figure out how things work14:15
bauzasbut honestly, I hate the host-evacuate method14:15
*** mcgiggler has joined #openstack-nova14:16
* sean-k-mooney note to self dont add bauzas to the review if i work on it :)14:16
sean-k-mooneyya i have personally neverr used it but i have used the horizon one in the past but mainly for testing14:17
*** awalende has quit IRC14:18
*** sridharg has quit IRC14:18
bauzassean-k-mooney: FWIW, I'm planning to ask for some time for me in the next cycle for OSC gap closure14:19
bauzason the next week :p14:19
bauzasso you could see my name somewhere in your OSC evacuate change :p14:20
sean-k-mooneyoh cool we brought it up at the last ptg but stephen and i never really got around to doing it14:20
*** markvoelker has joined #openstack-nova14:20
sean-k-mooneyit woudl be nice to swap entirely to osc at some point14:21
sean-k-mooneybut ya there are a number of challanges with that at present14:21
openstackgerritKashyap Chamarthy proposed openstack/nova-specs master: Re-propose the spec to allow specifying a list of CPU models  https://review.openstack.org/64203014:24
*** sunnaichuan has quit IRC14:29
*** awaugama has joined #openstack-nova14:30
*** sridharg has joined #openstack-nova14:31
*** IvensZambrano has joined #openstack-nova14:32
*** mrch has quit IRC14:32
sean-k-mooneyspeaking of spec i need to write and review a few14:34
*** pcaruana has joined #openstack-nova14:37
*** jchhatbar has joined #openstack-nova14:40
*** cdent has joined #openstack-nova14:41
*** phasespace has quit IRC14:41
*** jchhatba_ has joined #openstack-nova14:42
*** janki has quit IRC14:42
bauzassean-k-mooney: closing OSC gap is one thing, swap to OSC is a totally different thing14:44
bauzassean-k-mooney: I'm just talking of the next TC goal which is somehow close to the former14:44
bauzasat least at the last consensus point we had14:45
*** jchhatbar has quit IRC14:45
*** s10 has joined #openstack-nova14:50
*** snevi has joined #openstack-nova14:51
*** IvensZambrano has quit IRC14:51
*** efried is now known as fried_rice14:51
*** wolverineav has joined #openstack-nova14:51
*** mikal has joined #openstack-nova14:53
*** markvoelker has quit IRC14:54
*** wolverineav has quit IRC14:55
openstackgerritArtom Lifshitz proposed openstack/nova master: Dont' wait for VIF plugging during revert resize  https://review.openstack.org/63939614:57
artomAlright, I think the tests should pass on ^^ now14:57
*** mikal has quit IRC14:57
artommriedem, can I ask you to hit that when you get a moment? You're neck-deep in the resize flow anyways ;)14:57
dansmithmriedem: fried_rice: related failure to a novaclient release? http://logs.openstack.org/92/624592/11/gate/nova-live-migration/06aea60/job-output.txt.gz#_2019-03-08_12_45_00_25012314:58
artomstephenfin, how much do you know about resize? Can I pick you as the lucky RH core for ^^ ?14:59
mriedemdansmith: yes15:00
mriedemdansmith: https://review.openstack.org/#/c/641986/15:00
*** mrch has joined #openstack-nova15:01
dansmithah okay15:01
mriedemartom: did you sort that out with dansmith?15:01
dansmithmriedem: he's just removing a wait for events on resize, which is more a you think than me15:02
dansmithit doesn't break the context manager for waiting for events, so I'm happy(er)15:02
artommriedem, I think so. I also checked with slaweq who's in our Neutron team, and he confirmed that the VIF remains plugged on the source host15:03
dansmithartom: um, what? :)15:03
artomdansmith, what what?15:04
artomIs plugged the right word? Wired?15:04
mriedemi feel like i just had a patch like this15:04
mriedembut was maybe for confirm15:04
dansmithartom: that makes no sense for what you're saying here15:04
artomdansmith, entirely plausible, but explain how :)15:04
dansmithoh, this is finish_revert, I see15:04
dansmithI thought it was finish-finish15:05
artomSwedish finish?15:05
aspierssean-k-mooney: strong +1 for renaming evacuate, my preferred term would probably be "resurrect"15:05
dansmithbut still, you're not asserting this was always broken right?15:05
artomdansmith, always racy15:05
dansmithartom: what's it racing with?15:05
artomNeutron with the virt driver15:05
aspierssean-k-mooney: "revive" might also work15:06
mriedemartom: so you're reverting this essentially right? https://review.openstack.org/#/q/I9e0cffb889c94713c7f28812918103a5d97cefeb15:06
artommriedem, err, yes. WTF.15:06
mriedemok -1 until we have a good explanation of why https://review.openstack.org/#/c/595069/ was wrong15:08
*** Luzi has quit IRC15:08
mriedembut maybe that explains a regression you're seeing downstream?15:08
dansmithartom: okay, so tickling from compute manager will cause that event to start heading towards us because there are still interfaces available for neutron agent to find.. I kinda buy that, I guess...15:08
artommriedem, fair enough15:08
dansmithI also -1d for the drunk speak in the commit message15:09
artom...15:09
artomWell now you're just picking on em :(15:09
artom*me15:09
dansmithbut yeah, I'd like to hear from lyarwood I guess15:09
mriedemdansmith: it was my change :)15:09
mriedemlyarwood backported it15:09
mriedemand https://review.openstack.org/#/c/179228/ was your change :)15:10
mriedemit's great15:10
artomIn a meeting now, I think I'll have to play around with it afterwards15:10
artomUntil now I was relying on logs15:10
artomYo-yo patches...15:11
stephenfinoh, that's interesting15:11
dansmithoh15:15
mriedemartom: dansmith: ok so i see the race in https://review.openstack.org/#/c/595069/15:15
mriedemthe dest triggers the event,15:15
mriedemthe source is waiting for it,15:15
*** jchhatba_ has quit IRC15:15
mriedembut it might come before the source is registered for the callback15:15
*** janki has joined #openstack-nova15:16
dansmithso wait, we've gone back and forth twice now?15:16
mriedemdest unplugs vifs when it calls driver.destroy15:16
mriedemyes15:16
mriedemwhen the original change was made by dansmith we didn't have the code in the API which routes events to both the source and host if the instance had a migration context15:16
mriedemso when i made my change, the source will get the event routed to it,15:16
mriedembut, we might not be registered for the callback on the source by the time the event arrives15:17
mriedemas you can see from my comments in https://review.openstack.org/#/c/179228/ it's all very confusing15:17
mriedemthe sequence of events is kind of tribal knowledge with only 1.5 people in the tribe nowadays15:18
*** amodi has joined #openstack-nova15:18
mriedemso we should probably (1) revert my change and (2) add a comment in the libvirt driver finish_revert_migration code about why we don't wait for the event (like there is a comment in finish_migration)15:19
mriedembecause the event does come to the source, but we are racing to catch it15:20
*** udesale has quit IRC15:21
mriedemactually...finish_revert_migration does plug vifs,15:21
mriedemso why wouldn't we get an event for that on the source?15:21
*** udesale has joined #openstack-nova15:21
mriedemdest unplugs vifs on revert_resize because of driver.destroy,15:21
mriedemsource plugs vifs because of finish_revert_migration which re-spawns the guest15:21
mriedemartom: are you seeing this downstream with OVS or linuxbridge?15:22
artommriedem, ovs15:22
artommriedem, we do get the event on the source, just before we wait for it15:23
artomBecause as soon as Neutron gets the API call, it's able to wire them and send us the events15:23
*** janki has quit IRC15:23
mriedemthe network-vif-plugged event?15:23
artomYeah15:23
mriedemok that's from something else then15:24
mriedembecause when the source calls plug_vifs during finish_revert_migration, it's doing it within the context to wait for the event15:24
artommriedem, do you have access to slaweq's notes on our downstream BZ?15:25
artommriedem, https://bugzilla.redhat.com/show_bug.cgi?id=1678681#c815:25
openstackbugzilla.redhat.com bug 1678681 in openstack-nova "REVERT_RESIZE stuck for 300s: "VirtualInterfaceCreateException: Virtual Interface creation failed" [Medium,On_dev] - Assigned to alifshit15:25
mriedemartom: then i'm pretty sure it's what we've talked about before, finish_revert_resize on the source host calls migrate_instance_finish which updates the port binding to point at the source host, which triggers an event15:26
mriedem*before* the driver registers the callback15:26
artommriedem, right, but there's another thing at play here - Neutron will only wire the ports after we plug them15:27
artom... I think15:27
mriedemyes i can read his comments and i think they align with what i just said,15:28
mriedem"Nova then asks neutron to bind port on compute-0 again, at it happens:" - that's the migrate_instance_finish call on the source finish_revert_resize to update the port binding15:28
mriedem"And after that, L2 agent on compute-0 wire port again:" - i assume that's the plug_vifs call from the driver's finish_revert_migration on the source host15:29
mriedemartom: how recreateable is this? it would be a lot more clear if we just had some logging on the source that said, "updating port bindings to point at the source host" and then "finish revert migration in the driver which will plug vifs"15:30
mriedembecause if we saw the event come between those 2 messages we know it's the port binding change that is triggering the event before we're ready to wait for it15:30
artommriedem, like, 70% in our CI?15:30
mriedemare you able to test with patches from upstream?15:31
artomI'd need to check with the CI guys, but I could provide them test builds, yeah15:31
kashyapaspiers: You about...?15:31
*** s10 has quit IRC15:32
*** s10 has joined #openstack-nova15:32
*** shilpasd has quit IRC15:32
kashyapaspiers: When you are -- at the risk of adding more work ... wonder if we should simply split out the addition of getDomainCapablities() method into its own patch.15:32
*** s10 has quit IRC15:32
mriedemartom: ew do you have to patch an rpm or something?15:33
*** s10 has joined #openstack-nova15:33
kashyapIt is just my OCD of "one logical change per-patch thing".  And it allows quicker merge, too.  As it'll be easier on the reviewers eyes15:33
artommriedem, scratch build15:33
artomBackport a patch, build RPMs with that15:33
*** s10 has quit IRC15:33
*** s10 has joined #openstack-nova15:34
mriedemartom: actually you should be able to determine this with existing logs15:34
kashyap(A "scratch build" is something that is short-lived, and will be "scratched" from the build system)15:34
*** s10 has quit IRC15:34
*** s10 has joined #openstack-nova15:34
artom(Backport a patch, build a SRPM without pushing anything, scratch build, I should have said)15:34
mriedemartom: you should see this on the source https://github.com/openstack/nova/blob/4f9bc724010f0c935bf83a6d19bdd805e86b7086/nova/network/neutronv2/api.py#L335515:35
*** s10 has quit IRC15:35
mriedemwith binding:host_id changing to point at the source host15:35
*** s10 has joined #openstack-nova15:35
mriedemand then you should see this from the driver https://github.com/openstack/nova/blob/4f9bc724010f0c935bf83a6d19bdd805e86b7086/nova/virt/libvirt/driver.py#L896515:35
artommriedem, ack, looking15:35
*** s10 has quit IRC15:36
*** s10 has joined #openstack-nova15:36
mriedemgranted there is some other stuff that happens in the driver after that before unplug_vifs happens, so it's a tight window15:36
*** s10 has quit IRC15:36
*** s10 has joined #openstack-nova15:37
*** s10 has quit IRC15:37
*** s10 has joined #openstack-nova15:38
*** s10 has quit IRC15:38
*** s10 has joined #openstack-nova15:38
*** jangutter has quit IRC15:39
*** s10 has quit IRC15:39
*** s10 has joined #openstack-nova15:39
*** s10 has quit IRC15:40
*** s10 has joined #openstack-nova15:40
*** s10 has quit IRC15:40
*** s10 has joined #openstack-nova15:41
*** s10 has quit IRC15:41
*** s10 has joined #openstack-nova15:42
aspierskashyap: yeah could do, I kind of like there being an incentive to get SEV stuff merged though ;-)15:42
*** s10 has quit IRC15:42
*** s10 has joined #openstack-nova15:42
aspierswhen do we fork for stein?15:42
*** s10 has quit IRC15:43
kashyapaspiers: Yeah, I hear you.  But as we both know ... it can be used for multiple features :-)15:43
*** s10 has joined #openstack-nova15:43
kashyap(I won't insist on it, though.  But if you appetite...)15:43
aspierswe already hit feature freeze, right?15:43
aspiershttps://wiki.openstack.org/wiki/Nova/Stein_Release_Schedule15:44
kashyapaspiers: Yes, yesterday15:44
*** s10 has quit IRC15:44
kashyapmriedem: ^ Right?15:44
*** s10 has joined #openstack-nova15:44
*** s10 has quit IRC15:44
mriedemyes15:45
mriedemplease don't be approving anything that's not already approved15:45
kashyapaspiers: Also think of it this way: splitting it out allows it to be merged while SEV bits get reviewed :-)15:45
*** s10 has joined #openstack-nova15:45
aspiersI guess15:45
kashyapmriedem: This (AMD SEV work) was already approved for Stein15:45
*** s10 has quit IRC15:45
aspiersI think mriedem means W+115:45
kashyapaspiers: But sorry to be "that guy"; you're allowed to hate me for 5 minutes.15:45
aspiershaha15:46
sean-k-mooneyaspiers: we will fork stien at RC1 so master is feature frozen for the next 2 ish weeks until that is done i think15:46
aspierssean-k-mooney: OK thanks15:46
kashyapaspiers: This work will spill over into "Train", yes?15:47
sean-k-mooneyrc1 will be around march 21st15:47
aspierskashyap: https://review.openstack.org/#/c/641994/15:47
aspiersOh, you already saw that :)15:47
aspiersI forgot15:47
kashyapYeah, no worries.15:47
*** fried_rice is now known as efried_afk15:48
aspiersAnyway, ultimately I need to do whatever you guys think is best :)15:48
openstackgerritBalazs Gibizer proposed openstack/nova master: Documentation for bandwidth support  https://review.openstack.org/64206415:48
aspiersIf splitting it out helps then let's do that15:48
kashyapaspiers: Yeah, it also will be less "daunting" to those who don't normally dwell on this area of code15:49
aspiersSure15:49
mriedemwhat are you talking about splitting out?15:49
aspiersthe call to getDomainCapabilities15:49
*** marst has quit IRC15:50
*** TheJulia is now known as needssleep15:50
kashyapmriedem: Yeah, the adding of getDomainCapablities() in libvirt/host.py15:50
aspierskashyap: I was also thinking about splitting this code move out https://review.openstack.org/#/c/633855/11/nova/virt/libvirt/utils.py15:50
mriedemaspiers: ok, i see you already put the -W back on your own change that i don't want merged in stein at this point, so you know where i am on this15:50
kashyapmriedem: This bit (line 680): https://review.openstack.org/#/c/633855/11/nova/virt/libvirt/host.py15:50
*** markvoelker has joined #openstack-nova15:51
*** luksky has quit IRC15:51
mriedemsure, split that out even if to just make that change smaller and easier to grok15:51
mriedemtoo much code in one patch == hard to review15:51
aspiersmriedem: yes, that's also why I submitted https://review.openstack.org/#/c/641994/15:51
kashyap(Yeah, I agree with mriedem on AMD SEV for Stein, I'm afraid.)15:51
kashyapaspiers: Yes.  The machine_type_mappings() in a separate change is good, too.15:52
aspiersThere's not even a debate on that, since feature freeze was yesterday :)15:52
*** dtantsur is now known as creepy_owlet15:52
mriedemaspiers: you'd be surprised15:52
*** mdbooth has quit IRC15:52
aspiersHaha OK, well I'm sure some vendors are insane but not us15:52
kashyapIt's the same story with upstream _kernel_ as well :D15:54
aspiersYup15:54
aspiersSUSE knows all about that ...15:54
kashyapOnce they start cutting long-term 'stable' release, everyone starts running around headless-chickens15:54
kashyap... "we need to get this into the "long-term" release!"15:54
sean-k-mooneykashyap: honestly i think that is better the feature backports15:54
gibimriedem, melwitt: I'm wondering about how to close the https://blueprints.launchpad.net/nova/+spec/bandwidth-resource-provider the spec talks about the server move operations that was not implemented in Stein so if we use that as a scope then the bp cannot be marked implemented15:55
kashyapsean-k-mooney: Not saying it's bad :-)  Just the reality and the incentive structure I guess15:55
mriedemgibi: i think we can complete the blueprint in stein and have a separate blueprint for move operations in train15:55
aspiersOK I'm done for the week.  I'll look at splitting that out into 3 reviews next week15:55
aspiersHave a good w/e all! o/15:55
kashyapaspiers: Just a heads-up, next week the first three days I'm travelling; so, if I'm slow on responding ... you know why :-)15:56
kashyapHave a good one!15:56
aspiersOK np :)15:56
gibimriedem: shall I remove the move operation from the old spec when I update that now?15:56
mriedemgibi: are you going to amend the stein spec for the microversion and all the other stuff that came up?15:56
mriedemi guess i'm not sure what you're asking me :)15:57
gibimriedem: yeah I'm plannig to amend the spec15:57
gibimriedem: so I can remove the move from the scope there15:57
*** lbragstad is now known as elbragstad15:58
*** ccamacho has joined #openstack-nova15:58
mriedemok15:58
gibimriedem: ok15:59
*** dcdawg has joined #openstack-nova16:00
gibimriedem: there are two trailing patche for the bandwidth series both can be considered bugfixes, https://review.openstack.org/#/c/638711/ and https://review.openstack.org/#/c/639608/ what do you think, shall I push them forward before RC1?16:03
*** artom has quit IRC16:05
*** mmethot has quit IRC16:06
openstackgerritguang-yee proposed openstack/nova master: pass endpoint interface to Ironic client  https://review.openstack.org/64087916:09
*** awalende has joined #openstack-nova16:14
*** efried_afk is now known as fried_rice16:16
*** awalende has quit IRC16:19
*** artom has joined #openstack-nova16:20
*** markvoelker has quit IRC16:23
*** sean-k-mooney has quit IRC16:23
*** mmethot has joined #openstack-nova16:26
*** dcdawg has quit IRC16:27
mriedemgibi: on https://review.openstack.org/#/c/639608/ i think that's already a potential bug because of not handling errors from claim_resources16:27
mriedemso it's low priority for me, but i think it's a bug we can report and work on16:27
mriedemas i said before that whole while loop needs to be refactored at some point (that whole damn build_instances method actually)16:27
mriedemgibi: https://review.openstack.org/#/c/638711/ isn't really a bug per se, but it's a performance optimization, so again i don't know that we need to rush it, but it's still good to have16:28
mriedemyou could report a low priority bug for tracking that if you wanted - as a perf issue16:29
mriedemlike i said, if i'm multi-creating 100 instances in one request, that's 100 separate GET /allocations calls16:29
mriedemalthough, can you multi-create with a pre-created port...?16:29
mriedemi'm not sure that is supported16:29
mriedemthat would be like multi-create with the same pre-created volume16:30
mriedemso that likely doesn't work anyway16:30
openstackgerritEric Fried proposed openstack/nova master: update gate test for removal of force evacuate  https://review.openstack.org/64198616:31
fried_ricemriedem: sean-k-mooney stephenfin bauzas tssurya ^16:32
fried_ricehttp://logs.openstack.org/86/641986/2/check/nova-live-migration/6f6b678/job-output.txt.gz#_2019-03-08_14_32_56_00435716:32
*** artom has quit IRC16:33
tssuryafried_rice: oops16:33
*** belmoreira has quit IRC16:34
cfriesenmriedem: pretty sure you can't multi-create with pre-created port16:34
openstackgerritKashyap Chamarthy proposed openstack/nova-specs master: Re-propose the spec to allow specifying a list of CPU models  https://review.openstack.org/64203016:34
mriedemcfriesen: yeah i didn't think so..16:36
mriedemfried_rice: ah crap16:36
fried_ricemriedem: Did I do it right? I have very little experience with the CLI16:37
cfriesenCould someone take a look at this and clue me in why my new unit test (test_executable_exists_in_path(), in test_utils.py) is failing?  Is this a zuul thing?  It works fine locally.16:38
cfriesenhttps://review.openstack.org/#/c/64193216:38
mriedemcfriesen: i looked at that the last time you asked and i couldn't spot anything obvious, but i'm sure there is some global you're hitting16:38
mriedemfried_rice: yeah i think so, the 'common' options come before the command, and then the per-command options16:39
mriedemi'm not sure if osc is that picky but i think novaclient is16:39
fried_ricecfriesen: Have you tried full-pathing ls?16:39
cfriesenfried_rice: the "ls" is fine, it's the call to libvirt_utils.executable_exists_in_path() that is unexpectedly failing16:40
fried_riceduh, sorry, yeah16:40
fried_riceI hadn't really parsed yet, that was just something that jumped out at a glance.16:40
cfriesenit's totally reliable locally, fails every time in zuul16:40
* fried_rice pulls...16:40
fried_ricecfriesen: ah, `type`. Shell builtins can be funny. There's a way to invoke a builtin as an executable, looking...16:42
fried_ricecfriesen: Tough that it can't be reproduced locally; but you can try changing "type ..." to "builtin type ..."16:44
fried_ricecfriesen: also, based on the comment, if what you truly care about is that it's an executable (as opposed to a function, alias, or builtin), you could use /usr/bin/which instead of type.16:45
mriedemcfriesen: there is also this https://wiki.openstack.org/wiki/Testr#Reproducing_Failures16:45
mriedemif you get desperate16:45
mriedem^ is how you can try to spot racing globals16:45
*** artom has joined #openstack-nova16:45
fried_ricemriedem: whoosh, that could use some updating for stestr, huh16:46
mriedemyeah probably16:46
mriedemgood thing anyone with a lp account can change it :)16:47
fried_ricecfriesen: oh, sorry again, the problem is likely to be with $PATH, which definitely shouldn't include /tmp in real life.16:50
fried_ricecfriesen: I'm able to repro locally.16:50
cfriesenfried_rice: ah, excellent.  wonder why it works locally here16:52
cfriesenI don't have /tmp in PATH16:52
fried_riceyeah, that's really strange cfriesen, because even if you had /tmp in your $PATH, mktemp creates in a randomly-named subdirectory.16:52
cfriesenI think my bash must be messed up, with "type -P" returning whether the binary is executable even if it's not in $PATH16:56
*** ccamacho has quit IRC16:57
*** ttsiouts has quit IRC16:58
cfriesenfried_rice: interesting, fedora 29 behaves the same as mine16:58
cfriesenmaybe I'll just switch to "which".16:58
openstackgerritArtom Lifshitz proposed openstack/nova master: Dont' wait for VIF plugging during revert resize  https://review.openstack.org/63939616:58
mriedemgibi: smallish thing in https://review.openstack.org/#/c/638711/16:58
mriedemartom: are you going to ever fix that typo in your commit title?16:58
mriedemdon't make me -5 you16:59
artomChrist :(16:59
openstackgerritArtom Lifshitz proposed openstack/nova master: Don't wait for VIF plugging during revert resize  https://review.openstack.org/63939616:59
artomIt's, err, the possessive form?17:00
fried_ricecfriesen: using type -P with a full path is weird anyway.17:01
fried_ricecfriesen: Are you actually wanting to know if a fully-pathed file is executable, or are you wanting to know if a name (not a full path) is in your PATH *and* executable?17:02
fried_ricecause the former should really be done natively (os.stat | O_EXEC kind of thing)17:02
cfriesenfried_rice: the latter.  using the full path was just for the testcase.  I'm trying to make sure that "swtpm" exists on the system before advertising support for emulated TPM on that node.17:03
fried_ricecfriesen: Okay. Then the test case shouldn't use the full path either.17:04
cfriesenone could argue that's up to the operator to ensure, in which case I could skip the check entirely.17:04
*** udesale has quit IRC17:04
fried_ricemeh, it seems reasonable, if you can get it to work.17:05
mriedemartom: so i don't think you need to change that finish_revert_migration interface at all,17:07
mriedemwe just need to be registering an event callback prior to call migrate_instance_fininsh17:07
fried_ricecfriesen: Is / in your $PATH?17:07
cfriesennope17:08
cfriesenI think it must be a non-exclusive test, where it checks $PATH but also the full path to the executable if specified17:08
artommriedem, I thought that's what dansmith *didn't* want17:09
fried_riceweird that it's different on different platforms.17:09
mriedemartom: this is the kind of thing that the 3 of us would probably need to get on a hangout to discuss17:09
mriedemso we stop talking past each other while half paying attention17:09
dansmitharen't we just reverting mriedem's patch to expect an event here?17:10
artomdansmith, you just want to +W a mriedem revert, don't you :)17:10
artomI have to go do lunch with colleagues, but I like the hangouts idea17:10
mriedemyes he's reverting my patch but in an uglier way17:10
dansmithartom: no, I want to avoid the other thing, and since we've been in that "don't wait for the event" state a few times now, I think first step is just getting back to that, unless there's some reason not to17:10
*** xek has quit IRC17:11
cfriesenfried_rice: if I modify os.environ['PATH'] in the testcase to include the directory containing the temp dir, do you think that'd work?17:11
fried_ricecfriesen: I tried that locally and it didn't work, no.17:11
fried_ricebut I'm not sure why.17:11
*** xek has joined #openstack-nova17:11
artomdansmith, so you'd rather I just straight up revert mriedem's patch? Functionally it's the same17:11
cfriesenguess I'm switching to "which" then. :)17:11
mriedemartom: before doing that i'd like to have a plan,17:12
fried_ricecfriesen: Note that type -P doesn't check whether the thing is executable (at least on my bionic)17:12
mriedembecause if we know the event is coming, i don't see why we don't wait for it17:12
fried_riceefried:~/openstack/nova$ PATH=$PATH:/tmp type f17:12
fried_ricef is /tmp/f17:12
fried_riceefried:~/openstack/nova$ ll /tmp/f17:12
fried_rice-rw-r--r-- 1 efried efried 4169 Mar  8 08:53 /tmp/f17:12
dansmithartom: I'm not saying a full revert vs. tactical is important17:12
mriedemclearly my change worked albeit is racy17:12
artommriedem, ah, I see what you mean17:12
dansmithmriedem: because it's hard to do that based on the arrangement of the code17:12
*** pcaruana has quit IRC17:12
mriedemartom: because i'm pretty sure i made that change b/c of gate failures17:12
mriedemi.e. resize revert + ssh in tempest race fails17:13
mriedemb/c we aren't waiting for plugging17:13
mriedemso you revert this w/o fixing that and we've got new gate failures again17:13
dansmiththat's why this whole thing exists, yes17:13
mriedemwell, same old gate failures17:13
cfriesenfried_rice: oh, good catch.17:13
mriedemdansmith: we can do the same thing we do for live migration17:13
dansmithcan we not wait for the plug event before leaving the compute manager ?17:13
fried_ricecfriesen: `which` is much better, because it also doesn't fire if the file is executable, but not by you.17:13
dansmithlike way before we get to the driver.finish_revert_migration() ?17:14
*** amodi has quit IRC17:14
artomdansmith, ah, right. I thought that wasn't possible because we needed the virt driver to plug the vifs17:14
dansmithartom: you've been saying the virt driver isn't causing the vif plug event17:14
artomBut if they're already plugged, we can wait in the compute manager, and tell virt not to wait for it17:14
dansmithso which is it?17:14
dansmithright.17:14
artomdansmith, I'm not 100% on how the Neutron agent works, but my understanding is that Neutron will only wire the thing if the virt driver had plugged it17:15
dansmithartom: that contradicts what you said earlier,17:15
artomSo we need both: poke Neutron, and plug (or have it be already plugged)17:15
dansmithwhich is that the vif is already on the source, so the network_api call is what is causing neutron to send the event, which we might miss if we're not fast enough17:15
artomdansmith, because it was never unplugged to begin with17:16
mriedemdansmith: "like way before we get to the driver.finish_revert_migration() ?" yes that's what i' msaying17:16
mriedemartom: right here you are contradicting what you just said, https://review.openstack.org/#/c/639396/3/nova/compute/manager.py@418417:16
dansmithmriedem: gotcha, I'm with you now.. I thought yesterday you were saying that a straight revert was the best first step.. with this, we race in control plane code, without this, we race on the instance17:17
mriedemSo the times are:18:50:51.138 Update port binding 18:50:54.239 Event received 18:50:54.300 Event ignored on both computes 18:50:54.483 Driver finish_revert_migration start17:17
dansmithand you could argue that the latter is better17:17
dansmithI'm a little frustrated by this whole deal, so how about we table this until artom comes back from lunch and we get on a hangout?17:17
mriedemsure, so we could (1) revert my change saying oopsy we still race and then (2) stack on top of that a change to wait for the event prior to changing the port binding which triggers the event17:17
dansmithI don't care about the mechanics17:18
mriedemyeah hangout for sure, since i'm leaving for lunch soon too17:18
artomCool. Lunch then hangout. At 13... 30?17:18
artomErr, eastern time, so in an hour and 10?17:18
mriedema hangout will also give me an opportunity to show off my excessively manly beard17:18
mriedemartom: like 3pm ET for me17:19
dansmithmriedem: I assume you mean your sparse patchy white boy beard?17:19
mriedemwhich is noon for dan17:19
mriedemdansmith: of course17:19
dansmithum, +1:10 from now is not noon for me17:19
dansmithI can't be on a hangout right at noon, but before or shortly after is fine.. but noon is 2:50 from now17:20
mriedem2pm CT is noon PT yeah?17:20
artomSpeak in deltas :)17:20
mriedemyes i'm saying 3PM ET or thereabouts17:20
artom1:10 from now?17:20
mriedemno17:20
dansmithomg17:20
mriedemhow about we just ping later17:20
mriedembreak17:20
dansmith(the sound of a gunshot in oregon echoes through)17:20
*** markvoelker has joined #openstack-nova17:20
* artom off17:21
*** tesseract has quit IRC17:22
*** gyee has joined #openstack-nova17:22
*** erlon has quit IRC17:23
openstackgerritChris Friesen proposed openstack/nova master: Add emulated TPM support to Nova  https://review.openstack.org/63136317:26
*** creepy_owlet is now known as dtantsur|afk17:29
*** rpittau is now known as rpittau|afk17:31
*** erlon has joined #openstack-nova17:39
fried_ricemriedem: I just added https://review.openstack.org/#/c/640879/ to https://etherpad.openstack.org/p/nova-stein-rc-potential -- is that a right thing?17:39
mriedemthat's not a regression right? it's latent since pike or queens?17:40
fried_ricenot a regression in stein, yes latent since several releases, will need to be backported there.17:40
fried_riceso, not sure how that works; do we get it in now before rc and then backport it, or wait until stein is forked, drop it in master, wait for the dust to settle, then backport it to stable branches (including stein)?17:41
mriedemtechnically if it's not a regression in stein it doesn't need to be stein-rc-potential (tag)17:41
mriedemso the latter17:41
mriedemif it's latent, we can fix it whenever and backport17:41
*** snevi has quit IRC17:41
mriedem^ doesn't bar people from reviewing latent bug fixes before stein rc1 if they wnat17:42
mriedem*want17:42
mriedemjust dont want to push anything in that is high risk if we don't need to17:42
mriedemand i'm not saying this is high risk17:42
mriedemi haven't looked it over17:42
mriedemwhat i am saying is i'm going to lunch now17:43
*** mriedem is now known as mriedem_burgers17:43
fried_ricethanks mriedem_burgers17:43
*** fried_rice is now known as fried_rolls17:43
*** derekh has quit IRC17:44
*** snevi has joined #openstack-nova17:45
*** cdent has quit IRC17:47
*** panda|rover is now known as panda|rover|baby17:48
*** xek has quit IRC17:49
*** xek has joined #openstack-nova17:49
*** agopi is now known as agopi|afk17:50
*** mcgiggler has quit IRC17:51
*** markvoelker has quit IRC17:54
*** tssurya has quit IRC17:55
*** igordc has joined #openstack-nova17:59
*** wolverineav has joined #openstack-nova18:17
*** dcdawg has joined #openstack-nova18:23
*** cdent has joined #openstack-nova18:24
*** dcdawg has quit IRC18:27
*** wolverineav has quit IRC18:31
*** sridharg has quit IRC18:34
*** betherly has joined #openstack-nova18:34
*** wolverineav has joined #openstack-nova18:35
*** IvensZambrano has joined #openstack-nova18:37
*** igordc has quit IRC18:38
*** betherly has quit IRC18:39
*** igordc has joined #openstack-nova18:40
*** snevi has quit IRC18:40
*** erlon_ has joined #openstack-nova18:51
*** markvoelker has joined #openstack-nova18:51
*** erlon has quit IRC18:54
*** amodi has joined #openstack-nova18:55
artommriedem_burgers, dansmith, back, ready when you are19:03
*** wolverineav has quit IRC19:03
*** wolverineav has joined #openstack-nova19:03
dansmithI too am here19:04
*** wolverineav has quit IRC19:06
*** wolverineav has joined #openstack-nova19:06
*** snevi has joined #openstack-nova19:10
*** IvensZambrano has quit IRC19:10
*** mriedem_burgers is now known as mriedem19:16
mriedemi'm back but19:16
artombackbutt19:17
dansmithas opposed to front butt?19:17
* artom is a fan of the sidebutt19:17
mriedemgimme a minute19:18
artomFor what it's worth, revert https://review.openstack.org/#/c/595069/ + wait in compute manager is fine19:18
artomErr, *I think that revert blah blah19:18
mriedemhttps://hangouts.google.com/call/4MUfp6uNnY6ddII9gtXGAEEE19:19
*** hemna has quit IRC19:20
*** hemna has joined #openstack-nova19:21
*** markvoelker has quit IRC19:24
mriedemartom: see _get_neutron_events_for_live_migration for how live migration uses it19:25
artommriedem, ack19:26
mriedemso i think we just wrap that around the migrate_instance_finish in finish_revert_resize19:26
mriedemi'd probably split that code out to a new helper method for sanity..19:27
*** wolverineav has quit IRC19:34
*** wolverineav has joined #openstack-nova19:35
artomRight, I gotta run again, I'll try to have those 2 patches up before the weekend19:37
*** wolverineav has quit IRC19:40
*** artom has quit IRC19:42
*** waverider has joined #openstack-nova19:45
cfriesenso in devstack I'm running "nova migrate <uuid>" and I'm getting an error saying that it can't migrate to current host.  Shouldn't it be disregarding the current host for migration?20:00
dansmithcfriesen: do you only have one host?20:03
*** erlon_ has quit IRC20:03
cfriesendansmith: nope, two20:03
dansmithbut I think you'll get that if the scheduler doesn't pick the other one as a candidate20:04
cfriesenshed logs see it considering both right up till the weigher20:04
cfriesenbut for "migrate" we'd never want the current host20:04
cfriesenonly for resize20:04
*** mikal has joined #openstack-nova20:04
dansmiththey're the same operation, so if allow-to-same is enabled, I think it'll allow it20:04
openstackgerritMichael Still proposed openstack/nova master: Make privsep.chown mocking for libvirt snapshot tests less magic.  https://review.openstack.org/64213420:06
*** wolverineav has joined #openstack-nova20:06
cfriesenokay, but that makes no sense logically to the user.  a "migrate" operation should mean moving it elsewhere.20:06
dansmithcfriesen: yeah, I'm not arguing that it makes sense, I'm just stating what I think is the case20:06
dansmithalso, migrate is admin-only by default I think, so it might not get much visibility because of that20:06
*** mikal has quit IRC20:07
cfriesenI suspect you're right.  but it means that with "resize on same host" enabled you currently have to explicitly specify a destination.20:07
dansmithno,20:07
*** mikal has joined #openstack-nova20:07
cfriesenI'm pretty sure this is a regression, just not sure from when20:07
dansmithit only means that if your current host is the most obvious target right?20:07
cfriesendansmith: okay, fair enough.  it means you can't assume it won't land on the current host20:08
dansmithI dunno how common it is to have allow-to-current enabled in production either20:08
dansmithit was supposed to only be for gate testing20:08
cfriesenIt's useful in small clouds.20:09
dansmithagain, stating the original intent20:09
cfriesenwhat was the rationale for defaulting it to "off"?20:09
dansmithbecause it was only for testing20:09
cdenttautology!20:10
cfriesenI get that, but why was it only for testing?  You're trying to resize, why would you force a migration too?20:10
*** wolverineav has quit IRC20:11
*** mikal has quit IRC20:12
dansmithcfriesen: there were some reasons discussed originally when it was added, but it's been a long time..  probably mostly around how the major users of this stuff ran their clouds at the time20:12
dansmithi.e. packing like flavors together20:12
dansmithagain, not saying it makes sense today, just saying why it was done that way20:13
cfriesencurrently I'd expect the scheduler to trigger it to move in that case20:13
*** irclogbot_2 has quit IRC20:13
cfriesenanyways, reported it as a bug: https://bugs.launchpad.net/nova/+bug/181921620:16
openstackLaunchpad bug 1819216 in OpenStack Compute (nova) "in devstack, "nova migrate <uuid>" will try to migrate to the same host (and then fail)" [Undecided,New]20:16
mriedemdid someone say resize to same host problems?! https://review.openstack.org/#/c/638791/20:19
mriedemif you have servers in an affinity group, you can *only* resize to the same ohst20:19
mriedem*host20:19
mriedemcfriesen: there is also a bug for the thing you hit, semi-related anyway20:19
cfriesenmriedem: right, I knew about the affinity group20:20
mriedemthe api says resize to same host, sure, but the compute says 'no way'20:20
*** markvoelker has joined #openstack-nova20:21
mriedemhttps://bugs.launchpad.net/nova/+bug/181123520:22
openstackLaunchpad bug 1811235 in OpenStack Compute (nova) "instance's vm-state becomes error when cold-migrate instance to same host failed" [Medium,In progress] - Assigned to Matt Riedemann (mriedem)20:22
mriedemthe api check is here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L362620:23
*** dcdawg has joined #openstack-nova20:23
mriedemthe compute check is here https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L428720:23
mriedemwhich fails for everything except vmware20:24
mriedemsince vmware is a cluster and you can migrate within the same "host"20:24
*** mvkr has quit IRC20:24
mriedemso allow_resize_to_same_host is true if you're doing a resize, but not a cold migration *unless* you've got vcenter20:25
*** mvkr has joined #openstack-nova20:25
cfriesenarguably the scheduler should only allow cold migration to same host for vcenter20:26
mriedemthe scheduler doesn't know if it's vcenter20:26
cfriesenI guess it's actually the conductor20:26
mriedemor conductor20:26
mriedemthe control plane services are not supposed to know anything about the underlying compute driver20:26
cfriesencan't we look at host capabilities or hypervisor type or something?20:26
dansmithcfriesen: and make assumptions? what about vmware 2 which *can* do that thing.. it's a slippery slope20:27
dansmithseparation of responsibilities20:27
cdentvmware even has a downstream patch which adds config for "always resize to same "host" "20:27
mriedemcdent: that's kind of where https://review.openstack.org/#/c/638791/ is going20:28
cfriesenhow do we handle it for live migration?20:28
mriedemwe always ignore the source host20:28
mriedemsame for evac20:29
cfriesenseems weird to make live migration and cold migration behave differently20:29
* cdent adds to review queue20:29
mriedemso,20:31
mriedemnow that we report compute driver capabilities as traits,20:31
mriedemand supports_migrate_to_same_host is a driver capability,20:31
mriedemwe could technically check the compute node provider traits for the source compute node from the api and determind if it supports cold migrate to the same host and filter the request spec appropriately20:32
dansmithdon't you need the opposite of that?20:32
dansmithyou need to know if it *needs* the same host for the vmware case, not whether not it *can* yeah?20:33
mriedemyeah maybe, you can't cold migrate across clusters20:33
mriedemright?20:33
dansmithyeah20:34
dansmithso you could exclude the source host unless can-haz-must-be-same-host20:34
dansmiththat means looking up traits for the current host and then deciding what to ask the scheduler to do20:35
mriedemyeah that's what i was trying to get to20:35
mriedemfrom the api, if supports_migrate_to_same_host is false, exclude the source host before calling the scheduler20:36
mriedemif supports_migrate_to_same_host is true, then don't exclude the source host20:36
mriedemright?20:36
dansmithyou want libvirt to say it *can't* ?20:36
mriedemit already does20:36
mriedemvmware is the only driver that says supports_migrate_to_same_host=true20:36
dansmithoh okay I'm conflating the migrate and resize to same host cases I guess20:37
dansmithin that case,20:37
dansmiththe flag is really backwards,20:37
mriedemyeah we don't have a driver capability for resize to same host20:37
*** xek has quit IRC20:37
dansmithbecause you don't want to allow the source if it can, you want to limit to the source if it can, else exclude20:37
mriedemyou mean supports_migrate_to_same_host should be "must_migrate_to_same_host"?20:37
mriedemoh20:37
dansmithmriedem: right we have the config flag tho20:37
*** xek has joined #openstack-nova20:38
mriedemyeah again that's kind of where i was going with https://review.openstack.org/#/c/638791/1/nova/conf/workarounds.py20:38
mriedembut it's fugly20:38
mriedemi need to work on an alternative to that which doesn't bypass the scheduler filters20:39
mriedembecause that patch is really wild west at the moment20:39
mriedemit's more like --force to the source host20:39
mriedemwhich is like, -4.520:40
*** artom has joined #openstack-nova20:42
*** phasespace has joined #openstack-nova20:44
mriedemcfriesen: your test vm went to ERROR state right20:47
mriedem?20:47
cfriesenmriedem: yep20:47
mriedemhttps://review.openstack.org/#/q/topic:bug/1811235+(status:open+OR+status:merged) should fix that20:48
*** mikal has joined #openstack-nova20:48
*** IvensZambrano has joined #openstack-nova20:50
*** snevi has quit IRC20:51
*** markvoelker has quit IRC20:53
*** fried_rolls is now known as fried_rice20:56
*** artom has quit IRC20:57
*** slaweq has quit IRC20:57
*** slaweq has joined #openstack-nova20:58
*** mikal has quit IRC20:59
*** erlon_ has joined #openstack-nova21:01
*** agopi|afk is now known as agopi21:03
*** erlon_ has quit IRC21:15
mriedemdansmith: fried_rice: one of you want to hit https://review.openstack.org/#/c/641416/ to cut down the nova-next run time?21:16
mriedemshaves 20-30 minutes21:17
fried_rice...21:17
mriedemjust noticed that the gate blocker fix for evacuate is still sitting on the nova-next job21:17
*** bnemec is now known as beekneemech21:18
fried_ricemriedem: done21:20
fried_ricesuggestion inline about further efficientizing21:20
mriedemyeah i don't know what to do about tempest-slow21:21
mriedemi've been pushing back lately on adding more stuff to tempest, which kind of sucks21:21
*** artom has joined #openstack-nova21:22
*** dcdawg has quit IRC21:23
*** betherly has joined #openstack-nova21:26
*** wolverineav has joined #openstack-nova21:28
*** wolverineav has quit IRC21:30
*** igordc has quit IRC21:31
*** betherly has quit IRC21:31
*** wolverineav has joined #openstack-nova21:32
*** wolverineav has quit IRC21:32
*** wolverineav has joined #openstack-nova21:32
*** luksky has joined #openstack-nova21:35
*** betherly has joined #openstack-nova21:47
*** IvensZambrano has quit IRC21:50
*** markvoelker has joined #openstack-nova21:51
*** awaugama has quit IRC21:52
*** betherly has quit IRC21:52
*** betherly has joined #openstack-nova22:08
*** wolverineav has quit IRC22:08
*** betherly has quit IRC22:13
*** wolverineav has joined #openstack-nova22:18
*** wolverineav has quit IRC22:23
*** markvoelker has quit IRC22:24
*** Swami has joined #openstack-nova22:28
*** wolverineav has joined #openstack-nova22:32
*** betherly has joined #openstack-nova22:50
*** wolverineav has quit IRC22:54
*** betherly has quit IRC22:55
*** waverider has quit IRC22:58
*** irclogbot_2 has joined #openstack-nova23:00
*** wolverineav has joined #openstack-nova23:01
*** BjoernT_ has quit IRC23:14
*** igordc has joined #openstack-nova23:17
*** dcdawg has joined #openstack-nova23:19
*** markvoelker has joined #openstack-nova23:21
*** xek has quit IRC23:22
*** dcdawg has quit IRC23:24
*** hongbin has joined #openstack-nova23:24
*** mmethot has quit IRC23:27
*** betherly has joined #openstack-nova23:32
*** whoami-rajat has quit IRC23:33
*** Cardoe has quit IRC23:33
*** Cardoe has joined #openstack-nova23:34
*** betherly has quit IRC23:36
*** wolverineav has quit IRC23:37
*** wolverineav has joined #openstack-nova23:39
*** wolverineav has quit IRC23:44
*** wolverineav has joined #openstack-nova23:44
*** markvoelker has quit IRC23:54
*** fried_rice is now known as efried23:56
openstackgerritMatt Riedemann proposed openstack/nova master: Add finish_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63508023:57
openstackgerritMatt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask  https://review.openstack.org/63564623:57
openstackgerritMatt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field  https://review.openstack.org/61403523:57
openstackgerritMatt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask  https://review.openstack.org/63566823:57
openstackgerritMatt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize()  https://review.openstack.org/63568423:57
openstackgerritMatt Riedemann proposed openstack/nova master: Filter duplicates from compute API get_migrations_sorted()  https://review.openstack.org/63622423:57
openstackgerritMatt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells  https://review.openstack.org/61403723:57
openstackgerritMatt Riedemann proposed openstack/nova master: Add CrossCellWeigher  https://review.openstack.org/61435323:57
openstackgerritMatt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize  https://review.openstack.org/63625323:57
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source  https://review.openstack.org/63705823:57
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask  https://review.openstack.org/63707023:57
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize conductor RPC method  https://review.openstack.org/63707523:57
openstackgerritMatt Riedemann proposed openstack/nova master: Confirm cross-cell resize from the API  https://review.openstack.org/63731623:57
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add revert_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63763023:57
openstackgerritMatt Riedemann proposed openstack/nova master: Deal with cross-cell resize in _remove_deleted_instances_allocations  https://review.openstack.org/63945323:57
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add finish_revert_snapshot_based_resize_at_source compute method  https://review.openstack.org/63764723:57
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add RevertResizeTask  https://review.openstack.org/63804623:57
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add revert_snapshot_based_resize conductor RPC method  https://review.openstack.org/63804723:57
*** agopi has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!