Monday, 2019-09-02

« Sunday, 2019-09-01 Index Tuesday, 2019-09-03 »
*** rcernin has joined #openstack-nova00:00
openstackgerritArtom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration  https://review.opendev.org/67259500:29
*** brinzhang has joined #openstack-nova00:35
*** spsurya has joined #openstack-nova01:29
*** eandersson has quit IRC01:36
*** larainema has joined #openstack-nova01:38
*** eandersson has joined #openstack-nova01:39
*** tkajinam has quit IRC01:45
*** tkajinam has joined #openstack-nova01:47
*** jhesketh has joined #openstack-nova02:29
openstackgerritChason Chan proposed openstack/nova master: Fix the incorrect powershell command  https://review.opendev.org/67958803:13
*** yaawang has quit IRC03:39
alex_xustephenfin: with the current upgrade plan, does it means we lose the ability LM the instance?03:41
alex_xustephenfin: since the control-plane is asking PCPU, then there is no way to do that. If the opeartor is going to upgrade the OS also, then he need LM the existing instance to other host. But we there is PCPU on old nodes also.03:42
*** yaawang has joined #openstack-nova03:43
alex_xustephenfin: except we required the operator to do a upgrade of openstack first before upgrade OS. But I don't know whether it works always, is there any case the user have to upgrade the OS first before upgrade OpenStack since some dependence only available on the new version of OS03:43
alex_xusean-k-mooney: efried ^03:47
*** mvkr has joined #openstack-nova04:02
*** yaawang has quit IRC04:14
*** yaawang has joined #openstack-nova04:15
*** HagunKim has joined #openstack-nova04:19
HagunKimHello, I have a question for using openstack nova.04:22
HagunKimIn mariadb, there are many vm instances with vm_state=deleted. (more than 4000 over)04:22
HagunKimCan I delete them manually?04:22
alex_xuHagunKim: you should use nova-manage db archive_deleted_rows04:23
*** ircuser-1 has quit IRC04:37
openstackgerritya.wang proposed openstack/nova master: vCPU model selection  https://review.opendev.org/67029804:42
openstackgerritya.wang proposed openstack/nova master: Add compatibility checks for CPU mode and CPU models and extra flags  https://review.opendev.org/67029904:42
openstackgerritya.wang proposed openstack/nova master: Support reporting multi CPU model traits  https://review.opendev.org/67030004:42
*** ricolin has joined #openstack-nova04:47
HagunKimalex_xu: Thanks. I have some duplicated data in shadow tables. So "nova-manage db archive_deleted_rows" occurs an error and not working.04:51
HagunKimCan I delete all data in shadow tables manually?04:51
HagunKimeg. DELETE from shadow_instance_actions_events;04:51
alex_xuHagunKim: you can use nova-manage db purge to purge the shadow table when you think those data is useless for you04:51
HagunKimalex_xu: Because I use ocata version, I think I don't have purge command in nova-manage.04:52
alex_xuHagunKim: ah04:52
alex_xuHagunKim: probably you only can manually delete04:53
alex_xuHagunKim: why there are duplicated data?04:53
HagunKimalex_xu: Well.. I don't know why. But there are datas with same id value.04:56
*** ivve has joined #openstack-nova05:29
*** Luzi has joined #openstack-nova05:37
*** maciejjozefczyk has joined #openstack-nova06:19
*** psachin has joined #openstack-nova06:24
*** maciejjozefczyk has quit IRC06:32
*** maciejjozefczyk has joined #openstack-nova06:32
*** brinzhang has quit IRC06:40
*** brinzhang has joined #openstack-nova06:40
*** aojea has joined #openstack-nova06:54
*** ttsiouts has joined #openstack-nova06:55
*** tesseract has joined #openstack-nova06:56
*** ttsiouts has quit IRC06:57
*** slaweq has joined #openstack-nova06:59
*** trident has quit IRC07:04
*** rcernin has quit IRC07:08
openstackgerritLuyao Zhong proposed openstack/nova master: Add resources dict into _Provider  https://review.opendev.org/67844907:10
openstackgerritLuyao Zhong proposed openstack/nova master: Retrieve the allocations early  https://review.opendev.org/67845007:10
openstackgerritLuyao Zhong proposed openstack/nova master: Claim resources in resource tracker  https://review.opendev.org/67845207:10
openstackgerritLuyao Zhong proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces  https://review.opendev.org/67845307:10
openstackgerritLuyao Zhong proposed openstack/nova master: libvirt: report VPMEM resources by provider tree  https://review.opendev.org/67845407:10
openstackgerritLuyao Zhong proposed openstack/nova master: libvirt: Support VM creation with vpmems and vpmems cleanup  https://review.opendev.org/67845507:10
openstackgerritLuyao Zhong proposed openstack/nova master: Parse vpmem related flavor extra spec  https://review.opendev.org/67845607:10
openstackgerritLuyao Zhong proposed openstack/nova master: Add functional tests for virtual persistent memory  https://review.opendev.org/67847007:10
openstackgerritBrin Zhang proposed openstack/nova-specs master: Support re-configure deleted_on_termination in server  https://review.opendev.org/58033607:11
*** luksky has joined #openstack-nova07:12
*** trident has joined #openstack-nova07:13
*** jangutter has joined #openstack-nova07:21
*** rcernin has joined #openstack-nova07:25
*** ileixe has joined #openstack-nova07:25
ileixeHi nova!07:26
ileixeDoes anybody has an environment which assure CPU QoS within Nova?07:26
ileixeI mean have anybody ever used CPU pinning in production or something else to prevent noisy neighbors?07:27
ileixeI want to hear any experience which will be really appreciated for us07:28
*** ccamacho has joined #openstack-nova07:34
*** ccamacho has quit IRC07:44
*** ccamacho has joined #openstack-nova07:44
*** rcernin has quit IRC07:48
*** pcaruana has joined #openstack-nova07:49
*** ralonsoh has joined #openstack-nova07:51
openstackgerritya.wang proposed openstack/nova master: Support reporting multi CPU model traits  https://review.opendev.org/67030007:58
*** dougsz has joined #openstack-nova08:04
*** mdbooth has joined #openstack-nova08:15
*** arxcruz is now known as arxcruz_pto08:19
openstackgerritMatthew Booth proposed openstack/nova master: libvirt: Fix service-wide pauses caused by un-proxied libvirt calls  https://review.opendev.org/67773608:19
*** tkajinam has quit IRC08:21
*** panda|rover|off is now known as panda|rover08:23
*** dtantsur|afk is now known as dtantsur08:26
openstackgerritBrin Zhang proposed openstack/nova master: Add delete_on_termination to volume-attach API  https://review.opendev.org/67313308:34
*** jaosorior has joined #openstack-nova08:34
*** derekh has joined #openstack-nova08:39
*** ociuhandu has joined #openstack-nova08:47
*** cdent has joined #openstack-nova08:57
*** jistr has quit IRC09:09
*** jistr has joined #openstack-nova09:10
stephenfinalex_xu: Well, you can't migrate pinned instances at the moment anyway, so that's not an issue :)09:15
stephenfinalex_xu: However, assuming artom fixes it, we have that option to disable the translation feature in the scheduler09:16
stephenfinalex_xu: So if the operator sets that, the scheduler will keep requesting VCPUs09:16
*** xek has joined #openstack-nova09:17
alex_xustephenfin: okay, I thought people my blind to live migrate dedicated instance in case they have same hardware :)09:18
alex_xustephenfin: we don't have that option in the current version, right09:18
alex_xustephenfin: probably we can ask people live migration the instance before upgrade if they want09:19
stephenfinNo unless you explicitly set '[workarounds] enable_numa_live_migration'09:19
stephenfin*Not09:19
alex_xuoh, totally forget that option09:19
stephenfinalex_xu: nah, if all they do is upgrade their compute nodes, nothing will change. They'll still be reporting and consuming VCPUs09:20
openstackgerritGorka Eguileor proposed openstack/nova master: Use os-brick locking for volume attach and detach  https://review.opendev.org/61419009:20
stephenfinThings only change when the operator sets cpu_dedicated_set or cpu_shared_set on the controller, and set the as-yet-unnamed option on the controller09:20
stephenfinwhen 'cpu_dedicated_set' is set, the compute node will start reporting PCPUs and transform existing inventory of pinned instances (if any) from VCPU to PCPU09:21
alex_xuright09:21
stephenfinnot before that though09:21
alex_xustephenfin: so we still have that workaround option, in case people need it09:22
stephenfinalex_xu: Yes, see https://review.opendev.org/#/c/671801/32/nova/conf/workarounds.py09:23
stephenfinI'm going to have to rename and move that though, of course09:23
bauzasstephenfin: you got hit by a merge conflict FWIW09:24
bauzashttps://review.opendev.org/#/c/671793/1609:24
stephenfinyeah, I'll tackle that shortly09:24
stephenfinexpecting a lot of those09:24
* bauzas now works back on placement audit09:24
alex_xustephenfin: ok, if the operator wants to upgrade the host os, then he can rolling upgrade the host os and nova-compute, without PCPU translating and cpu_dedicated_set. After all the nodes upgrade the new os and new nova-compute, then update the cpu_dedicated_set, to reshape the inventory, is that correct?09:25
stephenfinyes09:26
alex_xuso cool09:26
alex_xustephenfin: we really document those somewhere, like release note upgrade section?09:26
stephenfinalex_xu: I've started here https://review.opendev.org/#/c/671793/16/releasenotes/notes/cpu-resources-d4e6a0c12681fa87.yaml09:27
stephenfinBut I'll add more as I go09:27
bauzasalex_xu: yeah that's the plan09:27
alex_xustephenfin: cool09:27
bauzasalex_xu: nothing changes unless the operator triggers it09:27
alex_xubauzas: yea, that is much simpler09:28
bauzasbut when they trigger, then it'll be a reshape09:28
*** Tianhao_Hu has joined #openstack-nova09:28
bauzasstephenfin: and that also implies we need to provide some nova-status check command09:28
bauzasfor upgrading to Unicorn09:28
alex_xustephenfin: also something fun https://review.opendev.org/#/c/674895/22/nova/virt/libvirt/driver.py, due to I updated a wrong option09:28
sean-k-mooneyalex_xu: dsid stephenfin answer your os upgrade question09:29
alex_xusean-k-mooney: yes, perfectly answered09:29
*** Tianhao_Hu has left #openstack-nova09:30
gibibauzas, stephenfin: hi! there are +2s from mriedem on the next couple patches in the support-move-ops-with-qos-ports series. starting here https://review.opendev.org/#/c/655110/09:30
bauzasgibi: yeah, I was planning to put my eyes on your series after stephenfin's one09:31
gibibauzas: cool, thanks09:31
yaawangstephenfin: alex_xu Hi, I had update the patches of vCPU models selection, can you take a look if you have time :) https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/cpu-model-selection09:44
alex_xuyaawang: got it, will check it again09:45
yaawangalex_xu: thk :)09:45
alex_xuyaawang: np09:45
*** Tianhao_Hu_ has joined #openstack-nova09:49
*** Tianhao_Hu_ is now known as Tianhao_Hu09:49
*** Tianhao_Hu has left #openstack-nova09:49
openstackgerritsean mooney proposed openstack/nova master: Libvirt: report storage bus traits  https://review.opendev.org/66691410:06
openstackgerritsean mooney proposed openstack/nova master: libvirt: use domain capabilities to get supported device models  https://review.opendev.org/66691510:06
openstackgerritsean mooney proposed openstack/nova master: Add transform_image_metadata request filter  https://review.opendev.org/66577510:06
*** mdbooth_ has joined #openstack-nova10:07
*** Garyx has joined #openstack-nova10:08
*** mdbooth has quit IRC10:10
*** mdbooth_ has quit IRC10:18
*** mdbooth has joined #openstack-nova10:19
*** ociuhandu has quit IRC10:19
openstackgerritBrin Zhang proposed openstack/nova-specs master: Support set port vnic type to create server  https://review.opendev.org/67240010:20
*** mdbooth_ has joined #openstack-nova10:23
*** jaosorior has quit IRC10:24
openstackgerritBrin Zhang proposed openstack/nova-specs master: Support set port vnic type to create server  https://review.opendev.org/67240010:25
*** mdbooth has quit IRC10:27
*** xek has quit IRC10:35
*** ociuhandu has joined #openstack-nova10:44
*** brinzhang_ has joined #openstack-nova10:50
openstackgerritStephen Finucane proposed openstack/nova master: trivial: Rewrap definitions of 'NUMACell'  https://review.opendev.org/67439510:51
openstackgerritStephen Finucane proposed openstack/nova master: libvirt: Simplify 'fakelibvirt.HostInfo' object  https://review.opendev.org/67886110:51
openstackgerritStephen Finucane proposed openstack/nova master: libvirt: Start checking compute usage in functional tests  https://review.opendev.org/67890210:51
openstackgerritStephen Finucane proposed openstack/nova master: objects: Rename 'fields' import to 'obj_fields'  https://review.opendev.org/67410310:51
openstackgerritStephen Finucane proposed openstack/nova master: libvirt: Fold in argument to '_update_provider_tree_for_vgpu'  https://review.opendev.org/67672910:51
openstackgerritStephen Finucane proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement  https://review.opendev.org/67179310:51
openstackgerritStephen Finucane proposed openstack/nova master: libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available'  https://review.opendev.org/67269310:51
openstackgerritStephen Finucane proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs  https://review.opendev.org/67180010:51
openstackgerritStephen Finucane proposed openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait  https://review.opendev.org/67557110:51
openstackgerritStephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta  https://review.opendev.org/67180110:51
openstackgerritStephen Finucane proposed openstack/nova master: Add reshaper for PCPU  https://review.opendev.org/67489510:51
*** brinzhang has quit IRC10:53
openstackgerritChris Dent proposed openstack/nova master: Use microversion in put allocations in test_report_client  https://review.opendev.org/67962210:54
stephenfinalex_xu, bauzas: Still working on changing the scheduler config option and adding resize functional tests for cpu-resources, but I've moved all the trivial patches up to the front of the series, if you fancy taking a look (starts here https://review.opendev.org/#/c/678861/)11:10
stephenfingibi: Yeah, that's on my list for today (t)11:11
stephenfin*(y)11:11
*** dougsz has quit IRC11:11
gibistephenfin: thanks11:11
*** ociuhandu has quit IRC11:22
*** jaosorior has joined #openstack-nova11:22
*** ociuhandu has joined #openstack-nova11:22
*** dougsz has joined #openstack-nova11:30
*** ociuhandu has quit IRC11:43
openstackgerritChris Dent proposed openstack/nova master: Tune up db.instance_get_all_uuids_by_hosts  https://review.opendev.org/67962711:49
openstackgerritBalazs Gibizer proposed openstack/nova master: Nice to have test coverage for If1f465112b8e9b0304b8b5b864b985f72168d839  https://review.opendev.org/67962911:59
*** mdbooth_ has quit IRC12:02
aspiersstephenfin: thanks for the reviews. BTW https://review.opendev.org/#/c/636334/ is missing W+112:07
aspiersstephenfin: I'm working on adding the checks you requested, for live migration / suspend etc.12:08
*** FlorianFa has joined #openstack-nova12:10
*** mdbooth has joined #openstack-nova12:12
*** panda|rover is now known as panda|rover|eat12:13
*** ociuhandu has joined #openstack-nova12:14
*** ociuhandu has quit IRC12:19
*** ociuhandu has joined #openstack-nova12:20
kashyapaspiers: Nice, seems like the SEV work is almost merged.12:21
aspierskashyap: Yes, pretty close now :)12:21
aspiersIt's almost a year since I uploaded the Stein spec :-o12:21
aspiersSo this has been a year-long project or more already12:22
kashyapIt required a lot of 'supporting infra' work, which is also useful for other features (machine types rework; domCapabilities, etc)12:23
aspiersRight12:23
*** priteau has joined #openstack-nova12:23
aspiersI'm glad it turned out to have other uses. Didn't anticipate that originally.12:23
kashyapaspiers: Unrelated - SEV _requires_ 'q35', isn't it?12:24
aspierskashyap: http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2019-07-30.log.html#t2019-07-30T09:21:3512:25
aspiers;-)12:25
*** brinzhang has joined #openstack-nova12:25
kashyapaspiers: Haha, my memory is like the bumblebee's -- can't recall anything more than past 7 seconds12:26
aspiersXD12:26
* kashyap goes to read the spec12:26
aspiersSimilar thing happened to me this morning12:26
kashyapaspiers: I asked because, the next question I was looking for was: is there a check in the code where: "if SEV && !q35 --> throw an error"12:27
sean-k-mooneykashyap: you could only do that check on the compute node currenlty12:27
*** xek has joined #openstack-nova12:28
*** luksky has quit IRC12:28
kashyapsean-k-mooney: Right, it matters only on compute nodes anyway.12:28
sean-k-mooneyi guess you could push it to placemetn by not reporting the sev resouces if the machine type is set to pc12:28
sean-k-mooneykashyap: my point is it woudl cause a late reschdule12:28
kashyapsean-k-mooney: What I was wondering is, _if_ such a thing exists already12:28
*** brinzhang_ has quit IRC12:28
aspiersGuys, no need to speculate ;-)12:28
aspiershttps://review.opendev.org/#/c/644565/49/nova/virt/libvirt/driver.py@510412:29
* kashyap clicks12:29
sean-k-mooney im not sure overriding the machine type is a good idea12:29
aspiersWe could bail there instead12:30
kashyapOh, at _what_ point does the override happens?12:30
aspierskashyap: At *that* point ^^^12:30
sean-k-mooneyits in the driver when its generating the xml12:31
kashyapRight.  Override can cause a potential surprise for the operator, doesn't it?12:31
sean-k-mooneykashyap: yes and maybe live migration issues12:31
aspiersLive migration is unsupported12:31
aspiersSee above message I wrote to stephenfin ^^^12:31
kashyapaspiers: Ah, nod, noticed.12:32
aspiersBut it will probably be supported eventually12:32
sean-k-mooneyalso you can request teh machine type in the image https://github.com/openstack/glance/blob/master/etc/metadefs/compute-libvirt-image.json#L34-L3812:32
aspierssean-k-mooney: Yes, that's why image_meta is a parameter to get_machine_type_no_default() in the code I just linked12:32
kashyapaspiers: Thinking out loud: do we need an additional function for handling SEV-specific quirks for machine types, instead of squeezing it into the generic guest_machine_type()?12:33
aspiershttps://review.opendev.org/#/c/644565/49/nova/virt/libvirt/utils.py@53912:33
sean-k-mooneyyes but overrideing an explcit machine_type in the image is not correct12:33
aspierssean-k-mooney: like I said, we could bail there instead12:33
aspiersthat would probably be fine12:33
sean-k-mooneyi would bail on conflict with the image12:33
aspiersYes that's what I'm saying12:34
sean-k-mooneythat leave the question of what to do if its set to pc in nova.conf12:34
sean-k-mooneyyour could bail then also or override12:34
kashyapaspiers: One more: It seems like if machine type is _not_ set, then we're defaulting to 'q35' -- https://review.opendev.org/#/c/644565/49/nova/virt/libvirt/driver.py@509912:35
sean-k-mooneyi think raising an exception in both case is less surpising12:35
sean-k-mooneybut i am ment ot be getting food so i should go do that12:35
sean-k-mooneybrb12:35
*** xek has quit IRC12:35
aspierskashyap: Yes of course, otherwise SEV won't work without explicit hw_machine_type every time12:35
kashyapaspiers: Okay, the function name confused me, then12:36
aspiersWhich name?12:36
*** panda|rover|eat is now known as panda|rover12:36
kashyapThe _guest_machine_type() --> should this be named more SEV-specific12:36
kashyapSomething like ... _guest_machine_type_for_sev()?12:37
aspiersYes sure I could extract the big chunk out into _guest_machine_type_for_sev()12:37
aspiersthen _guest_machine_type() would call that12:37
aspiersanother non-inlined function to annoy stephenfin ;-)12:37
kashyapaspiers: Yeah, thank you.  So in _guest_machine_type() -- we can call the _for_sev() based on some SEV-specific conditional12:38
aspiersIt would use the exact same conditional which is already there12:38
aspierson line 509012:38
kashyapRight, sev_enabled12:38
aspiersand X86_6412:38
aspiersWe're not supporting SEV on other architectures ;-)12:38
kashyapYep, agreed12:38
kashyapHehe12:39
aspiersOK I'm going for a run to preserve sanity12:39
aspiersbiab12:39
kashyap+112:40
kashyap(On a run)12:40
*** tkajinam has joined #openstack-nova12:45
openstackgerritMatthew Booth proposed openstack/nova master: libvirt: Fix service-wide pauses caused by un-proxied libvirt calls  https://review.opendev.org/67773612:46
openstackgerritMerged openstack/nova master: Add extra spec parameter and image property for memory encryption  https://review.opendev.org/66442012:47
openstackgerritMerged openstack/nova master: trivial: Rewrap definitions of 'NUMACell'  https://review.opendev.org/67439512:47
openstackgerritMerged openstack/os-vif master: Fix code bug in document  https://review.opendev.org/67938212:52
kashyapaspiers: When you're back, on a similar vein, I also to introduce such a method for Secure Boot: _guest_machine_type_for_secure_boot()12:58
*** tkajinam has quit IRC12:58
*** tkajinam has joined #openstack-nova12:59
openstackgerritLuyao Zhong proposed openstack/nova master: db: Add resources column in instance_extra table  https://review.opendev.org/67844713:00
openstackgerritLuyao Zhong proposed openstack/nova master: object: Introduce Resource and ResouceList objs  https://review.opendev.org/67844813:00
openstackgerritLuyao Zhong proposed openstack/nova master: Add resources dict into _Provider  https://review.opendev.org/67844913:00
openstackgerritLuyao Zhong proposed openstack/nova master: Retrieve the allocations early  https://review.opendev.org/67845013:00
openstackgerritLuyao Zhong proposed openstack/nova master: Claim resources in resource tracker  https://review.opendev.org/67845213:00
openstackgerritLuyao Zhong proposed openstack/nova master: libvirt: Enable driver discovering PMEM namespaces  https://review.opendev.org/67845313:00
openstackgerritLuyao Zhong proposed openstack/nova master: libvirt: report VPMEM resources by provider tree  https://review.opendev.org/67845413:00
openstackgerritLuyao Zhong proposed openstack/nova master: libvirt: Support VM creation with vpmems and vpmems cleanup  https://review.opendev.org/67845513:00
openstackgerritLuyao Zhong proposed openstack/nova master: Parse vpmem related flavor extra spec  https://review.opendev.org/67845613:00
openstackgerritLuyao Zhong proposed openstack/nova master: Add functional tests for virtual persistent memory  https://review.opendev.org/67847013:00
openstackgerritLuyao Zhong proposed openstack/nova master: libvirt: Enable driver configuring PMEM namespaces  https://review.opendev.org/67964013:00
*** ociuhandu has quit IRC13:06
*** tkajinam has quit IRC13:06
*** mmethot has joined #openstack-nova13:08
*** ociuhandu has joined #openstack-nova13:10
*** ociuhandu has quit IRC13:10
*** ociuhandu has joined #openstack-nova13:10
*** luksky has joined #openstack-nova13:13
*** gbarros has joined #openstack-nova13:25
openstackgerritChris Dent proposed openstack/nova master: single pass instance info fetch in host manager  https://review.opendev.org/62355813:26
*** Luzi has quit IRC13:32
cdentefried, mriedem: I've still not managed to unit test that ^ but I did do some tweaks to the loop so that it yields more simply, as efried suggested. As I said on the patch, if someone else wants to poke at it that would be welcome, I've got other things leaking into my priorities :(13:32
openstackgerritChris Dent proposed openstack/nova master: Tune up db.instance_get_all_uuids_by_hosts  https://review.opendev.org/67962713:35
openstackgerritChris Dent proposed openstack/nova master: single pass instance info fetch in host manager  https://review.opendev.org/62355813:35
*** xek has joined #openstack-nova13:42
openstackgerritBalazs Gibizer proposed openstack/nova master: Refactor MigrationTask._execute  https://review.opendev.org/67965113:52
openstackgerritBalazs Gibizer proposed openstack/nova master: unit test: do not fill rp mapping for failed re-schedule  https://review.opendev.org/67965313:55
*** gbarros has quit IRC13:55
bauzasgibi: to make it clear, I'm about to +W https://review.opendev.org/#/c/655110/ but do you want me to hold until we fully review the series ?13:55
bauzasgibi: I think we can safely merge the above given we don't call it yet13:56
*** larainema has quit IRC13:56
gibibauzas:I have no objection against merging that :)13:56
bauzasheh ok13:56
gibibauzas: and thank you13:57
luyaodansmith: Are you around?13:57
bauzasgibi: well, this one is easy13:57
bauzasall the credits go to you13:57
*** ricolin_ has joined #openstack-nova13:59
*** ricolin has quit IRC14:02
*** KeithMnemonic has joined #openstack-nova14:03
bauzasluyao: today's bank holiday for US folks https://en.wikipedia.org/wiki/Labor_Day14:04
openstackgerritsean mooney proposed openstack/nova master: [WIP] multi numa nfv testing job  https://review.opendev.org/67965614:05
luyaobauzas:   ok, thanks14:07
openstackgerritsean mooney proposed openstack/nova master: [WIP] multi numa nfv testing job  https://review.opendev.org/67965614:08
sean-k-mooneydonnyd: not sure if your around today but ^ should run against the FN multi numa lable and do some basic testing14:10
donnydOh nice14:11
donnydI will be in and out14:11
sean-k-mooneyits basicaly my fedora job but swaped to use the new lable with some minor tweaks.14:11
sean-k-mooneydonnyd: no worries its a holiday in the us right. im going to spend some time today tweaking the job and hopefully it will be working when people get back tomorow14:12
donnydPlease LMK if something isn't working right on my end14:14
sean-k-mooneyit seams to be workign fine so far http://zuul.openstack.org/stream/4778ebe053804dbb976917fd40c1a860?logfile=console.log14:15
*** boxiang has joined #openstack-nova14:15
*** mkrai has joined #openstack-nova14:19
*** mkrai has quit IRC14:20
*** mdbooth has quit IRC14:20
*** ricolin_ is now known as ricolin14:22
*** mdbooth has joined #openstack-nova14:30
*** mdbooth has quit IRC14:32
*** mdbooth_ has joined #openstack-nova14:33
*** aojea has quit IRC14:33
*** aojea has joined #openstack-nova14:34
*** mkrai has joined #openstack-nova14:36
*** xXraphXx has joined #openstack-nova14:37
*** mdbooth_ has quit IRC14:42
*** ociuhandu has quit IRC14:45
*** mdbooth has joined #openstack-nova14:49
*** jangutter has quit IRC14:51
*** xek has quit IRC14:52
sean-k-mooneydonnyd: so in the plus side some of the test that spawn vms are passing. on the minus side i for got to turn on the live migration tests14:59
*** zhubx has joined #openstack-nova15:01
donnydDo you think the extra memory helps?15:03
*** psachin has quit IRC15:03
*** Sundar has joined #openstack-nova15:05
*** boxiang has quit IRC15:05
sean-k-mooneyam it does to a degree in that i can more easily allocate memory to hugepages15:05
sean-k-mooneybut i could make it work with 8G15:05
sean-k-mooneydonnyd: looks like the job is almost done15:06
*** zhubx has quit IRC15:06
*** zhubx has joined #openstack-nova15:06
*** zhubx has quit IRC15:07
*** priteau has quit IRC15:09
sean-k-mooneydonnyd: uploading logs to swift takes a while15:12
sean-k-mooneydonnyd: https://d14c0e495b00eea27855-0acfe5e696fda6aec28b6c718c3c4841.ssl.cf2.rackcdn.com/679656/2/check/tempest-nfv-multi-numa-multinode/4778ebe/testr_results.html.gz :) thats not bad for a first try.15:15
aspierskashyap: what exception should be raised?15:16
donnydThat's not too bad at all15:16
*** Tianhao_Hu has joined #openstack-nova15:16
*** Tianhao_Hu has left #openstack-nova15:16
*** jangutter has joined #openstack-nova15:16
*** trident has quit IRC15:17
sean-k-mooneyi have a few bugs in my job definition mainly stuff i remvoed that i thought was not needed but is15:17
sean-k-mooneyonce i fix that i think everythin should pass15:17
kashyapaspiers: In a call.  But how about an error on "Invalid MachineType" or some such?15:18
* kashyap bbiab15:18
*** ociuhandu has joined #openstack-nova15:18
sean-k-mooneythe contoler node was deployed the way i wanted with kvm but the compute node is missing some of the setting15:18
aspierskashyap: you mean a new exception class?15:18
aspiersInvalidMachineType?15:19
kashyapaspiers: I just used CamelCase as an example.  But but a simple error should do, no?15:19
aspierskashyap: I need to decide on an exception class15:19
aspiersI can't just raise a string15:19
aspiersTypeError: exceptions must derive from BaseException15:20
kashyapaspiers: Right, InvalidMachineType sounds reasonable, then?15:20
aspiersso I'm not sure what you mean by "simple error"15:20
aspiersYeah I could add that15:20
kashyapForgive; split attention15:20
kashyapaspiers: Mind alerting me when you post that?15:22
*** ociuhandu has quit IRC15:23
*** ociuhandu has joined #openstack-nova15:23
*** trident has joined #openstack-nova15:24
*** mkrai has quit IRC15:30
aspierssure15:33
openstackgerritsean mooney proposed openstack/nova master: [WIP] multi numa nfv testing job  https://review.opendev.org/67965615:35
sean-k-mooneynow with 100% more live migration tests ^15:38
sean-k-mooneystill no cinder/swift ro concurancy however.@chat:button115:38
*** mgoddard has quit IRC15:41
*** mgoddard has joined #openstack-nova15:43
openstackgerritBalazs Gibizer proposed openstack/nova master: prepare func test env for moving servers with bandwidth  https://review.opendev.org/65510915:44
openstackgerritBalazs Gibizer proposed openstack/nova master: Func test for migrate server with ports having resource request  https://review.opendev.org/65511315:44
sean-k-mooneybauzas: care to +w on https://review.opendev.org/#/c/636334/49 it has 3 +2s and the follow up patch also has +2+w15:45
*** dtantsur is now known as dtantsur|afk15:45
*** luksky has quit IRC15:45
sean-k-mooneymy device model series that is in the runway is conflciting with the sev series so the sooner we merge one the better15:46
stephenfinffs, I need to rebase again15:49
*** ociuhandu has quit IRC15:49
stephenfingibi: Think you could send these trivial fixes on their way before I do? https://review.opendev.org/#/c/678861/ https://review.opendev.org/#/c/678902/ https://review.opendev.org/#/c/678902/15:50
stephenfinActually, bauzas, think you could hit https://review.opendev.org/#/c/676729/ https://review.opendev.org/#/c/674103/ ?15:51
stephenfinMore trivial patches that I don't want to have to rebase15:51
*** zbr is now known as zbr|ooo15:52
openstackgerritBalazs Gibizer proposed openstack/nova master: Make _rever_allocation nested allocation aware  https://review.opendev.org/67613815:53
openstackgerritBalazs Gibizer proposed openstack/nova master: Support reverting migration / resize with bandwidth  https://review.opendev.org/67614015:55
*** aojea has quit IRC15:56
bauzassean-k-mooney: on it15:56
bauzasstephenfin: ack15:56
stephenfinbauzas: Do mine first. I'll look at sean-k-mooney's ;-)15:56
stephenfinsean-k-mooney: '[scheduler] enable_legacy_pinning_policy_translation' (default = False)15:57
stephenfinHappy with that?15:57
*** tbachman has quit IRC15:57
bauzasstephenfin: lol, ok15:57
sean-k-mooneystephenfin: i would drop legacy15:57
stephenfinGood idea15:57
sean-k-mooneygiven by defualt we want to translate in the future15:58
sean-k-mooneybut yes15:58
stephenfinthat's a hangover from when we planned to kill these15:58
stephenfinkill hw:cpu_policy et al, that is15:58
openstackgerritBalazs Gibizer proposed openstack/nova master: Func test for migrate re-schedule with bandwidth  https://review.opendev.org/67697215:58
sean-k-mooneyya15:58
openstackgerritsean mooney proposed openstack/nova master: [WIP] multi numa nfv testing job  https://review.opendev.org/67965615:59
*** tbachman has joined #openstack-nova16:01
openstackgerritBalazs Gibizer proposed openstack/nova master: Support migrating SRIOV port with bandwidth  https://review.opendev.org/67698016:01
*** xek has joined #openstack-nova16:02
openstackgerritBalazs Gibizer proposed openstack/nova master: Allow migrating server with port resource request  https://review.opendev.org/67149716:04
openstackgerritBalazs Gibizer proposed openstack/nova master: Do not query allocations twice in finish_revert_resize  https://review.opendev.org/67882716:07
openstackgerritBalazs Gibizer proposed openstack/nova master: Allow resizing server with port resource request  https://review.opendev.org/67901916:07
*** ralonsoh has quit IRC16:10
*** ralonsoh has joined #openstack-nova16:20
*** maciejjozefczyk has quit IRC16:26
*** ociuhandu has joined #openstack-nova16:27
*** mmethot has quit IRC16:29
*** luksky has joined #openstack-nova16:29
*** ociuhandu has quit IRC16:31
openstackgerritStephen Finucane proposed openstack/nova master: Remove dead code  https://review.opendev.org/67966816:32
openstackgerritMerged openstack/nova master: allow getting resource request of every bound ports of an instance  https://review.opendev.org/65511016:35
*** N3l1x has joined #openstack-nova16:36
*** spsurya has quit IRC16:44
* bauzas calls it a day16:45
*** derekh has quit IRC17:00
* sean-k-mooney biab go to get food to cook dinner17:01
*** ociuhandu has joined #openstack-nova17:04
*** ociuhandu has quit IRC17:09
*** KeithMnemonic1 has joined #openstack-nova17:27
*** KeithMnemonic1 has quit IRC17:27
*** tesseract has quit IRC17:30
*** KeithMnemonic has quit IRC17:31
*** mgoddard has quit IRC17:32
openstackgerritDustin Cowles proposed openstack/nova master: Provider Config File: YAML file loading and schema validation  https://review.opendev.org/67334117:35
openstackgerritDustin Cowles proposed openstack/nova master: Provider Config File: Public method to retrieve custom resources  https://review.opendev.org/67602917:35
openstackgerritDustin Cowles proposed openstack/nova master: Provider Config File: Update provider tree with new custom resources  https://review.opendev.org/67652217:35
*** mgoddard has joined #openstack-nova17:35
*** ralonsoh has quit IRC17:36
*** mgoddard has quit IRC17:40
*** zbr|ooo is now known as zbr17:53
*** mgoddard has joined #openstack-nova17:53
*** mvkr has quit IRC17:55
openstackgerritsean mooney proposed openstack/nova master: [WIP] multi numa nfv testing job  https://review.opendev.org/67965618:02
openstackgerritMerged openstack/nova master: Pass network API to the conducor's MigrationTask  https://review.opendev.org/65511118:07
*** dougsz has quit IRC18:11
*** Sundar has quit IRC18:14
*** pcaruana has quit IRC18:19
*** pcaruana has joined #openstack-nova18:19
openstackgerritsean mooney proposed openstack/nova master: [WIP] multi numa nfv testing job  https://review.opendev.org/67965618:28
sean-k-mooneydonnyd: i seam to be getting node error. i notice that several of the ubuntu-bionic-expanded instance are deleting for quite a while https://zuul.openstack.org/nodes19:08
sean-k-mooneydonnyd: actully in general it looks lik fortnebula-regionone has more deleting vms in general then other providers19:10
sean-k-mooneydonnyd: looks like its fixing itself19:14
donnydWhenever new images are loaded it takes a little while to get it together19:16
*** ircuser-1 has joined #openstack-nova19:16
sean-k-mooneyah ok i assume zuul builder just uploaded images then19:17
donnydI have been trying to sort out max disk ops looking for the most reliable value for my setup19:17
sean-k-mooneyit all looks normal in anycase now19:17
sean-k-mooneyare you using local storage or ceph?19:18
donnydLocal19:18
donnydMy disks are all nvme, so ceph is too expensive19:18
sean-k-mooneywith qcow219:18
donnydYea19:18
sean-k-mooneythe sparse images should eb fairly cheap to create form the image cache then19:19
sean-k-mooneyassuming you fixed you issue where it kept gettin downloaded19:19
donnydYes they are19:19
donnydBut I am yet to get the glance issue sorted19:20
*** cdent has quit IRC19:20
*** logan- has quit IRC19:59
openstackgerritMerged openstack/nova master: Add request_spec to server move RPC calls  https://review.opendev.org/65572120:02
*** logan- has joined #openstack-nova20:03
sean-k-mooneywell thats new "Live Migration failure: nodeset attribute of hugepages of sizes 2048 and 2048 intersect: libvirt.libvirtError: nodeset attribute of hugepages of sizes 2048 and 2048 intersect"20:14
*** pcaruana has quit IRC20:36
sean-k-mooneydonnyd: i think the new job found its first bug :) im going to test manually to confirm20:41
sean-k-mooneyin my previous testing of the numa migration code i noticed that it did not seam to hanel explciti pages correctly20:42
sean-k-mooneyand the live migration test seam to be failing because of that20:42
*** dougsz has joined #openstack-nova20:48
*** dougsz has quit IRC20:53
donnydHow is that manual test coming20:53
sean-k-mooneystill stacking20:53
sean-k-mooneyalmost done however20:53
sean-k-mooneyit takes about 20 mins to stack on my old serivce that i use for sriov. well for the contoler20:54
sean-k-mooneyssd connected to sata2 are still faster then hardrives but alot slower then nvme20:55
sean-k-mooneyslightly different error because im testing it slighly differently but yes21:02
sean-k-mooneydonnyd: hugepage live migration is definetly broken so the test failures are valid21:02
sean-k-mooneylet me see if i can repodcue the exact error21:02
sean-k-mooneydonnyd: yep i can repoduce the error in the ci locally21:08
sean-k-mooneydonnyd: so there are at least 2 bugs with the proposed live migration code21:08
sean-k-mooneyboth related to how hugepages are calulated on live migration21:08
donnydWell at least its repeatable21:10
sean-k-mooneyyep. i think the job is working properly as is FN21:10
sean-k-mooneyi think artom need to do some more testing with hugepages and i need to review his code to see what he is currently doing21:10
sean-k-mooneydonnyd: thanks again for helping us test this.21:12
donnydNP, I'm glad I can help21:13
openstackgerritArtom Lifshitz proposed openstack/nova master: Introduce live_migration_claim()  https://review.opendev.org/63566921:17
openstackgerritArtom Lifshitz proposed openstack/nova master: New objects for NUMA live migration  https://review.opendev.org/63482721:17
openstackgerritArtom Lifshitz proposed openstack/nova master: LM: Use Claims to update numa-related XML on the source  https://review.opendev.org/63522921:17
openstackgerritArtom Lifshitz proposed openstack/nova master: NUMA live migration support  https://review.opendev.org/63460621:17
openstackgerritArtom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration  https://review.opendev.org/64002121:17
openstackgerritArtom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration  https://review.opendev.org/67259521:17
openstackgerritArtom Lifshitz proposed openstack/nova master: DNM: extra logging  https://review.opendev.org/67968121:17
sean-k-mooneydonnyd: for someone who is ment to be off today he is pushing alot of relvent changes like ^21:19
sean-k-mooneyalmost as if he know the ci job failed or found a bug. because that is exactlyu the logging that is missing to figure out why it failed21:19
openstackgerritArtom Lifshitz proposed openstack/nova master: DNM: extra logging  https://review.opendev.org/67968121:19
openstackgerritArtom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration  https://review.opendev.org/67259521:19
*** gbarros has joined #openstack-nova21:24
*** tobberydberg has quit IRC21:30
*** tridde has joined #openstack-nova21:31
openstackgerritDustin Cowles proposed openstack/nova master: Use SDK for validating instance and node  https://review.opendev.org/65602821:31
openstackgerritDustin Cowles proposed openstack/nova master: Use SDK for setting instance id  https://review.opendev.org/65969021:31
openstackgerritDustin Cowles proposed openstack/nova master: Use SDK for add/remove instance info from node  https://review.opendev.org/65969121:31
openstackgerritDustin Cowles proposed openstack/nova master: Use SDK for getting network metadata from node  https://review.opendev.org/67021321:31
*** tobberydberg has joined #openstack-nova21:31
*** trident has quit IRC21:32
*** slaweq has quit IRC21:45
openstackgerritArtom Lifshitz proposed openstack/nova master: NUMA live migration support  https://review.opendev.org/63460621:54
openstackgerritArtom Lifshitz proposed openstack/nova master: Deprecate CONF.workarounds.enable_numa_live_migration  https://review.opendev.org/64002121:54
openstackgerritArtom Lifshitz proposed openstack/nova master: DNM: extra logging  https://review.opendev.org/67968121:54
openstackgerritArtom Lifshitz proposed openstack/nova master: Functional tests for NUMA live migration  https://review.opendev.org/67259521:54
*** ivve has quit IRC22:06
*** ivve has joined #openstack-nova22:07
*** ivve has quit IRC22:17
*** luksky has quit IRC22:48
*** rcernin has joined #openstack-nova22:54
*** tkajinam has joined #openstack-nova22:56
*** owalsh has quit IRC23:43
*** owalsh has joined #openstack-nova23:51
openstackgerritMerged openstack/nova master: Extract SEV-specific bits on host detection  https://review.opendev.org/63633423:57
« Sunday, 2019-09-01 Index Tuesday, 2019-09-03 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!