Tuesday, 2019-02-19

*** wolverineav has joined #openstack-nova00:22
*** wolverineav has quit IRC00:26
*** sdake has joined #openstack-nova00:26
*** tetsuro has joined #openstack-nova00:34
*** lbragstad has quit IRC00:39
*** slaweq has quit IRC00:41
*** erlon has joined #openstack-nova00:45
*** sdake has quit IRC00:58
*** wolverineav has joined #openstack-nova00:59
*** ileixe has joined #openstack-nova01:03
*** wolverineav has quit IRC01:04
*** markvoelker has joined #openstack-nova01:10
*** sdake has joined #openstack-nova01:14
openstackgerritZhenyu Zheng proposed openstack/nova master: Bump compute service to indicate attach/detach root volume is supported  https://review.openstack.org/61475001:16
*** sdake has quit IRC01:16
*** tiendc has joined #openstack-nova01:19
*** whoami-rajat has joined #openstack-nova01:22
*** sdake has joined #openstack-nova01:22
*** liumk2233 has joined #openstack-nova01:31
*** lbragstad has joined #openstack-nova01:32
*** hongbin has joined #openstack-nova01:33
*** erlon has quit IRC01:38
openstackgerritMerged openstack/nova stable/queens: Lock detach_volume  https://review.openstack.org/63753501:39
*** sdake has quit IRC01:42
*** markvoelker has quit IRC01:44
openstackgerritAdam Spiers proposed openstack/nova master: Fix fake DELETE in PlacementFixture  https://review.openstack.org/63765701:44
*** sdake has joined #openstack-nova01:45
*** wolverineav has joined #openstack-nova01:53
*** sdake has quit IRC01:54
*** sdake has joined #openstack-nova01:59
*** Dinesh_Bhor has joined #openstack-nova02:10
*** sapd1_ has quit IRC02:13
openstackgerritmelanie witt proposed openstack/nova master: Add online data migration for populating user_id  https://review.openstack.org/63335102:16
*** wolverineav has quit IRC02:20
*** wolverineav has joined #openstack-nova02:20
*** sdake has quit IRC02:22
*** takashin has joined #openstack-nova02:26
*** wolverineav has quit IRC02:30
openstackgerritYongli He proposed openstack/nova master: Adds the server group info into show server detail API.  https://review.openstack.org/62147402:32
*** _alastor_ has joined #openstack-nova02:33
*** awalende has joined #openstack-nova02:35
*** _alastor_ has quit IRC02:38
*** sapd1 has joined #openstack-nova02:38
*** awalende has quit IRC02:39
*** markvoelker has joined #openstack-nova02:41
*** wolverineav has joined #openstack-nova02:47
*** bhagyashris has joined #openstack-nova02:50
*** wolverineav has quit IRC02:51
*** psachin has joined #openstack-nova02:59
*** wolverineav has joined #openstack-nova03:00
*** Dinesh_Bhor has quit IRC03:03
*** Dinesh_Bhor has joined #openstack-nova03:06
*** markvoelker has quit IRC03:13
*** dklyle has joined #openstack-nova03:23
*** yaawang has joined #openstack-nova03:26
*** gbarros has quit IRC03:28
*** mgariepy has quit IRC03:33
*** wolverineav has quit IRC03:34
*** sapd1 has quit IRC03:34
*** _fragatina has quit IRC03:37
openstackgerritZhenyu Zheng proposed openstack/nova master: Bump compute service to indicate attach/detach root volume is supported  https://review.openstack.org/61475003:44
*** dave-mccowan has joined #openstack-nova03:45
*** psachin has quit IRC03:50
*** sapd1 has joined #openstack-nova03:51
*** wolverineav has joined #openstack-nova03:54
*** udesale has joined #openstack-nova03:54
*** wolverineav has quit IRC03:54
*** psachin has joined #openstack-nova04:00
*** Dinesh_Bhor has quit IRC04:03
*** agopi has quit IRC04:06
*** mgariepy has joined #openstack-nova04:06
*** agopi has joined #openstack-nova04:07
*** andreaf has quit IRC04:08
*** andreaf has joined #openstack-nova04:09
*** belmoreira has quit IRC04:10
*** hamzy has quit IRC04:13
*** jlvillal has quit IRC04:14
*** janki has joined #openstack-nova04:14
*** jlvillal has joined #openstack-nova04:14
*** jamesdenton has quit IRC04:16
*** jamesdenton has joined #openstack-nova04:16
*** ade_lee_ has left #openstack-nova04:20
*** wolverineav has joined #openstack-nova04:25
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Fix output of interface-attach command  https://review.openstack.org/63767704:25
alex_xumelwitt: I probably didn't get what your mean about the unrelated between instance.project_id as target and removing project_only=True04:28
alex_xumelwitt: I'm thinking make the instance.project_id as target should be the first step, then we can remove the project_only=True. so...that I'm thinking they are related04:29
*** wolverineav has quit IRC04:29
*** cfriesen has joined #openstack-nova04:30
*** _alastor_ has joined #openstack-nova04:35
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix resetting non-persistent fields when saving obj  https://review.openstack.org/63627104:37
*** _alastor_ has quit IRC04:39
*** BlackDex has quit IRC04:43
*** zzzeek has quit IRC04:49
*** _fragatina has joined #openstack-nova04:49
*** _fragatina has quit IRC04:49
*** _fragatina has joined #openstack-nova04:50
*** zzzeek has joined #openstack-nova04:52
*** irclogbot_2 has quit IRC04:55
*** BlackDex has joined #openstack-nova04:56
*** medberry has quit IRC05:07
*** ratailor has joined #openstack-nova05:09
*** pbing19 has joined #openstack-nova05:10
*** dave-mccowan has quit IRC05:13
*** Dinesh_Bhor has joined #openstack-nova05:21
*** pbing19 has quit IRC05:22
*** pbing19 has joined #openstack-nova05:22
*** liumk2233 has quit IRC05:23
*** liumk2233 has joined #openstack-nova05:24
*** wolverineav has joined #openstack-nova05:27
*** pbing19 has quit IRC05:29
*** mgariepy has quit IRC05:29
*** mgariepy has joined #openstack-nova05:30
*** liumk2233 has quit IRC05:31
*** liumk2233 has joined #openstack-nova05:32
*** wolverineav has quit IRC05:32
*** udesale has quit IRC05:33
*** _fragatina has quit IRC05:36
*** hongbin has quit IRC05:41
*** markvoelker has joined #openstack-nova05:41
*** udesale has joined #openstack-nova05:43
*** itlinux has quit IRC05:45
*** moshele has joined #openstack-nova05:45
*** ralonsoh has joined #openstack-nova05:48
*** moshele has quit IRC05:50
*** wolverineav has joined #openstack-nova05:52
*** lbragstad has quit IRC05:53
*** sridharg has joined #openstack-nova05:53
*** wolverineav has quit IRC05:59
*** wolverineav has joined #openstack-nova06:01
*** wolverineav has quit IRC06:06
*** liumk_ has joined #openstack-nova06:07
*** liumk2233 has quit IRC06:09
*** liumk2233 has joined #openstack-nova06:11
*** itlinux has joined #openstack-nova06:11
*** liumk_ has quit IRC06:13
*** markvoelker has quit IRC06:14
*** itlinux has quit IRC06:16
*** liumk_ has joined #openstack-nova06:19
*** liumk2233 has quit IRC06:22
*** jackyzhu has joined #openstack-nova06:35
*** ccamacho has quit IRC06:35
*** jackyzhu has quit IRC06:37
openstackgerritYongli He proposed openstack/nova master: Adds the server group info into show server detail API.  https://review.openstack.org/62147406:37
openstackgerritYongli He proposed openstack/nova master: Add server subresouce toplogy API  https://review.openstack.org/62147606:40
*** wolverineav has joined #openstack-nova06:42
*** wolverineav has quit IRC06:47
*** Luzi has joined #openstack-nova06:48
openstackgerritZhenyu Zheng proposed openstack/nova master: Bump compute service to indicate attach/detach root volume is supported  https://review.openstack.org/61475006:49
bauzasgood morning Nova06:49
*** zhubx has joined #openstack-nova06:52
openstackgerritYongli He proposed openstack/nova master: Add server subresouce toplogy API  https://review.openstack.org/62147606:54
*** wolverineav has joined #openstack-nova06:56
*** pbing19 has joined #openstack-nova06:56
openstackgerritYongli He proposed openstack/nova master: Add server subresouce toplogy API  https://review.openstack.org/62147606:59
*** slaweq has joined #openstack-nova07:11
*** markvoelker has joined #openstack-nova07:11
*** wolverineav has quit IRC07:12
*** itlinux has joined #openstack-nova07:13
*** belmoreira has joined #openstack-nova07:16
*** itlinux has quit IRC07:17
*** yaawang has quit IRC07:19
*** rcernin has quit IRC07:25
*** yaawang has joined #openstack-nova07:27
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix cleaning up console tokens  https://review.openstack.org/63771607:38
*** takamatsu has joined #openstack-nova07:40
*** ociuhandu has joined #openstack-nova07:40
*** markvoelker has quit IRC07:43
*** ociuhandu has quit IRC07:47
*** ccamacho has joined #openstack-nova07:47
*** maciejjozefczyk has joined #openstack-nova07:48
*** pbing19 has quit IRC07:54
*** ileixe has quit IRC07:55
*** yan0s has joined #openstack-nova07:56
*** rpittau has joined #openstack-nova07:57
*** udesale has quit IRC07:58
*** takashin has left #openstack-nova08:01
*** udesale has joined #openstack-nova08:03
*** wolverineav has joined #openstack-nova08:06
*** udesale has quit IRC08:07
*** udesale has joined #openstack-nova08:08
*** pcaruana has joined #openstack-nova08:11
*** tkajinam has quit IRC08:12
*** tesseract has joined #openstack-nova08:13
*** helenafm has joined #openstack-nova08:19
*** itlinux has joined #openstack-nova08:22
*** itlinux has quit IRC08:22
*** sdake has joined #openstack-nova08:28
*** tssurya has joined #openstack-nova08:31
*** evrardjp_ is now known as evrardjp08:31
*** liuyulong_ has joined #openstack-nova08:33
*** ratailor has quit IRC08:35
*** _alastor_ has joined #openstack-nova08:36
*** sdake has quit IRC08:37
*** wolverineav has quit IRC08:39
*** _alastor_ has quit IRC08:40
*** markvoelker has joined #openstack-nova08:40
*** itlinux has joined #openstack-nova08:41
*** itlinux has quit IRC08:42
*** sdake has joined #openstack-nova08:45
*** liuyulong_ has quit IRC08:54
*** ttsiouts has joined #openstack-nova08:57
*** priteau has joined #openstack-nova08:59
*** ociuhandu has joined #openstack-nova09:02
*** itlinux has joined #openstack-nova09:03
*** itlinux has quit IRC09:03
openstackgerritYongli He proposed openstack/nova master: Add server subresouce toplogy API  https://review.openstack.org/62147609:04
*** liumk_ has quit IRC09:05
*** liumk_ has joined #openstack-nova09:06
*** ttsiouts has quit IRC09:08
*** pbing19 has joined #openstack-nova09:08
*** ttsiouts has joined #openstack-nova09:08
*** sapd1 has quit IRC09:09
*** panda|off is now known as panda09:11
kashyapThis needs a final +W, if anyone wants to put this through: https://review.openstack.org/#/c/632507/ (libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Stein")09:12
*** ttsiouts has quit IRC09:13
*** markvoelker has quit IRC09:14
gibikashyap: I'm here but I cannot help :)09:14
kashyapHeh, I know.09:14
kashyapThat's why I didn't ping anyone explicitly :-)09:14
gibi:)09:15
kashyapgibi: I _think_ this is related to my patch, isn't it: http://logs.openstack.org/78/637578/1/check/nova-tox-functional-py35/14da3dc/testr_results.html.gz09:16
kashyapComing from https://review.openstack.org/#/c/637578 ("libvirt: Omit needless check on 'CONF.serial_console'")09:16
* kashyap checks carefully09:16
gibikashyap: fells like there is a connection. Is it always fails for your patch?09:18
kashyapDon't know yet09:18
* gibi pulling the patch down09:18
*** sapd1 has joined #openstack-nova09:22
*** ociuhandu has quit IRC09:23
kashyapIt is this test 'test_volume_swap_server_with_error'  from nova/tests/functional/notification_sample_tests/test_instance.py09:23
*** sdake has quit IRC09:25
*** ttsiouts has joined #openstack-nova09:25
kashyapgibi: But the "IndexError: list index out of range09:26
kashyap"09:26
kashyap... give me a déjà vu, as I fixed similar errors in unit tests in main patch.09:27
kashyapUsually lowering the index of devices by one, because we removed one needless serial device.09:27
*** ratailor has joined #openstack-nova09:28
gibikashyap: that index out of range is a missing notification that is probably caused by the stack trace before it09:28
gibi  File "/home/zuul/src/git.openstack.org/openstack/nova/nova/compute/manager.py", line 5688, in _init_volume_connection09:28
gibi    new_cinfo['serial'] = new_volume_id09:28
gibiTypeError: 'tuple' object does not support item assignment09:28
*** sdake has joined #openstack-nova09:28
* gibi is running functional-py35 locally09:28
* kashyap looks09:29
*** ociuhandu has joined #openstack-nova09:31
kashyapgibi: What's your `tox` command-line?09:31
gibitox -e functional-py3509:31
*** slaweq has quit IRC09:31
*** ccamacho has quit IRC09:31
*** slaweq has joined #openstack-nova09:32
gibikashyap: I cannot reproduce the failure locally09:32
kashyapHmm09:33
gibibut the stacktrace is there in the log even if the test does not fail09:33
kashyapAh, I see.09:33
gibino I will check for the stack trace without your patch09:33
kashyapgibi: One more: did you apply the main bump patch as well?09:33
* kashyap runs 'functional-py35' locally09:34
*** dtantsur|afk is now known as dtantsur09:34
gibikashyap: I checked out your Omit patch so I have the whole series09:34
kashyapAh, good.09:35
*** ociuhandu has quit IRC09:35
gibikashyap: the stacktrace is there even on master without your series09:35
gibiso it is something unrelated09:35
kashyapOkay, noted.  Thanks for the quick check.09:36
*** zhubx has quit IRC09:36
kashyapWonder then should I 'recheck', as this could be a spurious failure?09:36
gibibottom line, the test fail in your patch due to some timing issue in the notification test, unrelated to that there is a ugly stack trace in the log even if it passes, all unrelated to your patch09:36
gibikashyap: yep, recheck it is09:36
*** wolverineav has joined #openstack-nova09:37
kashyapNoted, thanks for the quick double-check, feel a bit saner now09:37
*** ttsiouts has quit IRC09:38
*** ttsiouts has joined #openstack-nova09:38
*** cfriesen has quit IRC09:38
openstackgerritLee Yarwood proposed openstack/nova master: compute: Reject resize requests when the source host is down  https://review.openstack.org/62348909:38
lyarwoodsean-k-mooney: ^ I think you looked at this a while ago, should be ready for another review if you have time today.09:39
gibikashyap: filed a bug not to forget to look at that stacktrace https://bugs.launchpad.net/nova/+bug/181653809:40
openstackLaunchpad bug 1816538 in OpenStack Compute (nova) "test_volume_swap_server_with_error produce a suspicious stack trace" [Undecided,New]09:40
kashyapAh, excellent, thanks09:40
*** tetsuro has quit IRC09:41
*** derekh has joined #openstack-nova09:42
*** sdake has quit IRC09:49
openstackgerritSylvain Bauza proposed openstack/nova master: Use the correct mdev allocated from the pGPU  https://review.openstack.org/63659109:50
openstackgerritSylvain Bauza proposed openstack/nova master: libvirt: implement reshaper for vgpu  https://review.openstack.org/59920809:50
openstackgerritSylvain Bauza proposed openstack/nova master: Add functional test for libvirt vgpu reshape  https://review.openstack.org/63155909:50
*** bhdn has quit IRC09:53
*** awalende has joined #openstack-nova09:54
*** bhagyashris has quit IRC09:55
*** ociuhandu has joined #openstack-nova09:56
*** ttsiouts has quit IRC09:56
*** ociuhandu has quit IRC09:57
*** ociuhandu has joined #openstack-nova09:58
*** ttsiouts has joined #openstack-nova09:58
kashyapstephenfin: When you can: https://review.openstack.org/#/c/632507/09:58
kashyapgibi: Remind me again, if I touch-up the commit message, will I lose all the +2s / +1s?09:59
kashyapThere is a small grammar mistake; it absolutely needs no touching up, but it's just the OCD...09:59
*** xek has joined #openstack-nova10:01
gibikashyap: hm, if you lose it I can give it back10:04
gibikashyap: I think rebase only remove +A10:04
gibibut commit message change might remove +2 as well10:04
kashyapNot worth it; the `diff` is this:10:04
kashyap- create a duplicate 'pty' devices10:05
kashyap+ create duplicate 'pty' devices10:05
kashyap:D10:05
*** cdent has joined #openstack-nova10:06
gibikashyap: don't change it :D10:06
* gibi is not native English so he has no pain over such small thing obviously10:07
kashyapWon't :-)10:08
kashyapI'm not a native speaker either, but a stickler for (my own) language.10:09
kashyaps/language/words/10:09
*** wolverineav has quit IRC10:10
stephenfinkashyap: Yup, I left one comment there10:11
*** markvoelker has joined #openstack-nova10:11
stephenfinStill wondering if we should fix the s390x as a (backportable) precursor10:11
kashyapstephenfin: Yeah, I'm wondering that too.  I'll respond  in the afternoon; need to be AFK soon for a bit10:12
stephenfinkashyap: Cool. I'll hold off reviewing until then so10:13
stephenfinIt probably doesn't matter too much (no one has spotted this in ~2 years) but if it's an easy fix...10:13
kashyapstephenfin: Yeah.  And to backport it, it's easy to glean it off10:14
kashyapstephenfin: Probably not worth the churn in this patch, IMHO.  Not sure if you agree there10:14
kashyapLet me comment on the review, and then head out10:15
*** sajauddin has joined #openstack-nova10:18
kashyapstephenfin: Responded; let me know what you think there.10:19
kashyapstephenfin: Alright, will do.  I _began_ with a separate patch, originally, you know - then squashed it :-)10:30
*** ttsiouts has quit IRC10:40
*** priteau has quit IRC10:41
*** ttsiouts has joined #openstack-nova10:41
*** ttsiouts_ has joined #openstack-nova10:42
*** markvoelker has quit IRC10:43
*** ttsiouts has quit IRC10:45
*** lpetrut has joined #openstack-nova10:46
openstackgerritAdam Spiers proposed openstack/nova master: WIP: Convert driver supported capabilities to compute node provider traits  https://review.openstack.org/53849810:47
*** wolverineav has joined #openstack-nova11:07
*** udesale has quit IRC11:10
*** Dinesh_Bhor has quit IRC11:14
*** snevi has joined #openstack-nova11:25
*** tiendc has quit IRC11:29
*** pbing19 has quit IRC11:30
*** snevi has quit IRC11:31
*** wolverineav has quit IRC11:39
openstackgerritSilvan Kaiser proposed openstack/nova master: Added mount fstype based validation of Quobyte mounts  https://review.openstack.org/52224511:40
*** markvoelker has joined #openstack-nova11:40
*** thgcorrea has joined #openstack-nova11:41
*** awalende has quit IRC11:42
*** awalende has joined #openstack-nova11:42
*** awalende has quit IRC11:44
*** awalende has joined #openstack-nova11:44
*** tbachman has quit IRC11:44
openstackgerritsean mooney proposed openstack/os-vif master: [WIP] remove brctl from vif_plug_ovs  https://review.openstack.org/63682111:50
openstackgerritsean mooney proposed openstack/os-vif master: remove brctl from vif_plug_ovs  https://review.openstack.org/63682111:52
openstackgerritLee Yarwood proposed openstack/nova stable/queens: Avoid redundant initialize_connection on source post live migration  https://review.openstack.org/63782711:53
*** xek has quit IRC11:56
*** ttsiouts_ has quit IRC11:57
*** xek has joined #openstack-nova11:57
*** xek_ has joined #openstack-nova11:59
*** xek has quit IRC11:59
*** awalende has quit IRC12:01
*** liumk_ has quit IRC12:01
*** awalende has joined #openstack-nova12:01
*** awalende has quit IRC12:02
*** liumk_ has joined #openstack-nova12:02
*** awalende has joined #openstack-nova12:02
*** ratailor has quit IRC12:06
*** mvkr has quit IRC12:09
*** davidsha has joined #openstack-nova12:13
*** liumk_ has quit IRC12:13
*** erlon has joined #openstack-nova12:14
*** markvoelker has quit IRC12:14
*** erlon has quit IRC12:14
*** erlon has joined #openstack-nova12:15
*** erlon has quit IRC12:16
*** erlon has joined #openstack-nova12:16
*** sajauddin has quit IRC12:20
openstackgerritya.wang proposed openstack/nova master: Add configuration item cpu_models to libvirt  https://review.openstack.org/63783412:35
*** wolverineav has joined #openstack-nova12:36
*** janki has quit IRC12:41
*** pbing19 has joined #openstack-nova12:43
*** snevi has joined #openstack-nova12:43
*** janki has joined #openstack-nova12:44
*** dpawlik has quit IRC12:51
*** janki has quit IRC12:52
*** janki has joined #openstack-nova12:53
*** janki has quit IRC12:54
*** janki has joined #openstack-nova12:54
*** udesale has joined #openstack-nova12:55
*** xek_ has quit IRC13:01
*** xek_ has joined #openstack-nova13:02
*** yan0s has quit IRC13:03
*** tbachman has joined #openstack-nova13:05
*** tbachman_ has joined #openstack-nova13:09
*** wolverineav has quit IRC13:09
*** tbachman has quit IRC13:10
*** tbachman_ is now known as tbachman13:10
*** markvoelker has joined #openstack-nova13:11
*** ttsiouts has joined #openstack-nova13:11
aspierscdent: you around? I'm a bit confused whether a compute driver should be able to provide a CUSTOM_ trait or not.13:12
*** itlinux has joined #openstack-nova13:12
*** mvkr has joined #openstack-nova13:13
aspiershttps://specs.openstack.org/openstack/nova-specs/specs/pike/implemented/resource-provider-traits.html suggests not but https://review.openstack.org/#/c/538498/7/nova/virt/driver.py@934 seems to contradict that13:13
openstackgerritsean mooney proposed openstack/os-vif master: remove use of brctl from vif_plug_linux_bridge  https://review.openstack.org/63682213:15
cdentaspiers: I'm around now. What's up?13:15
aspiersthat spec suggests CUSTOM_* was just for admin/operator usage, but maybe it's for everything not in os-traits?13:16
aspiersI see the placement API prevents a PUT of any trait without CUSTOM_ prefix13:16
aspierswould a virt driver ever want to provide a trait not in os-traits?13:16
aspiers3rd-party drivers maybe?13:17
*** ttsiouts has quit IRC13:18
*** ttsiouts has joined #openstack-nova13:19
*** itlinux has quit IRC13:21
*** itlinux has joined #openstack-nova13:22
*** dpawlik has joined #openstack-nova13:23
cdentaspiers: sorry for the delay. It depends on who you ask.13:23
cdentThe hope is that anything that is the result of the virt driver "knowing things" about it's environment would be traits in os-traits13:24
*** sdake has joined #openstack-nova13:24
cdentand as things are needed by the virt drivers they get added to os-traits13:24
*** yan0s has joined #openstack-nova13:25
cdentbut it's pretty clear that's not always going to be realistic, but, as you say placement will only allow CUSTOM_* traits to be created over the api13:25
cdents/it's/its/13:25
* aspiers notes a fellow punctuation perfectionist ;-)13:26
kashyapstephenfin: Untangling the mess out of it turning out to be more painful than expected :-(13:26
cdentaspiers: it's more a matter of trying (and failing) to fix my hasty typing disease. It's especially bad with contractions and homophones.13:26
kashyap"the mess" -- remove the 'sclplm'-only first.13:26
aspierscdent: so you mean that for now at least, we have to allow for drivers providing traits not in os-traits? but would those traits always have the CUSTOM_ prefix, or not necessarily?13:27
cdentIf a driver needs to use something that's not in os-traits they have two choices:13:27
cdent* add it to os-traits13:27
cdent* use a CUSTOM_*13:27
cdentideally the former would be the go to thing13:27
*** itlinux has quit IRC13:28
aspiersgot it13:29
*** itlinux has joined #openstack-nova13:32
*** gokhani has joined #openstack-nova13:32
*** agopi has quit IRC13:33
*** med_ has joined #openstack-nova13:34
*** itlinux has quit IRC13:37
*** gbarros has joined #openstack-nova13:39
*** jmlowe has quit IRC13:41
*** itlinux has joined #openstack-nova13:42
*** markvoelker has quit IRC13:43
*** hamzy has joined #openstack-nova13:44
*** Dinesh_Bhor has joined #openstack-nova13:45
aspierscdent: so this? https://pasteboard.co/I1Tbzp2.png13:47
*** itlinux has quit IRC13:47
openstackgerritSilvan Kaiser proposed openstack/nova master: Exec systemd-run without --user flag in Quobyte driver  https://review.openstack.org/55419513:48
tssuryamriedem, dansmith: under which section of the admin docs would you prefer the down cell to be in ? or should I just start w new one?13:48
kaisersstephenfin: Hi! FYI: Followed up on your comments on https://review.openstack.org/#/c/522245/ and https://review.openstack.org/#/c/554195/13:49
*** dave-mccowan has joined #openstack-nova13:49
*** amorin has quit IRC13:50
*** janki has quit IRC13:54
*** eharney has joined #openstack-nova13:55
*** ttsiouts has quit IRC14:04
*** ttsiouts has joined #openstack-nova14:04
*** sdake has quit IRC14:05
*** ccamacho has joined #openstack-nova14:05
*** jmlowe has joined #openstack-nova14:07
*** wolverineav has joined #openstack-nova14:07
*** liuyulong has quit IRC14:08
*** ttsiouts has quit IRC14:09
*** ttsiouts has joined #openstack-nova14:09
*** Dinesh_Bhor has quit IRC14:10
*** sdake has joined #openstack-nova14:13
openstackgerritHamdy Khader proposed openstack/os-vif master: Add create_port field in VIFPortProfileOpenVSwitch profile  https://review.openstack.org/63606114:17
stephenfinkaisers: Sweet. I've left comments on the systemd one. Might be as easy discuss here if you disagree14:17
stephenfinkaisers: I'm well aware of how many revisions it's gone through too. Sorry :(14:17
*** ccamacho has quit IRC14:18
*** sdake has quit IRC14:19
*** itlinux has joined #openstack-nova14:22
*** ccamacho has joined #openstack-nova14:22
kaisersstephenfin: mmh, driver changes have outgrown the tests, I'm going through those...14:22
*** mriedem has joined #openstack-nova14:25
*** xek__ has joined #openstack-nova14:28
*** itlinux has quit IRC14:30
*** xek_ has quit IRC14:31
cdentaspiers: was on a walk. parsing that image...14:32
*** yan0s has quit IRC14:34
*** cfriesen has joined #openstack-nova14:34
*** lbragstad has joined #openstack-nova14:34
mriedemmelwitt: so on https://review.openstack.org/#/c/633351/ you're picking up a marker instance record from an earlier data migration14:36
mriedemand that marker instance doesn't have a user_id set14:36
*** sdake has joined #openstack-nova14:36
mriedemhence the infinite loop14:36
stephenfinkaisers: Addressed the other one too https://review.openstack.org/#/c/522245/14:37
kaisersstephenfin: thnx!14:38
mriedemmelwitt: this is the migration that creates the marker instance with the 0'ed uuid and no user_id https://github.com/openstack/nova/blob/ab5a9bba31664e74c1e866c620b0d284d1d4f4cb/nova/objects/virtual_interface.py#L30114:38
*** _alastor_ has joined #openstack-nova14:38
*** wolverineav has quit IRC14:39
mriedemthere must also be a data migration that creates a marker instance mapping with the same 00000000-0000-0000-0000-000000000000 uuid14:39
*** mriedem has quit IRC14:40
*** mriedem has joined #openstack-nova14:40
*** markvoelker has joined #openstack-nova14:41
*** _alastor_ has quit IRC14:43
aspiersmriedem: please can you sanity check my understanding too? https://pasteboard.co/I1Tbzp2.png14:44
mriedemthe capabilities patch being the one you're trying to rebase?14:45
mriedemthen yes i think so14:45
aspiersyes, that one14:46
*** munimeha1 has joined #openstack-nova14:46
aspierswell, I've succeeded in rebasing it ;-) the struggle now is finishing off the tests14:46
mriedemthe compute "owns" the capabilities it reports, and if something external removes it then it will get put back on the next periodic task run14:46
mriedemif an admin wants to schedule around hosts with a given capability, they use forbidden traits in their flavor14:46
aspiersbut the driver can still provide other traits outside this new capabilities-to-traits mechanism, right? like cpu flags etc.14:47
mriedemcorrect14:47
aspiersbut it can also provide CUSTOM_* stuff which isn't capabilities?14:47
mriedemthe libvirt driver would be reporting traits that are both cpu flags and capabilities, like COMPUTE_SUPPORTS_MULTIATTACH or whatever14:47
mriedemno i don't think so14:47
aspiersis that COMPUTE_SUPPORTS_MULTIATTACH or CUSTOM_COMPUTE_SUPPORTS_MULTIATTACH?14:48
mriedemnot custom14:48
mriedemit's a standard trait14:48
aspiersah OK14:48
mriedemhttps://github.com/openstack/os-traits/tree/master/os_traits/compute14:48
openstackgerritJan Gutter proposed openstack/os-vif master: Clean up versioned object backlevelling code  https://review.openstack.org/63097614:48
mriedemhttps://github.com/openstack/os-traits/blob/master/os_traits/compute/volume.py#L2414:49
mriedemCOMPUTE_TRUSTED_CERTS is another https://github.com/openstack/os-traits/blob/master/os_traits/compute/__init__.py#L1914:49
aspiersgot it14:49
mriedemthe *only* driver i'm not sure about is ironic since i think nova proxies the traits from the ironic node to placement14:50
mriedemi'm not sure if those are made CUSTOM_14:50
mriedemthat happens here https://github.com/openstack/nova/blob/master/nova/virt/ironic/driver.py#L83114:50
mriedemlooks like the traits on the node are just passed through as-is14:50
mriedemso they can be standard or custom14:51
mriedemjroll: ^ is that accurate?14:51
mriedemhttps://developer.openstack.org/api-ref/baremetal/?expanded=show-node-details-detail#node-traits-nodes14:51
mriedem"Traits can be either standard or custom."14:51
*** yan0s has joined #openstack-nova14:52
aspiersmriedem: so the puzzle I'm currently facing with tests is this: I want to remove a CUSTOM_ trait in order to test that the periodic update replaces it correctly. But if I delete using the placement API fixtures then the trait is still in the provider tree, so it doesn't recognise that the traits are out of sync due to https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L104614:52
*** sdake has quit IRC14:53
aspiersif I comment out that return then it works, but I'm not sure if that will break something else14:53
*** udesale has quit IRC14:53
*** itlinux has joined #openstack-nova14:53
jrollmriedem: they can only be custom: https://developer.openstack.org/api-ref/baremetal/#node-traits-nodes14:54
jroller wait14:54
mriedemso set_traits_for_provider is being called with the traits from the provider tree in the compute driver rather than the traits from the placement API?14:54
jrollno, they can be both, you're right14:54
aspiersmriedem: correct14:55
aspiers(I think anyway; my head is beginning to hurt a bit at this stage)14:55
mriedemaspiers: that sounds like an efried question14:56
*** itlinux has quit IRC14:56
aspiersOK14:56
mriedemand i'm about to jump on a call14:56
aspiersOK np, thanks for the help14:56
mriedemeric should be well rested after his presidents day celebrations14:56
efriedaspiers: gimme a sec14:56
aspiershaha14:56
*** itlinux has joined #openstack-nova14:56
efriedyeah, it wasn't what you would call a relaxing day.14:56
mriedemwas someone out protesting the border wall again14:57
*** itlinux has quit IRC14:57
aspiers%-/14:57
efriedaspiers: should I read back or you wanna summarize?14:58
aspiersI'll try to summarise14:58
efriedor that14:58
stephenfinkashyap: What Georg Hoesch has pointed out here looks correct https://review.openstack.org/#/c/632507/12/nova/virt/libvirt/driver.py14:58
aspiersit looks to me like set_traits_for_provider() is currently designed as a push from the provider tree to placement, rather than a bidirectional sync14:58
stephenfinkashyap: If you agree, I'm happy to +2 as-is14:58
stephenfin(Don't know how far you got in splitting up that patch)14:58
* efried will try to honour aspiers's spelling for a bit...14:58
kashyapstephenfin: Yeah, just read that.  In the last 2 hours I came to the conclusion that we should untangle it14:59
kashyapstephenfin: Phew.  I was just about to admit myself to the "sanity ward"14:59
aspiers:)14:59
efriedaspiers: true story. Up to now, virt drivers have been fully responsible for knowing what traits compute "owns".14:59
*** sdake has joined #openstack-nova14:59
kashyapstephenfin: That consoles stuff is mind-warping (to my weak brain, at least)14:59
aspiersefried: so the puzzle I'm currently facing with tests is this: I want to remove a CUSTOM_ trait in test_servers.py in order to test that the periodic update replaces it correctly. But if I delete using the placement API fixtures then the trait is still in the provider tree, so it doesn't recognise that the traits are out of sync due to14:59
stephenfinkashyap: It's a maze. So many conditionals14:59
aspiershttps://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L104614:59
kashyapAnd I'm glad I Cced Georg to also double-check.  He writes thorough comments14:59
efriedaspiers: This will be (I think) the first time code outside of the virt driver, but inside n-cpu, will assert "ownership" of traits.15:00
stephenfinkashyap: So, happy for me to go ahead and approve this as-is?15:00
kashyapstephenfin: Yeah, did you see the reworked table?  It was a cathartic relief to turn the tri-state to binary15:00
kashyapstephenfin: Yes, please15:00
stephenfindone and done15:00
efriedaspiers: okay, I understand the problem. Give us a tick to suss it out...15:00
* kashyap goes to comment15:01
kashyapstephenfin: Excellent.15:01
*** jobewan has joined #openstack-nova15:01
*** whoami-rajat has quit IRC15:01
kashyapstephenfin: Unrelated: on the top-most patch, on that nuking of 'if' conditional from _create_pty_device() -- I noticed it, had a draft comment to myself, before I saw yours.15:02
kashyapWill address that, too15:02
stephenfinkashyap: ack15:02
*** awaugama has joined #openstack-nova15:03
aspiersefried: this probably doesn't help, but this is how things work with the capabilities patch from mriedem / me: https://goo.gl/haARAc15:03
aspiersI didn't get as far as modelling the sync with placement15:04
aspiersbecause I didn't expect to need to change that15:04
efriedaspiers: Okay, so this is pretty tricky. We implemented a SIGHUP trap to trigger the report client to flush its cache and re-pull everything from placement. The operator is *supposed* to SIGHUP n-cpu any time they make an "out-of-band" change like the ones you're simulating here.15:04
efriedaspiers: So technically what you need to do is call ComputeManager.reset() before you hit the periodic and your test will work.15:05
*** awalende has quit IRC15:05
aspiersso that reset is more heavyweight than a periodic update?15:05
efriedaspiers: But it points to an interesting real-world issue, which is: what if the operator doesn't do that?15:05
aspiersright15:05
efriedaspiers: Yes, it triggers the report client to blow away its provider tree cache completely, so on the periodic it will go re-retrieve everything.15:05
*** awalende has joined #openstack-nova15:05
aspiersdo we really need that have_traits_changed() check? is it more than an optimisation?15:05
efriedit is exactly an optimiz^Hsation, so that we don't have to query placement every periodic as to whether things have changed.15:06
aspiersah yeah15:06
aspiershow often is the periodic?15:06
efrieduh15:07
*** awalende has quit IRC15:07
aspiersI guess with 10k compute nodes it would be a problem even if it was hourly15:07
*** awalende has joined #openstack-nova15:07
*** awalende has quit IRC15:07
efriedyes, exactly. And in fact we implemented a way to switch off the association refresh entirely. So there's a timer that's separate from the periodic timer, and if that one expires, we go re-pull anyway.15:08
aspiershrm15:08
efriedbut CERN wound up having to set that to ~inf or it brought their data center to its knees.15:08
aspiershaha15:08
efriedand in fact, we found we don't need it15:09
efriedso we enabled setting it to zero, meaning "off".15:09
efriedthis just merged a couple weeks ago.15:09
aspiersinteresting15:09
efriedso what would happen is, if the report client felt like it needed to update anything about the traits after your oob update, it would bounce on a generation conflict, re-GET, redrive its tweak, and redo the push.15:10
aspiersso my test could just call self.compute.manager.reportclient.clear_provider_cache() ?15:10
efriedyes. And that's the right thing... for your test.15:10
aspiersor reset() directly15:10
efriedI'm just not super happy with the real-world issue it points to.15:10
aspiersagreed15:10
*** awalende has joined #openstack-nova15:11
aspiersI guess longer term traits from compute drivers could be marked as read-only?15:12
aspiersso then they couldn't be deleted via placement API15:12
aspierseven CUSTOM_* ones15:12
*** mlavalle has joined #openstack-nova15:12
aspiersonly the driver which originated it could delete it15:13
aspiersor traits could have an owner associated explicitly15:13
aspiersso then creation and deletion would always both one-way pushes from provider tree to placement15:14
*** markvoelker has quit IRC15:14
*** rpittau has quit IRC15:14
*** awalende has quit IRC15:15
efriedaspiers: Have a gander at this: https://github.com/openstack/nova/blob/master/nova/tests/functional/libvirt/test_report_cpu_traits.py15:15
kashyapstephenfin: What do you mean by a "UT"?15:15
efriedaspiers: Yes, we discussed those things a bit at a PTG or two, decided on this.15:15
stephenfinkashyap: Unit Test15:15
kashyapAh, thanks.  :-)15:16
stephenfinI'd assume, anyway15:16
kashyapstephenfin: Yes, you win, because the `wtf` tool actually detects the acronym15:16
kashyap$> rpm -qf `which wtf`15:16
kashyapbsd-games-2.17-57.fc28.x86_6415:16
kashyapIt's super-tiny RPM.  And comes with an acryonym file15:17
aspiersefried: I found this post-Dublin conversation http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2018-03-12.log.html#t2018-03-12T16:09:2215:18
*** whoami-rajat has joined #openstack-nova15:18
*** Luzi has quit IRC15:19
aspiersI was reading through to try to catch up on a year's worth of thinking (not to mention several decades of nova experience...)15:19
aspiersit was linked from https://review.openstack.org/#/c/552122/415:19
aspiersefried: but OK, for now I'll just do reset()15:20
efriedaspiers: Yeah, that's the right thing. Not sure what to do about the other. Maybe we just leave it alone and hope it doesn't become a problem. Operators always do what they're supposed to, right?15:20
aspiershaha yeah sure, always ;-)15:21
*** pbing19 has quit IRC15:22
*** jmlowe has quit IRC15:23
*** jmlowe has joined #openstack-nova15:24
aspiersefried: it works!15:28
*** sdake has quit IRC15:28
efriedwoot!15:30
aspiersIt's been an unexpectedly rough 24 hours on this. Took me many hours yesterday trawling through with pdb to find this microversion issue https://review.openstack.org/#/c/637657/15:31
aspiersOnce things go wrong in the middleware that's a whole can of voodoo worms which I'm not used to opening up15:31
efriedholy unnecessary shim, Batman15:33
efriedI remember last time I encountered these fake_* methods I griped about how they should just diaf.15:33
aspiersX-)15:33
*** jmlowe has quit IRC15:34
efriedaspiers: in fact, I may demand that we do that instead of the fix you've proposed.15:35
aspiersThat would make me sad, because it would introduce a big new dependency to my SEV work15:36
*** sdake has joined #openstack-nova15:36
*** pbing19 has joined #openstack-nova15:36
efriedaspiers: It wouldn't be a big thang; just needs someone to do it. And since I'm the one being difficult...15:36
aspiersI'm trying to do the right thing by landing the capabilities-to-traits feature and then implementing the SEV trait on top of it15:36
*** wolverineav has joined #openstack-nova15:37
aspiersrather than just tweaking the libvirt driver's u_p_t()15:37
efriedokay, I hear you.15:37
aspiersThe latter would be the easy way out, but I wanted to give something useful upstream before asking for "our" feature work to be accepted15:37
efriedGive me half an hour?15:37
aspiersOh wow, well sure, if you're that fast :)15:38
jangutterkashyap: wtf's pretty nifty. It's got dry, crud, rest and a bunch of others that are tricky to google.15:38
efriedaspiers: I remember thinking it would be pretty easy. 25 minutes of that is going to be refreshing myself on the code.15:38
aspiersIt sounded to my inexperienced ears like it would take days15:38
efriedaspiers: What test were you running when you hit this?15:38
kashyapjangutter: Yeah.15:38
aspiersefried: the patch which sits on top of it15:38
aspiersefried: https://review.openstack.org/#/c/538498/1015:39
aspiersbut let me upload a working PS 1115:39
aspierswhich will even pass PEP8 *shock* :)15:39
openstackgerritsean mooney proposed openstack/nova master: libvirt: auto detach/attach sriov ports on migration  https://review.openstack.org/62958915:39
efriedaspiers: test_servers presumably?15:39
aspiersturns out that tox -e flake8 pretends to do something useful but actually checks nothing at all ...15:39
aspierscorrect15:39
aspiersstestr --test-path=./nova/tests/functional run -n nova.tests.functional.test_servers.TraitsTrackingTests.test_resource_provider_traits15:40
aspiersif you want the quicker version15:40
efriedokay, thanks.15:40
kashyapstephenfin: I'm still a bit squirmy about this whole muddling with s390x, and its console devices.  Because both "sclp" and "sclplm" are for different purposes, per s390x maintainers.  And I don't have access to a machine to test it all15:40
*** mdbooth has joined #openstack-nova15:40
efriedaspiers: btw, if you install tox 3.1.1 (uh, which I guess I forced you to do anyway), tox -e functional -- test_resource_provider_traits should be almost as quick.15:40
kashyapSo I was just poking the code to see until it poked me back...15:40
aspiersefried: yeah I figure that out, but I find that tox still adds unacceptable overhead15:41
efriedfor test discovery?15:41
aspiersthe {posargs} trick is handy though15:41
kashyapstephenfin: Anyway, so far from our code audit it is mergeable; so no need to belabour more on it now.  </EOM>15:41
* kashyap bbiab15:41
aspiersnot just test discovery15:41
efriedWe got rid of the bytecode15:41
aspiersefried: it takes ~2s on my machine to get to the point of launching stestr15:42
aspiersit has to go through all the virtualenv stuff first15:42
mdboothmriedem: If you recall this one: https://review.openstack.org/#/c/626952/ . It looks like the commit message there is rubbish. I haven't been able to reproduce the underlying issue and I suspect it's actually been fixed by upgrading libraries. Haven't worked out which ones, though.15:43
mdboothThat said, I still like this cleanup, and I suspect the approach is slightly more robust to similar issues. Is it worth rewriting the commit message and keeping it around?15:43
mdboothIf there's no interest I'll just drop it. Or lyarwood will, anyway :)15:44
mriedemi don't have a strong opinion15:45
*** lpetrut has quit IRC15:45
mdboothmriedem: ack. I'll probably drop it then as it's likely to languish. Thanks.15:46
mdboothlyarwood: ^^^ Lets drop it.15:46
openstackgerritAdam Spiers proposed openstack/nova master: WIP: Convert driver supported capabilities to compute node provider traits  https://review.openstack.org/53849815:49
aspiersefried: there's PS 11 with a working test which includes the reset()15:49
efriedaspiers: The test was working for me at ps1015:50
aspiersefried: yeah but it didn't test the reappearance of the trait15:51
aspiersefried: https://review.openstack.org/#/c/538498/10..11/nova/tests/functional/test_servers.py@227415:52
*** sdake has quit IRC15:52
efriedokay. It doesn't really matter; I just need something that's supposed to hit that fake_delete15:52
aspiersgood point15:52
*** maciejjozefczyk has quit IRC15:53
*** pbing19 has quit IRC15:55
*** sdake has joined #openstack-nova15:58
*** _alastor_ has joined #openstack-nova16:00
mriedemcdent: so the nova functional tests - they use the placement fixture from extracted placement right? not the in-tree placement code. so doing https://review.openstack.org/#/c/637892/ in nova would only be for runtime if we copied that16:00
cdentyes16:02
mriedemok16:02
*** sdake has quit IRC16:03
*** hongbin has joined #openstack-nova16:03
cdentmriedem: plus that osa thing that m*naser was gunna set up16:05
openstackgerritsean mooney proposed openstack/os-vif master: Change python3.5 job to python3.7 job on Stein+  https://review.openstack.org/61006816:08
mriedemcdent: yeah i'm more concerned (right now anyway) on the subunit parser thing16:08
* cdent nods16:08
*** wolverineav has quit IRC16:10
bauzasefried: others, I found yet another issue with vgpu reshapes...16:11
*** markvoelker has joined #openstack-nova16:11
bauzasefried: others, given we need to recreate mdevs when rebooting, we need to pass allocations when init_host()16:12
bauzasI wrote a bit of comment in the first change, but I'm working on a change that will do this16:12
*** pbing19 has joined #openstack-nova16:16
mdboothlyarwood: https://review.openstack.org/#/c/623489/16:17
mdboothlyarwood: Just looking at the unit tests, but feel free to consider ^^^ anyway.16:17
lyarwoodmdbooth: ack and ack looking16:20
kashyapBTW, sean-k-mooney: thanks for inadvertently reminding that this is a pending item a few weeks ago!  Almost lost track of it: https://review.openstack.org/#/c/63250716:24
sean-k-mooney  kashyap oh the libvirt version bump ya16:26
openstackgerritSylvain Bauza proposed openstack/nova master: libvirt: implement reshaper for vgpu  https://review.openstack.org/59920816:31
openstackgerritSylvain Bauza proposed openstack/nova master: Add functional test for libvirt vgpu reshape  https://review.openstack.org/63155916:31
*** jaypipes has quit IRC16:36
*** jaypipes has joined #openstack-nova16:36
*** markvoelker has quit IRC16:44
*** macza has joined #openstack-nova16:47
*** dtantsur is now known as dtantsur|afk16:48
*** hamzy has quit IRC16:50
*** hongbin has quit IRC16:54
*** ttsiouts has quit IRC16:56
sean-k-mooneyso are we still going ahead with https://review.openstack.org/#/c/634949/ since the cinder volume test in tempest-full are continuosly failing.16:56
*** ttsiouts has joined #openstack-nova16:56
*** pcaruana has quit IRC16:57
mriedemwhat cinder volume tests?16:59
mriedemwe did do https://review.openstack.org/#/c/634970/ to hopefully help with the tooz connection errors17:00
*** tssurya has quit IRC17:01
sean-k-mooneytempest.api.compute.volumes.test_attach_volume_negative.AttachVolumeNegativeTest.test_attach_attached_volume_to_same_server17:01
*** wolverineav has joined #openstack-nova17:01
*** ttsiouts has quit IRC17:01
*** helenafm has quit IRC17:01
sean-k-mooneyi dont know if it sthe same two but i have notice 1-2 cinder volume attachment test filaing in a few reivews i looked at recently17:02
sean-k-mooneyusually either with a timeout or sometime with Invalid volue errors17:03
sean-k-mooneylike http://logs.openstack.org/15/620115/23/check/tempest-full/fcabb4b/testr_results.html.gz17:03
smcginnismriedem: I haven't noticed that tooz issue anymore. Have you seen any since that change?17:03
mriedemsmcginnis: http://status.openstack.org/elastic-recheck/#181052617:04
mriedemthose could be stable branches, checking logstash17:05
smcginnisGrenade on the first one, so maybe?17:05
mriedemsean-k-mooney: you hit a nova bug http://logs.openstack.org/15/620115/23/check/tempest-full/fcabb4b/controller/logs/screen-n-cpu.txt.gz?level=TRACE#_Feb_19_14_49_49_27732517:06
mriedemhttp://status.openstack.org/elastic-recheck/#177021117:06
mriedem WARNING nova.virt.block_device [None req-2c3d004a-05e7-43d3-af5e-ccac75bed5fd tempest-AttachVolumeNegativeTest-1375016298 tempest-AttachVolumeNegativeTest-1375016298] [instance: 4b50efcf-0d26-4b01-91f0-1b8b1e9eef0f] Guest refused to detach volume aea37477-6c0c-4727-8092-da3b3d2f7c5c: DeviceDetachFailed: Device detach failed for vdb: Unable to detach the device from the live config.17:06
mriedemi remember cfriesen asking about this still being an issue the other day17:06
mriedemsmcginnis: yeah looks like it's hitting on grenade so it's just a stable branch issue with devstack at this poin,17:07
mriedem*point17:07
dansmithmriedem: I saw a ping from you fly by from last night.. can you link me again so I don't have to dig out mah logs?17:07
* dansmith is lazy17:07
mriedemi'll work on backporting those devstack / etcd changes17:07
mriedemdansmith: https://review.openstack.org/#/c/635315/117:07
*** _fragatina has joined #openstack-nova17:09
dansmithmriedem: nice17:10
cfriesenmriedem: smcginnis: I was hitting this:https://bugs.launchpad.net/nova/+bug/181594917:11
openstackLaunchpad bug 1815949 in OpenStack Compute (nova) "missing special-case libvirt exception during device detach" [Medium,Confirmed]17:11
smcginnisHmm, that's a new one to me.17:11
sean-k-mooneyok ya that makes sense. im pretty sure there are other voluem attachemtn failures like https://bugs.launchpad.net/cinder/+bug/1814916 too which are unrelated to tooz17:12
mriedemcfriesen: ok that looks different than the 'unable to detach from transient / live config'17:12
openstackLaunchpad bug 1814916 in Cinder "MessagingTimeout in cinder-api during volume attachment update due to VolumeAttachmentNotFound in cinder-volume" [Medium,Confirmed]17:12
mriedemyeah i don't know what's up with that volume attachment not found one17:12
mriedemthe api finds the record and then c-vol blows up b/c it's gone from the db, so clearly a race somewhere17:12
mriedemprobably just need someone to trace requests in a log to see when the PUT request is made and when the DELETE request is made17:12
cfriesenmriedem: smcginnis: but after fixing that I'm hitting the "Guest refused to detach" issue.  I'm guessing it's related to stalls we're seeing in the guest, but still digging into causes.17:12
openstackgerritAdam Spiers proposed openstack/nova master: WIP: Convert driver supported capabilities to compute node provider traits  https://review.openstack.org/53849817:14
aspiersmriedem, efried: I've just finished the first version of the tests which were missing for capabilities to traits: https://review.openstack.org/#/c/538498/12/nova/tests/functional/test_servers.py17:15
mriedemetcd changes for devstack backported https://review.openstack.org/#/q/topic:bug/1810526+(status:open+OR+status:merged)17:15
aspiersmriedem, efried: I'm sure you'll find plenty of things to improve, but hopefully this is a good base17:15
aspiersnot sure if I went overkill on the tests for standard vs. custom traits, and capabilities vs. update_provider_tree17:16
aspiersmriedem: BTW I kept you as the author and added myself via Co-Authored-By - was that the right thing to do?17:22
aspiersnot sure what the convention is here17:22
mriedemaspiers: yup17:24
*** artom has quit IRC17:24
mriedemmore testing > less testing17:24
aspiersmriedem: also, should I put "related to blueprint expose-host-capabilities" in the commit message?17:24
*** lpetrut has joined #openstack-nova17:27
mriedemhmm, that's not approved https://blueprints.launchpad.net/nova/+spec/expose-host-capabilities but the change is essentially trying to solve that same problem17:32
mriedemso sure17:32
*** ociuhandu_ has joined #openstack-nova17:33
mriedemthe spec for that bp predates traits https://review.openstack.org/#/c/222200/17:33
mriedemso it was trying to solve it with something different17:33
*** wolverineav has quit IRC17:34
*** ociuhandu has quit IRC17:36
*** eharney has quit IRC17:37
*** ociuhandu_ has quit IRC17:38
*** psachin has quit IRC17:39
*** lpetrut has quit IRC17:40
*** markvoelker has joined #openstack-nova17:41
*** agopi has joined #openstack-nova17:41
*** NewBruce has quit IRC17:42
*** mma has quit IRC17:42
sean-k-mooneyadrianc: o/ are you wokring on a new version of https://review.openstack.org/#/c/620115/17:43
*** NewBruce has joined #openstack-nova17:43
sean-k-mooneyadrianc: if not i might start adressing stephenfin nits after i get something to eat.17:44
*** ccamacho has quit IRC17:46
*** ociuhandu has joined #openstack-nova17:48
*** ociuhandu has quit IRC17:52
*** derekh has quit IRC17:53
openstackgerritBalazs Gibizer proposed openstack/nova master: Refactor _heal_allocations_for_instance to make place for port healing  https://review.openstack.org/63795317:54
openstackgerritBalazs Gibizer proposed openstack/nova master: Refactor _heal_allocations_for_instance (2)  https://review.openstack.org/63795417:54
openstackgerritBalazs Gibizer proposed openstack/nova master: nova-manage: heal port allocations  https://review.openstack.org/63795517:54
*** yan0s has quit IRC17:54
*** davidsha has quit IRC17:57
*** ociuhandu has joined #openstack-nova18:04
*** ociuhandu has quit IRC18:08
melwittmriedem: omg, thank you. I was completely baffled by the 0'ed out instance uuid and could not imagine how that could happen18:09
*** mdbooth_ has joined #openstack-nova18:11
melwittalex_xu: by unrelated, I meant that project_only=True isn't related to the instance target. the only thing that project_only=True does is enforce the request context project, not the instance project. so I was thinking that changing to use instance target is an unrelated change from removing project_only=True. if we remove project_only=True without changing instance target, we get the same behavior as today except that users choose to18:12
melwitt enforce request context project via policy instead of hard-coded18:12
*** markvoelker has quit IRC18:14
*** mdbooth has quit IRC18:14
efriedaspiers: Still around?18:16
*** hamzy has joined #openstack-nova18:17
aspiersefried: just got back18:20
efriedaspiers: Sorry that took a little longer than expected (interrupted by internal work, among other things), but I got it figured out. Now I have to go around and fix all the test cases that call into the shimmed API.18:20
aspiersefried: nice work :)18:20
efriedWIP incoming, lemme finish writing up the commit message.18:20
aspiersI bet whatever you changed would have taken me a week18:20
aspiersmriedem: OK thanks, I'll do that18:21
openstackgerritEric Fried proposed openstack/nova master: WIP: De-shim PlacementFixture  https://review.openstack.org/63795918:21
efriedaspiers: ^18:21
aspiersmriedem, melwitt: is it OK to reference 2 blueprints in a single commit message?18:21
aspiersefried: ack18:21
efriedI originally wrote it on top of your patch, to make sure your test case worked, and it did.18:22
efriedBut this version will be lacking your delete helper.18:22
efriedOnce I've got it polished I'll rebase it under yours.18:22
efriedor you can18:22
aspiersSure, I can do that18:22
aspiersI have some other tweaks to go in mine anyway18:22
melwittaspiers: I don't see why not. doing that would just put the link on both lp whiteboard AFAIK18:23
aspiersnot to mention fixes for the glaring errors you and mriedem are about to uncover ;-)18:23
aspiersmelwitt: cool thanks, that's what I expected18:23
aspierswoohoo, my version switcher patch landed on https://docs.openstack.org/18:25
aspiershopefully it doesn't break anything18:25
aspierss/version/release/18:25
*** tssurya has joined #openstack-nova18:28
*** ociuhandu has joined #openstack-nova18:30
*** artom has joined #openstack-nova18:32
*** artom has quit IRC18:33
*** artom has joined #openstack-nova18:33
*** hongbin has joined #openstack-nova18:34
*** hongbin has joined #openstack-nova18:35
*** mvkr has quit IRC18:36
*** lpetrut has joined #openstack-nova18:39
openstackgerritBoris Bobrov proposed openstack/nova master: Fallback to the first element if no nodes match  https://review.openstack.org/63796518:40
*** ccamacho has joined #openstack-nova18:42
*** wolverineav has joined #openstack-nova18:50
*** wolverineav has quit IRC18:50
*** wolverineav has joined #openstack-nova18:50
*** snevi has quit IRC18:53
*** gbarros has quit IRC19:00
*** pbing19 has quit IRC19:01
*** mvkr has joined #openstack-nova19:07
*** xek_ has joined #openstack-nova19:07
*** xek__ has quit IRC19:10
*** markvoelker has joined #openstack-nova19:11
*** xek_ has quit IRC19:16
*** panda is now known as panda|off19:17
*** lpetrut has quit IRC19:18
openstackgerritMerged openstack/nova master: api-ref: warn about changing/unsetting AZ name with instances  https://review.openstack.org/63531519:19
openstackgerritmelanie witt proposed openstack/nova master: Add user_id field to InstanceMapping  https://review.openstack.org/63335019:29
openstackgerritmelanie witt proposed openstack/nova master: Add online data migration for populating user_id  https://review.openstack.org/63335119:29
*** sridharg has quit IRC19:29
*** eharney has joined #openstack-nova19:34
*** sdake has joined #openstack-nova19:35
openstackgerritMerged openstack/nova master: api-ref: mention policy defaults for aggregates  https://review.openstack.org/63531719:35
efriedmriedem: easy +A if you please https://review.openstack.org/#/c/637657/19:37
*** jmlowe has joined #openstack-nova19:38
*** mgariepy has quit IRC19:40
*** markvoelker has quit IRC19:44
mriedem+W19:45
efriedthanks mriedem19:50
*** sdake has quit IRC19:52
*** wolverineav has quit IRC19:53
*** wolverineav has joined #openstack-nova19:54
*** sdake has joined #openstack-nova19:57
efriedaspiers: I had to give up. Your fix is gateward.19:57
*** sdake has quit IRC19:58
*** thgcorrea has quit IRC19:59
*** wolverineav has quit IRC20:00
*** openstackgerrit has quit IRC20:09
*** cdent has quit IRC20:24
*** openstackgerrit has joined #openstack-nova20:28
openstackgerritMerged openstack/nova master: Fix fake DELETE in PlacementFixture  https://review.openstack.org/63765720:28
*** efried has quit IRC20:29
*** efried has joined #openstack-nova20:29
*** wolverineav has joined #openstack-nova20:31
*** tssurya has quit IRC20:32
*** ttsiouts has joined #openstack-nova20:36
*** wolverineav has quit IRC20:36
*** awaugama has quit IRC20:36
*** markvoelker has joined #openstack-nova20:41
openstackgerritMerged openstack/os-vif master: Change python3.5 job to python3.7 job on Stein+  https://review.openstack.org/61006820:42
*** tbachman has quit IRC20:46
*** wolverineav has joined #openstack-nova20:50
*** wolverineav has quit IRC20:53
*** wolverineav has joined #openstack-nova20:55
*** erlon has quit IRC20:55
*** tesseract has quit IRC20:55
*** wolverineav has quit IRC20:55
*** wolverineav has joined #openstack-nova20:56
*** jmlowe has quit IRC20:57
*** igordc has joined #openstack-nova20:58
*** ttsiouts has quit IRC21:00
*** ttsiouts has joined #openstack-nova21:01
*** eharney has quit IRC21:03
*** ttsiouts has quit IRC21:05
mriedemi wonder how many operators are trying to recover an instance from a failed resize confirm/revert where the instance is in ERROR status and they fixed something in the compute and just want to retry the confirm/revert operation, but can't b/c the API doesn't allow those operations on an instance in ERROR status. seems weird that the os-resetStatus server action API doesn't allow you to put an instance in 'resized' status agai21:07
mriedemrom error)21:07
mriedemdansmith: ever thunk on that?^21:08
dansmithwell, it would be pretty bad if you put it into resized status when it wasn't,21:08
dansmithand it'd be hard to tell when and when not to do something like that21:08
mriedemif you got through revert_resize on the dest but finish_revert_resize fails on the source, you can maybe hard reboot your way out of it21:09
*** gbarros has joined #openstack-nova21:09
openstackgerritBoris Bobrov proposed openstack/nova master: Don't reference unassigned variable in xml matchers in test  https://review.openstack.org/63796521:09
mriedemif revert_resize on the dest fails, you're kind of stuck there now21:09
mriedemanywho21:09
dansmithright but is it realistic for an operator to know that it failed on finish_revert_resize?21:10
dansmitheven if they fix something,21:10
dansmiththey could reset into resized state from another place where they think it's likely to help and further break stuff21:10
dansmithI mean, I know what you're saying, but.. it's a dangerous tool21:10
mriedemsure, and i wouldn't expect most operators to know that unless they were pretty knowledgeable on low level nova21:10
mriedemwhich their time is probably just better spent on rebuilding the vm21:11
artomAnd, is it needed? I mean, if they have data they want to keep on a volume, they can just attach that to a new instance with the correct flavor?21:11
mriedemand writing an apology to the user21:11
mriedemartom: i'm thinking more about local storage, no volumes21:11
artomSo actually, is a resize destructive in that respect?21:12
mriedemgranted your data should be stored on a persistent volume21:12
mriedemno, the local disks are left intact on the source host when you resize21:12
mriedemthey are destroyed when you confirm the resize to keep the instance on the dest host21:12
mriedemand re-used on the source if you revert21:12
*** markvoelker has quit IRC21:13
artomAnd your idea is - if the resize went to ERROR, rather than just reverting back to the source, allow them to force completion, so to speak?21:13
mriedemnot force completion, retry21:14
mriedembut,21:14
mriedemthat retry could fail if we got part way the first time21:14
mriedemso retrying isn't without risk either21:14
mriedemi.e. revert_resize on dest fails the 2nd time b/c we already deleted some stuff the first time21:14
artomYeah...21:15
artomIn that situation, what's wrong with going through the resize again, after reverting the failed attempt?21:15
mriedemi've just got some sticky logic in the cross-cell revert code where if something fails part way through it's not so good21:15
mriedem"reverting the failed attempt"?21:15
mriedems/reverting/fixing/?21:16
mnasermaybe in an ideal world we can collect more of the cases where finish_revert_resize fails and test those out *before* starting to revert the resize21:16
artomOh. You can't revert out of an ERROR, can you?21:16
mriedemyou could reset status to active and try another resize21:16
mriedemno you can only confirm/revert a server in 'resized' status21:16
mnaserno you cant revert out of ERROR, only RESIZE_CONFIRM i think21:16
mriedemyou can hard reboot / rebuild a server in ERROR21:16
mnasertbh if you're going to hand a tool this dangerous, you might as well as have people login to their dbs and edit stuff manually21:17
mnaserbut thats my drive-by 2 cents21:17
artomWhich *they do*.21:17
artomUnfortunately.21:17
artomAnd not only that, but we help them do it.21:17
mriedemthat kind of goes back to my original question - allowing only setting active or error in os-resetState - i wonder how many operators are manually changing status to something else to retry an operation21:17
mnaserwait os-resetState lets you do more than active and error?21:18
mriedemno21:18
mnaseroo21:18
mriedemin cinder it lets you reset volume status to just about any known volume status i think21:18
mnaseryeah21:18
*** ralonsoh has quit IRC21:19
mnasertbh my workflow has always been "bring something back to as consistent of a state as it is"21:19
mnaserand then use api afterwards21:19
* artom decides to read the cross-cell resize spec.21:20
artomWhoa, "shards cells by hardware generation" is a thing?21:21
*** whoami-rajat has quit IRC21:21
mriedemyou don't need to read the spec, i'm going to go back in my cave21:22
artommriedem, just curious, and procrastinating this downstream thing :)21:23
mriedemfeel free to read it if you want21:23
mriedemit is quite masterful in it's word count21:24
artomYes. I'm a free man in a free country.21:24
mriedemsorry *its i know efried is watching21:24
artomUh huh. "Masterful count" is exactly what I think of you ;)21:24
*** wolverineav has quit IRC21:27
*** wolverineav has joined #openstack-nova21:28
mriedemi guess in the case of normal resize failures, there is stuff in the compute on restart that tries to clean that up,21:29
mriedemwhich isn't as possible with cross-cell since it's orchestrated across 2 cells from the top21:29
mriedembut on revert i think the idea is basically, best effort cleanup from the target and make sure the instance is mapped back to the source so if there is an error the user can work with it there on the source with the original flavor, since they were trying to revert in the first place21:30
*** xek has joined #openstack-nova21:31
artom(This isn't really related or asked for, but I feel that if you're using cells to segregate hardware you're doing it wrong? Wouldn't that what hosts aggregates are for?)21:31
mriedemwhat if you have 10K old hosts and 10K new hosts?21:32
*** gbarros has quit IRC21:32
mriedemyou want them all in the same cell with the same mq/db cluster?21:32
mriedembut 2 aggregates?21:32
mriedemyou can certainly do aggregates, but cells is also for the scaling issue21:32
mriedemcern also segregates by hardware type i believe,21:33
mriedemand by vm/bm flavors21:33
mriedemlots of ways to slice it up21:33
artomYeah, I'd understood that cells sharding was for scale21:33
artomSo if you want to scale different hardware generations/types, it'd be cells for scale, and then in each cell aggregates of hardware21:34
artomSo in your 10K example, it'd be 1K or whatever per cell, and 500/500 old/new hardware21:36
artomBut I guess that ignore the flow of time21:36
artomYou'll have set up your cells with old hardware, not necessarily leaving wiggling room to add new hardware21:36
artomMan, I'm really good at this procrastinating thing21:37
*** _pewp_ has quit IRC21:37
*** _pewp_ has joined #openstack-nova21:38
*** hamzy has quit IRC21:41
* artom heads home21:43
*** wolverineav has quit IRC21:43
*** artom has quit IRC21:47
*** wolverineav has joined #openstack-nova21:47
*** hongbin has quit IRC21:49
*** sdake has joined #openstack-nova21:58
*** dave-mccowan has quit IRC22:07
*** ttsiouts has joined #openstack-nova22:07
*** sdake has quit IRC22:09
*** markvoelker has joined #openstack-nova22:10
*** ttsiouts has quit IRC22:12
*** xek_ has joined #openstack-nova22:16
*** gbarros has joined #openstack-nova22:18
*** xek has quit IRC22:18
*** xek_ has quit IRC22:21
*** rcernin has joined #openstack-nova22:26
*** gbarros has quit IRC22:27
*** gbarros has joined #openstack-nova22:29
openstackgerritsean mooney proposed openstack/os-vif master: remove brctl from vif_plug_ovs  https://review.openstack.org/63682122:30
openstackgerritsean mooney proposed openstack/os-vif master: remove use of brctl from vif_plug_linux_bridge  https://review.openstack.org/63682222:30
*** agopi has quit IRC22:32
openstackgerritEric Fried proposed openstack/nova master: WIP: De-shim PlacementFixture  https://review.openstack.org/63795922:41
*** gbarros has quit IRC22:42
*** markvoelker has quit IRC22:44
*** tbachman has joined #openstack-nova22:47
*** tkajinam has joined #openstack-nova22:55
*** munimeha1 has quit IRC23:02
*** ccamacho has quit IRC23:13
*** ccamacho has joined #openstack-nova23:14
*** eharney has joined #openstack-nova23:26
*** ociuhandu has quit IRC23:30
*** ociuhandu_ has joined #openstack-nova23:30
*** imacdonn has joined #openstack-nova23:32
*** imacdonn_ has joined #openstack-nova23:32
*** imacdonn_ has quit IRC23:32
*** wolverineav has quit IRC23:34
*** wolverineav has joined #openstack-nova23:35
openstackgerritMatt Riedemann proposed openstack/nova master: Add TargetDBSetupTask  https://review.openstack.org/62789223:38
openstackgerritMatt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask  https://review.openstack.org/63158123:38
openstackgerritMatt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask  https://review.openstack.org/63385323:38
openstackgerritMatt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method  https://review.openstack.org/62131323:38
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add prep_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63329323:38
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtDestTask  https://review.openstack.org/62789023:38
openstackgerritMatt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method  https://review.openstack.org/63483223:38
openstackgerritMatt Riedemann proposed openstack/nova master: Add nova.compute.utils.delete_image  https://review.openstack.org/63760523:38
openstackgerritMatt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask  https://review.openstack.org/62789123:38
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add finish_snapshot_based_resize_at_dest compute method  https://review.openstack.org/63508023:38
openstackgerritMatt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask  https://review.openstack.org/63564623:38
openstackgerritMatt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask  https://review.openstack.org/63566823:38
openstackgerritMatt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize()  https://review.openstack.org/63568423:38
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Filter duplicates from compute API get_migrations_sorted()  https://review.openstack.org/63622423:38
openstackgerritMatt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize  https://review.openstack.org/63625323:38
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source  https://review.openstack.org/63705823:38
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask  https://review.openstack.org/63707023:38
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize conductor RPC method  https://review.openstack.org/63707523:38
*** liuyulong has joined #openstack-nova23:40
*** wolverineav has quit IRC23:40
*** jmlowe has joined #openstack-nova23:40
*** markvoelker has joined #openstack-nova23:41
*** mriedem has quit IRC23:41
*** yaawang has quit IRC23:53
*** tbachman has quit IRC23:54

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!