Friday, 2018-01-05

openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in put (2)  https://review.openstack.org/53126600:00
*** esberglu has quit IRC00:04
*** yassine has joined #openstack-nova00:05
*** yassine is now known as Guest1342100:05
*** yangyapeng has joined #openstack-nova00:09
*** jose-phillips has quit IRC00:09
*** jistr has quit IRC00:11
*** jaypipes has quit IRC00:14
*** yangyapeng has quit IRC00:14
*** jose-phillips has joined #openstack-nova00:14
*** jistr has joined #openstack-nova00:16
*** zhurong has joined #openstack-nova00:16
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Microversion 2.58 - Instance actions list pagination  https://review.openstack.org/52860100:17
*** yangyapeng has joined #openstack-nova00:20
*** sdague has quit IRC00:22
*** xnox has quit IRC00:23
*** xnox has joined #openstack-nova00:24
*** xnox has joined #openstack-nova00:24
*** yangyapeng has quit IRC00:25
*** esberglu has joined #openstack-nova00:28
*** esberglu has quit IRC00:29
mriedemalex_xu: takashin: important question in https://review.openstack.org/#/c/330406/78/api-ref/source/parameters.yaml@414800:29
mriedemthis microversion adds a uuid field to the response for GET /os-migrations,00:29
mriedemi suggested we could also take this opportunity to add uuid to the response for GET /servers/{uuid}/migrations as well00:29
mriedemthoughts on that?00:29
*** EmilienM has quit IRC00:30
*** markmcclain has quit IRC00:30
mriedemcomment on the review if you like, i have to make dinner00:30
*** markmcclain has joined #openstack-nova00:32
*** edmondsw has joined #openstack-nova00:36
*** edmondsw has quit IRC00:41
*** jobewan has quit IRC00:44
*** liuzz has joined #openstack-nova00:44
*** chyka has quit IRC00:46
*** dave-mccowan has joined #openstack-nova00:48
*** gyee has quit IRC00:49
*** liangy has quit IRC00:56
*** jose-phillips has quit IRC01:00
*** jose-phillips has joined #openstack-nova01:03
*** markmcclain has quit IRC01:05
*** purplerbot has joined #openstack-nova01:08
*** hshiina has joined #openstack-nova01:09
*** markmcclain has joined #openstack-nova01:13
*** nicolasbock has joined #openstack-nova01:17
*** tuanla____ has joined #openstack-nova01:22
openstackgerritMerged openstack/nova master: Imported Translations from Zanata  https://review.openstack.org/52479501:29
*** dansmith has joined #openstack-nova01:30
*** Apoorva_ has joined #openstack-nova01:33
*** Apoorva has quit IRC01:37
*** Apoorva_ has quit IRC01:38
*** hemna_ has quit IRC01:39
*** hui has joined #openstack-nova01:41
*** jroll has quit IRC01:46
*** jroll has joined #openstack-nova01:51
*** hieulq has quit IRC01:55
*** hoangcx has quit IRC01:55
*** hieulq has joined #openstack-nova01:55
*** hoangcx has joined #openstack-nova01:55
*** tuanla____ has quit IRC01:57
*** tuanla____ has joined #openstack-nova01:57
*** hui is now known as tianhui02:01
*** ljjjustin has joined #openstack-nova02:04
*** tuanla____ has quit IRC02:08
*** tuanla____ has joined #openstack-nova02:08
*** yangyapeng has joined #openstack-nova02:11
*** threestrands_ has joined #openstack-nova02:14
*** threestrands_ has quit IRC02:14
*** threestrands_ has joined #openstack-nova02:14
*** eharney has quit IRC02:16
*** threestrands has quit IRC02:16
*** zzzeek has quit IRC02:19
*** zzzeek has joined #openstack-nova02:20
*** namnh has joined #openstack-nova02:20
*** zzzeek has quit IRC02:20
*** zzzeek has joined #openstack-nova02:21
*** smatzek has joined #openstack-nova02:24
*** edmondsw has joined #openstack-nova02:24
*** edmondsw has quit IRC02:28
*** markmcclain has quit IRC02:29
*** eharney has joined #openstack-nova02:30
*** mriedem has quit IRC02:31
*** markmcclain has joined #openstack-nova02:34
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in put (3)  https://review.openstack.org/53128002:34
*** harlowja has quit IRC02:36
*** Dinesh_Bhor has joined #openstack-nova02:36
*** smatzek has quit IRC02:38
*** markmcclain has quit IRC02:43
*** markmcclain has joined #openstack-nova02:45
*** Tom-Tom has joined #openstack-nova02:51
*** xinliang has quit IRC02:52
*** markmcclain has quit IRC02:53
*** markmcclain has joined #openstack-nova02:55
*** xinliang has joined #openstack-nova03:05
*** Dinesh_Bhor has quit IRC03:12
*** tetsuro has quit IRC03:14
*** nicolasbock has quit IRC03:14
*** yamamoto has joined #openstack-nova03:28
*** links has joined #openstack-nova03:29
*** Apoorva has joined #openstack-nova03:30
*** Dinesh_Bhor has joined #openstack-nova03:32
*** links has quit IRC03:34
*** zhurong has quit IRC03:34
*** eharney has quit IRC03:38
*** eharney has joined #openstack-nova03:39
*** moshele has joined #openstack-nova03:39
*** jobewan has joined #openstack-nova03:44
*** markmcclain has quit IRC03:45
*** markmcclain has joined #openstack-nova03:47
openstackgerritHironori Shiina proposed openstack/nova master: virt/ironic: Implement rescue and unrescue  https://review.openstack.org/41648703:51
*** abhishekk has joined #openstack-nova03:51
*** annp has joined #openstack-nova03:56
*** links has joined #openstack-nova03:56
*** liuzz has quit IRC04:01
*** udesale has joined #openstack-nova04:08
*** liuzz has joined #openstack-nova04:10
*** moshele has quit IRC04:12
*** edmondsw has joined #openstack-nova04:13
*** Dinesh_Bhor has quit IRC04:14
*** mdnadeem has joined #openstack-nova04:15
*** Dinesh_Bhor has joined #openstack-nova04:17
*** edmondsw has quit IRC04:17
*** jobewan has quit IRC04:20
*** Tom-Tom has quit IRC04:21
*** Tom-Tom has joined #openstack-nova04:22
*** Eran_Kuris has quit IRC04:23
*** jobewan has joined #openstack-nova04:23
*** Tom-Tom has quit IRC04:26
*** sree has joined #openstack-nova04:27
*** psachin has joined #openstack-nova04:33
*** eharney has quit IRC04:34
*** Eran_Kuris has joined #openstack-nova04:36
*** chyka has joined #openstack-nova04:57
*** sree_ has joined #openstack-nova04:59
*** sree_ is now known as Guest4579004:59
*** sree has quit IRC05:02
*** chyka has quit IRC05:02
*** sridharg has joined #openstack-nova05:03
*** dave-mccowan has quit IRC05:06
*** liuzz has quit IRC05:10
*** markmcclain has quit IRC05:12
*** markmcclain has joined #openstack-nova05:15
*** ratailor has joined #openstack-nova05:17
*** janki has joined #openstack-nova05:24
*** liuzz has joined #openstack-nova05:24
*** Tom-Tom has joined #openstack-nova05:34
*** jobewan has quit IRC05:35
*** hongbin has joined #openstack-nova05:37
*** Tom-Tom has quit IRC05:38
*** Tom-Tom has joined #openstack-nova05:39
*** Brin has joined #openstack-nova05:45
openstackgerritAndreas Jaeger proposed openstack/os-vif master: zuul: Enable functional tests in gate  https://review.openstack.org/53096105:46
*** Dinesh_Bhor has quit IRC05:49
*** Apoorva has quit IRC05:55
*** Dinesh_Bhor has joined #openstack-nova06:03
*** Dinesh_Bhor has quit IRC06:04
*** liuzz has quit IRC06:07
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in get  https://review.openstack.org/53129906:10
*** threestrands_ has quit IRC06:13
*** tetsuro_ has joined #openstack-nova06:44
openstackgerritTakashi NATSUME proposed openstack/nova master: Remove a duplicate line in a unit test  https://review.openstack.org/53131706:53
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Fix 500 in test_resize_server_negative_invalid_state  https://review.openstack.org/53111707:04
*** claudiub has joined #openstack-nova07:14
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Fix 500 in test_resize_server_negative_invalid_state  https://review.openstack.org/53111707:24
*** armax has quit IRC07:29
*** Dinesh_Bhor has joined #openstack-nova07:32
*** zer0c00l has quit IRC07:35
openstackgerritTetsuro Nakamura proposed openstack/nova master: Add ComputeNode.virt_type field  https://review.openstack.org/53132707:37
openstackgerritTetsuro Nakamura proposed openstack/nova master: Use virt_type in /os-hypervisors/details API  https://review.openstack.org/53132807:37
openstackgerritChen Hanxiao proposed openstack/nova master: conf: hyperv: fix a comment typo  https://review.openstack.org/53132907:37
*** zer0c00l has joined #openstack-nova07:41
*** Dinesh_Bhor has quit IRC07:47
*** zer0c00l has quit IRC07:48
*** zhurong has joined #openstack-nova07:48
openstackgerritMarcin Juszkiewicz proposed openstack/nova master: libvirt: use 'host-passthrough' as default on AArch64  https://review.openstack.org/53096507:48
hrwstephenfin: added test for ^^ - can you take a look?07:49
*** edmondsw has joined #openstack-nova07:49
*** zer0c00l has joined #openstack-nova07:50
*** Dinesh_Bhor has joined #openstack-nova07:52
*** edmondsw has quit IRC07:53
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Add index(instance_uuid, updated_at) on instance_actions table  https://review.openstack.org/53042908:09
*** markvoelker has joined #openstack-nova08:09
*** fragatina has joined #openstack-nova08:09
*** migi has quit IRC08:11
*** tesseract has joined #openstack-nova08:12
*** fragatina has quit IRC08:12
*** fragatina has joined #openstack-nova08:13
*** gouthamr has quit IRC08:13
kashyaphrw: Oh, yes, I thought of the test in your change, but forgot to mention it.  Cool that you've worked it out.08:14
hrwkashyap: I was aware that test is needed. Just had no idea how to write it. Kevin wrote it for me.08:14
*** yangyapeng has quit IRC08:15
*** yangyapeng has joined #openstack-nova08:15
kashyapI see, ah, missed the co-author bit.08:15
kashyaphrw: I myself struggle with the unit tests as I don't look at it terribly often08:15
hrwkashyap: Kevin knows Nova code. I don't08:15
* kashyap nods08:16
*** zhurong has quit IRC08:16
*** sahid has joined #openstack-nova08:21
*** zhurong has joined #openstack-nova08:27
*** rcernin has quit IRC08:28
hrwmailed ML08:28
*** fragatin_ has joined #openstack-nova08:29
*** fragatina has quit IRC08:31
*** fragatin_ has quit IRC08:33
*** chyka has joined #openstack-nova08:33
*** fragatina has joined #openstack-nova08:33
*** Dinesh_Bhor has quit IRC08:36
*** fragatina has quit IRC08:37
*** chyka has quit IRC08:38
*** fragatina has joined #openstack-nova08:42
*** Brin has quit IRC08:43
*** markvoelker has quit IRC08:44
*** Dinesh_Bhor has joined #openstack-nova08:47
*** migi has joined #openstack-nova08:51
*** moshele has joined #openstack-nova08:52
*** armax has joined #openstack-nova08:55
openstackgerritYikun Jiang (Kero) proposed openstack/nova master: Add pagination and Changes-since filter support for os-migrations.  https://review.openstack.org/33040608:57
*** Tom-Tom has quit IRC08:59
*** Tom-Tom has joined #openstack-nova09:00
*** moshele has quit IRC09:12
*** armax has quit IRC09:14
*** johnthetubaguy has quit IRC09:17
*** johnthetubaguy has joined #openstack-nova09:18
stephenfinhrw: Sure09:18
hrwstephenfin: thanks.09:19
hrwstephenfin: hope that someone will take a look too. Not adding reviewers as I do not know who to add09:19
* hrw back in ~4009:21
*** fragatina has quit IRC09:21
*** fragatina has joined #openstack-nova09:22
*** derekh has joined #openstack-nova09:23
openstackgerritChangBo Guo(gcb) proposed openstack/nova master: Use method validate_integer from oslo.utils  https://review.openstack.org/52862009:29
*** takashin has left #openstack-nova09:30
*** lpetrut has joined #openstack-nova09:31
*** lucas-afk is now known as lucasagomes09:33
*** udesale__ has joined #openstack-nova09:33
*** cristicalin has joined #openstack-nova09:34
*** udesale has quit IRC09:35
*** edmondsw has joined #openstack-nova09:37
*** cristicalin has quit IRC09:39
*** markvoelker has joined #openstack-nova09:41
*** edmondsw has quit IRC09:42
*** liuyulong has joined #openstack-nova09:44
hrwstephenfin: by 'link to your blog' you mean https://marcin.juszkiewicz.com.pl/2018/01/04/today-i-was-fighting-with-nova-no-idea-who-won/ link?09:50
openstackgerritTetsuro Nakamura proposed openstack/nova master: Fix hypervisor_type in /os-hypervisors/details  https://review.openstack.org/53132809:50
openstackgerritTetsuro Nakamura proposed openstack/nova master: [libvirt] Filter hypervisor_type by virt_type  https://review.openstack.org/53134709:50
stephenfinhrw: Yup09:51
hrwok09:51
*** claudiub|2 has joined #openstack-nova09:51
*** hongbin has quit IRC09:52
openstackgerritMarcin Juszkiewicz proposed openstack/nova master: libvirt: use 'host-passthrough' as default on AArch64  https://review.openstack.org/53096509:53
hrwreno updated, link added09:53
*** claudiub has quit IRC09:54
kashyaphrw: Thanks for the quick feedback09:54
*** fragatina has quit IRC09:54
kashyaps/feedback/response/09:54
hrwkashyap: it's my job ;D09:55
hrwkashyap: and I prefer to have patches quickly updated to keep reviewers attention09:55
kashyaphrw: Is this pressing?  If so you have to state it, so reviewers calibrate their queue accordingly :-)09:56
kashyapThose who can +2, i.e.09:56
hrwkashyap: it would be lovely to get both my patches into Queens. That level of priority.09:56
kashyapYeah, it should.  Upstream is not _that_ terribly slow for self-contained stuff like this.09:57
kashyapThis is a straight-forward one.  No delay should be done in merging this.09:57
hrwWe have workaround for cpu_mode already merged into Kolla and I know that it is present in some other ways of deploying too.09:57
hrwUEFI one can not be workarounded09:59
* kashyap nods10:00
hrwadded some random core reviewers to both patches10:01
*** sean-k-mooney has joined #openstack-nova10:03
sean-k-mooneystephenfin: o/ happy new year10:03
stephenfinsean-k-mooney: o/ ditto10:03
sean-k-mooneysee my comment on https://review.openstack.org/#/c/530961/210:04
stephenfinsean-k-mooney: Hahaha10:04
stephenfinThis is what I'm currently reading10:04
stephenfinhttps://docs.python.org/2/library/subprocess.html#subprocess.Popen.communicate10:04
stephenfinI was trying to capture stderr/stdout to see if that was indeed the issue (I suspected it was)10:05
stephenfinsean-k-mooney: I'm thinking we just configure that to 'python -m privsep.[path_to_module]'10:06
*** Guest45790 has quit IRC10:06
*** sree has joined #openstack-nova10:07
sean-k-mooneystephenfin: the test rodoflo is adding will need at least cap_net_admin to run so it will need to launch privesp with sudo to get that10:07
*** sgordon has quit IRC10:08
*** Dinesh_Bhor has quit IRC10:09
stephenfinRight, I meant 'sudo python -m' as I thought we'd keep the virtualenv-configured Python, but we obviously won't actually10:09
sean-k-mooneystephenfin: i think we can alter the config fuction here https://review.openstack.org/#/c/484386/30/os_vif/tests/functional/base.py10:09
*** sree has quit IRC10:09
*** sree has joined #openstack-nova10:09
sean-k-mooneystephenfin: if we addin the a override for the oslo privsep helper_command form sudo privsep-helper to sudo -EH privsep-helper its shoudl resolve the issue10:10
* stephenfin checks10:11
sean-k-mooneytechnicall i theink just the -E is need to keep the virtual env intact but i always use -EH just to be safe10:11
sean-k-mooneymy spelling is on point as always...10:12
*** tetsuro_ has left #openstack-nova10:12
*** markvoelker has quit IRC10:15
*** hoangcx has quit IRC10:15
*** jafeha has quit IRC10:17
stephenfinsean-k-mooney: Turns out PATH isn't passed through via '-E'. Something about secure_path. Who knew!10:21
*** damien_r has joined #openstack-nova10:26
*** ljjjustin has quit IRC10:27
*** udesale has joined #openstack-nova10:28
*** cdent has joined #openstack-nova10:29
*** gcb has quit IRC10:29
*** udesale__ has quit IRC10:29
*** gszasz has joined #openstack-nova10:30
sean-k-mooneystephenfin: really ok... im pulling down the patch in a clean vm and ill see if i can reporduce the gate issue10:32
*** namnh_ has joined #openstack-nova10:33
*** namnh_ has quit IRC10:34
sean-k-mooneyyep same issue10:35
*** tuanla____ has quit IRC10:35
*** sree has quit IRC10:35
sean-k-mooneyand the privsep-helper is in .tox/functional/bin/privsep-helper as i taught but not in /bin10:36
*** sree has joined #openstack-nova10:36
*** namnh has quit IRC10:37
*** sambetts|afk is now known as sambetts10:37
stephenfinsean-k-mooney: This is as far as I've got http://paste.openstack.org/show/638995/10:38
stephenfinsean-k-mooney: But that's yielding this, ostensibly because we're using absolute paths to a package that is only installed in the virtualenv http://paste.openstack.org/show/638997/10:40
sean-k-mooneythat worked for me10:40
stephenfinIt did?10:40
stephenfinFailing for me on CentOS :(10:40
sean-k-mooneywell it passed all the tests10:40
sean-k-mooneyhttp://paste.openstack.org/show/638998/10:41
*** moshele has joined #openstack-nova10:42
*** dtantsur|afk is now known as dtantsur10:42
stephenfinHow weird10:42
*** sree has quit IRC10:42
*** sdague has joined #openstack-nova10:43
openstackgerritsahid proposed openstack/nova master: libvirt: disconnect volume from host during detach  https://review.openstack.org/51500810:43
stephenfinYou've other modifications though, to say that you have logging10:43
sean-k-mooneyno i just applied what you gave10:43
*** sree has joined #openstack-nova10:43
sean-k-mooneyhttp://paste.openstack.org/show/638999/10:43
sean-k-mooneythat was applied directly on top of your zuul patch10:44
*** moshele has quit IRC10:44
sean-k-mooney15264957c475a6ecc1648a9ca2f04c9beb95f98d10:44
sean-k-mooneyi can spin up a centos vm and test it there again if that helps10:45
stephenfinIf you wouldn't mind10:45
stephenfinI'll push this now for review/rework10:45
stephenfinSee what the gate thinks too10:45
*** nore_rabel has joined #openstack-nova10:46
openstackgerritStephen Finucane proposed openstack/os-vif master: zuul: Enable functional tests in gate  https://review.openstack.org/53096110:47
openstackgerritStephen Finucane proposed openstack/os-vif master: Configure privsep binary  https://review.openstack.org/53135810:47
sean-k-mooneyno prob it will only take like 5min to get the env setup i mainly the proxies...#10:47
*** sree has quit IRC10:48
*** abhishekk has quit IRC10:49
*** sree has joined #openstack-nova10:52
*** szaher has joined #openstack-nova10:54
*** nore_rabel is now known as rabel10:55
*** zhurong has quit IRC10:57
openstackgerritMatthew Booth proposed openstack/nova master: Remove redundant swap_volume tests  https://review.openstack.org/53117910:57
openstackgerritMatthew Booth proposed openstack/nova master: Don't generate fake disk_info in swap_volume  https://review.openstack.org/53078710:57
openstackgerritMatthew Booth proposed openstack/nova master: Local disk serial numbers for the libvirt driver  https://review.openstack.org/52938010:57
*** pcaruana has joined #openstack-nova11:01
*** udesale__ has joined #openstack-nova11:02
*** sgordon has joined #openstack-nova11:03
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Refactor encryptor attach and detach calls  https://review.openstack.org/46024311:04
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Attach and detach encryptors during swap_volume  https://review.openstack.org/53123311:04
*** udesale has quit IRC11:05
sean-k-mooneystephenfin: v2 fails v3 gives me the No module named internal.command.ip.test_impl_pyroute2 issue but only on centos. on ubunutu v3 works11:05
stephenfinTry this on Ubuntu -> sudo -E su -c 'echo $PATH'11:06
*** rabel has quit IRC11:06
*** Tom-Tom_ has joined #openstack-nova11:07
stephenfinCentOS has 'Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin'. I imagine Ubuntu might not11:07
sean-k-mooney/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games vs /sbin:/bin:/usr/sbin:/usr/bin on centos11:07
stephenfinHmm, still doesn't pass through the virtualenv directory. I'd expected it would11:08
sean-k-mooneywell i was not in a venves ill try that instead11:08
stephenfinOh, yeah :D source .tox/functional/bin/activate first :)11:09
*** Tom-Tom has quit IRC11:09
sean-k-mooneyi think " tox -e venv --  sudo -E su -c 'echo $PATH'" will also work11:09
*** Tom-Tom has joined #openstack-nova11:10
*** sree has quit IRC11:10
*** purplerbot has quit IRC11:11
sean-k-mooneystephenfin: not much is jumping out at me http://paste.openstack.org/show/639010/11:11
*** markvoelker has joined #openstack-nova11:11
*** purplerbot has joined #openstack-nova11:12
*** Tom-Tom_ has quit IRC11:13
*** sree has joined #openstack-nova11:14
stephenfinsean-k-mooney: Is that any help to us though?11:16
*** jafeha has joined #openstack-nova11:17
*** Tom-Tom has quit IRC11:18
*** sree has quit IRC11:18
*** rabel has joined #openstack-nova11:20
*** sridharg has quit IRC11:21
*** sree has joined #openstack-nova11:22
sean-k-mooneyi dont think its related to the system path.11:23
*** udesale has joined #openstack-nova11:27
*** sridharg has joined #openstack-nova11:28
*** udesale__ has quit IRC11:29
*** khappone has quit IRC11:29
*** fragatina has joined #openstack-nova11:32
*** fragatina has quit IRC11:32
*** fragatina has joined #openstack-nova11:33
*** Drankis has joined #openstack-nova11:33
sean-k-mooneystephenfin: i think this has someithing to do with the python path. specifically i do not think the privsep deamon spawnded via sudo -E on cetos is using the python path that with the virtual_env11:34
*** nicolasbock has joined #openstack-nova11:35
stephenfinYeah, I've tried setting 'PYTHONPATH' but haven't had any luck so far11:35
stephenfinI wonder if it's using a different Python interpreter too?11:35
sean-k-mooneyslightly 2.7.12 on ubuntu and 2.7.5 on centos11:35
stephenfinNo, I meant the one in the virtualenv vs the system one11:36
*** sree has quit IRC11:36
sean-k-mooneyoh you can force that but no not by default11:36
stephenfinI'm not actually sure how virtualenv configures PYTHONPATH. I was assuming it was done by way of the environment variable but apparently not11:37
sean-k-mooneyif i activeate teh virutal enve and run python i can run "from os_vif.internal.command.ip import impl_pyroute2" but it dose not work if i do sudo -E python11:37
stephenfinYeah, I tried 'sudo PYTHONPATH=.tox/functional/lib/python2.7/site-packages' too, but to no avail11:38
sean-k-mooneyhum that is the same on ubuntho though11:38
*** Tom-Tom has joined #openstack-nova11:40
*** sree has joined #openstack-nova11:40
*** Tom-Tom_ has joined #openstack-nova11:42
sean-k-mooneystephenfin: well python -c "import sys; print('\n'.join(sys.path))" is pretty clear. with sudo -E teh venv disapears11:43
*** udesale has quit IRC11:44
*** Tom-Tom has quit IRC11:44
*** markvoelker has quit IRC11:45
stephenfincdent: Finished https://review.openstack.org/#/c/513526/. One potential follow-up in there11:46
cdentstephenfin: roger that. I think that block being where it is is a holdover from an earlier version where the logic was somewhat different and a rebase required shifting the logic a bit11:48
cdentactually...11:48
* cdent re-learns to read11:48
cdentno, that statement's right11:50
*** tbachman has quit IRC11:50
*** Tom-Tom_ has quit IRC11:53
*** sree has quit IRC11:53
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Attach and detach encryptors during swap_volume  https://review.openstack.org/53123311:54
*** Tom-Tom has joined #openstack-nova11:55
*** purplerbot has quit IRC11:56
*** purplerbot has joined #openstack-nova11:56
*** sree has joined #openstack-nova11:56
*** annp has quit IRC12:00
*** chyka has joined #openstack-nova12:09
*** purplerbot has quit IRC12:11
*** purplerbot has joined #openstack-nova12:11
*** liuyulong has quit IRC12:13
*** Tom-Tom has quit IRC12:14
*** chyka has quit IRC12:14
*** Tom-Tom has joined #openstack-nova12:14
*** sree has quit IRC12:16
*** Tom-Tom has quit IRC12:19
*** Tom-Tom has joined #openstack-nova12:21
*** sree has joined #openstack-nova12:22
*** smatzek has joined #openstack-nova12:23
*** jamesdenton has quit IRC12:25
*** Tom-Tom has quit IRC12:26
*** fragatina has quit IRC12:30
*** sree has quit IRC12:30
*** sree has joined #openstack-nova12:34
sean-k-mooneyefried: o/12:35
sean-k-mooneyefried: is https://review.openstack.org/#/c/531260/ the top patch on the nested resouce provides series?12:39
*** sree has quit IRC12:39
*** moshele has joined #openstack-nova12:42
*** markvoelker has joined #openstack-nova12:42
*** sree has joined #openstack-nova12:42
*** damien_r1 has joined #openstack-nova12:43
*** lucasagomes is now known as lucas-hungry12:46
*** damien_r has quit IRC12:46
*** sree has quit IRC12:47
*** janki has quit IRC12:50
*** moshele has quit IRC12:51
*** ratailor has quit IRC12:52
*** tbachman has joined #openstack-nova12:53
*** tbachman_ has joined #openstack-nova12:56
*** sree has joined #openstack-nova12:56
*** mdnadeem has quit IRC12:57
*** pooja_jadhav has quit IRC12:57
*** hoonetorg has quit IRC12:57
*** tbachman has quit IRC12:58
*** tbachman_ is now known as tbachman12:58
*** hshiina has quit IRC12:59
*** dillaman has joined #openstack-nova13:03
*** sree has quit IRC13:05
* mdbooth wishes there was a way to mark a -1 review as "I looked at that, and it was idiotic"13:08
*** psachin has quit IRC13:09
* mdbooth curses drive-by low-energy reviews13:09
*** sree has joined #openstack-nova13:11
*** edmondsw has joined #openstack-nova13:13
*** markvoelker has quit IRC13:15
*** edmondsw has quit IRC13:17
*** fragatina has joined #openstack-nova13:19
*** sree has quit IRC13:20
*** dave-mccowan has joined #openstack-nova13:22
*** sree has joined #openstack-nova13:23
*** eharney has joined #openstack-nova13:24
*** fragatina has quit IRC13:25
*** sree has quit IRC13:27
*** sree has joined #openstack-nova13:28
*** slunkad has joined #openstack-nova13:32
*** markvoelker has joined #openstack-nova13:36
mhenkelhi All13:40
mhenkelI am trying to use the haproxy template similar to this one: https://review.openstack.org/#/c/474107/8/puppet/services/keystone.yaml13:41
mhenkelhowever, when I define server_names: "%{hiera('contrail_config_node_names')}" haproxy puppet fails with:13:42
mhenkel"Error: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/File[/etc/haproxy/haproxy.cfg]/content: change from {md5}1f337186b0e1ba5ee82760cb437fb810 to {md5}d12e9a54e85e6bb63391d61cf9eb1da6 failed: Execution of '/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg20180105-12-13t72r6 -c' returned 1: [ALERT] 004/024209 (275) : parsing [/etc/haproxy/haproxy.cfg20180105-12-13t72r6:53] :13:42
mhenkelcharacter '[' is not permitted in server name '[\"overcloud-contrailcontroller-0.internalapi.localdomain\"]'."13:42
*** david_10 has joined #openstack-nova13:46
*** tpatzig_3 has joined #openstack-nova13:46
*** carthaca_2 has joined #openstack-nova13:46
*** tpatzig_8 has joined #openstack-nova13:46
*** mkoderer_ has joined #openstack-nova13:46
mhenkelany idea why %{hiera('contrail_config_node_names')} returns [\"overcloud-contrailcontroller-0.internalapi.localdomain\"] instead of overcloud-contrailcontroller-0.internalapi.localdomain ?13:46
*** bhagyashri_s has joined #openstack-nova13:47
*** links has quit IRC13:47
stephenfinmhenkel: Sounds like something for #tripleo or #puppet13:48
*** sean-k-mooney has quit IRC13:49
*** sean-k-mooney has joined #openstack-nova13:49
*** bhagyashris has quit IRC13:49
*** tonyb has quit IRC13:49
*** tonyb has joined #openstack-nova13:50
*** mkoderer_3 has quit IRC13:51
*** tpatzig_2 has quit IRC13:51
*** david_ has quit IRC13:51
*** carthaca_ has quit IRC13:51
*** tpatzig_ has quit IRC13:51
*** jistr is now known as jistr|biab13:51
artommdbooth, I did have an idea of "meta-reviews" a while ago13:53
artomThe (very legitimate) response was "make it happen"13:53
*** edmondsw has joined #openstack-nova13:53
artomTo which my also very legitimate reaction was "ugh"13:53
artomAnd nothing came of it :/13:53
mdboothartom: My concern's more that I suspect people have review lists like 'open reviews with +1 from Zuul and no -1s'13:54
mdboothAnd my patch would now fall foul of that for a completely dumb reason13:54
*** sridharg has quit IRC13:54
mdboothAn alternate idea would be the ability to mark a -1 as 'I looked at that, and have no intention of addressing it'.13:55
*** mlavalle has joined #openstack-nova13:55
mdboothThen people might filter on unaddressed -1s instead.13:55
mhenkelstephenfin: sorry, wrong channel13:56
mdboothartom: Meta reviews would be good, though.13:57
artommdbooth, also, I suspect just showing who the -1 is from would help that13:58
mdboothAlthough, who would do them?13:58
*** READ10 has quit IRC13:59
*** lyan has joined #openstack-nova13:59
stephenfinmdbooth, artom: If you're using the dashboards from gerrit-dash-creator, they shouldn't matter. The only -1s used to filter stuff there are core ones https://github.com/openstack/gerrit-dash-creator/tree/master/dashboards14:00
stephenfinJust as an aside14:00
mdboothstephenfin: Ah, that's awesome14:00
*** sree has quit IRC14:00
mdboothWell, somewhat elitist, but better that than exclude based on drive-bys14:00
mdbooth;)14:01
*** sree has joined #openstack-nova14:01
stephenfinElitist, but the best we've got ;)14:01
openstackgerritLajos Katona proposed openstack/nova master: Deduplicate aggregate notification samples  https://review.openstack.org/53116214:03
*** sree has quit IRC14:05
hrwspeaking of reviews... anyone with free time to take a look at two simple aarch64 related patches? https://review.openstack.org/#/c/530965/ takes care of cpu_mode and https://review.openstack.org/#/c/489951/ makes aarch64 bootable out-of-the-box14:06
*** claudiub|2 is now known as claudiub14:08
*** edleafe is now known as figleaf14:11
*** mriedem has joined #openstack-nova14:12
mriedemkashyap: bad news, the shareable disk flag doesn't work when using raw images either http://paste.openstack.org/show/639185/14:16
efriedsean-k-mooney Howdy.  That's the top patch in *one* of the nrp series, yes.14:21
*** efried is now known as fried_rice14:23
ildikovmriedem: kashyap: :(14:25
openstackgerritLajos Katona proposed openstack/nova master: Deduplicate service status notification samples  https://review.openstack.org/53138114:25
openstackgerritStephen Finucane proposed openstack/nova master: Add debug output for selected page size  https://review.openstack.org/53066214:25
ildikovmriedem: which versions gave you that error?14:26
ildikovmriedem: I also saw this 'qcow2 would suffer metadata corruption in such case' in the latest comment of this bug: https://bugzilla.redhat.com/show_bug.cgi?id=141525014:28
openstackbugzilla.redhat.com bug 1415250 in openstack-nova "QEMU image file locking (RHOS)" [High,Post] - Assigned to lyarwood14:28
*** smatzek has quit IRC14:31
stephenfinmdbooth: What's the name of the pattern where you create an instance of object() and use that as a unique identifier?14:32
*** ljjjustin has joined #openstack-nova14:32
*** smatzek has joined #openstack-nova14:32
mdboothstephenfin: I don't know. I stole it from zzzeek14:33
mdboothIt's a sentinel, though, so that might be what it's called?14:33
stephenfinYup, sentinel is the word I was looking for14:34
* zzzeek wouldnt have thought of that14:35
*** smatzek has quit IRC14:37
mriedemildikov: this is still qemu 2.10 and libvirt 3.614:38
*** esberglu has joined #openstack-nova14:39
mriedemkashyap was saying the libvirt devs were saying that qcow2 wouldn't work before libvirt 3.10 for shareable disks but raw should work, so this test run is setting CONF.libvirt.images_type=raw and CONF.use_cow_images=False14:39
mriedembut still doesn't work14:39
mnaserdid i hit a really weird race condition possibly...? "Conflict updating instance e8cc3ace-2441-413e-bee7-d3a667e3298f. Expected: {'task_state': [u'migrating']}. Actual: {'task_state': None}"14:40
mriedempossibly, might depend on what it's trying to change the task_state to14:41
mnaserreset-state --active and doing a live-migration again triggers it.. let me check14:41
Roamer`okay, so I'll try to also reach johnthetubaguy through a couple of other channels, but maybe I should also try to start a conversation here... What do people think about adding a pretty much trivial StorPool volume attachment driver as in https://review.openstack.org/140733/ ?  I know that milestone 1 has passed and we're actually getting pretty close to milestone 2, but the Cinder and os-brick dri14:41
Roamer`vers made it in last week, so right now we're in a funny situation where the main use case for the StorPool driver still needs manual patching.  There's actually a major downstream OpenStack distributor with a client that wants to use StorPool, and the distributor would strongly prefer if all three StorPool drivers made it into OpenStack... so is there a chance for that? :)14:42
mdboothmriedem: I don't think those config params would affect volumes anyway fwiw.14:42
*** esberglu_ has joined #openstack-nova14:42
mriedemwas wondering that,14:43
mriedemi guess those are only used for ephemeral imagebackend devices14:43
mdboothYep14:43
mriedemgah14:43
mriedemok14:43
*** jmlowe has quit IRC14:43
mriedemalright, trying again w/o pike uca so we use qemu 2.514:43
mriedemyikun: i think we should probably just add uuid to the response for the server migrations API in this change as well https://review.openstack.org/#/c/330406/ - is that something you think you'll have time for? if not, i can add that part of the change14:45
*** esberglu has quit IRC14:45
*** jmlowe has joined #openstack-nova14:45
*** esberglu_ is now known as esberglu14:45
*** jmlowe has quit IRC14:48
*** hongbin has joined #openstack-nova14:48
*** jaypipes has joined #openstack-nova14:48
*** burt has joined #openstack-nova14:50
mlavallemriedem, hongbin: https://bugs.launchpad.net/neutron/+bug/1718605 is approved14:50
openstackLaunchpad bug 1718605 in neutron "[RFE] Support sub-string matching when filtering port with IP address" [Wishlist,Triaged] - Assigned to hongbin (hongbin034)14:50
*** jaypipes is now known as leakypipes14:51
hongbinmlavalle: great. thanks14:51
*** nicolasbock has quit IRC14:51
*** lucas-hungry is now known as lucasagomes14:52
ildikovmriedem: got it, sadness, I guess it means we're falling back to play with the versions in Devstack then14:55
mriedemildikov: yeah14:56
mriedemmlavalle: nice14:56
fried_ricefigleaf What's the status of the alternate hosts series (for purposes of the sched mtg agenda)?14:57
fried_ricefigleaf Are there open patches?14:58
fried_ricefigleaf Or just that bug?14:58
* fried_rice bonks self, goes to look at the bug...14:58
mriedemfried_rice: i can tell you15:01
fried_riceTell me, tell me.15:01
mriedemresize is the last piece https://review.openstack.org/#/c/526436/15:01
mriedembut while figleaf was testing that, he found a race15:01
mriedemso he's working on fixing that race15:01
mriedemhttps://bugs.launchpad.net/nova/+bug/174112515:01
openstackLaunchpad bug 1741125 in OpenStack Compute (nova) "Instance resize always fails when rescheduling" [High,Confirmed] - Assigned to Ed Leafe (ed-leafe)15:01
*** smatzek has joined #openstack-nova15:02
fried_ricemriedem Thanks.  Is the forked bug https://bugs.launchpad.net/nova/+bug/1741307 also related to alt hosts?15:03
openstackLaunchpad bug 1741307 in OpenStack Compute (nova) "Resize always fails when using the CachingScheduler" [High,In progress] - Assigned to Matt Riedemann (mriedem)15:03
mriedemfried_rice: not to alternate hosts, just something i noticed when he pointed out the reschedule issue with resize15:03
fried_ricecool, thanks mriedem15:03
mriedemresize just doesn't work with the caching scheduler, period15:04
melwittmriedem: do you think this libvirt patch has had enough bake time to propose backports now? https://review.openstack.org/#/c/400384/15:04
mriedembecause of the migration allocation swap stuff15:04
mriedemmelwitt: heh, can you give it about another 18 months for us to know?15:04
mriedem:)15:04
*** ljjjustin has quit IRC15:04
*** jistr|biab is now known as jistr15:04
melwittD:15:04
mriedemmelwitt: honestly we probably won't know until people actually upgrade to start using that, and mnaser will probably be the first one once queens is released15:05
mriedemmelwitt: i'm fine with doing backports though15:05
mriedemjust not to newton15:05
mriedemildikov: the devstack patch is now just enablement https://review.openstack.org/#/c/266633/15:06
mnaseri'll gladly take release notes for us :D15:06
mriedemthe patch on top of that turns on multiattach and disables the pike uca15:06
melwittmriedem: okay, cool. I shall propose them then. thanks15:07
*** sree has joined #openstack-nova15:07
*** hoonetorg has joined #openstack-nova15:10
mnaserif there's any stable cores that can have a look at this backport: https://review.openstack.org/#/c/52938515:13
*** ljjjustin has joined #openstack-nova15:17
openstackgerritLajos Katona proposed openstack/nova master: Deduplicate service status notification samples  https://review.openstack.org/53138115:17
mriedemdansmith: ^15:17
dansmithmriedem: dun already got it15:17
mriedemthe stable thing, not the notificatoins thing15:17
mriedemsympatico15:17
dansmithbottom one needs a recheck?15:17
lyarwoodit already has15:18
dansmithah nm,15:18
dansmithyeah15:18
lyarwoodshould be in the gate by now15:18
dansmithnot quite15:18
mriedemstill check queue, but going15:18
lyarwoodurgh15:18
mriedemdansmith: btw, i had to fix a test in https://review.openstack.org/#/c/530982/15:18
mriedemlyarwood: since you're around https://review.openstack.org/#/q/I1aa3ca6cc70cef65d24dec1e7db9491c9b73f7ab15:19
mriedemthat should be the final thing for newton15:19
mriedemmelwitt: this is a simple bp driver feature parity change https://review.openstack.org/#/c/448034/15:20
mriedemneeds final +215:20
lyarwoodmriedem: ack15:20
melwittmriedem: ack, will review15:20
mriedemthanks15:20
*** sree has quit IRC15:21
mriedemi see the allocation candidates limit change is also approved https://review.openstack.org/#/c/513526/ - nice, might get at least 2 bps merged today15:21
openstackgerritLajos Katona proposed openstack/nova master: Deduplicate service status notification samples  https://review.openstack.org/53138115:21
*** gouthamr has joined #openstack-nova15:22
mriedemandreykurilin: can you take a look at https://review.openstack.org/#/c/528128/ ? it will close out that bp and it's holding up other microversion changes in novaclient15:22
ildikovmriedem: ok, tnx15:27
openstackgerritLajos Katona proposed openstack/nova master: Deduplicate aggregate notification samples  https://review.openstack.org/53116215:27
*** weshay is now known as weshay_interview15:28
mriedemstephenfin: http://logs.openstack.org/50/530950/1/check/legacy-tempest-dsvm-py35/f814425/logs/screen-n-novnc.txt.gz?level=TRACE#_Jan_04_17_19_48_24267915:30
mriedemstephenfin: maybe try using encodeutils.safe_encode here? https://review.openstack.org/#/c/345399/37/nova/console/securityproxy/rfb.py15:30
stephenfinmriedem: I can't catch a break with that thing :D That's a good idea though15:31
* stephenfin is currently deploying devstack for the third time today because he forgot to enable Python 315:31
stephenfinsilly stephenfin15:31
*** jmlowe has joined #openstack-nova15:32
sean-k-mooneyfried_rice: thanks. ya i noticed that there seemed to be multiple nrp patch series interwoven. i think https://review.openstack.org/#/c/384807/ is the patch i need to start using nrp to model hw acclerated vhost-vfio interfaces.15:37
*** jafeha has quit IRC15:37
*** jafeha has joined #openstack-nova15:39
fried_ricesean-k-mooney "hw accelerated vhost-vfio interfaces".  Sure.15:41
mriedemi can't wait until sean-k-mooney stabs one of you guys15:42
sean-k-mooneyfried_rice: they are hardware offloaded port that use vfio-mdev instead of sriov to offload the dataplane and vhost to offload the control plane15:42
*** liverpooler has joined #openstack-nova15:43
fried_ricesean-k-mooney Okay, cool.  So how are you planning to represent that from libvirt without using update_provider_tree?15:43
hrwmelwitt: thanks for +2 +W :)15:43
sean-k-mooneyfried_rice: step too is the libvirt xml generation but effectivly i am planning on having neutorn create teh resouce providers in placement and using a new vnic type to generate a request to placement for an allocation of the custom resouce class.15:45
mriedemjohnthetubaguy: is there some known thing with using the cachingscheduler where the compute services don't check in on time? or you have to adjust the compute service heartbeat?15:45
johnthetubaguymriedem: yeah, that is a thing15:46
melwitthrw: thanks for enhancing the commit message and the bug, makes it a lot easier to review15:46
fried_ricesean-k-mooney This in Queens??15:46
mriedemjohnthetubaguy: orly15:46
mriedemyeah because i'm seeing this in the scheduler logs15:46
mriedemJan 04 20:06:18.133057 ubuntu-xenial-rax-ord-0001690099 nova-scheduler[9487]: WARNING nova.scheduler.filters.compute_filter [None req-370806fa-5640-485d-adc2-21808de9ccb4 tempest-MigrationsAdminTest-1101936831 tempest-MigrationsAdminTest-1101936831] (ubuntu-xenial-rax-ord-0001690099, ubuntu-xenial-rax-ord-0001690099) ram: 6446MB disk: 58368MB io_ops: 15 instances: 15 has not been heard from in a while15:46
mriedembut no errors in the n-cpu logs15:46
hrwmelwitt: o yes ;D15:46
hrwmelwitt: can you take a look at https://review.openstack.org/#/c/530965/ too?15:46
johnthetubaguymriedem: if your cache interval is larger than than how long you thing it is dead, it starts to ignore some of your nodes randomly15:46
mriedemah15:46
hrwmelwitt: I am open for any questions15:46
mnasermriedem: i think i found out how i got that weird state.  doing a host-evacuate-live sends a live-migration request for all instances, as nova processes them one by one, the states of all vms in the compute host are in "migrating".. but then its possible that a delete request comes in while the live migration is happening.15:46
sean-k-mooneyfried_rice: not upstream, libvirt and qemu dont even have support for vhost-vfio yet. way to early to propose upstrem in openstack15:47
melwitthrw: I'll add it to my list15:47
hrwmelwitt: thanks15:47
hrwmelwitt: with those two merged getting openstack on aarch64 is much easier ;D15:47
mnaserin the rpc code, there doesn't seem to be any handling if the VM is in migrating state15:48
sean-k-mooneyfried_rice: i would like to have a poc working ptg/early rocky timeframe but i doubt the dependcies will align for it to land even in rocky15:48
mnaserfor deletes15:48
hrwmelwitt: uefi change is most important as with it in nova we cut 90% of users questions ;d15:48
kashyapmriedem: Hey was AFK15:48
fried_ricesean-k-mooney Okay, gotcha.  You going to be around for the sched meeting on Monday?  This is exactly the kind of rationale cdent and Radoslav were bringing up in https://review.openstack.org/#/c/526539/ (to pose the question, "Why are we artificially restricting in-tree virt drivers from talking directly to placement?"15:49
kashyapReading the scroll15:49
fried_rice)15:49
mriedemjohnthetubaguy: i don't see a specific config option for how long the cachingscheduler caches it's host state information15:49
mriedemis that somewhere outside the actual CachingScheduler class?15:49
sean-k-mooneyfried_rice: am what time is it?( checking meeting page)15:49
johnthetubaguymriedem: ah, I think its in scheduler manager, sadly15:50
johnthetubaguyjust digging15:50
mriedem    @periodic_task.periodic_task(spacing=CONF.scheduler.periodic_task_interval,15:50
mriedem                                 run_immediately=True)15:50
mriedem    def _run_periodic_tasks(self, context):15:50
mriedem        self.driver.run_periodic_tasks(context)15:50
mriedem?15:50
johnthetubaguymriedem, yeah https://github.com/openstack/nova/blob/d19e1252d5f2f8e8b54a276003f842ee00641026/nova/scheduler/manager.py#L7915:50
mriedemyup that's it15:50
sean-k-mooneyfried_rice: 14:00 UTC i should be able to attend.15:51
openstackgerritStephen Finucane proposed openstack/nova master: console: introduce framework for RFB authentication  https://review.openstack.org/34539715:51
openstackgerritStephen Finucane proposed openstack/nova master: console: introduce the VeNCrypt RFB authentication scheme  https://review.openstack.org/34539815:51
openstackgerritStephen Finucane proposed openstack/nova master: console: Provide an RFB security proxy implementation  https://review.openstack.org/34539915:51
openstackgerritStephen Finucane proposed openstack/nova master: doc: Document TLS security setup for noVNC proxy  https://review.openstack.org/50054415:51
fried_ricesean-k-mooney Cool.15:51
leakypipescdent, fried_rice: see my comment on https://review.openstack.org/#/c/526539/15:51
openstackgerritMatt Riedemann proposed openstack/nova master: [libvirt] Allow multiple volume attachments  https://review.openstack.org/26758715:51
openstackgerritMatt Riedemann proposed openstack/nova master: Pass multiattach flag to reserve_block_device_name  https://review.openstack.org/53124415:51
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: [api] Allow multi-attach in compute api  https://review.openstack.org/27104715:51
johnthetubaguymriedem: trying to find the docks I wrote about this, failing :(15:51
*** Drankis has quit IRC15:51
*** awaugama has joined #openstack-nova15:52
stephenfinmriedem: Could you hit this again at some point today? Got an open question on it for you https://review.openstack.org/#/c/526329/15:53
mriedemthe config option help for that actually points out the service_down_time option too15:53
cdentleakypipes: thanks reading. I think I tend to use "arbitrary" incorrectly or at least with different connotations that most peole do. I tend to mean it as "something we chose" (as opposed to "a thing we must do")15:53
mriedemso i'm pleasantly surprised15:53
johnthetubaguymriedem: https://github.com/openstack/nova/blob/d19e1252d5f2f8e8b54a276003f842ee00641026/nova/conf/scheduler.py#L7815:53
johnthetubaguymriedem: yeah, I knew I put that somewhere!15:53
mriedemjohnthetubaguy: yup, was just reading that15:53
mriedemnice15:53
stephenfinoh15:53
*** stephenfin is now known as finucannot15:53
leakypipescdent: sure, understood.15:54
sean-k-mooneyfried_rice: im hoping i wont actully need to modify the virt driver as part of the poc beyond the xml generation. on of the advantages of mdevs is you create them using arbitray uuids(neutron port uuid in my case) dynamically so i may be able to skip the resouce tracker entrily as i basically just need a count of howmany can be created per host which the placement inventories will already have.15:54
*** Tom-Tom has joined #openstack-nova15:54
johnthetubaguymriedem: there is a type there s/scheduler service as down/compute service as down/15:55
fried_riceleakypipes cdent Good deal.  I can see both sides.  I don't mind the idea of having intra-nova placement work funneled through this spot for the reasons stated (thread safety, code commonality) as long as we're not doing it with the expectation that e.g. OOT drivers have to conform.15:55
fried_ricesean-k-mooney So where are you going to get that request info from?  Just directly from the flavor?15:55
mriedemfinucannot: replied15:56
mriedemfinucannot: we could backport that to stable/pike upstream, but no further given ocata g-r requires glanceclient>=2.515:56
mriedemjohnthetubaguy: yup i noticed that too15:56
mriedemjohnthetubaguy: i'm going to remove the deprecated names for that option, so i'll fix the typo at the same time15:57
leakypipesfried_rice: well, it's Python so of course OOT drivers can always do whatever they want with things ;) but that said, yes, I'd prefer to have the virt drivers just update the ProviderTree object and rely on something else (the reportclient called from the resource tracker most likely) do the work of making placement REST API calls.15:57
finucannotmriedem: Ta15:57
johnthetubaguymriedem: sweet, thank you15:57
sean-k-mooneyfried_rice: of course when i try to do that perhaps ill discovery why mriedem thinks ill what to stabs ye lol. am the requst for the port is encode by the new vnic type + neutron port bining profile info for mdev_type if needed.15:57
*** gszasz has quit IRC15:59
fried_ricesean-k-mooney IMO what you're doing here is closer to what should be happening long term.  It would be nice if the scheduler and resource tracker could accept requests for resources, allocate them via placement, and pass those allocations through to virt without introspecting or special-casing them at all.15:59
mriedemjohnthetubaguy: actually, a bit confused15:59
mriedemjohnthetubaguy: it says, "This is because the scheduler driver is responsible for sending a heartbeat and it will only do that as often as this option allows."16:00
fried_ricesean-k-mooney I.e. compute code outside of virt doesn't need to care if a resource is a CPU, GPU, mdev (whatever that is), or peanut.16:00
mriedemjohnthetubaguy: i don't see how the compute service down time and the scheduler periodic task interval are linke16:00
mriedem*linked16:00
mriedemsean-k-mooney: you will get stabby because of the constant spelling corrections16:00
mriedemlike, "spell knife in the kidneys!"16:01
fried_riceOh, did I do that?  Not consciously.16:01
sean-k-mooneyfried_rice: yes i had hopped to work on this as part of the nova neutron negociation stuff in queens and the multiple port binding before we change from upstream first to more poc/internal focoused.16:02
*** jmlowe has quit IRC16:02
sean-k-mooneymriedem: haha i would have taught i would be the other way round. jay has very kindly help me with this area over the past few years and im very greatful for that.16:03
openstackgerritEd Leafe proposed openstack/nova master: Add functional test for resize reschedules  https://review.openstack.org/53102216:03
openstackgerritEd Leafe proposed openstack/nova master: Change compute RPC to use alternates for resize  https://review.openstack.org/52643616:03
openstackgerritEd Leafe proposed openstack/nova master: Add regression test for resizing failing during retries  https://review.openstack.org/53140516:03
figleafmriedem: ^^ added regression test and fix16:03
mriedemcool16:04
*** tbachman has quit IRC16:04
leakypipessean-k-mooney: you would have *thought*.16:04
leakypipessean-k-mooney: you're welcome.16:04
sean-k-mooneyleakypipes: :)16:04
leakypipessean-k-mooney: also... it's *grateful", not greatful. you're welcome, too.16:04
leakypipessean-k-mooney: :P16:04
kashyapmriedem: Hope you recorded your observation about raw & the shareable thing in the multi-attach review16:05
openstackgerritmelanie witt proposed openstack/nova stable/pike: libvirt: Re-initialise volumes, encryptors, and vifs on hard reboot  https://review.openstack.org/53140716:06
kashyapCurrently, I'm in split-brains mode, and trying to finishing something before the 'cache is flushed16:06
mriedemkashyap: i did, and it was just something i should have realized wouldn't apply here earlier on16:06
mriedemsince imagebackend != volumes16:06
* kashyap nods16:07
mriedemjohnthetubaguy: oh i think i see,16:08
figleaffried_rice: sorry, missed your question earlier - network issues.16:08
mriedemthe ComputeFilter is running every 60 seconds due to the caching scheduler's periodic task interval,16:08
mriedemwhich means it's potentially using a stale version of the compute service reference,16:08
figleaffried_rice: mriedem answered what I would have said, so no worries16:08
johnthetubaguymriedem: sorry, yeah, that's it16:08
mriedemwhich means it's last_seen_up would be old16:08
johnthetubaguymriedem: yeah16:08
*** mtreinish has quit IRC16:08
*** mtreinish has joined #openstack-nova16:09
johnthetubaguymriedem: hence the note about it not seeing instances being deleted, and being really bad if you run two schedulers16:09
*** dhellmann has left #openstack-nova16:09
* johnthetubaguy all hail placement16:09
mriedemheh16:09
leakypipesjohnthetubaguy: make checks payable to fried_rice.16:10
mriedemif you're wondering why i'm asking https://review.openstack.org/#/c/531020/16:10
Roamer`johnthetubaguy, did you catch my earlier messages about your procedural -2 on https://review.openstack.org/140733/?  or am I being annoying already? :)16:10
mriedempeople are still using the cachingscheduler but we don't really test it functionally anywhere, so i'm trying to flush out some squirrels16:10
leakypipestotally off-topic, but why does gerrit allow someone to hit a "quote" button on a review comment that is on a source file but won't provide a "quote" button to respond to a review comment that is on the review (not a source file)...16:11
johnthetubaguyRoamer: ah, sorry, I missed those, was visiting a customer this morning16:11
melwittleakypipes: there's a button but it doesn't say "quote". it's a curved arrow next to the comment timestamp16:12
mriedemRoamer`: you should probably be talking to me - i haven't been tracking that blueprint for nova in queens at all16:12
johnthetubaguymriedem: I don't think this once has been re-approved: https://blueprints.launchpad.net/nova/+spec/libvirt-storpool-volume-attach16:12
mriedemRoamer`: where is the patch that adds the storpool driver to cinder?16:12
leakypipesmelwitt: well, holy crap.16:13
mriedemjohnthetubaguy: yeah, because i didn't know about it until now and we're <2 weeks from FF16:13
mriedem:)16:13
leakypipesmelwitt: I thought that was to close the comment...16:13
Roamer`mriedem, yeah, actually things happened 1. a bit suddenly at the start of December, and then 2. a bit slower than we expected with getting our CI system running, so our timing is a bit less than impeccable16:13
* johnthetubaguy nods16:13
mriedemRoamer`: you also need an os-brick release16:13
Roamer`mriedem, https://review.openstack.org/#/c/220155/ is Cinder and https://review.openstack.org/#/c/192639/ is os-brick16:13
*** stvnoyes has joined #openstack-nova16:13
* johnthetubaguy learns new things about gerrit16:13
leakypipesmelwitt: heh, and the more you click the button, the more gerrit repeats the comment in the reply box. fun!16:14
melwittleakypipes: hah, awesome16:14
* leakypipes thinks up some evil DDoS that emulates clicking on comment quote buttons16:15
Roamer`mriedem, and, yeah, my last comment to 140733 was "oof, yeah, Zuul cannot pass without an os-brick release"16:15
mriedemthe unit tests are also all failing16:15
Roamer`yep, since our driver refers to initiator.STORPOOL16:15
Roamer`I could make it pass the "storpool" literal string, then it might fail in a slightly different way somewhat further16:16
Roamer`but I think that since os-brick has the constants, it'd be best to use them16:16
sean-k-mooneyleakypipes: did you not tell me once you used gertty mainly for reviews or has the new mac forced you to the web in protest.16:17
*** moshele has joined #openstack-nova16:17
openstackgerritMerged openstack/nova master: conf: hyperv: fix a comment typo  https://review.openstack.org/53132916:17
Roamer`mriedem, to make it clear: yes, the unit tests are also failing because the released version of os-brick doesn't know about the StorPool driver... if I patch it manually within the tox venv, the tests pass16:17
mriedemRoamer`: so you need an os-brick release and a bump to the required minimum version of os-brick in global-requirements in the openstack/requirements repo,16:19
mriedemnon-client library freeze is jan 1816:19
mriedemnova feature freeze is jan 2516:19
leakypipessean-k-mooney: I don't use the Mac for anything other than Verizon Slack and email. I use gertty as well but also use the Gerrit web UI just as much nowadays now that the whole "comment on this and the whole page goes haywire" bug was solved by mriedem.16:19
mriedems/mriedem/smcginnis/16:19
Roamer`mriedem, so you're saying that if I bother the os-brick folks enough, there might be a chance?  Thanks!16:19
*** moshele has quit IRC16:19
mriedemRoamer`: i'm not terribly keen on approving a nova blueprint 1 week before feature freeze16:20
mriedemsince it's a libvirt volume driver, it might get an exception16:20
sean-k-mooneyleakypipes: wait that has been fixed?16:20
mriedemRoamer`: but the ducks need to be in a row first16:20
Roamer`mriedem, okay, thanks a lot!  I'll be on my way to bother others now16:20
*** jmlowe has joined #openstack-nova16:21
Roamer`hmm, does this mean that, at least for the next couple of days, it might be a good idea for johnthetubaguy to remove his procedural -2, or shall we think about that only if we do get our ducks in a row first?16:21
leakypipescdent: answered :)16:23
cdentthanks16:23
openstackgerritMatt Riedemann proposed openstack/nova master: Remove the deprecated scheduler_driver_task_period option  https://review.openstack.org/53141316:24
mriedemjohnthetubaguy: see how that wording sounds ^16:24
mriedemRoamer`: it's fine for johnthetubaguy to remove his -216:24
mriedemwe can replace it with mine :)16:24
openstackgerritMerged openstack/nova master: Remove a duplicate line in a unit test  https://review.openstack.org/53131716:24
Roamer`:)16:24
leakypipessean-k-mooney: yeah. go into your profile -> settings -> diff preferences and set Render to "Slow"16:25
leakypipesmriedem: is that right? ^16:25
mriedemyup16:26
*** gszasz has joined #openstack-nova16:26
*** yamamoto has quit IRC16:26
leakypipescdent: great questions, btw16:28
cdentthanks, I'll try to make some more, by monday's meeting we can have a grand chat16:28
sean-k-mooneyleakypipes: mriedem awsome that bug was painful.16:29
leakypipessean-k-mooney: yeah, it sucks.16:29
leakypipesit's kinda funny. the Gerrit UI seems to deliberately try and emulate late 1990s style.16:30
*** weshay_interview is now known as weshay16:30
mriedemwhat is 2018 style? animal avitars like the iphone x?16:30
leakypipesmriedem: you know it.16:31
mriedemi'll gladly fork out $1000 for that piece of shit16:31
leakypipesnot.16:31
*** damien_r1 has quit IRC16:33
openstackgerritmelanie witt proposed openstack/nova stable/ocata: libvirt: Re-initialise volumes, encryptors, and vifs on hard reboot  https://review.openstack.org/53142216:35
*** damien_r has joined #openstack-nova16:37
*** armax has joined #openstack-nova16:39
*** openstackstatus has quit IRC16:40
cdentleakypipes: made a response, in which besides opening up some more worm cans, I note that the good questions still stand16:41
*** openstackstatus has joined #openstack-nova16:42
*** ChanServ sets mode: +v openstackstatus16:42
*** mikal has quit IRC16:43
mriedemmelwitt: i think we should split the index add out of this https://review.openstack.org/#/c/334614/ and do that in a separate change16:45
*** damien_r has quit IRC16:45
*** mikal has joined #openstack-nova16:45
*** jaosorior has quit IRC16:45
melwittrats. okay16:45
sean-k-mooneyleakypipes: well gerrit is written in java useing gwt so its not exactly html5 native. its better then github pull requests though for code review so im not complaining16:47
leakypipescdent: re: allocations for vCenter-owned (not OpenStack-owned) resources... the allocation records in placement don't care who creates them, of course. so yes, if there were some external agent that claimed resources for some non-OpenStack consumers, that would be fine I suppose. As long as that agent uses the POST /allocations interface to ensure read and write consistency, that doesn't bother me at all.16:47
leakypipescdent: I just want to make sure that for *OpenStack consumers*, it's the scheduler (or conductor for move operations) that is the thing that claims resources for the system.16:47
leakypipescdent: for all the reasons that we moved to scheduler doing claims in Pike...16:48
*** chyka has joined #openstack-nova16:48
* cdent nods16:48
sean-k-mooneyleakypipes: speaking of code review. regarding your nrp api patch that merged in early december, is the best way to play around with that just use curl/postman and do it manually or have there been any osc patches created?16:49
*** damien_r has joined #openstack-nova16:49
leakypipessean-k-mooney: :) that's actually the patch series we're currently discussing :)16:49
melwittmriedem: I changed the name of the ConsoleConnection object too. let me know if that wasn't a good idea16:50
leakypipessean-k-mooney: https://review.openstack.org/#/q/topic:bp/nested-resource-providers+(status:open)16:50
*** yamamoto has joined #openstack-nova16:50
mriedemmelwitt: i like it, going through that diff now16:50
leakypipessean-k-mooney: I'm currently working on integrating n-r-p with the allocation candidates stuff. fried_rice is focusing on the integration with the virt driver piece.16:50
leakypipessean-k-mooney: so besides playing around with the hierarchy stuff in providers, there's not much you can yet do.16:51
*** Apoorva has joined #openstack-nova16:51
sean-k-mooneyleakypipes: yep i was brefily talking to fried_rice about a poc im hoping to do in the next 6-8 to use it to support vhost-vfio interfaces.16:52
leakypipes++16:53
sean-k-mooney*6-8 weeks... units are important16:54
mriedemmelwitt: which service should i see these log statements? cauth? https://review.openstack.org/#/c/320063/25/nova/objects/console_auth_token.py@11616:54
mriedemor will this not actually run yet in ci in this change?16:54
melwittmriedem: yeah I think you won't see it until the last change in the series, in the novnc service log16:56
mriedemah yes16:57
mriedemhttp://logs.openstack.org/12/527812/1/check/tempest-full/26f5e5f/controller/logs/screen-n-novnc-cell1.txt16:57
mriedemJan 05 01:44:25.338490 ubuntu-xenial-citycloud-sto2-0001695415 nova-novncproxy[10280]: DEBUG nova.objects.console_auth_token [None req-166854c4-3f01-4db1-890b-212b7ba78e95 None None] Validated token - console connection is ConsoleAuthToken(access_url_base='http://10.0.1.151:6080/vnc_auto.html',console_type='novnc',created_at=2018-01-05T01:44:25Z,host='127.0.0.1',id=1,instance_uuid=4453afe4-e6eb-4479-a073-0f99a76c71f1,internal16:57
mriedemess_path=None,port=5900,token='***',updated_at=None) {{(pid=6917) validate /opt/stack/nova/nova/objects/console_auth_token.py:159}}16:57
mriedemNOICE16:57
melwitt:D16:57
mriedemfinucannot: you were +2 on https://review.openstack.org/#/c/320063/ before a rebase and some changes16:58
leakypipesfried_rice: question...16:58
melwittI did a bunch of token masking for the logs and then realized the websocketproxy logs the whole thing on its own, not our code16:58
leakypipesfried_rice: did we say that for allocation candidates responses, we would include "intermediary nodes" in the provider_summaries section even if those nodes weren't involved in any of the allocation_requests?16:59
melwittbut it's at level INFO so if someone cares they can turn the level up16:59
leakypipesfried_rice: for example, say we have a compute node provider, two numa cell providers and two pfs that are children, one of each numa cell. and the allocation_requests contains the compute node (for VCPU and MEMORY_MB) and both PFs (for SRIOV_NET_VF resources), should the provider_summaries include the numa cell providers as well even though no resources are directly consumed from them?17:00
*** damien_r1 has joined #openstack-nova17:01
cdentbbs17:01
*** cdent has quit IRC17:01
mriedemmelwitt: on that index, i'd be fine with it in the same change as long as you call it out in the commit message and check the columns instead of the index name - i doubt anyone would want/need to backport that index add17:02
*** jackie-truong has joined #openstack-nova17:02
mriedemif that makes life easier17:02
*** damien_r has quit IRC17:02
figleafleakypipes: it would make sense to include the whole "tree" of RPs, so that filters etc. could operate on that info17:03
leakypipesfigleaf: ack, that's what I thought I remembered. thanks for reminding me. :)17:04
sean-k-mooneyleakypipes: that is assuimg the resouce provider tree has 3 branches  basically host->numa_node->pf, host->vcpu, host->memory_mb and vcpu and memory are not inventories of the numa nodes correct17:04
*** mriedem is now known as mriedem_luncharo17:04
*** Apoorva has quit IRC17:04
melwittmriedem: okay. it probably makes more sense to be separate since the query isn't till a later change anyway. but curious, how could it end up part of a backport?17:04
leakypipessean-k-mooney: right.17:04
*** Tom-Tom has quit IRC17:05
leakypipessean-k-mooney: plus, keep in mind the allocation_requests and the provider_summaries sections of GET /allocation_candidates response are different things... the former section is the providers that would be actually allocated against. the latter section is just informational.17:05
openstackgerritMerged openstack/nova master: Use UEFI as the default boot for AArch64  https://review.openstack.org/48995117:06
*** ljjjustin has quit IRC17:08
sean-k-mooneyleakypipes: if the summery is just informational and wont be just to make desision then i dont see any harm in simplfying the representation and just weak referncing the numa afinity of the resouces.17:09
leakypipessean-k-mooney: could you elaborate on what you mean by that?17:09
leakypipessean-k-mooney: the numa affinity of the PFs is not a tag/trait. rather, it's the structure of the tree itself.17:10
sean-k-mooneyleakypipes: if the provider_summaries was used by a weigher for example then it would be good to keep the numa afinity info in the summary however if the chose beteewn which allocation candiate is made based on the tree structure then the summary could be less detailed17:12
melwittmriedem_luncharo: nevermind, my question is answered in your review comment17:12
leakypipessean-k-mooney: understood.17:14
leakypipesthx17:14
*** jafeha__ has joined #openstack-nova17:14
*** jafeha has quit IRC17:16
*** jafeha__ has quit IRC17:20
*** markmcclain has quit IRC17:20
*** markmcclain has joined #openstack-nova17:21
*** pcaruana has quit IRC17:21
*** dtantsur is now known as dtantsur|afk17:22
openstackgerritMerged openstack/nova stable/pike: Use instance.project_id when creating request specs for old instances  https://review.openstack.org/52938417:23
openstackgerritMerged openstack/nova stable/pike: Workaround missing RequestSpec.project_id when moving an instance  https://review.openstack.org/52938517:24
fried_riceleakypipes Sorry, was otp.  My understanding was that we did intend to include resource *classes* that weren't part of the request.  I don't know if we talked about including other unused providers in the same tree.  I don't see the harm in doing it, and rather have it and not need it than need it and not have it.  But no strong feelings either way.17:25
fried_riceAnd to be clear, that's in the provider summaries, *not* in the allocations portion.17:25
*** damien_r has joined #openstack-nova17:28
*** Guest13421 has quit IRC17:28
*** damien_r1 has quit IRC17:29
leakypipesfried_rice: ack17:30
openstackgerritEd Leafe proposed openstack/nova master: Fix race condition in retrying migrations  https://review.openstack.org/53102217:30
openstackgerritEd Leafe proposed openstack/nova master: Change compute RPC to use alternates for resize  https://review.openstack.org/52643617:30
*** lpetrut has quit IRC17:31
*** Guest13421 has joined #openstack-nova17:31
sean-k-mooneyfried_rice: leakypipes haveing the extra info in the summery could help debuging i guess but do ye enviosion end users reading them directly? e.g. would it be stored in the instance info when you do an openstack server show like the cached flavor or something similar?17:33
*** cdent has joined #openstack-nova17:33
sean-k-mooneyleakypipes: fried_rice basically is the summary for people or python?17:34
leakypipessean-k-mooney: no, this is an admin-only API endpoint that is only for the scheduler to make better decisions. none of this information is available to end users.17:35
openstackgerritJay Pipes proposed openstack/nova master: func tests for nested providers in alloc candidate  https://review.openstack.org/53144317:35
leakypipesfried_rice, figleaf, sean-k-mooney: thou comments welcome... ^^17:35
sean-k-mooneyleakypipes: i like that the fist think i see is an ascii diagram of what is about to be tested :)17:36
leakypipessean-k-mooney: you can thank gibi_away for that practice :)17:37
*** danpawlik has quit IRC17:37
sean-k-mooneyleakypipes: out of scope of queens but rocky+ could the VCPU and memory_mb(or perhaps just hugepages) inventories be moved to the numa node resouce providers?17:41
leakypipessean-k-mooney: possible. definitely something we need to be (very carefully) handle and discuss in Dublion.17:42
leakypipesor even Dublin17:42
*** inara has quit IRC17:42
sean-k-mooneymemory_mb is basically the non hugepage memory(e.g. 4k pages) which is basically managed by the kernel auptomatically so may that shoudl stay at the compute node level but vcpus i think would be a gould fit for the numa resouce provider. im just not sure how straight forword it would be do do the db migration required.17:44
*** inara has joined #openstack-nova17:45
leakypipessean-k-mooney: right17:45
cdentdb migration? presumably the resource tracker would intead choose to change the inventory for its tree17:46
cdentwe _really_ need to not think of placement as authoritative on that stuff. reality should be.17:46
cdentwith placement second17:46
cdentideally reality should be able to recover all of placement17:47
*** yamamoto has quit IRC17:47
leakypipesfried_rice, figleaf: do you think it would be better to have an entirely different code path in the allocation candidates code when we notice that there are provider trees? you know how we currently have a fast-path for when there are no sharing providers? should we do the same kind of thing for provider trees?17:47
leakypipescdent: ^^17:47
cdentmy initial reaction is "yes"17:48
leakypipesdidn't know you were back from your walkabout... ;)17:48
sean-k-mooneycdent: well what i ment by db migration was going from 1 inventory of all the cpus to n, one per numa node and updating the inventries to match what were actully used17:48
cdentmy walkabout was "drive home from the place where that dog was"17:48
leakypipescdent: yes, as in "separate code path for the provider tree deployments"?17:48
*** derekh has quit IRC17:48
cdentyeah, if only to make it more scrutable17:49
leakypipesack17:49
leakypipesthat was my initial thought as well.17:49
cdentI still kinda wish, however, that there was one single magical sql expression (that we never have to see but we know works) that did the whole thing17:50
* cdent likes to dream17:50
sean-k-mooneyleakypipes: as long as the second code path is not like cells v1 and solves a proablem maybe.17:50
leakypipessean-k-mooney: no, it would be just fancier SQL/Python to determine the results.17:51
leakypipescdent: there is... but it would be inscrutable...17:51
cdentthus "that we never have to see"17:51
leakypipesheh17:51
leakypipesalways remember, all of OpenStack can be rewritten entirely in SQL.17:51
cdentas well as rewritten without any17:52
leakypipesor Golang.17:52
sean-k-mooneyleakypipes: you could tripple your consluting fee to debug it if you wrote it lol17:52
leakypipessean-k-mooney: zero times three is zero. :)17:52
SamYapleleakypipes: you mispelled rust17:54
*** Apoorva has joined #openstack-nova17:54
sean-k-mooneyyou know if you joke about rewriting openstack in golang 3 times it might summon the Ciao gods...17:54
leakypipesSamYaple: :)17:54
leakypipesSamYaple: you misspelled misspelled.17:54
SamYaplei speel with the best of em17:54
leakypipes:P17:54
leakypipessean-k-mooney: Beetlejuice!17:55
SamYaplei did start on a libvirt replacement in rust (not 100% replacement, just spawn/monitor qemu process replacement)17:55
*** elod has quit IRC17:55
SamYaplerust is suuuuuuuper ugly syntax wise17:55
sean-k-mooneySamYaple: danial berage is considing that too. or golang17:56
*** elod has joined #openstack-nova17:56
SamYapleim pretty sure i pushed *something* to git about it17:56
*** sree has joined #openstack-nova17:56
SamYaplei had it spawning and interfacing with the monitor socket17:56
SamYaplehttps://github.com/SamYaple/qemuwatch17:56
SamYaplesean-k-mooney: i would love to not have to use libvirt anymore17:57
*** dave-mccowan has quit IRC17:57
sean-k-mooneySamYaple: there was a faily active thread on the libvirt mailing list back in december not sure if they reached a decision or not17:57
leakypipesSamYaple: libvirt is a fine library. except for the XML. :)17:57
SamYapleleakypipes: and the 500MB of deps needed to compile it17:57
leakypipesSamYaple: the XML deps you mean, yeah?17:58
SamYapleprobably. i never dug into why libvirt was such a pita to build from source17:58
SamYapleprobably xml :)17:58
*** elod has quit IRC17:58
*** elod has joined #openstack-nova17:58
*** amodi has joined #openstack-nova17:59
SamYaplein truth, i was writing the above to spawn vms in kubernetes, libvirt is a bit overkill for that scenario (container per vm)17:59
SamYaplebut i stopped working on it because there are 1001 projects doing that right now and i prefer to get a more openstack native solution going17:59
sean-k-mooneyi havent done anything to maintain this since i gave it rodolfo to upstream but this made it less of a pain https://github.com/openstack/devstack-plugin-libvirt-qemu18:00
sean-k-mooneyi mean you are replaceing the pain of building libvirt from source with using devstack but at least you get to chose your poision18:01
*** lpetrut has joined #openstack-nova18:02
*** lucasagomes is now known as lucas-afk18:02
SamYaplei just want to use the native qemu interface more directly. live differential backups. add/remove hardware. better metrics. that kind of thing18:02
SamYaplebut libvirt will get all that.... eventually.... probably18:02
*** yamamoto has joined #openstack-nova18:03
figleafwin 1518:04
figleafugh18:04
leakypipesSamYaple: ack18:05
*** yamamoto has quit IRC18:07
figleafleakypipes: +1 on separate code path18:08
figleafleakypipes: I'd rather avoid lots of nested logic18:08
leakypipesya18:08
*** harlowja has joined #openstack-nova18:09
*** sree has quit IRC18:09
*** mriedem_luncharo is now known as mriedem18:17
*** sree has joined #openstack-nova18:17
*** _pewp_ has quit IRC18:20
*** _pewp_ has joined #openstack-nova18:22
mriedemmelwitt: how required are the dependent os-brick changes for these backports? https://review.openstack.org/#/q/Id188d48609f3d22d14e16c7f6114291d547a898618:25
mriedembecause we can bump upper-constraints in stable but not global-requirements18:25
*** sree has quit IRC18:26
melwittmriedem: without the os-brick changes, the 'nova start' after a compute host reboot would fail because the device won't be found during the detach during re-initialization18:27
mriedemhmm18:27
mriedemwell then18:27
mriedemi don't think we can backport those then18:28
melwittI've requested .z release for os-brick stable/pike and stable/ocata, jungleboyj said he can do that next week18:28
mriedemunless you have some compat code in nova to handle os-brick not having that functionality18:28
*** damien_r has quit IRC18:28
melwittyeah, we can handle the os-brick exit code thing in nova. do you think that's more appropriate for backport land? it would make things easier for me18:29
melwittI had been thinking we had to go the .z stable release + upper-constraints route18:29
*** sree has joined #openstack-nova18:30
mriedemupper-constraints doesn't mean people are going to use that versoin18:30
mriedemthat's what g-r is for18:30
mriedemrhosp can of course require whichever package deps it wants for patch releases, but upstream we generally don't bump the minimum required version of a dependency on stable b/c we have to assume people have already shipped and possibly frozen those deps18:31
mriedem"we can handle the os-brick exit code thing in nova" is probably best, if you can determine the failure and log something like, "make sure you upgrade os-brick to avoid this problem" or whatever18:32
melwittyeah, I know. I thought it was going to be like, people will get the compatible os-brick because the latest .z release would be picked up automatically. is that not how it works? does it grab the minimum only?18:32
mriedemour CI uses what's in upper-constraints18:33
mriedemwe can't assume that's how people deploy though18:33
mriedemat least post-GA18:33
melwittI see. okay18:34
openstackgerritsean mooney proposed openstack/os-vif stable/pike: Check if interface belongs to a Linux Bridge before removing  https://review.openstack.org/53146518:35
melwittthe os-brick thing is just that we added an additional exit code for a detach call to ignore, that means "ignore if not found". we can catch that in nova too, from os-brick, and look at the exit code and ignore it if the code == "not found"18:35
mriedemby parsing the message?18:35
melwittit's just more correct to do the check in os-brick since os-brick is the one making the detach call18:36
mriedemoh18:36
melwittno18:36
mriedemok18:36
mriedemwell that doesn't seem too terrible18:36
melwittos-brick will raise ProcessExecutionError and one of the attributes is what exit code it was18:36
mriedemyeah18:36
melwittyeah, it's not terrible IMHO either18:36
*** sambetts is now known as sambetts|afk18:37
*** pooja has joined #openstack-nova18:37
*** damien_r has joined #openstack-nova18:37
poojaHappy Friday all! I had an upgrade related question for nova database.. is it safe to truncate the nova.reservations table when upgrading the controller? As per my understanding, when there is no instance creation in progress, all rows will always be in soft-deleted state.18:38
mriedempooja: why not just run the "nova-manage db archive_deleted_rows" CLI?18:40
poojamriedem: I ran that db command, but it doesn't seem to cleanup reservations table rows18:43
mriedemwhich release are you on? does the command complete without errors? what value are you using for --max_rows?18:45
poojaI am on Newton release.. Ran the command as - nova-manage --debug --config-file /etc/nova/nova.conf   db archive_deleted_rows --max_rows 10018:45
poojaIt didn't throw any errors but I also don't see any logs if it did anything18:46
mriedem--max_rows is likely not a big enough number18:46
mriedem--max_rows is the total number, so if you remove 100 records in the first table processed, you likely are done before getting to the reservations table18:47
poojaah okay.. I thought that number was per table. Let me try with a higher value18:47
mriedemildikov: well, running without the pike uca now but n-cpu fails to start because: "libvirtError: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Permission denied"18:47
*** sree has quit IRC18:47
*** yamamoto has joined #openstack-nova18:48
mriedempooja: you can run it with the --verbose option18:48
mriedemhttps://github.com/openstack/nova/blob/stable/newton/nova/cmd/manage.py#L83618:48
mriedempooja: in ocata there is a --until_complete option too https://github.com/openstack/nova/blob/stable/ocata/nova/cmd/manage.py#L66518:48
mriedemso you don't have to deal with --max_rows18:49
mriedemyou could run that in a venv/container, or backport that change to your internal repo18:49
poojamriedem: Thanks! Running it with max_rows=1000 couple of times seems to have purged the reservations table. Out of curiosity, is truncating the table safe operation during downtime though?18:49
ildikovmriedem: hmm, I saw this error before but I couldn't tell right now whether it was during testing multi-attach or not18:50
mriedempooja: archive_deleted_rows takes records where deleted != 0 and moves them to the shadow table, then deletes them from the main table18:50
mriedemshadow tables are named shadow_<table_name>18:50
mriedemso shadow_instances, shadow_reservations, etc18:50
mriedemildikov: i have a feeling it's related to not using pike uca18:50
mriedempooja: archive deleted records should be safe about any time, but yeah certainly during downtime/maintenance18:50
mriedem*downtime of the control plane18:50
ildikovmriedem: in what sense do you mean?18:51
mriedemildikov: that's the only thing that's different about this CI run,18:51
mriedemis that it's not using the pike uca, which is the default in devstack - so i'm guessing maybe something changed in devstack and implicitly relied on newer libvirt/qemu,18:51
mriedemand now doesn't work if you're not using that18:51
poojaGot it.. makes sense. It may be a good practice to run archive_deleted_rows periodically as a cronjob with low max_rows count such that there is no visible performance impact then?18:52
ildikovmriedem: I was just wondering what's different in the setup that can lead to a permission denied error18:52
mriedemcould be that libvirtd isn't even running18:52
mriedempooja: i'd ask that in the #openstack-operators channel or mailing list18:52
mriedemto see what other operators are doing18:52
*** yamamoto has quit IRC18:52
poojamriedem: Sounds good, will do that. Thanks a lot for your help!18:53
mriedemyw18:53
*** gszasz has quit IRC18:53
mriedemlooks like libvirtd is running http://logs.openstack.org/86/531386/2/check/tempest-full/4eacc3c/controller/logs/libvirt/18:53
*** damien_r has quit IRC18:54
mriedemhmm18:56
mriedem2018-01-05 16:50:52.125+0000: 11136: error : virSecurityDriverLookup:80 : unsupported configuration: Security driver apparmor not enabled18:56
*** damien_r has joined #openstack-nova18:58
*** damien_r has joined #openstack-nova18:58
mriedemhmm https://review.openstack.org/#/c/451492/18:59
SamYapleive definetely seen that error19:00
SamYaplealot19:00
mriedemwhich version?19:00
mriedemor maybe we just don't care about the apparmor error19:00
ildikovthere are a couple other errors there too19:01
SamYaplemriedem: primarily on 14.04, but that would have been with UCA libvirt 1.3.119:01
SamYaplemriedem: ive seen it on 16.04 too, but only when some old 14.04 configs werent cleaned up properly19:01
mriedemthis is xenial so 1.3.119:01
openstackgerritJay Pipes proposed openstack/nova master: add _has_provider_trees() utility function  https://review.openstack.org/53147419:01
SamYaplei do not see it when running libvirt in containers with UCA libvirt19:02
mriedemii  libvirt-bin                         1.3.1-1ubuntu10.1519:02
SamYapleyep, thats the version ive seen it with, but it was 14.04 with mitaka UCA19:02
mriedemthis is just devstack with ENABLE_UBUNTU_CLOUD_ARCHIVE=False19:03
SamYaplenow that were talking about this, i dont know if that error broke anything, i just remember seeing it19:03
SamYapleit could be a red herring19:03
openstackgerritmelanie witt proposed openstack/nova master: Add access_url_base to console_auth_tokens table  https://review.openstack.org/33461419:06
openstackgerritmelanie witt proposed openstack/nova master: Optionalize instance_uuid in console_auth_token_get_valid()  https://review.openstack.org/48170019:06
openstackgerritmelanie witt proposed openstack/nova master: Add ConsoleAuthToken object  https://review.openstack.org/32006319:06
openstackgerritmelanie witt proposed openstack/nova master: Add periodic task to clean expired console tokens  https://review.openstack.org/32538119:06
openstackgerritmelanie witt proposed openstack/nova master: Use ConsoleAuthToken object to generate authorizations  https://review.openstack.org/32541419:06
openstackgerritmelanie witt proposed openstack/nova master: Convert websocketproxy to use db for token validation  https://review.openstack.org/33399019:06
openstackgerritmelanie witt proposed openstack/nova master: Add index on token_hash and instance_uuid for console_auth_tokens  https://review.openstack.org/53147719:06
clarkbmriedem: SamYaple the issue with non UCA libvirt was segfaulting and other memory access faults19:06
clarkbwhen we upgraded to UCA libvirt those went away (not sure if that is helpful)19:07
mriedemclarkb: i'm getting a permission denied error on startup19:07
mriedemconnecting to libvirt-sock19:07
clarkbmriedem: thats the group issue I think19:07
clarkb(I thought we handled both possible groups but maybe that is broken?)19:07
mriedemclarkb: right i was looking at this https://review.openstack.org/#/c/451492/15/stackrc19:07
SamYapleclarkb: that helps me, and i think its safe to ignore those apparmor issues fwiw19:07
clarkbSamYaple: we never root caused the memory issues because upstream basically said EOLD and upgrade19:08
melwittmriedem: when I separate the index add, is it supposed to get its own migration file under migrate_repo/versions/?19:08
mriedemmelwitt: yeah it would have to19:08
melwittoops. okay19:08
SamYapleclarkb: upstream as in libvirt? or ubuntu? because ubuntu should be fixing that for the next 3(?) years or so19:08
SamYapleclarkb: did you pop a bug with canonical?19:08
clarkbSamYaple: upstream as in libvirt, ya we filed a bug against canonical/ubuntu19:09
mriedemhmm, do we not have a worlddump anymore?19:09
mriedemdid that dump the groups?19:09
*** openstack has joined #openstack-nova21:15
*** ChanServ sets mode: +o openstack21:15
dansmithand I dunno what to do with the randomize thing.. maybe also a flag about whether to do that or not (spread/pack) ?21:15
mriedemdansmith: that's in that change21:15
mriedemso that's already done21:15
dansmithoh,21:16
leakypipesdansmith: that's already there.21:16
dansmithplacement has a config not a query param for the randomize?21:16
mriedemi think the idea being, if your limit is smaller, you do random, and if it's bigger, you don't21:16
leakypipessorry, that's why I was confused :)21:16
dansmithgotcha21:16
mriedemdansmith: correct21:16
mriedemdoes seem like it should be a per-request on the client side thing21:16
dansmithright I hadn't actually paid attention to that part21:16
mriedemconfig-driven api behavior21:16
dansmiththe randomize? I would have kinda thought so, since cinder and nova might have different requirements for which 1000 to return21:17
mriedemyeah21:17
mriedemwelp, i guess that's a future improvement for rocky21:17
dansmithyeah, that's what I had been assuming, but it21:17
dansmithis something we can do later21:17
dansmithyeah21:17
*** jose-phillips has quit IRC21:17
mriedemi wonder if we'd want to make the limit option on the nova side a multiple of the number of instances to build in a given request...or is that getting too cute?21:18
*** jose-phillips has joined #openstack-nova21:18
dansmithopens a DoS though21:18
mriedemtrue21:19
dansmithI would make it static and then if people need cuteness, we can ask for safe suggestions21:19
*** smatzek has quit IRC21:20
mriedemalright, now how to track this - could do it against the blueprint if someone is going to work on it now, else i can open a bug since it's kind of a bug on the nova side that we don't limit21:20
mriedemi'm sure no one cares about that tracking business except me21:20
dansmithI can cook up a patch while you decide how to appease your paperwork fetish21:20
mriedemcool, because i'm already getting rashy21:20
*** smatzek has joined #openstack-nova21:20
*** tonyb has quit IRC21:21
edmondswthe placement api-ref isn't linked from https://developer.openstack.org/api-guide/quick-start/ ... is that intentional or an oversight?21:21
*** smatzek has quit IRC21:25
fried_ricemriedem leakypipes dansmith FYI using GET /allocation_candidates limiting is on the sched mtg agenda for Monday.21:27
leakypipescool, thanks21:27
leakypipesedmondsw: good question...21:28
mriedemedmondsw: likely an oversight21:28
mriedemedmondsw: you should totally add that21:28
edmondswmriedem leakypipes alright, will do21:29
*** jose-phillips has quit IRC21:29
*** lpetrut has quit IRC21:29
*** jose-phillips has joined #openstack-nova21:29
*** lpetrut has joined #openstack-nova21:30
dansmithI look forward to the rash of "you didn't write this conf option help text right" comments21:35
leakypipes:w21:36
openstackgerritMerged openstack/nova master: VMware: add support for different firmwares  https://review.openstack.org/44803421:40
*** sree has joined #openstack-nova21:44
*** tidwellr has joined #openstack-nova21:45
*** gouthamr has joined #openstack-nova21:47
*** Apoorva has joined #openstack-nova21:48
*** tidwellr has quit IRC21:48
*** dave-mccowan has quit IRC21:50
openstackgerritMerged openstack/nova stable/pike: Retry _trait_sync on deadlock  https://review.openstack.org/52809421:52
*** jose-phillips has quit IRC21:55
*** Apoorva_ has joined #openstack-nova21:56
*** sree has quit IRC21:58
*** jose-phillips has joined #openstack-nova21:59
*** Apoorva has quit IRC21:59
*** sree has joined #openstack-nova22:02
*** yamamoto has joined #openstack-nova22:04
*** sree has quit IRC22:06
*** gouthamr has quit IRC22:07
*** sree has joined #openstack-nova22:10
*** gouthamr has joined #openstack-nova22:12
mriedemstvnoyes: tempest patch updated with the detach logic and todos for more tests https://review.openstack.org/26660522:12
dansmithhmm, between 1.10 and 1.16 /allocation_candidates result changed ?22:13
*** gouthamr has quit IRC22:15
*** gouthamr has joined #openstack-nova22:16
*** jose-phillips has quit IRC22:16
*** gouthamr has quit IRC22:16
*** yamamoto has quit IRC22:17
figleafdansmith: yep - https://github.com/openstack/nova/blob/master/nova/api/openstack/placement/microversion.py#L52-L5522:18
dansmithyeah22:18
dansmithfigleaf: have links to what they looked like before/after?22:18
dansmithI would go for api_samples normally22:19
*** edmondsw has quit IRC22:22
*** sree has quit IRC22:22
*** edmondsw has joined #openstack-nova22:23
figleafdansmith: IIRC, they were identical, with only the addition of 'project_id' and 'user_id'22:26
figleafdansmith: but no, I don't have such links handy22:26
dansmithfigleaf: no i think the "format now matches that of get" was more of a change22:26
*** edmondsw has quit IRC22:26
dansmithI'm getting some issues that are related to trying to treat an array like a dict22:26
*** smatzek has joined #openstack-nova22:27
*** sree has joined #openstack-nova22:27
fried_riceleakypipes Looks like today we don't handle RP generation when we update aggregates.22:28
fried_riceleakypipes At least according to the API ref as it stands.  I think that's a problem.22:28
*** jose-phillips has joined #openstack-nova22:29
openstackgerritJay Pipes proposed openstack/nova master: placement: _get_trees_matching_all()  https://review.openstack.org/53151222:30
leakypipesfried_rice: add it as an item on Monday?22:30
fried_riceleakypipes Ight.22:30
fried_riceleakypipes It comes into play in the discussion around coordinating shared providers from multiple compute nodes22:30
leakypipesack22:31
fried_riceUnless the agg UUIDs are deterministic, we absolutely need generation handling there.22:31
*** smatzek has quit IRC22:32
leakypipesfried_rice: to be discussed :) (I am heading off for dinner...)22:32
fried_riceleakypipes Enjoy22:32
mriedemdansmith: https://developer.openstack.org/api-ref/placement/#list-allocation-candidates22:34
dansmithoh I didn't realize both versions were in there22:35
dansmithawesome22:35
mriedemlbragstad: looks like your patch fixes the issue http://logs.openstack.org/97/531497/2/check/legacy-tempest-dsvm-neutron-src/a196420/logs/screen-n-cpu.txt.gz?level=WARNING22:38
mriedemlbragstad: my question was really, let's say nova has 10 deprecated policies in code,22:39
mriedemand i've overridden one of those deprecated policy rules in my policy.yaml,22:39
mriedembut not the other 922:39
mriedemwill i see warnings about all 10, or just the 1 that overrode the default?22:39
lbragstadyou will see a warning for the one you overrode22:39
mriedemok i figured self.file_rules was just checking if there are any22:40
mriedembut not specific ones22:40
lbragstadthat's what https://review.openstack.org/#/c/531497/2/oslo_policy/tests/test_policy.py@1004 tests for22:40
lbragstadsince that specific test isn't writing a temp file for policy rules (or overrides)22:40
mriedemoh duh22:41
mriedem default.name in self.file_rules22:41
mriedemi didn't realize it was looping over the rules22:41
lbragstadyup22:41
lbragstadyeah - it goes through everything22:41
*** amodi has quit IRC22:41
mriedem+122:41
mriedemthanks22:41
lbragstadand logs a single warning (so warnings aren't constantly emitted every time a poilcy is run through enforce())22:41
lbragstadno problem - thanks for raising the issue22:42
openstackgerritMatt Riedemann proposed openstack/nova master: [libvirt] Allow multiple volume attachments  https://review.openstack.org/26758722:42
openstackgerritMatt Riedemann proposed openstack/nova master: Pass multiattach flag to reserve_block_device_name  https://review.openstack.org/53124422:42
openstackgerritMatt Riedemann proposed openstack/nova master: WIP: [api] Allow multi-attach in compute api  https://review.openstack.org/27104722:42
*** jose-phillips has quit IRC22:42
*** burt has quit IRC22:44
*** hongbin has quit IRC22:46
dansmithbah, have to figure that format thing out on monday22:48
openstackgerritDan Smith proposed openstack/nova master: WIP: Pass limit to /allocation_requests  https://review.openstack.org/53151722:48
*** lyan has quit IRC22:48
*** slaweq_ has quit IRC22:48
*** sree has quit IRC22:49
*** sree has joined #openstack-nova22:53
*** amodi has joined #openstack-nova22:57
*** sree has quit IRC22:58
*** vishwanathj has quit IRC22:59
*** figleaf is now known as edleafe23:01
*** sree has joined #openstack-nova23:02
*** david-lyle has joined #openstack-nova23:04
*** sree has quit IRC23:07
*** sree has joined #openstack-nova23:11
*** markmc has quit IRC23:11
*** ltomasbo has quit IRC23:11
*** moshele has joined #openstack-nova23:13
*** jpena|off has quit IRC23:14
*** sree has quit IRC23:15
*** lpetrut has quit IRC23:19
*** lpetrut has joined #openstack-nova23:20
*** moshele has quit IRC23:21
*** sree has joined #openstack-nova23:23
*** sree has quit IRC23:28
*** lpetrut has quit IRC23:28
*** Apoorva_ has quit IRC23:28
*** ltomasbo has joined #openstack-nova23:30
*** vishwanathj has joined #openstack-nova23:31
*** sree has joined #openstack-nova23:32
*** esberglu has quit IRC23:34
*** markmc has joined #openstack-nova23:34
*** esberglu has joined #openstack-nova23:34
*** gyee has quit IRC23:35
*** stvnoyes has quit IRC23:35
*** esberglu has quit IRC23:39
*** slaweq has joined #openstack-nova23:39
*** slaweq has quit IRC23:44
*** sree has quit IRC23:44
*** jackie-truong_ has joined #openstack-nova23:45
*** sree has joined #openstack-nova23:45
*** jackie-truong has quit IRC23:48
*** liverpooler has quit IRC23:48
*** liverpooler has joined #openstack-nova23:50
*** jackie-truong_ has quit IRC23:52
*** jose-phillips has joined #openstack-nova23:52
*** Guest13421 has quit IRC23:57
*** sdague has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!