Thursday, 2018-01-11

*** jmlowe has quit IRC00:00
*** edmondsw has quit IRC00:00
*** jmlowe has joined #openstack-nova00:01
openstackgerritHironori Shiina proposed openstack/nova master: [WIP] Ironic: Get IP address for volume connector from ports/portgroups  https://review.openstack.org/51732200:03
*** hshiina has joined #openstack-nova00:12
*** suresh12 has joined #openstack-nova00:14
*** hoonetorg has quit IRC00:15
openstackgerritHironori Shiina proposed openstack/nova master: Ironic: Get IP address for volume connector from ports/portgroups  https://review.openstack.org/51732200:16
*** hoonetorg has joined #openstack-nova00:16
*** felipemonteiro has joined #openstack-nova00:19
*** jobewan has quit IRC00:23
*** yamamoto has quit IRC00:24
*** yamamoto has joined #openstack-nova00:27
*** armax has joined #openstack-nova00:31
openstackgerritTakashi NATSUME proposed openstack/nova master: List/show all server migration types (1/2)  https://review.openstack.org/43060800:32
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in delete (2)  https://review.openstack.org/52626300:34
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in delete (3)  https://review.openstack.org/52655700:34
*** browne has joined #openstack-nova00:36
*** browne has quit IRC00:36
openstackgerritTakashi NATSUME proposed openstack/nova master: List/show all server migration types (1/2)  https://review.openstack.org/43060800:38
*** yamamoto has quit IRC00:39
*** tetsuro has joined #openstack-nova00:41
*** liuzz_ has quit IRC00:42
*** liuzz has joined #openstack-nova00:42
*** claudiub has quit IRC00:43
*** sree has joined #openstack-nova00:43
*** Dinesh_Bhor has joined #openstack-nova00:46
*** sree has quit IRC00:48
*** jichen has joined #openstack-nova00:51
*** jamiec has quit IRC00:52
*** lyan has joined #openstack-nova00:58
*** hiro-kobayashi has joined #openstack-nova00:58
*** jamiec has joined #openstack-nova00:59
*** mvk has quit IRC01:01
*** tidwellr has joined #openstack-nova01:02
*** threestrands has quit IRC01:03
*** zhaochao has joined #openstack-nova01:04
*** phuongnh has joined #openstack-nova01:08
*** Dinesh_Bhor has quit IRC01:19
*** slaweq_ has joined #openstack-nova01:21
*** mdnadeem has joined #openstack-nova01:22
*** slaweq_ has quit IRC01:25
*** Dinesh_Bhor has joined #openstack-nova01:26
*** masayukig_ has quit IRC01:28
*** masayukig has joined #openstack-nova01:30
*** zhurong has joined #openstack-nova01:34
*** kumarmn has joined #openstack-nova01:34
*** felipemonteiro has quit IRC01:35
*** kumarmn has quit IRC01:35
Kevin_ZhengIt seems to me that this one is useless https://github.com/openstack/nova/blob/master/nova/compute/api.py#L230401:39
*** yamamoto has joined #openstack-nova01:39
Kevin_Zhengsince fixed_ip is not considered as an valid filter both in schema and api-ref01:39
Kevin_Zhengafter remap, its value is put into filters['ip']01:40
Kevin_Zhengas we consider 'ip' and 'ip6' as valid filters01:40
Kevin_Zhengthose values didn't got remapped as what has been done for fixed_ip filter01:41
Kevin_Zhengseems strange to me01:41
*** rcernin has quit IRC01:42
*** edmondsw has joined #openstack-nova01:44
*** yamamoto has quit IRC01:47
*** edmondsw has quit IRC01:48
*** markvoelker has joined #openstack-nova01:50
*** rcernin has joined #openstack-nova01:50
*** Dinesh_Bhor has quit IRC01:52
openstackgerritZhenyu Zheng proposed openstack/nova master: Use neutron port_list when filtering instance by ip  https://review.openstack.org/52550501:53
*** Apoorva has quit IRC01:58
*** mlavalle has quit IRC01:58
*** lyan has quit IRC02:00
*** moshele has joined #openstack-nova02:01
*** kumarmn has joined #openstack-nova02:02
*** kumarmn has quit IRC02:04
*** Dinesh_Bhor has joined #openstack-nova02:08
Kevin_Zhengmriedem alex_xu ^^02:09
Kevin_ZhengI mean the fixed_ip thing02:09
*** gouthamr has joined #openstack-nova02:10
*** annp has joined #openstack-nova02:11
*** Apoorva has joined #openstack-nova02:14
*** gouthamr_ has joined #openstack-nova02:15
*** gouthamr has quit IRC02:15
*** yamahata has joined #openstack-nova02:18
*** namnh has joined #openstack-nova02:18
mriedemKevin_Zheng: we should probably have a release note for that,02:19
*** gouthamr_ has quit IRC02:19
mriedemsomething about how nova will proxy ip filtering to neutron as long as the neutron extension is available02:19
*** yamamoto has joined #openstack-nova02:19
*** Apoorva has quit IRC02:19
Kevin_ZhengOK02:19
mriedemyou can do that on top02:19
mriedemso you don't lose jay's +202:19
Kevin_ZhengOK02:19
Kevin_Zhengwhat do you think about the fixed_ip thing?02:20
Kevin_ZhengI was mislead by it at the beginning02:20
mriedemi'm not sure what you're referring to02:21
Kevin_ZhengIt seems to me that this one is useless https://github.com/openstack/nova/blob/master/nova/compute/api.py#L230402:21
Kevin_Zhengsince fixed_ip is not considered as an valid filter both in schema and api-ref02:21
Kevin_Zheng09:38 after remap, its value is put into filters['ip']02:21
Kevin_Zheng09:39 as we consider 'ip' and 'ip6' as valid filters02:21
Kevin_Zheng09:39 those values didn't got remapped as what has been done for fixed_ip filter02:21
Kevin_Zheng09:39 seems strange to me02:21
mriedemthe schema won't block it though02:22
mriedembecause additionalProperties=True02:22
Kevin_Zhengyeah02:22
mriedemotherwise yes i agree the fixed_ip filter mapping doesn't really make sense02:22
mriedembut i wouldn't propose removing that in this same change02:22
Kevin_Zhengbut why it has been mapped but ip and ip6 didn't02:23
mriedemno idea02:23
Kevin_Zhengis what I'm curious about02:23
mriedemprobably was written 6 years ago02:23
mriedemit could predate the ip/ip6 filters02:23
Kevin_Zhengyeah, I checked the history02:23
mriedemor was a rackspace only thing02:23
mriedemwho knows02:23
*** gouthamr has joined #openstack-nova02:23
Kevin_Zhengseems for ec202:23
mriedemah ok02:24
Kevin_Zhengso probably just leave it alone02:24
mriedemyeah i would02:25
mriedemi don't see anything obvious about it in here though https://github.com/openstack/ec2-api/blob/master/ec2api/api/instance.py02:25
openstackgerritChen Hanxiao proposed openstack/nova master: libvirt: add Linux distribution guest only description for inject_xxx options  https://review.openstack.org/52844102:28
*** kwathore has quit IRC02:28
*** suresh12 has quit IRC02:28
mriedemKevin_Zheng: comments inline02:29
*** kwathore has joined #openstack-nova02:29
mriedemeasy stuff though02:29
mriedemlooks really good otherwise02:29
Kevin_ZhengI missread02:30
Kevin_Zhengit has a logic bellow in the previous code in 201102:30
Kevin_Zhengand it says ec2 specific02:30
mriedemyou said the neutron extension wasn't being caught when you did the performance test right?02:31
mriedemthat's probably because the neutron patch changed the name to "IP address substring filtering"02:31
Kevin_Zhengyeah02:31
Kevin_ZhengIt should be OK now02:31
mriedemhttps://review.openstack.org/#/c/521683/19/neutron/extensions/ip_substring_port_filtering_lib.py#3402:32
mriedemyou're using the old ext name in the nova patch02:32
mriedemhttps://review.openstack.org/#/c/521683/19/neutron/extensions/ip_substring_port_filtering_lib.py@3402:32
Kevin_Zhengoh, no02:33
Kevin_Zhengthe previous error was in neutron02:33
Kevin_Zhengit cannot list the new extension02:33
Kevin_Zhengbecause the name is not correct, in neutron02:33
Kevin_Zhengnow it is good02:33
*** suresh12 has joined #openstack-nova02:34
Kevin_ZhengI will change the name in nova02:34
mriedemok02:34
mriedemadd the missing tests and release note and i think it's ready02:34
mriedemi can't +2 b/c i'm co-author though02:34
*** Dinesh_Bhor has quit IRC02:36
*** markvoelker has quit IRC02:36
*** Dinesh_Bhor has joined #openstack-nova02:36
*** yamahata has quit IRC02:38
*** suresh12 has quit IRC02:38
*** yamahata has joined #openstack-nova02:38
mriedemalex_xu: i addressed your issue on that migrations changes-since thing https://review.openstack.org/#/c/330406/ - also found a couple of issues in the unit tests that were already in there, so fixed those as well02:39
*** gouthamr has quit IRC02:40
mriedemCI has been way backed up all day though02:40
*** hongbin has joined #openstack-nova02:44
*** tidwellr has quit IRC02:46
*** tidwellr has joined #openstack-nova02:46
*** threestrands has joined #openstack-nova02:47
*** threestrands has quit IRC02:47
*** threestrands has joined #openstack-nova02:47
*** tidwellr has quit IRC02:50
*** yamahata has quit IRC02:54
*** mriedem has quit IRC02:56
*** moshele has quit IRC02:58
*** suresh12 has joined #openstack-nova02:59
*** Dinesh_Bhor has quit IRC03:01
*** masber has quit IRC03:02
openstackgerritTakashi NATSUME proposed openstack/nova master: List/show all server migration types (1/2)  https://review.openstack.org/43060803:03
*** suresh12 has quit IRC03:03
*** masber has joined #openstack-nova03:05
*** yamamoto_ has joined #openstack-nova03:12
*** yamamoto has quit IRC03:16
*** gyee has quit IRC03:16
*** felipemonteiro has joined #openstack-nova03:18
*** harlowja_ has quit IRC03:21
*** slaweq_ has joined #openstack-nova03:21
*** slaweq_ has quit IRC03:26
*** kumarmn has joined #openstack-nova03:28
openstackgerritjichenjc proposed openstack/nova master: WIP: add zvm document into  https://review.openstack.org/53272003:33
*** vladikr has quit IRC03:45
*** kumarmn has quit IRC03:54
*** abhishekk has joined #openstack-nova03:54
*** felipemonteiro has quit IRC03:57
*** diga has joined #openstack-nova03:59
*** kumarmn has joined #openstack-nova04:03
*** Dinesh_Bhor has joined #openstack-nova04:05
openstackgerritZhenyu Zheng proposed openstack/nova master: Use neutron port_list when filtering instance by ip  https://review.openstack.org/52550504:07
*** diga has quit IRC04:08
*** kumarmn has quit IRC04:09
*** kumarmn has joined #openstack-nova04:09
*** esberglu has quit IRC04:10
*** kumarmn has quit IRC04:10
*** kumarmn has joined #openstack-nova04:11
*** zhurong has quit IRC04:12
*** felipemonteiro has joined #openstack-nova04:15
*** kumarmn has quit IRC04:16
*** sree has joined #openstack-nova04:21
*** armax has quit IRC04:24
*** links has joined #openstack-nova04:25
*** hiro-kobayashi has quit IRC04:26
*** kumarmn has joined #openstack-nova04:28
*** andreas_s has joined #openstack-nova04:32
*** karthiks has joined #openstack-nova04:33
*** andreas_s has quit IRC04:37
*** Dinesh_Bhor has quit IRC04:38
*** kumarmn has quit IRC04:38
*** felipemonteiro has quit IRC04:39
*** kumarmn has joined #openstack-nova04:39
*** tuanla____ has joined #openstack-nova04:41
*** kumarmn has quit IRC04:44
*** diga has joined #openstack-nova04:45
*** suresh12 has joined #openstack-nova04:45
*** Dinesh_Bhor has joined #openstack-nova04:53
*** Dinesh_Bhor has quit IRC04:56
*** links has quit IRC04:56
*** links has joined #openstack-nova04:57
*** suresh12 has quit IRC04:58
*** suresh12 has joined #openstack-nova04:58
*** Dinesh_Bhor has joined #openstack-nova04:58
*** hshiina has quit IRC05:00
*** ratailor has joined #openstack-nova05:06
*** janki has joined #openstack-nova05:07
openstackgerritTakashi NATSUME proposed openstack/nova master: Add a warning in 'nova-manage cell_v2 delete_cell'  https://review.openstack.org/51377105:11
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix 500 error while passing 4-byte unicode data  https://review.openstack.org/40751405:11
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix missing marker functions  https://review.openstack.org/51457905:12
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add functional tests for traits API  https://review.openstack.org/52409405:13
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add functional tests for traits API  https://review.openstack.org/52409405:14
*** itlinux has quit IRC05:14
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Move body examples to an isolated directory  https://review.openstack.org/52999805:15
*** claudiub has joined #openstack-nova05:15
openstackgerritTakashi NATSUME proposed openstack/nova master: Adds view builders for keypairs controller  https://review.openstack.org/34728905:16
openstackgerritTakashi NATSUME proposed openstack/nova stable/pike: Add 'delete_host' command in 'nova-manage cell_v2'  https://review.openstack.org/51371705:18
*** sree has quit IRC05:19
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix the order of target host checks  https://review.openstack.org/52622505:19
*** edmondsw has joined #openstack-nova05:20
*** psachin has joined #openstack-nova05:21
*** edmondsw has quit IRC05:24
*** psachin has quit IRC05:27
*** zhurong has joined #openstack-nova05:28
*** rcernin has quit IRC05:29
*** harlowja has joined #openstack-nova05:29
*** sree has joined #openstack-nova05:29
*** gcb has quit IRC05:30
*** gcb has joined #openstack-nova05:31
*** hongbin has quit IRC05:32
*** sridharg has joined #openstack-nova05:32
*** psachin has joined #openstack-nova05:35
openstackgerritTakashi NATSUME proposed openstack/nova master: List/show all server migration types (2/2)  https://review.openstack.org/45948305:42
*** hiro-kobayashi has joined #openstack-nova05:44
*** markvoelker has joined #openstack-nova05:47
*** karthiks has quit IRC05:50
*** Dinesh_Bhor has quit IRC05:52
*** suresh12 has quit IRC05:54
*** xinliang has quit IRC05:56
*** Dinesh_Bhor has joined #openstack-nova05:56
*** Tom-Tom has joined #openstack-nova05:57
*** moshele has joined #openstack-nova05:58
*** sandanar has joined #openstack-nova05:58
*** karthiks has joined #openstack-nova06:02
*** Dinesh_Bhor has quit IRC06:04
*** hoonetorg has quit IRC06:08
*** xinliang has joined #openstack-nova06:08
*** slaweq has joined #openstack-nova06:09
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in post  https://review.openstack.org/52682306:09
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in put (1)  https://review.openstack.org/53125806:10
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in put (2)  https://review.openstack.org/53126606:10
*** Dinesh_Bhor has joined #openstack-nova06:12
*** slaweq has quit IRC06:13
*** suresh12 has joined #openstack-nova06:14
*** Dinesh_Bhor has quit IRC06:17
*** hoonetorg has joined #openstack-nova06:20
*** Eran_Kuris has joined #openstack-nova06:24
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in put (3)  https://review.openstack.org/53128006:28
*** suresh12 has quit IRC06:30
*** Dinesh_Bhor has joined #openstack-nova06:32
*** moshele has quit IRC06:37
*** annp has quit IRC06:38
*** lajoskatona has joined #openstack-nova06:40
*** markvoelker has quit IRC06:40
openstackgerritChen Hanxiao proposed openstack/nova master: libvirt: add Linux distribution guest only description for inject_xxx options  https://review.openstack.org/52844106:41
*** markvoelker has joined #openstack-nova06:42
*** yamamoto has joined #openstack-nova06:45
*** alexchadin has joined #openstack-nova06:46
*** harlowja has quit IRC06:47
*** yamamoto_ has quit IRC06:49
*** tianhui has joined #openstack-nova06:52
*** Dinesh_Bhor has quit IRC06:59
*** Dinesh_Bhor has joined #openstack-nova06:59
*** Dinesh_Bhor has quit IRC06:59
*** threestrands has quit IRC07:00
*** threestrands has joined #openstack-nova07:01
*** threestrands has quit IRC07:01
*** threestrands has joined #openstack-nova07:01
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add sending global request ID in get  https://review.openstack.org/53129907:01
*** threestrands has quit IRC07:02
*** threestrands has joined #openstack-nova07:03
*** threestrands has quit IRC07:03
*** threestrands has joined #openstack-nova07:03
*** threestrands has quit IRC07:04
*** threestrands has joined #openstack-nova07:04
*** edmondsw has joined #openstack-nova07:08
*** yjk has joined #openstack-nova07:08
*** hshiina has joined #openstack-nova07:09
*** kholkina has joined #openstack-nova07:10
*** edmondsw has quit IRC07:13
*** pcaruana has joined #openstack-nova07:21
*** dims_ has quit IRC07:21
*** hemna_ has quit IRC07:21
*** jafeha__ is now known as jafeha07:23
*** dims has joined #openstack-nova07:25
*** sahid has joined #openstack-nova07:33
*** xnox has quit IRC07:33
*** moshele has joined #openstack-nova07:35
*** xnox has joined #openstack-nova07:36
*** xnox has joined #openstack-nova07:36
*** Eran_Kuris has quit IRC07:38
*** yjk has quit IRC07:39
*** kumarmn has joined #openstack-nova07:39
*** Eran_Kuris has joined #openstack-nova07:42
*** kumarmn has quit IRC07:44
*** andreas_s has joined #openstack-nova07:53
*** xnox has quit IRC07:54
*** xnox has joined #openstack-nova07:56
*** xnox has joined #openstack-nova07:56
*** sapcc-bot has quit IRC07:59
*** annp has joined #openstack-nova08:01
*** mkoderer_6 has quit IRC08:03
*** tpatzig_8 has quit IRC08:03
*** david_10 has quit IRC08:03
*** tpatzig_3 has quit IRC08:03
*** carthaca_2 has quit IRC08:03
*** mkoderer_ has quit IRC08:03
*** AlexeyAbashkin has joined #openstack-nova08:08
*** threestrands has quit IRC08:08
*** slaweq_ has joined #openstack-nova08:10
*** matrohon has joined #openstack-nova08:11
*** slaweq_ has quit IRC08:14
ameedaMorning :)08:15
maciejjozefczykMorning!08:15
ameedacan you please check this question https://stackoverflow.com/questions/48202352/write-functional-regression-test-for-nova08:15
*** ttsiouts has quit IRC08:16
*** ttsiouts has joined #openstack-nova08:17
*** tesseract has joined #openstack-nova08:17
ameedamaciejjozefczyk: I try to deploy openstack on baremetal from undercloud installed at vm, I got this error "CREATE_FAILED  ResourceInError: resources.NovaCompute: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500""08:19
*** damien_r has joined #openstack-nova08:20
*** yjk has joined #openstack-nova08:21
ameedaI found this error "ComputeHostNotFound: Compute host undercloud could not be found." in nova-compute.log08:22
*** slaweq has joined #openstack-nova08:22
*** avolkov has joined #openstack-nova08:22
*** alexchadin has quit IRC08:28
*** suresh12 has joined #openstack-nova08:30
maciejjozefczykameeda: It shoudl be quite similiar to https://github.com/openstack/nova/blob/master/nova/tests/functional/regressions/test_bug_1554631.py08:31
maciejjozefczykabout functional regression test08:31
ameedamaciejjozefczyk: thanks you very much. I will check it now08:32
maciejjozefczykameeda: I mean you need some mocking and it could be possible to check if the field is not truncated08:32
ameedamaciejjozefczyk: the problem is that I am beginner with openstack and also I never write regression code, I will do my best to do that08:33
maciejjozefczykameeda: I've never written any regression test also :D08:34
*** jpena|off is now known as jpena08:34
maciejjozefczykameeda: about second question; check scheduler logs08:34
ameedamaciejjozefczyk: great :D08:35
*** yjk has quit IRC08:35
ameedamaciejjozefczyk: let me check it now08:35
maciejjozefczykameeda: Do understand it well I recommend to move log_level to debug and then restart scheduler; you'll find which filter removes the host from list08:35
*** suresh12 has quit IRC08:35
ameedalet me see how to change log_level :$08:37
*** alexchadin has joined #openstack-nova08:41
*** pcaruana has quit IRC08:44
*** ragiman has joined #openstack-nova08:46
*** links has quit IRC08:46
*** jaosorior has joined #openstack-nova08:47
openstackgerritSilvan Kaiser proposed openstack/nova master: Added mount fstype based validation of Quobyte mounts  https://review.openstack.org/52224508:48
*** ralonsoh has joined #openstack-nova08:50
*** ratailor_ has joined #openstack-nova08:53
*** tianhui has quit IRC08:54
*** tianhui has joined #openstack-nova08:54
*** ratailor has quit IRC08:55
maciejjozefczykameeda: /etc/nova/nova.conf - add debug=True08:56
*** edmondsw has joined #openstack-nova08:56
ameedamaciejjozefczyk:  yes I did that, and I restart the services and I need to empty log files contents08:57
*** hiro-kobayashi has quit IRC08:58
*** links has joined #openstack-nova08:59
*** edmondsw has quit IRC09:01
bauzasgood morning Novaers09:01
ameedaMorning :)09:01
ameedamaciejjozefczyk: I will start openstack deploy.09:02
*** hshiina has quit IRC09:06
*** sree_ has joined #openstack-nova09:10
*** sree_ is now known as Guest3737509:11
*** sree has quit IRC09:12
*** Tom-Tom has quit IRC09:12
*** sree has joined #openstack-nova09:12
*** Tom-Tom has joined #openstack-nova09:13
kashyapmdbooth: Hi, when you're about: Given this — https://review.openstack.org/#/c/505673/ (which adds the '--force-share' option to Nova Git master).  For operators that are running already released version (e.g. Pike), updating to QEMU 2.10, would there be any impact?09:13
kashyapOr we'd be expecting them to _not_ upgrade as each Nova release comes with its specific version cap of libvirt & QEMU?09:13
*** Guest37375 has quit IRC09:15
*** ljjjustin has joined #openstack-nova09:15
kashyapOh, it's actually cherry-picked to Pike09:16
*** Tom-Tom has quit IRC09:17
kashyapBut not to Ocata (because it's in Phase-II).09:18
*** lucas-afk is now known as lucasagomes09:26
mdboothkashyap: I suspect there would be an impact.09:31
mdboothI think downstreams will have to backport that as required.09:32
kashyapmdbooth: Yeah, was thinking so.  So for Ocata and Newton users (https://releases.openstack.org/) who upgrade to 2.10 QEMU will see problems09:32
kashyapYeah09:32
mdboothkashyap: Yep.09:32
kashyapThe Pike backport was already done to alleviate an Ubuntu user running 2.10 -- https://review.openstack.org/#/c/509774/09:32
redondo-mkMorning. Question...does anyone know some fast/simple way to estimate available capacity in terms of number of instances if you want to estimate that for specific flavor?09:36
*** dtantsur|afk is now known as dtantsur09:38
*** takashin has left #openstack-nova09:38
*** gnuoy has quit IRC09:43
*** hrw has left #openstack-nova09:46
*** links has quit IRC09:51
*** Tom-Tom has joined #openstack-nova09:58
*** kholkina has quit IRC10:02
*** Tom-Tom has quit IRC10:02
*** kholkina has joined #openstack-nova10:03
*** jichen has quit IRC10:04
*** pcaruana has joined #openstack-nova10:04
*** links has joined #openstack-nova10:05
*** ljjjustin has quit IRC10:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Default some flavor fields loaded from instance  https://review.openstack.org/52919410:06
openstackgerritBalazs Gibizer proposed openstack/nova master: Default some flavor fields loaded from instance  https://review.openstack.org/52919410:07
*** phuongnh has quit IRC10:10
*** slaweq_ has joined #openstack-nova10:11
*** yjk_ has joined #openstack-nova10:11
yjk_Hi,  I am currently on Newton release. (I am new to openstack)  I am facing the following issue: - Event notifications generated by nova do not show the actual user who initiated the event. It rather shows the VM/object owner.  For Example, If a "test-tenant" has two users an admin "test-admin" and a SSO user "test-ssu".   If "test-admin" reboots a VM created by "test-ssu", event notification shows "test-admin" rather than "tes10:12
*** moshele has quit IRC10:12
yjk_This I believe is because while generating the event notification payload, information from instance is used instead of context information.  Code wise, in this line https://github.com/openstack/nova/blob/808d36475103e373f1deb3344b6829ce68d6cdd5/nova/notifications/base.py#L432  Here we use: user_id=instance.user_id  Should'nt we use, user that has come from context, something like user_id=context.user_id  Just wanted to know co10:12
*** markvoelker has quit IRC10:13
*** sambetts|afk is now known as sambetts10:14
*** namnh has quit IRC10:14
*** mvk has joined #openstack-nova10:14
*** slaweq_ has quit IRC10:15
bauzasmmm, I'm slapped by an InventoryInUse exception when I start testing VGPU allocations10:16
gibiyjk_: interesting finding.10:16
bauzasjianghuaw_: have you faced the same ?10:16
bauzasjianghuaw_: I had an exception when I started a new instance, so I destroyed it (it was in ERROR state)10:16
bauzasbut since now, every inventory update leads to InventoryInUse thingy10:17
gibiyjk_: do you see the same problem for versioned notifications?10:17
jianghuaw_bauzas, I don't have the issue at the moment. Is the exception reported when deleting inventory?10:19
jianghuaw_Is it the first instance booted with a vgpu?10:20
gibiyjk_: anyhow could you please file a bug about it?10:20
bauzasjianghuaw_: no, the story is more like that :10:20
bauzasJan 11 05:10:18 virtlab606.virt.lab.eng.bos.redhat.com nova-compute[80886]: INFO nova.scheduler.client.report [None req-4fdc9878-61e9-4b2f-8e92-1913486f0c9a demo admin] [req-b7b27fd5-31f8-43da-9d52-c2066d0b7af2] Inventory update conflict for e4304148-2f21-417b-bf78-9f3bc90f0ec7 with generation ID 710:20
bauzasbecause of the generation bit10:20
bauzasthat would lead to deleting the inventory as it's planned *but* the scheduler client is checking whether that inventory is in use - and yes, it's the case10:21
bauzasbecause I have 3 fully functional VMs10:21
bauzaswith VGPU allocations to them10:21
bauzascdent is off, edleafe, jaypipes and efried aren't there yet10:21
jianghuaw_Ah, then it should be a bug.10:21
bauzasso I dunno who could help me10:22
bauzasyes, this is a bug I guess10:22
jianghuaw_Is it in a single compute node env or multiple nodes?10:23
bauzassingle one10:23
bauzasso, it tries to update with generation=8 but the RP already has generation=8 so I guess it's the problem10:24
*** sree_ has joined #openstack-nova10:25
*** sree_ is now known as Guest8860010:26
gibiyjk_: the code you linked above has a comment "Owner properties" so I think the original intention was to include the user id of the owner of the VM there10:27
gibiyjk_: but still I can understand the need to know the user id of the user who initiated the action10:28
jianghuaw_bauzas, checking in my side. May you share me your procedure? Just boots 3 VMs with vGPU in sequence? And met the exception as the 3rd vm boots up?10:29
gibiyjk_: I think we can add that as a separate field, so both the owner and the action initiator is available in the notification10:29
*** sree has quit IRC10:29
bauzasjianghuaw_: #1 just create a new instance using a VGPU allocation10:29
bauzas#2 then, make sure you'll get an exception (and the instance being in ERROR) for a second instance asking for the same VGPU class10:30
bauzas#3 restart nova-compute10:30
bauzas#4 you'll see that InventoryInUse exception then10:30
yjk_@gibi: That sounds good.10:30
*** cdent has joined #openstack-nova10:30
jianghuaw_bauzas, for #2 how did you trigger the exception?10:31
jianghuaw_or what's the exception met in #2?10:33
*** alexchadin has quit IRC10:34
*** diga has quit IRC10:35
*** zhurong has quit IRC10:35
bauzasjianghuaw_: it was just an exception raised by privsep when I was trying to create a new mdev10:35
bauzasjianghuaw_: but I guess any exception raised by the virt driver is fine10:35
bauzasI'll need to disappear for 2 hours10:36
jianghuaw_yep, understood.10:36
*** aloga_ has joined #openstack-nova10:37
yjk_Thanks for the reply gibi.10:39
*** dr_gogeta86 has joined #openstack-nova10:39
yjk_I have another query.10:39
*** dr_gogeta86 has quit IRC10:39
*** dr_gogeta86 has joined #openstack-nova10:39
yjk_I have an openstack Newton setup with 3 compute nodes. I have booted VMs from a bootable volume. Setup uses iSCSI disk shared between computes.  Whenever, I try nova evacuate, it fails with an ERROR: FailedISCSITargetPortalLogin: Could not login to any iSCSI portal  It seems that "connect_volume" call on the new host fails because it is not able to authenticate with iSCSI.10:39
*** sandanar has quit IRC10:39
yjk_I believe it (whoever responsible) creates isci credentials when volume attach is initiated from a host. During migration, it should cleanly update the credentials so new host can access.10:40
*** sandanar has joined #openstack-nova10:40
yjk_Any ideas on what may be causing this issue?10:41
*** edmondsw has joined #openstack-nova10:44
*** slaweq_ has joined #openstack-nova10:46
*** maciejjo1 has joined #openstack-nova10:46
*** danpawlik_ has joined #openstack-nova10:46
*** andreas_s has quit IRC10:48
*** slaweq_ has quit IRC10:48
*** maciejjozefczyk has quit IRC10:48
*** danpawlik has quit IRC10:48
*** slaweq_ has joined #openstack-nova10:48
*** slaweq has quit IRC10:48
*** edmondsw has quit IRC10:49
*** andreas_s has joined #openstack-nova10:58
*** hiro-kobayashi has joined #openstack-nova11:02
jianghuaw_bauzas, I've tried several ways to trigger the exception in #2 but I didn't see the issue " Inventory update conflict " on my side.11:02
cdentbauzas: I'm here now (re [t 3csz] ). What's up?11:03
purplerbot<bauzas> cdent is off, edleafe, jaypipes and efried aren't there yet [2018-01-11 10:21:54.269899] [n 3csz]11:03
*** moshele has joined #openstack-nova11:03
*** abhishekk has quit IRC11:03
yjk_Compute host logs where the VM was getting migrated (evacuated) to are available on http://paste.openstack.org/show/642545/. Your help would be really appreciated. Thanks11:04
*** AlexeyAbashkin has quit IRC11:06
bauzasjianghuaw_: cdent: nevermind, I found the root cause : PEBKAC11:06
bauzaslemme tell you the story tho11:06
* cdent listens11:06
bauzasso, in my changes, I said that the min version for mdevs is 3.4.011:06
jianghuaw_listening11:06
bauzasbut for RHEL7.4, it's 3.2.011:07
bauzasso I directly modified the version in the libvirt driver11:07
bauzasafter that, I had a problem (for the #2 exception) so I just rebased my git repo into a separate change11:07
bauzasbut then the min version was 3.4.0 again11:08
bauzasso when looking at the inventory, it wasn't supporting mdevs, so the scheduler client tried to delete the inventory11:08
bauzashence the problem11:08
cdentah11:08
cdentI can see how that would have been hard to uncover :(11:09
bauzasthe problem with VGPU resource classes is that we only use them if needed11:09
bauzasso when you want to stop supporting it, then it would be problme11:10
bauzaswe don't have the other problem for other standard RCs11:10
bauzasbut it *could* be a problem later11:10
bauzasat least with nested RPs11:10
bauzasI totally understand why we provide that exception11:11
bauzasbut maybe it could be a confusion for operatoes11:11
cdentI suspect we're going to find a lot if interesting bits of a confusion as we start using the stuff for real. Which I hope we can start doing sooner than later.11:12
bauzasanyway, thanks for your help11:13
bauzasI'm just punting my thoughts, just keeping it in my mind11:13
bauzasfood for thoughts, that's it11:14
jianghuaw_bauzas, understood. thanks for the sharing.11:14
*** gszasz has joined #openstack-nova11:15
*** andreas_s has quit IRC11:19
mdboothbauzas: Is 'allocations' persisted anywhere? That is, can we recall it after spawn e.g. during live migration/reboot/rescue?11:20
* mdbooth guesses it's not really for that?11:20
*** tuanla____ has quit IRC11:21
*** AlexeyAbashkin has joined #openstack-nova11:25
*** derekh has joined #openstack-nova11:25
*** maciejjo1 is now known as maciejjozefczyk11:27
bauzasmdbooth: you mean where are persisted 'allocations' ? in the API DB11:27
bauzasI'm just amazed to see a feature working with new inventory classes :è)11:28
bauzas:-)11:28
bauzasthe era of tech debt is just behind us now, huzzah \o/11:28
gibiyjk_: I filed a bp about your notification ehancement https://blueprints.launchpad.net/nova/+spec/add-action-initiator-to-instance-action-notifications11:28
bauzasPCIManager, trust me, you're going to have a bad day !11:28
cdentmdbooth: they are recalled during migration11:31
*** andreas_s has joined #openstack-nova11:36
*** sdague has joined #openstack-nova11:39
*** kumarmn has joined #openstack-nova11:40
*** andreas_s has quit IRC11:40
*** tetsuro has quit IRC11:44
*** kumarmn has quit IRC11:45
*** hiro-kobayashi has quit IRC11:45
*** andreas_s has joined #openstack-nova11:50
*** liuzz has quit IRC11:54
*** andreas_s has quit IRC11:55
mdboothcdent: Do you have a pointer?11:57
cdentmdbooth: yeah, just a sec11:58
*** gszasz has quit IRC11:58
mdboothlocations = self.reportclient.get_allocations_for_consumer(11:59
mdbooth            instance.uuid)11:59
mdbooth^^^ ?11:59
mdbooths/locations/allocations/11:59
cdentI was thinking of https://github.com/openstack/nova/blob/master/nova/conductor/tasks/migrate.py#L12911:59
*** janki has quit IRC11:59
cdentwhich calls  replace_allocation_with_migration11:59
*** yjk_ has quit IRC11:59
mdboothcdent: Does an instance have 2 sets of allocations during a migration, btw?12:00
*** yamamoto has quit IRC12:01
cdentyes, but one of them is identified by the uuid of the migration (in queens). For a while there were two, both identified by the instance12:01
*** yamamoto has joined #openstack-nova12:01
mdboothcdent: Thanks12:01
*** icey has quit IRC12:02
*** pcaruana has quit IRC12:02
*** pcaruana|afk| has joined #openstack-nova12:02
*** cdent has quit IRC12:02
*** icey has joined #openstack-nova12:02
*** gszasz has joined #openstack-nova12:03
*** jpena is now known as jpena|lunch12:04
*** annp has quit IRC12:05
*** slaweq has joined #openstack-nova12:11
*** markvoelker has joined #openstack-nova12:14
*** cdent has joined #openstack-nova12:16
*** slaweq has quit IRC12:16
*** sandanar has quit IRC12:18
*** smatzek has joined #openstack-nova12:21
*** yjk has joined #openstack-nova12:23
yjkHi gibi, I have opened an openstack bug. Please check https://bugs.launchpad.net/nova/+bug/174267412:23
openstackLaunchpad bug 1742674 in OpenStack Compute (nova) "[Newton] Nova notifications do not show the user which initiates the event." [Undecided,New]12:23
*** redondo-mk has quit IRC12:24
*** redondo-mk has joined #openstack-nova12:24
*** tomhambleton_ has quit IRC12:24
*** jose-phi_ has joined #openstack-nova12:25
*** mvk has quit IRC12:25
*** lucasagomes is now known as lucas-hungry12:25
*** markmc has quit IRC12:27
*** jose-phillips has quit IRC12:27
*** takashin has joined #openstack-nova12:27
*** markmc has joined #openstack-nova12:28
gibiyjk: I think it needs to be handled as a new feature, hence the blueprint I opened12:29
gibiyjk: if you have time to work on such a feature I can help you12:29
*** Eran_Kuris has quit IRC12:29
*** tetsuro has joined #openstack-nova12:29
*** edmondsw has joined #openstack-nova12:33
yjkgibi: I though you had asked me to open an openstack bug. That is why opened a bug.12:33
gibiyjk: yes, but then based on the code comment I realized that it was the original intention to send the owner's user_id12:35
gibiyjk: so I think we should add a separate field for the user id of the user that called the action12:35
gibiyjk: and that will be more like a new feature than a bugfix12:35
yjkgibi: I though you had asked me to open an openstack bug. That is why opened a bug.12:36
*** edmondsw has quit IRC12:38
gibiyjk: yes, first I thought that it is a bug then I realzied that it is more like a new feature12:38
gibiyjk: sorry for the confusion12:38
ameedaHello, Please help me , please check this "http://paste.openstack.org/show/642597/" that caused when I introspect the node.12:38
*** Eran_Kuris has joined #openstack-nova12:42
*** ljjjustin has joined #openstack-nova12:44
*** ljjjustin has quit IRC12:44
*** sapcc-bot has joined #openstack-nova12:44
*** moshele has quit IRC12:46
*** markvoelker has quit IRC12:48
*** ratailor_ has quit IRC12:49
*** pcaruana|afk| has quit IRC12:49
*** moshele has joined #openstack-nova12:50
*** krtaylor_ has quit IRC12:50
*** ygl has joined #openstack-nova12:50
*** yjk has quit IRC12:54
yglhi all can someone help me with nova-scheduler please ?12:55
bauzasygl: sure?12:55
*** andreas_s has joined #openstack-nova12:58
ameedabauzas: can you please help me ?12:59
yglbauzas: recently my openstack setup took a hit. it is not launching any instances. I cant see any use ful logs indicative of the problem, even though I enabled debug in nova.conf. After a lot of investigation I found that one of the computes is not reachable by it mgmt IP13:00
yglbauzas: why is it that the scheduler was not lookin gto for other good computes even though they are available but looking always for that unrechable compute node ?13:00
yglbauzas: ununtil I removed that compute from the nova zone, it did not work13:00
*** jpena|lunch is now known as jpena13:00
yglbauzas: why is it so with the scheduler ?13:00
*** pcaruana|afk| has joined #openstack-nova13:02
*** kholkina has quit IRC13:02
yglbauzas: any idea ?13:02
*** kholkina has joined #openstack-nova13:03
bauzasygl: not sure I understand your problem13:03
*** ljjjustin has joined #openstack-nova13:03
*** andreas_s has quit IRC13:04
bauzasmdbooth: thanks for your review on https://review.openstack.org/#/c/528832/13:04
bauzasmdbooth: there are 3 "stages" on having VGPU capabiliities with libvirt13:05
mdboothbauzas: Thanks for a morning of discovery :)13:05
bauzasmdbooth: first of all, you need a vGPU capable device13:05
yglbauzas: because of unreachable compute by its management IP, the nova-scheduler  is not allowing to launch new instances on remaining compute nodes13:05
bauzasmdbooth: this is expressed by a "mdev capable" PCI device13:05
*** Tom-Tom has joined #openstack-nova13:06
bauzasie. a PCI device having mdev "capabilities" that say "for that type, you can have X vGPUs"13:06
*** gszasz has quit IRC13:06
bauzasbut a single PCI device can support multiple profiles, ie. types13:06
bauzasthat's only when you *create* a mediated device that the other types become unavailable13:07
bauzascreating a mediated device is stage #213:07
bauzasfor creating a mediated device, you can't call the libvirt API but you rather need to use sysfs directly13:07
bauzasthat will generate a mediated device that you can see in the sysfs, but no guests are allocated yet to that host13:08
yglbauzas: any idea ?13:08
bauzasygl: seriously, no, we filter out unreachable computes but the others are still checked13:08
yglbauzas: but in my case it was failing13:09
mdboothbauzas: I think the above is in line with my (very newly acquired) understanding.13:09
bauzasmdbooth: for having a guest having VGPUs, you need a stage #313:09
yglbauzas: until I dsiabled the bad compute, it didi not work13:09
*** jaosorior has quit IRC13:09
bauzasmdbooth: which is basically use that mediated device in the guest config13:09
mdboothbauzas: Yep. This is the parallel I drew with host-connected volumes.13:10
bauzasmdbooth: so, about the risk of having all the mdevs not cleaned up, it's a non-problem13:10
*** Tom-Tom has quit IRC13:10
mdboothStep 0 is create the volume (in cinder)13:10
bauzasmdbooth: because we only care of how many mediated devices are still unassigned to a guest13:10
mdboothStep 1 is create a host device corresponding to that volume13:10
bauzasif all the mediated devices are already created by the operator, that's not an issue13:11
mdboothStep 2 is add it to a guest.13:11
bauzasand btw. there is a kernel race that prevents libvirtd to see the newly created mediated device, see my relnote13:11
mdboothYeah, I read that. That's annoying.13:11
bauzasso, in general, it's better to just create by yourself all the mediated devices beforehand13:11
mdboothFWIW I wouldn't bother working round that in Nova. Just note that here be dragons and wait for the fix.13:12
mdboothThere are enough caveats for the experimental flag already, I think.13:12
bauzasso, see, given each mediated device is cleaned up when it's detached from the guest, I don't see any problem with reusing them in between all the instances13:12
mdboothThanks for that release note, btw. Best I've read.13:12
bauzas(wrt. your last comment)13:12
bauzas"Looking forward, we're not cleaning up mdevs. I don't think this is a  problem right now when we only support 1 type, but IIUC allocating an  mdev of type X can reduce the availability for type Y, right? If that's  the case, I think we should consider cleaning up. This wouldn't be an  immediate priority, though.'"13:13
mdboothbauzas: Got it. Yeah, that would only apply if these are mix and match. i.e. You can have 6 X and 6 Y.13:13
mdboothBut I think you're saying as soon as you have 1 X you can only have X13:13
bauzasbecause, and I told you that, if you have a physical device that can support X and Y types, once you create *one* single mediated device of X, then you litterally have 0 available for Y13:14
bauzasexactly13:14
*** janki has joined #openstack-nova13:14
bauzas(at least for the nvidia driver)13:14
mdboothRight, so the only advantage to cleanup is that you could potentially repurpose it later to be Y if you removed all X without requiring a restart.13:14
bauzasmdbooth: http://images.nvidia.com/content/grid/pdf/GRID-vGPU-User-Guide.pdf section 1.3.2 "homogeneous vGPUs"13:15
mdboothThat's in the realms of optimisation, though. Very much not important.13:15
bauzasmdbooth: right, as an operator, you really need to consider which type to use, and not try to mix them up in between a single node13:15
mdboothThe major issues were the races, the inconsistent XML generation, and that as a maintainer of the libvirt driver, in general I don't want _get_guest_xml doing even more surprising things.13:16
bauzasI'd be in favor to limit as the maximum the fact to mix up different types across a single compute, even if you have multiple GPUs13:16
bauzasmdbooth: the races are a concern, but I'd be in favor of adding the locks in a follow-up if you don't mind13:16
mdboothSo if you create one of these instances, stop it, then start it, you've lost your vgpu.13:16
bauzasmdbooth: that code is already pretty heavy13:16
bauzasmdbooth: that's part of the extra testing I'm doing on my box13:17
*** sahid has quit IRC13:17
mdboothbauzas: I think we can safely say it won't work without testing though, right?13:17
bauzasI'm just tackling the last bits of privsep, but I was considering a shelve13:17
*** ameeda has quit IRC13:17
mdboothBecause we don't pass allocations to power_on.13:17
bauzasmdbooth: we could if necessary13:18
mdboothIt would also be nice to block live migrations, don't know if we do that alrady.13:18
*** gszasz has joined #openstack-nova13:18
bauzasmdbooth: well, live migrating an instance having a VGPU attached is like "hold my beer"13:18
mdboothSimilarly, if you rescue one, when you unrescue it you'll have lost your vgpu13:18
bauzasmdbooth: okay, here is what I propose13:19
mdboothbauzas: Hehe, yeah. I wouldn't expect that to work, tbh.13:19
bauzasmdbooth: step #1 amend the relnote and mention the caveats you noticed13:19
mdboothCan we split the mdev allocation code into a function which does:13:19
bauzasmdbooth: step #2 do a follow-up on the race13:19
bauzasmdbooth: step #3 and beyond : patch each caller of get_guest_config to pass the allocation13:20
mdbooth_grab_mdevs(allocations) -> list of mdev uuids13:20
mdboothThen pass list of mdev uuids to _get_guest_xml13:20
mdboothIt would also give us the opportunity to do explicitly different things to generate that list in different contexts.13:21
mdboothe.g. at startup, our mdevs may exist already or they may not.13:21
bauzasright, that's another caveat, mdevs aren't persistent13:22
mdboothIf we just restarted Nova, they will exist. If we rebooted the host, they won't.13:22
bauzasbut I was considering to use init_host to just lookup all the guest XMLs and recreate the mdevs13:22
mdboothWe don't want to be trying to handle weird situations like that in the middle of _get_guest_xml13:22
bauzas(if of course your node kernel doesn't have that uevent race)13:22
mdboothSo my structural concern is with my 'maintainer of the libvirt driver' hat on. It isn't specifically related to mdevs.13:23
*** tetsuro has quit IRC13:23
mdboothI'd prefer that we didn't do host configuration in _get_guest_xml.13:23
mdbooths/do/do more/13:24
bauzasmdbooth: btw. issue confirmed : if you stop/start your instance, then goodbye my dev13:25
*** tetsuro_ has joined #openstack-nova13:25
bauzasok, I'm gonna tackle one issue at time13:25
mdboothbauzas: We can get allocations back, I think. I'm pretty sure we can do this without having to persist it anywhere, although that was my first thought.13:26
bauzasmdbooth: I'll just spin another revision modifying the privsep call, could you please post your comments back on the next rev ?13:26
mdboothbauzas: Sure thing.13:26
bauzasmdbooth: this is more a scheduling problem13:26
*** mvk has joined #openstack-nova13:26
bauzasmdbooth: because if we stop the guest, then we loose the mediated device13:26
mdboothOh, true.13:27
bauzasmdbooth: I wonder if we keep the mdev attachment when we restart13:27
bauzasmdbooth: that needs to be tested, because I ask to lookup all libvirt guests, including those stopped13:27
mdboothI think the domain sticks around of you just stop it.13:27
mdboothI think you're good there.13:27
bauzasso, in theory, we still get the association even if the domain is destroyed13:27
bauzasbecause it's not undefined13:27
bauzasso I think we're good13:28
mdboothI'm also assuming, btw, that it doesn't matter *which* vgpu we get.13:28
bauzaswe could just need to pass the allocations to power_on13:28
mdboothSo stop/start, get a new vgpu, we don't care.13:28
bauzasmdbooth: right, all VGPUs are equally equal :p13:28
bauzasbased on the same type, of course :)13:28
bauzasit's cloud, brah13:28
mdboothHehe.13:29
* mdbooth submits a patch to delete live-migration13:29
* bauzas submits a patch to rename live-migration to 'hold-my-beer'13:29
openstackgerritStephen Finucane proposed openstack/nova master: console: Send bytes to sockets  https://review.openstack.org/53183413:31
openstackgerritStephen Finucane proposed openstack/nova master: console: introduce framework for RFB authentication  https://review.openstack.org/34539713:31
openstackgerritStephen Finucane proposed openstack/nova master: console: introduce the VeNCrypt RFB authentication scheme  https://review.openstack.org/34539813:31
openstackgerritStephen Finucane proposed openstack/nova master: console: Provide an RFB security proxy implementation  https://review.openstack.org/34539913:31
openstackgerritStephen Finucane proposed openstack/nova master: doc: Document TLS security setup for noVNC proxy  https://review.openstack.org/50054413:31
stephenfinmelwitt: Addressed concerns in ^13:31
yglbauzas: any idea about the scheduler behaving this way ?13:31
openstackgerritBalazs Gibizer proposed openstack/nova master: Increase notification wait timeout in functional tests  https://review.openstack.org/53281613:32
bauzasmdbooth: https://twitter.com/sylvainbauza/status/95144671547057766413:33
gibimelwitt: hi!, https://bugs.launchpad.net/nova/+bug/1742688 and https://review.openstack.org/532816 It seems we are the victim of the gate slowness13:33
openstackLaunchpad bug 1742688 in OpenStack Compute (nova) "test_live_migration_actions notification sample test fails intermittently with 'notification instance.live_migration_rollback.start hasn't been received'" [Undecided,In progress] - Assigned to Balazs Gibizer (balazs-gibizer)13:33
bauzasgibi: yup, the gate is like super-slow13:36
bauzasmdbooth: so, tell me more about what you want13:39
*** mdnadeem has quit IRC13:39
bauzasmdbooth: you said to have a magical _grab_mdevs(allocations) that would return you a list of mdev UUIDs, right?13:40
bauzasmdbooth: so basically identical to _guest_add_vgpus() except the very last line ?13:40
mdboothbauzas: Sec, lemme look13:40
bauzasmdbooth: tl;dr: I only care about the guest parameter because I eventually do a guest.add_device()13:42
bauzasbut at the very very end of that method13:42
mdboothbauzas: Yes. There's only like 3 lines of that function which relate to xml generation :)13:42
bauzasand 2 of them are unrelated to the guest itself13:43
bauzasso, then, what would be the interest of splitting that ?13:43
mdboothbauzas: A few13:43
mdboothThe first as I mentioned is that it's surprising13:44
mdboothSomebody coming along looking for where this stuff is configured in the workflow isn't going to see _get_guest_xml() and guess that it's there.13:44
*** lucas-hungry is now known as lucasagomes13:44
*** markvoelker has joined #openstack-nova13:44
bauzasmdbooth: that said, we do that *a lot*, right? :p13:44
mdboothWe do a whole lot of surprising stuff, sure :)13:45
bauzasmdbooth: I mean, that's precisely there where we attach what we want, right ?13:45
mdboothSecond is I suspect we're going to want to have different ways to create this list.13:45
bauzasit's not only the GPU thing, it's all the devices we create13:45
openstackgerritStephen Finucane proposed openstack/nova master: trivial: Resolve 'oslo.context' deprecation warnings  https://review.openstack.org/53282213:45
mdboothe.g. creating new mdevs vs re-using existing mdevs13:45
mdbooths/existing/already allocated to this instance/13:46
bauzasmdbooth: that is precisely something that should kept hidden from the other libvirt places13:46
bauzasfor tech debt reasons13:46
mdboothWhere creating == create or reallocate13:46
bauzashow the vGPUs are implemented in libvirt should be kept very isolated from the rest13:46
mdboothI'm talking about the code in libvirt/driver.py here13:47
bauzassure13:47
mdboothSo it shouldn't affect the decisions of other drivers.13:47
bauzasI still think we need to limit the driver knowledge about what is a vGPU13:47
mdboothI think the guest needs to end up with a list of mdevs it should add13:48
bauzasmdbooth: that's just basically splitting the method in twice13:48
bauzasand introducing a new level of indirection, but okay13:48
bauzasmdbooth: I still don't see why we really need to split in twice but okay13:49
mdboothWell I think the split is helpful to the reader, because it calls out a different type of operation in a different place. It also allows us to generate the list in different ways based on caller context without having to pass the context all the way down into _get_guest_xml13:50
bauzasit just makes the interfaces a bit cleaner, but that's it13:50
mdboothe.g. If we're creating a new guest, allocating new instances, vs re-generating our existing xml with our current mdevs.13:50
bauzasmdbooth: a-ha, that's different13:50
bauzasmdbooth: I see your thoughts13:51
*** liverpooler has joined #openstack-nova13:51
*** abhishekk has joined #openstack-nova13:51
bauzasmdbooth: ie. not passing allocated all the way down to _get_guest_xml() but rather just call _give_me_vgpus() in the caller itself ?13:51
mdboothbauzas: Something like that would be my preference.13:51
bauzasand then the caller attaching those vgpus *after* it got the XML ?13:51
mdboothNo, you have to do it before13:52
*** masber has quit IRC13:52
bauzasmdbooth: you understand that it's mostly a convenience problem ?13:52
mdboothSo you have to do the host config first, then you can generate the guest xml based on how you did it13:52
*** rabel_b1 has joined #openstack-nova13:52
mdboothIt's the same as the volume attachment problem. You can't generate the guest xml until you can tell libvirt where you mounted the volume.13:53
bauzasmdbooth: so you would pass the extra bits down _get_guest_xml, right?13:53
*** efried has joined #openstack-nova13:53
mdboothYes.13:53
bauzasmdbooth: and then the whole _give_me_mdevs() would be a synchronized section?13:54
*** lyan_ has joined #openstack-nova13:54
bauzasgibi: chairing the nova meeting, right?13:55
mdboothbauzas: Yes, it would have to be.13:55
bauzasmdbooth: I still consider the synchronized section to be in a follow-up13:55
bauzasmdbooth: timeline wise13:56
bauzasbecause we're 2 weeks away from the FF, and I don't want that feature to be punted13:56
mdboothbauzas: Adding that would be way simpler than moving code around out of _get_guest_xml13:56
mdboothJust a simple host-wide local lock13:57
bauzasI agree13:57
bauzashere are my thoughts13:57
bauzasbased on what I see important to be merged before end of Queens13:58
bauzas#1 the patch itself13:58
bauzas(with the synchronized lock)13:58
bauzas#2 the possibility to disallow nova to create mdevs13:58
*** alexchadin has joined #openstack-nova13:59
bauzas#3 the fix for power-on and others13:59
bauzas#4 the possibility to recreate mdevs after a reboot13:59
bauzasand the rest looks optional to me13:59
bauzaswe can refine the interfaces in Rocky13:59
bauzasbesides the fact that the problem you mention isn't particular to the VGPU feature, but rather for a shit ton of extra devices and features we add to the guest14:00
bauzasmdbooth: agree ?14:00
mdboothOnly volumes, I think.14:00
mdboothI agree with your prioritisation.14:00
gibibauzas: I did not get the memo that I'm the chair14:00
bauzasmdbooth: no, we litterally generate all the devices at the same place14:00
gibibauzas: but I can jump in14:01
efriedgibi bauzas Somebody oughtta do it.  The time is now :)14:01
*** vladikr has joined #openstack-nova14:01
mdboothbauzas: I really think it's only volumes.14:03
mdboothThe others we're just adding devices to the guest. The different with volumes and now vgpus is that we're reconfiguring the host.14:03
mdboothSo we're doing _get_guest_xml() and it has surprising side-effects14:03
bauzasmdbooth: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4882 ?14:03
mdboothbauzas: Yeah, that's just pulling them out of the instance.14:04
mdboothIIUC they've already been allocated elsewhere.14:04
bauzasmdbooth: what's the difference between attaching a PCI device or a mediated device ?14:04
mdboothBecause you're not creating the pci device, or doing any kind of 'scheduling'14:04
*** ragiman has quit IRC14:05
mdboothYou're not calling _get_guest_xml() and suddenly you've created a new pci device14:05
mdboothAnyway, I agree with your prioritisation. That said, I really don't like it about volumes, and I'm uncomfortable adding more of it.14:05
bauzasmdbooth: but the allocation was made way earlier :)14:06
bauzasfor the VGPU14:06
bauzasif you go to that compute, that's because it has free space for a vGPU14:06
bauzasbecause placement told you14:06
bauzasexactly like NUMA things14:06
bauzasthe scheduler or placement won't know which specific device you're assigned14:07
*** eharney has joined #openstack-nova14:07
bauzasbut it will know there is room for you14:07
bauzasin theory, you should never raise the exception I added14:07
mdboothbauzas: There's also the case of recreating the domain xml with existing mdevs.14:08
mdboothAlthough perhaps that doesn't matter as long as we're absolutely certain the domain isn't running.14:09
mdboothBut again that's an embedded assumption which makes everything harder to understand.14:09
bauzasmdbooth: the problem is that the XML isn't persisted when you stop the instance14:10
bauzasI mean in Nova14:10
bauzasthat's surprising to me14:10
bauzaswe regenerate the XML when we start again the instance14:10
bauzaswhile it's already been defined !14:11
bauzasmdbooth: any idea you know why ?14:11
mdboothYeah, that's... suboptimal.14:11
bauzasthat's rather errorprone14:11
mdboothA legacy design choice that Nova is the ultimate source of truth. It means we lose stuff like device ordering and address assignments.14:11
*** slaweq has joined #openstack-nova14:12
bauzasI can understand the reasoning14:13
bauzasbut then, you somehow need to persist that14:13
bauzasand rather not rebuilding it14:13
bauzasbut that's MHO14:13
bauzasand I'm possibly missing some design choices14:13
artomIt may have come from the need to support other drivers?14:14
artomWho don't necessarily keep state like libvirt does?14:14
*** ygl has quit IRC14:14
mdboothartom: This is an internal libvirt driver detail.14:14
bauzasanyway14:15
bauzasI have some stupid formatting to do, because pci names are different from filesystem14:15
bauzasI mean pci names in libvirt14:15
bauzasgrrrr14:15
*** yangyapeng has quit IRC14:15
mdboothbauzas: I agree that moving that stuff is less important, though.14:15
mdboothI think where we differ is much less important we think it is ;)14:16
*** dave-mccowan has joined #openstack-nova14:16
cdent"source of truth" is definitely a common issue14:16
bauzasdistributed systems is a NP-problem by definition14:17
bauzasso I'm not afraid14:17
*** slaweq has quit IRC14:17
*** markvoelker has quit IRC14:18
*** ragiman has joined #openstack-nova14:19
mdboothbauzas: The other things are bugs, anyway. This is an issue of ongoing maintainability.14:20
kashyapstephenfin: Hey, have you setup DevStack lately on an older branch (Mitaka) on CentOS?14:21
*** edmondsw has joined #openstack-nova14:21
stephenfinkashyap: Newton is as far back I've gone in a while, I think14:21
kashyapI don't see Mitaka branch in DevStack, do they "delete" the older branches?14:21
*** tomhambleton_ has joined #openstack-nova14:21
stephenfinand that was before it was EOL14:21
stephenfinI guess they do. Look for the mitaka-eol tag14:21
kashyapstack@devstack-centos ~/src/cloud/devstack (master) $ git branch -a | grep -i mitaka14:22
kashyapstack@devstack-centos ~/src/cloud/devstack (master) $ echo $?14:22
kashyap114:22
stephenfingit tag -a | grep -i mitaka14:22
*** moshele has quit IRC14:22
kashyaps/-a/-l/14:23
stephenfinIndeed14:23
kashyapYep, could fetch from the tag.  Thx.14:23
stephenfinOnce you've that checked out, you'll also want to modify the branches that are checked out, e.g. https://gist.github.com/stephenfin/d1c000f7e52ff181c4673322b28eaf47#file-local-conf-L71-L81https://gist.github.com/stephenfin/d1c000f7e52ff181c4673322b28eaf47#file-local-conf-L71-L8114:23
* kashyap clicks14:23
stephenfinI guess they should all be 'mitaka-eol'14:23
*** edmondsw_ has joined #openstack-nova14:24
*** psachin has quit IRC14:24
kashyapstephenfin: Yeah, my attempt on Fedora for Mitaka failed14:24
*** pchavva has joined #openstack-nova14:24
kashyapFailed as in, the libvirt-python wonkiness: http://paste.openstack.org/show/642707/14:25
stephenfinAye, an LTS distro is probably the way to go. Guess either Ubuntu 14.04 or CentOS 7.4 is your best bet14:25
kashyapIt's missing two of the events, when building libvirt-python14:25
kashyapDidn't wanted to duke around there, so just went w/ CentOS.  Let's see how that goes.14:25
kashyapstephenfin: I used to use -- https://kashyapc.fedorapeople.org/virt/openstack/Minimal-DevStack-local.conf14:27
kashyapI know it could be improved (Neutron is now default, etc).14:27
*** esberglu has joined #openstack-nova14:28
*** edmondsw has quit IRC14:28
*** awaugama has joined #openstack-nova14:29
*** ljjjustin has quit IRC14:29
*** tetsuro_ has quit IRC14:34
*** archit has joined #openstack-nova14:40
*** zhaochao has quit IRC14:40
*** kholkina has quit IRC14:43
*** kholkina has joined #openstack-nova14:43
*** edmondsw_ is now known as edmondsw14:45
*** abhishekk has quit IRC14:46
*** ragiman has quit IRC14:48
*** hongbin has joined #openstack-nova14:48
*** jobewan has joined #openstack-nova14:49
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Collocate encryptor and volume driver calls  https://review.openstack.org/46024314:50
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Introduce disk encryption config classes  https://review.openstack.org/46400814:50
openstackgerritLee Yarwood proposed openstack/nova master: WIP libvirt: QEMU native LUKS decryption for volumes  https://review.openstack.org/52395814:50
*** Swami has joined #openstack-nova14:52
*** moshele has joined #openstack-nova14:54
*** moshele has quit IRC14:56
*** bkopilov has quit IRC14:58
*** sahid has joined #openstack-nova14:58
*** mlavalle has joined #openstack-nova14:58
*** eharney has quit IRC15:00
*** yangyapeng has joined #openstack-nova15:00
*** gouthamr has joined #openstack-nova15:01
*** takashin has left #openstack-nova15:01
*** ragiman has joined #openstack-nova15:01
*** markvoelker has joined #openstack-nova15:02
*** markvoelker has quit IRC15:02
*** mriedem has joined #openstack-nova15:02
mriedemoops, totally forgot about the team meeting15:03
*** rabel__b1 has joined #openstack-nova15:03
*** eharney has joined #openstack-nova15:03
mriedemgibi: thanks for running that15:03
openstackgerritsahid proposed openstack/nova master: libvirt: slow live-migration to ensure network is ready  https://review.openstack.org/49745715:05
gibimriedem: no problem, I was not prepared but I could run it15:05
*** ihrachys has quit IRC15:05
*** rabel_b1 has quit IRC15:05
*** ihrachys has joined #openstack-nova15:05
efriedHi cores, would anyone be willing to look at the first two patches for the PowerVM integration bp?  https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/powervm-nova-it-compute-driver15:10
efriedstephenfin gibi bauzas perhaps if you have time?15:11
stephenfinefried: I've been meaning to review the OVS VIF one, but I'll have to leave the storage patches to someone else15:11
bauzasefried: I'm still rushing on my feature branch :/15:11
bauzasefried: blame mdbooth15:11
mdboothbauzas: Sorry :P15:12
*** Apoorva has joined #openstack-nova15:12
bauzasmdbooth: no worries, you just owe me a beer :p15:12
bauzaset voila!15:12
bauzasinteresting, creating a new mdev even if you have a kernel race works15:13
bauzasthat's awesome news15:13
bauzassince the mdev is *really* created, you can ask libvirt to use it15:13
bauzaseven if libvirt isn't able to show it per its API15:13
bauzasamazing15:13
bauzasso, one big caveat isn't really a big deal and not requiring a separate change15:14
bauzasjust docs15:14
bauzashuzzah15:14
efriedstephenfin Thanks, anything helps.  TBH, there's not a lot of storage (or network) understanding required for these.  The guts are in the pypowervm lib; these patches are really just translating the nova-ese to powervm-ese.15:14
*** kumarmn has joined #openstack-nova15:14
*** hamzy has quit IRC15:17
*** tidwellr has joined #openstack-nova15:18
*** edmondsw has quit IRC15:18
mriedemstephenfin: do you see anything obviously wrong with https://review.openstack.org/#/c/532689/ ? i ran it last night through the experimental queue but results were never posted back on the patch15:19
* stephenfin looks15:19
*** cleong has joined #openstack-nova15:20
mriedemi can see it queued up in the experimental queue in zuul right now15:20
*** READ10 has joined #openstack-nova15:20
mriedemgmann: https://review.openstack.org/#/c/330406/85/nova/api/openstack/compute/migrations.py@11915:22
mriedemwe have 2.0 and 2.1 all over the API, it doesn't seem to make a difference15:22
openstackgerritsahid proposed openstack/nova master: hardware: only take into account small pages  https://review.openstack.org/53216815:24
*** markvoelker has joined #openstack-nova15:24
*** edmondsw has joined #openstack-nova15:24
*** lajoskatona has quit IRC15:26
hongbinmrodden: mlavalle : want to make a call on the full substring or partial substring debate https://review.openstack.org/#/c/521683/ ?15:26
mlavallehongbin: I will later today. Thanks for the heads up15:27
*** alexchad_ has joined #openstack-nova15:28
*** Eran_Kuris has quit IRC15:28
gibiefried: opened the patches, but it won't fitt to my today's schedule15:29
efriedgibi Thanks in advance15:29
mriedemhongbin: full substring15:30
hongbinmlavalle: mriedem : ack15:30
*** slaweq_ has quit IRC15:30
*** slaweq has joined #openstack-nova15:31
*** alexchadin has quit IRC15:31
kashyapDoes anyone know what provides this one?  I'm on CentOS 7.4, and DevStack fails with:15:32
kashyap $ rpm -qf `which generate-subunit`15:32
kashyapfile /usr/bin/generate-subunit is not owned by any package15:32
kashyap(Err the failure message was: "./stack.sh: line 488: generate-subunit: command not found")15:32
mriedemask in qa15:33
*** yangyapeng has quit IRC15:33
kashyapYep, checking there15:33
*** Swami has quit IRC15:33
*** kumarmn has left #openstack-nova15:35
*** janki has quit IRC15:35
kashyapmriedem: Hehe, I found a bug with your comment (#5) -- https://bugs.launchpad.net/devstack/+bug/154737915:35
openstackLaunchpad bug 1547379 in devstack "./stack.sh: line 488: generate-subunit: command not found" [Undecided,Invalid]15:35
*** slaweq has quit IRC15:35
kashyapIt's indeed a mistake in my local.conf!15:36
mriedemexellent15:36
mriedem*excellent event15:36
mriedem*gdi15:36
kashyap(Because I copied my F27 local.conf to the old and sloth-like CentOS)15:36
*** itlinux has joined #openstack-nova15:37
*** yamamoto has quit IRC15:37
*** felipemonteiro has joined #openstack-nova15:37
*** alexchad_ has quit IRC15:37
*** links has quit IRC15:38
*** yamamoto has joined #openstack-nova15:38
*** felipemonteiro_ has joined #openstack-nova15:38
*** yangyapeng has joined #openstack-nova15:38
openstackgerritEric Berglund proposed openstack/nova master: PowerVM Driver: vSCSI volume driver  https://review.openstack.org/52609415:40
*** yamamoto has quit IRC15:43
*** felipemonteiro has quit IRC15:43
stephenfinmriedem: I can't see anything obvious, no. I'm guessing it's just zuul funkiness15:44
stephenfinThere's been a lot of that this week15:44
*** sshwarts has joined #openstack-nova15:45
*** itlinux has quit IRC15:47
*** hamzy has joined #openstack-nova15:48
mriedemstvnoyes: ildikov: jgriffith: about 10pm last night i thought about this https://etherpad.openstack.org/p/multi-attach-volume-queens - see L4815:48
mriedemstephenfin: yeah, thanks for looking. have'nt merged any patches in 2 days...15:48
*** ragiman has quit IRC15:50
ildikovmriedem: does that apply to live_migrate as well?15:50
*** kholkina has quit IRC15:50
*** tidwellr_ has joined #openstack-nova15:50
*** kholkina has joined #openstack-nova15:51
openstackgerritEric Berglund proposed openstack/nova master: PowerVM Driver: vSCSI volume driver  https://review.openstack.org/52609415:51
mriedemildikov: no15:51
mriedemildikov: with live migration, it's the same volume attached on both the source and dest nodes,15:51
mriedemand it's using the same disk config xml which already has the shareable flag set for the source guest xml15:51
*** bkopilov has joined #openstack-nova15:51
ildikovok, I wasn't sure about the config on the dest15:51
mriedemi think this is purely a swap volume wrinkle15:51
*** esberglu has quit IRC15:52
mriedemso i think i need to handle that in between the libvirt patch and https://review.openstack.org/#/c/531244/15:52
mriedemsince https://review.openstack.org/#/c/531244/ bumps the rpc version15:52
stvnoyesmriedem: at this point in Queens, what about explicitly not allowing swap volume if the volume is MA'd? Then we can test swap more exhaustively for rocky...15:52
*** xnox has quit IRC15:53
*** sshwarts has quit IRC15:53
*** tidwellr has quit IRC15:53
mriedemstvnoyes: because that means you'd need a new microversion for swap volume with multiattach support in rocky, and that would mean cinder (using nova client) would have to know to send that microversoin if swapping with a MA volume15:53
*** alexchadin has joined #openstack-nova15:53
mriedemstvnoyes: what needs to happen is like a 3 line code change i think15:54
ildikovmriedem: I'm good with your solution15:54
stvnoyesah ok15:54
mriedemif new_volume['multattach']: new_connection_info['multiattach'] = True15:54
ildikovyep15:54
mriedemthinking about a tempest test scenario to tickle this...15:54
mriedemattach non-multiattach volume A, swap to multiattach volume B, then attach B to a 2nd serer15:55
mriedem*server15:55
mriedemif we didn't fix the swap volume piece, the 2nd attach for B on the 2nd server should be wonky15:55
ildikovmriedem: do we track if the old volume was multiattach, but the new one is not?15:55
*** kholkina has quit IRC15:55
mriedemildikov: no. we can figure that out if needed, but i'm not sure if we care.15:55
mriedemwe did say in the cinder spec that you can't retype an in-use volume to change it's multiattach value15:56
ildikovbut with swap it's a new volume?15:56
ildikovanyway, if the shareable flag cannot appear in the xml by accident for a non-multiattach volume, then good15:57
openstackgerritSylvain Bauza proposed openstack/nova master: libvirt: create vGPU for instance  https://review.openstack.org/52883215:57
*** xnox has joined #openstack-nova15:58
*** xnox has joined #openstack-nova15:58
mriedemi'm not entirely sure how the various retype and volume migration stuff works on the cinder side15:58
mriedemretype is like a resize in nova right? new flavor/volume type15:58
mriedemvolume migration is not a change in volume type, but it migrates the volume from one backend to another of the same type?15:58
*** derekh has quit IRC15:59
mriedemanywho, meeting time15:59
ildikovyeah, I guess you're right, I always confuse myself with the different types of swap...15:59
*** Guest88600 has quit IRC16:03
*** sree has joined #openstack-nova16:04
*** itlinux has joined #openstack-nova16:08
*** sree has quit IRC16:09
*** slaweq has joined #openstack-nova16:09
*** miguel1 has joined #openstack-nova16:09
bauzasmdbooth: updated based on your comments16:10
*** slaweq has quit IRC16:13
*** slaweq_ has joined #openstack-nova16:13
*** mlavalle has quit IRC16:14
*** miguel1 has quit IRC16:14
*** mlavalle has joined #openstack-nova16:15
*** tidwellr_ has quit IRC16:17
*** slaweq_ has quit IRC16:17
openstackgerritMatt Riedemann proposed openstack/nova master: Address nits in change I7e01f95d7173d9217f76e838b3ea71555151ef56  https://review.openstack.org/53289816:18
*** tidwellr has joined #openstack-nova16:18
*** sahid has quit IRC16:19
*** alexchadin has quit IRC16:24
*** alexchadin has joined #openstack-nova16:25
*** esberglu has joined #openstack-nova16:26
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Amend the migration paging spec for uuid in server migrations response  https://review.openstack.org/53290416:27
mriedemnova-multiattach job failed with NODE_FAILURE in the experimental queue :(16:28
mdboothbauzas: Hey, can you try a couple of things on your mdev box and document the results: create more mdevs than permitted, and create an mdev with an invalid name.16:28
bauzasmdbooth: I'm just trying the former :)16:28
*** alexchadin has quit IRC16:29
mdboothbauzas: The BFL around that function is fine by me. How do you plan to address all the other places which call _get_guest_xml?16:29
*** sree has joined #openstack-nova16:30
ildikovmriedem: sadness :(16:31
*** Eran_Kuris has joined #openstack-nova16:34
*** sree has quit IRC16:34
*** maciejjozefczyk has quit IRC16:35
mnaserfrom what i understand, once an instance is scheduled in a cell, it will forever remain in that cell (rebuilds, live migrations, etc?)16:36
*** maciejjozefczyk has joined #openstack-nova16:36
openstackgerritEric Fried proposed openstack/nova master: Track provider traits in report client  https://review.openstack.org/52168616:36
openstackgerritEric Fried proposed openstack/nova master: Track associated sharing RPs in report client  https://review.openstack.org/52653916:36
openstackgerritEric Fried proposed openstack/nova master: Raise on API errors getting aggregates/traits  https://review.openstack.org/52654016:36
openstackgerritEric Fried proposed openstack/nova master: ProviderTree.populate_from_iterable  https://review.openstack.org/52075616:36
openstackgerritEric Fried proposed openstack/nova master: Track tree-associated providers in report client  https://review.openstack.org/52654116:36
openstackgerritEric Fried proposed openstack/nova master: Raise conflict exception on RP create 409  https://review.openstack.org/53256316:36
openstackgerritEric Fried proposed openstack/nova master: SchedulerReportClient.set_traits_for_provider  https://review.openstack.org/53256416:36
openstackgerritEric Fried proposed openstack/nova master: WIP: Scheduler[Report]Client.get_provider_tree  https://review.openstack.org/52109816:36
openstackgerritEric Fried proposed openstack/nova master: WIP: ComputeDriver.update_provider_tree()  https://review.openstack.org/52118716:36
openstackgerritEric Fried proposed openstack/nova master: WIP: Use update_provider_tree from resource tracker  https://review.openstack.org/52024616:36
openstackgerritEric Fried proposed openstack/nova master: Fix nits in update_provider_tree series  https://review.openstack.org/53126016:36
bauzasmdbooth: yup, in separate change16:36
efriedgibi If you're still around, would you please re+W https://review.openstack.org/521686 (just rebased)16:37
bauzasmdbooth: oh shit, I forgot to amend the relnote telling that limitation16:37
bauzas(basically, don't touch your instance for the moment once it's up)16:37
mdboothbauzas: Commented.16:37
efriedmgoddard Un-WIP'd SchedulerReportClient.set_traits_for_provider  https://review.openstack.org/532564 -- see how ya like it.16:37
efriedmgoddard And maybe let's talk about peeling it out of that series so you can make use of it.16:38
mriedemneed a couple of cores to review the 2.57 novaclient change https://review.openstack.org/#/c/528128/16:38
mriedemand 2.58 above it16:38
mriedemand 2.59 is approved16:38
mriedemclient freeze is in 2 weeks16:38
maciejjozefczykmriedem: about https://review.openstack.org/#/c/520024/16:38
maciejjozefczykmriedem: found :)16:38
mdboothbauzas: Is it really useful to land in that state? The code is pretty cleanly separated. It's kinda annoying that the libvirt folks half-landed mdev for eg, and now we can't tell which version we're supposed to required.16:39
maciejjozefczykmriedem: https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py?utf8=✓#L61316:39
bauzasmriedem: ack, top prio list of reviews, starred16:40
bauzasAPI microversion client side changes are like more important than feature prios to me16:40
*** burt has joined #openstack-nova16:40
mriedemmaciejjozefczyk: so it overwrites the ComputeNode allocation ratios which are set in code, to the self allocation ratios set in the RT via the config options?16:41
mriedemmaciejjozefczyk: i'm guessing that's in there in case the operator changed the configs and restarted the service16:41
maciejjozefczykmriedem: yes16:42
mriedembut maybe https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py?utf8=%E2%9C%93#L148 should match what's in the compute node object...16:42
maciejjozefczykmriedem: if not 0; then rewrite ;P ?16:42
mriedemCONF.cpu_allocation_ratio or 16.016:42
mriedemsomething like that16:42
mriedemmaciejjozefczyk: or that16:42
*** gszasz has quit IRC16:42
mriedemit's definitely shitty that we have multiple places writing this value into the ocmpute node record16:42
maciejjozefczykmriedem: so.. should i create other bug and then fix it?16:43
mriedemas evidenced by the fact it took 3 of us 3 hours yesterday to figure this out16:43
maciejjozefczykmriedem: maybe move it to common16:43
mriedemmaciejjozefczyk: sure16:43
mriedemi think we have to leave what's in the compute node object16:43
mriedembut the RT stuff definitely isn't obvious16:43
mriedemfyi to anyone that's interested in what's going on with the gate http://grafana.openstack.org/dashboard/db/zuul-status16:44
mriedemexecutors are maxed out so throughput is down16:45
mriedemrecheck grind is also killing things16:45
mriedemso if you have a patch queued up in zuul, no need to recheck it16:45
mriedemadd the change number to http://status.openstack.org/zuul/ if you want to see if it's queue16:45
mriedem*queued16:45
mriedemor use https://github.com/openstack/gerrit-dash-creator16:46
*** cdent has quit IRC16:47
*** mlavalle has quit IRC16:47
bauzasmdbooth: very interesting, I'm trying to allocate 2 vGPUs on a single VM16:47
bauzaslibvirt fails miserably16:47
*** dtantsur is now known as dtantsur|afk16:47
*** mlavalle has joined #openstack-nova16:47
bauzasI guess because of the nvidia driver16:48
mdboothbauzas: Other fun things: allocate 1 vgpu to 2 vms. Create an mdev of each of 2 different types.16:48
*** mlavalle has quit IRC16:48
mdboothbauzas: It would be awesome to document all these failure modes somewhere in a code comment. Maybe the privsep function?16:49
bauzasmdbooth: my code prevents that16:49
bauzasmdbooth: we only support one type16:49
mdboothYeah, I know.16:49
bauzasand that's documented in the relnote16:49
mdboothbauzas: Right, but I'm just thinking it's interesting to understand our failure modes, and most people don't have the hardware to hand to test it.16:49
bauzasfor the multi-assignment of the same mediated device, that should also be in theory avoided by my code, except if bugs16:49
mdboothWill help reviewers/maintainers, for eg16:50
bauzasmdbooth: I mean, what would you like to see ?16:50
*** aloga_ has quit IRC16:50
* mdbooth is interested to know what the bugs would look like if they happened16:50
bauzasmdbooth: I can try to change the conf opt16:50
mdboothSo do we see a failure on creating the mdev?16:50
*** mlavalle has joined #openstack-nova16:50
mdboothOr subsequently when we try to use it?16:50
bauzasyou mean the privsep call ?16:50
mdboothAnd in either case, what form does the failure take?16:51
mdboothYeah16:51
*** damien_r has quit IRC16:51
bauzaswhat I haven't checked is what happens if I modify the conf opt and ask for another type16:51
mdboothbauzas: That's an extremely nice to have, btw.16:51
*** aloga_ has joined #openstack-nova16:51
bauzasin theory, it should say 0 available resources16:51
bauzasbecause some mediated devices are already created for the existing guests, so the other resources are 016:52
mdboothIf you're testing this stuff anyway, if you could squirrel away what real failures look like it would be really useful.16:52
bauzaswell, for that physical GPU16:52
*** cdent has joined #openstack-nova16:52
bauzasmdbooth: well, an instance leading to an ERROR state is okay to me, if that's documented why16:53
bauzaslike, if you make a flavor that asks for 2 VGPUs, it won't work16:53
bauzasthat's the caveats I mentioned in my note16:53
*** AlexeyAbashkin has quit IRC16:53
*** chyka has joined #openstack-nova16:55
*** pcaruana|afk| has quit IRC16:57
*** felipemonteiro_ has quit IRC16:58
*** penick has joined #openstack-nova17:00
*** mlavalle has quit IRC17:01
*** penick has quit IRC17:04
*** matrohon has quit IRC17:07
openstackgerritEric Berglund proposed openstack/nova master: PowerVM Driver: vSCSI volume driver  https://review.openstack.org/52609417:07
*** fragatina has joined #openstack-nova17:11
dansmithmriedem: cdent: I just want to say that us passing around dicts of placement api goo in our scheduler paths is really not ideal17:12
dansmithI'm changing the microversion I make a call at, which changes that format,17:12
dansmithand thus a bunch of obscure dict manipulation code paths have to change in weird ways17:12
cdentyeah, I made a similar comment on a recent bug report17:13
dansmithif we had a proper client-side model of this it would be a lot easier to maintain17:13
cdentof the 'when we bump to the new microversion this stuff gonna go funkity'17:13
mriedemclient side model like an object?17:13
dansmithmriedem: yes17:13
mriedemleave a todo in the patch?17:14
dansmithif I ever get it working17:14
*** mlavalle has joined #openstack-nova17:15
*** links has joined #openstack-nova17:17
*** links has quit IRC17:17
*** lucasagomes is now known as lucas-afk17:19
*** suresh12 has joined #openstack-nova17:21
*** fragatina has quit IRC17:23
*** fragatina has joined #openstack-nova17:23
*** amotoki has quit IRC17:24
*** penick has joined #openstack-nova17:24
*** rabel__b1 has quit IRC17:27
*** Apoorva has quit IRC17:28
efrieddansmith An object like ProviderTree?  Though also presumably an equivalent for allocation candidates.17:29
dansmithefried: well, kinda yeah17:29
edleafedansmith: aren't we treating them as opaque blobs, as originally designed? I know there was the doubking up hack, but that should no longer be necessary17:31
cdentedleafe: they've changed shape17:31
cdentand they get manipulated17:31
cdentso not really opaque (last I checked)17:31
* edleafe dies a little more17:31
bauzasdansmith: not sure you saw the long convo I had with mdbooth but before I leave for 2.5 hours (will be back later), just food for thoughts17:32
cdentI can't remember why they get manipulated17:32
cdentMy ability to index this stuff for memory is broken17:32
dansmithedleafe: not in the scheduler17:33
dansmithedleafe: I'm not having any problems with the code that treats it as opaque17:33
bauzasdansmith: since we create mediated devices on the fly (or allocate them) based on allocations, and generate the guest config with that, thanks to allocations, it means that when you power off and then power on again, you get a guest that no longer has VGPUs because we don't pass (yet) the allocations to power_on()17:33
*** slaweq has joined #openstack-nova17:33
dansmithbauzas: okay17:33
bauzasdansmith: everything works seamlessly when booting, so https://review.openstack.org/#/c/528832/ is ready for reviews, but I'll need to address that issue in a follow-up17:33
*** homeski has joined #openstack-nova17:34
homeskiQuestion for anyone ...17:34
homeskiI have a instance booted from volume, where the volume gets attached using the IDE bus17:34
homeskiis there a way, using OSP CLI, to have the boot device remounted using virtio ?17:34
homeskiight now I have to manually `virsh edit` the XML17:34
homeskiAlso, if I attach any additional volumes, they always get attached using IDE, even if I have `disk_bus=virtio` set on the volume metadata17:35
homeskiI have to keep messing with the XML manually17:35
dansmithedleafe: and we can't get rid of the doubling until rocky, AFAIK, because pike computes still depends on it17:35
dansmithhomeski: see topic please17:36
homeskigotcha, thx17:36
edleafedansmith: sure, what I was asking was if that was a one-off thing, and that we aren't doing any more of it17:36
dansmithedleafe: that's the biggest wart, yeah17:37
*** slaweq has quit IRC17:37
openstackgerritEric Fried proposed openstack/nova master: ProviderTree.generation_for  https://review.openstack.org/53292217:39
*** aloga_ has quit IRC17:40
openstackgerritEric Berglund proposed openstack/nova master: PowerVM Driver: vSCSI volume driver  https://review.openstack.org/52609417:46
-openstackstatus- NOTICE: Due to an unexpected issue with zuulv3.o.o, we were not able to preserve running jobs for a restart. As a result, you'll need to recheck your previous patchsets17:47
*** Apoorva has joined #openstack-nova17:53
*** sambetts is now known as sambetts|afk17:54
*** jistr is now known as jistr|afk17:55
*** penick has quit IRC17:56
openstackgerritMaciej Jozefczyk proposed openstack/nova master: WIP: Do not normalize allocation_ratios  https://review.openstack.org/53292417:57
maciejjozefczykmriedem: https://review.openstack.org/53292417:58
*** suresh12 has quit IRC17:58
maciejjozefczykmriedem: IMO instead doing additional if statements in RT we should do it clean and set default values in config files as it should be done eariler (starting from newton) - if I understand the problem correctly17:58
maciejjozefczykits draft, without test changes17:59
*** liverpooler has quit IRC17:59
maciejjozefczyktests*17:59
mriedemmaciejjozefczyk: i don't know about the implications of changing the defaults in the config file17:59
*** jdillaman has quit IRC17:59
mriedemthat's all bauzas territory17:59
*** yamamoto has joined #openstack-nova18:00
*** nicolasbock has joined #openstack-nova18:00
mriedemi don't know the history behind why we had 0.0 in config and hard-coded values in code18:00
maciejjozefczykmriedem: It was (at least I can read it from comments) about backwards compatibility to Liberty, i think18:01
maciejjozefczykbauzas: hello :) could you please check https://review.openstack.org/532924 if the idea is ok for you? thanks!18:02
*** liverpooler has joined #openstack-nova18:02
*** ralonsoh has quit IRC18:04
*** yamamoto has quit IRC18:04
*** penick has joined #openstack-nova18:05
dansmithmriedem: for the overcommit ratios?18:05
dansmithallocation_ratio yeah18:05
dansmiththat is our sentinel to the scheduler to obey what the compute reports in the db instead of using config18:06
dansmithwhen we moved that stuff18:06
dansmithif 0.0, then 16.0 for cpu, etc18:06
dansmithheh that patch just undoes it without historical context I think18:06
dansmithbauzas: ^18:06
dansmithhe has the full story in his head I bet18:07
openstackgerritDan Smith proposed openstack/nova master: WIP: Pass limit to /allocation_requests  https://review.openstack.org/53151718:08
*** maciejjozefczyk is now known as maciejjozefczyk_18:09
openstackgerritMatt Riedemann proposed openstack/nova master: Pass multiattach flag to reserve_block_device_name  https://review.openstack.org/53124418:10
openstackgerritMatt Riedemann proposed openstack/nova master: [api] Allow multi-attach in compute api  https://review.openstack.org/27104718:10
openstackgerritMatt Riedemann proposed openstack/nova master: Handle swapping to a multiattach volume  https://review.openstack.org/53292818:10
mriedemildikov: gibi: johnthetubaguy: ^ swap volume change is in the middle18:10
ildikovmriedem: gorgeous, thank you!18:11
*** weshay is now known as weshay_interview18:14
*** jpena is now known as jpena|off18:16
*** jdillaman has joined #openstack-nova18:16
*** AlexeyAbashkin has joined #openstack-nova18:20
*** AlexeyAbashkin has quit IRC18:24
*** mvk has quit IRC18:32
*** suresh12 has joined #openstack-nova18:33
*** Tom-Tom has joined #openstack-nova18:36
*** suresh12 has quit IRC18:39
*** Tom-Tom has quit IRC18:40
*** Tom-Tom has joined #openstack-nova18:41
*** sree has joined #openstack-nova18:43
*** suresh12 has joined #openstack-nova18:44
*** tesseract has quit IRC18:45
*** sree has quit IRC18:47
*** sridharg has quit IRC18:48
*** penick has quit IRC18:50
*** yamahata has joined #openstack-nova18:53
*** felipemonteiro has joined #openstack-nova18:55
*** felipemonteiro_ has joined #openstack-nova18:56
*** AlexeyAbashkin has joined #openstack-nova18:57
*** cdent has quit IRC18:59
*** hemna_ has joined #openstack-nova18:59
*** sree has joined #openstack-nova19:00
*** felipemonteiro has quit IRC19:00
*** AlexeyAbashkin has quit IRC19:01
*** penick has joined #openstack-nova19:04
*** sree has quit IRC19:05
*** sree has joined #openstack-nova19:06
*** sree has quit IRC19:10
*** penick has quit IRC19:10
*** suresh12 has quit IRC19:15
*** penick has joined #openstack-nova19:15
*** liverpooler has quit IRC19:15
*** weshay_interview is now known as weshay19:16
*** suresh12 has joined #openstack-nova19:18
*** penick has quit IRC19:20
*** slaweq has joined #openstack-nova19:20
*** avolkov has quit IRC19:22
*** karthiks has quit IRC19:23
*** karthiks has joined #openstack-nova19:24
openstackgerritEric Fried proposed openstack/nova master: ProviderTree.populate_from_iterable  https://review.openstack.org/52075619:24
openstackgerritEric Fried proposed openstack/nova master: Track tree-associated providers in report client  https://review.openstack.org/52654119:24
openstackgerritEric Fried proposed openstack/nova master: Raise conflict exception on RP create 409  https://review.openstack.org/53256319:24
openstackgerritEric Fried proposed openstack/nova master: SchedulerReportClient.set_traits_for_provider  https://review.openstack.org/53256419:24
openstackgerritEric Fried proposed openstack/nova master: WIP: Scheduler[Report]Client.get_provider_tree  https://review.openstack.org/52109819:24
openstackgerritEric Fried proposed openstack/nova master: WIP: ComputeDriver.update_provider_tree()  https://review.openstack.org/52118719:24
openstackgerritEric Fried proposed openstack/nova master: WIP: Use update_provider_tree from resource tracker  https://review.openstack.org/52024619:24
openstackgerritEric Fried proposed openstack/nova master: Fix nits in update_provider_tree series  https://review.openstack.org/53126019:24
openstackgerritEd Leafe proposed openstack/nova master: Fix invalid UUIDs in test_compute.py  https://review.openstack.org/53147919:24
*** itlinux has quit IRC19:26
*** suresh12 has quit IRC19:26
*** smatzek has quit IRC19:27
mriedemedleafe: comments in the resize + alternates patch https://review.openstack.org/#/c/526436/19:28
*** smatzek has joined #openstack-nova19:28
mriedemas far as i can tell, there is nothing in there claiminig resources on the alternate hosts19:28
*** alexchadin has joined #openstack-nova19:29
mriedemdansmith: we need to get this fixed before rc1 https://review.openstack.org/#/q/topic:bug/1741307+project:openstack/nova19:30
*** suresh12 has joined #openstack-nova19:30
*** sree has joined #openstack-nova19:30
dansmithokay19:30
*** smatzek has quit IRC19:33
*** sree has quit IRC19:34
*** liverpooler has joined #openstack-nova19:38
sean-k-mooneymriedem: if we do a release of os-vif on monday are you ok with bumping the minium version for queens19:38
*** suresh12 has quit IRC19:39
mriedemi won't be around on monday19:40
mriedemi was going to do the os-vif release today, and os-traits for that matter19:40
mriedemnow that you mention it19:40
mriedemdansmith: melwitt: thanks for getting those; i'll add something to the ptg etherpad about migrating caching scheduler people to filter scheduler19:40
sean-k-mooneymriedem: it would be nice to include https://review.openstack.org/#/q/status:open+project:openstack/os-vif+branch:master+topic:bug/1672812 and https://review.openstack.org/#/q/status:open+project:openstack/os-vif+branch:master+topic:bug/1666917 in queens19:41
mriedemjesus19:42
*** eharney has quit IRC19:42
mriedemand what requires the minimum version bumps for those?19:42
*** harlowja has joined #openstack-nova19:43
mriedemwith jay on his deathbed i'm not sure how that's going to get done19:43
sean-k-mooneymriedem: sorry i ment to say upperconstriats not minium version19:43
mriedemu-c bumps are automatic19:43
mriedemg-r bumps are when some consumer needs to rely on the changes19:43
mriedemand presumably we're doing this work so something can use it19:44
*** eharney has joined #openstack-nova19:44
sean-k-mooneyits used by os-vif itself it allow use to not use process exec to create port and insted use the openvswitch python libs19:44
mriedemsean-k-mooney: ok, but that's not a super high priority is it?19:45
mriedemat this point in queens i think that's a bit risky19:45
mriedemand would be better deferred to an early release in rocky19:45
*** suresh12 has joined #openstack-nova19:45
sean-k-mooneyits just a nice to have yes so it we branched today and did a release it would not break anything19:46
mriedemhas claudiub seen this? https://review.openstack.org/#/c/487405/19:47
mriedemi think the hyperv driver is using os-vif so how are they not blowing up?19:47
sean-k-mooneythe code that is pertected by the if would not fail if the if was removed so it does not explode19:47
*** edmondsw_ has joined #openstack-nova19:48
*** edmonds__ has joined #openstack-nova19:48
sean-k-mooneythat said  checking if an interface exists via a file exits check under sys on windows is not exactly well fucntional19:48
mriedemthe other 2 things in https://review.openstack.org/#/q/status:open+project:openstack/os-vif+branch:master+topic:bug/1672812 are not really something we need to get released19:49
mriedemthe testing related patches i mean19:49
mriedemso i'm going to push an os-vif release today, and if something else gets in afteward before next thursday, bauzas or someone can propose a release19:49
sean-k-mooneymriedem: ok that sound fine to me19:50
sean-k-mooneyas i said they are just nice to have not required19:50
mriedemsean-k-mooney: kit me up when rocky rolls around and we can push for the native ovs ones19:50
mriedem*hit me up19:50
*** alexchadin has quit IRC19:51
sean-k-mooneysure. we are keeping support for process exec and not enableing it by default anyway. we had push out makeing that change to S instead.19:51
*** edmondsw has quit IRC19:52
sean-k-mooneyhas a name been chosen for S yet by the way19:52
*** edmondsw_ has quit IRC19:52
*** itlinux has joined #openstack-nova19:54
*** itlinux has quit IRC19:55
*** jistr|afk is now known as jistr19:57
*** smatzek has joined #openstack-nova19:57
*** sree has joined #openstack-nova19:59
*** Apoorva has quit IRC20:01
*** sree has quit IRC20:04
mriedemsquish20:07
mriedemshizer?20:07
*** mvk has joined #openstack-nova20:08
bauzasmriedem: sylvain :p20:10
mriedemthere is nothing german about that20:10
mriedemshizer it is20:10
bauzasschnauze :p20:11
bauzasmriedem: question for https://review.openstack.org/#/c/528128/7/novaclient/exceptions.py20:12
bauzasmriedem: why ?20:12
mriedembecause if you do six.text_type(ex) you don't get the message from the exception20:12
*** sree has joined #openstack-nova20:12
bauzasinteresting to know20:13
mriedemhttps://review.openstack.org/#/c/528128/7/novaclient/tests/unit/v2/test_servers.py@164020:13
mriedemif you remove it, that stuff blows up20:13
dansmithbauzas: if we name it that, people will be pronouncing it wrong for a long time20:13
bauzasI know :p20:14
*** sree has quit IRC20:16
sean-k-mooneylike meltdown and specter im sure marketing will prevent any name being chossen that would not be family frendly20:22
sean-k-mooneyor well are inside/techy jokes20:23
bauzasdansmith: got a sec ?20:25
openstackgerritLee Yarwood proposed openstack/nova master: WIP libvirt: QEMU native LUKS decryption for volumes  https://review.openstack.org/52395820:25
dansmithbauzas: finishing an email, but then yes20:25
bauzasdansmith: cool20:25
bauzasdansmith: that's just about the libvirt issue about regenerating a wrong XML because missing allocs20:26
bauzasso take your time20:26
openstackgerritEric Berglund proposed openstack/nova master: PowerVM Driver: vSCSI volume driver  https://review.openstack.org/52609420:27
efriedCould one of y'all please re+W https://review.openstack.org/#/c/521686/ (auto rebase)20:27
*** penick has joined #openstack-nova20:27
bauzasefried: .20:28
efriedbauzas Merci bien.20:28
*** READ10 has quit IRC20:29
*** Tom-Tom has quit IRC20:31
*** pchavva has quit IRC20:31
mriedemstvnoyes: on those tempest patches, when you push the changes, you need to use git review -R20:32
*** Tom-Tom has joined #openstack-nova20:32
mriedemyou pulled my approved changes out of the gate20:32
mriedemstvnoyes: i thought you were going to push a separate patch on top of https://review.openstack.org/#/c/266605/ ? not add to it.20:32
*** imacdonn has quit IRC20:33
stvnoyeshmm cr*p. what's the best way to undo that?20:33
*** imacdonn has joined #openstack-nova20:33
stvnoyesi can submit a new change that pulls it out...20:33
claudiubmriedem: so, at the moment, hyperv-neutron-agent is still used in the CI. But i've fetched that patch and instances still spawn when using OVS.20:34
mriedemstvnoyes: too late now20:34
mriedemwell too late for the git review -R20:34
mriedemstvnoyes: you could just put the resize test on top in a different change20:34
mriedemjust don't git commit --amend20:34
*** tbachman has quit IRC20:35
stvnoyesjust to be clear, on top, you mean by using depends on?20:35
*** penick has quit IRC20:35
mriedemno20:35
mriedemstack the changes like i've done in that series20:35
mriedemyou don't need depends-on within the same repo20:35
mriedemso just remove the resize test from the top patch, then commit it into a new change at the top of the series,20:36
sean-k-mooneyclaudiub: ya the way os-vif is written it should work with or without the patch but it annoys me that it works just because we happened ot make the functions not fail if there preconditions were violated20:36
mriedemand then git review -R -y20:36
stvnoyesok20:36
*** Tom-Tom has quit IRC20:36
mriedemif you need to go back within the series to change something, you do an interactive rebase (git rebase -i HEAD~4) or whatever20:36
*** tbachman has joined #openstack-nova20:37
efriedgit restack is your friend20:39
efriedWell, *my* friend, anyway.20:40
sean-k-mooneyefried: git restack?20:40
sean-k-mooneyi use git rebase -i often but never heard of git restack20:41
efriedsean-k-mooney It rolls up rebase -i and some of the other paperwork associated with managing a big series.20:41
*** eharney has quit IRC20:41
efriedsean-k-mooney I think it was developed by stackers.  Separate plugin.  pip install git-restack (I think)20:41
sean-k-mooneyhttps://github.com/openstack-infra/git-restack so yes20:41
sean-k-mooneygit review is also an infra tool20:41
efriedno kidding, really?  TIL20:42
efriedWas using that long before I got involved with openstack.20:42
sean-k-mooneyyep https://github.com/openstack-infra/git-review20:42
*** matrohon has joined #openstack-nova20:43
sean-k-mooneyefried: stephenfin: also intoduced me to gerrithub.io which allows you to add gerrit to any github project so you can use the openstack/gerrit workflow with it insteadk of pull requests20:44
efriedoooooooo20:44
bauzasI don't like git restack20:44
*** penick has joined #openstack-nova20:44
bauzasI prefer the old but clean way20:44
efriedWould you like it better if it was git restaque?20:45
bauzasbecause something you can be messed up in a long series and then git reflog is your friend20:45
* efried looks up reflog...20:45
*** penick has quit IRC20:45
dansmithbauzas: you never started talking20:45
bauzasdansmith: I was holding my breath till your email :p20:46
dansmithbauzas: I was expecting you to start asking your question and I'd get to it when I was done20:46
dansmithbauzas: but then I got distracted with no ping :)20:46
bauzasmy bad, misunderstanding20:46
dansmithanyway, shoot20:46
bauzasso20:46
bauzasdansmith: in case you don't know yet, we're attaching a mdev device based on the allocation20:47
bauzasthat works fine20:47
bauzasbut the issue we have is that there are a shit ton of other methods that rebuild that guest XML, and those don't have the allocations passed yet20:47
dansmith1.0 shit tons?\20:48
bauzasI mean, spawn() is calling it, but also for example hard_reboot20:49
bauzaswhich itself is called by power_on()...20:49
bauzashttps://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L265820:49
dansmithyeah, I get it20:49
bauzasso, I tested on my node, and indeed, if you stop/start your instance, you're not missing the mdev20:50
bauzasyou're missing*20:50
dansmithnot missing?20:50
dansmithheh yeah okay20:50
bauzasso there are 2 options20:50
*** eharney has joined #openstack-nova20:51
bauzas#1 pass the allocations down to where we need, exactly like we did for spawn()20:51
bauzasthat means we need to modify every caller20:51
*** aloga_ has joined #openstack-nova20:51
bauzas#2 introspect the flavor rather than allocations20:51
dansmith#2 is cheating20:52
*** eharney has quit IRC20:52
dansmiththe reason I asked about the fraction of shit tons is... how many actual calls are we talking about here?20:52
*** penick has joined #openstack-nova20:52
*** mm6021 has joined #openstack-nova20:53
bauzaslemme grep that20:53
bauzasmy grep tells me 6 times20:54
dansmiththat doesn't seem untenable to me20:54
dansmithgranted you have to do it in multiple drivers, but.. it's not a billion20:55
dansmiththat's like 0.0001 shit tons :)20:55
bauzasresume, unrescue, finish_migration, power_on and so far20:55
dansmithI guess the other thing we need to do is cache those in compute manager for when we need them because spawn brings them, but the other calls won't20:55
dansmithso the other way to look at this is,20:55
bauzasdansmith: 'those' == allocs ?20:56
dansmiththat maybe this is a libvirt problem20:56
dansmithyes those == allocs20:56
dansmithlike, I wonder if the other drivers would have this problem, or if they would just not blindly regenerate the vm definition20:56
*** bigdogstl has joined #openstack-nova20:56
dansmithyou could just read the xml to get the mdev info before you blow it away, right?20:57
*** matrohon has quit IRC20:57
dansmithand you wouldn't need the full allocation20:57
bauzasdansmith: that's something I wondered earlier in the day20:57
bauzasI mean I was surprised to see we were generating the XML every time20:57
dansmithwell,20:57
dansmithI don't think that refreshing it all the time is bad,20:57
dansmithbecause it gives us a chance to update things like the ceph monitor ip address and such20:58
dansmithbut we don't have to do it blindly20:58
bauzasyeah I can understand that reasoning20:58
bauzasbut yeah, instead of just amending what's needed, we're just wiping out the whole config20:58
bauzasanyway20:58
bauzasthis is what it is20:58
dansmithyou could probably write a clean utility method that is "get me the mdev(s) from $guest" and just call that everywhere20:58
*** kenperkins has joined #openstack-nova20:59
dansmitheverywhere you need it that is.. don't call it EVERYWHERE :P20:59
dansmithand don't call it everywhere() because that won't be very obvious20:59
* dansmith is a barrel of laughs this afternoon20:59
bauzasI need to look at the driver code21:00
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Fix being able to create a reno using tox -e venv  https://review.openstack.org/53297121:00
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated MigrationManager.list cell_name kwarg  https://review.openstack.org/53297221:00
*** Apoorva has joined #openstack-nova21:02
*** krtaylor has joined #openstack-nova21:02
cfriesenIs it documented anywhere that deleting a quota-set for a tenant will also delete it for any users within that tenant?  (This seems to be how it behaves, but I don't see it in the API ref.)21:02
*** david-lyle has quit IRC21:03
melwittcfriesen: user quota is a subset of tenant quota so I think it would behave that way. and I doubt it's documented21:03
*** krtaylor has quit IRC21:03
*** david-lyle has joined #openstack-nova21:04
cfriesenmelwitt: I can see the rationale, was just a bit unexpected.21:04
*** krtaylor has joined #openstack-nova21:04
bauzasdansmith: mmm, your last approach could work, except for migrations (but I think it's a silverlining)21:05
dansmithbauzas: we should have allocations passed for migrations I imagine21:05
bauzasdansmith: I mean, we can get the guest, hence its mdevs, before we regenerate it21:05
dansmithwe have to look them up anyway21:05
melwittcfriesen: yeah. user quota was an afterthought. originally it was tenant quota only (and I think it might go back that way eventually). so to tack on user quota, all they did was add a user column and filter on project AND user if user quota21:05
bauzasdansmith: right, I just feel migrations are off the scope for queens21:06
*** Apoorva has quit IRC21:06
dansmithyeah, but it's good to think about how it'd be handled21:06
stvnoyesmriedem: how exactly do I remove the test from the top patch? I don't want to make things worse.  (git is not my strong point).21:06
dansmithwe don't even know if libvirt could do that I guess21:06
mriedemstvnoyes: you just undo your change21:07
mriedemstvnoyes: want me to do it?21:07
*** gouthamr has quit IRC21:07
stvnoyesthat might be easier, just let me know what you did, for future reference.21:07
*** edmonds__ is now known as edmondsw21:07
stvnoyesonce you do that I'll submit a new change21:08
edmondswmriedem sdague the PowerVM vSCSI patch is also ready for review now, which is the last one for that bp21:09
sean-k-mooneydansmith: bauzas just reading back im not that surprised to seee the xml being regenerated on hard reboots its requried for some config changes21:09
dansmithsean-k-mooney: yup21:09
*** liverpooler has quit IRC21:10
dansmithI've argued for doing it more often than we need even so we can freshen it, since some instances run for years without us getting to tweak things21:10
sean-k-mooneybauzas: you will need to be careful not to miss livemigration too as we regenerate part of the xml there also + resize21:10
dansmithwe just don't need to do it blindly21:10
*** masber has joined #openstack-nova21:10
*** matrohon has joined #openstack-nova21:12
*** sree has joined #openstack-nova21:12
cfriesensean-k-mooney: and for "proper" resource tracking on live migration with we need to update the XML before the live-migration call and update it with the resources to use on the new compute node.21:14
mriedemstvnoyes: done21:14
*** gouthamr has joined #openstack-nova21:15
sean-k-mooneycfriesen: yep we discussed that in some depth in the multiple port binding spec21:15
mriedemstvnoyes: i did this basically:21:15
mriedem1. git review -d 26660521:15
*** bigdogstl has quit IRC21:15
mriedemto pull down the series from gerrit since it's no longer my local changes21:15
mriedem2. removed your test and committed that21:15
sean-k-mooneycfriesen: in that specific case we were talking about nics but its the same problem statement just in a different domain21:16
mriedem3. added your test as a new commit21:16
mriedem4. changed the author to you: git commit --amend --author <steve-o>21:16
mriedem5. push it up: git review -R -y21:16
*** sree has quit IRC21:17
stvnoyesmriedem: thanks21:18
*** penick has quit IRC21:18
*** penick has joined #openstack-nova21:23
*** aloga_ has quit IRC21:25
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecate certs CLIs and python bindings  https://review.openstack.org/53297421:25
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated certs CLIs and python bindings  https://review.openstack.org/53297421:25
*** suresh12 has quit IRC21:30
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated cloudpipe CLIs and python API bindings  https://review.openstack.org/53297621:31
*** mlavalle has quit IRC21:32
*** mlavalle has joined #openstack-nova21:33
*** oomichi_ has joined #openstack-nova21:33
*** cleong has quit IRC21:33
*** smatzek has quit IRC21:42
bauzassean-k-mooney: dansmith: understood, will consider migrations in general by looking at allocations21:42
bauzassean-k-mooney: dansmith: that said, my change is working for just spawning new instances https://review.openstack.org/#/c/528832/21:43
bauzasso you can review it21:43
*** sree has joined #openstack-nova21:43
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated MigrationManager.list cell_name kwarg  https://review.openstack.org/53297221:45
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated certs CLIs and python bindings  https://review.openstack.org/53297421:45
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated cloudpipe CLIs and python API bindings  https://review.openstack.org/53297621:45
*** bigdogstl has joined #openstack-nova21:45
*** eharney has joined #openstack-nova21:46
*** rcernin has joined #openstack-nova21:46
*** jackie-truong has joined #openstack-nova21:47
*** sree has quit IRC21:48
*** eharney has quit IRC21:48
*** suresh12 has joined #openstack-nova21:51
*** bigdogstl has quit IRC21:53
*** slaweq has quit IRC21:55
*** slaweq has joined #openstack-nova21:56
*** threestrands has joined #openstack-nova21:58
*** threestrands has quit IRC21:58
*** threestrands has joined #openstack-nova21:58
*** hamzy has quit IRC21:58
*** threestrands has quit IRC21:59
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecate os-hosts CLIs and python API bindings  https://review.openstack.org/53298921:59
*** threestrands has joined #openstack-nova21:59
*** threestrands has quit IRC21:59
*** threestrands has joined #openstack-nova21:59
mriedemlet's fing remove everything21:59
*** threestrands has quit IRC22:00
*** threestrands has joined #openstack-nova22:01
*** threestrands has quit IRC22:02
*** gyee has joined #openstack-nova22:02
*** threestrands has joined #openstack-nova22:02
ildikovmriedem: made my day; 'fing' in Hungarian means 'fart' :)22:03
*** Apoorva has joined #openstack-nova22:03
mriedemexcellent22:03
mriedemmtreinish: how do you say "staying alive" in czech again?22:03
mriedemhodina ho?22:04
*** sree has joined #openstack-nova22:06
*** penick has quit IRC22:07
*** penick has joined #openstack-nova22:07
*** bjolo has quit IRC22:09
*** nicolasbock has quit IRC22:10
*** sree has quit IRC22:10
*** bigdogstl has joined #openstack-nova22:12
*** kumarmn has joined #openstack-nova22:13
openstackgerritEric Fried proposed openstack/nova master: Raise conflict exception on RP create 409  https://review.openstack.org/53256322:14
openstackgerritEric Fried proposed openstack/nova master: SchedulerReportClient.set_traits_for_provider  https://review.openstack.org/53256422:15
openstackgerritEric Fried proposed openstack/nova master: Track associated sharing RPs in report client  https://review.openstack.org/52653922:15
openstackgerritEric Fried proposed openstack/nova master: Raise on API errors getting aggregates/traits  https://review.openstack.org/52654022:15
openstackgerritEric Fried proposed openstack/nova master: ProviderTree.populate_from_iterable  https://review.openstack.org/52075622:15
openstackgerritEric Fried proposed openstack/nova master: Track tree-associated providers in report client  https://review.openstack.org/52654122:15
openstackgerritEric Fried proposed openstack/nova master: WIP: Scheduler[Report]Client.get_provider_tree  https://review.openstack.org/52109822:15
openstackgerritEric Fried proposed openstack/nova master: WIP: ComputeDriver.update_provider_tree()  https://review.openstack.org/52118722:15
openstackgerritEric Fried proposed openstack/nova master: WIP: Use update_provider_tree from resource tracker  https://review.openstack.org/52024622:15
openstackgerritEric Fried proposed openstack/nova master: Fix nits in update_provider_tree series  https://review.openstack.org/53126022:15
*** matrohon has quit IRC22:15
openstackgerritEric Fried proposed openstack/nova master: WIP: SchedulerReportClient.set_aggregates_for_provider  https://review.openstack.org/53299522:15
*** dave-mccowan has quit IRC22:16
*** dave-mccowan has joined #openstack-nova22:16
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated fixedip/floatingip/virtual interface CLIs/APIs  https://review.openstack.org/53299922:21
*** awaugama has quit IRC22:22
*** Apoorva_ has joined #openstack-nova22:23
*** tidwellr has quit IRC22:23
*** tidwellr has joined #openstack-nova22:23
*** NostawRm has quit IRC22:23
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated os-hosts CLIs and python API bindings  https://review.openstack.org/53298922:24
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated fixedip/floatingip/virtual interface CLIs/APIs  https://review.openstack.org/53299922:24
*** NostawRm has joined #openstack-nova22:25
*** dave-mccowan has quit IRC22:25
mtreinishmriedem: "Hodina H" is the title22:25
mriedemyeah but the song is called staying alive22:25
*** Apoorva has quit IRC22:26
*** lyan_ has quit IRC22:26
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated fixedip/floatingip/virtual interface CLIs/APIs  https://review.openstack.org/53299922:26
*** bigdogstl has quit IRC22:26
*** bigdogstl has joined #openstack-nova22:26
*** tidwellr has quit IRC22:28
*** Apoorva_ has quit IRC22:30
*** kumarmn has quit IRC22:30
*** itlinux has joined #openstack-nova22:30
*** mlavalle has quit IRC22:31
*** Apoorva has joined #openstack-nova22:31
*** mlavalle has joined #openstack-nova22:31
*** sree has joined #openstack-nova22:32
*** sree has quit IRC22:36
*** esberglu has quit IRC22:36
*** sree has joined #openstack-nova22:38
*** sree has quit IRC22:43
*** vladikr has quit IRC22:43
*** bjolo has joined #openstack-nova22:44
*** kenperkins has quit IRC22:45
*** slaweq has quit IRC22:46
*** markvoelker has quit IRC22:49
*** markvoelker has joined #openstack-nova22:49
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated fixedip/floatingip/virtual interface CLIs/APIs  https://review.openstack.org/53299922:52
openstackgerritMatt Riedemann proposed openstack/python-novaclient master: Remove deprecated services binary CLI arg  https://review.openstack.org/53300522:52
mriedemVek is earning his salt today22:52
*** Jeffrey4l has quit IRC22:53
mriedemmelwitt: i'm going to try and get https://review.openstack.org/#/q/topic:bug/1668743+status:open into novaclient and then we'll cut the major 10.0.0 release22:53
*** markvoelker has quit IRC22:54
*** andreas_s has joined #openstack-nova22:54
mriedemit's always nice to flush the toilet once every 6 months22:54
*** sree has joined #openstack-nova22:54
*** edmondsw has quit IRC22:54
melwittlol22:54
*** penick has quit IRC22:54
*** edmondsw has joined #openstack-nova22:55
*** felipemonteiro_ has quit IRC22:55
*** itlinux has quit IRC22:55
*** Jeffrey4l has joined #openstack-nova22:56
*** markvoelker has joined #openstack-nova22:56
*** itlinux has joined #openstack-nova22:57
*** penick has joined #openstack-nova22:57
*** sree has quit IRC22:58
*** jackie-truong has quit IRC22:58
*** edmondsw has quit IRC22:59
*** markvoelker has quit IRC22:59
*** andreas_s has quit IRC23:00
*** markvoelker has joined #openstack-nova23:00
*** penick has quit IRC23:01
*** markvoelker has quit IRC23:05
*** sree has joined #openstack-nova23:05
*** sree has quit IRC23:10
*** itlinux has quit IRC23:12
*** browne has joined #openstack-nova23:14
*** browne has quit IRC23:15
openstackgerritLee Yarwood proposed openstack/nova master: WIP libvirt: QEMU native LUKS decryption for volumes  https://review.openstack.org/52395823:17
*** bigdogstl has quit IRC23:19
*** bigdogstl has joined #openstack-nova23:31
*** bigdogstl has quit IRC23:36
mriedemedleafe: left comments on the resize patch with what i think needs to happen wrt allocations getting moved from the primary selected host to the alternate23:41
*** smatzek has joined #openstack-nova23:43
*** sree has joined #openstack-nova23:43
*** bigdogstl has joined #openstack-nova23:46
*** takashin has joined #openstack-nova23:47
*** smatzek has quit IRC23:47
*** sree has quit IRC23:47
*** hongbin has quit IRC23:53
gmannmriedem: re: https://review.openstack.org/#/c/330406/85/nova/api/openstack/compute/migrations.py23:53
gmannmriedem: it will stop schema validation for legacy v2 compatibility mode23:54
gmannmriedem: https://github.com/openstack/nova/blob/90a92d33edaea2b7411a5fd528f3159a486e1fd0/nova/api/validation/__init__.py#L7423:54
rybridgesGood afternoon! Might have found a bug with user_storage_quota in glance-api.conf. It looks like when we are out of quota and try to directly upload an image we get the error on the command line "Image exceeds the storage quota". However, when we attempt to make a snapshot with "openstack server image create", we do not see the error on the command line. The snapshot gets created and is in queued23:57
rybridgesstate, but then just disappears. We then see a stack trace on the hypervisor with the error in the log, but the user is never directly informed23:57
rybridgesIs this the expected behavior?23:57
gmannmriedem: 1 query on this. why we did not add paging on GET /servers/{server_id}/migrations -https://review.openstack.org/#/c/532904/123:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!