| *** agopi|out has quit IRC | 00:00 | |
| *** macza has quit IRC | 00:04 | |
| *** markvoelker has quit IRC | 00:05 | |
| *** hongbin has quit IRC | 00:13 | |
| *** tetsuro has joined #openstack-nova | 00:16 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: docs: ComputeDriver.update_provider_tree in nova https://review.openstack.org/633375 | 00:17 |
|---|---|---|
| *** liuyulong has quit IRC | 00:22 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: Add descriptions for vol-backed snapshots https://review.openstack.org/615084 | 00:24 |
| *** markvoelker has joined #openstack-nova | 00:26 | |
| *** mriedem has quit IRC | 00:29 | |
| *** tbachman_ has joined #openstack-nova | 00:29 | |
| *** tbachman has quit IRC | 00:30 | |
| *** tbachman_ is now known as tbachman | 00:30 | |
| *** tbachman_ has joined #openstack-nova | 00:35 | |
| *** tbachman has quit IRC | 00:36 | |
| *** tbachman_ is now known as tbachman | 00:36 | |
| *** erlon_ has joined #openstack-nova | 00:36 | |
| *** betherly has joined #openstack-nova | 00:53 | |
| *** bhagyashris has joined #openstack-nova | 00:57 | |
| *** ileixe has joined #openstack-nova | 00:58 | |
| *** betherly has quit IRC | 00:58 | |
| *** wolverineav has quit IRC | 01:00 | |
| *** sapd1 has joined #openstack-nova | 01:01 | |
| *** erlon_ has quit IRC | 01:04 | |
| *** liumk_ has joined #openstack-nova | 01:12 | |
| *** gyee has quit IRC | 01:15 | |
| *** whoami-rajat has joined #openstack-nova | 01:19 | |
| *** wolverineav has joined #openstack-nova | 01:28 | |
| *** cdent has quit IRC | 01:42 | |
| *** tiendc has joined #openstack-nova | 01:43 | |
| *** hongbin has joined #openstack-nova | 02:07 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix resetting non-persistent fields when saving obj https://review.openstack.org/636271 | 02:12 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix resetting non-persistent fields when saving obj https://review.openstack.org/636271 | 02:12 |
| *** Dinesh_Bhor has joined #openstack-nova | 02:18 | |
| *** wolverineav has quit IRC | 02:34 | |
| *** psachin has joined #openstack-nova | 02:55 | |
| openstackgerrit | Merged openstack/nova master: Document how to make tests log at DEBUG level https://review.openstack.org/636744 | 02:56 |
| openstackgerrit | Merged openstack/nova master: Drop specific versions of openSUSE-based distributions https://review.openstack.org/636745 | 02:56 |
| openstackgerrit | Merged openstack/nova master: tox: Don't write byte code (maybe) https://review.openstack.org/636378 | 02:56 |
| openstackgerrit | Merged openstack/nova master: Use tox 3.1.1 fixes https://review.openstack.org/582392 | 02:56 |
| *** betherly has joined #openstack-nova | 03:00 | |
| *** betherly has quit IRC | 03:04 | |
| *** Cardoe has joined #openstack-nova | 03:18 | |
| *** markvoelker has quit IRC | 03:27 | |
| *** markvoelker has joined #openstack-nova | 03:27 | |
| *** markvoelker has quit IRC | 03:32 | |
| *** agopi|out has joined #openstack-nova | 03:40 | |
| *** Dinesh_Bhor has quit IRC | 03:40 | |
| *** lbragstad has joined #openstack-nova | 03:46 | |
| *** tzumainn has quit IRC | 03:56 | |
| *** Dinesh_Bhor has joined #openstack-nova | 04:03 | |
| openstackgerrit | Merged openstack/nova master: Update help messages for weight multipliers https://review.openstack.org/633446 | 04:05 |
| openstackgerrit | Merged openstack/nova master: Use :oslo-config: role in hypervisor-kvm doc https://review.openstack.org/634798 | 04:05 |
| openstackgerrit | sean mooney proposed openstack/os-vif master: [WIP] remove brctl from vif_plug_ovs https://review.openstack.org/636821 | 04:07 |
| openstackgerrit | sean mooney proposed openstack/os-vif master: [WIP] remove use of brctl from vif_plug_linux_bridge https://review.openstack.org/636822 | 04:07 |
| *** wolverineav has joined #openstack-nova | 04:17 | |
| *** wolverineav has quit IRC | 04:21 | |
| *** markvoelker has joined #openstack-nova | 04:28 | |
| *** udesale has joined #openstack-nova | 04:36 | |
| *** ileixe has quit IRC | 04:39 | |
| *** wolverineav has joined #openstack-nova | 04:39 | |
| *** ileixe has joined #openstack-nova | 04:42 | |
| *** owalsh_ has joined #openstack-nova | 04:49 | |
| *** owalsh has quit IRC | 04:52 | |
| *** sridharg has joined #openstack-nova | 04:56 | |
| *** markvoelker has quit IRC | 05:02 | |
| *** ileixe has quit IRC | 05:03 | |
| *** ileixe has joined #openstack-nova | 05:04 | |
| *** ratailor has joined #openstack-nova | 05:09 | |
| *** wolverineav has quit IRC | 05:18 | |
| *** tetsuro has quit IRC | 05:40 | |
| *** hongbin has quit IRC | 05:44 | |
| *** abhishekk has joined #openstack-nova | 05:56 | |
| *** mrjk_ has joined #openstack-nova | 05:58 | |
| *** markvoelker has joined #openstack-nova | 05:58 | |
| *** mrjk_ has quit IRC | 06:00 | |
| *** macza has joined #openstack-nova | 06:01 | |
| *** tetsuro has joined #openstack-nova | 06:03 | |
| *** tetsuro has quit IRC | 06:05 | |
| *** macza has quit IRC | 06:05 | |
| *** _fragatina has quit IRC | 06:12 | |
| *** Cardoe has quit IRC | 06:18 | |
| *** cfriesen has quit IRC | 06:21 | |
| *** _fragatina has joined #openstack-nova | 06:22 | |
| *** mmethot has quit IRC | 06:22 | |
| *** cfriesen has joined #openstack-nova | 06:25 | |
| *** Cardoe has joined #openstack-nova | 06:30 | |
| *** dpawlik has joined #openstack-nova | 06:31 | |
| *** markvoelker has quit IRC | 06:32 | |
| *** _fragatina has quit IRC | 06:37 | |
| *** Cardoe has quit IRC | 06:40 | |
| *** ccamacho has quit IRC | 06:41 | |
| *** _fragatina has joined #openstack-nova | 06:48 | |
| *** Cardoe has joined #openstack-nova | 06:50 | |
| *** dpawlik has quit IRC | 06:55 | |
| *** Luzi has joined #openstack-nova | 06:58 | |
| *** wolverineav has joined #openstack-nova | 07:05 | |
| *** dpawlik has joined #openstack-nova | 07:05 | |
| *** slaweq has joined #openstack-nova | 07:08 | |
| *** wolverineav has quit IRC | 07:09 | |
| *** janki has joined #openstack-nova | 07:11 | |
| *** janki has quit IRC | 07:13 | |
| *** janki has joined #openstack-nova | 07:13 | |
| *** liumk_ has quit IRC | 07:21 | |
| *** jangutter has joined #openstack-nova | 07:21 | |
| *** gokhani has quit IRC | 07:24 | |
| *** markvoelker has joined #openstack-nova | 07:28 | |
| *** macza has joined #openstack-nova | 07:32 | |
| *** macza has quit IRC | 07:36 | |
| *** ralonsoh has joined #openstack-nova | 07:36 | |
| *** _fragatina has quit IRC | 07:57 | |
| openstackgerrit | Merged openstack/nova master: Provide a useful error message when trying to update non-compute services https://review.openstack.org/620667 | 08:00 |
| openstackgerrit | Merged openstack/nova master: Ignore VolumeAttachmentNotFound exception in compute.manager https://review.openstack.org/632657 | 08:00 |
| *** markvoelker has quit IRC | 08:01 | |
| *** tkajinam has quit IRC | 08:09 | |
| *** cfriesen has quit IRC | 08:13 | |
| *** tesseract has joined #openstack-nova | 08:14 | |
| *** awalende has joined #openstack-nova | 08:16 | |
| *** rpittau has joined #openstack-nova | 08:17 | |
| *** ccamacho has joined #openstack-nova | 08:22 | |
| *** xek has joined #openstack-nova | 08:40 | |
| *** helenafm has joined #openstack-nova | 08:42 | |
| openstackgerrit | Yongli He proposed openstack/nova master: Add server subresouce toplogy API https://review.openstack.org/621476 | 08:42 |
| openstackgerrit | garyk proposed openstack/nova master: Lock detach_volume https://review.openstack.org/636701 | 08:43 |
| *** lbragstad has quit IRC | 08:47 | |
| *** helenafm has quit IRC | 08:50 | |
| *** tssurya has joined #openstack-nova | 08:54 | |
| *** wolverineav has joined #openstack-nova | 08:57 | |
| *** helenafm has joined #openstack-nova | 08:58 | |
| *** markvoelker has joined #openstack-nova | 08:58 | |
| *** panda|off is now known as panda | 09:00 | |
| *** wolverineav has quit IRC | 09:01 | |
| *** sridharg has quit IRC | 09:15 | |
| *** dtantsur|afk is now known as dtantsur | 09:20 | |
| *** markvoelker has quit IRC | 09:31 | |
| * bauzas wrapping his head around triyng to understand some reshape issue with func test | 09:34 | |
| *** derekh has joined #openstack-nova | 09:37 | |
| *** stakeda has quit IRC | 09:40 | |
| *** bhagyashris has quit IRC | 09:42 | |
| bauzas | gibi: around ? | 09:46 |
| bauzas | gibi: I don't get why when entering the reshape here https://review.openstack.org/#/c/599208/13/nova/virt/libvirt/driver.py@7000 we don't have the VGPU inventory | 09:48 |
| gibi | bauzas: just from a phone. I will be around properly during my afternoon (UTC+1) | 09:48 |
| bauzas | gibi: ack, I probably need your help to understand the problem | 09:49 |
| bauzas | gibi: just to explain the issue, we don't trigger the reshape since the root inventory we have at first time doesn't have yet the VGPU RC | 09:49 |
| bauzas | it will be there later | 09:49 |
| *** tetsuro has joined #openstack-nova | 09:50 | |
| gibi | good hint, can be some race condition in the test | 09:50 |
| *** tetsuro has quit IRC | 09:51 | |
| bauzas | gibi: I feel we probably trigger the reshape *before* we add the inventory to the root RP | 09:52 |
| *** ociuhandu has joined #openstack-nova | 09:56 | |
| ioni | hello | 10:02 |
| ioni | did you guys noticed that novnc sometimes doesn't have the right keymap, i think. the problem is that i cannot type | in the console, whenever i try to use shift+\ for |, the character is > instead of | | 10:02 |
| openstackgerrit | Chris Dent proposed openstack/nova master: Correct lower-constraints.txt and the related tox job https://review.openstack.org/622972 | 10:02 |
| lyarwood | mdbooth_: https://review.openstack.org/#/c/551349/ - Anything more you wanted to add to this btw? | 10:05 |
| stephenfin | ioni: What version of noVNC are you using? You need 1.0 for proper keymap support | 10:05 |
| ioni | stephenfin, it's the one that comes with openstack ansible queens version | 10:06 |
| ioni | novncproxy_git_repo: https://github.com/novnc/noVNC | 10:06 |
| ioni | novncproxy_git_install_branch: 37b4d13db81e0e80e117c07b86ff98714c7b6b1a # Frozen HEAD of "master" as of 10.02.2018 | 10:06 |
| *** sridharg has joined #openstack-nova | 10:06 | |
| ioni | stephenfin, i think on rocky is the same | 10:06 |
| ioni | stephenfin, i tried latest master and had the same problem but it was giving me the wrong server console for some reason and i reverted | 10:07 |
| stephenfin | ioni: Oh yeah, that's 1.0.0, which should be fine | 10:07 |
| ioni | had the same problem with | | 10:07 |
| stephenfin | Then I'm not sure. You should not have the '[vnc] keymap' nova.conf setting configured | 10:08 |
| stephenfin | So if that's set, unset it | 10:08 |
| ioni | ok so keymap = None ? | 10:08 |
| gibi | bauzas: the test first boot an instance while the vgpu resource is still on the root RP | 10:09 |
| ioni | default on queens is en-us | 10:09 |
| gibi | bauzas: and check the allocation and usages | 10:09 |
| ioni | 2019-02-14 12:09:31.814 2988 DEBUG oslo_service.service [req-b9b6b811-d6c2-49f7-9006-47e60b8d17a7 - - - - -] vnc.keymap = en-us log_opt_values /openstack/venvs/nova-17.1.7/lib/python2.7/site-packages/oslo_config/cfg.py:2898 | 10:09 |
| ioni | 2019-02-14 12:09:31.874 2988 DEBUG oslo_service.service [req-b9b6b811-d6c2-49f7-9006-47e60b8d17a7 - - - - -] spice.keymap = en-us log_opt_values /openstack/venvs/nova-17.1.7/lib/python2.7/site-packages/oslo_config/cfg.py:2898 | 10:09 |
| mdbooth_ | lyarwood: Not specifically. | 10:09 |
| ioni | it's starts with this default if is not set | 10:10 |
| *** mdbooth_ is now known as mdbooth | 10:10 | |
| gibi | bauzas: anyhow, i will know more in 2 hours when i finally download and run the test | 10:10 |
| bauzas | gibi: so, I tried to look at where we were, and I think we have a race | 10:10 |
| stephenfin | ioni: You might want to look at commits d983234288728427235ef2c1f355ec135119b865 and cab8139498c7ea6b05cfdc8b4997276051b943fc in nova | 10:10 |
| stephenfin | The first of those probably should have been backported, if it wasn't | 10:11 |
| stephenfin | Looks like it is in stable/queens, actually | 10:11 |
| bauzas | gibi: when https://review.openstack.org/#/c/631559/5/nova/tests/functional/libvirt/test_reshape.py@123, we don't have yet VGPU inventory since we just changed the conf opt | 10:11 |
| bauzas | gibi: so, when entering .start(), we eventually go into https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L942 | 10:12 |
| stephenfin | ioni: OK, so override the default and set it to '' (the empty string) | 10:12 |
| ioni | stephenfin, ok, testing | 10:13 |
| *** cdent has joined #openstack-nova | 10:13 | |
| bauzas | gibi: but the provider tree we pass is not already having VGPUs, hence the reshape being not triggered | 10:13 |
| ioni | i think i did test that but i just found out that i need to restart the instance | 10:13 |
| bauzas | gibi: which is normal | 10:13 |
| ioni | to start with the right options in libvirt | 10:13 |
| bauzas | gibi: now, I think I see the problem actually | 10:13 |
| bauzas | gibi: we should directly return inventories with VGPU on the child | 10:13 |
| bauzas | gibi: but I made it conditional | 10:14 |
| bauzas | gibi: anyway, continuing to dig | 10:14 |
| bauzas | I feel I made a mistake and I'm close to find it | 10:14 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/rocky: Avoid redundant initialize_connection on source post live migration https://review.openstack.org/636895 | 10:14 |
| openstackgerrit | sahid proposed openstack/nova master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610694 | 10:17 |
| *** moshele has joined #openstack-nova | 10:17 | |
| stephenfin | ioni: Right. You might even need to rebuild, actually. I'm not sure | 10:18 |
| stephenfin | ioni: Just boot an new instance | 10:18 |
| stephenfin | *a | 10:18 |
| *** whoami-rajat has quit IRC | 10:19 | |
| openstackgerrit | sahid proposed openstack/os-traits master: Change python3.5 job to python3.7 job on Stein+ https://review.openstack.org/610065 | 10:21 |
| ioni | stephenfin, thanks. it works | 10:23 |
| stephenfin | ioni: \o/ I would suggest filing a bug against openstack-ansible to get them to override that default out of the box | 10:24 |
| ioni | stephenfin, with default, i ment that nova starts with that default | 10:24 |
| ioni | default nothing is confgured in nova.conf related to keymap | 10:24 |
| stephenfin | ioni: We can't do it upstream because that would require everyone use noVNC 1.0. OSA can because they're deciding what version of that package gets installed | 10:25 |
| stephenfin | ioni: Correct. I'm suggesting OSA override the nova default | 10:25 |
| ioni | stephenfin, alright | 10:25 |
| lyarwood | mdbooth: ack thanks, would you mind throwing a +1 on there to indicate that please? | 10:28 |
| *** markvoelker has joined #openstack-nova | 10:28 | |
| *** ileixe has quit IRC | 10:29 | |
| *** sapd1 has quit IRC | 10:29 | |
| *** helenafm has quit IRC | 10:38 | |
| *** abhishekk has quit IRC | 10:40 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova stable/rocky: tox: Don't write byte code (maybe) https://review.openstack.org/636917 | 10:40 |
| *** tetsuro has joined #openstack-nova | 10:41 | |
| *** tetsuro has quit IRC | 10:42 | |
| *** betherly has joined #openstack-nova | 10:44 | |
| *** betherly has quit IRC | 10:44 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova stable/queens: tox: Don't write byte code (maybe) https://review.openstack.org/636918 | 10:46 |
| openstackgerrit | Stephen Finucane proposed openstack/nova stable/pike: tox: Don't write byte code (maybe) https://review.openstack.org/636919 | 10:47 |
| openstackgerrit | Stephen Finucane proposed openstack/nova stable/ocata: tox: Don't write byte code (maybe) https://review.openstack.org/636921 | 10:51 |
| stephenfin | lyarwood: There's a couple of super easy, totally trivial backports there for you, if you fancy getting some stat padding in ^^^ :D | 10:53 |
| stephenfin | sean-k-mooney: Thoughts on mriedem's comments here? https://review.openstack.org/#/c/635170/2/nova/virt/libvirt/vif.py | 10:56 |
| *** udesale has quit IRC | 10:58 | |
| *** mdbooth_ has joined #openstack-nova | 11:01 | |
| *** markvoelker has quit IRC | 11:01 | |
| *** mdbooth has quit IRC | 11:04 | |
| *** helenafm has joined #openstack-nova | 11:06 | |
| lyarwood | stephenfin: ack | 11:10 |
| *** sapd1 has joined #openstack-nova | 11:24 | |
| *** tiendc has quit IRC | 11:29 | |
| *** jangutter_ has joined #openstack-nova | 11:35 | |
| *** jangutter has quit IRC | 11:38 | |
| *** tbachman has quit IRC | 11:40 | |
| *** maciejjozefczyk has joined #openstack-nova | 11:41 | |
| *** asettle has joined #openstack-nova | 11:41 | |
| *** priteau has joined #openstack-nova | 11:43 | |
| maciejjozefczyk | sean-k-mooney: Hey, if you're interested I can provide you cherry-picks to Newton of patches about cache, it works well on our production | 11:44 |
| *** vdrok_ has quit IRC | 11:44 | |
| *** erlon_ has joined #openstack-nova | 11:45 | |
| *** vdrok has joined #openstack-nova | 11:48 | |
| *** janki has quit IRC | 11:48 | |
| *** janki has joined #openstack-nova | 11:48 | |
| *** tetsuro has joined #openstack-nova | 11:51 | |
| openstackgerrit | Stephen Finucane proposed openstack/os-vif master: Add 'SUPPORT_BW_CONFIG' option to VIFs https://review.openstack.org/636933 | 11:53 |
| *** janki has quit IRC | 11:55 | |
| *** janki has joined #openstack-nova | 11:55 | |
| *** rpittau has quit IRC | 11:58 | |
| *** markvoelker has joined #openstack-nova | 11:58 | |
| *** sapd1 has quit IRC | 12:03 | |
| *** ratailor has quit IRC | 12:06 | |
| *** jangutter_ has quit IRC | 12:10 | |
| *** jangutter has joined #openstack-nova | 12:11 | |
| *** maciejjozefczyk has quit IRC | 12:14 | |
| *** tetsuro has quit IRC | 12:16 | |
| gibi | bauzas: If I understand correctly the functional test creates VGPU inventory on the root, and a VM even boots with a VGPU before the reshape | 12:17 |
| gibi | bauzas: I saw in the debug log of the test run that the VM has an mdev in the xml | 12:18 |
| gibi | <hostdev mode="subsystem" type="mdev" managed="no" model="vfio-pci"> | 12:18 |
| gibi | <source> | 12:18 |
| gibi | <address uuid="4b20d080-1b54-4048-85b3-a6a62d165c01"/> | 12:18 |
| gibi | </source> | 12:18 |
| gibi | </hostdev> | 12:18 |
| gibi | bauzas: I guess that the problem can be that the VGPU inventory is not added by the compute itself to the root RP but added by the functional test manually. | 12:19 |
| *** tetsuro has joined #openstack-nova | 12:19 | |
| *** tetsuro has quit IRC | 12:21 | |
| gibi | bauzas: somehow if not the compute adds the VGPU to the root RP then the compute does not see it later when checking if reshape is needed. | 12:23 |
| *** sapd1 has joined #openstack-nova | 12:23 | |
| *** tetsuro has joined #openstack-nova | 12:24 | |
| gibi | bauzas: it seems like the provider_tree cache is not updated after the VGPU is added to the root RP by the functional test, even if the functional test simulated the compute restart | 12:25 |
| gibi | bauzas: I will dig into this. It can be an artifact of the way the functional test a) adds the VGPU inventory to the root RP or b) the way the tests simulates the compute service restart | 12:25 |
| *** markvoelker has quit IRC | 12:26 | |
| *** _fragatina has joined #openstack-nova | 12:30 | |
| alex_xu | stephenfin: thanks for take care https://blueprints.launchpad.net/nova/+spec/remove-force-flag-from-live-migrate-and-evacuate | 12:33 |
| *** wolverineav has joined #openstack-nova | 12:33 | |
| *** priteau has quit IRC | 12:36 | |
| *** priteau has joined #openstack-nova | 12:38 | |
| *** wolverineav has quit IRC | 12:38 | |
| *** janki has quit IRC | 12:41 | |
| *** udesale has joined #openstack-nova | 12:45 | |
| openstackgerrit | Andrey Volkov proposed openstack/nova master: AZ list performance optimiztion: avoid double service list DB fetch https://review.openstack.org/636947 | 12:48 |
| gibi | bauzas: yepp, it seems that when the func test simulates the compute restart the provider tree kept in the the report client is not dropped and therefore during the restart it is not refreshed | 12:50 |
| gibi | bauzas: does the reportclient went to become a singleton? | 12:50 |
| *** owalsh_ is now known as owalsh_afk | 12:51 | |
| *** rpittau has joined #openstack-nova | 12:56 | |
| *** tetsuro has quit IRC | 12:56 | |
| gibi | bauzas: nvm | 12:58 |
| gibi | bauzas: it doesn't | 12:58 |
| openstackgerrit | Andrey Volkov proposed openstack/nova master: AZ list performance optimization: avoid double service list DB fetch https://review.openstack.org/636947 | 13:02 |
| gibi | bauzas: I managed to find the root case | 13:07 |
| gibi | bauzas: it is in the test env | 13:07 |
| gibi | bauzas: the service.stop service.start in the functional test does not properly simulate a real service restart, a lot of state are kept, like the resource tracker and the reportclient provider_tree cache. | 13:08 |
| *** sapd1 has quit IRC | 13:08 | |
| gibi | bauzas: I manually dropped the provider_tree cache between stop and start and now the test passes | 13:08 |
| gibi | bauzas: pushing the fix to the test soon | 13:10 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add functional test for libvirt vgpu reshape https://review.openstack.org/631559 | 13:11 |
| gibi | bauzas: ^^ | 13:11 |
| *** priteau has quit IRC | 13:13 | |
| bauzas | gibi: yeah, I was about telling you that I didn't had a problem with the reshape when testing it in some hardware | 13:18 |
| *** Dinesh_Bhor has quit IRC | 13:18 | |
| bauzas | gibi: see http://paste.openstack.org/show/745093/ | 13:19 |
| openstackgerrit | Silvan Kaiser proposed openstack/nova master: Added mount fstype based validation of Quobyte mounts https://review.openstack.org/522245 | 13:21 |
| bauzas | gibi: I just removed the -2 | 13:21 |
| *** tbachman has joined #openstack-nova | 13:23 | |
| *** mvkr has quit IRC | 13:26 | |
| *** agopi|out is now known as agopi|brb | 13:28 | |
| *** moshele has quit IRC | 13:28 | |
| gibi | bauzas: the whole test problem boils down to the fact that it is not possible to create an old compute RP inventory directly from the compute code as that code was replaced by the new vgpu code. So the test should simulate that there is an old RP inventory with VGPU on the root | 13:29 |
| *** ade_lee_ has quit IRC | 13:31 | |
| *** moshele has joined #openstack-nova | 13:32 | |
| *** agopi|brb has quit IRC | 13:32 | |
| bauzas | gibi: if you wanna test the real reshape, I agree | 13:36 |
| *** mdbooth_ is now known as mdbooth | 13:36 | |
| bauzas | gibi: what does the test atm is making sure that on greenfields, you start with VGPU resources to be nested | 13:37 |
| kaisers | stephenfin: Hi! FYI, i just ran an update to https://review.openstack.org/#/c/522245 based on yesterdays feedback :) | 13:37 |
| mdbooth | Anybody got experience configuring devstack to use an https endpoint with a self-signed cert? | 13:37 |
| *** dave-mccowan has joined #openstack-nova | 13:37 | |
| * mdbooth is having trouble getting requests to validate the self-signed cert. | 13:37 | |
| mdbooth | To the best of my knowledge the cert is correctly installed in the system-wide ca-bundle, and certainly curl will now connect without --insecure. | 13:38 |
| mdbooth | But openstackcli will not. | 13:38 |
| mdbooth | And neither will Nova. | 13:38 |
| *** mriedem has joined #openstack-nova | 13:39 | |
| mdbooth | '$ openstack --insecure volume list' *does* work, but afaik there's no similar option to disable nova's certificate validation when using cinderclient. | 13:40 |
| * mdbooth is currently hacking on the assumption that if I can fix the requirement for openstackcli to take the --insecure argument, nova will Just Work(tm) too. | 13:41 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Ignore VolumeAttachmentNotFound exception in compute.manager https://review.openstack.org/636954 | 13:41 |
| mdbooth | But... is there some alternate ca-bundle somewhere? | 13:41 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Provide a useful error message when trying to update non-compute services https://review.openstack.org/636955 | 13:42 |
| bauzas | gibi: I'm not super happy with https://review.openstack.org/#/c/631559/5..6/nova/tests/functional/libvirt/test_reshape.py | 13:45 |
| *** sridharg has quit IRC | 13:47 | |
| *** jaosorior has quit IRC | 13:47 | |
| stephenfin | kaisers: Still got comments, unfortunately https://review.openstack.org/#/c/522245 | 13:48 |
| *** s10 has joined #openstack-nova | 13:48 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Change sqlalchemy warnings filter to an error https://review.openstack.org/636957 | 13:49 |
| *** lbragstad has joined #openstack-nova | 13:50 | |
| *** jaosorior has joined #openstack-nova | 13:51 | |
| *** jmlowe has quit IRC | 13:54 | |
| openstackgerrit | Mark Goddard proposed openstack/nova stable/queens: Don't emit warning when ironic properties are zero https://review.openstack.org/608611 | 13:55 |
| *** priteau has joined #openstack-nova | 13:56 | |
| *** liuyulong has joined #openstack-nova | 13:59 | |
| *** agopi|brb has joined #openstack-nova | 14:00 | |
| *** owalsh_afk is now known as owalsh | 14:01 | |
| *** cdent has quit IRC | 14:04 | |
| mdbooth | Hmm, this is a difference in behaviour between urllib.requests (urllib2) and requests, which appears to use urllib3 | 14:04 |
| *** artom has quit IRC | 14:04 | |
| *** agopi_ has joined #openstack-nova | 14:04 | |
| mdbooth | The former appears to use the system ca bundle, the latter does not | 14:05 |
| gibi | efried: I answered your questions / comments in https://review.openstack.org/#/c/616239/24 | 14:06 |
| *** agopi|brb has quit IRC | 14:06 | |
| mdbooth | certifi, j'accuse | 14:07 |
| gibi | bauzas: how can I make you happier about that reshape functional test? | 14:07 |
| *** jdillaman has quit IRC | 14:08 | |
| openstackgerrit | Vlad Gusev proposed openstack/nova stable/queens: Handle unicode characters in migration params https://review.openstack.org/636678 | 14:08 |
| bauzas | gibi: self.compute is generated by nova.test.py which returns a ServiceFixture | 14:10 |
| bauzas | gibi: if we really want to mock a real compute stop, we need to update the fixture | 14:11 |
| bauzas | gibi: https://github.com/openstack/nova/blob/master/nova/test.py#L399 | 14:12 |
| bauzas | gibi: somehow, you say in your comment that https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L83 doesn't work correctly | 14:13 |
| bauzas | because we still have a cache | 14:13 |
| bauzas | so we need to invalidate the cache there ^ | 14:13 |
| gibi | bauzas: yeah, it would be nice to improve the fixture to support a real service restart | 14:14 |
| gibi | bauzas: let me dig a bit about that | 14:14 |
| gibi | bauzas: I remember I had issues with the restart in the past | 14:14 |
| bauzas | for placement I guess ? | 14:14 |
| bauzas | because I don't see any problems with the existing fixture | 14:15 |
| *** cdent has joined #openstack-nova | 14:15 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: Add descriptions for vol-backed snapshots https://review.openstack.org/615084 | 14:15 |
| bauzas | gibi: AFAICS, we kill the service | 14:15 |
| bauzas | so I guess the problem is more about the python module | 14:16 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Address nits from I9e30a24a4c0640f282f507d0a96640d3cdefe43c https://review.openstack.org/636967 | 14:16 |
| bauzas | gibi: actually, I think I voiced about this when I was reviewing the scheduler reportclient cut | 14:17 |
| bauzas | we shouldn't cache it | 14:17 |
| *** agopi_ is now known as agopi | 14:18 | |
| gibi | bauzas: the fixture.stop() does not kill the rela service all the module level things are in place. fixture.kill and a starting a new compute service through the fixture would be better | 14:18 |
| *** ade_lee_ has joined #openstack-nova | 14:21 | |
| *** psachin has quit IRC | 14:22 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Drop the integrated-gate (py27) template https://review.openstack.org/634949 | 14:23 |
| *** matoef1 has joined #openstack-nova | 14:24 | |
| *** jmlowe has joined #openstack-nova | 14:25 | |
| matoef1 | Hi guys. After enable SSL on DevStack I got the following message from nova.. ..devstack@n-api.service[30584]: DEBUG nova.api.openstack.wsgi [None req-00203afa-89e4-4c28-bed8-cbbc985ee67c demo demo] Returning 400 to user: Invalid input for field/attribute user_data. Value: Q29udGVud... | 14:26 |
| matoef1 | But my user_data value does not have more then 64k. | 14:26 |
| matoef1 | But the error message is : Invalid input for field/attribute user_data .. is too long | 14:28 |
| matoef1 | Is there any way how to investigate this error ? | 14:28 |
| matoef1 | Many thanks | 14:28 |
| *** mvkr has joined #openstack-nova | 14:31 | |
| mdbooth | lyarwood: So... after *finally* getting a python3 devstack up and running with nova talking to cinder over ssl on F29... I don't get the recursion error. | 14:34 |
| openstackgerrit | Vlad Gusev proposed openstack/nova stable/queens: Handle unicode characters in migration params https://review.openstack.org/636678 | 14:35 |
| mdbooth | lyarwood: I downgraded eventlet to 0.21.0 | 14:35 |
| mdbooth | Nada. | 14:35 |
| * mdbooth -> coffee | 14:36 | |
| *** maciejjozefczyk has joined #openstack-nova | 14:44 | |
| gibi | bauzas: simply killing the compute service and starting a new compute service leads to a HostMapping issue which reminds me of https://review.openstack.org/#/c/512552/ | 14:47 |
| *** moshele has quit IRC | 14:49 | |
| openstackgerrit | Jack Ding proposed openstack/nova master: Flavor extra spec and image properties validation https://review.openstack.org/620706 | 14:51 |
| *** matoef1 has quit IRC | 14:51 | |
| gibi | bauzas: I will refresh https://review.openstack.org/#/c/512552/ and try to combine that with the reshape test to see if that helps | 14:53 |
| *** mlavalle has joined #openstack-nova | 14:55 | |
| *** Luzi has quit IRC | 14:55 | |
| gibi | bauzas: I don't actually have to complicate that much | 14:57 |
| gibi | bauzas: there is already a restart_compute_service() call that does the trick we need | 14:57 |
| *** mchlumsky has joined #openstack-nova | 14:57 | |
| *** awalende has quit IRC | 14:58 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add functional test for libvirt vgpu reshape https://review.openstack.org/631559 | 14:58 |
| gibi | bauzas: what about this ^^ | 14:59 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove deprecated 'os-flavor-manage' policy https://review.openstack.org/633656 | 15:00 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix a missing policy in test policy data https://review.openstack.org/633686 | 15:01 |
| *** artom has joined #openstack-nova | 15:01 | |
| *** mrch_ has quit IRC | 15:04 | |
| openstackgerrit | Elod Illes proposed openstack/nova master: DNM: test pip 19 - pip freeze with git editable https://review.openstack.org/636980 | 15:06 |
| bauzas | gibi: looking | 15:09 |
| bauzas | gibi: sorry was afk | 15:09 |
| *** artom has quit IRC | 15:09 | |
| bauzas | gibi: FWIW, I checked and we still need to have https://review.openstack.org/#/c/636591/1 | 15:10 |
| bauzas | gibi: that said, looks like it's not a problem | 15:10 |
| bauzas | gibi: I'll use a specific type on my node, and see whether we have problems | 15:10 |
| gibi | bauzas: OK | 15:11 |
| bauzas | gibi: http://paste.openstack.org/show/745109/ | 15:11 |
| bauzas | gibi: the fdfd8f9b-e645-4a6f-aa60-477b00ba32b7 instance is having the mdev_27840b52_31e3_4fb7_a391_ec529e73be93 | 15:12 |
| bauzas | gibi: placement said it should be in virtlab606.virt.lab.eng.bos.redhat.com_pci_0000_84_00_0 which is RP #3 | 15:12 |
| bauzas | gibi: but when looking at libvirt, the mdev parent is pci_0000_87_00_0 | 15:13 |
| bauzas | ie. not the same pGPU | 15:13 |
| bauzas | given all pGPUs have the same type, I don't think it's a problem, but I'll test it | 15:13 |
| openstackgerrit | Merged openstack/nova-specs master: Add more detail and fix code block formatting https://review.openstack.org/628702 | 15:14 |
| gibi | bauzas: so you have more than one pGPU and nova allocates VGPU from one RP in placement then use mdev from another PGPU on the host | 15:14 |
| bauzas | correct | 15:14 |
| *** wolverineav has joined #openstack-nova | 15:15 | |
| gibi | do you have more than one pgpu RP in placement? or you only report all the mdevs from the host on a single RP in placement? | 15:15 |
| bauzas | gibi: we have a RP per pPGU, yes | 15:16 |
| bauzas | gibi: see http://paste.openstack.org/show/745093/ | 15:17 |
| bauzas | we have 4 inventories for VGPU RC | 15:17 |
| bauzas | while previously we only had 1 | 15:17 |
| gibi | bauzas: then there will be problem when you remove a pgpu from the host that has no mdev allocated from, but in placement the allocation is against that RP | 15:17 |
| bauzas | you can't remove a PGPU | 15:18 |
| bauzas | it's a PCI device | 15:18 |
| gibi | stop the host, remove the pci device, start the host | 15:18 |
| bauzas | ah that, then yes | 15:18 |
| bauzas | but honestly, how many people would do this ? | 15:18 |
| bauzas | I can add a documentation of course | 15:18 |
| gibi | bauzas: yeah | 15:18 |
| bauzas | actually, I need to do it | 15:18 |
| bauzas | and remove it by https://review.openstack.org/#/c/636591/ | 15:19 |
| gibi | bauzas: it could be problem in future when we start supporting different device types per pgpu, but our RP inventory does not match the physical usage of the pgpu | 15:19 |
| *** wolverineav has quit IRC | 15:19 | |
| gibi | bauzas: I feel this inconsistency is dangerous | 15:19 |
| bauzas | but lemme use one type that gives 1 VGPU per pGPU | 15:19 |
| bauzas | gibi: right, I said it, we don't have a problem for the moment | 15:20 |
| bauzas | gibi: until we have multiple types | 15:20 |
| bauzas | where two pGPUs could have different types | 15:20 |
| bauzas | I documented it tho | 15:20 |
| gibi | bauzas: but if we release this code, then there will be deployments where the allocation in placement will not reflect that usage on the host, and fixing those deployments will be painful | 15:20 |
| *** s10 has quit IRC | 15:20 | |
| gibi | bauzas: the fix will be like moving allocations between RPs based on parent child relationship | 15:21 |
| gibi | of the mdev and the pgpu | 15:21 |
| gibi | I agree that it is not a problem today, but it will cause pain in the future if we are not handling it now | 15:22 |
| bauzas | gibi: actually that's a good point, we should need to reshape allocations :( | 15:22 |
| *** udesale has quit IRC | 15:22 | |
| gibi | bauzas: it will be even more complicated if a server requests two VGPUs in a numbered groups, and therefore it needs to be filled from one pgpu which is represented as on RP in placement. But the RP in placement won't have an exact inventory for a single pgpu | 15:25 |
| bauzas | gibi: we don't accept requests for > 1 vGPU atm | 15:25 |
| bauzas | for many reasons, one being at least that nvidia driver doesn't support more than one mdev attached to a guest per pGPU | 15:26 |
| gibi | bauzas: I see | 15:26 |
| gibi | bauzas: that hepls | 15:26 |
| gibi | helps | 15:26 |
| bauzas | ideally, the patch is ready | 15:26 |
| gibi | bauzas: still I think we should fix the mapping now to avoid the pain of reshape in the future | 15:27 |
| bauzas | but the problem is about the RP UUID > name transtive relationship | 15:27 |
| bauzas | yeah | 15:27 |
| *** awalende has joined #openstack-nova | 15:27 | |
| bauzas | so, I need to find a quick and clean solution for this | 15:27 |
| bauzas | I should probably ask for a hangout session if people agree | 15:27 |
| bauzas | to get feedback | 15:27 |
| gibi | bauzas: yeah that would make sense. Especially if we already have different solution proposals to choose from | 15:28 |
| bauzas | gibi: the alternative would be asking operators to rebuild :p | 15:28 |
| gibi | bauzas: hm, that could be option N+1 ;) | 15:28 |
| bauzas | gibi: actually, lemme look at the driver code and see what I can do | 15:29 |
| bauzas | we already have the providertree info | 15:29 |
| bauzas | now the big concern I have is caching it | 15:29 |
| bauzas | so that the method could use it | 15:29 |
| bauzas | instead of passing straight the tree down to that method | 15:29 |
| bauzas | which I dislike | 15:30 |
| gibi | at least we have options | 15:30 |
| gibi | In general I don't like caching as that is hard to make it right | 15:30 |
| *** itlinux has joined #openstack-nova | 15:30 | |
| *** awalende has quit IRC | 15:31 | |
| *** hongbin has joined #openstack-nova | 15:38 | |
| mriedem | stephenfin: replies in https://review.openstack.org/#/c/633620/ | 15:41 |
| bauzas | gibi: yup me too | 15:42 |
| * stephenfin clicks | 15:42 | |
| stephenfin | mriedem: right back at ya (tl;dr: you do need to drop that line) | 15:47 |
| *** artom has joined #openstack-nova | 15:47 | |
| mriedem | ok updating | 15:49 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix deps for api-samples tox env https://review.openstack.org/633620 | 15:52 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Make VolumeAttachmentsSampleV249 test other methods https://review.openstack.org/633621 | 15:52 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add microversion to expose virtual device tags https://review.openstack.org/631948 | 15:52 |
| *** maciejjozefczyk has quit IRC | 15:53 | |
| *** itlinux has quit IRC | 15:55 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove duplicate cleanup in functional tests https://review.openstack.org/636996 | 15:56 |
| *** itlinux has joined #openstack-nova | 15:56 | |
| *** itlinux has quit IRC | 16:07 | |
| *** kaisers has quit IRC | 16:07 | |
| *** hongbin has quit IRC | 16:09 | |
| *** tbachman has quit IRC | 16:09 | |
| bauzas | efried: thanks for commenting https://review.openstack.org/#/c/636591/1/nova/virt/libvirt/driver.py | 16:11 |
| efried | fo sho, look forward to seeing that patch. | 16:11 |
| stephenfin | mriedem: Said it to lyarwood but I've backported that PYTHONDONTWRITEBYTECODE setting all the way back to queens. If you could take a look, I'd appreciate it https://review.openstack.org/#/q/topic:PYTHONDONTWRITEBYTECODE+status:open | 16:11 |
| bauzas | efried: given we update the provider tree at compute startup, I just wonder if it's safe enough to assume that the libvirt in-memory cached attribute for the tree would be there anyway when an instance is spawned ? | 16:12 |
| bauzas | mriedem: thoughts on that too ? | 16:12 |
| bauzas | mriedem: context is https://review.openstack.org/#/c/636591/1/nova/virt/libvirt/driver.py@6347 | 16:12 |
| efried | bauzas: We're pretty much *relying* on there being a match between upt and spawn | 16:13 |
| bauzas | efried: there could be a race tho | 16:14 |
| efried | bauzas: Also, pretty sure we run upt right before spawn, not just in the periodic, I could be wrong. So it should be pretty durn recent by then. | 16:14 |
| bauzas | efried: where spawn() could run *before* upt | 16:14 |
| bauzas | yeah | 16:14 |
| efried | bauzas: But like I've said already (uh, possibly about something else): if there's a race, it's going to f up more than just this. | 16:14 |
| mriedem | we run update_available_resource on compute start | 16:15 |
| mriedem | which calls upt | 16:15 |
| bauzas | i know | 16:15 |
| mriedem | that's before the service is ready for spawn | 16:15 |
| mriedem | so what's the issue | 16:15 |
| bauzas | okay | 16:15 |
| *** yan0s has quit IRC | 16:15 | |
| bauzas | I thought we were allowing instances to spawn, while compute was starting | 16:15 |
| mriedem | f i hope not | 16:15 |
| *** kaisers has joined #openstack-nova | 16:15 | |
| mriedem | https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L1259 | 16:15 |
| mriedem | https://github.com/openstack/nova/blob/master/nova/service.py#L180 | 16:15 |
| bauzas | oh good catch this is pre hook | 16:16 |
| bauzas | perfect | 16:16 |
| *** dtantsur is now known as dtantsur|afk | 16:16 | |
| bauzas | mriedem: you're not worried by libvirt keeping a copy of the provider tree structure ? | 16:16 |
| mriedem | i haven't been following along | 16:16 |
| bauzas | mriedem: nah, it's just because we need to find the name of a RP by its uuid | 16:17 |
| mriedem | sounds like more cache-tastic fun | 16:17 |
| bauzas | mriedem: as we only get RP UUID from the allocations list we pass over spawn | 16:17 |
| mriedem | which RP? | 16:17 |
| efried | mriedem: TL;DR: the virt driver needs to be responsible for maintaining a mapping between RP and "real thing". The provider tree already has that information, so don't bother building something special, just save it off during upt. | 16:17 |
| bauzas | mriedem: the RP which was allocated a VGPU | 16:17 |
| mriedem | so the pgpu? | 16:17 |
| mriedem | the child provider | 16:17 |
| bauzas | yup | 16:18 |
| bauzas | so that libvirt can use or create a mdev from *this* pgpu | 16:18 |
| bauzas | http://paste.openstack.org/show/745109/ | 16:18 |
| mriedem | what's the alternative? the driver holding it's own mapping of rp_uuid -> rp_name? which is something ProviderTree already does | 16:18 |
| efried | yes ^ | 16:18 |
| bauzas | above is the fact that when we create a new instance, we use a random mdev which is unrelated to the pgpu which was allocated by placement | 16:18 |
| efried | (to both) | 16:18 |
| mriedem | i don't have a strong feeling either way | 16:19 |
| *** itlinux has joined #openstack-nova | 16:19 | |
| mriedem | ProviderTree is convenient albeit heavy weight and hopefully someone doesn't abuse it somehow, but doing our own mapping dict is redundant | 16:19 |
| mriedem | so shrug | 16:19 |
| bauzas | yeah me too, but I'm opinionated enough to make it the smoothiest | 16:20 |
| *** moshele has joined #openstack-nova | 16:20 | |
| bauzas | and caching the value in the libvirt driver object seems the smoothiest approach | 16:20 |
| mriedem | so the provider tree would just get reset on every upt call? | 16:20 |
| mriedem | if so, then sure | 16:20 |
| bauzas | yeah | 16:20 |
| mriedem | wfm | 16:20 |
| bauzas | i'm done with complex cache invalidation mechanisms | 16:21 |
| mriedem | are you SO done with them? | 16:21 |
| *** liuyulong has quit IRC | 16:23 | |
| *** hongbin has joined #openstack-nova | 16:24 | |
| bauzas | mriedem: /me tries to not think hard of AZs | 16:25 |
| bauzas | or server groups | 16:25 |
| mriedem | hey if you like azs i have some good news for you | 16:25 |
| mriedem | https://review.openstack.org/#/c/509206/ | 16:25 |
| mriedem | finally fixing that shitty old bug | 16:25 |
| bauzas | \o/ | 16:26 |
| bauzas | mriedem: straight adding it into my queue | 16:26 |
| bauzas | gibi: you'll be happy to hear that functests work now, btw. :) | 16:27 |
| *** macza has joined #openstack-nova | 16:27 | |
| gibi | bauzas: awesome | 16:28 |
| *** wwriverrat has joined #openstack-nova | 16:29 | |
| gibi | every time I start kicking https://review.openstack.org/#/c/635859/ I got depressed after 15 minutes about I capabilities to understand what happening | 16:29 |
| melwitt | o/ | 16:29 |
| melwitt | gibi: me too. I tried for awhile and gave up | 16:30 |
| gibi | melwitt: but I feel something deep is behind that so I cannot gave up :) | 16:31 |
| *** jackding has quit IRC | 16:31 | |
| melwitt | gibi: that's good, I think you will get to the bottom of it :) | 16:32 |
| mriedem | well if you want to get really depressed http://logs.openstack.org/47/635147/11/check/nova-tox-functional-py35/5c3c2a9/job-output.txt.gz#_2019-02-13_13_33_41_103932 | 16:32 |
| mriedem | that traceback wins the grand prize | 16:32 |
| mriedem | and i don't know what to do about it | 16:32 |
| mriedem | looks like the db dropped in the middle of the test | 16:32 |
| gibi | mriedem: nice :) | 16:33 |
| mriedem | the dumping of os-traits contents to the logs on each functional test run with the PlacementFixture isn't helping either | 16:33 |
| mriedem | http://logs.openstack.org/47/635147/11/check/nova-tox-functional-py35/5c3c2a9/job-output.txt.gz#_2019-02-13_13_33_41_193979 | 16:34 |
| gibi | mriedem: I think we can move that to DEBUG | 16:34 |
| mriedem | yeah probably http://logs.openstack.org/59/636459/2/check/tempest-full/3226eb1/controller/logs/screen-placement-api.txt.gz#_Feb_13_13_54_30_464250 | 16:35 |
| gibi | I'm affraid that the drop of the databease could be another interfering test case. Did you managed to reproduce that locally? | 16:35 |
| mriedem | nope | 16:35 |
| bauzas | gibi: efried: what's the best way to copy a providertree object ? copy.deepcopy() ? | 16:35 |
| openstackgerrit | Elod Illes proposed openstack/nova master: DNM: test pip 19 - pip freeze with git editable https://review.openstack.org/636980 | 16:35 |
| bauzas | or do we have other built-in mechanism ? | 16:35 |
| bauzas | tbc, I want to have a separate reference to be stored on the libvirt object | 16:36 |
| *** itlinux has quit IRC | 16:36 | |
| gibi | bauzas: I would be affraid of copying the self.lock = lockutils.internal_lock(_LOCK_NAME) inside the ProviderTree | 16:36 |
| efried | bauzas: upt is already operating on a copy... but I'm actually not positive that ufpt doesn't dork with it. So yeah, copy.deepcopy is probably the best bet. | 16:37 |
| gibi | bauzas: besides the self.lock the rest of the internals seems copiable | 16:37 |
| bauzas | gibi: so, ideally we should provide a copy method | 16:38 |
| efried | gibi: hm, interesting point about the lock. This impl that bauzas is doing should be read-only. But that still wouldn't stop you from bouncing off the lock if the other instance is using it. | 16:38 |
| bauzas | efried: what I want is purely read-only | 16:38 |
| *** itlinux has joined #openstack-nova | 16:38 | |
| *** markvoelker has joined #openstack-nova | 16:38 | |
| efried | However, I would be surprised if the lockutils lock doesn't override __copy__ to *not* copy itself. | 16:38 |
| bauzas | actually, you know what ? I'm gonna make it explicit | 16:38 |
| bauzas | by using a propertgy | 16:38 |
| bauzas | we're not intended to manipulate this copy | 16:39 |
| gibi | mriedem: I think I will stick to the remove-the-assert-that-blow-the-world issue, that I can reproduce locally at least | 16:39 |
| *** moshele has quit IRC | 16:39 | |
| efried | bauzas: There are methods to get a read-only copy of the data out of ProviderTree, but I'm pretty sure they only get one provider at a time. | 16:39 |
| efried | Yeah, ProviderTree.data() | 16:40 |
| mriedem | gibi: efried: fwiw i'm also holding off on the bw provider series until https://review.openstack.org/#/c/616239/ makes progress | 16:40 |
| mriedem | and we've got 3 weeks from FF today | 16:40 |
| efried | I guess that's my action | 16:40 |
| * efried reads gibi's responses... | 16:41 | |
| mriedem | i'm just waiting for efried to +2 so i can nit pick a few things and make it look like i get it | 16:41 |
| gibi | efried: thanks | 16:41 |
| bauzas | efried: anyway, I'm rushing to propose a new rev | 16:41 |
| bauzas | efried: you will be able to comment on the best approach by reviewing | 16:41 |
| bauzas | not a big deal | 16:41 |
| gibi | efried: to your surprise there is no copy overried in https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/lockutils.py ;) | 16:41 |
| efried | gibi: d'oh! | 16:41 |
| *** ociuhand_ has joined #openstack-nova | 16:42 | |
| efried | uh, wtf. gibi, bauzas, when I try to deepcopy an object with a lockutils.internal_lock in it, I get an error. | 16:44 |
| efried | so how tf would that be working in the rt? /me looks... | 16:44 |
| *** agopi is now known as agopi|FOOD | 16:44 | |
| *** ociuhandu has quit IRC | 16:45 | |
| *** trident has quit IRC | 16:46 | |
| *** ccamacho has quit IRC | 16:48 | |
| *** ociuhand_ has quit IRC | 16:48 | |
| efried | gibi, bauzas: http://paste.openstack.org/show/745111/ <== so how tf is https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L885 working?? | 16:51 |
| bauzas | heh | 16:51 |
| bauzas | sorcery ! | 16:51 |
| *** dpawlik has quit IRC | 16:52 | |
| bauzas | gibi: looking at the func test | 16:54 |
| bauzas | gibi: you lost me | 16:54 |
| bauzas | gibi: in theory, if you request a VGPU but we don't have inventories because the conf option is unset, you should end up with a beautiful NoValidHost | 16:54 |
| bauzas | so I wonder why it works | 16:54 |
| *** pbing19 has joined #openstack-nova | 16:55 | |
| bauzas | oh wait | 16:55 |
| bauzas | I missed the point you're creating the inventory manually | 16:55 |
| bauzas | oki doki | 16:55 |
| *** gyee has joined #openstack-nova | 16:55 | |
| bauzas | so you do this to fake the fact you were running an old version | 16:55 |
| gibi | bauzas: yeah, you are correct | 16:56 |
| bauzas | ok, the good news is that I saw this when I looked at the provider_tree for the allocation RP UUID | 16:56 |
| bauzas | and I was wondering why I was getting a root RP | 16:57 |
| bauzas | gotcha | 16:57 |
| bauzas | perfect, it works then | 16:57 |
| efried | jaypipes: maybe you have some insight? | 17:00 |
| efried | jaypipes: on: http://paste.openstack.org/show/745111/ <== so how tf is https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L885 working?? | 17:00 |
| jaypipes | looking... | 17:01 |
| *** ociuhandu has joined #openstack-nova | 17:02 | |
| *** helenafm has quit IRC | 17:03 | |
| *** _fragatina has quit IRC | 17:03 | |
| gibi | efried: sorry I'm expired for today | 17:04 |
| efried | gibi: I think you've done enough, don't you? :P | 17:04 |
| efried | "Ohh, I'm gibi, I notice things like lock copies, nya nya nya" | 17:05 |
| gibi | :P | 17:05 |
| openstackgerrit | Merged openstack/nova master: Use math.gcd starting with python 3.5 https://review.openstack.org/636669 | 17:05 |
| openstackgerrit | Merged openstack/nova master: doc: specify --os-compute-api-version when setting flavor description https://review.openstack.org/635508 | 17:05 |
| openstackgerrit | Merged openstack/nova master: Remove PLACEMENT_DB_ENABLED from nova-next job config https://review.openstack.org/634953 | 17:05 |
| *** ociuhandu has quit IRC | 17:06 | |
| *** _fragatina has joined #openstack-nova | 17:06 | |
| jaypipes | efried: I have no idea how that works. | 17:10 |
| efried | jaypipes: I figured we must be monkey-patching lockutils somewhere. Couldn't find it at a glance, but haven't finished looking. | 17:11 |
| *** igordc has joined #openstack-nova | 17:11 | |
| jaypipes | no idea. :( | 17:12 |
| *** markvoelker has quit IRC | 17:12 | |
| openstackgerrit | Stephen Finucane proposed openstack/os-vif master: WIP: Add API docs for various VIF types https://review.openstack.org/637009 | 17:13 |
| openstackgerrit | melanie witt proposed openstack/nova master: WIP Remove project_only=True from database queries https://review.openstack.org/637010 | 17:15 |
| cdent | efried: if you simply want to fix it, you can implement __deepcopy__ on the ProviderTree class | 17:15 |
| *** ociuhandu has joined #openstack-nova | 17:15 | |
| cdent | but it's confusing that it hasn't come up before | 17:15 |
| *** wwriverrat has left #openstack-nova | 17:16 | |
| *** wolverineav has joined #openstack-nova | 17:17 | |
| *** wolverineav has quit IRC | 17:17 | |
| *** wolverineav has joined #openstack-nova | 17:17 | |
| openstackgerrit | Merged openstack/nova master: Don't set bandwidth limits for vhostuser, hostdev interfaces https://review.openstack.org/635170 | 17:18 |
| *** rpittau has quit IRC | 17:22 | |
| *** betherly has joined #openstack-nova | 17:23 | |
| *** cfriesen has joined #openstack-nova | 17:23 | |
| *** ociuhandu has quit IRC | 17:24 | |
| *** mdbooth has quit IRC | 17:24 | |
| efried | cdent: Exactly so. | 17:25 |
| efried | Aside: I, for one, am looking forward to hearing all the gweilo trying to pronounce the U release name, whatever it winds up being :P | 17:27 |
| efried | self included, of course | 17:27 |
| *** hongbin has quit IRC | 17:29 | |
| artom | efried, heh, we'll just cop out and call it U | 17:29 |
| *** wolverineav has quit IRC | 17:32 | |
| cfriesen | melwitt: mriedem: question about https://github.com/openstack/nova/blob/master/nova/virt/libvirt/guest.py#L416-L422 I'm seeing this (on pike): http://paste.openstack.org/show/745113/ Which I think means that the original exception is lost (see last line in paste), and so the check for "if 'no target device' in errmsg" will never actually succeed. This results in me hitting the "libvirt.libvirtError" exception instead | 17:32 |
| cfriesen | of the "exception.DeviceNotFound" exception. | 17:32 |
| *** wolverineav has joined #openstack-nova | 17:32 | |
| melwitt | looking | 17:33 |
| *** itlinux has quit IRC | 17:35 | |
| *** wolverineav has quit IRC | 17:36 | |
| *** tbachman has joined #openstack-nova | 17:36 | |
| *** wolverineav has joined #openstack-nova | 17:36 | |
| efried | gibi: I don't want to delay this another day, but I still don't get it. Are you around to hold my hand and explain it? | 17:37 |
| melwitt | cfriesen: the last line looks like it did the right thing, raised DeviceNotFound after catching 'no target device'. but the previous traceback is different, 'internal error' with a 'not found' message, which isn't handled in any way | 17:41 |
| efried | gibi: nm, I think I see it now. And I am suitably shocked that this is the case. | 17:42 |
| cfriesen | melwitt: it looks like the intent in LibvirtDriver.detach_volume() was to swallow the DeviceNotFound exception, but I seem to be hitting the libvirtError exception path which gets re-raised and handled in DriverVolumeBlockDevice.driver_detach | 17:42 |
| artom | Weird question, but does oslo.config support *writing* config files? Sorta like python's configparser, but hopefully without getting rid of comments when writing (which configparser does :( ) | 17:44 |
| melwitt | cfriesen: right. it looks like you're hitting an unexpected combination where you have an libvirt 'internal error' with a 'not found' message, whereas we expect a 'operation failed' libvirt error to go with the 'not found' message | 17:44 |
| cfriesen | got it...I hadn't clued in we were dealing with two separate exceptions | 17:45 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Use the correct mdev allocated from the pGPU https://review.openstack.org/636591 | 17:46 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: libvirt: implement reshaper for vgpu https://review.openstack.org/599208 | 17:46 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add functional test for libvirt vgpu reshape https://review.openstack.org/631559 | 17:46 |
| bauzas | efried: updated based on our discussions ^ | 17:46 |
| bauzas | now the pGPU finding is a dependency for the reshape patch | 17:46 |
| bauzas | because if not, it would mean we would need to trigger another reshape for allocations wrongly set | 17:47 |
| melwitt | cfriesen: I expect the one you're getting is VIR_ERR_INTERNAL_ERROR and we're only handling VIR_ERR_OPERATION_FAILED and VIR_ERR_INVALID_ARG | 17:47 |
| cfriesen | melwitt: thanks, that gives me something to dig into at least. | 17:48 |
| melwitt | cfriesen: np. looks like we might need a 'if errcode in (libvirt.VIR_ERR_OPERATION_FAILED, libvirt.VIR_ERR_INTERNAL_ERROR):' on L409 | 17:50 |
| cfriesen | melwitt: it looks like we expect the detach from the persistent config to sometimes fail...what would cause that? | 17:50 |
| melwitt | cfriesen: it's only expected if the device is not there (already detached). that is, libvirt will raise an error if it's already detached | 17:51 |
| cfriesen | melwitt: so why would it be already detached? a previous iteration of the loop? | 17:52 |
| *** mvkr has quit IRC | 17:52 | |
| melwitt | that's why we specifically look for 'not found' (in the case of persistent) or 'no target device' (in the case of live) | 17:52 |
| melwitt | cfriesen: we've seen things like, someone requests a detach, the detach from the libvirt domain succeeds, but the detach from the compute host fails, so a fail is returned up to the user | 17:53 |
| melwitt | then later, the user tries to detach the volume again, the device was detached from the libvirt domain last time, so we expect 'not found' at that point, but we still need to detach from the compute host | 17:54 |
| cfriesen | got it, thanks | 17:55 |
| melwitt | np | 17:55 |
| *** jmlowe has quit IRC | 17:57 | |
| efried | mriedem: I'm +2 now on https://review.openstack.org/#/c/616239/ | 18:08 |
| efried | bauzas: Yes, I agree. Looking... | 18:08 |
| *** derekh has quit IRC | 18:09 | |
| *** markvoelker has joined #openstack-nova | 18:09 | |
| mriedem | efried: ack cool | 18:09 |
| mriedem | will try to hit that later today or tomorrow morning | 18:10 |
| efried | My confusion is probably because I missed the bulk of the spec discussion and the bottom of the series. | 18:10 |
| efried | Trying to catch up seemed futile, so I'm trying to jump in in the middle, and suffering the consequences. | 18:10 |
| mriedem | i haven't gone through the comments, but looks like you were expecting us to track vcpu/disk/ram in the request group? | 18:11 |
| mriedem | we still pull those values off the request_spec.flavor during scheduling prior to calling GET /a_c | 18:11 |
| efried | mriedem: Yes, I expected that the request spec's list of request groups would contain the request groups. | 18:11 |
| efried | Not a subset of the request groups | 18:11 |
| efried | afaict it actually only contains the bw-related rgs. | 18:12 |
| mriedem | yup | 18:12 |
| mriedem | it's new as of this series | 18:12 |
| efried | so like even if you requested a VGPU in a numbered group, and your VCPU in another numbered group, and... | 18:12 |
| efried | it would still only have the bw groups | 18:12 |
| *** mrch_ has joined #openstack-nova | 18:13 | |
| efried | it's eventually going to need to contain everything | 18:13 |
| *** priteau has quit IRC | 18:13 | |
| mriedem | did he put a comment about that in the code or going to document the limitation in a follow up? | 18:13 |
| efried | or if not "need", it'll soon get to a point where the things that are being excluded won't make any sense. | 18:13 |
| efried | Well, for someone who has been following along, I'm sure it's not a thang. | 18:14 |
| efried | But no, he just responded in the comments. | 18:14 |
| efried | If the rgs member in the request spec was called bandwidth_request_groups, I wouldn't have batted an eye | 18:14 |
| mriedem | i don't disagree that the 10 roads to get to our GET /a_c is a problem | 18:14 |
| *** wolverineav has quit IRC | 18:14 | |
| efried | but then we would have had to change it later | 18:15 |
| efried | and changing things on an ovo is suck | 18:15 |
| mriedem | i think it was VGPUs that made me think at one point about how we now have resources we pull from the explicit flavor attributes (ram/vcpu/disk) and VGPU is not a top-level attribute on the flavor, it's an extra_spec | 18:16 |
| efried | I guess what would help is if RequestSpec.requested_resources had a big NOTE on it saying "currently only houses bandwidth groups" | 18:16 |
| *** wolverineav has joined #openstack-nova | 18:16 | |
| mriedem | efried: right - that's good for a follow up | 18:16 |
| mriedem | fup 3 at this point | 18:16 |
| efried | I'm adding a note. | 18:17 |
| openstackgerrit | Adrian Chiris proposed openstack/nova master: Libvirt: do not set MAC when unplugging macvtap VF https://review.openstack.org/624842 | 18:18 |
| openstackgerrit | Adrian Chiris proposed openstack/nova master: Add free for claimed, allocated devices https://review.openstack.org/616120 | 18:18 |
| openstackgerrit | Adrian Chiris proposed openstack/nova master: Allow per-port modification of vnic_type and profile https://review.openstack.org/607365 | 18:18 |
| openstackgerrit | Adrian Chiris proposed openstack/nova master: Add get_instance_pci_request_from_vif https://review.openstack.org/619929 | 18:18 |
| openstackgerrit | Adrian Chiris proposed openstack/nova master: SR-IOV Live migration indirect port support https://review.openstack.org/620115 | 18:18 |
| openstackgerrit | Adrian Chiris proposed openstack/nova master: libvirt: auto detach/attach sriov ports on migration https://review.openstack.org/629589 | 18:18 |
| *** wolverineav has quit IRC | 18:18 | |
| mriedem | like, flavor at this point doesn't even really need to have vcpu/memory_mb/root_gb at top level fields - root_gb=0 was already problematic since people used it to denote volume-backed server flavors | 18:18 |
| *** wolverineav has joined #openstack-nova | 18:18 | |
| *** agopi|FOOD is now known as agopi | 18:19 | |
| mriedem | which reminds me https://review.openstack.org/#/c/603910/ | 18:19 |
| *** prometheanfire has joined #openstack-nova | 18:20 | |
| prometheanfire | is it possible to request the status of pci devices on hypervisors? | 18:20 |
| prometheanfire | https://wiki.openstack.org/wiki/Pci-api-support#List_and_show_PCI_devices_on_VMs doesn't seem to be used | 18:21 |
| mriedem | prometheanfire: not in the api | 18:21 |
| mriedem | prometheanfire: the review for https://specs.openstack.org/openstack/nova-specs/specs/stein/approved/show-server-numa-topology.html talked about that since it was originally munging that in with NUMA topology for a server | 18:22 |
| mriedem | but that would be allocated pci devices per instance anyway | 18:22 |
| mriedem | if you're looking for inventory, it has to be retrieved outside of the compute API right now | 18:22 |
| mriedem | if/when pci device inventory is tracked in placement you'd use the placement API | 18:22 |
| prometheanfire | ya, placement makes sense | 18:22 |
| mriedem | oh, and because of the [pci]passthrough_whitelist config option per compute the actual pci devices on the host might not be used by nova | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Stub out port binding create/delete in NeutronFixture https://review.openstack.org/636413 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid https://review.openstack.org/614012 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.openstack.org/614035 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Instance.hidden field https://review.openstack.org/631123 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add TargetDBSetupTask https://review.openstack.org/627892 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask https://review.openstack.org/631581 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask https://review.openstack.org/633853 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method https://review.openstack.org/621313 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add prep_snapshot_based_resize_at_dest compute method https://review.openstack.org/633293 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add PrepResizeAtDestTask https://review.openstack.org/627890 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.openstack.org/634832 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.openstack.org/627891 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add finish_snapshot_based_resize_at_dest compute method https://review.openstack.org/635080 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask https://review.openstack.org/635646 | 18:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.openstack.org/635668 | 18:23 |
| *** mlavalle has quit IRC | 18:25 | |
| prometheanfire | right, what I want to know is the status/usage of the passthrough_whitelist and alias for the compute hosts | 18:27 |
| prometheanfire | having that in hv show would be nice | 18:27 |
| mriedem | throw it in the old train ptg etherpad! | 18:28 |
| mriedem | but the answer is likely going to be "it will be available in placement (eventually, in a couple of years maybe)" | 18:28 |
| mriedem | you could write a tool that mines that data from the database.... | 18:29 |
| prometheanfire | meh, not that important | 18:29 |
| mriedem | the pci_devices table is both inventory and allocations | 18:29 |
| efried | soon you will be able to query this from the cyborg API :P | 18:31 |
| efried | using incantations formulated from the charred remains of pci.passthrough_whitelist | 18:32 |
| *** mlavalle has joined #openstack-nova | 18:32 | |
| *** ralonsoh has quit IRC | 18:37 | |
| *** markvoelker has quit IRC | 18:42 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add user_id field to InstanceMapping https://review.openstack.org/633350 | 18:45 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add online data migration for populating user_id https://review.openstack.org/633351 | 18:45 |
| *** macza has quit IRC | 18:46 | |
| *** dims has quit IRC | 18:47 | |
| *** ccamacho has joined #openstack-nova | 18:47 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add user_id column to the instance_mappings table https://review.openstack.org/633349 | 18:52 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add user_id field to InstanceMapping https://review.openstack.org/633350 | 18:52 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add online data migration for populating user_id https://review.openstack.org/633351 | 18:52 |
| *** moshele has joined #openstack-nova | 18:55 | |
| *** macza has joined #openstack-nova | 18:55 | |
| *** awalende has joined #openstack-nova | 18:59 | |
| *** wolverineav has quit IRC | 19:02 | |
| *** awalende has quit IRC | 19:03 | |
| *** wolverineav has joined #openstack-nova | 19:03 | |
| *** wolverineav has quit IRC | 19:06 | |
| *** wolverineav has joined #openstack-nova | 19:06 | |
| openstackgerrit | Merged openstack/nova master: Ensure config regexes match the entire string https://review.openstack.org/636627 | 19:08 |
| openstackgerrit | Merged openstack/nova master: Replace glance command with openstack command https://review.openstack.org/635102 | 19:08 |
| openstackgerrit | Merged openstack/nova master: Switch to using os-resource-classes https://review.openstack.org/628278 | 19:08 |
| *** ccamacho has quit IRC | 19:17 | |
| *** macza_ has joined #openstack-nova | 19:18 | |
| *** macza_ has quit IRC | 19:18 | |
| *** macza has quit IRC | 19:18 | |
| *** macza has joined #openstack-nova | 19:18 | |
| *** wolverineav has quit IRC | 19:22 | |
| *** wolverineav has joined #openstack-nova | 19:23 | |
| openstackgerrit | Merged openstack/nova master: Change live_migration_wait_for_vif_plug=True by default https://review.openstack.org/635360 | 19:24 |
| *** ociuhandu has joined #openstack-nova | 19:24 | |
| efried | cdent, jaypipes, gibi, bauzas: I figured out the locking pickle. eventlet.monkey_patch replaces thread with eventlet.green.thread. The former blows up when you try to copy it; the latter does not. And the latter is doing the right thing by creating a new semaphore. http://paste.openstack.org/show/745119/ | 19:28 |
| openstackgerrit | Surya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells https://review.openstack.org/591657 | 19:28 |
| openstackgerrit | Surya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells Documentation https://review.openstack.org/635147 | 19:28 |
| *** ociuhandu has quit IRC | 19:30 | |
| efried | I also confirmed that, even though they're created with the same key, they are in fact different lock contexts. I.e. I don't deadlock by calling f.acquire() followed by f2.acquire() (but I do deadlock by calling f.acquire() twice). | 19:30 |
| * efried lunches | 19:31 | |
| *** mrch_ has quit IRC | 19:34 | |
| *** moshele has quit IRC | 19:38 | |
| *** markvoelker has joined #openstack-nova | 19:39 | |
| *** wolverineav has quit IRC | 19:41 | |
| *** dave-mccowan has quit IRC | 19:42 | |
| *** tomtom001 has joined #openstack-nova | 19:42 | |
| tomtom001 | Hello, I'm running OpenStack Queens and am trying to work with LUKS encryption. My nova-compute node keeps throwing the following error when trying to decrypt the volume: http://paste.openstack.org/show/745122/ - I've verified the code exists under the following path: /openstack/venvs/nova-17.1.2/lib/python2.7/site-packages/castellan/key_manager/barbican_key_manager.py Is there a patch or can | 19:45 |
| tomtom001 | I verify the api_class in the config? | 19:45 |
| *** igordc has quit IRC | 19:47 | |
| *** igordc has joined #openstack-nova | 19:47 | |
| *** tssurya has quit IRC | 19:48 | |
| *** igordc has quit IRC | 19:48 | |
| *** igordc has joined #openstack-nova | 19:49 | |
| cdent | efried: will you believe me if I said "i thought of that while having dinner"? | 19:55 |
| cdent | nice sluething | 19:55 |
| *** prometheanfire has left #openstack-nova | 19:59 | |
| *** _fragatina has quit IRC | 20:02 | |
| *** moshele has joined #openstack-nova | 20:02 | |
| *** moshele has quit IRC | 20:05 | |
| *** erlon_ has quit IRC | 20:05 | |
| melwitt | tomtom001: where are you configuring the api_class? nova.conf? | 20:05 |
| *** dave-mccowan has joined #openstack-nova | 20:06 | |
| *** jmlowe has joined #openstack-nova | 20:07 | |
| tomtom001 | melwitt: yes nova.conf api_class = castellan.key_manager.barbican_key_manager.BarbicanKeyManager | 20:08 |
| efried | cdent: did you really? | 20:11 |
| efried | I mean, I figured it must be because of a monkey patch somewhere, but I couldn't imagine what/where. | 20:11 |
| jaypipes | efried: interesting. good sleuthing. | 20:11 |
| * jaypipes always wondered what cdent thought about while having dinner... question answered. | 20:12 | |
| *** markvoelker has quit IRC | 20:12 | |
| *** moshele has joined #openstack-nova | 20:14 | |
| *** mchlumsky has quit IRC | 20:15 | |
| melwitt | tomtom001: ok. I think maybe you aren't supposed to set api_class. see this example config from a gate run in the [key_manager] section http://logs.openstack.org/51/633351/1/check/tempest-full/8473acf/controller/logs/etc/nova/nova_conf.txt.gz | 20:16 |
| melwitt | [key_manager]/backend = nova.keymgr.conf_key_mgr.ConfKeyManager | 20:16 |
| *** moshele has quit IRC | 20:17 | |
| *** wolverineav has joined #openstack-nova | 20:23 | |
| *** tssurya has joined #openstack-nova | 20:24 | |
| melwitt | tomtom001: according to the docs, it looks like you could also use backend='barbican', the api_class setting is deprecated https://docs.openstack.org/nova/queens/configuration/config.html?highlight=fixed_key#key_manager.backend | 20:25 |
| tomtom001 | melwitt: thanks I'll check it out | 20:25 |
| *** dave-mccowan has quit IRC | 20:28 | |
| *** wolverineav has quit IRC | 20:30 | |
| tomtom001 | melwitt: looks to me like it doesn't support rbd yet. http://paste.openstack.org/show/745124/ Also, I can't instantiate the other one. Weird. That backend may only work for a fixed key as well. | 20:33 |
| tomtom001 | melwitt: what do you think? | 20:33 |
| *** dave-mccowan has joined #openstack-nova | 20:35 | |
| melwitt | lyarwood is the best person to answer that but he's in EU time zone and off by now, so let me look through the feature patches quickly. I didn't remember rbd being excluded | 20:35 |
| melwitt | the commit message mentions rbd, implying it should work https://review.openstack.org/523958 but I don't know more than that, unfortunately | 20:39 |
| *** mrjk has quit IRC | 20:40 | |
| melwitt | but that error is coming from os-brick. going to see when that was added | 20:40 |
| melwitt | oh, wait, that's because it's falling back on the old, non-native encryption in your case | 20:41 |
| melwitt | only the native luks encryption through qemu will work with rbd | 20:42 |
| tomtom001 | melwitt: I'm struggling to find how to set that up. Do you know? | 20:42 |
| *** wolverineav has joined #openstack-nova | 20:42 | |
| tomtom001 | melwitt: or at least docs on how to? | 20:43 |
| melwitt | yeah, there's not a special doc because it's mostly automatic. the release notes describe what versions of qemu and libvirt are required for it to use native encryption https://docs.openstack.org/releasenotes/nova/queens.html | 20:44 |
| melwitt | there, in the "New Features" section is some info about it. for example, if you have instances using encryption prior to queens, you need to reboot them (maybe hard reboot) to get them to generate new xml needed to do the native encryption. or live migrate them between queens compute hosts | 20:46 |
| melwitt | new instances in queens should Just Work(tm) | 20:47 |
| melwitt | nova meeting in 10 minutes | 20:50 |
| tomtom001 | melwitt: thanks! really appreciate your help on this | 20:51 |
| melwitt | tomtom001: np, hope you get it working. let us know | 20:52 |
| *** takashin has joined #openstack-nova | 20:52 | |
| tomtom001 | will do | 20:53 |
| *** markvoelker has joined #openstack-nova | 21:09 | |
| *** _fragatina has joined #openstack-nova | 21:15 | |
| *** pbing19 has quit IRC | 21:19 | |
| *** eharney has quit IRC | 21:23 | |
| *** xek has quit IRC | 21:27 | |
| -openstackstatus- NOTICE: Jobs are failing due to ssh host key mismatches caused by duplicate IPs in a test cloud region. We are disabling the region and will let you know when jobs can be rechecked. | 21:32 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (3) https://review.openstack.org/574104 | 21:35 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4) https://review.openstack.org/574106 | 21:35 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5) https://review.openstack.org/574110 | 21:35 |
| *** agopi_ has joined #openstack-nova | 21:39 | |
| *** whoami-rajat has joined #openstack-nova | 21:41 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Start functional testing for cross-cell resize https://review.openstack.org/636253 | 21:41 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source https://review.openstack.org/637058 | 21:41 |
| *** agopi has quit IRC | 21:42 | |
| *** markvoelker has quit IRC | 21:42 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source https://review.openstack.org/637058 | 21:43 |
| openstackgerrit | Merged openstack/nova master: Change sqlalchemy warnings filter to an error https://review.openstack.org/636957 | 21:46 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova stable/rocky: Replace glance command with openstack command https://review.openstack.org/637060 | 21:46 |
| openstackgerrit | Merged openstack/nova master: Lock detach_volume https://review.openstack.org/636701 | 21:46 |
| *** burt has quit IRC | 21:47 | |
| *** dave-mccowan has quit IRC | 21:49 | |
| *** burt has joined #openstack-nova | 21:51 | |
| *** cdent has quit IRC | 21:57 | |
| *** wolverineav has quit IRC | 22:05 | |
| -openstackstatus- NOTICE: The test cloud region using duplicate IPs has been removed from nodepool. Jobs can be rechecked now. | 22:14 | |
| * melwitt upgrades tox | 22:16 | |
| *** mvkr has joined #openstack-nova | 22:16 | |
| *** hongbin has joined #openstack-nova | 22:20 | |
| *** ade_lee_ has quit IRC | 22:23 | |
| *** wolverineav has joined #openstack-nova | 22:30 | |
| efried | melwitt: #sorrynotsorry | 22:32 |
| efried | melwitt: fwiw, you will notice marked speedups | 22:32 |
| melwitt | haha, I was hoping you'd see that :) | 22:32 |
| *** wolverineav has quit IRC | 22:35 | |
| *** markvoelker has joined #openstack-nova | 22:39 | |
| *** tesseract has quit IRC | 22:42 | |
| *** agopi_ has quit IRC | 22:43 | |
| *** wolverineav has joined #openstack-nova | 22:49 | |
| *** mdrabe has quit IRC | 22:53 | |
| *** wolverineav has quit IRC | 22:53 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize_at_source https://review.openstack.org/637058 | 22:56 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask https://review.openstack.org/637070 | 22:56 |
| *** wolverineav has joined #openstack-nova | 22:56 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/nova master: ironic: check fresh data when sync_power_state doesn't line up https://review.openstack.org/636699 | 22:58 |
| *** moshele has joined #openstack-nova | 22:59 | |
| *** moshele has quit IRC | 23:01 | |
| *** tkajinam has joined #openstack-nova | 23:02 | |
| *** ade_lee_ has joined #openstack-nova | 23:03 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add user_id field to InstanceMapping https://review.openstack.org/633350 | 23:06 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add online data migration for populating user_id https://review.openstack.org/633351 | 23:06 |
| openstackgerrit | Merged openstack/nova stable/rocky: Ignore VolumeAttachmentNotFound exception in compute.manager https://review.openstack.org/636954 | 23:07 |
| openstackgerrit | Merged openstack/nova stable/rocky: Provide a useful error message when trying to update non-compute services https://review.openstack.org/636955 | 23:08 |
| *** ade_lee_ has quit IRC | 23:10 | |
| *** ade_lee_ has joined #openstack-nova | 23:11 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add ConfirmResizeTask https://review.openstack.org/637070 | 23:12 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add confirm_snapshot_based_resize conductor RPC method https://review.openstack.org/637075 | 23:12 |
| *** markvoelker has quit IRC | 23:12 | |
| *** erlon_ has joined #openstack-nova | 23:21 | |
| *** wolverineav has quit IRC | 23:23 | |
| *** wolverineav has joined #openstack-nova | 23:26 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: ksa auth conf and client for cyborg access https://review.openstack.org/631242 | 23:29 |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Add Cyborg device profile groups to spec obj. https://review.openstack.org/631243 | 23:29 |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Create and bind Cyborg ARQs. https://review.openstack.org/631244 | 23:29 |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Get resolved Cyborg ARQs and add PCI BDFs to VM's domain XML. https://review.openstack.org/631245 | 23:29 |
| *** hongbin has quit IRC | 23:35 | |
| *** agopi_ has joined #openstack-nova | 23:51 | |
| *** tssurya has quit IRC | 23:54 | |
| *** macza has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!