| *** Guest10778 has quit IRC | 00:00 | |
| mriedem | melwitt: this is what i wanted the retrospective link for http://lists.openstack.org/pipermail/openstack-dev/2018-February/127402.html | 00:02 |
|---|---|---|
| *** acormier has quit IRC | 00:02 | |
| melwitt | mriedem: a-ha, nice. I was just looking for that email to link in the agenda for tomorrow's meeting | 00:02 |
| *** Guest10778 has joined #openstack-nova | 00:04 | |
| *** acormier has joined #openstack-nova | 00:05 | |
| *** jobewan has quit IRC | 00:06 | |
| mnaser | here goes nothing :> | 00:06 |
| *** hshiina|afk is now known as hshiina | 00:08 | |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Drop extra loop which modifies Cinder volume status https://review.openstack.org/539758 | 00:08 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Clean up ports and volumes when deleting ERROR instance https://review.openstack.org/340614 | 00:08 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Add functional tests to ensure BDM removal on delete https://review.openstack.org/544747 | 00:08 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Store block device mappings in cell0 https://review.openstack.org/544748 | 00:08 |
| *** sdague has quit IRC | 00:09 | |
| mnaser | i guess at some point we'll have to squash the func. tests and clean up commit to get them to merge | 00:14 |
| melwitt | mriedem: just updated the agenda for tomorrow's meeting https://wiki.openstack.org/wiki/Meetings/Nova | 00:15 |
| mriedem | mnaser: what we usually do, | 00:15 |
| mriedem | is land the regression test that asserts the failre, | 00:16 |
| melwitt | mnaser: why? the way we do that is the func test is introduced with the after-bug-fix asserts commented out with a note (and I'll let mriedem finish) | 00:16 |
| mriedem | then the patch that fixes it changes the functional test to show it's passing | 00:16 |
| mnaser | ok i see so then my functional test is wrong | 00:17 |
| mnaser | because it is a failing one (which will pass in the patch above of it) | 00:17 |
| *** acormier has quit IRC | 00:17 | |
| *** acormier has joined #openstack-nova | 00:18 | |
| mnaser | so i will switch the best to make it assert the failure and then change the fixing patch as explained | 00:18 |
| melwitt | mnaser: that's nearly there, just have to comment out the failing assert and add TODO(mnaser) above it to explain it should be uncommented in the same patch as the fix, then add an assert for the wrong (but expected) thing with a TODO to remove it in the fix patch | 00:19 |
| *** rmcall has joined #openstack-nova | 00:20 | |
| *** rmcall has quit IRC | 00:20 | |
| *** jdurgin has joined #openstack-nova | 00:23 | |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Store block device mappings in cell0 https://review.openstack.org/544748 | 00:26 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Add functional tests to ensure BDM removal on delete https://review.openstack.org/544747 | 00:26 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Clean up ports and volumes when deleting ERROR instance https://review.openstack.org/340614 | 00:26 |
| mnaser | that should do it | 00:26 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Drop extra loop which modifies Cinder volume status https://review.openstack.org/539758 | 00:27 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Store block device mappings in cell0 https://review.openstack.org/544748 | 00:27 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Add functional tests to ensure BDM removal on delete https://review.openstack.org/544747 | 00:27 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Clean up ports and volumes when deleting ERROR instance https://review.openstack.org/340614 | 00:27 |
| *** chyka has quit IRC | 00:35 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova-specs master: Enable NUMA Features for Libvirt/QEMU Driver https://review.openstack.org/533077 | 00:36 |
| *** tetsuro has joined #openstack-nova | 00:37 | |
| *** s1061123_ has quit IRC | 00:39 | |
| mriedem | mnaser: melwitt: ok i've gone through the first 3, left some nits but nothing major | 00:39 |
| mriedem | i can get the final big patch in the morning | 00:40 |
| melwitt | thanks | 00:42 |
| mnaser | mriedem: thanks, im addressing them now and rebasing | 00:43 |
| *** s1061123 has joined #openstack-nova | 00:43 | |
| *** moshele has joined #openstack-nova | 00:47 | |
| *** mriedem has quit IRC | 00:50 | |
| *** acormier has quit IRC | 00:50 | |
| tonyb | melwitt: It is indeed confusing. The "next pahse" shoudl just go strainght to EOL but there is a window in there where it's in Phase III while we do the EOLing ... and then we went and confused ourselves with the Long lived stable branches so I don't know what'll happen | 00:53 |
| tonyb | melwitt: but there's a session at the PTG to discuss it so hopefully shortly after that I can at least make that table look more sane | 00:54 |
| *** dave-mccowan has joined #openstack-nova | 00:54 | |
| melwitt | tonyb: thanks for the glimpse at what it means :) | 00:54 |
| tonyb | melwitt: ;P I guess you have to care more about that stuff now:) | 00:55 |
| melwitt | nah, total coincidence | 00:55 |
| tonyb | melwitt: anytime I can help you know where I am | 00:55 |
| melwitt | :) thanks | 00:55 |
| tonyb | melwitt: ohh my bad ;P | 00:55 |
| melwitt | heh | 00:56 |
| *** r-daneel has quit IRC | 00:56 | |
| *** acormier has joined #openstack-nova | 01:08 | |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Store block device mappings in cell0 https://review.openstack.org/544748 | 01:09 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Add functional tests to ensure BDM removal on delete https://review.openstack.org/544747 | 01:09 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Clean up ports and volumes when deleting ERROR instance https://review.openstack.org/340614 | 01:09 |
| mnaser | addressed all comments ^ | 01:10 |
| *** acormier has quit IRC | 01:12 | |
| *** acormier has joined #openstack-nova | 01:13 | |
| *** suresh12 has quit IRC | 01:14 | |
| *** acormier has quit IRC | 01:20 | |
| *** Guest10778 has quit IRC | 01:26 | |
| *** yassine has joined #openstack-nova | 01:27 | |
| *** acormier has joined #openstack-nova | 01:27 | |
| *** yassine is now known as Guest48782 | 01:27 | |
| *** jdurgin has quit IRC | 01:30 | |
| *** acormier has quit IRC | 01:30 | |
| *** acormier has joined #openstack-nova | 01:30 | |
| *** moshele has quit IRC | 01:40 | |
| *** lbragstad has quit IRC | 01:45 | |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Store block device mappings in cell0 https://review.openstack.org/544748 | 01:46 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Add functional tests to ensure BDM removal on delete https://review.openstack.org/544747 | 01:46 |
| openstackgerrit | Mohammed Naser proposed openstack/nova master: Clean up ports and volumes when deleting ERROR instance https://review.openstack.org/340614 | 01:46 |
| mnaser | ..broke some functional tests and fixed them.. | 01:46 |
| *** itlinux has joined #openstack-nova | 01:47 | |
| *** jdurgin has joined #openstack-nova | 01:53 | |
| *** chyka has joined #openstack-nova | 01:57 | |
| *** jdurgin has quit IRC | 01:57 | |
| *** suresh12 has joined #openstack-nova | 01:58 | |
| *** yamahata has quit IRC | 02:01 | |
| *** chyka has quit IRC | 02:01 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add periodic task to clean expired console tokens https://review.openstack.org/325381 | 02:04 |
| openstackgerrit | melanie witt proposed openstack/nova master: Use ConsoleAuthToken object to generate authorizations https://review.openstack.org/325414 | 02:04 |
| openstackgerrit | melanie witt proposed openstack/nova master: Convert websocketproxy to use db for token validation https://review.openstack.org/333990 | 02:04 |
| *** hiro-kobayashi has joined #openstack-nova | 02:06 | |
| *** tbachman has quit IRC | 02:12 | |
| *** slaweq has joined #openstack-nova | 02:13 | |
| *** acormier has quit IRC | 02:15 | |
| *** slaweq has quit IRC | 02:18 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: [libvirt] Add _get_vcpu_realtime_scheduler() https://review.openstack.org/527630 | 02:18 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: [libvirt] Add _get_numa_memnode() https://review.openstack.org/529906 | 02:18 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: [libvirt] Add _get_XXXpin_cpuset() https://review.openstack.org/527631 | 02:18 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add NumaTopology support for libvirt/qemu driver https://review.openstack.org/530451 | 02:18 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: disable cpu pinning with libvirt/qemu driver https://review.openstack.org/531049 | 02:18 |
| *** acormier has joined #openstack-nova | 02:35 | |
| *** slaweq has joined #openstack-nova | 02:36 | |
| *** slaweq has quit IRC | 02:39 | |
| *** salv-orlando has joined #openstack-nova | 02:44 | |
| *** suresh12 has quit IRC | 02:44 | |
| *** suresh12 has joined #openstack-nova | 02:46 | |
| *** acormier has quit IRC | 02:47 | |
| *** salv-orl_ has quit IRC | 02:47 | |
| *** acormier has joined #openstack-nova | 02:47 | |
| *** bhujay has joined #openstack-nova | 02:50 | |
| *** suresh12 has quit IRC | 02:51 | |
| *** acormier has quit IRC | 02:51 | |
| *** harlowja has quit IRC | 03:08 | |
| *** bhujay has quit IRC | 03:09 | |
| *** chyka has joined #openstack-nova | 03:12 | |
| *** chyka has quit IRC | 03:17 | |
| *** yamahata has joined #openstack-nova | 03:17 | |
| *** suresh12 has joined #openstack-nova | 03:21 | |
| *** amodi has joined #openstack-nova | 03:22 | |
| *** suresh12 has quit IRC | 03:26 | |
| *** amodi has quit IRC | 03:29 | |
| *** hongbin has joined #openstack-nova | 03:34 | |
| *** itlinux has quit IRC | 03:37 | |
| *** yamamoto has joined #openstack-nova | 03:47 | |
| *** yamamoto has quit IRC | 03:47 | |
| *** yamamoto has joined #openstack-nova | 03:51 | |
| openstackgerrit | Hironori Shiina proposed openstack/nova master: ironic: Clean up resources after unprovision fails https://review.openstack.org/544772 | 03:54 |
| *** bkopilov has quit IRC | 04:00 | |
| *** itlinux has joined #openstack-nova | 04:01 | |
| *** bkopilov has joined #openstack-nova | 04:03 | |
| *** bhujay has joined #openstack-nova | 04:08 | |
| openstackgerrit | Merged openstack/nova master: Fix nits in allocation candidate limit handling https://review.openstack.org/536784 | 04:13 |
| *** jogo has quit IRC | 04:13 | |
| *** udesale has joined #openstack-nova | 04:13 | |
| *** gyee has quit IRC | 04:14 | |
| *** itlinux has quit IRC | 04:17 | |
| *** elod has quit IRC | 04:18 | |
| *** dave-mccowan has quit IRC | 04:25 | |
| *** psachin has joined #openstack-nova | 04:27 | |
| *** masber has joined #openstack-nova | 04:33 | |
| *** links has joined #openstack-nova | 04:34 | |
| *** elod has joined #openstack-nova | 04:35 | |
| *** vladikr has quit IRC | 04:37 | |
| *** lpetrut has joined #openstack-nova | 04:39 | |
| *** vladikr has joined #openstack-nova | 04:40 | |
| *** abhishekk has joined #openstack-nova | 04:41 | |
| *** links has quit IRC | 04:42 | |
| *** hongbin has quit IRC | 04:43 | |
| *** links has joined #openstack-nova | 04:45 | |
| *** r-daneel has joined #openstack-nova | 04:57 | |
| *** janki has joined #openstack-nova | 04:57 | |
| *** janki has quit IRC | 05:03 | |
| *** janki has joined #openstack-nova | 05:03 | |
| *** janki has quit IRC | 05:06 | |
| *** janki has joined #openstack-nova | 05:06 | |
| *** lpetrut has quit IRC | 05:08 | |
| *** janki has quit IRC | 05:08 | |
| *** janki has joined #openstack-nova | 05:09 | |
| *** threestrands_ has joined #openstack-nova | 05:09 | |
| *** janki has quit IRC | 05:09 | |
| *** threestrands_ has quit IRC | 05:09 | |
| *** threestrands_ has joined #openstack-nova | 05:09 | |
| *** threestrands has quit IRC | 05:09 | |
| *** jogo has joined #openstack-nova | 05:10 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: [libvirt] Add _get_XXXpin_cpuset() https://review.openstack.org/527631 | 05:10 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add NumaTopology support for libvirt/qemu driver https://review.openstack.org/530451 | 05:10 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: disable cpu pinning with libvirt/qemu driver https://review.openstack.org/531049 | 05:10 |
| *** janki has joined #openstack-nova | 05:10 | |
| *** threestrands_ has quit IRC | 05:10 | |
| *** threestrands_ has joined #openstack-nova | 05:11 | |
| *** threestrands_ has quit IRC | 05:11 | |
| *** threestrands_ has joined #openstack-nova | 05:11 | |
| *** harlowja has joined #openstack-nova | 05:15 | |
| *** threestrands has joined #openstack-nova | 05:15 | |
| *** threestrands has quit IRC | 05:15 | |
| *** threestrands has joined #openstack-nova | 05:15 | |
| *** sree has joined #openstack-nova | 05:15 | |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: [libvirt] Add _get_vcpu_realtime_scheduler() https://review.openstack.org/527630 | 05:17 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: [libvirt] Add _get_numa_memnode() https://review.openstack.org/529906 | 05:17 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: [libvirt] Add _get_XXXpin_cpuset() https://review.openstack.org/527631 | 05:17 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add NumaTopology support for libvirt/qemu driver https://review.openstack.org/530451 | 05:17 |
| openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: disable cpu pinning with libvirt/qemu driver https://review.openstack.org/531049 | 05:17 |
| *** threestrands_ has quit IRC | 05:18 | |
| *** lpetrut has joined #openstack-nova | 05:19 | |
| *** threestrands has quit IRC | 05:28 | |
| *** lpetrut has quit IRC | 05:34 | |
| *** jaosorior has quit IRC | 05:37 | |
| *** lajoskatona has joined #openstack-nova | 05:39 | |
| *** janki has quit IRC | 05:40 | |
| *** janki has joined #openstack-nova | 05:40 | |
| *** tetsuro has left #openstack-nova | 05:42 | |
| *** sridharg has joined #openstack-nova | 05:44 | |
| *** jaosorior has joined #openstack-nova | 05:44 | |
| *** mdnadeem has joined #openstack-nova | 05:46 | |
| *** udesale has quit IRC | 05:48 | |
| *** udesale has joined #openstack-nova | 05:48 | |
| *** lpetrut has joined #openstack-nova | 06:06 | |
| *** jchhatbar has joined #openstack-nova | 06:09 | |
| *** janki has quit IRC | 06:09 | |
| *** harlowja has quit IRC | 06:14 | |
| *** slaweq has joined #openstack-nova | 06:15 | |
| *** lpetrut has quit IRC | 06:19 | |
| *** slaweq has quit IRC | 06:19 | |
| *** ameeda has joined #openstack-nova | 06:20 | |
| *** Eran_Kuris has joined #openstack-nova | 06:26 | |
| *** rcernin has quit IRC | 06:44 | |
| *** stakeda has joined #openstack-nova | 06:51 | |
| *** mrodden has quit IRC | 06:54 | |
| *** jchhatbar has quit IRC | 06:56 | |
| *** hoonetorg has quit IRC | 06:56 | |
| *** janki has joined #openstack-nova | 06:56 | |
| *** hoonetorg has joined #openstack-nova | 06:57 | |
| *** sree has quit IRC | 07:01 | |
| *** sree has joined #openstack-nova | 07:01 | |
| *** sapcc-bot2 has quit IRC | 07:01 | |
| *** sapcc-bot has joined #openstack-nova | 07:02 | |
| *** ccamacho has quit IRC | 07:02 | |
| *** sree has quit IRC | 07:06 | |
| *** udesale_ has joined #openstack-nova | 07:07 | |
| *** udesale has quit IRC | 07:09 | |
| *** hshiina has quit IRC | 07:09 | |
| *** threestrands has joined #openstack-nova | 07:09 | |
| *** threestrands has quit IRC | 07:09 | |
| *** threestrands has joined #openstack-nova | 07:09 | |
| *** hshiina has joined #openstack-nova | 07:09 | |
| *** janki has quit IRC | 07:09 | |
| *** janki has joined #openstack-nova | 07:10 | |
| *** jamesdenton has quit IRC | 07:10 | |
| *** moshele has joined #openstack-nova | 07:18 | |
| *** sree has joined #openstack-nova | 07:20 | |
| openstackgerrit | Merged openstack/nova master: Avoid inventory DELETE API (no conflict detection) https://review.openstack.org/539712 | 07:22 |
| *** udesale__ has joined #openstack-nova | 07:23 | |
| *** psachin has quit IRC | 07:24 | |
| *** udesale_ has quit IRC | 07:26 | |
| *** rcernin has joined #openstack-nova | 07:28 | |
| *** chyka has joined #openstack-nova | 07:29 | |
| *** lpetrut has joined #openstack-nova | 07:29 | |
| *** sshwarts has joined #openstack-nova | 07:29 | |
| *** chyka has quit IRC | 07:33 | |
| *** pcaruana has joined #openstack-nova | 07:40 | |
| *** andreas_s has joined #openstack-nova | 07:40 | |
| *** edmondsw has joined #openstack-nova | 07:41 | |
| *** slaweq has joined #openstack-nova | 07:45 | |
| *** edmondsw has quit IRC | 07:45 | |
| *** tssurya has joined #openstack-nova | 07:50 | |
| *** ragiman has joined #openstack-nova | 07:55 | |
| *** belmoreira has joined #openstack-nova | 07:55 | |
| *** psachin has joined #openstack-nova | 07:59 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/541561 | 08:01 |
| *** AlexeyAbashkin has joined #openstack-nova | 08:08 | |
| *** gibi has joined #openstack-nova | 08:13 | |
| gibi | good morning | 08:13 |
| *** slaweq_ has joined #openstack-nova | 08:15 | |
| *** tesseract has joined #openstack-nova | 08:17 | |
| *** slaweq_ has quit IRC | 08:20 | |
| *** alexchadin has joined #openstack-nova | 08:22 | |
| *** ansiwen has joined #openstack-nova | 08:23 | |
| hrw | morning | 08:23 |
| *** mdbooth has joined #openstack-nova | 08:24 | |
| *** danpawlik has quit IRC | 08:24 | |
| *** danpawlik has joined #openstack-nova | 08:25 | |
| *** jpena|off is now known as jpena | 08:26 | |
| *** lajoskatona has quit IRC | 08:30 | |
| *** links has quit IRC | 08:35 | |
| ameeda | morning :) | 08:42 |
| ameeda | gibi: thanks for reviews. what about this https://review.openstack.org/#/c/542378/ ? I am not sure why bug reporter asked to add this file, do you guess that I have to remove it ? | 08:43 |
| *** mgoddard_ has joined #openstack-nova | 08:44 | |
| ameeda | also regarding this https://review.openstack.org/#/c/543348/, I will ask openstack-requirements channel to get suitable answer | 08:44 |
| gibi | ameeda: by reading the bug report I think dhellmann wants to move the doc only requirements to a separate file and _then_ change tox to use that requirement file for doc generation | 08:47 |
| gibi | ameeda: I think you missed that second part | 08:48 |
| ameeda | gibi: let me check. brb | 08:48 |
| *** jaianshu has joined #openstack-nova | 08:50 | |
| *** links has joined #openstack-nova | 08:52 | |
| *** lajoskatona has joined #openstack-nova | 08:53 | |
| ameeda | gibi: thank for the update, I guess the edit of tox.ini will be here https://review.openstack.org/#/c/543348/1/tox.ini , so the old tox.ini was running correctly since the test-requirements have all requirements. | 08:55 |
| ameeda | gibi: it make since ? | 08:55 |
| *** alexchadin has quit IRC | 08:56 | |
| *** alexchadin has joined #openstack-nova | 08:57 | |
| gmann_ | gibi: your fix looks fine just comment for newton branch - https://review.openstack.org/#/c/538908/3 | 09:01 |
| *** sambetts|afk has quit IRC | 09:03 | |
| *** ccamacho has joined #openstack-nova | 09:03 | |
| gibi | ameeda: I don't get it. This patch https://review.openstack.org/#/c/543348/1/tox.ini still does not refer to the doc/requirements.txt you created in https://review.openstack.org/#/c/542378/ | 09:04 |
| *** ccamacho has quit IRC | 09:04 | |
| gibi | ameeda: so something is missing | 09:04 |
| *** ccamacho has joined #openstack-nova | 09:04 | |
| ameeda | gibi: so I have to edit this https://review.openstack.org/#/c/543348/1/tox.ini to have -r{toxinidir}/doc/requirements.txt ? so my edit must be in this review https://review.openstack.org/#/c/543348/ to avoid merge conflict ? | 09:06 |
| *** sambetts_ has joined #openstack-nova | 09:07 | |
| *** amoralej|off is now known as amoralej | 09:09 | |
| gibi | ameeda: I would keep your two change separate. I would change the tox.ini in https://review.openstack.org/#/c/542378/ to use the new doc/requirements.txt | 09:09 |
| gibi | ameeda: for the testenv:docs and the testenv:releasenotes target | 09:09 |
| ameeda | gibi: what should I write to use it ? | 09:10 |
| *** hiro-kobayashi has quit IRC | 09:10 | |
| ameeda | gibi: deps = -r{toxinidir}/doc/requirements.txt, right ? | 09:11 |
| gibi | ameeda: I guess deps = -r{toxinidir}/doc/test-requirements.txt | 09:11 |
| gibi | ameeda: yes | 09:11 |
| ameeda | okay, Thanks for help ^_^ | 09:11 |
| *** derekh has joined #openstack-nova | 09:12 | |
| gibi | gmann_: looking | 09:13 |
| ameeda | gibi: is this correct https://pasteboard.co/H7JnfYu.png ? | 09:13 |
| gibi | ameeda: at least that is what I would try :) | 09:14 |
| ameeda | gibi: so I will upload new patch now ^_^ | 09:15 |
| openstackgerrit | Ameed Ashour proposed openstack/osc-placement master: Change documentation theme https://review.openstack.org/542378 | 09:15 |
| *** chyka has joined #openstack-nova | 09:18 | |
| *** dtantsur|afk is now known as dtantsur | 09:21 | |
| *** threestrands_ has joined #openstack-nova | 09:21 | |
| *** threestrands has quit IRC | 09:21 | |
| *** chyka has quit IRC | 09:23 | |
| *** ralonsoh has joined #openstack-nova | 09:23 | |
| *** hshiina is now known as hshiina|afk | 09:25 | |
| openstackgerrit | Ameed Ashour proposed openstack/osc-placement master: Change documentation theme https://review.openstack.org/542378 | 09:26 |
| *** edmondsw has joined #openstack-nova | 09:29 | |
| openstackgerrit | Ameed Ashour proposed openstack/osc-placement master: Change documentation theme https://review.openstack.org/542378 | 09:32 |
| *** edmondsw has quit IRC | 09:34 | |
| *** alexchadin has quit IRC | 09:37 | |
| *** alexchadin has joined #openstack-nova | 09:38 | |
| bauzas | morning Nova | 09:40 |
| ameeda | bauzas: morning :) | 09:41 |
| gibi | gmann_: I removed the newton branches from https://review.openstack.org/#/c/538908/ | 09:43 |
| stephenfin | bauzas: Oh hai. I'm going to be pushing up a reworked version of https://review.openstack.org/#/c/541290/ shortly and would love your thoughts :) | 09:44 |
| stephenfin | Let me know if you want me to return the favour anywhere | 09:44 |
| bauzas | stephenfin: sure thing | 09:45 |
| kashyap | bauzas: Hey, when you get a sec, a question for you on #openstack-dev (on writing a new scheduler filter) | 09:45 |
| bauzas | kashyap: ack | 09:45 |
| *** cdent has joined #openstack-nova | 09:45 | |
| bauzas | kashyap: why #openstack-dev btw. ? :) | 09:46 |
| bauzas | since I very rarely scroll it | 09:46 |
| kashyap | Heh | 09:46 |
| * stephenfin didn't even know #openstack-dev existed | 09:46 | |
| kashyap | bauzas: It's not me, someone else asked the question | 09:46 |
| bauzas | hah ok | 09:46 |
| bauzas | np | 09:46 |
| kashyap | I noticed the question and prompted you :-) | 09:46 |
| kashyap | I _think_ he's left the channel :-( | 09:46 |
| kashyap | I recognize the name of the developer from Kubernetes community | 09:47 |
| cdent | stephenfin: it's funny how that works isn't it? so much stuff can stay under the radar, no matter how long you're around. | 09:47 |
| stephenfin | cdent: For sure. You never stop learning new stuff | 09:48 |
| *** acormier has joined #openstack-nova | 09:48 | |
| stephenfin | Though I suppose if you did, it would probably be time to move on | 09:48 |
| cdent | true | 09:49 |
| *** bhujay has quit IRC | 09:49 | |
| *** bhujay has joined #openstack-nova | 09:50 | |
| *** acormier has quit IRC | 09:53 | |
| openstackgerrit | Bence Romsics proposed openstack/nova master: Clarify 'capacity' in placement api-ref https://review.openstack.org/544347 | 10:00 |
| *** hshiina|afk has quit IRC | 10:01 | |
| *** yamahata has quit IRC | 10:05 | |
| *** sree has quit IRC | 10:06 | |
| *** sree has joined #openstack-nova | 10:07 | |
| openstackgerrit | Chris Dent proposed openstack/nova master: WIP: Move resource provider objects into placement hierarchy https://review.openstack.org/540049 | 10:08 |
| openstackgerrit | Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766 | 10:08 |
| openstackgerrit | Chris Dent proposed openstack/nova master: WIP: Isolate placement database config https://review.openstack.org/541435 | 10:08 |
| openstackgerrit | Chris Dent proposed openstack/nova master: Isolate config parse_args for placement https://review.openstack.org/543495 | 10:08 |
| *** sree has quit IRC | 10:11 | |
| bauzas | does anyone know if our nova documentation includes HTML rendered docstrings of our internal interfaces ? | 10:13 |
| bauzas | stephenfin: ^ | 10:13 |
| bauzas | I'd like to point in a document some object structures | 10:14 |
| stephenfin | bauzas: Nope | 10:14 |
| bauzas | okay, then I'll say look at the code | 10:14 |
| stephenfin | That would need Sphinx's 'apidoc' tool, which we don't use enable | 10:14 |
| stephenfin | That'd be the best call, yeah | 10:14 |
| *** slaweq_ has joined #openstack-nova | 10:16 | |
| *** sambetts_ is now known as sambetts | 10:21 | |
| *** slaweq_ has quit IRC | 10:21 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: doc: Clarify how to create your own filter https://review.openstack.org/544836 | 10:23 |
| bauzas | stephenfin: gibi: doc nits fix in https://review.openstack.org/544836 | 10:23 |
| bauzas | related to kashyap's proxying someone for clarifications about scheduler filters | 10:24 |
| kashyap | bauzas: Thanks; will look at the review in a few | 10:24 |
| stephenfin | bauzas: One newline missing and I'm +2 | 10:24 |
| bauzas | stephenfin: ack, thanks | 10:29 |
| *** udesale_ has joined #openstack-nova | 10:30 | |
| *** udesale__ has quit IRC | 10:33 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: doc: Clarify how to create your own filter https://review.openstack.org/544836 | 10:37 |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec https://review.openstack.org/541290 | 10:38 |
| stephenfin | bauzas: Boom ^ | 10:38 |
| *** sdague has joined #openstack-nova | 10:38 | |
| bauzas | stephenfin: okay, I'll look at your spec once I'm done with jaypipes's one about aggregate ratios | 10:39 |
| *** stakeda has quit IRC | 10:41 | |
| *** sridharg has quit IRC | 10:41 | |
| openstackgerrit | Ameed Ashour proposed openstack/osc-placement master: Change documentation theme https://review.openstack.org/542378 | 10:42 |
| *** priteau has joined #openstack-nova | 10:59 | |
| *** jistr is now known as jistr|mtg | 11:00 | |
| *** yamamoto has quit IRC | 11:01 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add 'numa-aware-vswitches' spec https://review.openstack.org/541290 | 11:02 |
| *** lucas-afk is now known as lucasagomes | 11:04 | |
| *** chyka has joined #openstack-nova | 11:04 | |
| *** yamamoto has joined #openstack-nova | 11:06 | |
| *** tbachman has joined #openstack-nova | 11:08 | |
| *** chyka has quit IRC | 11:09 | |
| openstackgerrit | Ameed Ashour proposed openstack/osc-placement master: Change documentation theme https://review.openstack.org/542378 | 11:12 |
| *** udesale__ has joined #openstack-nova | 11:14 | |
| *** abhishekk has quit IRC | 11:15 | |
| *** udesale_ has quit IRC | 11:16 | |
| openstackgerrit | Ameed Ashour proposed openstack/osc-placement master: Change documentation theme https://review.openstack.org/542378 | 11:28 |
| *** udesale__ has quit IRC | 11:30 | |
| openstackgerrit | Merged openstack/nova master: doc: Clarify how to create your own filter https://review.openstack.org/544836 | 11:31 |
| *** tssurya has quit IRC | 11:34 | |
| *** acormier has joined #openstack-nova | 11:36 | |
| *** Tom-Tom has joined #openstack-nova | 11:36 | |
| *** abalutoiu_ has quit IRC | 11:42 | |
| *** tssurya has joined #openstack-nova | 11:43 | |
| *** abalutoiu has joined #openstack-nova | 11:43 | |
| *** sree has joined #openstack-nova | 11:45 | |
| *** sree has quit IRC | 11:50 | |
| *** jpena is now known as jpena|lunch | 11:52 | |
| *** tbachman has quit IRC | 11:53 | |
| *** sree has joined #openstack-nova | 11:55 | |
| *** sree has quit IRC | 11:55 | |
| *** sree has joined #openstack-nova | 11:56 | |
| *** sree_ has joined #openstack-nova | 11:57 | |
| *** sree_ is now known as Guest43911 | 11:57 | |
| *** sree has quit IRC | 12:01 | |
| *** acormier has quit IRC | 12:03 | |
| *** yamamoto has quit IRC | 12:09 | |
| *** bhujay has quit IRC | 12:09 | |
| *** dave-mccowan has joined #openstack-nova | 12:10 | |
| *** bhujay has joined #openstack-nova | 12:10 | |
| *** janki is now known as janki|aafk | 12:12 | |
| *** janki|aafk is now known as janki|afk | 12:12 | |
| *** slaweq_ has joined #openstack-nova | 12:17 | |
| *** slaweq_ has quit IRC | 12:22 | |
| *** yamamoto has joined #openstack-nova | 12:22 | |
| *** jistr|mtg is now known as jistr | 12:23 | |
| openstackgerrit | Chris Dent proposed openstack/nova-specs master: VMware: add support for live migration https://review.openstack.org/299207 | 12:33 |
| *** sridharg has joined #openstack-nova | 12:35 | |
| *** chyka has joined #openstack-nova | 12:37 | |
| *** yamamoto has quit IRC | 12:39 | |
| *** chyka has quit IRC | 12:41 | |
| *** mgoddard_ has quit IRC | 12:42 | |
| *** tssurya has quit IRC | 12:47 | |
| *** yamamoto has joined #openstack-nova | 12:54 | |
| *** jaianshu has quit IRC | 13:02 | |
| *** yamamoto has quit IRC | 13:05 | |
| *** edmondsw has joined #openstack-nova | 13:06 | |
| *** tssurya has joined #openstack-nova | 13:08 | |
| *** edmondsw has quit IRC | 13:10 | |
| *** pooja-jadhav is now known as pooja_jadhav | 13:10 | |
| openstackgerrit | Bence Romsics proposed openstack/nova master: Clarify 'capacity' in placement api-ref https://review.openstack.org/544347 | 13:13 |
| *** Guest43911 has quit IRC | 13:14 | |
| *** sree has joined #openstack-nova | 13:14 | |
| *** sree has quit IRC | 13:21 | |
| *** mgoddard_ has joined #openstack-nova | 13:21 | |
| *** edmondsw has joined #openstack-nova | 13:24 | |
| *** udesale has joined #openstack-nova | 13:25 | |
| openstackgerrit | Thomas Goirand proposed openstack/nova master: Python 3 fix for sphinx doc https://review.openstack.org/544956 | 13:26 |
| ameeda | Hello, what do you think if I take this review and complete on it ? https://review.openstack.org/#/c/525253/ | 13:30 |
| *** kaisers has quit IRC | 13:31 | |
| *** acormier has joined #openstack-nova | 13:32 | |
| *** kaisers has joined #openstack-nova | 13:32 | |
| *** pchavva has joined #openstack-nova | 13:33 | |
| *** acormier has quit IRC | 13:34 | |
| *** acormier has joined #openstack-nova | 13:34 | |
| *** jpena|lunch is now known as jpena | 13:37 | |
| *** acormier has quit IRC | 13:38 | |
| *** abhishekk has joined #openstack-nova | 13:42 | |
| *** elmaciej has joined #openstack-nova | 13:42 | |
| hrw | ameeda: you mean "take it, update, send for review"? | 13:48 |
| hrw | ameeda: if you feel that change is needed but abandoned by original author then why not | 13:48 |
| ameeda | yes | 13:48 |
| hrw | ameeda: my way from 'wth is kolla' to 'kolla core reviewer' started from taking over one patch | 13:49 |
| ameeda | original author doesn't do any activity from awhile "5 weeks" | 13:49 |
| ameeda | hrw: I am not sure if that is legal to take this patch, I want approval, what if I comment on the bug to ask current assignee if he still working on the bug. also I am not sure about the rank of this bug. | 13:54 |
| ameeda | hrw: since I have patches need for review from awhile. | 13:55 |
| *** ragiman has quit IRC | 13:57 | |
| *** esberglu has joined #openstack-nova | 13:58 | |
| hrw | ameeda: do what you think. nova is foss and mia maintainer can be overriden imho with new ver of patch | 13:59 |
| *** yamamoto has joined #openstack-nova | 14:00 | |
| hrw | ameeda: sometimes during review it is easier for reviewer to send new version than comment | 14:00 |
| *** mriedem has joined #openstack-nova | 14:01 | |
| ameeda | hrw: so you guess me to mark current patch as abandoned then upload new patch | 14:02 |
| jroll | ameeda: cfriesen is the author, he is active in this channel, might as well ask him | 14:02 |
| hrw | ameeda: no. | 14:02 |
| ameeda | cfriesen: are you around ? | 14:02 |
| *** dtantsur is now known as dtantsur|brb | 14:02 | |
| hrw | ameeda: git review 525253 -d; edit, git commit --amend, git review | 14:03 |
| ameeda | hrw: thanks | 14:03 |
| mriedem | heh, mnaser check out https://review.openstack.org/#/c/525253/ being discussed; look familiar? | 14:04 |
| ameeda | mriedem: Hello :) | 14:05 |
| mnaser | mriedem: lol, well now that I’ve learned a whole lot, I can tell that shouldn’t work for scenarios involving new flow :p | 14:06 |
| mnaser | I don’t think it would be a fun time to replicate all the code used in the clean up section (but maybe it could be moved out to utils somewhere and reused) | 14:07 |
| mnaser | But imho transactionally It feels more correct that bdms for failed instances aren’t disappearing | 14:07 |
| mriedem | mnaser: yup, that's part of my -1 | 14:07 |
| mnaser | If the detach fails there should still be some sort of reference, right now there’s nothing | 14:07 |
| mnaser | mriedem: as much as I’d hate to rebase the series again, should we add a release note for operators that “hey, if you have a lot of failed scheduled instances, your bdm table might get bigger because they’re saved now” | 14:09 |
| *** jmlowe has quit IRC | 14:09 | |
| *** lucasagomes is now known as lucas-hungry | 14:09 | |
| mriedem | mnaser: those will be deleted when the failed instance is deleted | 14:10 |
| mnaser | mriedem: oh okay so thats not a big impact, i guess bdms are not soft deleted | 14:10 |
| mriedem | i doubt anyone would need to know that we start putting bdms in cell0 now | 14:10 |
| mriedem | bdms are soft deleted | 14:10 |
| mriedem | like instances and everything else (except tags) in the 'nova' table | 14:10 |
| mriedem | so you'd still have to archive/purge that stuff at some point | 14:10 |
| mnaser | gotcha | 14:10 |
| *** sree has joined #openstack-nova | 14:11 | |
| mnaser | anyways, whole series is ready to be reviewed (and i don't think i have a lot of review karma saved up for my lack of nova reviews to use :p) | 14:11 |
| mriedem | i've got +2s on the first 3 | 14:12 |
| mnaser | it's causing a ton of annoyance for some customers because they cant delete their volumes | 14:12 |
| mriedem | there is a force detach in cinder | 14:12 |
| mnaser | afaik thats an admin op | 14:12 |
| mriedem | yeah maybe | 14:13 |
| mriedem | yup it's an admin action by default os-force_detach | 14:13 |
| mriedem | http://git.openstack.org/cgit/openstack/cinder/tree/cinder/api/contrib/admin_actions.py#n203 | 14:14 |
| mnaser | anyways no worries, i know the first 3 are much more simple, the last one is the tough one but i'm ready to pick up any other work that comes on top so ping me if anything :> | 14:14 |
| *** janki|afk has quit IRC | 14:16 | |
| *** jmlowe has joined #openstack-nova | 14:18 | |
| *** bhujay has quit IRC | 14:19 | |
| *** bhujay has joined #openstack-nova | 14:19 | |
| *** yamahata has joined #openstack-nova | 14:21 | |
| *** rmcall has joined #openstack-nova | 14:21 | |
| *** acormier has joined #openstack-nova | 14:22 | |
| efried | melwitt: I have to duck out of the nova meeting around quarter after, FYI. On the hook for school runs again. | 14:22 |
| *** dave-mccowan has quit IRC | 14:23 | |
| *** jaosorior has quit IRC | 14:23 | |
| *** lbragstad has joined #openstack-nova | 14:29 | |
| *** ccamacho has quit IRC | 14:29 | |
| *** tbachman has joined #openstack-nova | 14:29 | |
| *** alexchadin has quit IRC | 14:29 | |
| *** Eran_Kuris has quit IRC | 14:31 | |
| *** awaugama has joined #openstack-nova | 14:31 | |
| *** acormier has quit IRC | 14:31 | |
| *** doude has quit IRC | 14:33 | |
| *** tbachman has quit IRC | 14:34 | |
| *** esberglu has quit IRC | 14:34 | |
| *** threestrands_ has quit IRC | 14:39 | |
| *** dtruong has quit IRC | 14:40 | |
| *** dtruong has joined #openstack-nova | 14:41 | |
| *** yamamoto has quit IRC | 14:41 | |
| *** tbachman has joined #openstack-nova | 14:41 | |
| *** amodi has joined #openstack-nova | 14:42 | |
| jaypipes | bauzas: would you mind pinging sahid to hop on #openstack-nova please? just for a few minutes. have a question for him.. | 14:44 |
| bauzas | sure, lemme try ;) | 14:44 |
| *** mlavalle has joined #openstack-nova | 14:44 | |
| jaypipes | merci | 14:44 |
| *** doude has joined #openstack-nova | 14:44 | |
| *** doude has quit IRC | 14:44 | |
| *** doude has joined #openstack-nova | 14:44 | |
| bauzas | mmm, he's not on our internal IRC, lemme verify if he has some PTO | 14:44 |
| *** ccamacho has joined #openstack-nova | 14:46 | |
| *** yamamoto has joined #openstack-nova | 14:48 | |
| jaypipes | bauzas: ok, no worries. | 14:48 |
| bauzas | jaypipes: well, I don't see any PTO on our agenda | 14:49 |
| jaypipes | bauzas: danpb isn't available is he? | 14:49 |
| *** dtantsur|brb is now known as dtantsur | 14:49 | |
| jaypipes | bauzas: specifically, I am looking to find out whether Dan's comment here: https://review.openstack.org/#/c/527631/9/nova/virt/libvirt/driver.py@a4299 (that was removed by tetsuro) is still valid. | 14:50 |
| bauzas | jaypipes: I can ask danpb to go here | 14:50 |
| jaypipes | bauzas: cool, cheers :) | 14:50 |
| bauzas | even if he's no longer working on nova | 14:50 |
| *** READ10 has joined #openstack-nova | 14:51 | |
| *** psachin has quit IRC | 14:51 | |
| stephenfin | jaypipes: kashyap is the person to ask about that | 14:52 |
| * kashyap blinks | 14:52 | |
| stephenfin | Far as I know, that comment is still valid. We've got support or emulator threads enabled but not IO threads | 14:52 |
| kashyap | And clicks on the URL | 14:53 |
| stephenfin | However, iirc, kashyap was in talks where the value of IO threads was called into question | 14:53 |
| *** yamamoto has quit IRC | 14:53 | |
| stephenfin | Well, IO threads > 1 anyway | 14:53 |
| kashyap | stephenfin: jaypipes: (I have a discussion for it (IO Threads at Dublin too) | 14:53 |
| kashyap | That said... | 14:53 |
| kashyap | Yeah, the IO Threads value is in contention | 14:55 |
| kashyap | Recently, I saw a presentation at KVM Forum where someone from oVirt claimed the "ideal" number of IO Threads is ...1! | 14:55 |
| kashyap | (In their benchmarks) | 14:55 |
| kashyap | But I won't believe it. "Seeing is believing" --> Need `fio` benchmarks for that | 14:56 |
| kashyap | jaypipes: Also see: https://review.openstack.org/#/c/230968/ "iothreads for disk devices" | 14:57 |
| jaypipes | kashyap: so, bottom line, that comment from danpb is still valid, yeah? | 14:59 |
| bauzas | jaypipes: when you have time, I'd also like to discuss abotu https://review.openstack.org/#/c/544683/1 | 14:59 |
| kashyap | jaypipes: Yes, it is still valid; it's better to retain that | 14:59 |
| kashyap | (I.e. I agree with your comment on Gerrit) | 14:59 |
| kashyap | Nova isn't yet using IO Threads. | 15:00 |
| * kashyap goes to comment there | 15:00 | |
| jaypipes | danke | 15:00 |
| jaypipes | kashyap: dan just responded. | 15:01 |
| *** rcernin has quit IRC | 15:01 | |
| kashyap | jaypipes: I think you're trying to speak Dutch, in that case: "Dank je" / "Heel erg bedant" :P | 15:02 |
| kashyap | (You wrote German) | 15:02 |
| kashyap | If that was intentional; disreregard me | 15:02 |
| openstackgerrit | Dan Smith proposed openstack/nova-specs master: Support member_of param for allocation candidates https://review.openstack.org/544694 | 15:03 |
| jaypipes | kashyap: oh, I wasn't trying to write Dutch... I just say danke all the time... | 15:04 |
| jaypipes | dansmith: why thank you dan | 15:04 |
| kashyap | True; I've noticed it before | 15:04 |
| dansmith | cha | 15:04 |
| mriedem | melwitt: mnaser: i've gone through the local delete https://review.openstack.org/#/c/340614/ change in detail, lots of questions and head scratching | 15:05 |
| jaypipes | stephenfin: I -W'd https://review.openstack.org/#/c/527630/ since the blueprint isn't yet approved.. | 15:05 |
| stephenfin | Oh, good catch. It is just cleanup though, right? | 15:06 |
| mriedem | 527630 is in the gate | 15:06 |
| efried | cdent, jaypipes: Can we talk about the "2001 providers" issue from https://review.openstack.org/#/c/540111/ ? | 15:06 |
| mriedem | you're going to have to rebase it to pull it out | 15:06 |
| mriedem | or change the commit message | 15:06 |
| mriedem | jaypipes: stephenfin: ^ | 15:06 |
| stephenfin | Do we want to? | 15:06 |
| cdent | efried: I can if you like but it will be partial attention, doing tc office hours then api-s | 15:07 |
| cdent | ig | 15:07 |
| *** tidwellr has joined #openstack-nova | 15:07 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Add request filter functionality to scheduler https://review.openstack.org/544730 | 15:07 |
| openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Add require_tenant_aggregate request filter https://review.openstack.org/545002 | 15:07 |
| stephenfin | jaypipes: I'll let you call that | 15:07 |
| jaypipes | mriedem: it's just a small refactor. if you want to keep it in, that's OK with me. but the series is associated with a blueprint that is not approved, as you know. | 15:07 |
| jaypipes | efried: sure. hangout or IRC? | 15:08 |
| mriedem | i haven't been tracking it, just saw you mention that | 15:08 |
| efried | cdent, jaypipes, edleafe: If y'all are okay with limiting based on the MISC_SHARES_VIA_AGGREGATE trait, at least for now, I'll make it so. If I'm the only one who's uncomfortable about that, and I can't give a good reason/counterexample, then I'm okay to let it ride. | 15:08 |
| jaypipes | mriedem: for some reason I thought -W would prevent it from gating... | 15:08 |
| dansmith | jaypipes: nay | 15:08 |
| efried | jaypipes, mriedem: A -2 might | 15:08 |
| jaypipes | ah | 15:09 |
| jaypipes | well I don't want to do that... | 15:09 |
| mriedem | no, the best thing with stuff that's not bp approved right now, is don't +2 it | 15:09 |
| *** lajoskatona has quit IRC | 15:09 | |
| dansmith | a -2 will, but it will reset the gate at the last moment | 15:09 |
| bauzas | dansmith: oh, saw https://review.openstack.org/#/c/544585/ | 15:09 |
| mriedem | +1 if you wanted to review it and said lgtm but the bp isn't approved yet | 15:09 |
| *** r-daneel has quit IRC | 15:09 | |
| *** hamzy_ has quit IRC | 15:10 | |
| dansmith | bauzas: came out of discussing some concerns with CERN | 15:10 |
| bauzas | okay, I need to review it carefully then | 15:10 |
| bauzas | I wasn't really paying attention to specs yet | 15:10 |
| jaypipes | stephenfin: I -W'd the following patch. | 15:11 |
| dansmith | bauzas: it's pretty simple, and you can see my prototype code which is probably easier to grok | 15:11 |
| jaypipes | stephenfin: and left the one small refactoring in the gate. | 15:11 |
| bauzas | dansmith: yeah, will look | 15:11 |
| stephenfin | jaypipes: Cool. I had concerns about that one anyway (though it's also a refactoring change) | 15:12 |
| jaypipes | efried: let me comment on the spec. | 15:12 |
| efried | jaypipes: ack, thx. I'd like to answer the question in the text before it's published. | 15:12 |
| jaypipes | ya | 15:13 |
| efried | edleafe: btw, we're talking about https://review.openstack.org/#/c/540111/4/specs/rocky/approved/update-provider-tree.rst@48 | 15:13 |
| *** dave-mccowan has joined #openstack-nova | 15:13 | |
| *** abhishekk has quit IRC | 15:14 | |
| dansmith | mriedem: you're going to let me know when I can un -W this right? https://review.openstack.org/#/c/543580/ | 15:16 |
| *** mdnadeem has quit IRC | 15:17 | |
| mriedem | preferably after rc2 | 15:18 |
| *** lucas-hungry is now known as lucasagomes | 15:18 | |
| jaypipes | efried: done | 15:18 |
| *** bhujay has quit IRC | 15:19 | |
| *** Nil_ has joined #openstack-nova | 15:22 | |
| *** jaosorior has joined #openstack-nova | 15:27 | |
| *** elmaciej has quit IRC | 15:28 | |
| edleafe | efried: was getting coffee. Makes sense to limit the tree by that trait. | 15:29 |
| mriedem | melwitt: i thought about this earlier today for some reason: if anyone, new contributor, stephen, whoever :) wants to start converting mox to mock in nova, they should definitely either way for cellsv1 and nova-net removal first, or be sure to stear clear from converting any tests that touch those code paths | 15:29 |
| mriedem | *wait | 15:29 |
| efried | edleafe: Rgr, thx | 15:29 |
| *** ralonsoh_ has joined #openstack-nova | 15:32 | |
| *** yamamoto has joined #openstack-nova | 15:33 | |
| bauzas | dansmith: question for you, why can't we leave Placement return all the hosts and only select the ones from that or this in a filter ? | 15:34 |
| bauzas | dansmith: you described that in https://review.openstack.org/#/c/544585/6/specs/rocky/approved/placement-req-filter.rst@115 IIRC but it's a bit confusing for me | 15:35 |
| dansmith | bauzas: because cern would have to process 9000 hosts that couldn't possibly work before they get to the first one that would | 15:35 |
| dansmith | bauzas: if you have a 10k node deployment with lots of space, and you boot a small instance, you get back a ridiculous number of hosts that you're not allowed to use and we have to run filters on all of them | 15:36 |
| bauzas | dansmith: it ties to a Placement performance question, right? | 15:36 |
| *** ralonsoh has quit IRC | 15:36 | |
| dansmith | no | 15:36 |
| dansmith | placement is fast at doing that, | 15:36 |
| dansmith | scheduler is slow at processing the result | 15:36 |
| bauzas | dansmith: because for that kind of purpose, we always say to use the more important filters first | 15:36 |
| mriedem | melwitt: i've started a technical debt / cleanup section in https://etherpad.openstack.org/p/nova-ptg-rocky L157 | 15:36 |
| dansmith | bauzas: at CERN-level scale, that is still very wasteful | 15:37 |
| bauzas | dansmith: since each filter is processed one after the other, depending on the ordering of the list | 15:37 |
| bauzas | dansmith: did they found some bottleneck ? | 15:37 |
| *** hamzy_ has joined #openstack-nova | 15:37 | |
| dansmith | bauzas: did you read the irc conversation? | 15:37 |
| bauzas | because the last benches I had from the scheduler, the filtering process itself was way fast | 15:37 |
| *** yamamoto has quit IRC | 15:38 | |
| bauzas | dansmith: from the cells meeting ? unfortunately no | 15:38 |
| dansmith | bauzas: we also only request 1000 hosts from placement, so if your hosts are after the first thousand, we wouldn't get back any that work for us | 15:38 |
| dansmith | bauzas: no, linked in the spec | 15:38 |
| mriedem | bauzas: cern ties tenants to cells, | 15:38 |
| mriedem | so top level cells v1 scheduler will pick the cell for the tenant, | 15:38 |
| mriedem | but with cells v2, we have a flat scheduler, | 15:39 |
| dansmith | bauzas: so if you have more than 1000 hosts that could fit your size flavor, but the first one you're allowed to use is after that, you could never schedule, or have to bump that limit higher | 15:39 |
| mriedem | so rather than a request from that tenant being just the computes in the chosen child cell, now it's all hosts from all cells, | 15:39 |
| mriedem | as dan pointed out | 15:39 |
| dansmith | and, any hard requirement we (nova) knows about that it can communicate to placement ahead of time will only improve performance, and ensure we get back a richer set of hosts to choose from | 15:39 |
| dansmith | and "can never be on hosts in aggregate X" is a pretty concrete requirement :) | 15:40 |
| bauzas | dansmith: mriedem: I understand that concern | 15:40 |
| bauzas | I just wonder how big it can become | 15:40 |
| mriedem | i like to also think of this as a way to eventually get rid of these other post-placement filters | 15:40 |
| dansmith | yeah | 15:40 |
| bauzas | the fact we said placement was efficient is that because it doesn't care of anything specific | 15:40 |
| dansmith | that is also nice | 15:40 |
| mriedem | we have already said don't use ram/disk/core filter b/c placement | 15:40 |
| mriedem | this would also remove the aggregate tenant isolation filter | 15:40 |
| dansmith | bauzas: placement already does aggregates, this is well within its scope | 15:41 |
| mriedem | bauzas: i'm not aware of anyone that has done benchmarking before and after placement at scale | 15:41 |
| mriedem | to say that we're way better now | 15:41 |
| dansmith | bauzas: performance aside, you get the point about the request limit right? | 15:41 |
| *** hamzy_ has quit IRC | 15:41 | |
| bauzas | mriedem: no, the benches were pre-placement, but saw the filtering process very fast | 15:41 |
| bauzas | dansmith: sure | 15:41 |
| dansmith | if we'reonly getting back 1000 hosts, then we could get back none | 15:41 |
| openstackgerrit | Claudiu Belu proposed openstack/nova master: compute: Cleans up allocations after failed resize https://review.openstack.org/543971 | 15:42 |
| bauzas | well, I don't know what to say | 15:42 |
| *** lpetrut has quit IRC | 15:42 | |
| dansmith | bauzas: performance aside, we want to get back a list of hosts that are candidates, and hosts that are in aggregates we're not allowed to be on are not candidates, so they're just busy work and they defeat the point of getting a solid fast set of canidates from placement based on things it knows about | 15:42 |
| *** dave-mccowan has quit IRC | 15:43 | |
| bauzas | dansmith: sure | 15:43 |
| bauzas | anyway, here is my take | 15:43 |
| bauzas | I'm not opposed at that | 15:43 |
| bauzas | I just need to consider all the implications | 15:44 |
| bauzas | because filters are like the nova things the most tweaked | 15:44 |
| bauzas | by ops | 15:44 |
| mriedem | dansmith: i'm assuming the pre-placement filters would also be pluggable? | 15:44 |
| dansmith | mriedem: modular, like our scheduler filters, not necessarily pluggable | 15:45 |
| *** acormier has joined #openstack-nova | 15:45 | |
| bauzas | if we're about to introduce a new hook for cheating placement to only return the subset we want, chances are that in the future ops will come with solid use-cases | 15:45 |
| dansmith | bauzas: cheating placement? I can't understand how this is that :) | 15:45 |
| bauzas | and at the end of the day, we'd just introduce more and more complexity to placement | 15:45 |
| dansmith | this doesn't change placement at all | 15:45 |
| dansmith | not a single bit | 15:45 |
| mriedem | these pre-placement filters are restricted to what placement api supports | 15:45 |
| bauzas | dansmith: for your proposal, I agree | 15:45 |
| mriedem | which is resource classes, traits and aggregates | 15:45 |
| dansmith | they operate on request spec | 15:46 |
| bauzas | dansmith: again, I'm not opposed to *your* use case which is tenant filtering | 15:46 |
| mriedem | i also have a use case for this | 15:46 |
| mriedem | if you care | 15:46 |
| bauzas | that said, we could do that without using aggregateds | 15:46 |
| bauzas | I guess | 15:46 |
| bauzas | anyway | 15:46 |
| bauzas | finding use-cases is not the thing I'm worried | 15:47 |
| bauzas | I'm more worried by how big it can become | 15:47 |
| mriedem | we knew we wanted to move filters in sql via placement as much as possible since the beginning right? | 15:47 |
| dansmith | I think I need a more concrete argument than that to understand the concern :) | 15:47 |
| bauzas | dansmith: hence why I'm discussing over IRC | 15:48 |
| bauzas | FUD isn't a valid argument and I respect that | 15:48 |
| dansmith | IMHO, this is the natural evolution of us asking placement for hosts based on resources (which it deals with) and then traits (which it deals with) and now aggregates (which it deals with) | 15:48 |
| bauzas | I just need to understand the context | 15:48 |
| dansmith | bauzas: why don't you want to hear mriedem's use case? | 15:49 |
| bauzas | dansmith: I'm open to his use-case | 15:49 |
| dansmith | bauzas: I also had a BFV use case in the first rev of the spec, | 15:49 |
| dansmith | but removed it because it was distracting to some people | 15:50 |
| bauzas | the limitation thing is certainly a valid argument | 15:50 |
| *** felipemonteiro has joined #openstack-nova | 15:50 | |
| bauzas | because we need to get all the corresponding hosts from placement | 15:50 |
| *** hamzy has joined #openstack-nova | 15:50 | |
| bauzas | if we say we won't allow pluggability, I think I'd be okay | 15:50 |
| dansmith | personally I don't want this to be pluggable, just modular | 15:50 |
| dansmith | (brb) | 15:51 |
| bauzas | then okay | 15:51 |
| bauzas | if we're all clear that we will never open that to ops | 15:51 |
| openstackgerrit | Eric Fried proposed openstack/nova-specs master: Update Provider Tree https://review.openstack.org/540111 | 15:51 |
| mriedem | bauzas: what i want to use this for is making sure we can pick a host that supports multiattach volumes for bfv, and tagged attach during bfv | 15:51 |
| efried | edleafe, jaypipes, cdent: There's that reword ^ | 15:51 |
| mriedem | bauzas: because today that's just luck of the host and we don't reschedule on failed volume attaching during bfv | 15:51 |
| bauzas | mriedem: you could achieve that with a filter, right? | 15:51 |
| *** felipemonteiro_ has joined #openstack-nova | 15:51 | |
| bauzas | atm, we have two placing mechanisms | 15:52 |
| bauzas | #1 is placement | 15:52 |
| bauzas | #2 is filters | 15:52 |
| mriedem | do we have a filter that actually has the driver.capabilities information exposed to it? | 15:52 |
| bauzas | I agree with both of you to try to make #1 prioritary | 15:52 |
| cdent | efried: good, thanks | 15:52 |
| *** moshele has quit IRC | 15:52 | |
| efried | bauzas: IMO we should do any filtering that's possible/practical to do in placement, in placement. Cause that'll be more efficient. | 15:52 |
| bauzas | mriedem: we have one caring about the compute capabilities themselves | 15:52 |
| *** sridharg has quit IRC | 15:53 | |
| mriedem | bauzas: but i'm not sure if that filter actually gets that data | 15:53 |
| bauzas | efried: yeah, hence my | 15:53 |
| bauzas | (16:52:33) bauzas: I agree with both of you to try to make #1 prioritary | 15:53 |
| efried | cool | 15:53 |
| bauzas | okay, looks like I'm settling down | 15:53 |
| mriedem | i have a hard time even grokking the code in the compute capabilities filter | 15:53 |
| bauzas | #1 scheduling (placement) shouldn't never be open to operators | 15:53 |
| bauzas | #2 scheduling (filters) would eventually only be custom | 15:54 |
| bauzas | if we could | 15:54 |
| bauzas | looks like a deal then | 15:54 |
| mriedem | the ComputeCapabilitiesFilter works on the HostState.stats right? | 15:54 |
| *** felipemonteiro has quit IRC | 15:55 | |
| mriedem | anyway, it would be easy to make one of these pre-placement filters that sees, 'oh you're doing bfv with a multiattach volume, let me add required trait CUSTOM_COMPUTE_SUPPORTS_MULTIATTACH to the request to placement for filtering' | 15:56 |
| efried | bauzas: Not sure I agree that ops don't affect placement-level scheduling. They do so via traits etc. | 15:56 |
| efried | mriedem: ++, assuming the compute got tagged with that trait somehow | 15:56 |
| jaypipes | efried: +2 | 15:56 |
| mriedem | efried: that's https://review.openstack.org/#/c/538498/ | 15:56 |
| mriedem | http://lists.openstack.org/pipermail/openstack-dev/2018-January/126653.html | 15:56 |
| mnaser | mriedem: left some comments about the change .. to the best of my knowledge | 15:56 |
| bauzas | efried: no, I meant custom code modifying behaviour of placement | 15:57 |
| *** r-daneel has joined #openstack-nova | 15:58 | |
| bauzas | mriedem: IIRC, the ComputeCapabilitiesFilter matches any HostState field | 15:58 |
| bauzas | that's magic | 15:58 |
| cdent | yes, definitely custom code should not happen to placement | 15:58 |
| dansmith | cdent: that's not what he's saying here htough | 15:58 |
| bauzas | dansmith: so, modularity is good, provided we don't open the entry points | 15:59 |
| dansmith | cdent: nobody is talking about touching placement at all, he's talking about in this code that we use to query placement | 15:59 |
| bauzas | dansmith: a big fat comment in the code would be fine for me | 15:59 |
| dansmith | bauzas: we don't comment code that isn't pluggable so I don't see why this would be any different :) | 15:59 |
| bauzas | yeah, what dansmith said | 15:59 |
| cdent | ah, sorry, I misinterpreted "I meant custom code modifying behaviour of placement" | 15:59 |
| dansmith | bauzas: I will have to say, that scheduler filters and these pre-filters would be the _best_ place to introduce custom behavior if I was an operator | 16:00 |
| bauzas | dansmith: you surely imagine the creativity of people using our code :) | 16:00 |
| cdent | presumably the only way to modify placement is by sending different queries and creating various entities. in which case it's not "custom" is it? | 16:00 |
| dansmith | because placement is a solid interface and all this is doing is augmenting how we call it | 16:00 |
| cdent | it's what it's for | 16:00 |
| dansmith | cdent: exactly my point | 16:00 |
| bauzas | the scheduler base filter is an external interface | 16:00 |
| cdent | yay, agreeing with dan is fun | 16:01 |
| bauzas | what would be the base pre-filter isn't an external interface and I'd vote -2 on any change that would open that | 16:01 |
| dansmith | bauzas: why do you think the scheduler filters are an external interface? | 16:01 |
| bauzas | dansmith: because it's pluggable :) | 16:02 |
| bauzas | shit, meeting | 16:02 |
| dansmith | bauzas: we don't load them externally anymore right? | 16:02 |
| bauzas | oh hell, we still do that alot | 16:02 |
| bauzas | unless I missed the bus | 16:02 |
| bauzas | and cutting that flexibility would be a huge thing | 16:03 |
| dansmith | where is the loading code for that? | 16:03 |
| bauzas | sec | 16:03 |
| dansmith | either way, the spec says I'm not even adding a list of pre-filters, but making each one a semantic config toggle | 16:04 |
| dansmith | so it should be pretty clear we're not adding a new external interface here :) | 16:04 |
| bauzas | dansmith: https://docs.openstack.org/nova/latest/user/filter-scheduler.html?#writing-your-own-filter | 16:04 |
| dansmith | ah, I thought we removed the classloading from available_filters | 16:05 |
| *** ralonsoh_ has quit IRC | 16:05 | |
| bauzas | no, we removed the possibility to hook a custom scheduler *driver* | 16:05 |
| dansmith | yeah I remember that | 16:06 |
| mriedem | no we didn't | 16:06 |
| *** ralonsoh_ has joined #openstack-nova | 16:06 | |
| mriedem | that was reverted | 16:06 |
| bauzas | okay, I thought we never agreed on removing the custom filter | 16:06 |
| dansmith | well, anyway, this is not that :) | 16:06 |
| bauzas | mriedem: orly? | 16:06 |
| mriedem | yes you were on the reno for it | 16:06 |
| bauzas | man, did I slept 100 years like the Sleeping Beauty ? | 16:06 |
| bauzas | mriedem: on reverting the removal of the classloading ? | 16:07 |
| bauzas | if so, I apologize for my memory lack | 16:07 |
| dansmith | bauzas: I think he's talking about the driver pluggability | 16:07 |
| mriedem | you can't classload the driver, but you can add an entry point for the scheduler driver in setup.cfg | 16:07 |
| bauzas | righrt | 16:08 |
| mriedem | https://github.com/openstack/nova/commit/1e5c7b52a403e708dba5a069dd86b628a4cb952c | 16:08 |
| *** tbachman has quit IRC | 16:08 | |
| bauzas | seriously, I'm sometimes afraid of my memory lacks :/ | 16:09 |
| bauzas | I totally forgot that story | 16:09 |
| bauzas | apologies for that | 16:09 |
| *** pcaruana has quit IRC | 16:09 | |
| bauzas | it's 6 months ago and I don't recall it | 16:10 |
| bauzas | creepy | 16:10 |
| dansmith | so, um, | 16:10 |
| dansmith | are we okay on this non-pluggable modular pre-filter thing or what? | 16:11 |
| cdent | bauzas: too much chaud-verte (or was it verte-chaud or whatever)? | 16:11 |
| *** andreas_s has quit IRC | 16:11 | |
| bauzas | cdent: no, Chartreuse | 16:11 |
| bauzas | dansmith: oui | 16:11 |
| bauzas | dansmith: I'm currently re-reviewing your spec | 16:11 |
| *** andreas_s has joined #openstack-nova | 16:11 | |
| *** tbachman has joined #openstack-nova | 16:12 | |
| *** damien_r1 has joined #openstack-nova | 16:17 | |
| *** yamamoto has joined #openstack-nova | 16:18 | |
| *** READ10 has quit IRC | 16:18 | |
| *** damien_r has quit IRC | 16:18 | |
| bauzas | cdent: edleafe: I mostly see your concerns by having a spec approved in 1 day, and pre-PTG | 16:19 |
| bauzas | cdent: edleafe: please note that I don't feel it's a problem | 16:19 |
| *** slaweq_ has joined #openstack-nova | 16:19 | |
| bauzas | we can approve a spec and amend it later based on feedback that can come from the PTG or elsewhere | 16:19 |
| edleafe | bauzas: the concern was that a series of related specs came out all at once | 16:19 |
| bauzas | or we can at the end of the day have a spec that is approved but leading to a dead-end, that's not a problem to me | 16:20 |
| edleafe | At first I thought I missed the discussions about them | 16:20 |
| dansmith | I don't think the concern is over PTG discussion, | 16:20 |
| dansmith | we can't tie spec approvals to ptg discussion globally.. not everyone can or will go, nor will we have enough time for that | 16:20 |
| edleafe | dansmith: agreed. | 16:20 |
| bauzas | edleafe: the relationship between specs is surely a thing to consider, but which shouldn't hold an approval if that's not blocking (heh, tautology) | 16:20 |
| *** andreas_s has quit IRC | 16:21 | |
| bauzas | dansmith: sure, hence my "or elsewhere" | 16:21 |
| *** slaweq has quit IRC | 16:21 | |
| *** tbachman has quit IRC | 16:21 | |
| edleafe | bauzas: it was because there were a few unstated assumptions, such as the idea that we would be syncing nova aggs to placement aggs, that were surprising | 16:21 |
| dansmith | edleafe: I dunno why that is surprising, I feel like we've covered that multiple times | 16:22 |
| *** links has quit IRC | 16:22 | |
| *** slaweq has joined #openstack-nova | 16:22 | |
| dansmith | but that said, I'm totally cool with letting the ink dry a bit on things before they go in so all the timezones have a chance to comment | 16:22 |
| edleafe | dansmith: it's possible, but I must have missed those discussions | 16:22 |
| dansmith | I actually thought we already were doing that syncing, | 16:22 |
| dansmith | because of the method for doing so in the scheduler client, which is apparently unused as of yet | 16:22 |
| edleafe | dansmith: I still repeated the mantra "placement aggs are not nova aggs" | 16:22 |
| dansmith | ...which is still true :) | 16:23 |
| *** yamamoto has quit IRC | 16:23 | |
| *** slaweq_ has quit IRC | 16:24 | |
| edleafe | now it's "placement aggs *are* nova aggs, and then some" | 16:24 |
| edleafe | :) | 16:24 |
| mriedem | fwiw i never thought we were syncing nova aggs to placement aggs, or intended to do so, before yesterday | 16:24 |
| mriedem | edleafe: but they aren't | 16:24 |
| mriedem | placement aggs don't have metadata | 16:25 |
| mriedem | they don't have to exist for anyone outside of nova | 16:25 |
| *** tbachman has joined #openstack-nova | 16:25 | |
| edleafe | mriedem: I don't mean they are exactly the same | 16:25 |
| mriedem | i believe jay said they'd be a superset of nova host aggregates, which i think is correct | 16:25 |
| edleafe | mriedem: just that if a compute node is in a nova agg, it will also be a matching placement agg | 16:25 |
| dansmith | right, a superset | 16:25 |
| *** tbachman has quit IRC | 16:25 | |
| mriedem | because my compute node providers could be in an aggregate that mirror a nova host aggregate, and also in a shared storage provider aggregate with cinder | 16:25 |
| *** andreas_s has joined #openstack-nova | 16:26 | |
| dansmith | aggregates are a thing that placement provides for consumers to group resource providers, | 16:26 |
| bauzas | mriedem: edleafe: if we only keep placement aggregates as a bag of a collection of hosts, I'm fine with sync'ing that to nova | 16:26 |
| edleafe | again, I don't think that this is a bad approach | 16:26 |
| edleafe | it was just a big surprise | 16:26 |
| *** slaweq has quit IRC | 16:26 | |
| dansmith | nova is a user of placement and thus would use it to augment its grouping when placing instances, | 16:26 |
| bauzas | what I'm not okay is if placement aggs begin to have metadata information that carries some superseding logic | 16:26 |
| dansmith | as I would expect neutron to create aggregates to mirror is grouping constructs when it helps to place things, cinder the same | 16:26 |
| bauzas | yeah that | 16:27 |
| *** chyka has joined #openstack-nova | 16:27 | |
| mriedem | bauzas: no one said they would, and i'm sure jaypipes' head would explode | 16:27 |
| mriedem | if we said that | 16:27 |
| bauzas | mriedem: just stating loud things | 16:27 |
| mriedem | maybe i should update https://review.openstack.org/#/c/539033/ | 16:27 |
| *** lpetrut has joined #openstack-nova | 16:27 | |
| *** suresh12 has joined #openstack-nova | 16:28 | |
| bauzas | mriedem: please, CC'ing that change | 16:28 |
| bauzas | since my brain is untrustable, I need to star things | 16:28 |
| *** suresh12 has quit IRC | 16:28 | |
| *** suresh12 has joined #openstack-nova | 16:28 | |
| *** jaosorior has quit IRC | 16:29 | |
| *** hongbin has joined #openstack-nova | 16:30 | |
| mriedem | edleafe: ^ replied in that change, see what you think | 16:31 |
| mriedem | the point of that patch is to provide an example, | 16:31 |
| * jaypipes reads back after escorting the exterminator through a field of feral pugs. | 16:31 | |
| mriedem | since mgagne correctly pointed out that the api-ref today only talks about what placement aggregates "aren't", not an example of what they "are" | 16:31 |
| hrw | my next patch will look weird for some. There will be comments added to explain why that way | 16:32 |
| bauzas | jaypipes: don't use Bravecto | 16:33 |
| bauzas | on your pugs | 16:33 |
| openstackgerrit | Merged openstack/nova master: [libvirt] Add _get_vcpu_realtime_scheduler() https://review.openstack.org/527630 | 16:33 |
| hrw | does virt/libvirt/driver.py have access to config entries from nova.conf? | 16:34 |
| bauzas | hrw: of course | 16:34 |
| *** ccamacho has quit IRC | 16:34 | |
| hrw | pcie.numberofslots (int8) will be new option | 16:34 |
| jaypipes | hrw: yes. via the global CONF variable. | 16:35 |
| hrw | or rather libvirt.pcie.numberofslots or sth like that | 16:35 |
| jaypipes | hrw: python doesn't have an int8. :) you're Golanging. | 16:35 |
| hrw | so admin can set how many 'slots' VM will have | 16:36 |
| hrw | jaypipes: int5 even ;D and I know that python is typeless | 16:36 |
| jaypipes | hrw: it's int5 today. I'm sure a certain hardware manufacturer will change that at some point in the future... | 16:37 |
| *** AlexeyAbashkin has quit IRC | 16:37 | |
| *** imacdonn has quit IRC | 16:37 | |
| *** imacdonn has joined #openstack-nova | 16:37 | |
| *** READ10 has joined #openstack-nova | 16:38 | |
| hrw | jaypipes: have nothing against it in real hw | 16:38 |
| hrw | jaypipes: there are cpus with >32 pcie lines so it is (in theory) possible | 16:38 |
| jaypipes | hrw: yup. | 16:38 |
| *** Tom-Tom has quit IRC | 16:38 | |
| hrw | jaypipes: I remember time when my arm64 machine boot was ~13 minutes due to pci bus scanning | 16:39 |
| mriedem | mnaser: you magnificent bastard, you clarified my big sticking point in melwitt's patch, | 16:40 |
| mriedem | the main thing is that when the instance is in cell0, instance.host is None but the vm_state is ERROR, so we don't enter that first block of code because "if not has_been_scheduled" is False | 16:40 |
| jaypipes | hrw: I remember a time when we weren't coding hardware-defined software ;) | 16:40 |
| mriedem | and the context is targeted at cell0 because the api code does that when we pull the instance out of cell0 | 16:40 |
| mriedem | so we can get bdms using that context too | 16:40 |
| mnaser | mriedem: i won't take credit, i had the same train of thought, and melwitt corrected me :p | 16:41 |
| mriedem | the has_been_scheduled variable name is confusing i think | 16:41 |
| mriedem | but i don't have a better alternative | 16:41 |
| hrw | jaypipes: I am not getting young as well ;D | 16:42 |
| mriedem | has_been_scheduled_or_should_be_in_cell0_or_idk_whatever | 16:42 |
| *** udesale has quit IRC | 16:44 | |
| *** andreas_s has quit IRC | 16:44 | |
| *** tbachman has joined #openstack-nova | 16:46 | |
| hrw | https://paste.fedoraproject.org/paste/A9Gtb~sliIFo2-yvA32ymw - preliminary version | 16:48 |
| bauzas | dansmith: I left some comments but only one is blocking me from +2 on https://review.openstack.org/#/c/544585/6 | 16:50 |
| bauzas | dansmith: the fact that I wonder if we should persist the changes in the RequestSpec or not | 16:51 |
| dansmith | bauzas: why wouldn't we? | 16:51 |
| bauzas | maybe it's just an implementation detail, but we've seen in the past some problems with the RequestSpec fields that were badly designed whether they should be persisted or not | 16:51 |
| mriedem | like instance groups? | 16:52 |
| bauzas | dansmith: for the usecase you described, I agree | 16:52 |
| dansmith | bauzas: we'll re-run the pre filters on reschedule, so persisted data could be kept, re-evaluated, updated, etc | 16:52 |
| bauzas | dansmith: but since we'll define a modular mechanism, we could end up having cases where it's necessary and some not | 16:52 |
| bauzas | dansmith: that answer suits me | 16:52 |
| *** slaweq has joined #openstack-nova | 16:52 | |
| dansmith | yeah, each one could do a different thing with the persisted data | 16:53 |
| mriedem | yeah we'd definitely need this for any move operations | 16:53 |
| bauzas | dansmith: we could persist that, but we shouldn't use that value on a later reschedule but rather recalculate it | 16:53 |
| bauzas | that being a per-filter policy | 16:53 |
| dansmith | bauzas: depends | 16:53 |
| dansmith | yeah | 16:53 |
| dansmith | I will reply | 16:53 |
| bauzas | cool | 16:53 |
| bauzas | one last point | 16:53 |
| bauzas | but it's a nit | 16:53 |
| bauzas | I'm poor at wording | 16:53 |
| bauzas | but I'm afraid the word "filter" carries a lot of context | 16:54 |
| bauzas | so I'd prefer to use a synonym for that mechanism | 16:54 |
| bauzas | I know it's sneaky | 16:54 |
| bauzas | so feel free to NACK me | 16:54 |
| dansmith | bauzas: it's not exposed as a filter to the user or admin at all | 16:54 |
| dansmith | bauzas: so we can change it either now or later if it becomes a problem | 16:55 |
| bauzas | yeah it's just wording | 16:55 |
| dansmith | but, IMHO, it's not a big enough deal to rename the things I have already :) | 16:55 |
| bauzas | like we have nova aggs and placement aggs | 16:55 |
| bauzas | dansmith: not requiring a respin of your spec | 16:55 |
| bauzas | dansmith: but maybe we should consider that in your series :) | 16:55 |
| dansmith | bauzas: well feel free to take exception to it in the code if you still feel that way yeah | 16:56 |
| bauzas | scheduler filters and request filters seem very identical to me in terms | 16:56 |
| bauzas | again, it's maybe me overreacting on some blueprints I could see in the future | 16:56 |
| *** tidwellr has quit IRC | 16:56 | |
| dansmith | what word do you like better? | 16:57 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova master: Allow to configure amount of PCIe slots in aarch64 instance https://review.openstack.org/545034 | 16:57 |
| *** slaweq has quit IRC | 16:57 | |
| *** tidwellr has joined #openstack-nova | 16:57 | |
| hrw | cna someone take a look at that patch? mriedem stephenfin bauzas? | 16:57 |
| bauzas | dansmith: from a French folk, the Thesaurus tells me "strain" | 16:57 |
| dansmith | lol | 16:57 |
| bauzas | but anyway, I feel ashamed of nitpicking on that | 16:57 |
| dansmith | that would make no sense to an english speaker | 16:58 |
| bauzas | the history of nova told me that there are 3 NP problems | 16:58 |
| bauzas | distibuted locks | 16:58 |
| bauzas | NFV | 16:58 |
| bauzas | or BFV even | 16:58 |
| bauzas | and naming things | 16:58 |
| bauzas | naming a new thing requires 3 cycles at least | 16:59 |
| bauzas | in order to get consensus | 16:59 |
| bauzas | so, anyway, let's just pretend I never asked for that | 16:59 |
| dansmith | bauzas: as the only one who has (thus far) taken exception with the name, it's funny for you to be complaining about that :) | 16:59 |
| bauzas | I know, hence my apologies | 17:00 |
| bauzas | let's pretend that discussion never existed | 17:00 |
| bauzas | (17:57:55) bauzas: but anyway, I feel ashamed of nitpicking on that | 17:00 |
| *** sree has quit IRC | 17:00 | |
| *** sree has joined #openstack-nova | 17:01 | |
| mriedem | hrw: busy atm, can you mark that WIP if it's the initial version w/o tests? | 17:02 |
| hrw | mriedem: ok | 17:02 |
| hrw | W-1 | 17:02 |
| *** yamamoto has joined #openstack-nova | 17:03 | |
| openstackgerrit | Merged openstack/nova master: Don't JSON encode instance_info.traits for ironic https://review.openstack.org/543574 | 17:04 |
| openstackgerrit | Merged openstack/nova master: Python 3 fix for sphinx doc https://review.openstack.org/544956 | 17:04 |
| mriedem | ameeda: i think you're missing a unit test in https://review.openstack.org/#/c/528385/ | 17:05 |
| *** Matias_ has joined #openstack-nova | 17:06 | |
| *** sree has quit IRC | 17:06 | |
| mriedem | otherwise i think that's mostly ok, outside of the commit message cleanup | 17:06 |
| openstackgerrit | Mark Goddard proposed openstack/nova stable/queens: Don't JSON encode instance_info.traits for ironic https://review.openstack.org/545037 | 17:06 |
| *** Matias_ is now known as Matias | 17:07 | |
| *** yamamoto has quit IRC | 17:08 | |
| *** Tom-Tom has joined #openstack-nova | 17:09 | |
| *** Tom-Tom has quit IRC | 17:14 | |
| *** sshwarts has quit IRC | 17:14 | |
| *** abhishekk has joined #openstack-nova | 17:15 | |
| *** lpetrut has quit IRC | 17:18 | |
| *** sambetts is now known as sambetts|afk | 17:19 | |
| *** harlowja has joined #openstack-nova | 17:19 | |
| *** pcaruana has joined #openstack-nova | 17:20 | |
| *** yamahata has quit IRC | 17:20 | |
| mriedem | stephenfin: i don't know what people want in here https://review.openstack.org/#/c/544015/ | 17:22 |
| *** suresh12 has quit IRC | 17:22 | |
| bauzas | mriedem: dansmith: edleafe: cdent: I'm about to +W https://review.openstack.org/#/c/544585/ | 17:22 |
| bauzas | if anyone has concerns, that's the moment | 17:22 |
| bauzas | oops, jaypipes too ^ | 17:22 |
| *** suresh12 has joined #openstack-nova | 17:23 | |
| dansmith | my only concern would be the dependent specs, if people want those landed first | 17:23 |
| dansmith | but I don't mind either way really | 17:24 |
| stephenfin | mriedem: I think you identified what he was looking for, which was different to what I thought. Given that the caching scheduler is deprecated (I didn't know that), we don't need to worry about IMO | 17:24 |
| *** Tom-Tom has joined #openstack-nova | 17:25 | |
| mriedem | bauzas: it is certainly possible to know if we're doing a first schedule or a reschedule, | 17:25 |
| mriedem | RequestSpec.retry has that information | 17:25 |
| bauzas | mriedem: for a reschedule yes | 17:26 |
| bauzas | mriedem: for a move operation, nope | 17:26 |
| edleafe | bauzas: that's cool. At least there has been some discussion since my comment | 17:26 |
| mriedem | bauzas: that's because https://review.openstack.org/#/c/505771/ | 17:26 |
| *** damien_r1 has quit IRC | 17:26 | |
| *** stephenfin has left #openstack-nova | 17:26 | |
| mriedem | well, the issue in ^ confuses the fact that you're doing a reschedule or a move | 17:26 |
| *** stephenfin has joined #openstack-nova | 17:26 | |
| bauzas | shit | 17:26 |
| *** suresh12 has quit IRC | 17:27 | |
| *** belmoreira has quit IRC | 17:27 | |
| bauzas | how many times will I regret to not have thought more on persisted fields for the RequestSpec object ? | 17:27 |
| openstackgerrit | Claudiu Belu proposed openstack/nova master: compute: Cleans up allocations after failed resize https://review.openstack.org/543971 | 17:27 |
| jaypipes | bauzas: hold please. | 17:27 |
| cdent | bauzas: feel free to +w the spec if there's been enough feedback | 17:27 |
| jaypipes | still reviewing | 17:28 |
| mriedem | i thought that would be an easy fix but then second guessed myself, and need to write a functional test to be sure | 17:28 |
| cdent | bauzas: except for jay :) | 17:28 |
| bauzas | jaypipes: ack | 17:28 |
| efried | Could rebase it on top of the other spec | 17:29 |
| *** cdent has quit IRC | 17:30 | |
| dansmith | or someone could slam it in: https://review.openstack.org/#/c/544694/ | 17:31 |
| dansmith | it's really simple, has a lot of +1s from stakeholders and only needs a +W | 17:31 |
| mriedem | looking | 17:31 |
| mriedem | i will ram this down your throat like obamacare to the republicans | 17:32 |
| * dansmith nods | 17:32 | |
| mriedem | lemme call nancy quick | 17:32 |
| dansmith | lolol | 17:32 |
| *** yamamoto has joined #openstack-nova | 17:32 | |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: mirror nova host aggregates to placement API https://review.openstack.org/545057 | 17:33 |
| *** yamamoto has quit IRC | 17:33 | |
| dansmith | apparently by "lots" I meant "two" but.. you know | 17:34 |
| *** lpetrut has joined #openstack-nova | 17:34 | |
| mriedem | consider it rammed | 17:34 |
| mriedem | thanks obama | 17:35 |
| mriedem | the two that matter | 17:35 |
| jaypipes | dansmith: cern doesn't use the cachingscheduler does it? | 17:38 |
| mriedem | tssurya: ^ | 17:39 |
| dansmith | jaypipes: presumably not because they're working through placement issues right now | 17:39 |
| jaypipes | ak | 17:39 |
| openstackgerrit | Merged openstack/nova-specs master: Support member_of param for allocation candidates https://review.openstack.org/544694 | 17:40 |
| *** moshele has joined #openstack-nova | 17:41 | |
| *** yamamoto has joined #openstack-nova | 17:41 | |
| tssurya | mriedem, jaypipes, dansmith : no we don't | 17:42 |
| mriedem | excellente | 17:42 |
| *** moshele has quit IRC | 17:44 | |
| *** yamamoto has quit IRC | 17:46 | |
| *** amodi has quit IRC | 17:47 | |
| gibi | dansmith: I also left some comments / questions on https://review.openstack.org/#/c/544585 just now | 17:47 |
| *** itlinux has joined #openstack-nova | 17:47 | |
| openstackgerrit | Surya Seetharaman proposed openstack/nova master: Extending delete_cell --force to delete instance_mappings https://review.openstack.org/540073 | 17:51 |
| *** tesseract has quit IRC | 17:52 | |
| bauzas | gibi: dansmith: I got the name. Let's call it "scheduler transformers" and name the first one "autobot" | 17:52 |
| dansmith | gibi: replying | 17:53 |
| mriedem | gibi: i cleaned up the commit message on this and added a simple unit test for detach failing, see if you still like it ^ | 17:53 |
| mriedem | oops | 17:53 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Detach volumes when VM creation fails https://review.openstack.org/528385 | 17:53 |
| mriedem | gibi: ^ | 17:53 |
| bauzas | please folks, don't tell me about Shia LaBeouf or anything stupid like the movies | 17:53 |
| gibi | mriedem: looking | 17:53 |
| bauzas | they killed my childhood | 17:53 |
| *** suresh12 has joined #openstack-nova | 17:54 | |
| *** yamahata has joined #openstack-nova | 17:58 | |
| *** suresh12 has quit IRC | 17:58 | |
| *** amodi has joined #openstack-nova | 17:58 | |
| mriedem | gibi: and left a comment | 17:58 |
| *** tssurya has quit IRC | 17:59 | |
| *** derekh has quit IRC | 17:59 | |
| mriedem | deleting the volume before we reschedule is kind of dumb | 17:59 |
| mriedem | let's say i boot from volume with a pre-existing volume, | 17:59 |
| gibi | mriedem: oops, +Wd it | 17:59 |
| mriedem | spawn on the first host fails after i've attached the instance to the volume, | 17:59 |
| mriedem | we then detach and delete the volume, and reschedule, | 17:59 |
| mriedem | then we get to the 2nd host, and attach will fail b/c the volume is gone | 17:59 |
| mriedem | well i was +2, | 18:00 |
| mriedem | and might still be, | 18:00 |
| mriedem | because this is a latent thing that's always been in here, but i just don't think it's ever worked because of the bug he's fixing | 18:00 |
| gibi | mriedem: I see. I guess this is a new bug report then | 18:00 |
| *** AlexeyAbashkin has joined #openstack-nova | 18:01 | |
| gibi | mriedem: we fixed something that allowed another latent bug to manifest itself | 18:02 |
| mriedem | at least delete_on_termination defaults to False... | 18:03 |
| mriedem | this is also kind of why i didn't like the spec proposed for doing delete_on_termination with ports | 18:03 |
| mriedem | since nova is not good at orchestrating this | 18:03 |
| *** mgoddard_ has quit IRC | 18:04 | |
| gibi | I also think that nova should only delete these if nova was the one that created them in the first place. But it would be even better if nova would never create these things | 18:04 |
| mriedem | yeah that's the exact thing i was thinking | 18:04 |
| *** dtantsur is now known as dtantsur|afk | 18:04 | |
| mriedem | and we handle that with ports by storing a flag if nova created the port or not | 18:04 |
| mriedem | in the nw info cache | 18:05 |
| mriedem | because if nova did create this volume and attach it, then we fail and reschedule but don't delete it, we'll create another volume on the next host we try | 18:05 |
| mriedem | and never cleanup the old one | 18:05 |
| mriedem | and given enough retries, you could also hit quota limits on creating volumes | 18:06 |
| *** AlexeyAbashkin has quit IRC | 18:06 | |
| mriedem | oh, well we do know if nova created it actually, | 18:06 |
| mriedem | if source_type=volume, it's pre-created | 18:06 |
| mriedem | ok i left some notes in the review so we can remember this conversation 2 years from now | 18:08 |
| mriedem | when it's still a bug :) | 18:08 |
| imacdonn | was something fixed recently-ish w.r.t. ports on reschedule .... seems I've seen issues with an exception, something like PortInUse, when an instance creation fails and gets rescheduled on another compute node, but that probably was with Ocata | 18:09 |
| openstackgerrit | Merged openstack/nova-specs master: Add placement-req-filter spec https://review.openstack.org/544585 | 18:09 |
| mriedem | imacdonn: that's pretty vague | 18:10 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: DNM: Test LM with encrypted volumes https://review.openstack.org/536350 | 18:10 |
| mriedem | https://github.com/openstack/nova/commit/08d24b733ee9f4da44bfbb8d6d3914924a41ccdc went into newton | 18:10 |
| mriedem | *mitaka | 18:10 |
| gibi | mriedem: thanks for that note | 18:10 |
| mriedem | imacdonn: i have this https://review.openstack.org/#/c/520248/ | 18:11 |
| mriedem | but might not be what you're asking about | 18:11 |
| imacdonn | mriedem: yeah, I know ... I'd have to try to come up with a way to reproduce it ... but it's likely that the port was not created by nova, in my case .. because I use heat a lot, and usually create the port separately there | 18:11 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: DNM: Test LM with encrypted volumes https://review.openstack.org/545074 | 18:11 |
| *** yamamoto has joined #openstack-nova | 18:11 | |
| mriedem | imacdonn: in that case nova shouldn't try to delete the port | 18:12 |
| mriedem | just unbind it | 18:12 |
| gibi | dansmith: thanks for the reply on the request filter spec. I have a better view now, so I'm +1 | 18:12 |
| mriedem | if nova didn't unbind it and a reschedule tried to bind that host to another host, that would fail | 18:12 |
| dansmith | gibi: np, thanks for those comments | 18:13 |
| * gibi calls it a day | 18:13 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Detach volumes when VM creation fails https://review.openstack.org/544144 | 18:13 |
| imacdonn | mriedem: yeah. I'm fairly sure I've seen this with Ocata .. but it's one of those things that only happens when something else goes wrong (to cause the initial creation failure), so it never got hot enough for me to really dig into ... you just reminded me of if when you mentioned ports above | 18:14 |
| *** jpena is now known as jpena|off | 18:14 | |
| *** kukacz has quit IRC | 18:15 | |
| mriedem | at the risk of angering the gods, i think we actually do a relatively decent job of managing ports and cleaning up after ourselves | 18:15 |
| mriedem | with volumes, not so much | 18:15 |
| imacdonn | heh. I'll try to find time to attempt to reproduce it ... but my "round tuit" supply is low | 18:16 |
| mriedem | i understand | 18:16 |
| *** yamamoto has quit IRC | 18:16 | |
| *** ralonsoh_ has quit IRC | 18:17 | |
| imacdonn | I guess you really can by ANYTHING on Amazon! https://www.amazon.com/Round-TUIT-Tokens-Multi-Pack-Encouragement/dp/B00J8KVHLY | 18:18 |
| mriedem | heh, find me a hang in there poster with a kitten that doesn't suck | 18:18 |
| mriedem | and doesn't say "baby" on it | 18:18 |
| imacdonn | added to my todo list .. I'm try when it .... well, you know... | 18:19 |
| mriedem | https://play.google.com/store/books/details?id=KEMFmlv1uKcC&source=productsearch&utm_source=HA_Desktop_US&utm_medium=SEM&utm_campaign=PLA&pcampaignid=MKTAD0930BO1&gclid=Cj0KCQiA_JTUBRD4ARIsAL7_VeXOQT1TCYHJZb7ifQDAGqbTm2uSwLsNB0_bR7o0pTSXHwROSrbpsSUaAtxPEALw_wcB&gclsrc=aw.ds&dclid=CLC_ypvEqNkCFYzdwAod8RYOVg | 18:19 |
| mriedem | oh there is one on amazon now, well goes to show demand is up since last i checked | 18:19 |
| *** kukacz_ has joined #openstack-nova | 18:20 | |
| *** slaweq has joined #openstack-nova | 18:20 | |
| mriedem | jaypipes: even have one for you https://ih0.redbubble.net/image.475009417.6220/flat,800x800,070,f.u5.jpg | 18:20 |
| *** amoralej is now known as amoralej|off | 18:21 | |
| *** gibi has quit IRC | 18:22 | |
| *** slaweq has quit IRC | 18:25 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Add require_tenant_aggregate request filter https://review.openstack.org/545002 | 18:28 |
| openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Add a require_tenant_trait request filter https://review.openstack.org/545079 | 18:28 |
| dansmith | efried: ^ example of using traits to do the tenant isolation as an alternative | 18:28 |
| efried | dansmith: Nice, looking. | 18:29 |
| dansmith | efried: and a question in there you can probably answer and save me a few minutes of reading | 18:29 |
| *** suresh12 has joined #openstack-nova | 18:29 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Detach volumes when VM creation fails https://review.openstack.org/544143 | 18:30 |
| melwitt | efried: ack, np | 18:33 |
| *** yamamoto has joined #openstack-nova | 18:35 | |
| *** yamamoto has quit IRC | 18:35 | |
| melwitt | mriedem: yep, good point to avoid nova-net and cells v1 for mox -> mock | 18:36 |
| efried | dansmith: Responded | 18:38 |
| *** lpetrut has quit IRC | 18:38 | |
| *** lpetrut has joined #openstack-nova | 18:39 | |
| *** mgoddard_ has joined #openstack-nova | 18:39 | |
| dansmith | efried: tanks | 18:40 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Clean up volumes on boot failure https://review.openstack.org/545086 | 18:42 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Detach volumes when VM creation fails https://review.openstack.org/545087 | 18:42 |
| openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Add require_tenant_trait request filter https://review.openstack.org/545079 | 18:42 |
| TheJulia | dansmith: so we managed to get nova not to crash in ironic's grenade jobs.... we had to inject -B on to the python command line to prevent the .pyc files from being placed on disk | 18:42 |
| dansmith | TheJulia: I heard.. that sounds a lot like a python bug to me | 18:43 |
| dansmith | TheJulia: it's one thing if we get some broken call and an exception or something, but a segv seems way out of the realm of reasonable to me | 18:43 |
| *** jamesdenton has joined #openstack-nova | 18:43 | |
| TheJulia | dansmith: I believe it is officially a feature.... | 18:43 |
| dansmith | TheJulia: ... | 18:43 |
| TheJulia | certian app toolsets allow dynamic recompliation/reloading of python code in the app during runtime, the trick afaik is to remove the .pyc file which was likely occuring during upgrade | 18:44 |
| TheJulia | regardless, we're hunting something breaking with placement | 18:44 |
| *** abhishekk has quit IRC | 18:44 | |
| dansmith | TheJulia: right, but that can't cause a segv and not be called a bug, IMHO | 18:45 |
| efried | dansmith: The request_spec.project_id is always a UUID, yes? | 18:45 |
| dansmith | TheJulia: dynamic recompile is cool, even if it causes some python call imcompatibility or something, but not a segv | 18:45 |
| dansmith | efried: I think it depends on your keystone backend, no? | 18:45 |
| dansmith | TheJulia: what placement thing are you chasing now? | 18:45 |
| dansmith | efried: I bet lbragstad knows | 18:46 |
| dansmith | efried: https://github.com/midokura/python-midonetclient/issues/19 | 18:47 |
| efried | dansmith: no idea, swhy I'm asking. Cause it's gonna make a difference how much you have to sanitize it, etc. | 18:47 |
| efried | But | 18:47 |
| efried | I think we may be barking up the wrong tree anyway. Don't we actually want the trait to be CUSTOM_HOST_AGGREGATE_{agg_id} ? | 18:47 |
| lbragstad | efried it depends on the resource backend being used | 18:47 |
| TheJulia | dansmith: http://logs.openstack.org/50/544750/8/check/ironic-grenade-dsvm-multinode-multitenant/5713fb8/logs/screen-placement-api.txt.gz#_Feb_15_18_00_31_109054 after everything gets up and running and nova-compute is able to post data back out, we're getting a conflict, but I'm afraid we don't understand the mechanisms in that publishing/use of data | 18:47 |
| dansmith | efried: I don't :) | 18:47 |
| TheJulia | dansmith: to then go backwards and figure out what is truly causing that failure | 18:48 |
| dansmith | TheJulia: hmm, that's interesting | 18:48 |
| dansmith | jaypipes: ^ | 18:48 |
| lbragstad | efried if keystone is told to pull projects from something other than the default sql backend, then we can't guarantee them to be uuids | 18:49 |
| dansmith | TheJulia: I'm not sure that's really a blocking thing.. is there a matching failure in the n-cpu log? | 18:49 |
| TheJulia | http://logs.openstack.org/50/544750/8/check/ironic-grenade-dsvm-multinode-multitenant/5713fb8/logs/screen-n-cpu.txt.gz#_Feb_15_18_00_31_112744 | 18:49 |
| efried | lbragstad: Thanks. | 18:49 |
| lbragstad | efried yep | 18:50 |
| dansmith | TheJulia: okay that almost maybe kinda looks like someone changed ids or there's some confusion going on | 18:50 |
| dansmith | TheJulia: multiple nova-computes? multiple ironic nodes? | 18:50 |
| dansmith | actually, efried ^ | 18:52 |
| TheJulia | dansmith: 2x n-cpu running pike, 2x ironic-conductor (1 master, 1 queens) 1x ironic api running queens. | 18:52 |
| jroll | and multiple ironic nodes | 18:52 |
| TheJulia | yup | 18:52 |
| TheJulia | 7 of them | 18:52 |
| dansmith | efried: this is during a upgrade, any chance the report client being providertreeish after the upgrade is breaking something? | 18:52 |
| *** claudiub has joined #openstack-nova | 18:52 | |
| dansmith | because it's ensuring that the provider exists, and then freaking out because it does | 18:52 |
| jroll | reminder it could be a real incompatibility between pike nova and queens ironic, given this test has been off for weeks | 18:52 |
| jroll | dansmith: still pike nova | 18:52 |
| * efried looks | 18:52 | |
| dansmith | jroll: ah, right | 18:53 |
| TheJulia | jroll: it should presently be queens -> master, I'll double check (since the devstack-gate patch was involved) | 18:53 |
| jroll | oh, right | 18:53 |
| dansmith | okay so queens nova code yeah? | 18:53 |
| jroll | which shouldn't be much different than queens/queens which has been working | 18:54 |
| jroll | dansmith: yes, sorry | 18:54 |
| jroll | queens nova on both sides | 18:54 |
| dansmith | ack | 18:54 |
| TheJulia | jroll: we should propably cherry-pick the patch to stable/queens and let it run to see what the case is there... | 18:54 |
| jaypipes | mriedem: heh | 18:54 |
| jroll | TheJulia: wouldn't hurt | 18:54 |
| TheJulia | jroll: going to click button | 18:55 |
| jroll | thank you | 18:55 |
| *** rmcall has quit IRC | 18:55 | |
| dansmith | well, queens code still has the provider tree stuff so still worth efried looking at I think | 18:55 |
| dansmith | although weird that it would be different/broken across such a small upgrade boundary | 18:56 |
| jroll | yeah, that's my thought | 18:56 |
| TheJulia | dansmith: in the mean time, we can check the pike -> queens job results and see if it is broken the same way | 18:56 |
| efried | Yup, I'm looking. We shouldn't be able to get here except by really bad timing. | 18:56 |
| *** oomichi has joined #openstack-nova | 18:56 | |
| efried | TheJulia: And you said it happened more than once? | 18:56 |
| dansmith | more than once in the logs even | 18:56 |
| dansmith | on each sync | 18:56 |
| jroll | could also be a problem with our multinode, whether it's queens+queens or queens+master | 18:56 |
| TheJulia | efried: we have _not_ tried to recheck in case it was some process fluke | 18:57 |
| efried | No, this simply shouldn't happen, still looking... | 18:57 |
| *** lucasagomes is now known as lucas-afk | 19:00 | |
| *** harlowja has quit IRC | 19:00 | |
| efried | Okay, this is a *name* conflict, not a *uuid* conflict. This means we somehow got two RPs with different UUIDs but the same name. | 19:00 |
| efried | and the name is a UUID, which is nice and confusing. | 19:00 |
| cfriesen | mriedem: with respect to https://review.openstack.org/#/c/544748/ and my proposed change https://review.openstack.org/#/c/525253/. In our case it passes scheduling but then fails for whatever reason on the compute node. As such, the proposed fix is not sufficient because we would not end up calling _bury_in_cell0(). | 19:00 |
| efried | I thought johnthetubaguy was banging his head against this last Fall. | 19:00 |
| jroll | efried: the name would be the ironic uuid, right? | 19:00 |
| efried | jroll: I'm not an expert there, but yeah, something like that. | 19:01 |
| dansmith | efried: I wonder if we detected the ironic node go away and come back and we're trying to recreate it but never deleted it? | 19:01 |
| dansmith | jroll: yes | 19:01 |
| jroll | I seem to remember this being john's head banging https://review.openstack.org/#/c/508555/ | 19:01 |
| jroll | but there shouldn't be a rebalance happening | 19:02 |
| efried | jroll: Beat me to it, yeah, that's the patch I was thinking of. | 19:02 |
| efried | but there was more to it than that, I thought. | 19:03 |
| jroll | not sure | 19:03 |
| *** gyee has joined #openstack-nova | 19:03 | |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: mirror nova host aggregates to placement API https://review.openstack.org/545057 | 19:04 |
| jroll | the title on the bug certainly looks like this problem: https://bugs.launchpad.net/nova/+bug/1714248 | 19:04 |
| openstack | Launchpad bug 1714248 in OpenStack Compute (nova) pike "Compute node HA for ironic doesn't work due to the name duplication of Resource Provider " [High,In progress] - Assigned to Matt Riedemann (mriedem) | 19:04 |
| efried | There was another patch where we mucked with one of the rt update methods to look for things with different names. | 19:04 |
| dansmith | jroll: if ironic was down and the two nova computes notice it is back at different times maybe they could be disagreeing briefly on who owns what? | 19:04 |
| jroll | dansmith: yeah, trying to track down if it's long enough to trigger a rebalance | 19:05 |
| jroll | servicegroup api tells us | 19:05 |
| melwitt | mriedem: FYI, digging into the difference between the cleanup volumes vs ports bugs today | 19:06 |
| melwitt | based on your comments in the patch | 19:06 |
| *** burt has joined #openstack-nova | 19:06 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: DNM: Test LM with encrypted volumes https://review.openstack.org/545093 | 19:08 |
| jroll | dansmith: ah yep, the subnode n-cpu considers itself down at this point, I believe http://logs.openstack.org/50/544750/8/check/ironic-grenade-dsvm-multinode-multitenant/5713fb8/logs/subnode-2/screen-n-cpu.txt.gz#_Feb_15_17_59_05_738069 | 19:09 |
| jroll | ironic is unreachable for like 5 minutes | 19:10 |
| jroll | or rather a full resource tracker run and then some | 19:10 |
| openstackgerrit | Jackie Truong proposed openstack/python-novaclient master: Microversion 2.61 - Add trusted_image_certificates https://review.openstack.org/500396 | 19:11 |
| mriedem | cfriesen: then what you have here https://review.openstack.org/#/c/525253/1/nova/conductor/manager.py doesn't help you | 19:12 |
| mriedem | cfriesen: so i'm confused | 19:12 |
| mriedem | cfriesen: is https://review.openstack.org/#/c/528385/ what you are looking for? | 19:12 |
| dansmith | jroll: yeah, but with johnthetubaguy's reuse-compute-node patch I would think this wouldn't be a problem right? | 19:12 |
| jroll | aaaand we have some problems deleting RPs: http://logs.openstack.org/50/544750/8/check/ironic-grenade-dsvm-multinode-multitenant/5713fb8/logs/screen-n-cpu.txt.gz#_Feb_15_17_58_22_418875 | 19:12 |
| jroll | (wtf) | 19:13 |
| jroll | dansmith: I would think so too, just confirming there is likely a rebalance, so they could be disagreeing | 19:13 |
| dansmith | yeah | 19:13 |
| dansmith | jroll: ah, that 503 during deleting is weird | 19:13 |
| jroll | dansmith: indeed | 19:14 |
| dansmith | jroll: do you guys have to restart apache? | 19:14 |
| TheJulia | dansmith: we do | 19:14 |
| dansmith | okay | 19:14 |
| dansmith | also | 19:14 |
| TheJulia | we update the configuration to load a vhost | 19:14 |
| TheJulia | we also shutdown services at 17:55 for the upgrade, nova would have remained running | 19:15 |
| dansmith | if placement is crashing the same way as conductor, maybe placement is dead under apache, hence the 503? | 19:15 |
| jroll | good lord keystone db migrations are spammy | 19:15 |
| dansmith | until you restart? | 19:15 |
| TheJulia | so 17:58 is when everything is down | 19:15 |
| dansmith | thus nova never got to delete the RPs | 19:15 |
| TheJulia | I think the restart is during the ironic upgrade, checking to see when it actually occured | 19:15 |
| dansmith | right, but if placement started crashing during the upgrade of packages, | 19:16 |
| dansmith | which is when nova would have noticed ironic went away and tried to delete RPs or something, | 19:16 |
| jroll | OH | 19:16 |
| jroll | keystone is upgrading there | 19:16 |
| jroll | and so placement can't validate the token | 19:16 |
| dansmith | and then you restart apache.. | 19:16 |
| dansmith | ohh | 19:16 |
| jroll | http://logs.openstack.org/50/544750/8/check/ironic-grenade-dsvm-multinode-multitenant/5713fb8/logs/screen-placement-api.txt.gz#_Feb_15_17_58_22_463228 | 19:16 |
| *** felipemonteiro_ has quit IRC | 19:16 | |
| mriedem | melwitt: i'm likely also going to start writing a functional test for the case that we delete a build request for a bfv instance during local delete, because we aren't cleaning up volumes there either | 19:17 |
| *** felipemonteiro_ has joined #openstack-nova | 19:17 | |
| dansmith | and you get a bug, and you get a bug, and you get a bug... | 19:17 |
| * jroll makes a note to file a bug to not 503 placement here | 19:17 | |
| dansmith | jroll: good catch | 19:17 |
| melwitt | mriedem: ack | 19:17 |
| TheJulia | http://logs.openstack.org/50/544750/8/check/ironic-grenade-dsvm-multinode-multitenant/5713fb8/logs/grenade.sh.txt.gz#_2018-02-15_18_00_28_322 is when we restart apache | 19:17 |
| dansmith | jroll: surely you're not intending to have keystone be upgradingin this task right? | 19:17 |
| dansmith | s/task/job/ | 19:17 |
| jroll | dansmith: I think we just haven't cared either way in the past | 19:18 |
| jroll | keystone's one of those hand-wavy things to me, it's always there, don't care what version | 19:18 |
| dansmith | jroll: do you care now? :P | 19:18 |
| jroll | heh | 19:18 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add admin guide doc on volume multiattach support https://review.openstack.org/544090 | 19:21 |
| TheJulia | jroll: we could just skip restarting apache and fire up the local ironic-api, however then we're not forcing traffic through an older API endpoint which changes the scenario, although we would still have the rpc version pin | 19:21 |
| mriedem | bauzas: since you helped review the multiattach series, can you check out ^ so we can backport that for queens RC2? | 19:21 |
| jroll | TheJulia: eh, I'd rather not | 19:22 |
| jroll | also these times aren't lining up :/ | 19:22 |
| TheJulia | actually, other services will still likely need to restart things | 19:22 |
| TheJulia | so we shouldn't try to avoid restarting apache | 19:22 |
| jroll | oh it does line up, okay | 19:22 |
| jroll | TheJulia: maybe we um, try to make apache not need 66 seconds to restart | 19:23 |
| TheJulia | heh, 28 apache restarts in the grenade log | 19:23 |
| jroll | jesus | 19:23 |
| dansmith | wow | 19:23 |
| dansmith | that's impressive | 19:23 |
| mriedem | the edge people said openstack needed to be slimmed down | 19:23 |
| TheJulia | yeah.... | 19:24 |
| cfriesen | mriedem: I think you're right, I got messed up with which patch was fixing what. :) the fixes at https://review.openstack.org/#/c/528385 and https://review.openstack.org/#/c/340614/ look like they might do the trick. | 19:24 |
| mriedem | cfriesen: cool | 19:24 |
| mriedem | it is confusing | 19:24 |
| mriedem | cfriesen: the point of creating the bdms in cell0 also was so that the local delete in the api can remove them | 19:24 |
| mriedem | and thus avoid us having nova-compute, nova-api AND nova-conductor doing volume cleanup | 19:25 |
| TheJulia | jroll: what makes you think 66 seconds? | 19:25 |
| jroll | TheJulia: this line to the next one: http://logs.openstack.org/50/544750/8/check/ironic-grenade-dsvm-multinode-multitenant/5713fb8/logs/screen-keystone.txt.gz#_Feb_15_17_57_44_538878 | 19:26 |
| jroll | it isn't 66 seconds to restart | 19:26 |
| jroll | nor is it an apache thing | 19:26 |
| jroll | but we shut down keystone for the entire keystone upgrade | 19:26 |
| TheJulia | ahh, yeah | 19:26 |
| jroll | which is silly | 19:27 |
| *** r-daneel has quit IRC | 19:27 | |
| TheJulia | Well.... are we sure it is not silly? | 19:27 |
| jroll | I would think keystone should be able to do rolling upgrades very easily | 19:28 |
| jroll | but they don't have the magic tag, so who knows | 19:28 |
| * jroll needs to stand up and move for a bit | 19:29 | |
| * jroll is thinking placement should be able to handle this, but needs to load in all the context to be sure | 19:29 | |
| dansmith | well, it's not really placement, it's nova-compute right? | 19:30 |
| dansmith | we're probably not handling the 503 very gracefully in compute whilst trying to do things | 19:30 |
| *** mriedem has left #openstack-nova | 19:32 | |
| *** mriedem has joined #openstack-nova | 19:32 | |
| *** slaweq has joined #openstack-nova | 19:33 | |
| mriedem | is the 503 this? https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L93 | 19:33 |
| dansmith | I woudn't think so, we're getting a response | 19:34 |
| oomichi | hi, can someone take a look at https://review.openstack.org/#/c/532822 ? that is easy and completely the same patch is posted. so want to avoid the same thing | 19:34 |
| dansmith | we just raise and kill the RT | 19:34 |
| *** harlowja has joined #openstack-nova | 19:35 | |
| *** yamamoto has joined #openstack-nova | 19:35 | |
| oomichi | thanks, mriedem | 19:37 |
| *** harlowja_ has joined #openstack-nova | 19:37 | |
| *** harlowja has quit IRC | 19:39 | |
| mriedem | melwitt: i went ahead and created this for tracking the community wide goal and linked it to the other 4 bp's we've had in the past for the same thing https://blueprints.launchpad.net/nova/+spec/mox-removal | 19:40 |
| *** lpetrut has quit IRC | 19:40 | |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Account for host agg allocation ratio in placement https://review.openstack.org/544683 | 19:40 |
| mriedem | and marked the other as done https://review.openstack.org/545100 | 19:41 |
| melwitt | mriedem: great, thank you | 19:41 |
| jaypipes | bauzas, cdent, edleafe, dansmith, mriedem, efried: ^^ round two on the aggregate allocation ratio one. | 19:41 |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Account for host agg allocation ratio in placement https://review.openstack.org/544683 | 19:43 |
| mriedem | melwitt: i also assume we'll want 2 blueprints for removing nova-network and cellsv1 | 19:44 |
| mriedem | as those likely won't be a single change | 19:44 |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: mirror nova host aggregates to placement API https://review.openstack.org/545057 | 19:44 |
| melwitt | mriedem: yeah, I don't think they would be a single change | 19:44 |
| mriedem | i'm not sure on the order there though, probably nova-network first since you can't start nova-network w/o cells v1 enabled | 19:44 |
| mriedem | and some people run cellsv1 with neutron | 19:45 |
| *** mgoddard_ has quit IRC | 19:45 | |
| *** yamamoto has quit IRC | 19:45 | |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Account for host agg allocation ratio in placement https://review.openstack.org/544683 | 19:46 |
| efried | jaypipes: Ya done yet? | 19:50 |
| *** munishmehan has joined #openstack-nova | 19:53 | |
| jaypipes | efried: yeah, sorry :) | 19:58 |
| *** pchavva has quit IRC | 20:01 | |
| *** pcaruana has quit IRC | 20:03 | |
| *** suresh12 has quit IRC | 20:04 | |
| *** suresh12 has joined #openstack-nova | 20:05 | |
| *** suresh12 has quit IRC | 20:09 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Convert driver supported capabilities to compute node provider traits https://review.openstack.org/538498 | 20:10 |
| *** sree has joined #openstack-nova | 20:11 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Convert driver supported capabilities to compute node provider traits https://review.openstack.org/538498 | 20:14 |
| *** tssurya has joined #openstack-nova | 20:14 | |
| *** AlexeyAbashkin has joined #openstack-nova | 20:15 | |
| *** sree has quit IRC | 20:15 | |
| *** kukacz_ has quit IRC | 20:17 | |
| *** AlexeyAbashkin has quit IRC | 20:19 | |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: mirror nova host aggregates to placement API https://review.openstack.org/545057 | 20:19 |
| mnaser | mriedem: do i have to make any changes at this point for the clean up patch .. i know there's a lot of discussion but anything actionable for me right now? | 20:24 |
| mriedem | mnaser: nope | 20:25 |
| * jaypipes wondering if mnaser ever sleeps... | 20:25 | |
| *** tidwellr has quit IRC | 20:25 | |
| *** tidwellr has joined #openstack-nova | 20:25 | |
| mnaser | jaypipes: ahah. words that once i settle down with someone + kids, time disappers | 20:26 |
| mnaser | enjoying it while it lasts, ha | 20:26 |
| *** READ10 has quit IRC | 20:29 | |
| mriedem | you can still have a family, and ignore them | 20:29 |
| jaypipes | mnaser: :) | 20:29 |
| mnaser | haha | 20:29 |
| * mnaser has honestly enjoyed every bit of working in openstack for the past few years | 20:29 | |
| jaypipes | mnaser: that's awesome to hear! | 20:29 |
| jroll | that is awesome, but I'm skeptical, *every* bit? | 20:30 |
| jroll | :P | 20:30 |
| mnaser | jaypipes: :D | 20:30 |
| mnaser | jroll: it helps having super awesome understanding customers that know all about open source projects | 20:30 |
| *** hongbin has quit IRC | 20:31 | |
| mnaser | any issues we usually work with them to iron out and work with openstack teams to roll fixes out and everyone's happy | 20:31 |
| jroll | sweet :) | 20:31 |
| *** r-daneel has joined #openstack-nova | 20:31 | |
| jaypipes | mnaser: perhaps you can come visit with my internal customers ... ;) | 20:31 |
| mnaser | maybe they should just use us and make your life easier </salesman> | 20:32 |
| mnaser | haha | 20:32 |
| jaypipes | mnaser: ++ :) | 20:32 |
| *** hongbin has joined #openstack-nova | 20:32 | |
| jroll | jaypipes: wow, trying to drag mnaser down so quickly after mentioning the fun | 20:32 |
| mnaser | i'm just a bit bummed that i don't think we'll be able to get to queens right away | 20:33 |
| jaypipes | lol | 20:33 |
| mnaser | mainly because we have consumers using the images v1 api | 20:33 |
| openstackgerrit | Matthew Edmonds proposed openstack/nova-specs master: PowerVM Virt Integration (Rocky) https://review.openstack.org/545111 | 20:34 |
| mnaser | jaypipes: will you be at the ptg? | 20:36 |
| jaypipes | mnaser: yessir | 20:36 |
| mnaser | jaypipes: lets chat money sinks aka cars one day (maybe if nova is having a dinner? :p) | 20:37 |
| dansmith | ooh ooh, me me | 20:38 |
| mnaser | :D | 20:39 |
| openstackgerrit | Matthew Edmonds proposed openstack/nova-specs master: PowerVM Virt Integration (Rocky) https://review.openstack.org/545111 | 20:40 |
| jaypipes | dansmith, mnaser: I am always up for that :) | 20:41 |
| mnaser | does anyone want to organize a lunch or a dinner for the nova team? :> | 20:42 |
| *** andreas_s has joined #openstack-nova | 20:43 | |
| mriedem | mnaser: i think you just did | 20:43 |
| mriedem | thursday night | 20:43 |
| mriedem | cburgess was the food czar but i don't know if he'll be in dublin | 20:44 |
| mriedem | actually, we should make stephenfin organize a dinner since he's local | 20:44 |
| *** mgoddard_ has joined #openstack-nova | 20:46 | |
| *** andreas_s has quit IRC | 20:47 | |
| melwitt | nova meeting in 10 minutes | 20:50 |
| openstackgerrit | Matthew Edmonds proposed openstack/nova-specs master: PowerVM Virt Integration (Rocky) https://review.openstack.org/545111 | 20:51 |
| efried | mriedem: Small delta reproposal from Q to R ^ | 20:52 |
| *** takashin has joined #openstack-nova | 20:56 | |
| mriedem | your call will be processed in the order in which it was received | 20:59 |
| dansmith | handled | 20:59 |
| dansmith | you don't process calls | 20:59 |
| dansmith | duh. | 20:59 |
| mriedem | i made a phone call today | 20:59 |
| mriedem | with my phone | 20:59 |
| mnaser | mriedem: a local organizing it might be way better than me :> | 20:59 |
| mriedem | i like to consider myself enterprise business solutions | 21:00 |
| mriedem | mnaser: i've got a recreate functional test for the bfv + delete build request issue, will push up shortly | 21:01 |
| *** ttsiouts_ has joined #openstack-nova | 21:04 | |
| *** mgoddard_ has quit IRC | 21:05 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional recreate test of deleting a BFV server pre-scheduling https://review.openstack.org/545123 | 21:05 |
| mriedem | mnaser: ^ | 21:05 |
| cburgess | mriedem Tragically I will not be in dublin. I think I can make it to Vancouver, but probably won't be in Berlin. Might be able to swing Fall PTG is its North America. | 21:11 |
| dansmith | cburgess: boo :( | 21:13 |
| cburgess | dansmith Yeah, since the product is no in sustaining mode I don't have budget to travel. I do have budget for "training" so I can swing 1 or 2 openstack events a year that way but I can do all 4 now. | 21:14 |
| cburgess | s/no/now | 21:14 |
| dansmith | yeah :/ | 21:14 |
| cburgess | dansmith But like I said, I'm working on being in Vancouver. | 21:16 |
| dansmith | well, I'll take it | 21:16 |
| openstackgerrit | Merged openstack/nova master: libvirt: remove TODO on validation of scsi model https://review.openstack.org/525055 | 21:17 |
| *** burt has quit IRC | 21:18 | |
| *** sree has joined #openstack-nova | 21:20 | |
| openstackgerrit | Jackie Truong proposed openstack/python-novaclient master: Microversion 2.61 - Add trusted_image_certificates https://review.openstack.org/500396 | 21:23 |
| cburgess | have fun in dublin for me. Always wanted to go to Ireland. | 21:23 |
| edmondsw | melwitt you asked about 3rd party CI failures... the powervm CI (and presumably others) was failing for the tinyrpc issue that is hopefully fixed by https://review.openstack.org/#/c/545033/ | 21:24 |
| *** burt has joined #openstack-nova | 21:24 | |
| *** sree has quit IRC | 21:24 | |
| edmondsw | efried FYI ^ | 21:25 |
| *** tidwellr_ has joined #openstack-nova | 21:26 | |
| *** tidwellr has quit IRC | 21:26 | |
| melwitt | edmondsw: ah, thanks for that info | 21:27 |
| edmondsw | np | 21:27 |
| *** ttsiouts_ has quit IRC | 21:27 | |
| *** suresh12 has joined #openstack-nova | 21:29 | |
| edmondsw | melwitt looks like that hasn't merged yet for master, but has +W: https://review.openstack.org/#/c/545025/ | 21:30 |
| *** pcaruana has joined #openstack-nova | 21:31 | |
| melwitt | edmondsw: ack, thanks | 21:31 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: List/show all server migration types (1/2) https://review.openstack.org/430608 | 21:31 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: List/show all server migration types (2/2) https://review.openstack.org/459483 | 21:32 |
| openstackgerrit | Merged openstack/nova master: Detach volumes when VM creation fails https://review.openstack.org/528385 | 21:33 |
| openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Microversion 2.61 - List/Show all server migration types https://review.openstack.org/430839 | 21:33 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Adds view builders for keypairs controller https://review.openstack.org/347289 | 21:34 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional recreate test of deleting a BFV server pre-scheduling https://review.openstack.org/545123 | 21:37 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Detach volumes when deleting a BFV server pre-scheduling https://review.openstack.org/545132 | 21:37 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: api-ref: Parameter verification for servers.inc https://review.openstack.org/528201 | 21:37 |
| mriedem | melwitt: mnaser: ^ thar she blar | 21:37 |
| mriedem | that was actually pretty easy | 21:37 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: api-ref: Example verification for servers.inc https://review.openstack.org/529520 | 21:37 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix 500 error while passing 4-byte unicode data https://review.openstack.org/407514 | 21:38 |
| melwitt | BFV + easy ... that might be a first | 21:38 |
| *** tidwellr_ has quit IRC | 21:38 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: api-ref: Fix parameter order in rebuild https://review.openstack.org/529718 | 21:38 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Transform live_migration_post_dest notification https://review.openstack.org/469784 | 21:39 |
| *** tbachman has quit IRC | 21:39 | |
| mriedem | easy like bfv morning | 21:42 |
| melwitt | lol | 21:42 |
| *** Lingwu has quit IRC | 21:43 | |
| *** openstack has joined #openstack-nova | 21:45 | |
| *** ChanServ sets mode: +o openstack | 21:45 | |
| *** cdent has joined #openstack-nova | 21:45 | |
| *** hamzy has quit IRC | 21:46 | |
| *** tidwellr has joined #openstack-nova | 21:47 | |
| *** pcaruana has quit IRC | 21:48 | |
| *** felipemonteiro_ has quit IRC | 21:48 | |
| *** rcernin has joined #openstack-nova | 21:50 | |
| mriedem | melwitt: replied in https://review.openstack.org/#/c/340614/ - i think it's mostly a matter of cleaning up the commit message and comments for clarity | 21:51 |
| mriedem | the rest i'm OK with after mnaser sorted me out earlier today | 21:51 |
| melwitt | mriedem: that's what I gathered too, thanks. just have to take another pass to try and make it not confusing. which has been surprisingly hard | 21:52 |
| *** oomichi has quit IRC | 21:52 | |
| mriedem | dansmith: how would you like to do something exciting like go through the stable/queens backports and figure out what we want to get into RC2? | 21:55 |
| mriedem | dansmith: or do you want me to take a pass, +2 what i think is safe, and then prod you | 21:55 |
| dansmith | definitely the latter :P | 21:56 |
| mriedem | heh i figured | 21:56 |
| *** awaugama has quit IRC | 21:56 | |
| imacdonn | mriedem: FYI, I have been able to reproduce the issue where a rescheduled instance fails with a PortInUse exception (https://pastebin.com/6HAL944i)... on Ocata .. I guess I'll try on Pike next (in spare time) | 21:59 |
| *** tssurya has quit IRC | 21:59 | |
| *** tssurya has joined #openstack-nova | 22:00 | |
| *** tssurya has quit IRC | 22:00 | |
| *** tssurya has joined #openstack-nova | 22:00 | |
| *** tssurya has quit IRC | 22:01 | |
| mriedem | dansmith: while i do that, mayhap you'd be so kind as to review these simple docs changes https://review.openstack.org/#/c/544090/ (and the one below it) so i can backport those to queens as well | 22:02 |
| dansmith | mriedem: mayhap done | 22:05 |
| mriedem | great | 22:05 |
| mriedem | do you feel sufficiently up to date on volume multiattach now? | 22:06 |
| *** suresh12 has quit IRC | 22:06 | |
| dansmith | oh totes | 22:08 |
| *** felipemonteiro__ has quit IRC | 22:09 | |
| *** felipemonteiro__ has joined #openstack-nova | 22:10 | |
| *** belmoreira has joined #openstack-nova | 22:11 | |
| mriedem | dansmith: how do you feel about this one for queens? https://review.openstack.org/#/c/543489/ | 22:13 |
| mriedem | RC2 i mean | 22:13 |
| dansmith | idk, I'd like to see those backported, but they don't _have_ to be in rc2 | 22:13 |
| mriedem | if it's low impact/low risk i think it's ok | 22:14 |
| belmoreira | dansmith thanks for https://review.openstack.org/#/c/544585 I added few comments to clarify how we use the cell-scheduler in cellsV1 | 22:14 |
| dansmith | belmoreira: I think you're probably going to have to be willing to tweak some of your current mappings to make things work, | 22:16 |
| dansmith | but yes, we can have more filters with more behaviors, | 22:16 |
| dansmith | and/or tweak the stuff I have proposed | 22:17 |
| dansmith | belmoreira: https://review.openstack.org/#/c/545002/ | 22:17 |
| dansmith | belmoreira: I would like to understand more about why aggregates are more complicated at your scale, as they're pretty much intended to simplify stuff at higher scale | 22:17 |
| openstackgerrit | Merged openstack/nova master: Cleanup the manage-volumes admin doc https://review.openstack.org/544066 | 22:18 |
| dansmith | maybe we can hear from you/surya in dublin about it | 22:18 |
| openstackgerrit | Merged openstack/nova master: Add admin guide doc on volume multiattach support https://review.openstack.org/544090 | 22:18 |
| mriedem | dansmith: ok +2s on the stuff that i thought was fine; https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/queens - mine that aren't -W are up for review also, since i can't +2 those; i also didn't do takashi's for the placement global req id since they are rather large, but could maybe convinced otherwise | 22:18 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Cleanup the manage-volumes admin doc https://review.openstack.org/545141 | 22:20 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Add admin guide doc on volume multiattach support https://review.openstack.org/545142 | 22:20 |
| mriedem | and these ^ | 22:20 |
| dansmith | okay let me hit your +2s first | 22:21 |
| dansmith | ah yeah all those +2s other than the UC ones are things I had reviewed on master | 22:22 |
| mnaser | melwitt: mriedem so i guess just updating the commit message remains but i'll leave that for melwitt i think she'll do a better job than i will :p | 22:24 |
| belmoreira | dansmith sure, I'm willing to tweak the current mappings to work with the new scheduler schema. I just wanted to describe how we currently do it for reference | 22:27 |
| dansmith | belmoreira: okay, definitely appreciate that :) | 22:27 |
| mriedem | belmoreira: hard-coding projects in config is a bit odd | 22:27 |
| mriedem | https://gitlab.cern.ch/belmiro/cellsv1-filters/blob/master/target_cell_project.py#L29 | 22:27 |
| *** priteau has quit IRC | 22:29 | |
| *** priteau has joined #openstack-nova | 22:29 | |
| belmoreira | mriedem I agree, that was the initial approach. Now the cell mapping is a property of the project. Just didn't remove that old code just in case... | 22:29 |
| *** jaypipes has quit IRC | 22:30 | |
| *** jaypipes has joined #openstack-nova | 22:31 | |
| mriedem | oh yeah i see now | 22:31 |
| mriedem | cells_mapping = client_key.get_cells_mapping(instance_project_id) | 22:31 |
| mriedem | if config, use those, else get the mappings from keystone | 22:31 |
| belmoreira | mriedem it was kept for the transition phase, when not all the projects had the cell mapping as a property | 22:31 |
| belmoreira | I have >3000 projects :) | 22:32 |
| mriedem | belmoreira: i assume the complexity with host aggregates at that scale is that a lot of hosts are in a lot of different aggregates | 22:34 |
| *** priteau has quit IRC | 22:34 | |
| mriedem | how much of the aggregate management is automated? | 22:34 |
| belmoreira | dansmith mriedem also because is very difficult to automate add/remove nodes | 22:35 |
| dansmith | why | 22:35 |
| dansmith | ? | 22:35 |
| dansmith | (is it difficult) | 22:35 |
| belmoreira | using a configuration management tool is not that safe/easy for this operations | 22:35 |
| *** felipemonteiro_ has joined #openstack-nova | 22:35 | |
| dansmith | sure, but this is done via the api | 22:36 |
| belmoreira | that is one of the main reasons. The conf tool will need to have credentials for these operations | 22:36 |
| mriedem | removing nodes is a problem i can see, there is no api for that really - we have delete service, but that doesn't cleanup the compute_nodes table entry | 22:37 |
| dansmith | belmoreira: oh okay, I guess my point was .. maybe using conf management for this is not a good idea ;) | 22:37 |
| dansmith | mriedem: removing nodes what? you can remove a node from an aggregate via the api | 22:38 |
| mriedem | i assumed he was talking about dropping hosts | 22:38 |
| mriedem | not add/remove aggregate members | 22:38 |
| dansmith | hmm | 22:39 |
| belmoreira | mriedem was talking about aggregate management using a conf management tool | 22:39 |
| mriedem | which spurred the discussion the other day about delete_cell needing a --ignore-placement flag or whatever | 22:39 |
| mriedem | ok, ignore me then, i'm talking about something else | 22:39 |
| mriedem | 'decomissioning' nodes | 22:39 |
| *** felipemonteiro__ has quit IRC | 22:40 | |
| belmoreira | that is a one time operation. I'm not that worry if is not full automated | 22:41 |
| *** slaweq has joined #openstack-nova | 22:42 | |
| dansmith | belmoreira: are you going to be in dublin or just tssurya? | 22:42 |
| belmoreira | dansmith just tssurya | 22:42 |
| dansmith | okay | 22:42 |
| belmoreira | I will be Vancouver | 22:42 |
| dansmith | okay | 22:43 |
| *** munishmehan has quit IRC | 22:43 | |
| *** threestrands has joined #openstack-nova | 22:43 | |
| *** slaweq has quit IRC | 22:44 | |
| belmoreira | my previous point was that if for a specific use case we need to create aggregates/add nodes it will be very difficult to manage this at scale | 22:44 |
| melwitt | mriedem: replied on https://review.openstack.org/#/c/340614 about instance.host = None + vm_state = ERROR | 22:45 |
| belmoreira | especially because aggregates and cells need to be in sync. Missing a node in this mapping could mean that it will not be used even if it's in the cell | 22:46 |
| openstackgerrit | Arvind Nadendla proposed openstack/nova-specs master: Support traits in Glance https://review.openstack.org/541507 | 22:47 |
| mriedem | edmondsw: efried: done https://review.openstack.org/#/c/545111/ | 22:48 |
| efried | mriedem: Thanks! | 22:49 |
| mriedem | melwitt: we expect that in a *very specific scenario* | 22:53 |
| mriedem | the commit message makes it sound like the instance is ever only in error state because of a failed build, which is not the case | 22:53 |
| melwitt | okay, so the words should be "we can expect" instead of "we expect" | 22:56 |
| mriedem | let me get cochran on the horn | 22:56 |
| melwitt | "on the horn" means phone? that's new to me | 22:57 |
| mriedem | yes | 22:58 |
| mriedem | did you get the cochran reference at least? | 22:59 |
| melwitt | like johnny cochran? | 22:59 |
| mriedem | yes | 22:59 |
| melwitt | yeah | 22:59 |
| mriedem | how about just saying, "If the instance is in ERROR because of a failed build" | 23:00 |
| mriedem | my council informs me that would be satisfactory | 23:00 |
| melwitt | thank you, council people | 23:01 |
| *** mrhillsman has joined #openstack-nova | 23:06 | |
| imacdonn | mriedem: I reproduced the PortInUse thing on Pike, and also confirmed it only happens when using a pre-existing port - created https://bugs.launchpad.net/nova/+bug/1749838 | 23:08 |
| openstack | Launchpad bug 1749838 in OpenStack Compute (nova) "Rescheduled instace with pre-existing port fails with PortInUse exception" [Undecided,New] | 23:08 |
| *** itlinux has quit IRC | 23:09 | |
| mriedem | ok | 23:10 |
| mriedem | imacdonn: this is latest stable/pike? | 23:12 |
| imacdonn | It's RDO - nova 16.0.3 | 23:12 |
| mriedem | ok so port.device_id is set | 23:12 |
| mriedem | is what it's failing on | 23:13 |
| mriedem | when we unbind the port before rescheduling, we should wipe that out | 23:13 |
| mriedem | port_req_body = {'port': {'device_id': '', 'device_owner': ''}} | 23:13 |
| mriedem | https://github.com/openstack/nova/blob/stable/pike/nova/network/neutronv2/api.py#L511 | 23:14 |
| mriedem | do you see "Unable to clear device ID" in the logs? | 23:14 |
| mriedem | on the first host? | 23:14 |
| imacdonn | will look .. I did attach debug lots from both nodes to the bug just now | 23:14 |
| imacdonn | logs* | 23:14 |
| imacdonn | I don't see that message (grep for "clear" only finds a couple of config options) | 23:15 |
| mriedem | i feel like i had a patch that added debug logging in this code for when we tore down, but i probably abandoned it | 23:16 |
| mriedem | imacdonn: oh i bet this is the fix you need https://review.openstack.org/#/c/520248/ | 23:17 |
| *** belmoreira has quit IRC | 23:17 | |
| mriedem | in the case of nova creating a port, | 23:18 |
| mriedem | it doesn't fail because nova orphans the port created from the first host, and creates a new port when going through the 2nd host | 23:18 |
| mriedem | so you end up with 2 ports for the instance that nova created even though you're only using 1 | 23:18 |
| mriedem | in the case that you bring a port, nova doesn't unbind it before rescheduling, and that's why we fail to use it on the 2nd host | 23:18 |
| imacdonn | seems plausible ... looks like a one-line change, so I can try to drop it in | 23:18 |
| mriedem | imacdonn: you could try applying that patch and see if it resolves it | 23:18 |
| *** acormier has quit IRC | 23:18 | |
| *** acormier has joined #openstack-nova | 23:19 | |
| *** acormier has quit IRC | 23:23 | |
| *** r-daneel has quit IRC | 23:24 | |
| *** amodi has quit IRC | 23:24 | |
| *** felipemonteiro_ has quit IRC | 23:24 | |
| *** felipemonteiro_ has joined #openstack-nova | 23:24 | |
| imacdonn | mriedem: yup, that fixed it - thanks! | 23:35 |
| *** hshiina has joined #openstack-nova | 23:36 | |
| *** hshiina2 has joined #openstack-nova | 23:38 | |
| *** hshiina has quit IRC | 23:40 | |
| *** sdague has quit IRC | 23:41 | |
| *** cdent has quit IRC | 23:45 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: List/show all server migration types (1/2) https://review.openstack.org/430608 | 23:46 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: List/show all server migration types (2/2) https://review.openstack.org/459483 | 23:46 |
| *** stakeda has joined #openstack-nova | 23:52 | |
| cfriesen | does anyone know offhand what the rules are for using instance.info_cache.network_info vs calling self.network_api.get_instance_nw_info(context, instance) ? | 23:54 |
| *** hshiina2 is now known as hshiina | 23:55 | |
| *** hongbin has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
