| *** claudiub has quit IRC | 00:00 | |
| *** tetsuro has joined #openstack-nova | 00:03 | |
| openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: Snapshot https://review.openstack.org/543023 | 00:09 |
|---|---|---|
| *** felipemonteiro has joined #openstack-nova | 00:12 | |
| *** hemna_ has joined #openstack-nova | 00:13 | |
| *** felipemonteiro_ has joined #openstack-nova | 00:13 | |
| *** itlinux has joined #openstack-nova | 00:16 | |
| *** liverpoo1er has quit IRC | 00:17 | |
| *** felipemonteiro has quit IRC | 00:17 | |
| *** liverpooler has quit IRC | 00:17 | |
| *** edmondsw has quit IRC | 00:18 | |
| *** liverpooler has joined #openstack-nova | 00:21 | |
| *** liverpoo1er has joined #openstack-nova | 00:21 | |
| *** lbragstad has quit IRC | 00:22 | |
| *** hshiina has joined #openstack-nova | 00:23 | |
| *** slaweq has joined #openstack-nova | 00:31 | |
| *** slaweq has quit IRC | 00:36 | |
| *** Dinesh_Bhor has joined #openstack-nova | 00:37 | |
| *** AlexeyAbashkin has joined #openstack-nova | 00:40 | |
| *** links has joined #openstack-nova | 00:40 | |
| *** mlavalle has quit IRC | 00:40 | |
| *** chyka has quit IRC | 00:42 | |
| *** chyka has joined #openstack-nova | 00:42 | |
| *** acormier has joined #openstack-nova | 00:44 | |
| *** AlexeyAbashkin has quit IRC | 00:44 | |
| *** acormier has quit IRC | 00:45 | |
| *** chyka has quit IRC | 00:47 | |
| *** mdnadeem has joined #openstack-nova | 00:50 | |
| *** sdague has joined #openstack-nova | 00:51 | |
| *** zhaochao has joined #openstack-nova | 00:55 | |
| *** sdague has quit IRC | 00:57 | |
| *** jichen has joined #openstack-nova | 00:59 | |
| gmann_ | mriedem_afk: gibi stephenfin as we faced the nova jobs irrelevant_files and branch restriction issues and have to fix on project-config side, i am thinking to move pipeline list jobs on project side. | 00:59 |
| gmann_ | mriedem_afk: gibi stephenfin this will give nova a full control on those bits. i started a generic discussion on ML- http://lists.openstack.org/pipermail/openstack-dev/2018-February/127320.html | 01:00 |
| *** esberglu has quit IRC | 01:00 | |
| gmann_ | please check and if that looks ok/not ok from nova perspective. | 01:00 |
| gmann_ | melwitt: ^^ | 01:01 |
| gmann_ | next issue is irrelevant_files which does not work in case of inheritance or pipeline list (current issue where lot of nova jobs run on doc/api-ref changes), i will push some debug patch and then debug with infra team. | 01:02 |
| *** Dinesh_Bhor has quit IRC | 01:08 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: [placement] Add functional tests for traits API https://review.openstack.org/524094 | 01:09 |
| *** Dinesh_Bhor has joined #openstack-nova | 01:10 | |
| openstackgerrit | Rong Han proposed openstack/nova stable/pike: detach instance volumes when VM creation fails https://review.openstack.org/544143 | 01:11 |
| *** moshele has joined #openstack-nova | 01:15 | |
| openstackgerrit | Rong Han proposed openstack/nova stable/queens: detach instance volumes when VM creation fails https://review.openstack.org/544144 | 01:18 |
| *** itlinux has quit IRC | 01:19 | |
| *** hongbin has joined #openstack-nova | 01:24 | |
| *** Dinesh_Bhor has quit IRC | 01:25 | |
| *** gyee has quit IRC | 01:25 | |
| *** Dinesh_Bhor has joined #openstack-nova | 01:26 | |
| *** Dinesh_Bhor has quit IRC | 01:31 | |
| *** felipemonteiro__ has joined #openstack-nova | 01:33 | |
| *** felipemonteiro_ has quit IRC | 01:33 | |
| tetsuro | efried, dansmith, mriedem, jaypipes: Just finished reading back the dictussion about the update_provider_tree. Thank you for picking it up! | 01:34 |
| *** hemna_ has quit IRC | 01:34 | |
| *** liverpooler has quit IRC | 01:35 | |
| *** Dinesh_Bhor has joined #openstack-nova | 01:36 | |
| *** Dinesh_Bhor has quit IRC | 01:41 | |
| *** itlinux has joined #openstack-nova | 01:44 | |
| *** acormier has joined #openstack-nova | 01:44 | |
| *** Dinesh_Bhor has joined #openstack-nova | 01:45 | |
| *** edmondsw has joined #openstack-nova | 01:46 | |
| *** felipemonteiro_ has joined #openstack-nova | 01:46 | |
| *** rmcall has joined #openstack-nova | 01:46 | |
| *** moshele has quit IRC | 01:47 | |
| *** felipemonteiro__ has quit IRC | 01:49 | |
| *** edmondsw has quit IRC | 01:51 | |
| *** esberglu has joined #openstack-nova | 01:57 | |
| *** takashin has quit IRC | 01:57 | |
| mriedem_afk | takashin: i have a couple of comments in https://review.openstack.org/#/c/540256/ | 01:57 |
| mriedem_afk | if you can address and then i'll +W tomorrow | 01:57 |
| *** esberglu has quit IRC | 01:57 | |
| *** takashin has joined #openstack-nova | 01:57 | |
| *** felipemonteiro_ has quit IRC | 01:59 | |
| *** takashin has quit IRC | 01:59 | |
| *** takashin has joined #openstack-nova | 02:00 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add admin guide doc on volume multiattach support https://review.openstack.org/544090 | 02:02 |
| *** takashin has quit IRC | 02:05 | |
| *** takashin has joined #openstack-nova | 02:05 | |
| *** chyka has joined #openstack-nova | 02:09 | |
| *** takashin has quit IRC | 02:09 | |
| *** takashin has joined #openstack-nova | 02:09 | |
| *** zhaochao has quit IRC | 02:09 | |
| *** itlinux has quit IRC | 02:10 | |
| *** chyka has quit IRC | 02:13 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Amend the migration paging spec for uuid in server migrations response https://review.openstack.org/532904 | 02:16 |
| openstackgerrit | jichenjc proposed openstack/nova master: WIP: Add zvm admin doc https://review.openstack.org/533125 | 02:20 |
| openstackgerrit | jichenjc proposed openstack/nova master: WIP: Add zvm CI information https://review.openstack.org/533512 | 02:20 |
| *** zhaochao has joined #openstack-nova | 02:22 | |
| *** Dinesh_Bhor has quit IRC | 02:23 | |
| *** vivsoni has quit IRC | 02:23 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova-specs master: Amend volume multi-attach spec https://review.openstack.org/544152 | 02:28 |
| *** takashin has quit IRC | 02:28 | |
| *** takashin has joined #openstack-nova | 02:29 | |
| takashin | 02:31 | |
| *** itlinux has joined #openstack-nova | 02:32 | |
| *** slaweq has joined #openstack-nova | 02:33 | |
| *** mriedem_afk has quit IRC | 02:34 | |
| *** lbragstad has joined #openstack-nova | 02:35 | |
| *** slaweq has quit IRC | 02:37 | |
| *** Tom-Tom has joined #openstack-nova | 02:40 | |
| *** mdnadeem has quit IRC | 02:42 | |
| *** yamahata has quit IRC | 02:42 | |
| *** liusheng has quit IRC | 02:43 | |
| *** liusheng has joined #openstack-nova | 02:43 | |
| *** itlinux has quit IRC | 02:44 | |
| *** salv-orl_ has joined #openstack-nova | 02:45 | |
| *** salv-orlando has quit IRC | 02:48 | |
| *** suresh12 has quit IRC | 02:53 | |
| *** suresh12 has joined #openstack-nova | 03:00 | |
| *** takashin has quit IRC | 03:00 | |
| *** takashin has joined #openstack-nova | 03:01 | |
| *** suresh12 has quit IRC | 03:04 | |
| *** takashin has quit IRC | 03:06 | |
| *** takashin has joined #openstack-nova | 03:06 | |
| *** Tom-Tom has quit IRC | 03:15 | |
| *** takashin has quit IRC | 03:15 | |
| *** takashin has joined #openstack-nova | 03:15 | |
| openstackgerrit | jichenjc proposed openstack/nova master: fix race condition of instance host https://review.openstack.org/494458 | 03:19 |
| *** takashin has quit IRC | 03:19 | |
| *** takashin has joined #openstack-nova | 03:19 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova-specs master: WIP https://review.openstack.org/544158 | 03:23 |
| *** r-daneel has joined #openstack-nova | 03:23 | |
| *** takashin has quit IRC | 03:24 | |
| *** acormier has quit IRC | 03:28 | |
| *** harlowja has quit IRC | 03:30 | |
| *** acormier has joined #openstack-nova | 03:32 | |
| *** edmondsw has joined #openstack-nova | 03:34 | |
| *** takashin has joined #openstack-nova | 03:38 | |
| *** edmondsw has quit IRC | 03:39 | |
| *** acormier has quit IRC | 03:43 | |
| *** yamamoto has joined #openstack-nova | 03:43 | |
| *** Tom-Tom has joined #openstack-nova | 03:46 | |
| *** links has quit IRC | 03:50 | |
| *** Tom-Tom has quit IRC | 03:50 | |
| *** felipemonteiro has joined #openstack-nova | 03:54 | |
| *** sree has joined #openstack-nova | 03:59 | |
| *** udesale has joined #openstack-nova | 03:59 | |
| *** lbragstad has quit IRC | 04:02 | |
| *** links has joined #openstack-nova | 04:07 | |
| *** dave-mccowan has quit IRC | 04:10 | |
| *** harlowja has joined #openstack-nova | 04:12 | |
| *** takashin has quit IRC | 04:12 | |
| *** takashin has joined #openstack-nova | 04:12 | |
| *** efried1 has joined #openstack-nova | 04:20 | |
| *** efried has quit IRC | 04:22 | |
| *** efried1 is now known as efried | 04:22 | |
| openstackgerrit | jichenjc proposed openstack/nova master: WIP: fix validator confusing info https://review.openstack.org/544163 | 04:23 |
| *** takashin has quit IRC | 04:25 | |
| *** Tom-Tom has joined #openstack-nova | 04:29 | |
| *** harlowja has quit IRC | 04:30 | |
| *** slaweq has joined #openstack-nova | 04:34 | |
| *** suresh12 has joined #openstack-nova | 04:37 | |
| *** lpetrut has joined #openstack-nova | 04:38 | |
| *** slaweq has quit IRC | 04:38 | |
| *** suresh12 has quit IRC | 04:41 | |
| openstackgerrit | Arvind Nadendla proposed openstack/nova-specs master: Support traits in Glance https://review.openstack.org/541507 | 04:41 |
| *** felipemonteiro has quit IRC | 04:45 | |
| *** Tom-Tom has quit IRC | 04:45 | |
| *** abhishekk has joined #openstack-nova | 04:51 | |
| *** bhagyashri_s is now known as bhagyashris | 04:52 | |
| *** suresh12 has joined #openstack-nova | 04:54 | |
| *** hongbin has quit IRC | 04:55 | |
| *** psachin` has joined #openstack-nova | 05:04 | |
| *** psachin has joined #openstack-nova | 05:04 | |
| *** suresh12 has quit IRC | 05:04 | |
| *** dillaman has joined #openstack-nova | 05:08 | |
| *** jdillaman has quit IRC | 05:09 | |
| *** liverpoo1er has quit IRC | 05:09 | |
| *** lpetrut has quit IRC | 05:11 | |
| *** Tom-Tom has joined #openstack-nova | 05:12 | |
| *** takashin has joined #openstack-nova | 05:13 | |
| *** harlowja has joined #openstack-nova | 05:14 | |
| *** Tom-Tom has quit IRC | 05:17 | |
| *** moshele has joined #openstack-nova | 05:17 | |
| *** lpetrut has joined #openstack-nova | 05:19 | |
| *** Tom-Tom has joined #openstack-nova | 05:20 | |
| *** Tom-Tom has quit IRC | 05:21 | |
| *** claudiub has joined #openstack-nova | 05:21 | |
| *** Tom-Tom has joined #openstack-nova | 05:21 | |
| *** vladikr has quit IRC | 05:24 | |
| *** vladikr has joined #openstack-nova | 05:25 | |
| *** moshele has quit IRC | 05:27 | |
| *** lpetrut has quit IRC | 05:29 | |
| *** jichen has quit IRC | 05:32 | |
| *** tbachman has joined #openstack-nova | 05:33 | |
| *** jichen has joined #openstack-nova | 05:33 | |
| *** sridharg has joined #openstack-nova | 05:34 | |
| *** vladikr has quit IRC | 05:35 | |
| *** janki has joined #openstack-nova | 05:36 | |
| *** takashin has quit IRC | 05:39 | |
| *** takashin has joined #openstack-nova | 05:39 | |
| openstackgerrit | jichenjc proposed openstack/nova master: Avoid live migrate to same host https://review.openstack.org/542689 | 05:40 |
| *** lbragstad has joined #openstack-nova | 05:42 | |
| *** bhujay has joined #openstack-nova | 05:42 | |
| *** acormier has joined #openstack-nova | 05:43 | |
| *** psachin has quit IRC | 05:45 | |
| *** psachin` has quit IRC | 05:45 | |
| *** hiro-kobayashi has joined #openstack-nova | 05:46 | |
| *** acormier has quit IRC | 05:48 | |
| *** psachin has joined #openstack-nova | 05:51 | |
| *** takashin has quit IRC | 05:54 | |
| *** takashin has joined #openstack-nova | 05:55 | |
| *** tbachman has quit IRC | 05:55 | |
| *** slaweq has joined #openstack-nova | 05:59 | |
| *** slaweq has quit IRC | 06:04 | |
| *** takashin has quit IRC | 06:14 | |
| *** takashin has joined #openstack-nova | 06:14 | |
| *** takashin has quit IRC | 06:16 | |
| *** takashin has joined #openstack-nova | 06:17 | |
| *** psachin has quit IRC | 06:20 | |
| *** takashin has quit IRC | 06:21 | |
| *** takashin has joined #openstack-nova | 06:22 | |
| *** masber has joined #openstack-nova | 06:22 | |
| *** tbachman has joined #openstack-nova | 06:22 | |
| *** harlowja has quit IRC | 06:24 | |
| *** psachin has joined #openstack-nova | 06:28 | |
| *** moshele has joined #openstack-nova | 06:30 | |
| openstackgerrit | jichenjc proposed openstack/nova master: Fix warning for user_id and project_id https://review.openstack.org/544180 | 06:32 |
| *** suresh12 has joined #openstack-nova | 06:43 | |
| *** Eran_Kuris has joined #openstack-nova | 06:43 | |
| *** suresh12 has quit IRC | 06:47 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova-specs master: Fix missing blueprint URLs https://review.openstack.org/542543 | 06:54 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova-specs master: Fix missing blueprint URLs https://review.openstack.org/542543 | 06:55 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova-specs master: List/show all server migration types https://review.openstack.org/540256 | 07:00 |
| *** belmoreira has joined #openstack-nova | 07:01 | |
| tonyb | stephenfin: You can remove your -W from https://review.openstack.org/#/c/538954/1 when you're online :) | 07:04 |
| *** jchhatbar has joined #openstack-nova | 07:05 | |
| *** janki has quit IRC | 07:09 | |
| *** edmondsw has joined #openstack-nova | 07:11 | |
| *** jchhatbar is now known as janki | 07:14 | |
| *** edmondsw has quit IRC | 07:15 | |
| *** AlexeyAbashkin has joined #openstack-nova | 07:16 | |
| *** lbragstad has quit IRC | 07:17 | |
| *** alexchadin has joined #openstack-nova | 07:17 | |
| *** AlexeyAbashkin has quit IRC | 07:21 | |
| *** AlexeyAbashkin has joined #openstack-nova | 07:26 | |
| *** takashin has left #openstack-nova | 07:30 | |
| *** mdnadeem has joined #openstack-nova | 07:30 | |
| openstackgerrit | Merged openstack/nova master: Bindep does not catch missing libpcre3-dev on Ubuntu https://review.openstack.org/542340 | 07:31 |
| *** ragiman has joined #openstack-nova | 07:37 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/541561 | 07:40 |
| *** lpetrut has joined #openstack-nova | 07:42 | |
| *** lajoskatona has joined #openstack-nova | 07:44 | |
| *** slaweq has joined #openstack-nova | 07:48 | |
| *** andreas_s has joined #openstack-nova | 07:48 | |
| *** slaweq has quit IRC | 07:53 | |
| *** slaweq has joined #openstack-nova | 07:54 | |
| *** pcaruana has joined #openstack-nova | 07:56 | |
| *** bhagyashri_s has joined #openstack-nova | 07:56 | |
| *** pooja-jadhav has joined #openstack-nova | 07:58 | |
| *** nsingh has joined #openstack-nova | 07:58 | |
| *** niraj_singh has quit IRC | 07:59 | |
| *** bhagyashris has quit IRC | 07:59 | |
| *** pooja_jadhav has quit IRC | 07:59 | |
| *** slaweq has quit IRC | 07:59 | |
| *** Tom-Tom has quit IRC | 08:04 | |
| *** masber has quit IRC | 08:05 | |
| *** slaweq has joined #openstack-nova | 08:05 | |
| *** slaweq has quit IRC | 08:10 | |
| *** damien_r has joined #openstack-nova | 08:13 | |
| *** jpena|off is now known as jpena | 08:16 | |
| *** amoralej|off is now known as amoralej | 08:20 | |
| *** yamamoto has quit IRC | 08:21 | |
| *** yamamoto has joined #openstack-nova | 08:22 | |
| *** tesseract has joined #openstack-nova | 08:28 | |
| *** bhagyashris_ has joined #openstack-nova | 08:28 | |
| *** bhagyashris_ has quit IRC | 08:28 | |
| *** pooja_jadhav has joined #openstack-nova | 08:30 | |
| *** niraj_singh has joined #openstack-nova | 08:31 | |
| *** pooja-jadhav has quit IRC | 08:31 | |
| *** bhagyashris has joined #openstack-nova | 08:31 | |
| *** stakeda has quit IRC | 08:31 | |
| *** nsingh has quit IRC | 08:32 | |
| *** bhagyashri_s has quit IRC | 08:32 | |
| *** rcernin has quit IRC | 08:33 | |
| *** slaweq has joined #openstack-nova | 08:40 | |
| *** mgoddard_ has joined #openstack-nova | 08:43 | |
| *** slaweq has quit IRC | 08:44 | |
| *** sshwarts has joined #openstack-nova | 08:47 | |
| *** tetsuro has left #openstack-nova | 08:48 | |
| openstackgerrit | Claudiu Belu proposed openstack/nova master: compute: Cleans up allocations after failed resize https://review.openstack.org/543971 | 08:49 |
| *** ttx has joined #openstack-nova | 08:50 | |
| *** ralonsoh has joined #openstack-nova | 08:52 | |
| *** links has quit IRC | 08:53 | |
| hrw | morning | 08:56 |
| *** hiro-kobayashi has quit IRC | 09:03 | |
| *** tssurya has joined #openstack-nova | 09:05 | |
| *** ociuhandu has joined #openstack-nova | 09:05 | |
| *** links has joined #openstack-nova | 09:06 | |
| *** yamahata has joined #openstack-nova | 09:07 | |
| *** ociuhandu has quit IRC | 09:10 | |
| *** jichen has quit IRC | 09:10 | |
| *** slaweq has joined #openstack-nova | 09:18 | |
| *** jangutter has quit IRC | 09:20 | |
| gibi | gmann_: thanks for the ML post, I put my 2 cents in a reply. | 09:21 |
| *** janki has quit IRC | 09:22 | |
| *** slaweq has quit IRC | 09:23 | |
| *** slaweq has joined #openstack-nova | 09:24 | |
| *** yamahata has quit IRC | 09:24 | |
| gmann_ | gibi: thanks, i am not sure about moving from integrated-job but over time we can check those if creating issues | 09:27 |
| gmann_ | gibi: and on irrelevant_files issue, i reply on that your ML thread with additional query. i think avoiding the irrelevant_files in base job/template solve the most of the issue. | 09:27 |
| *** slaweq_ has joined #openstack-nova | 09:27 | |
| *** Tom-Tom has joined #openstack-nova | 09:28 | |
| *** slaweq has quit IRC | 09:28 | |
| *** yamahata has joined #openstack-nova | 09:31 | |
| gibi | gmann_: I'm not sure. The integrated-gate list jobs without irrelevant-files and that means those jobs will run on ever patch and we cannot override that later in project-config or in-tree | 09:37 |
| gibi | gmann_: at least that how I understood the problem in https://review.openstack.org/#/c/538908/ | 09:38 |
| gmann_ | gibi: no, we can still define irrelevant-files in-tree. if those job does not have irrelevant-files in base definition | 09:38 |
| gibi | gmann_: then I'm still confused about the problem in https://review.openstack.org/#/c/538908/ :) | 09:39 |
| *** derekh has joined #openstack-nova | 09:40 | |
| gibi | gmann_: what I see there is that we list tempest-full job twice, once in the integrated-gate template without irrelevant-files and once in the project-config with irrelevant-files | 09:40 |
| gibi | gmann_: still the tempest-full is triggered on every patches as the above two listings are understood as two different possible triggers for the job | 09:41 |
| gibi | gmann_: and the definition in integrated-gate matches to every patch (as no irrelevant-files define there) | 09:41 |
| *** yikun_jiang has quit IRC | 09:45 | |
| *** umbSublime has quit IRC | 09:48 | |
| gmann_ | gibi: yea that seems the behavior. | 09:50 |
| gmann_ | gibi: but that is opposite for 'branch' case, which get overridden from template to pipeline list definition - https://review.openstack.org/#/c/542484/2/zuul.d/projects.yaml | 09:51 |
| gmann_ | gibi: i got your point on https://review.openstack.org/#/c/538908/ | 09:54 |
| gmann_ | gibi: after your fix, i ll move those complete bits to nova-tree and there we control and do not need to use template if they create issues. | 09:56 |
| *** alexchadin has quit IRC | 09:56 | |
| *** alexchadin has joined #openstack-nova | 09:57 | |
| gibi | gmann_: ohh, so branch behaves differently, I did not know that | 10:03 |
| *** Tom-Tom has quit IRC | 10:04 | |
| stephenfin | tonyb: Done :) | 10:05 |
| *** pcaruana has quit IRC | 10:05 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/pike: libvirt: Block swap volume attempts with encrypted volumes https://review.openstack.org/543569 | 10:08 |
| gmann_ | gibi: but issue is same i think, zuul found it in 2 place and run on stable branch also as it did not find any branch restriction on project pipeline list | 10:08 |
| *** sree has quit IRC | 10:09 | |
| *** sree has joined #openstack-nova | 10:10 | |
| gibi | gmann_: I see. Then we cannot simply disallow defining branch and irrelevant-files for jobs in project-templates | 10:10 |
| gibi | gmann_: we either have a job in the templates that has irrelevant-files and branch definition that is good for every porject using that template, or we cannot have jobs in the template | 10:11 |
| openstackgerrit | Merged openstack/osc-placement stable/queens: Update UPPER_CONSTRAINTS_FILE for stable/queens https://review.openstack.org/537697 | 10:11 |
| *** cdent has joined #openstack-nova | 10:12 | |
| gmann_ | gibi: +1. totally agree. as we cannot have generic irrelevant-files for each project i think we remove the integrated-job as you mentioned in ML | 10:12 |
| *** sree has quit IRC | 10:15 | |
| *** slaweq_ is now known as slaweq | 10:16 | |
| gibi | gmann_: cool. As I mentioned I'm happy to help with the actual work as well if needed | 10:17 |
| gmann_ | gibi: replied. | 10:17 |
| gmann_ | gibi: thanks that will be helpful. | 10:17 |
| gmann_ | gibi: ll coordinate with you once get final confirmation from on ML | 10:18 |
| gibi | gmann_: thanks | 10:20 |
| *** pcaruana has joined #openstack-nova | 10:20 | |
| * cdent waves good morning | 10:20 | |
| * gibi waves back | 10:21 | |
| *** zhaochao has quit IRC | 10:24 | |
| * bauzas forgot to wave earlier, but since morning greetings are subjective for everyone, then... | 10:24 | |
| bauzas | good morning Nova | 10:25 |
| *** owalsh is now known as owalsh_afk | 10:28 | |
| cdent | lately it feels like morning (pre-coffee) all the time | 10:28 |
| *** lucas-pto is now known as lucasagomes | 10:30 | |
| bauzas | I feel always undercaffeinated | 10:33 |
| *** yamahata has quit IRC | 10:34 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Block swapping to an encrypted volume when using QEMU to decrypt https://review.openstack.org/544238 | 10:34 |
| gibi | I have some coffee beans covered in chocolate nearby all the time, that helps with my 'balanced' sugar and caffein intake | 10:36 |
| openstackgerrit | Hironori Shiina proposed openstack/nova-specs master: Ironic: Instance switchover https://review.openstack.org/449155 | 10:37 |
| cdent | gibi is wise | 10:37 |
| bauzas | we have something called "Green chaud" here | 10:39 |
| bauzas | https://www.chartreuse.fr/en/recettes/green-chaud/ | 10:39 |
| gmann_ | gibi: bauzas cdent enjoy your morning coffee, it's dinner time here :). | 10:39 |
| gibi | gmann_: actually I'm also closer to lunch than morning coffee. Enjoy your dinner | 10:40 |
| *** ttsiouts_ has joined #openstack-nova | 10:41 | |
| gibi | bauzas: that drink looks to me that somethin that helps to got to sleep at the end of a long day | 10:42 |
| bauzas | gibi: rather at the beginning of a skiing day | 10:42 |
| gibi | bauzas: interesting | 10:42 |
| cdent | a) looks tasty, b) agree it looks sleep, not ski, inducing | 10:43 |
| *** abhishekk has quit IRC | 10:44 | |
| *** hegemoOn has left #openstack-nova | 10:46 | |
| *** edmondsw has joined #openstack-nova | 10:47 | |
| *** sambetts|afk is now known as sambetts | 10:48 | |
| *** slaweq_ has joined #openstack-nova | 10:50 | |
| *** priteau has joined #openstack-nova | 10:52 | |
| *** edmondsw has quit IRC | 10:52 | |
| ameeda | Hi, just a question about gerrit review, I have 2 nova reviews waiting for reviewers (one of them got +2c, waiting for +2w) and another one doesn't have activities from reviewers from awhile , do you follow scheduler to check them or I have to ask you to check them ? | 10:55 |
| *** slaweq_ has quit IRC | 10:55 | |
| cdent | bauzas, gibi, stephenfin ^ | 10:58 |
| *** damien_r has quit IRC | 11:02 | |
| *** damien_r has joined #openstack-nova | 11:02 | |
| gibi | ameeda: it depends. For example priority feature work and high and critical bug fixes tend to get attention automatically the rest of the patches might need some encuragement | 11:04 |
| *** AlexeyAbashkin has quit IRC | 11:05 | |
| gibi | ameeda: also I suggest to try to communicate why something is important / needs attention instead of just pinging people with reviews. It helps us reviewers to prioritize properly | 11:05 |
| gibi | ameeda: when I have time I sometimes look at random reviews on gerrit having a clean CI run and some code review | 11:07 |
| gibi | ameeda: but most of the time I have a list of things to make progress with | 11:07 |
| openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: RP list: member_of and resources parameters (v1.3, v1.4) https://review.openstack.org/511183 | 11:19 |
| openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: RP delete inventories (v1.5) https://review.openstack.org/514642 | 11:19 |
| openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI for traits (v1.6) https://review.openstack.org/514643 | 11:19 |
| openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: Resource class set (v1.7) https://review.openstack.org/514644 | 11:19 |
| openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: Usages per project and user (v1.8, v1.9) https://review.openstack.org/514646 | 11:19 |
| openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: CLI allocation candidates (v1.10) https://review.openstack.org/514647 | 11:19 |
| openstackgerrit | Andrey Volkov proposed openstack/osc-placement master: PUT dict format to /allocations (v1.11, v1.12) https://review.openstack.org/542819 | 11:19 |
| bauzas | cdent: https://bugs.launchpad.net/nova/+bug/1749410 | 11:26 |
| openstack | Launchpad bug 1749410 in OpenStack Compute (nova) "placement api-ref unclear if capacity is meant to be total or current" [Wishlist,Confirmed] | 11:26 |
| openstackgerrit | Claudiu Belu proposed openstack/nova master: compute: Cleans up allocations after failed resize https://review.openstack.org/543971 | 11:26 |
| * cdent looks | 11:26 | |
| cdent | bauzas, hmmm, yeah, not entirely clear what the best way to clarify that is. It might be useful for there to be separate glossary of placement terms, so that the tables in the api-ref don't get too clogged up | 11:28 |
| bauzas | cdent: anyway, just FYI | 11:28 |
| * cdent nods | 11:28 | |
| *** pcaruana has quit IRC | 11:30 | |
| mdbooth | Hmm, instance snapshot is root disk only. | 11:32 |
| *** udesale_ has joined #openstack-nova | 11:36 | |
| *** sree has joined #openstack-nova | 11:36 | |
| bauzas | cdent: yet again some good point https://bugs.launchpad.net/nova/+bug/1749404 | 11:37 |
| openstack | Launchpad bug 1749404 in OpenStack Compute (nova) "nova-compute resource tracker ignores 'reserved' while reporting 'max_unit'" [Undecided,New] | 11:37 |
| *** alexchadin has quit IRC | 11:38 | |
| *** udesale has quit IRC | 11:38 | |
| *** AlexeyAbashkin has joined #openstack-nova | 11:38 | |
| *** sree has quit IRC | 11:40 | |
| cdent | bauzas: yup, that is a good point. I suppose an argument could be made that "it depends on allocation ratio" but I think the intent is as stated in the bug. | 11:40 |
| bauzas | cdent: I'll mark it as Wontfix | 11:40 |
| bauzas | it's a semantic point of view | 11:41 |
| cdent | no, I think it should be fixed | 11:41 |
| *** udesale has joined #openstack-nova | 11:41 | |
| *** udesale_ has quit IRC | 11:41 | |
| bauzas | cdent: if so, reopen the bug with your comment please | 11:41 |
| bauzas | I'm fine with reopening it | 11:41 |
| cdent | if allocation_ratio is one and reserved is > 0 then it is wrong | 11:41 |
| cdent | will do | 11:41 |
| *** moshele has quit IRC | 11:42 | |
| *** pcaruana has joined #openstack-nova | 11:42 | |
| *** alexchadin has joined #openstack-nova | 11:47 | |
| *** udesale has quit IRC | 11:47 | |
| *** sree has joined #openstack-nova | 11:51 | |
| openstackgerrit | Chen Hanxiao proposed openstack/nova master: libvirt: remove TODO on validation of scsi model https://review.openstack.org/525055 | 11:52 |
| *** cdent has quit IRC | 11:56 | |
| *** sree has quit IRC | 11:56 | |
| *** dave-mccowan has joined #openstack-nova | 11:56 | |
| *** ttsiouts_ has quit IRC | 11:57 | |
| *** sree has joined #openstack-nova | 11:58 | |
| openstackgerrit | Chen Hanxiao proposed openstack/nova master: libvirt: add Linux distribution guest only description for inject_xxx options https://review.openstack.org/528441 | 12:00 |
| *** sree_ has joined #openstack-nova | 12:00 | |
| *** sree_ is now known as Guest18968 | 12:01 | |
| *** sree has quit IRC | 12:03 | |
| *** elmaciej has joined #openstack-nova | 12:11 | |
| *** slaweq_ has joined #openstack-nova | 12:11 | |
| *** slaweq_ has quit IRC | 12:16 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements https://review.openstack.org/544277 | 12:18 |
| *** slaweq_ has joined #openstack-nova | 12:19 | |
| *** patriciadomin_ is now known as patriciadomin | 12:21 | |
| *** dtantsur|afk is now known as dtantsur | 12:21 | |
| *** slaweq_ has quit IRC | 12:23 | |
| *** yamamoto has quit IRC | 12:23 | |
| *** yamamoto has joined #openstack-nova | 12:24 | |
| *** cdent has joined #openstack-nova | 12:26 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-novaclient master: Updated from global requirements https://review.openstack.org/544282 | 12:30 |
| *** sdague has joined #openstack-nova | 12:30 | |
| *** moshele has joined #openstack-nova | 12:30 | |
| *** edmondsw has joined #openstack-nova | 12:36 | |
| *** moshele has quit IRC | 12:40 | |
| *** edmondsw has quit IRC | 12:40 | |
| *** moshele has joined #openstack-nova | 12:43 | |
| *** acormier has joined #openstack-nova | 12:46 | |
| *** edmondsw has joined #openstack-nova | 12:50 | |
| *** udesale has joined #openstack-nova | 12:51 | |
| *** jpena is now known as jpena|lunch | 12:51 | |
| *** r-daneel has quit IRC | 12:54 | |
| *** takedakn has joined #openstack-nova | 12:56 | |
| *** mvk_ has quit IRC | 12:58 | |
| *** weshay_PTO is now known as weshay | 13:01 | |
| *** takedakn has quit IRC | 13:01 | |
| *** r-daneel has joined #openstack-nova | 13:04 | |
| *** Guest18968 has quit IRC | 13:09 | |
| *** sree has joined #openstack-nova | 13:10 | |
| *** acormier has quit IRC | 13:10 | |
| *** acormier has joined #openstack-nova | 13:10 | |
| *** sree has quit IRC | 13:15 | |
| *** yamamoto has quit IRC | 13:17 | |
| *** yamamoto has joined #openstack-nova | 13:18 | |
| openstackgerrit | Lajos Katona proposed openstack/osc-placement master: Random names for functional tests https://review.openstack.org/542745 | 13:19 |
| *** yamamoto has quit IRC | 13:22 | |
| *** lbragstad has joined #openstack-nova | 13:23 | |
| *** vladikr has joined #openstack-nova | 13:23 | |
| *** mvk_ has joined #openstack-nova | 13:24 | |
| *** priteau has quit IRC | 13:27 | |
| *** amoralej is now known as amoralej|lunch | 13:30 | |
| *** awaugama has joined #openstack-nova | 13:38 | |
| * cdent has no brain, gives up and takes black panther break | 13:41 | |
| *** cdent has quit IRC | 13:42 | |
| *** rmcall has quit IRC | 13:43 | |
| *** acormier has quit IRC | 13:45 | |
| *** hrw has quit IRC | 13:45 | |
| *** pchavva has joined #openstack-nova | 13:45 | |
| *** hrw has joined #openstack-nova | 13:46 | |
| *** yamamoto has joined #openstack-nova | 13:48 | |
| *** liverpooler has joined #openstack-nova | 13:50 | |
| *** jpena|lunch is now known as jpena | 13:50 | |
| *** owalsh_afk is now known as owalsh | 13:52 | |
| *** mriedem has joined #openstack-nova | 13:57 | |
| mriedem | happy valentines day everyone <3 | 13:58 |
| *** ttsiouts_ has joined #openstack-nova | 13:58 | |
| *** ttsiouts_ has quit IRC | 13:59 | |
| gibi | mriedem: same to you :) | 13:59 |
| openstackgerrit | Bence Romsics proposed openstack/nova master: Clarify 'capacity' in placement api-ref https://review.openstack.org/544347 | 14:02 |
| gibi | mriedem: I left two questions in https://review.openstack.org/#/c/543971/3/nova/compute/manager.py@4457 | 14:03 |
| *** andreas_s has quit IRC | 14:06 | |
| *** andreas_s has joined #openstack-nova | 14:06 | |
| *** andreas_s has quit IRC | 14:06 | |
| *** alexchadin has quit IRC | 14:08 | |
| *** alexchadin has joined #openstack-nova | 14:08 | |
| *** alexchadin has quit IRC | 14:08 | |
| *** alexchadin has joined #openstack-nova | 14:09 | |
| *** elmaciej has quit IRC | 14:09 | |
| mriedem | replied | 14:09 |
| *** hshiina is now known as hshiina|afk | 14:09 | |
| *** alexchadin has quit IRC | 14:09 | |
| *** alexchadin has joined #openstack-nova | 14:10 | |
| *** alexchadin has quit IRC | 14:10 | |
| *** alexchadin has joined #openstack-nova | 14:11 | |
| *** r-daneel has quit IRC | 14:11 | |
| *** Eran_Kuris has quit IRC | 14:11 | |
| *** alexchadin has quit IRC | 14:11 | |
| *** alexchadin has joined #openstack-nova | 14:11 | |
| *** alexchadin has quit IRC | 14:12 | |
| mriedem | ildikov: i realized yesterday that we never handled this https://review.openstack.org/#/c/544152/1/specs/queens/implemented/multi-attach-volume.rst@136 | 14:12 |
| *** alexchadin has joined #openstack-nova | 14:12 | |
| mriedem | ildikov: but then realized, the only volume types that rely on that, don't support multiattach, so it's not really a problem we need to worry about right now | 14:12 |
| *** alexchadin has quit IRC | 14:12 | |
| mriedem | things like NFS | 14:12 |
| ildikov | mriedem: good points, thanks for looking into it | 14:13 |
| ildikov | agreed on let's worry about it later | 14:14 |
| ildikov | do you think it should be a PTG micro topic or we're good for now? | 14:14 |
| mriedem | nah | 14:15 |
| mriedem | unless someone really cares about making NFS support multiattach | 14:15 |
| mriedem | which i doubt | 14:15 |
| mriedem | well, NFS + multiattach + snapshot | 14:15 |
| ildikov | yeah, let's not encourage people if don't have to :) | 14:16 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Test websocketproxy with TLS in the nova-next job https://review.openstack.org/513160 | 14:17 |
| *** bhujay has quit IRC | 14:17 | |
| *** eharney has joined #openstack-nova | 14:19 | |
| *** lucasagomes is now known as lucas-hungry | 14:21 | |
| ildikov | mriedem: I also didn't talk about shared_targets in the docs for now, wonder whether I should or not? | 14:22 |
| *** alexchadin has joined #openstack-nova | 14:22 | |
| mriedem | probably not necessary; nova uses them for locking if the microversion is available | 14:23 |
| mriedem | but it's not required | 14:23 |
| *** rmcall has joined #openstack-nova | 14:24 | |
| *** mdnadeem has quit IRC | 14:24 | |
| *** alexchadin has quit IRC | 14:26 | |
| ildikov | ok, fine, I'll leave it then | 14:28 |
| *** udesale has quit IRC | 14:29 | |
| *** jafeha__ has joined #openstack-nova | 14:29 | |
| ildikov | it's something more interesting for driver maintainers anyway so does not seem to be admin guide content, but wanted to check | 14:29 |
| *** r-daneel has joined #openstack-nova | 14:29 | |
| *** udesale has joined #openstack-nova | 14:29 | |
| *** jafeha has quit IRC | 14:30 | |
| *** amoralej|lunch is now known as amoralej | 14:31 | |
| *** links has quit IRC | 14:33 | |
| *** elmaciej has joined #openstack-nova | 14:34 | |
| *** udesale has quit IRC | 14:36 | |
| *** READ10 has joined #openstack-nova | 14:38 | |
| *** hshiina|afk has quit IRC | 14:41 | |
| *** vladikr has quit IRC | 14:47 | |
| *** lbragsta_ has joined #openstack-nova | 14:49 | |
| *** bhagyashris has quit IRC | 14:50 | |
| *** nsingh has joined #openstack-nova | 14:50 | |
| *** nsingh has quit IRC | 14:51 | |
| *** lbragstad has quit IRC | 14:51 | |
| *** bhagyashris has joined #openstack-nova | 14:52 | |
| *** jobewan has joined #openstack-nova | 14:52 | |
| *** pooja-jadhav has joined #openstack-nova | 14:53 | |
| *** yamamoto has quit IRC | 14:53 | |
| *** yamahata has joined #openstack-nova | 14:54 | |
| *** yamamoto has joined #openstack-nova | 14:54 | |
| *** pooja_jadhav has quit IRC | 14:55 | |
| openstackgerrit | Merged openstack/nova-specs master: Fix missing blueprint URLs https://review.openstack.org/542543 | 14:55 |
| *** niraj_singh has quit IRC | 14:55 | |
| mriedem | dansmith: i won't be around for the cells v2 meeting today (if there was going to be one) | 14:56 |
| *** yamamoto has quit IRC | 14:57 | |
| *** yamamoto has joined #openstack-nova | 14:57 | |
| openstackgerrit | Merged openstack/nova-specs master: List/show all server migration types https://review.openstack.org/540256 | 14:58 |
| *** ragiman has quit IRC | 14:58 | |
| dansmith | mriedem: ack | 14:59 |
| dansmith | I'm around for it, but I don't really have anything specific other than my reviews | 14:59 |
| *** vladikr has joined #openstack-nova | 15:00 | |
| *** alexchadin has joined #openstack-nova | 15:00 | |
| *** moshele has quit IRC | 15:02 | |
| *** moshele has joined #openstack-nova | 15:03 | |
| *** alexchadin has quit IRC | 15:03 | |
| -openstackstatus- NOTICE: Due to a race in stable/queens branch creation and some job removals, Zuul has reported syntax errors for the past hour; if you saw a syntax error reported for "Job tripleo-ci-centos-7-ovb-containers-oooq not defined" you can safely recheck now | 15:05 | |
| belmoreira | have some placement related questions. dansmith, mriedem do you have some time? | 15:07 |
| dansmith | belmoreira: I can try | 15:07 |
| belmoreira | thanks dansmith | 15:08 |
| belmoreira | we are running local placements in ocata. Trying to consolidate them and get some performance numbers. | 15:08 |
| belmoreira | But, when nodes are retired (removed from the cloud) they continue to exist in placement. Are they eventually removed? not seeing how | 15:08 |
| dansmith | no they're not | 15:08 |
| belmoreira | and they are included in the candidates? | 15:09 |
| mriedem | hmm, they probably are, | 15:09 |
| dansmith | they would be from placement yeah, but scheduler would filter them out | 15:09 |
| mriedem | similarly, we don't have a direct hook to remove entries from the compute_nodes table | 15:09 |
| belmoreira | we are seeing that placement continue to include them | 15:10 |
| dansmith | this isn't really any different from our compute nodes befoe placement | 15:10 |
| mriedem | belmoreira: when you retire a node, do you manually remove it's entry from the nova.compute_nodes table? | 15:10 |
| belmoreira | the difference is that the scheduling is now global | 15:10 |
| belmoreira | removing few hundred nodes could impact the performance of the entire cloud | 15:11 |
| *** acormier has joined #openstack-nova | 15:11 | |
| mriedem | what did you do when you removed them before this? | 15:11 |
| belmoreira | mriedem currently we remove entire cells, so we don't have this problem | 15:11 |
| mriedem | ok so you drop the entire nova db | 15:12 |
| mriedem | for that cell | 15:12 |
| belmoreira | correct | 15:12 |
| dansmith | hmm, and compute nodes don't get mirrored up? | 15:12 |
| *** moshele has quit IRC | 15:12 | |
| dansmith | I can't remember how that works, but maybe they don't | 15:12 |
| mriedem | we've talked before about adding a nova-manage CLI to remove a compute_nodes entry | 15:12 |
| belmoreira | dansmith what do you mean? | 15:12 |
| mriedem | the host_mappings table would be pointing at hosts that no longer exist too | 15:13 |
| dansmith | belmoreira: well, in cellsv1 a bunch of stuff gets mirrored up to the top db | 15:13 |
| dansmith | belmoreira: I would have expected compute_nodes would too, but I don't really know I guess | 15:13 |
| dansmith | it doesn't matter though | 15:13 |
| belmoreira | dansmith compute nodes are only in the child cells | 15:13 |
| dansmith | going forward, pruning will have to happen when you remove stuff | 15:13 |
| dansmith | ack | 15:14 |
| *** r-daneel has quit IRC | 15:14 | |
| belmoreira | is there any work already done in that? | 15:14 |
| dansmith | so one way to do that, I think, | 15:14 |
| mriedem | we likely could use a nova-manage CLI that removes the compute_nodes table entry in the cell, the host_mappings entry in the api db, and the resource_providers entry in placement | 15:14 |
| *** ragiman has joined #openstack-nova | 15:14 | |
| dansmith | would be to go through host_mappings, find any mappings that don't have a cell, remove them and tell placement about it | 15:14 |
| dansmith | mriedem: yeah, but if you remove a cell first, you'll need a way to clean up the mess too | 15:14 |
| tssurya | mriedem : yes that would be a useful command | 15:15 |
| mriedem | can we delete a cell mapping that has host mappings? | 15:15 |
| mriedem | don't we have a fkey? | 15:15 |
| dansmith | idk | 15:15 |
| mriedem | cell_id = Column(Integer, ForeignKey('cell_mappings.id'), | 15:15 |
| mriedem | nullable=False) | 15:15 |
| mriedem | and nullable=False | 15:16 |
| mriedem | yeah, so we have to remove host mappings if we remove a cell mapping | 15:16 |
| dansmith | but, | 15:16 |
| dansmith | you can delete a cell with delete_cell and it will nuke all the host mappings | 15:16 |
| dansmith | so then you're really toast | 15:16 |
| mriedem | if you use --force yeah | 15:16 |
| dansmith | so we have delete_host | 15:17 |
| *** AlexeyAbashkin has quit IRC | 15:17 | |
| dansmith | which unmaps the compute node, and deletes the hostmapping | 15:17 |
| dansmith | so in both of those I guess we need to update placement | 15:18 |
| mriedem | i'm not sure why delete_host requires a --cell_uuid arg, | 15:18 |
| mriedem | but that's unrelated | 15:18 |
| dansmith | although nova-manage doesn't do any rpc or calls to other services right now, AFAIK, so we kinda need a --no-placement flag to each of those in case you're operating in an island | 15:18 |
| mriedem | well, | 15:19 |
| *** lbragsta_ is now known as lbragstad | 15:19 | |
| mriedem | nova-manage could use the scheduler report client and handle an http error response if it can't talk to placement | 15:19 |
| dansmith | yes, but you dont' want to proceed then, | 15:19 |
| dansmith | unless you tell it to | 15:19 |
| dansmith | because if we can't talk to placement, we can't proceed without corrupting the linkage (as we are now) | 15:20 |
| dansmith | so you need to fail in that case, unless instructed to ignore | 15:20 |
| dansmith | someone with placement down during an upgrade pruning a host wants to know that they're about to orphan some placement records | 15:20 |
| mriedem | could just throw another --force on delete_host | 15:20 |
| dansmith | we need it for both though | 15:20 |
| dansmith | delete_cell and delete_host | 15:20 |
| mriedem | sure, we have --force on delete_cell already | 15:21 |
| dansmith | and delete_cell has --force already which means something else though | 15:21 |
| dansmith | it means "delete all the things, yeah", but this would be "and also, don't delete some stuff" | 15:21 |
| mriedem | ha | 15:21 |
| dansmith | right? | 15:21 |
| mriedem | it's just confusing ux | 15:21 |
| *** cdent has joined #openstack-nova | 15:21 | |
| dansmith | but really, they're two meanings of force | 15:21 |
| dansmith | what we have now is really --recursive | 15:22 |
| *** r-daneel has joined #openstack-nova | 15:22 | |
| *** lpetrut has quit IRC | 15:23 | |
| stephenfin | dansmith: This NUMA aware vSwitch spec is basically turning into a book. Be prepared! | 15:23 |
| dansmith | regardless, we need that thing and some appropriate flaggage to control intent | 15:23 |
| dansmith | stephenfin: noted | 15:23 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old flavor_get_by_name db api method https://review.openstack.org/544379 | 15:24 |
| *** tidwellr has joined #openstack-nova | 15:24 | |
| dansmith | belmoreira: tssurya: can one of you open a bug for this so I can point a patch at it? | 15:24 |
| tssurya | dansmith : sure | 15:24 |
| belmoreira | dansmith yes we can do it, we can also work on it if you prefer | 15:24 |
| dansmith | or that ;) | 15:25 |
| mriedem | dansmith: in the scenario you said, "someone with placement down during an upgrade pruning a host wants to know that they're about to orphan some placement records" - if i'm removing a compute host (node really here), are you just saying that you want to *not* delete the compute node if placement is down, right? | 15:25 |
| *** mlavalle has joined #openstack-nova | 15:25 | |
| dansmith | mriedem: right | 15:25 |
| mriedem | default behavior would be, we try to remove from placement and if that fails, we fail, unless instructed to ignore it | 15:25 |
| dansmith | mriedem: unless they say "I know what I'm doing" or something | 15:25 |
| dansmith | mriedem: yes | 15:26 |
| mriedem | that still seems like --force should cover that case to me | 15:26 |
| dansmith | but they may want --force to mean "delete all the hosts too", | 15:26 |
| dansmith | but then they would be opting into "and orphan all the data in placement if it's down" | 15:26 |
| dansmith | so let's say you're deleting a cell while you're down for an upgrade, and that cell has 10 hosts | 15:27 |
| mriedem | well, --force is a destructive option | 15:27 |
| dansmith | it is but it's not a "and corrupt my data" | 15:27 |
| dansmith | so you're down and deleting a cell with ten hosts | 15:27 |
| dansmith | you run delete_cell --force, to delete all ten hosts and the cell record | 15:27 |
| dansmith | if placement is down, | 15:27 |
| dansmith | you now just orphaned the placement records for those ten hosts, | 15:28 |
| dansmith | so when you come back up, ten providers in placement are no longer present, but will be returned by placement forever, | 15:28 |
| dansmith | and unless you know what the uuids were of those records you deleted, | 15:28 |
| dansmith | you can't really clean that up | 15:28 |
| openstackgerrit | Merged openstack/nova-specs master: Update Queens NRP spec to reflect reality https://review.openstack.org/544086 | 15:28 |
| dansmith | our current --force is poorly named.. it should be --recursive | 15:29 |
| * cdent checks into getting dansmith an "astuteness merit badge" | 15:29 | |
| dansmith | because --recursive means "and delete everything you have to to delete the cell" | 15:29 |
| dansmith | cdent: do you not get what I'm saying here? | 15:29 |
| cdent | No, I totally get it. I'm not kidding. It is an astute observation. | 15:30 |
| dansmith | okay | 15:30 |
| mriedem | i get the scenario, i just, idk | 15:30 |
| *** alexchadin has joined #openstack-nova | 15:30 | |
| cdent | one way around the problem is "don't have placement down" | 15:30 |
| cdent | which is probably a good rule of thumb anyway | 15:30 |
| *** lbragstad has quit IRC | 15:31 | |
| dansmith | mriedem: in belmoreira's case, he would run --force to delete all the hosts, and then he would be in exactly the same position that started this conersation | 15:31 |
| dansmith | cdent: aye, it'd be better to not allow skipping the placement cleanup at all | 15:31 |
| mriedem | dansmith: did belmoreira say placement was down? | 15:31 |
| *** lucas-hungry is now known as lucasagomes | 15:31 | |
| dansmith | mriedem: no, but he'd want to use delete_cell --force to delete the cell and all the hosts, | 15:31 |
| *** hongbin has joined #openstack-nova | 15:31 | |
| dansmith | but if placement was down when doing that (because maintenance), then we'd orphan all the records that he's currently saying we orphan, which is a problem | 15:32 |
| dansmith | i.e. you can't opt into the recursiveness without choosing the "and also corrupt the data if you need to" | 15:32 |
| *** felipemonteiro has joined #openstack-nova | 15:33 | |
| *** dklyle has joined #openstack-nova | 15:33 | |
| mriedem | so we could just say, you can't do that if placement is down, but sure i agree people will want a backdoor | 15:33 |
| mriedem | to bypass that | 15:33 |
| *** lbragstad has joined #openstack-nova | 15:33 | |
| *** david-lyle has quit IRC | 15:33 | |
| *** slaweq has quit IRC | 15:33 | |
| dansmith | right, which is why I'm saying we need another flag | 15:33 |
| dansmith | since we already burned --force to mean --recursive, we need something else | 15:33 |
| mriedem | --ignore-placement? | 15:34 |
| mriedem | --fudge-it? | 15:34 |
| *** slaweq has joined #openstack-nova | 15:34 | |
| *** lajoskatona has quit IRC | 15:34 | |
| dansmith | yeah, I said --no-placement above, but exactly.. something like that | 15:34 |
| *** gibi has quit IRC | 15:34 | |
| mriedem | apparently --force doesn't also remove the instances/mappings | 15:35 |
| tssurya | mriedem : I have a patch for that | 15:35 |
| tssurya | in gerrit | 15:36 |
| dansmith | yeah and I think I'm +2 on it already right? | 15:36 |
| mriedem | force deleting instances from nova-manage is pretty scary | 15:36 |
| mriedem | you're likely orphaning ports, volumes, etc | 15:36 |
| tssurya | mriedem : no --force does not delete instances | 15:36 |
| mriedem | just the mappings? | 15:36 |
| *** felipemonteiro_ has joined #openstack-nova | 15:36 | |
| tssurya | mriedem : it only removes/cleans mappings for deleted instances | 15:36 |
| tssurya | yes | 15:36 |
| tssurya | just mappings | 15:36 |
| dansmith | ah, nm was thinking of the other one | 15:36 |
| *** breton has joined #openstack-nova | 15:37 | |
| tssurya | we under no circumstance delete a cell if there are instances | 15:37 |
| mriedem | i'm saying delete_cell --force will fail today if there are instance mappings in the cell | 15:37 |
| mriedem | ok this patch https://review.openstack.org/#/c/540073/ | 15:37 |
| tssurya | mriedem : yes , my after merging my patch, delete_cell will fail if there any living instances | 15:38 |
| *** slaweq has quit IRC | 15:38 | |
| mriedem | ok i see | 15:39 |
| jaypipes | stephenfin: https://review.openstack.org/#/c/538217/3/specs/rocky/approved/integrate-mypy-type-checking.rst <-- well, sheeet, let's just rewrite Nova in Golang (or Rust). :P | 15:39 |
| *** links has joined #openstack-nova | 15:40 | |
| stephenfin | jaypipes: If you can find me a tool to autoconvert one language to another, I'm all for it ;) | 15:40 |
| *** felipemonteiro has quit IRC | 15:40 | |
| mriedem | vish suggested java after joining oracle | 15:40 |
| mriedem | i know java, not go or rust | 15:41 |
| jaypipes | stephenfin: :) | 15:41 |
| belmoreira | a concern that I also have with placement is that it doesn't know anything about cells, that is now an official think in nova | 15:41 |
| mnaser | i've been looking through the nova codebase throughout this morning -- we're running into an issue where nova fails to boot an instance "Invalid volume: Volume status must be available to reserve, but the status is attaching. (HTTP 400)" | 15:41 |
| jaypipes | I'd be up for a rewrite in C++. | 15:41 |
| mriedem | belmoreira: placement shouldn't know about cells | 15:41 |
| belmoreira | we map projects to cells. Meaning that to handle a "small" VM the placement will return "all" nodes and then is the scheduler filtering to find the right cell | 15:41 |
| stephenfin | jaypipes: I do suggest checking out the sample patches I've thrown up through. An afternoons work and it really is minimally invasive | 15:41 |
| belmoreira | is there a way to get a more filtered answer from placement? | 15:42 |
| jaypipes | stephenfin: will do :) | 15:42 |
| mnaser | it seems like *somehow*, the cinder volume is in attaching state, and nova tries to reserve it, and so it fails after | 15:42 |
| *** AlexeyAbashkin has joined #openstack-nova | 15:42 | |
| stephenfin | jaypipes: and whatever you do, _don't_ listen to mdbooth! :D | 15:42 |
| jaypipes | heh | 15:42 |
| mriedem | mnaser: this is pike 16.0.4? | 15:42 |
| dansmith | belmoreira: aggregates | 15:42 |
| dansmith | belmoreira: that's how you get a more filtered answer | 15:42 |
| mdbooth | stephenfin: Generally good advice. | 15:42 |
| * mdbooth wonders why specifically | 15:42 | |
| belmoreira | dansmith placement aggregates? | 15:42 |
| stephenfin | mdbooth: https://review.openstack.org/#/c/538217/3/specs/rocky/approved/integrate-mypy-type-checking.rst | 15:42 |
| dansmith | belmoreira: aye | 15:42 |
| mdbooth | stephenfin: Haha | 15:43 |
| mnaser | mriedem: i believe the control plane is stable/pike, i think computes might be a bit older i'm not sure if they're 16.0.4 | 15:43 |
| jaypipes | dansmith: aggregate filtering isn't supported yet in allocation candidates... | 15:43 |
| jaypipes | dansmith: or at least, I don't think it is.. | 15:43 |
| *** felipemonteiro_ has quit IRC | 15:43 | |
| * jaypipes checks | 15:43 | |
| dansmith | jaypipes: I'm saying that's the plan | 15:43 |
| mnaser | we're running into a lot of weird conditions with heat | 15:43 |
| jaypipes | ah | 15:43 |
| belmoreira | dansmith can they the used in Pike? | 15:43 |
| dansmith | jaypipes: we're not asking for it now anyway | 15:43 |
| mdbooth | stephenfin: Spit shine that turd ;) | 15:43 |
| dansmith | belmoreira: no | 15:43 |
| *** felipemonteiro_ has joined #openstack-nova | 15:43 | |
| mnaser | rather heavy heat usage of create volume + attach to instance | 15:43 |
| mriedem | mnaser: so this is boot from volume - is the volume provided to nova, or is nova creating the volume? | 15:43 |
| mriedem | i.e. does the failure happen in nova-api or nova-compute? | 15:43 |
| mnaser | mriedem: boot from volume, volume created by heat and then given to bdm_v2 | 15:44 |
| belmoreira | dansmith are there any docs that describes how to use them? can find a lot of info about them | 15:44 |
| mnaser | http://paste.openstack.org/show/672350/ <-- pretty much this heat template | 15:45 |
| * mriedem has never used heat | 15:45 | |
| jaypipes | I prefer a cold press for pain. | 15:45 |
| dansmith | belmoreira: it's not going to solve your immediate problem, I'm just saying that aggregates, resources, and traits are the ways you can slice the result from placement (not cells) | 15:46 |
| mnaser | :P | 15:46 |
| jaypipes | :) | 15:46 |
| mriedem | mnaser: ok likely getting here https://github.com/openstack/nova/blob/stable/pike/nova/compute/api.py#L1347 | 15:46 |
| mnaser | technically speaking heat is creating a volume and then giving it to nova (so boot from volume with provided volume id) | 15:46 |
| jaypipes | mnaser: well, that heat template isn't exactly complicated... | 15:46 |
| dansmith | belmoreira: tbh, I'm not sure how to address your case of wanting to pre-filter or slice hosts by tenant | 15:46 |
| openstackgerrit | Aditya Vaja proposed openstack/nova master: remove IVS plug/unplug as they're moved to separate plugin https://review.openstack.org/534371 | 15:47 |
| mnaser | mriedem: yeah that's as far as i got, but it seems that once it gets there, the volume is already in attaching state.. so something kicked off the attach somewhere | 15:47 |
| belmoreira | dansmith thanks, we will continue to dig into this | 15:47 |
| mnaser | i can see the volume stuck in 'attaching' state right now in cinder still | 15:47 |
| mriedem | mnaser: do you see anything in the cinder logs for that volume id before nova tries to reserve it? | 15:48 |
| dansmith | belmoreira: let me think on it a bit | 15:48 |
| mnaser | mriedem: good question, i should have thought of that | 15:48 |
| belmoreira | dansmith but for Pike the only option will be the have a "normal" aggregate per cell and let the filter scheduler do the job. concerned about the performance though | 15:48 |
| mriedem | mnaser: also this is likely something you want in pike https://github.com/openstack/nova/commit/6d1877bf1da098b3be17fbd2dd5e53eecd2f048c | 15:49 |
| mriedem | should be in 16.1.0 | 15:49 |
| dansmith | belmoreira: yeah, although you can have more schedulers and parallelize that load | 15:49 |
| mnaser | oh that's interesting, that might be related | 15:50 |
| belmoreira | yes, that's the plan. but in the worst case scenario each scheduler will need to go through 9k nodes | 15:50 |
| *** itlinux has joined #openstack-nova | 15:50 | |
| belmoreira | that's why in ocata i decided to have local placements | 15:50 |
| mriedem | mnaser: yeah you might be hiting this https://github.com/openstack/nova/blob/stable/pike/nova/compute/api.py#L1406 | 15:51 |
| dansmith | belmoreira: aye, well, might need a hack for that case in pike until we think of something else | 15:51 |
| mriedem | so nova isn't actually trying to call cinder to reserve the volume | 15:51 |
| openstackgerrit | Eric Berglund proposed openstack/nova master: Add check for redundant import aliases https://review.openstack.org/543995 | 15:51 |
| mriedem | it's doing it's own conditional state check and failure | 15:51 |
| mriedem | *failing | 15:51 |
| mriedem | although i don't know why the volume status would be 'attaching' | 15:51 |
| *** tidwellr has quit IRC | 15:51 | |
| mnaser | interestingly enough i see a POST to /action | 15:52 |
| *** esberglu has joined #openstack-nova | 15:52 | |
| dansmith | belmoreira: what's the reasoning for that split? was it for scheduling initially, or security or hardware ownership? | 15:52 |
| *** tidwellr has joined #openstack-nova | 15:53 | |
| belmoreira | dansmith manly specific hardware for a use-case | 15:53 |
| dansmith | belmoreira: okay well that should really be a trait thing (although that's not done either :) | 15:54 |
| mnaser | ok so i see a volume create in cinder-volume | 15:54 |
| mriedem | dansmith: well, | 15:54 |
| belmoreira | dansmith until now we isolate the hardware type per cell and allocate the right project to it | 15:54 |
| mriedem | we do have traits-based scheduling, | 15:54 |
| mriedem | so you could have per-project flavors that have a trait for the hw you want for those projects | 15:55 |
| dansmith | mriedem: we're not passing it to placement yet right? | 15:55 |
| mriedem | we are | 15:55 |
| mriedem | in queens | 15:55 |
| dansmith | in pike? | 15:55 |
| dansmith | right okay | 15:55 |
| mriedem | oh | 15:55 |
| *** lajoskatona has joined #openstack-nova | 15:55 | |
| mriedem | yeah not pike | 15:55 |
| mriedem | queens solves all problems, i swear | 15:55 |
| dansmith | maybe they could backport | 15:55 |
| mriedem | the flavor plumbing stuff isn't versioned in anyway, the placement apis to use it are, | 15:55 |
| cdent | dansmith: traits in non-neste scenarios is working in queens last I checked | 15:55 |
| mriedem | but you could upgrade placement on the side, and backport the nova plumbing | 15:56 |
| belmoreira | dansmith i will have a look into that | 15:56 |
| mriedem | cdent: cern is on ocata | 15:56 |
| mriedem | queens placement + ocata nova (+patches) should be ok | 15:56 |
| mnaser | ok so | 15:56 |
| mnaser | it looks nova reserved the volume twice | 15:56 |
| mriedem | *should* == test in pre-prod please | 15:56 |
| mnaser | reserved it the first time succesfully, then shortly after it got reserved again | 15:57 |
| belmoreira | mriedem :) | 15:57 |
| cdent | mriedem: yeah, I know, it just sounded like dan was saying it's _still_ not done | 15:57 |
| cdent | and for some limited definitions of "done" it is, a bit | 15:57 |
| dansmith | cdent: well, tbh I didn't think we were even passing it in allocation_candidates :) | 15:58 |
| dansmith | I just hadn't been paying enough attention | 15:58 |
| mriedem | i know https://github.com/openstack/nova/commit/477a14e81e8c69b91a39395d78d845c8603e10fd#diff-b4a2926790df668d47bf16d43128c4cf | 15:58 |
| mnaser | so 12 seconds later, cinder got a second reserve request for the same volume and thats why it failed (i guess a volume goes to attaching state when its being reserved) | 15:58 |
| * mnaser looks more | 15:58 | |
| mriedem | mnaser: yeah it does, | 15:58 |
| mriedem | so i'm guessing the 2nd reserve request was made by nova-compute... | 15:59 |
| mriedem | looking | 15:59 |
| mnaser | im gonna investigate and see if that theory adds up | 15:59 |
| mriedem | this is where compute starts doing the attach work https://github.com/openstack/nova/blob/stable/pike/nova/compute/manager.py#L1595 | 15:59 |
| belmoreira | thanks all | 16:00 |
| mriedem | mnaser: so what's probably happening is it's hitting an ocata compute | 16:00 |
| mriedem | which calls check_attach again, and fails | 16:00 |
| mriedem | https://github.com/openstack/nova/blob/stable/ocata/nova/virt/block_device.py#L250 | 16:00 |
| mriedem | mnaser: that's why the API has that min compute service version check, but that was busted with the multi-cell cells v2 stuff, unless you have melwitt's patch | 16:01 |
| mriedem | mnaser: so i think that's your issue | 16:01 |
| mriedem | can you verify the compute it failed on is running ocata? | 16:01 |
| mnaser | mriedem: that's what i'm double checing | 16:01 |
| mnaser | checking* | 16:01 |
| *** alexchadin has quit IRC | 16:02 | |
| mnaser | mriedem: i wonder if that was the wsgi registration bug we ran into a while back | 16:05 |
| mnaser | "MariaDB [nova]> select * from services where deleted_at is null order by version;" shows everything at 22 except for 3 control plane services | 16:05 |
| mnaser | "nova-osapi_compute" | 16:05 |
| mnaser | which is at version 16 | 16:05 |
| mriedem | mnaser: the problem is w/o that patch, the version comes back at 0 | 16:05 |
| mriedem | mnaser: your nova-api config is probably pointing [database] at cell0? | 16:06 |
| mnaser | mriedem: oh you're right that's an old record, updated_at '2017-09-02 15:34:18' | 16:06 |
| mriedem | if your api config is pointing the [database] entry at nova_cell0, there are no nova-compute services in cell0, so it will return 0 as the version | 16:07 |
| mriedem | and hit that else block in the api code | 16:07 |
| mriedem | in other words, the api wasn't properly iterating non-cell0 cells for nova-compute service versions | 16:07 |
| mnaser | makes a lot of sense | 16:08 |
| mriedem | so that api patch will make your stuff to the right thing while you still have ocata computes | 16:09 |
| mnaser | we shouldn't have ocata computes so im still tracking down that issue right now | 16:09 |
| *** pcaruana has quit IRC | 16:13 | |
| *** yamamoto has quit IRC | 16:13 | |
| mnaser | mriedem: i think i might have another theory.. heat creates a volume, tries to create an instance, instance fails creation for some reason (but after reserving volume), it retries, and nova fails | 16:19 |
| mnaser | because im seeing a POST to /os-volumes_boot returning 400 | 16:19 |
| mriedem | hmm | 16:20 |
| mnaser | ill have to dig up heat logs | 16:20 |
| mriedem | mnaser: yeah we apparently don't unreserve the volume in the api if something fails later in the api | 16:21 |
| mriedem | or in compute | 16:22 |
| mnaser | interesting.. i see heat doing the create twice in logs 10 seconds apart, let me see if i can find an error there | 16:23 |
| mnaser | but yeah, it's masking a failure | 16:23 |
| mriedem | does heat delete the failed instance (which reserved the volume) before trying to create the 2nd instance? | 16:23 |
| mnaser | digging into that now | 16:23 |
| mriedem | because that would likely be related to https://review.openstack.org/#/c/340614/ | 16:24 |
| *** lajoskatona has quit IRC | 16:24 | |
| mnaser | ok there was a scheduler error (not enough hosts), i'll dig into that | 16:24 |
| *** suresh12 has joined #openstack-nova | 16:24 | |
| mnaser | "Task pause timed out" | 16:25 |
| mnaser | and then it tried to create again | 16:25 |
| mriedem | i don't know what that is, | 16:25 |
| mriedem | but for the scheduler novalidhost, | 16:25 |
| mriedem | the api would reserve the volume, rpc cast to conductor which rpc calls the scheduler for a host, that raises NoValidHost, and the instance is put into ERROR state, and the volume would still be reserved | 16:25 |
| mnaser | the "Task pause timed out" is a heat thing so i'll look into why it didn't attempt to delete before retry | 16:25 |
| mriedem | so if heat deletes the failed instance, nova can clean up the volume, but that's probably where we need https://review.openstack.org/#/c/340614/ i think | 16:26 |
| mnaser | mriedem: i think heat's normal behaviour is to just set the stack to failed | 16:27 |
| mnaser | and not retry | 16:27 |
| *** yamamoto has joined #openstack-nova | 16:27 | |
| mnaser | googling "Task pause timed out" yields nothing so that's a nice start, but we're in heat problem land | 16:27 |
| mriedem | ok, but even if heat did try to delete the instance before retrying, the 2nd attempt will likely fail because the volume is already reserved because of the bug that https://review.openstack.org/#/c/340614/ is trying to fix | 16:28 |
| *** belmoreira has quit IRC | 16:28 | |
| *** psachin has quit IRC | 16:29 | |
| *** tidwellr has quit IRC | 16:29 | |
| cfriesen | does anyone know off the top of their head which operations would cause an instance with a qcow2-backed rootfs (that normally has the glance image file as a backing file) to get flattened? | 16:30 |
| *** tidwellr has joined #openstack-nova | 16:30 | |
| mnaser | mriedem: yeah that's ideal, looks like melwitt was doing work on that, i can pick it up and work on whats missing if melwitt is busy with other things | 16:31 |
| *** yamamoto has quit IRC | 16:31 | |
| mriedem | mnaser: it's ready for review, just needs review | 16:32 |
| mriedem | that patch has a long history | 16:32 |
| mriedem | of merging, breaking hte gate, getting reverted, repeat | 16:32 |
| *** suresh12 has quit IRC | 16:32 | |
| mnaser | mriedem: there was a comment suggesting removing instance.host is None that's why i asked | 16:33 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old flavor_access_get_by_flavor_id db api method https://review.openstack.org/539720 | 16:33 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old flavor_destroy db api method https://review.openstack.org/539729 | 16:33 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old flavor_get_by_flavor_id db api method https://review.openstack.org/539733 | 16:33 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old flavor_get_by_name db api method https://review.openstack.org/544379 | 16:33 |
| *** mriedem is now known as mriedem_afk | 16:34 | |
| *** slaweq has joined #openstack-nova | 16:34 | |
| *** slaweq_ has joined #openstack-nova | 16:36 | |
| *** imacdonn has quit IRC | 16:37 | |
| *** imacdonn has joined #openstack-nova | 16:37 | |
| *** tssurya has quit IRC | 16:39 | |
| *** slaweq has quit IRC | 16:39 | |
| *** alexchadin has joined #openstack-nova | 16:40 | |
| *** slaweq_ has quit IRC | 16:40 | |
| *** tssurya has joined #openstack-nova | 16:41 | |
| *** itlinux has quit IRC | 16:41 | |
| *** elmaciej has quit IRC | 16:42 | |
| *** krtaylor has joined #openstack-nova | 16:43 | |
| *** alexchadin has quit IRC | 16:44 | |
| cfriesen | mriedem: do you know of any work in progress to deal with https://bugs.launchpad.net/nova/+bug/1583145 or https://bugs.launchpad.net/nova/+bug/1644248 ? | 16:47 |
| openstack | Launchpad bug 1644248 in OpenStack Compute (nova) ocata "duplicate for #1583145 Nova incorrectly tracks live migration progress" [High,In progress] - Assigned to Matt Riedemann (mriedem) | 16:47 |
| openstack | Launchpad bug 1644248 in OpenStack Compute (nova) ocata "Nova incorrectly tracks live migration progress" [High,In progress] - Assigned to Matt Riedemann (mriedem) | 16:47 |
| *** hemna_ has joined #openstack-nova | 16:47 | |
| jaypipes | mriedem_afk, dansmith: https://review.openstack.org/#/c/539712/ is pretty straightforward. | 16:50 |
| *** READ10 has quit IRC | 16:51 | |
| *** tssurya has quit IRC | 16:51 | |
| *** itlinux has joined #openstack-nova | 16:52 | |
| *** slaweq has joined #openstack-nova | 16:56 | |
| *** ragiman has quit IRC | 16:56 | |
| *** rmcall has quit IRC | 16:58 | |
| dansmith | melwitt: mriedem_afk isn't around for the cells meeting, we talked to tssurya and belmiro earlier and neither are online now, so I assume we're canceling | 16:59 |
| *** slaweq has quit IRC | 17:00 | |
| *** slaweq_ has joined #openstack-nova | 17:00 | |
| *** efried is now known as efried_rollin | 17:02 | |
| *** lpetrut has joined #openstack-nova | 17:03 | |
| *** rmcall has joined #openstack-nova | 17:03 | |
| *** READ10 has joined #openstack-nova | 17:04 | |
| *** slaweq_ has quit IRC | 17:05 | |
| *** jgriffith has joined #openstack-nova | 17:05 | |
| *** ameeda has quit IRC | 17:06 | |
| *** jgriffith has quit IRC | 17:07 | |
| *** AlexeyAbashkin has quit IRC | 17:07 | |
| *** rmcall has quit IRC | 17:08 | |
| *** beagles is now known as beagles_brb | 17:08 | |
| *** jgriffith has joined #openstack-nova | 17:08 | |
| *** dtantsur is now known as dtantsur|afk | 17:09 | |
| *** gyee has joined #openstack-nova | 17:11 | |
| *** chyka has joined #openstack-nova | 17:12 | |
| *** yamamoto has joined #openstack-nova | 17:12 | |
| cdent | jaypipes: if you're chasing easy wins: https://review.openstack.org/#/c/536784/ | 17:13 |
| *** AlexeyAbashkin has joined #openstack-nova | 17:15 | |
| *** alexchadin has joined #openstack-nova | 17:16 | |
| *** pcaruana has joined #openstack-nova | 17:17 | |
| *** yamamoto has quit IRC | 17:17 | |
| *** AlexeyAbashkin has quit IRC | 17:20 | |
| *** links has quit IRC | 17:21 | |
| *** slaweq has joined #openstack-nova | 17:22 | |
| *** tssurya has joined #openstack-nova | 17:24 | |
| *** tssurya has quit IRC | 17:27 | |
| *** yamahata has quit IRC | 17:31 | |
| *** beagles_brb is now known as beagles | 17:33 | |
| *** mgoddard_ has quit IRC | 17:37 | |
| *** felipemonteiro__ has joined #openstack-nova | 17:45 | |
| *** jmlowe has quit IRC | 17:46 | |
| *** felipemonteiro_ has quit IRC | 17:48 | |
| *** lpetrut has quit IRC | 17:48 | |
| *** jmlowe has joined #openstack-nova | 17:49 | |
| *** alexchadin has quit IRC | 17:50 | |
| openstackgerrit | Merged openstack/python-novaclient master: Updated from global requirements https://review.openstack.org/544282 | 17:52 |
| *** moshele has joined #openstack-nova | 17:52 | |
| *** arvindn05 has quit IRC | 17:52 | |
| *** sshwarts has quit IRC | 17:54 | |
| *** rmcall has joined #openstack-nova | 17:56 | |
| *** lpetrut has joined #openstack-nova | 17:56 | |
| *** sambetts is now known as sambetts|afk | 17:56 | |
| *** moshele has quit IRC | 17:58 | |
| *** jmlowe has quit IRC | 18:00 | |
| *** derekh has quit IRC | 18:03 | |
| *** hoonetorg has quit IRC | 18:05 | |
| *** yamahata has joined #openstack-nova | 18:06 | |
| *** mvk_ has quit IRC | 18:08 | |
| mnaser | so it looks like this bug wasn't visible because we had old disabled but inactive pre-pike compute nodes | 18:09 |
| mnaser | removing them made the minimum pike and landed us in this situation | 18:09 |
| *** sridharg has quit IRC | 18:10 | |
| openstackgerrit | Dan Smith proposed openstack/nova-specs master: Add placement-req-filter spec https://review.openstack.org/544585 | 18:15 |
| *** mgoddard_ has joined #openstack-nova | 18:16 | |
| *** jackie-truong has joined #openstack-nova | 18:16 | |
| mnaser | "The missing 'reserve_volume' call is added to the BFV flow. In case of build failure the volume will be locked in 'attaching' state until the instance in ERROR state is cleaned up." -- i guess the problem here is that it doesn't account for scheduling failures :x | 18:16 |
| *** jpena is now known as jpena|off | 18:17 | |
| *** cdent has quit IRC | 18:21 | |
| *** hoonetorg has joined #openstack-nova | 18:22 | |
| *** suresh12 has joined #openstack-nova | 18:24 | |
| *** psachin has joined #openstack-nova | 18:26 | |
| *** jmlowe has joined #openstack-nova | 18:26 | |
| *** yamamoto has joined #openstack-nova | 18:27 | |
| *** suresh12 has quit IRC | 18:29 | |
| *** felipemonteiro__ has quit IRC | 18:29 | |
| openstackgerrit | Merged openstack/nova master: Updated from global requirements https://review.openstack.org/544277 | 18:31 |
| *** alexchadin has joined #openstack-nova | 18:32 | |
| *** yamamoto has quit IRC | 18:32 | |
| openstackgerrit | Dan Smith proposed openstack/nova-specs master: Add placement-req-filter spec https://review.openstack.org/544585 | 18:32 |
| mriedem_afk | mnaser: i want to say that "so it looks like this bug wasn't visible because we had old disabled but inactive pre-pike compute nodes" was also a bug fix | 18:32 |
| *** mriedem_afk is now known as mriedem | 18:32 | |
| *** openstackgerrit has quit IRC | 18:33 | |
| mriedem | mnaser: oh nvm we excluded forced_down services, but we do include disabled services in that min version check | 18:33 |
| mriedem | i was thinking of this so it looks like this bug wasn't visible because we had old disabled but inactive pre-pike compute nodes | 18:34 |
| *** suresh12 has joined #openstack-nova | 18:34 | |
| mriedem | oops | 18:34 |
| mriedem | https://github.com/openstack/nova/commit/c79770e615799cd4457ac603dcad4fb3452fe2bc | 18:34 |
| mnaser | mriedem: yeah so right now i'm not too sure why this occurs because it looks like if it fails to schedule, it means when nova attempts to delete it, it does a local delete, which calls '_local_cleanup_bdm_volumes' | 18:35 |
| mnaser | or maybe im wrong? | 18:36 |
| *** alexchadin has quit IRC | 18:36 | |
| mnaser | because if it fails to schedule, its in cell0 so deletes will be a local delete based on what i think i understand from the code | 18:36 |
| mriedem | mnaser: presumably that's why melwitt has https://review.openstack.org/#/c/340614/ | 18:36 |
| mriedem | i think the problem is that we do find the error'ed instance in cell0, | 18:38 |
| mriedem | and delete it in the api, but return early and don't cleanup bdms or ports | 18:38 |
| mriedem | here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L1805 | 18:39 |
| mriedem | and then we exit here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L1809 | 18:39 |
| mnaser | mriedem: i dont know if logically this is clean but i was thinking maybe cleanup bdms in the _bury_in_cell0 codebase? | 18:42 |
| mnaser | because that will also release the volume right after failure, rather than until the volume is deleted | 18:42 |
| mriedem | yeah idk, we end up with a lot of cleanup volume logic strewn all over nova then | 18:43 |
| mriedem | b/c we already have some of this in nova-compute too | 18:43 |
| mriedem | and api already | 18:43 |
| mnaser | yeah thats what was making it a bit iffy for me is the fact that now multiple components are handling reserve/cleanup (which is probably how we ended up in this place) | 18:44 |
| mnaser | mriedem: i think what you mentioned isn't the case, because that 'return' happens if there is an instance.host, and in this case, the error'd instance probably doesnt have one set if it failed to be scheduled | 18:47 |
| mnaser | but thats what i think based on what little understanding of the code | 18:48 |
| melwitt | if the instance fails to schedule, it shouldn't have done anything with the volumes yet, right? | 18:48 |
| mnaser | melwitt: the api layer calls cinder reserve volume when it gets the request | 18:49 |
| melwitt | mnaser: oh :\ | 18:49 |
| mnaser | melwitt: not sure if your patch fixes it (or if its technically another bug), but this was my write up - http://paste.openstack.org/show/672535/ | 18:50 |
| *** jpena|off is now known as jpena | 18:50 | |
| mriedem | mnaser: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L1787 | 18:50 |
| mriedem | we only go down that route if the instance doesn't have a host yet | 18:51 |
| mnaser | mriedem: my bad, the comment above it tricked me | 18:51 |
| mriedem | yeah, it's confusing, the "Normal delete should be attempted." means "skip this weird block" | 18:51 |
| mriedem | so i think that's likely the issue, is we get the instance from cell0, and delete it in this block of code and return w/o dealing with volumes/ports | 18:52 |
| mriedem | granted we shouldn't have any ports if it's never been scheduled to a host | 18:52 |
| mriedem | as ports are created/bound in compute | 18:52 |
| mriedem | but volumes is definitely a gap here | 18:52 |
| mnaser | mriedem: i think so too, because if i follow the rest of the code, it should technically do a local_delete which does clean up bdms | 18:52 |
| mriedem | yup | 18:52 |
| *** yamamoto has joined #openstack-nova | 18:52 | |
| *** yamamoto has quit IRC | 18:52 | |
| *** tesseract has quit IRC | 18:54 | |
| mnaser | catch-22... if compute reserves volume, it means api layer won't give the error right away.. if api reserves volume, get an error right away but have to get it cleaned up later | 18:55 |
| *** lucasagomes is now known as lucas-afk | 18:56 | |
| *** eharney has quit IRC | 18:57 | |
| *** jpena is now known as jpena|off | 18:59 | |
| mnaser | im not even sure if this issue exists in queens | 19:01 |
| mnaser | looks like it creates an attachment rather than reserve a volume | 19:01 |
| *** claudiub has quit IRC | 19:02 | |
| mriedem | dansmith: so i think i get it, but https://review.openstack.org/#/c/544585/ | 19:02 |
| mriedem | mnaser: it does exist in queens, because we don't delete the attachment, same thing as not unreserving | 19:03 |
| mriedem | a volume with an empty attachment is in 'attaching' status just like reserve | 19:03 |
| dansmith | mriedem: in my playing around code, I have a note to self to go find that key and see if I can use it :) | 19:03 |
| melwitt | mnaser: with my patch, I think you'd still see the volume stuck in 'attaching' until you delete the instance | 19:03 |
| mriedem | melwitt: yes you would b/c we don't cleanup volumes in conductor | 19:04 |
| melwitt | without the patch, even after you delete the instance, the volume would be stuck. with the patch, the volume should be released | 19:04 |
| melwitt | after deleting | 19:04 |
| mnaser | melwitt: so _has_been_scheduled(instance) would return true (because vm_state == ERROR), which means that we would enter that `if not has_been_scheduled:` conditional. however, as mriedem pointed out, in your patch, line #1809 it returns so it doesn't follow the rest of the code | 19:06 |
| mnaser | which means it never makes it to the code under it which does clean ups etc (if i understand correctly) | 19:06 |
| *** suresh12 has quit IRC | 19:06 | |
| mriedem | it should be fairly easy to have a functional test to recreate that | 19:07 |
| *** alexchadin has joined #openstack-nova | 19:07 | |
| melwitt | mnaser: if _has_been_scheduled is True, then it would not enter 'if not _has_been_scheduled' | 19:07 |
| *** suresh12 has joined #openstack-nova | 19:07 | |
| mnaser | gah | 19:08 |
| mriedem | the functional tests that melwitt updated in that patch likely aren't doing boot from volume | 19:08 |
| mnaser | these programming 101 mistakes | 19:08 |
| * mnaser needs coffee | 19:08 | |
| *** lpetrut has quit IRC | 19:10 | |
| *** suresh12 has quit IRC | 19:10 | |
| melwitt | I have boot from volume unit tests, but need to check if any of the functional tests covering boot from volume could be enhanced to check this | 19:10 |
| *** suresh12 has joined #openstack-nova | 19:10 | |
| melwitt | originally I hadn't updated any functional tests in the patch, something changed recently to where we needed to manually start the consoleauth service. not yet sure what caused that change | 19:11 |
| *** tidwellr has quit IRC | 19:11 | |
| mriedem | we don't have many bfv functional tests, but a couple | 19:11 |
| mriedem | local delete will try to delete console auth entries | 19:11 |
| *** tidwellr has joined #openstack-nova | 19:11 | |
| mriedem | since you weren't going down that path before your change, that's why those tests didn't need it | 19:12 |
| mriedem | here is an easy example test https://review.openstack.org/#/c/543257/ | 19:12 |
| melwitt | yeah, I mean, in previous revisions I didn't need it | 19:12 |
| mnaser | i think `test_delete_error_state_with_no_host` covers this case, which makes sure that local_delete is actually being called (and local delete does clean up bdms) | 19:12 |
| mriedem | mnaser: that's a unit test | 19:13 |
| mriedem | we really should have a functional regression test for this, given the various service interactions and cell0 stuff | 19:13 |
| mnaser | is there an example of functional regression tests involving other services (i.e. cinder) | 19:14 |
| mriedem | https://review.openstack.org/#/c/543257/ | 19:14 |
| mnaser | i dont mind trying to write one if that helps | 19:14 |
| mnaser | melwitt: if you have other things at hand, i can write a functional regression test for this right now | 19:16 |
| *** suresh12_ has joined #openstack-nova | 19:17 | |
| melwitt | mnaser: sure, sounds good, thanks for the help | 19:17 |
| mnaser | ok perfect, i'll work on it locally and then i can tack it on top of the change | 19:17 |
| *** suresh12 has quit IRC | 19:20 | |
| mriedem | ideally the regression test patch comes before the fix, | 19:21 |
| mriedem | to show that the volume is still 'attaching' even after we delete the instance | 19:21 |
| mriedem | then mel's patch goes on top of that and changes the test to show the volume is properly unreserved | 19:21 |
| *** harlowja has joined #openstack-nova | 19:21 | |
| mriedem | the cinder fixture we have should be able to give you the volume state tracking and API support for figuring that out | 19:21 |
| mnaser | https://git.openstack.org/cgit/openstack/nova/tree/nova/tests/functional/regressions/test_bug_1675570.py | 19:24 |
| mnaser | well this is nice to have, heh | 19:25 |
| *** openstackgerrit has joined #openstack-nova | 19:26 | |
| openstackgerrit | Dan Smith proposed openstack/nova-specs master: Add placement-req-filter spec https://review.openstack.org/544585 | 19:26 |
| *** READ10 has quit IRC | 19:31 | |
| mriedem | mnaser: yeah you could probably build on that with a bfv scenario, and nix the shelve offload part | 19:33 |
| *** jackie-truong has quit IRC | 19:33 | |
| *** dklyle has quit IRC | 19:33 | |
| *** mgoddard_ has quit IRC | 19:34 | |
| *** moshele has joined #openstack-nova | 19:34 | |
| openstackgerrit | Dan Smith proposed openstack/nova-specs master: Add placement-req-filter spec https://review.openstack.org/544585 | 19:36 |
| mriedem | dansmith: couple small docs issues and then i'm +2 | 19:37 |
| *** ralonsoh has quit IRC | 19:38 | |
| mriedem | i really like the idea of using this for multiattach bfv | 19:38 |
| mriedem | assuming people don't hate the idea of storing the compute driver capabilities as traits | 19:39 |
| mriedem | we can also use it for tagged bfv | 19:39 |
| mriedem | which artom should like | 19:39 |
| *** alexchadin has quit IRC | 19:39 | |
| * artom would indeed | 19:40 | |
| artom | I hope I wouldn't be the only one | 19:40 |
| artom | It seems that sort of thing has come up for other problems | 19:40 |
| dansmith | mriedem: jaypipes was also saying this will let us clean up the pci request -> nets mapping thing, which sounds nice | 19:40 |
| dansmith | I have to go dig back into that stuff to see though | 19:40 |
| openstackgerrit | Dan Smith proposed openstack/nova-specs master: Add placement-req-filter spec https://review.openstack.org/544585 | 19:41 |
| openstackgerrit | Jackie Truong proposed openstack/nova master: Add trusted_certs to instance_extra https://review.openstack.org/537897 | 19:45 |
| openstackgerrit | Jackie Truong proposed openstack/nova master: Add trusted_certs object https://review.openstack.org/489408 | 19:45 |
| openstackgerrit | Jackie Truong proposed openstack/nova master: Implement certificate_utils https://review.openstack.org/479949 | 19:45 |
| openstackgerrit | Jackie Truong proposed openstack/nova master: Add trusted_image_certificates to REST API https://review.openstack.org/486204 | 19:45 |
| *** suresh12_ has quit IRC | 19:47 | |
| *** moshele has quit IRC | 19:49 | |
| mriedem | jackie should be in irc | 19:49 |
| openstackgerrit | melanie witt proposed openstack/nova-specs master: Re-propose convert consoles code to use objects framework https://review.openstack.org/543662 | 19:50 |
| *** efried_rollin is now known as efried | 19:50 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old flavor_get db api method https://review.openstack.org/544621 | 19:53 |
| *** yamamoto has joined #openstack-nova | 19:53 | |
| dansmith | mriedem: related to ^, I'll remove aggregate-in-cell-db compatibility before I add the methods to AggregateList that I need | 19:54 |
| dansmith | just so you don't do it if you were looking for deprecations to remove | 19:54 |
| mriedem | dansmith: i had just been slowly working on this list https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L385 | 19:59 |
| mriedem | last thing i removed was the flavor compat | 19:59 |
| mriedem | or maybe it was keypairs, don't remmeber | 19:59 |
| dansmith | mriedem: okay well, I've got the two aggregate ones in my sights | 19:59 |
| mriedem | ok wfm, | 19:59 |
| *** eharney has joined #openstack-nova | 19:59 | |
| mriedem | we can definitely remove these old newton compat things | 19:59 |
| mriedem | s/compat/migration/ | 19:59 |
| dansmith | yeah | 20:00 |
| *** felipemonteiro has joined #openstack-nova | 20:02 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add periodic task to clean expired console tokens https://review.openstack.org/325381 | 20:02 |
| openstackgerrit | melanie witt proposed openstack/nova master: Use ConsoleAuthToken object to generate authorizations https://review.openstack.org/325414 | 20:02 |
| openstackgerrit | melanie witt proposed openstack/nova master: Convert websocketproxy to use db for token validation https://review.openstack.org/333990 | 20:02 |
| *** yamamoto has quit IRC | 20:02 | |
| *** tidwellr has quit IRC | 20:03 | |
| mriedem | anyone want to send this home and the one below it? https://review.openstack.org/#/c/544066/ - the cinder admin guide patch for multiattach is dependent on this | 20:03 |
| mriedem | i think we should probably backport that to queens too | 20:04 |
| *** amoralej is now known as amoralej|off | 20:07 | |
| *** tidwellr has joined #openstack-nova | 20:07 | |
| mnaser | hey .. is this a bug in the cinder fixtures? https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L1390-L1394 .. shouldn't it modify volume['attach_status'] and not volume['status']? | 20:07 |
| mnaser | so it should be volume['status'] = 'in-use' if has_attachment else 'available' | 20:07 |
| mnaser | volume['attach_status'] = 'attached' if has_attachment else 'detached' | 20:08 |
| mriedem | jogo: can you remove your -2 on this? https://review.openstack.org/#/c/159382/1 | 20:08 |
| mnaser | my little functional test so far is failing because: HTTP exception thrown: Invalid volume: volume 'f8580fcb-b22d-4e50-b69a-d3d91f8f3a6b' status must be 'available'. Currently in 'detached' | 20:08 |
| mriedem | mnaser: yes | 20:08 |
| mnaser | ok, ill fix that first | 20:08 |
| mriedem | mnaser: same problem here https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L1574 | 20:08 |
| melwitt | mnaser: is that what I proposed here? https://review.openstack.org/#/c/539758/ | 20:09 |
| mnaser | melwitt: exactly that | 20:09 |
| mnaser | ill rebasemy work on that, thank you | 20:09 |
| mriedem | lemme look, we could rebase quick | 20:09 |
| melwitt | I ran into the same problem recently working on func tests | 20:09 |
| mnaser | melwitt: do we want to add in-use for volume status too? | 20:09 |
| mnaser | but that might be a bit more work | 20:10 |
| mnaser | because im not sure what the status is when using multiattach (in-use or available?) | 20:10 |
| melwitt | mnaser: we might. I didn't try to go further with changing the fixture | 20:10 |
| mriedem | i just left that comment | 20:10 |
| mriedem | if there are attachments, the status should be at least 'in-use' | 20:11 |
| melwitt | once it's attached, it should be 'in-use' from what I understand | 20:11 |
| mriedem | it might not be technically correct, but it's more correct than saying 'available' | 20:11 |
| mnaser | yeah, makes sense | 20:11 |
| mriedem | with the new fixture, there can be 'empty' attachments, which puts the volume status in 'attaching' status, and the volume doesn't go to in-use until the attachment is 'completed' | 20:11 |
| mriedem | in the old fixture, completed means calling os-attach | 20:11 |
| mriedem | but we likely don't need to worry about those details in these fixtures | 20:12 |
| melwitt | mnaser: you want to add 'in-use' and 'available' to that patch while you're rebasing? | 20:12 |
| mnaser | melwitt: sure, i can do that | 20:12 |
| melwitt | cool, thanks | 20:12 |
| *** david-lyle has joined #openstack-nova | 20:12 | |
| mriedem | actually, the loop above deals with this already | 20:13 |
| mriedem | so likely don't need a change | 20:13 |
| mriedem | i don't even know why we have this block of code now | 20:14 |
| mriedem | the loop above handles both of those status values | 20:14 |
| openstackgerrit | Peter Hamilton proposed openstack/nova-specs master: Add support for certificate validation https://review.openstack.org/540879 | 20:16 |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Account for host agg allocation ratio in placement https://review.openstack.org/544683 | 20:20 |
| *** alexchadin has joined #openstack-nova | 20:21 | |
| *** amodi has joined #openstack-nova | 20:24 | |
| *** alexchadin has quit IRC | 20:25 | |
| openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: Snapshot https://review.openstack.org/543023 | 20:25 |
| *** felipemonteiro_ has joined #openstack-nova | 20:27 | |
| *** felipemonteiro has quit IRC | 20:30 | |
| cfriesen | mriedem: are you aware of any work going on as far as live migration monitoring? (related to the broken progress monitor) | 20:30 |
| mnaser | sorry for the questions, is there a way to turn on debug logging for functional tests? | 20:31 |
| mriedem | mnaser: run with OS_DEBUG=True | 20:32 |
| mnaser | mriedem: thank you | 20:32 |
| mriedem | cfriesen: nope | 20:32 |
| mriedem | jaypipes: thanks for starting https://review.openstack.org/#/c/544683/ - a few thoughts inline | 20:40 |
| mriedem | will definitely want to talk with dansmith about the new upcall | 20:41 |
| *** suresh12 has joined #openstack-nova | 20:43 | |
| *** suresh12 has quit IRC | 20:48 | |
| mriedem | efried: i moved https://blueprints.launchpad.net/nova/+spec/nested-resource-providers back to queens and completed it | 20:49 |
| mriedem | let us never speak of it again | 20:49 |
| efried | ack | 20:49 |
| efried | Someone still owes a NRP-in-alloc-cands spec for Rocky. | 20:49 |
| efried | I think I'm still on the hook for that. | 20:50 |
| mriedem | you just smelled it | 20:50 |
| mriedem | melwitt: you gonna take the nova meeting baton tomorrow afternoon? | 20:52 |
| *** mchlumsky_ has joined #openstack-nova | 20:52 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old flavor_get_all db api method https://review.openstack.org/544688 | 20:53 |
| dansmith | good gawd this aggregate de-compat is a mess | 20:54 |
| *** mchlumsky has quit IRC | 20:54 | |
| *** suresh12 has joined #openstack-nova | 20:54 | |
| *** awaugama has quit IRC | 20:57 | |
| *** mgariepy has quit IRC | 21:00 | |
| *** suresh12 has quit IRC | 21:00 | |
| jaypipes | mriedem: yup | 21:03 |
| jogo | mriedem: done | 21:05 |
| *** suresh12 has joined #openstack-nova | 21:07 | |
| *** tidwellr has quit IRC | 21:08 | |
| *** tidwellr has joined #openstack-nova | 21:08 | |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs master: Support member_of param for allocation candidates https://review.openstack.org/544694 | 21:08 |
| *** esberglu has quit IRC | 21:08 | |
| *** esberglu has joined #openstack-nova | 21:09 | |
| *** liverpooler has quit IRC | 21:09 | |
| *** acormier has quit IRC | 21:11 | |
| *** acormier has joined #openstack-nova | 21:12 | |
| *** psachin has quit IRC | 21:12 | |
| *** esberglu has quit IRC | 21:13 | |
| *** suresh12 has quit IRC | 21:14 | |
| mriedem | jogo: thanks, and o/ | 21:15 |
| *** acormier has quit IRC | 21:16 | |
| jogo | mriedem: o/ glad gerrit lets you drop a -2 even if you can't -2 new things | 21:17 |
| *** suresh12 has joined #openstack-nova | 21:18 | |
| *** slaweq has quit IRC | 21:18 | |
| mriedem | jogo: you can always come back | 21:20 |
| *** felipemonteiro_ has quit IRC | 21:20 | |
| *** felipemonteiro_ has joined #openstack-nova | 21:20 | |
| jaypipes | mriedem, dansmith, efried: don't forget about this other aggregate-related spec... https://review.openstack.org/#/c/529135/ | 21:20 |
| jaypipes | dansmith: https://blueprints.launchpad.net/nova/+spec/placement-mirror-host-aggregates is ready for you to link to in your spec. | 21:21 |
| efried | jaypipes: I can look again; maybe my opinion has changed. | 21:21 |
| jaypipes | dansmith: working on the spec for that one now. | 21:21 |
| efried | (the "opinion" that I don't understand most of it) | 21:21 |
| dansmith | jaypipes: thanks | 21:21 |
| jaypipes | efried: np. was just pointing out all my agg-related stuffs... | 21:21 |
| *** esberglu has joined #openstack-nova | 21:22 | |
| efried | No need to get all aggro | 21:22 |
| dansmith | jaypipes: the one I really need is the bp for adding aggregates to the allocation_candidates api | 21:22 |
| efried | dansmith: https://review.openstack.org/#/c/544694/1 | 21:22 |
| dansmith | ah cool | 21:23 |
| jaypipes | dansmith: https://blueprints.launchpad.net/nova/+spec/alloc-candidates-member-of | 21:25 |
| jaypipes | dansmith: yup, pushed it up about 20 minutes ago | 21:25 |
| dansmith | jaypipes: yeah, found it thanks | 21:25 |
| jaypipes | np | 21:25 |
| dansmith | efried: yawna look at my replies before I push up this revision with the linkages? | 21:26 |
| mriedem | god i see i need to read https://review.openstack.org/#/c/544585/ again | 21:26 |
| efried | dansmith: You mean on ^ ? Looking... | 21:27 |
| dansmith | yeah | 21:27 |
| *** pcaruana has quit IRC | 21:28 | |
| mnaser | functional tests done (and failing), crossing fingers that putting the change on top fixes things | 21:29 |
| jogo | mriedem: :) | 21:29 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Remove deprecated aggregate DB compatibility https://review.openstack.org/544698 | 21:29 |
| jaypipes | aggregates are just the topic du jour I guess. | 21:31 |
| efried | dansmith: Can you please clarify the traits thing for me? What I suggested in L119 was yes, "bulk adding traits to large swaths of compute nodes" - but instead of getting the trait to the API call by pulling it from the flavor, you get it from the RequestSpec (or whatever) and fold it into any traits provided from the flavor. | 21:32 |
| efried | dansmith: Is that what you're suggesting could be a solution for boot-from-volume in the future? | 21:32 |
| dansmith | efried: clarify how a trait in a flavor doesn't help? | 21:32 |
| dansmith | efried: ha | 21:32 |
| dansmith | *ah | 21:32 |
| dansmith | efried: you could have flavors you allow BFV from but that aren't dedicated to bfv | 21:32 |
| efried | dansmith: Getting the trait from not-flavor to avoid Skittles. | 21:32 |
| *** suresh12 has quit IRC | 21:33 | |
| *** pchavva has quit IRC | 21:33 | |
| dansmith | efried: so if you want to make sure that BFV requests using generic flavors don't get routed to any old host, you might want to slap a trait on those requests so that you can send them to the right place | 21:33 |
| *** suresh12 has joined #openstack-nova | 21:33 | |
| mriedem | my idea was that the pre-placement filter thing dan's spec is adding, would shove a trait into the flavor embedded in the reqspec b/c that's currently how we get the traits from the user to the scheduler to placement | 21:33 |
| efried | Right, so we're merging traits-from-flavor with traits-from-elsewhere | 21:34 |
| dansmith | we could do that too | 21:34 |
| mriedem | so if we know we're doing bfv on a multiattach volume (or a tagged bdm), this code puts a CUSTOM_COMPUTE_SUPPORTS_MULTIATTACH on the flavor within the reqspec | 21:34 |
| mriedem | efried: yeah | 21:34 |
| efried | mriedem: Not on the flavor - on the allocation_candidates request | 21:35 |
| mriedem | we could always do something else like the pre-placement filters add RequestSpec.system_traits (like system_metadata) or something | 21:35 |
| mriedem | efried: but we have to get them to the allocation candidates request somehow | 21:35 |
| dansmith | efried: on the flavor in the reqspec if we want it to be honored for a resize or migration kinda thing | 21:35 |
| efried | Do we make other "modifications" to flavor data between when we pull it from the flavor db and when we attach it to the instance? | 21:36 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove old flavor_create db api method https://review.openstack.org/544703 | 21:36 |
| dansmith | efried: we suggested doing this for zero-root-disk BFV things I think | 21:36 |
| dansmith | efried: we also probably need traits to be stored in the reqspec proper for when the user asked for them right/ | 21:37 |
| mriedem | https://review.openstack.org/#/c/428505/ ? | 21:37 |
| dansmith | mriedem: no, mine below that | 21:37 |
| efried | My point is that it feels kinda weird for me to say 'boot an instance with this flavor, that has traits X,Y,Z' and then when I look at the flavor on the instance later, it also has traits A,B,C. I'm like where tf did those come from? | 21:37 |
| mriedem | efried: you wouldn't see those | 21:38 |
| efried | Okay, then I'm cool. | 21:38 |
| efried | If they're on the RequestSpec and not on the flavor per se. | 21:38 |
| mriedem | the instance.flavor would show the public facing flavor you used to create the instace | 21:38 |
| *** mvk_ has joined #openstack-nova | 21:38 | |
| dansmith | instance.flavor not reqspec.flavor | 21:38 |
| mriedem | right, they'd be on the flavor embedded within the request spec, which doesn't get exposed out of the api today | 21:38 |
| mriedem | ? | 21:38 |
| mriedem | GET /servers/{id}/detail shows the flavor embedded in the instance | 21:39 |
| dansmith | although we probably create instance.flavor from reqspec.flavor | 21:39 |
| mriedem | we don't | 21:39 |
| *** ansiwen has quit IRC | 21:39 | |
| *** mdbooth has quit IRC | 21:39 | |
| mriedem | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L910 | 21:40 |
| mriedem | https://github.com/openstack/nova/blob/master/nova/compute/api.py#L891 | 21:40 |
| efried | So yeah. dansmith What I'm saying is that this is an alternative that should be at least documented in the Alternatives section; *and* some more explanation along these lines is needed for the reader to make sense of the statement on L47 that mentions traits for the first/last/only time. | 21:40 |
| dansmith | it doesn't really matter anyway, as that'd be a conversation for how we're going to do a thing with traits, | 21:40 |
| dansmith | unrelated to the general idea of augmenting the request to placement | 21:40 |
| dansmith | efried: I'm not sure I get what is an alternative | 21:40 |
| dansmith | efried: traits for the tenant isolation is not an option, IMHO | 21:41 |
| dansmith | if you mean it's an alternative to the "and also maybe later we could do a bfv thingy", then agree, but it's not really the subject of this spec | 21:41 |
| efried | So | 21:41 |
| mriedem | "Although not covered by this spec directly, the mechanism added here | 21:41 |
| mriedem | would enable other such request-limiting filters in the future. For | 21:41 |
| mriedem | example," | 21:41 |
| mriedem | that seems pretty clear that this is a framework type thing | 21:42 |
| mriedem | the spec is dealing with the specific tenant aggregate problem, | 21:42 |
| dansmith | this could have been two specs, but I felt it was way better to not have a tiny spec for each | 21:42 |
| mriedem | but is generic enough that we can build on it for other problems | 21:42 |
| mriedem | other problems like bfv | 21:42 |
| dansmith | maybe I'm missing what efried is saying is an alternative though | 21:42 |
| *** jackie-truong has joined #openstack-nova | 21:43 | |
| *** esberglu has quit IRC | 21:43 | |
| mriedem | i think he's saying, if i have A, B, C in aggregate X, rather than model it as an aggregate, the resource providers for A,B,C would have an X trait | 21:43 |
| mriedem | which is a PITA | 21:43 |
| mriedem | so i agree we don't want to do that | 21:43 |
| dansmith | for tenant isolation for sure | 21:44 |
| efried | dansmith: I agree that [traits for tenant isolation if you have to specify the traits in flavors] is not an option - but you describe it in Alternatives and explain why it's not an option (Skittles). IMO, [traits for tenant isolation where we merge the traits from the RequestSpec that we got from conf just like you're talking about doing with aggregates] *may* also not be viable, but it doesn't suffer from your main objec | 21:44 |
| efried | therefore at least closer to being a viable option, and should therefore be described in the Alternatives section, and dismissed with an explanation as to why it's being dismissed if indeed there's a good reason for that. | 21:44 |
| mnaser | mriedem: good call on suggesting functional test. while the codebase seems to actually not stop there, it looks like it's not getting any bdms in _delete() .. BlockDeviceMappingList(objects=[]) | 21:44 |
| efried | And then you get the added bonus of being able to refer to that model from L46-7, because otherwise L46-7 doesn't make sense in the context of the rest of the spec. | 21:45 |
| mnaser | is it possible that bdms are created later? :< | 21:45 |
| dansmith | efried: ah okay, that's what I'm saying is also not an option IMHO, but for other reasons than flavor explosion | 21:45 |
| mriedem | mnaser: i bet i know why, | 21:45 |
| mriedem | mnaser: i bet the bdms aren't buried in cell0 | 21:45 |
| efried | dansmith: Cool, please say that in the spec and I'm good. | 21:45 |
| mnaser | yay.. | 21:45 |
| mnaser | should they be buried in cell0? | 21:45 |
| mriedem | mnaser: just a guess, but that's my guess | 21:45 |
| dansmith | efried: because we have to label each compute individually with the trait and I think that's what aggregates are for, but yep, I'll put it in as a slightly different thing.. just for you :) | 21:46 |
| efried | Thanks dansmith. Kisses. | 21:46 |
| mriedem | it's a valentines day miracle | 21:46 |
| efried | dansmith: Having to label each compute individually with the trait - what I'm confused about on L46-7 is why that argument doesn't apply for BFV too. | 21:46 |
| dansmith | efried: well, the way I phrased that is silly, since that's traits all up and down, but let me say it in the spec and then you can argue about it, if it's clear | 21:47 |
| mriedem | efried: for one thing, nova-compute can report the BFV traits capabilities automatically, the operator / admin doesn't have to do that for every tenant | 21:47 |
| mnaser | select * from block_device_mapping => Empty set | 21:48 |
| mnaser | i guess i'll get them added.. | 21:48 |
| efried | mriedem: It can do that for host aggs too, though, nah? | 21:48 |
| dansmith | mriedem: and compute can report aggregates automaticaly | 21:48 |
| mriedem | that's an up-call for one thing | 21:48 |
| mriedem | nova-compute would have to get the aggregates that the host is in, find the tenant_id meta on those, and convert that to traits on the RP | 21:49 |
| mriedem | i agree we could do that, | 21:49 |
| efried | mriedem, dansmith: Oh, so yeah, that was my point: we're talking about automatically mirroring host aggs as placement aggs; in the alternative I describe above, we would be instead automatically mirroring host aggs as traits on the compute node RP. | 21:49 |
| mriedem | but it's an up-call | 21:49 |
| mriedem | the bfv trait is much more clear to me as a 'trait', | 21:49 |
| dansmith | efried: right, but then you create a tenant trait, apply to all computes. if you want to reassign a cell to a different tenant, you remove/reapply against all those computes | 21:50 |
| mriedem | the tenant isolation thing is very much less a 'trait' in my head | 21:50 |
| dansmith | efried: compared with: | 21:50 |
| efried | ...same but for agg associations | 21:50 |
| dansmith | efried: this cell is aggregate cell-foo, and add "tenant bar is allowed" to the cell-foo aggregate, | 21:50 |
| mriedem | mnaser: oh i know the problem, | 21:50 |
| dansmith | efried: then you can reassign a cell to a tenant in one place instead of mass add/remove of the other trait | 21:50 |
| efried | Sorry, I don't get the difference. | 21:51 |
| dansmith | efried: hangout? | 21:51 |
| efried | If the trait is CUSTOM_CELL_FOO | 21:51 |
| mriedem | mnaser: so in conductor schedule_and_build_instances, we call the scheduler for a host and if we get one, we look up what cell that host is in and create the instance and bdms in that cell, | 21:51 |
| efried | dansmith: Sure | 21:51 |
| mriedem | mnaser: if scheduling fails, we create the instance in cell0, | 21:51 |
| mriedem | but we don't create the bdms in cell0 | 21:51 |
| mnaser | but not the bdms | 21:51 |
| mriedem | exactamundo | 21:51 |
| mriedem | melwitt: ^ | 21:51 |
| dansmith | efried: required https://hangouts.google.com/call/vebCoylDfu_0tvlCDPwmAAEE, edleafe mriedem jaypipes optional | 21:51 |
| mnaser | i guess i'll look into making it create bdms in cell0 | 21:52 |
| mriedem | pants required? | 21:52 |
| dansmith | mriedem: yes | 21:52 |
| mriedem | mnaser: that or we unreserve the volumes for the bdms right there in conductor... | 21:52 |
| mriedem | so that the api doesn't have to delete them locally | 21:52 |
| mriedem | but then we get the "how many different nova services does it take to detach a volume" problem | 21:52 |
| mnaser | that means that the volume will become available right away | 21:53 |
| mnaser | but yeah | 21:53 |
| mnaser | that | 21:53 |
| mriedem | mnaser: melwitt: left notes on https://review.openstack.org/#/c/340614/ - thanks for the testing | 21:56 |
| *** rcernin has joined #openstack-nova | 22:00 | |
| openstackgerrit | Jackie Truong proposed openstack/python-novaclient master: Microversion 2.61 - Add trusted_image_certificates https://review.openstack.org/500396 | 22:01 |
| mnaser | mriedem: melwitt i can do both (though storing bdms seems easier).. i'll leave the decision up to you | 22:01 |
| *** slaweq has joined #openstack-nova | 22:04 | |
| *** rmcall has quit IRC | 22:05 | |
| *** slaweq has quit IRC | 22:11 | |
| mriedem | before all of this we'd have the bdms created in the api along with the instance (before cellsv2 happened), so might as well just go with that for starters and see how it goes | 22:11 |
| mriedem | not api, the 'nova' db, but the nova-api service would create them in the nova db | 22:12 |
| melwitt | mriedem: sure, I can take the meeting baton tomorrow. let me know if I need to update the agenda or if you already did | 22:13 |
| mriedem | i have not updated the agenda | 22:13 |
| melwitt | k. I shall agenda then | 22:13 |
| melwitt | and that sucks about the bdms thing. | 22:15 |
| mriedem | right so we'll likely need to stagger these patches for sanity | 22:15 |
| openstackgerrit | Dan Smith proposed openstack/nova-specs master: Add placement-req-filter spec https://review.openstack.org/544585 | 22:15 |
| dansmith | efried: ^ | 22:15 |
| mriedem | regression test, do something about bdms in conductor on scheduling failure, then the local delete api patch | 22:16 |
| efried | dansmith: ack | 22:16 |
| melwitt | yeah, I was about to say ... should mnaser take that issue as a separate patch? | 22:16 |
| mriedem | melwitt: i think so | 22:16 |
| mriedem | we're going to want to backport all of this to ocata too | 22:16 |
| melwitt | good times | 22:16 |
| mriedem | this is the most fun i've had all week | 22:16 |
| melwitt | SAME | 22:16 |
| mnaser | I have 3 patches right now, fixing fixtures, functional tests and melwitt final patch on top. I’ll add the bdm creation in the stack. | 22:18 |
| mnaser | I’ll make sure to keep them | 22:18 |
| mnaser | Under the same topic | 22:18 |
| mriedem | mnaser: thanks | 22:18 |
| mnaser | (But first supper) | 22:18 |
| *** acormier has joined #openstack-nova | 22:19 | |
| *** suresh12 has quit IRC | 22:20 | |
| *** dave-mccowan has quit IRC | 22:20 | |
| mlavalle | melwitt: I added some topics to the Nova PTG etherpad. With those topics, I figured 2 hours. Right now I have the Neutron / Nova session from 3 to 5 on Thursday. When you have a chance let me know what you think. No need to rush | 22:20 |
| mlavalle | fro 3 to 5 in the Neutron etherpad^^^^ | 22:21 |
| mlavalle | but I am keeping that afternoon open until you and I firm up the times | 22:21 |
| melwitt | mlavalle: k, probably will be fine. just have to sync up with the ironic team. I'll let you know | 22:23 |
| *** acormier has quit IRC | 22:24 | |
| *** tidwellr has quit IRC | 22:31 | |
| mlavalle | melwitt: :-) | 22:31 |
| *** threestrands has joined #openstack-nova | 22:31 | |
| *** suresh12 has joined #openstack-nova | 22:31 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Scheduler multiple workers support https://review.openstack.org/159382 | 22:40 |
| *** claudiub has joined #openstack-nova | 22:43 | |
| *** claudiub has quit IRC | 22:43 | |
| *** artom_ has joined #openstack-nova | 22:48 | |
| *** abalutoiu_ has joined #openstack-nova | 22:49 | |
| *** felipemonteiro__ has joined #openstack-nova | 22:51 | |
| *** liverpooler has joined #openstack-nova | 22:51 | |
| *** abalutoiu has quit IRC | 22:51 | |
| *** itlinux has quit IRC | 22:53 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Check for leaked server resource allocations in post_test_hook https://review.openstack.org/538510 | 22:53 |
| *** artom has quit IRC | 22:53 | |
| *** mriedem has quit IRC | 22:55 | |
| *** felipemonteiro_ has quit IRC | 22:55 | |
| *** jackie-truong has quit IRC | 22:59 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Remove deprecated aggregate DB compatibility https://review.openstack.org/544698 | 23:04 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Add AggregateList.get_by_metadata_value() query method https://review.openstack.org/544728 | 23:04 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Add aggregates list to Destination object https://review.openstack.org/544729 | 23:04 |
| openstackgerrit | Dan Smith proposed openstack/nova master: WIP: Add request filter functionality and first filter https://review.openstack.org/544730 | 23:04 |
| *** hongbin has quit IRC | 23:06 | |
| *** burt has quit IRC | 23:10 | |
| *** acormier has joined #openstack-nova | 23:11 | |
| *** liverpooler has quit IRC | 23:11 | |
| *** edmondsw has quit IRC | 23:12 | |
| melwitt | mlavalle: I've confirmed that 3-5 on thursday works for nova/neutron. thanks | 23:12 |
| mlavalle | melwitt: Thank you! | 23:12 |
| *** acormier has quit IRC | 23:16 | |
| *** amodi has quit IRC | 23:19 | |
| *** mlavalle has quit IRC | 23:20 | |
| mnaser | yay adding bdms to cell0 fixed it | 23:26 |
| melwitt | tonyb: this says ocata enters phase 3 support on 2/26 but also says 2/26 is the EOL date https://releases.openstack.org/ does that make sense to you? | 23:35 |
| *** mriedem has joined #openstack-nova | 23:38 | |
| *** hshiina|afk has joined #openstack-nova | 23:38 | |
| mriedem | melwitt: do you want/plan on doing a queens retrospective at the ptg? | 23:39 |
| *** acormier has joined #openstack-nova | 23:40 | |
| melwitt | mriedem: I'm not opposed to having one but I hadn't thought about it because no one had mentioned it (whereas we have had obvious interest in retro in the past). do you think we should just plan for one? | 23:41 |
| *** acormier has joined #openstack-nova | 23:42 | |
| melwitt | I also might have missed it if anyone has expressed interest in having a queens retro | 23:42 |
| mriedem | it hasn't come up, but i assume we should at least make it an option | 23:45 |
| mriedem | if no one has anything, it's a short thing | 23:46 |
| melwitt | yeah, makes sense | 23:46 |
| mnaser | i have 4 patches right, is the following order ok or should i reorder: fix cinder fixtures -> add func regr. tests -> add bdms to cell0 -> patch to remove ports and bdms | 23:46 |
| *** acormier has quit IRC | 23:46 | |
| melwitt | mriedem: maybe we could start off with that on wednesday since any takeaways could apply to later discussions | 23:46 |
| mriedem | https://etherpad.openstack.org/p/nova-queens-retrospective | 23:47 |
| mriedem | mnaser: sounds correct | 23:47 |
| mnaser | ok awesome, i have to fix a few unit tests because of the changes in _bury_in_cell0 which now create bdms and should be ready for review soon | 23:47 |
| *** acormier has joined #openstack-nova | 23:48 | |
| *** acormier has quit IRC | 23:48 | |
| *** acormier has joined #openstack-nova | 23:48 | |
| *** acormier has quit IRC | 23:52 | |
| openstackgerrit | Murali Annamneni proposed openstack/nova master: Enables MySQL Cluster Support for Nova https://review.openstack.org/446643 | 23:53 |
| *** acormier has joined #openstack-nova | 23:54 | |
| *** acormier has joined #openstack-nova | 23:54 | |
| *** felipemonteiro__ has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!