| *** acormier has joined #openstack-nova | 00:00 | |
| *** baoli has joined #openstack-nova | 00:01 | |
| *** jaypipes has quit IRC | 00:02 | |
| mriedem | gah | 00:04 |
|---|---|---|
| mriedem | superconductor foils me again | 00:04 |
| mriedem | nova-manage db archive_deleted_rows in devstack uses nova.conf by default, which is cell0 in devstack | 00:04 |
| *** gbarros has quit IRC | 00:04 | |
| *** acormier has quit IRC | 00:05 | |
| *** gbarros has joined #openstack-nova | 00:05 | |
| *** baoli has quit IRC | 00:06 | |
| mriedem | http://logs.openstack.org/58/506458/2/check/gate-tempest-dsvm-neutron-nova-next-full-ubuntu-xenial-nv/0d589b6/logs/devstack-gate-post_test_hook.txt.gz | 00:07 |
| mriedem | yar | 00:07 |
| mriedem | we could make nova-manage db archive_deleted_rows use the api database to get the cells... | 00:08 |
| *** itlinux has quit IRC | 00:08 | |
| *** yangyape_ has quit IRC | 00:09 | |
| *** yangyapeng has joined #openstack-nova | 00:10 | |
| *** acormier has joined #openstack-nova | 00:11 | |
| mriedem | https://bugs.launchpad.net/nova/+bug/1719487 | 00:12 |
| openstack | Launchpad bug 1719487 in OpenStack Compute (nova) "nova-manage db archive_deleted_rows is not multi-cell aware" [Medium,Triaged] | 00:12 |
| *** yangyapeng has quit IRC | 00:14 | |
| *** acormier has quit IRC | 00:16 | |
| *** yamamoto has quit IRC | 00:16 | |
| *** acormier has joined #openstack-nova | 00:17 | |
| *** Swami has quit IRC | 00:17 | |
| *** yamamoto has joined #openstack-nova | 00:21 | |
| *** acormier has quit IRC | 00:22 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: For map_instances argument destination is not defined https://review.openstack.org/502236 | 00:24 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix --max-count handling for nova-manage cell_v2 map_instances https://review.openstack.org/502236 | 00:26 |
| *** crushil_ has joined #openstack-nova | 00:28 | |
| *** Shunli has joined #openstack-nova | 00:31 | |
| *** mikal_ has quit IRC | 00:32 | |
| *** mikal has joined #openstack-nova | 00:34 | |
| *** kenperkins has quit IRC | 00:36 | |
| *** crushil_ has quit IRC | 00:38 | |
| *** zhurong has joined #openstack-nova | 00:42 | |
| *** litao__ has joined #openstack-nova | 00:53 | |
| *** acormier has joined #openstack-nova | 00:54 | |
| *** Apoorva_ has joined #openstack-nova | 00:57 | |
| *** acormier has quit IRC | 00:58 | |
| *** Apoorva has quit IRC | 01:00 | |
| *** ijw has quit IRC | 01:02 | |
| *** Apoorva_ has quit IRC | 01:02 | |
| *** ijw has joined #openstack-nova | 01:02 | |
| *** acormier has joined #openstack-nova | 01:06 | |
| *** ijw has quit IRC | 01:07 | |
| *** acormier has quit IRC | 01:07 | |
| *** acormier has joined #openstack-nova | 01:08 | |
| *** yangyapeng has joined #openstack-nova | 01:12 | |
| *** phuongnh has joined #openstack-nova | 01:14 | |
| *** phuongnh has quit IRC | 01:20 | |
| openstackgerrit | jichenjc proposed openstack/nova master: propagate OSError to MigrationPreCheckError https://review.openstack.org/469779 | 01:20 |
| *** moshele has quit IRC | 01:20 | |
| *** gbarros_ has joined #openstack-nova | 01:25 | |
| *** gbarros has quit IRC | 01:28 | |
| openstackgerrit | jichenjc proposed openstack/nova master: check query param for server groups function https://review.openstack.org/500347 | 01:31 |
| *** esberglu has quit IRC | 01:33 | |
| *** yufei has joined #openstack-nova | 01:33 | |
| *** hongbin has joined #openstack-nova | 01:36 | |
| *** mingyu_ has joined #openstack-nova | 01:38 | |
| *** mingyu has quit IRC | 01:39 | |
| *** yangyapeng has quit IRC | 01:40 | |
| *** yangyapeng has joined #openstack-nova | 01:40 | |
| openstackgerrit | jichenjc proposed openstack/nova master: fix race condition of instance host https://review.openstack.org/494458 | 01:44 |
| *** yufei has quit IRC | 01:47 | |
| *** yufei has joined #openstack-nova | 01:47 | |
| *** yamamoto has quit IRC | 01:50 | |
| *** yamamoto has joined #openstack-nova | 01:50 | |
| *** crushil has quit IRC | 01:50 | |
| *** yangyapeng has quit IRC | 01:50 | |
| *** yangyapeng has joined #openstack-nova | 01:51 | |
| *** yamamoto_ has joined #openstack-nova | 01:51 | |
| *** gcb has joined #openstack-nova | 01:52 | |
| *** yufei has quit IRC | 01:54 | |
| *** yufei has joined #openstack-nova | 01:54 | |
| *** yamamoto has quit IRC | 01:55 | |
| *** yufei has left #openstack-nova | 01:55 | |
| *** yufei has joined #openstack-nova | 01:55 | |
| *** yangyape_ has joined #openstack-nova | 01:56 | |
| *** yangyapeng has quit IRC | 01:56 | |
| yufei | @alex_xu, could you please help take a look at this patch when you are free? a small patch which remove re-auth logic for service-users when nova call ironic. https://review.openstack.org/#/c/502382/ | 01:57 |
| *** hieulq has joined #openstack-nova | 02:05 | |
| *** yangyape_ has quit IRC | 02:12 | |
| *** yangyapeng has joined #openstack-nova | 02:13 | |
| *** yangyapeng has quit IRC | 02:15 | |
| *** hieulq has quit IRC | 02:15 | |
| *** yangyapeng has joined #openstack-nova | 02:15 | |
| *** mingyu_ has quit IRC | 02:17 | |
| *** yangyapeng has quit IRC | 02:20 | |
| openstackgerrit | wanghao proposed openstack/nova master: Set min_disk in the image meta based on the root volume's size https://review.openstack.org/407397 | 02:25 |
| *** gbarros_ has quit IRC | 02:26 | |
| *** OctopusZhang has joined #openstack-nova | 02:26 | |
| *** yufei has quit IRC | 02:30 | |
| *** OctopusZhang is now known as yufei | 02:30 | |
| *** Tom has joined #openstack-nova | 02:32 | |
| *** yingjun has joined #openstack-nova | 02:39 | |
| *** mingyu has joined #openstack-nova | 02:39 | |
| *** erlon has quit IRC | 02:46 | |
| *** hieulq has joined #openstack-nova | 02:48 | |
| *** thorst has quit IRC | 02:51 | |
| *** thorst has joined #openstack-nova | 02:51 | |
| *** hongbin has quit IRC | 02:51 | |
| *** hongbin has joined #openstack-nova | 02:52 | |
| *** bhagyashris has joined #openstack-nova | 02:54 | |
| *** Dinesh_Bhor has joined #openstack-nova | 02:54 | |
| *** pooja_jadhav has joined #openstack-nova | 02:54 | |
| *** neha_alhat has joined #openstack-nova | 02:54 | |
| *** neha_alhat has quit IRC | 02:55 | |
| *** thorst has quit IRC | 02:56 | |
| *** bhagyashris has quit IRC | 02:56 | |
| *** neha_alhat has joined #openstack-nova | 02:56 | |
| *** bhagyashris has joined #openstack-nova | 02:56 | |
| *** ps_jadhav has joined #openstack-nova | 02:57 | |
| *** Dinesh_Bhor has quit IRC | 02:57 | |
| *** jichen has joined #openstack-nova | 02:57 | |
| *** Dinesh_Bhor has joined #openstack-nova | 02:57 | |
| *** mikal has quit IRC | 02:59 | |
| *** pooja_jadhav has quit IRC | 02:59 | |
| *** yangyapeng has joined #openstack-nova | 03:01 | |
| *** mikal has joined #openstack-nova | 03:02 | |
| *** yangyape_ has joined #openstack-nova | 03:03 | |
| openstackgerrit | Steven Webster proposed openstack/nova master: Update nova network info when doing rebuild for evacuate operation https://review.openstack.org/382853 | 03:05 |
| openstackgerrit | Steven Webster proposed openstack/nova master: Race condition between audit and migrate/resize revert https://review.openstack.org/400995 | 03:06 |
| *** yangyapeng has quit IRC | 03:07 | |
| *** dave-mcc_ has quit IRC | 03:09 | |
| *** crushil has joined #openstack-nova | 03:14 | |
| *** crushil has quit IRC | 03:17 | |
| *** itlinux has joined #openstack-nova | 03:19 | |
| *** ps_jadhav has quit IRC | 03:20 | |
| *** bhagyashris has quit IRC | 03:20 | |
| *** itlinux has quit IRC | 03:22 | |
| *** neha_alhat has quit IRC | 03:22 | |
| *** itlinux has joined #openstack-nova | 03:24 | |
| *** Dinesh_Bhor has quit IRC | 03:24 | |
| gmann | alex_xu: do you remember why limit and marker are single_param in this - https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/hypervisors.py#L22 | 03:26 |
| *** pooja_jadhav has joined #openstack-nova | 03:27 | |
| *** bhagyashris has joined #openstack-nova | 03:27 | |
| *** Dinesh_Bhor has joined #openstack-nova | 03:28 | |
| *** neha_alhat has joined #openstack-nova | 03:28 | |
| gmann | mriedem: alex_xu: because it was added in microversion ? because we agreed on keep accepting the multi param for 'limit' and other pagination query param and let controller code to fetch the one(which is second one as default due to dict) | 03:29 |
| *** acormier has quit IRC | 03:29 | |
| *** acormier has joined #openstack-nova | 03:30 | |
| *** pooja_jadhav has quit IRC | 03:30 | |
| *** neha_alhat has quit IRC | 03:30 | |
| *** Apoorva has joined #openstack-nova | 03:30 | |
| *** bhagyashris has quit IRC | 03:34 | |
| *** acormier has quit IRC | 03:34 | |
| *** Dinesh_Bhor has quit IRC | 03:34 | |
| *** vladikr has quit IRC | 03:35 | |
| *** vladikr has joined #openstack-nova | 03:35 | |
| *** itlinux has quit IRC | 03:35 | |
| *** gouthamr has quit IRC | 03:35 | |
| *** Tom has quit IRC | 03:37 | |
| *** itlinux has joined #openstack-nova | 03:38 | |
| *** acormier has joined #openstack-nova | 03:39 | |
| gmann | mriedem: alex_xu similar for additionalProperties, it is False. i think those are correct as those query are introduced with microversion. I will keep 2.33 schema also same way to avoid any inconsistency between 2.33 and 2.53 | 03:39 |
| *** hongbin has quit IRC | 03:40 | |
| *** vladikr has quit IRC | 03:45 | |
| *** neha_alhat has joined #openstack-nova | 03:45 | |
| *** vladikr has joined #openstack-nova | 03:45 | |
| *** Dinesh_Bhor has joined #openstack-nova | 03:46 | |
| *** bhagyashris has joined #openstack-nova | 03:47 | |
| *** yufei has quit IRC | 03:47 | |
| *** pooja_jadhav has joined #openstack-nova | 03:49 | |
| *** avolkov has joined #openstack-nova | 03:51 | |
| *** thorst has joined #openstack-nova | 03:53 | |
| *** vladikr has quit IRC | 03:53 | |
| *** vladikr has joined #openstack-nova | 03:53 | |
| *** Tom has joined #openstack-nova | 03:55 | |
| *** wlfightup has joined #openstack-nova | 03:56 | |
| wlfightup | hello | 03:56 |
| wlfightup | if resize same host, the schuduler don't filter delta, only filter new flavor. i think that it is don't right | 03:57 |
| *** udesale has joined #openstack-nova | 03:59 | |
| *** acormier has quit IRC | 03:59 | |
| *** Tom has quit IRC | 04:00 | |
| *** itlinux has quit IRC | 04:00 | |
| wlfightup | anyone there ? | 04:11 |
| *** mtreinish has quit IRC | 04:24 | |
| *** vladikr has quit IRC | 04:25 | |
| *** cfriesen has quit IRC | 04:25 | |
| *** cfriesen_ has joined #openstack-nova | 04:25 | |
| *** psachin has joined #openstack-nova | 04:26 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Implement query param schema for GET hypervisor(2.33) https://review.openstack.org/507344 | 04:32 |
| gmann | mriedem: alex_xu this one for 2.33- https://review.openstack.org/507344 | 04:33 |
| *** mtreinish has joined #openstack-nova | 04:34 | |
| *** Apoorva has quit IRC | 04:36 | |
| *** Apoorva has joined #openstack-nova | 04:37 | |
| *** bhagyashri_s has joined #openstack-nova | 04:37 | |
| *** crushil has joined #openstack-nova | 04:38 | |
| *** lajoskatona has joined #openstack-nova | 04:38 | |
| *** bhagyashris has quit IRC | 04:40 | |
| *** Apoorva has quit IRC | 04:41 | |
| *** trungnv has joined #openstack-nova | 04:42 | |
| *** Apoorva has joined #openstack-nova | 04:45 | |
| *** gcb has quit IRC | 04:48 | |
| *** claudiub has joined #openstack-nova | 04:49 | |
| *** ratailor has joined #openstack-nova | 04:54 | |
| alex_xu | gmann: we should keep backward-compatible in 2.33? | 05:02 |
| *** Apoorva has quit IRC | 05:02 | |
| *** Apoorva has joined #openstack-nova | 05:05 | |
| *** Apoorva has quit IRC | 05:05 | |
| *** sree has joined #openstack-nova | 05:06 | |
| *** Apoorva has joined #openstack-nova | 05:08 | |
| *** sree has quit IRC | 05:09 | |
| *** yangyape_ has quit IRC | 05:09 | |
| *** sree has joined #openstack-nova | 05:09 | |
| *** yangyapeng has joined #openstack-nova | 05:09 | |
| *** avolkov has quit IRC | 05:11 | |
| *** avolkov has joined #openstack-nova | 05:11 | |
| *** mriedem has quit IRC | 05:12 | |
| *** adreznec has quit IRC | 05:13 | |
| *** bnemec has quit IRC | 05:14 | |
| *** yangyapeng has quit IRC | 05:14 | |
| gmann | alex_xu: but there was no query param before 2.33 so making those as single value and no additional property should be fine? | 05:15 |
| gmann | alex_xu: only issue was, we did not restrict and documented about that while doing 2.33 | 05:15 |
| gmann | alex_xu: which can break people if someone using it with multi param and additional params >2.33 (<2.53 as it restrict those with schema) | 05:16 |
| *** Apoorva has quit IRC | 05:18 | |
| gmann | alex_xu: i think i agree to keep those same as it is behaving currently. | 05:19 |
| alex_xu | gmann: yea, keep the same behaving currently | 05:23 |
| *** gcb has joined #openstack-nova | 05:24 | |
| *** crushil_ has joined #openstack-nova | 05:27 | |
| *** Eran_Kuris has joined #openstack-nova | 05:28 | |
| *** crushil has quit IRC | 05:29 | |
| *** felipemonteiro has joined #openstack-nova | 05:35 | |
| *** yamamoto_ has quit IRC | 05:37 | |
| *** yamamoto has joined #openstack-nova | 05:37 | |
| *** yufei has joined #openstack-nova | 05:43 | |
| *** Tom has joined #openstack-nova | 05:48 | |
| *** trinaths has joined #openstack-nova | 05:54 | |
| *** sahid has joined #openstack-nova | 05:56 | |
| *** armax has joined #openstack-nova | 05:59 | |
| *** felipemonteiro has quit IRC | 06:00 | |
| *** adreznec has joined #openstack-nova | 06:01 | |
| *** cfriesen_ has quit IRC | 06:02 | |
| wlfightup | can care me? | 06:02 |
| *** thorst has quit IRC | 06:03 | |
| *** Oku_OS-away is now known as Oku_OS | 06:06 | |
| *** mdnadeem has joined #openstack-nova | 06:06 | |
| *** trinaths1 has joined #openstack-nova | 06:06 | |
| *** trinaths2 has joined #openstack-nova | 06:07 | |
| *** trinaths has quit IRC | 06:07 | |
| *** trinaths1 has quit IRC | 06:11 | |
| *** jaosorior has quit IRC | 06:12 | |
| *** pcaruana has joined #openstack-nova | 06:20 | |
| *** andreas_s has joined #openstack-nova | 06:21 | |
| *** zhurong has quit IRC | 06:22 | |
| *** chyka has quit IRC | 06:24 | |
| *** jaosorior has joined #openstack-nova | 06:35 | |
| *** moshele has joined #openstack-nova | 06:36 | |
| *** Oku_OS is now known as Oku_OS-away | 06:40 | |
| *** crushil_ has quit IRC | 06:45 | |
| openstackgerrit | Merged openstack/nova master: Live Migration sequence diagram https://review.openstack.org/506370 | 06:45 |
| *** rcernin has joined #openstack-nova | 06:47 | |
| *** thorst has joined #openstack-nova | 06:59 | |
| *** Oku_OS-away is now known as Oku_OS | 06:59 | |
| *** sree has quit IRC | 07:02 | |
| *** sree has joined #openstack-nova | 07:02 | |
| *** karthiks has quit IRC | 07:03 | |
| *** karthiks has joined #openstack-nova | 07:03 | |
| *** Atom1234 has joined #openstack-nova | 07:05 | |
| *** sree has quit IRC | 07:07 | |
| *** belmoreira has joined #openstack-nova | 07:11 | |
| *** dtantsur|afk is now known as dtantsur | 07:11 | |
| *** Tom has quit IRC | 07:13 | |
| *** tesseract has joined #openstack-nova | 07:14 | |
| *** Tom has joined #openstack-nova | 07:15 | |
| *** Tom has quit IRC | 07:15 | |
| *** Tom has joined #openstack-nova | 07:16 | |
| *** belmoreira has quit IRC | 07:19 | |
| openstackgerrit | Alex Xu proposed openstack/nova-specs master: Add trait support in the allocation candidates API https://review.openstack.org/497713 | 07:19 |
| *** wlfightup has quit IRC | 07:20 | |
| *** wlfightup has joined #openstack-nova | 07:21 | |
| *** ratailor_ has joined #openstack-nova | 07:24 | |
| ratailor_ | help | 07:24 |
| *** belmoreira has joined #openstack-nova | 07:25 | |
| *** jangutter has quit IRC | 07:25 | |
| ratailor | anybody expert in openstackclient, I want to know why the default version is taken as minimum api microversion 2.1, whereas in novaclient it is maximum the client can support. Is there any design consensus about it. | 07:26 |
| *** ragiman has joined #openstack-nova | 07:28 | |
| *** armax has quit IRC | 07:29 | |
| *** tssurya has joined #openstack-nova | 07:29 | |
| *** belmoreira has quit IRC | 07:29 | |
| *** ratailor_ has quit IRC | 07:29 | |
| ratailor | amotoki, ^^ | 07:31 |
| *** phuongnh has joined #openstack-nova | 07:39 | |
| bauzas | good morning Novaz | 07:40 |
| *** sree has joined #openstack-nova | 07:41 | |
| gibi | bauzas: good morning | 07:42 |
| *** ratailor is now known as ratailor|Lunch | 07:43 | |
| bauzas | ta | 07:44 |
| * bauzas has a lovely talk to prepare for telling about Pike and the PTG | 07:44 | |
| *** belmoreira has joined #openstack-nova | 07:44 | |
| bauzas | 3 days, 20 slides | 07:45 |
| bauzas | meh | 07:45 |
| bauzas | I mean, 3 days left, 20 slides to write | 07:45 |
| bauzas | I can do it :) | 07:45 |
| gmann | bauzas: all the best :) | 07:45 |
| *** sree has quit IRC | 07:46 | |
| gibi | I feel your pain. I created a one slider for a local meetup. | 07:48 |
| openstackgerrit | jichenjc proposed openstack/nova-specs master: Adds spec for instance live resize https://review.openstack.org/141219 | 07:50 |
| openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix test_get_volume_config method https://review.openstack.org/489467 | 07:50 |
| *** xinliang has quit IRC | 07:55 | |
| bauzas | gibi: that's the problem when you say to your friends 'sure, I can talk about that'. | 07:56 |
| bauzas | then, you should know you're doomed for the whole week | 07:56 |
| *** tiendc has joined #openstack-nova | 07:59 | |
| gibi | bauzas: I decided to do it a bit blindly and only created a single slide. I will talk about what I will remember. Bit more stress during the talk lot less stress during the week before the talk :) | 07:59 |
| *** belmoreira has quit IRC | 07:59 | |
| *** alexchadin has joined #openstack-nova | 08:00 | |
| *** sree has joined #openstack-nova | 08:02 | |
| openstackgerrit | jichenjc proposed openstack/nova-specs master: Adds spec for instance live resize https://review.openstack.org/141219 | 08:02 |
| *** Tom__ has joined #openstack-nova | 08:02 | |
| bauzas | gibi: probably a good idea | 08:02 |
| bauzas | gibi: just some pointers and job done | 08:02 |
| bauzas | more interactive | 08:03 |
| bauzas | gibi: you saved my day, exactly like you saved our bugs | 08:03 |
| bauzas | you saved *from* our bugs :) | 08:04 |
| bauzas | unrelated, haven't seen yet the new zuul | 08:04 |
| *** aloga has quit IRC | 08:04 | |
| *** aloga has joined #openstack-nova | 08:04 | |
| *** Tom has quit IRC | 08:06 | |
| *** markvoelker has quit IRC | 08:06 | |
| gibi | at least the status page looks the same for zuul but I guess that means nothing | 08:08 |
| *** alexchadin has quit IRC | 08:08 | |
| *** xinliang has joined #openstack-nova | 08:08 | |
| gibi | bauzas: do you have the link for melwitt's interview about what-happened-in-Pike-in-nova? | 08:09 |
| gibi | bauzas: I can even play that video on the meetup :) | 08:10 |
| bauzas | gibi: I can find it | 08:11 |
| bauzas | I wouldn't like to be talking alot of Nova | 08:11 |
| gibi | bauzas: here it is https://www.youtube.com/watch?v=d7vFqM3ZEdI | 08:11 |
| gibi | bauzas: ohh you have to cover the whole OpenStack? that scary | 08:12 |
| bauzas | :) | 08:12 |
| *** Atom1234 has quit IRC | 08:15 | |
| *** zhurong has joined #openstack-nova | 08:20 | |
| *** OctopusZhang has joined #openstack-nova | 08:20 | |
| *** yufei has quit IRC | 08:23 | |
| *** OctopusZhang is now known as yufei | 08:23 | |
| *** mvk has quit IRC | 08:26 | |
| *** belmoreira has joined #openstack-nova | 08:27 | |
| *** Shunli has quit IRC | 08:27 | |
| *** Shunli has joined #openstack-nova | 08:28 | |
| openstackgerrit | Shan Guo proposed openstack/nova master: Remove SCREEN_LOGDIR from devstack install setting https://review.openstack.org/507425 | 08:28 |
| *** belmoreira has quit IRC | 08:30 | |
| *** ralonsoh has joined #openstack-nova | 08:31 | |
| *** yamamoto has quit IRC | 08:31 | |
| *** alexchadin has joined #openstack-nova | 08:31 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Implement query param schema for GET hypervisor(2.33) https://review.openstack.org/507344 | 08:33 |
| *** yamamoto has joined #openstack-nova | 08:34 | |
| *** TuanLA has joined #openstack-nova | 08:35 | |
| *** ratailor|Lunch is now known as ratailor | 08:37 | |
| *** yamamoto has quit IRC | 08:39 | |
| openstackgerrit | Merged openstack/nova master: Add functional for live migrate delete https://review.openstack.org/499583 | 08:41 |
| *** wlfightup has quit IRC | 08:42 | |
| *** wlfightup has joined #openstack-nova | 08:42 | |
| *** udesale has quit IRC | 08:45 | |
| *** yamamoto has joined #openstack-nova | 08:47 | |
| *** yamamoto has quit IRC | 08:47 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Implement query param schema for GET hypervisor(2.33) https://review.openstack.org/507344 | 08:50 |
| *** udesale has joined #openstack-nova | 08:51 | |
| *** mvk has joined #openstack-nova | 08:54 | |
| *** cdent has joined #openstack-nova | 08:55 | |
| *** OctopusZhang has joined #openstack-nova | 08:55 | |
| *** jpena|off is now known as jpena | 08:57 | |
| *** yufei has quit IRC | 08:58 | |
| *** OctopusZhang is now known as yufei | 08:58 | |
| *** belmoreira has joined #openstack-nova | 08:59 | |
| *** sree has quit IRC | 09:00 | |
| *** sree has joined #openstack-nova | 09:00 | |
| *** derekh has joined #openstack-nova | 09:00 | |
| *** sree has quit IRC | 09:05 | |
| *** moshele has quit IRC | 09:05 | |
| *** purplerbot has joined #openstack-nova | 09:06 | |
| *** moshele has joined #openstack-nova | 09:07 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova-specs master: Abort Cold Migration https://review.openstack.org/334732 | 09:07 |
| openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Microversion 2.54 - Enable cold migration with target host https://review.openstack.org/406707 | 09:08 |
| *** yamamoto has joined #openstack-nova | 09:10 | |
| *** yangyapeng has joined #openstack-nova | 09:10 | |
| *** udesale has quit IRC | 09:11 | |
| *** phuongnh has quit IRC | 09:11 | |
| *** udesale has joined #openstack-nova | 09:11 | |
| *** yamamoto has quit IRC | 09:12 | |
| *** yangyapeng has quit IRC | 09:12 | |
| *** belmoreira has quit IRC | 09:13 | |
| *** yangyapeng has joined #openstack-nova | 09:13 | |
| *** yamamoto has joined #openstack-nova | 09:14 | |
| *** mvk has quit IRC | 09:15 | |
| *** yamamoto has quit IRC | 09:16 | |
| *** yamamoto has joined #openstack-nova | 09:16 | |
| *** dtantsur is now known as dtantsur|lunch | 09:21 | |
| *** mvk has joined #openstack-nova | 09:28 | |
| *** yangyapeng has quit IRC | 09:30 | |
| *** wlfightup has quit IRC | 09:31 | |
| *** yamamoto has quit IRC | 09:31 | |
| *** Shunli has quit IRC | 09:31 | |
| *** yangyapeng has joined #openstack-nova | 09:31 | |
| *** wlfightup has joined #openstack-nova | 09:33 | |
| *** yangyapeng has quit IRC | 09:34 | |
| *** yangyapeng has joined #openstack-nova | 09:35 | |
| *** yufei has quit IRC | 09:35 | |
| *** yamamoto has joined #openstack-nova | 09:35 | |
| *** gszasz has joined #openstack-nova | 09:36 | |
| *** liuyulong has quit IRC | 09:36 | |
| *** yangyapeng has quit IRC | 09:39 | |
| *** yassine_ has quit IRC | 09:51 | |
| *** zhurong has quit IRC | 09:52 | |
| openstackgerrit | Lei Zhang proposed openstack/nova-specs master: Request traits in Nova https://review.openstack.org/468797 | 09:53 |
| *** takashin has left #openstack-nova | 09:54 | |
| *** jichen has quit IRC | 09:56 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova-specs master: Enable SR-IOV NIC offload feature discovery https://review.openstack.org/504895 | 09:57 |
| *** sdague has joined #openstack-nova | 10:00 | |
| *** yingjun has quit IRC | 10:02 | |
| *** markvoelker has joined #openstack-nova | 10:07 | |
| *** ijw has joined #openstack-nova | 10:08 | |
| *** trinaths2 has quit IRC | 10:11 | |
| *** wlfightup has quit IRC | 10:11 | |
| *** sree has joined #openstack-nova | 10:11 | |
| *** wlfightup has joined #openstack-nova | 10:12 | |
| *** ijw has quit IRC | 10:13 | |
| *** armax has joined #openstack-nova | 10:15 | |
| *** sree has quit IRC | 10:16 | |
| *** sshwarts has joined #openstack-nova | 10:21 | |
| *** yangyapeng has joined #openstack-nova | 10:22 | |
| *** trinaths has joined #openstack-nova | 10:23 | |
| *** trinaths has quit IRC | 10:26 | |
| *** yangyapeng has quit IRC | 10:26 | |
| *** trinaths has joined #openstack-nova | 10:28 | |
| openstackgerrit | Yikun Jiang proposed openstack/nova master: Update Instance action's updated_at when action event updated. https://review.openstack.org/507473 | 10:34 |
| openstackgerrit | Yikun Jiang proposed openstack/nova master: Update Instance action's updated_at when action event updated. https://review.openstack.org/507473 | 10:36 |
| *** zhurong has joined #openstack-nova | 10:36 | |
| *** udesale has quit IRC | 10:37 | |
| *** markvoelker has quit IRC | 10:41 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: Change 'InstancePCIRequest' spec field https://review.openstack.org/449257 | 10:45 |
| *** yufei has joined #openstack-nova | 10:47 | |
| *** grim-lock has joined #openstack-nova | 10:53 | |
| *** udesale has joined #openstack-nova | 10:53 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Implement query param schema for simple_tenant_usage https://review.openstack.org/507480 | 10:54 |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: Read Neutron port 'binding_profile' during boot https://review.openstack.org/507481 | 10:54 |
| *** grim-lock has quit IRC | 10:54 | |
| *** alexchadin has quit IRC | 10:58 | |
| *** alexchadin has joined #openstack-nova | 10:59 | |
| *** Tom__ has quit IRC | 11:01 | |
| *** Tom_ has joined #openstack-nova | 11:01 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: cover migration cases with functional tests https://review.openstack.org/493865 | 11:02 |
| *** cdent has quit IRC | 11:03 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: use context mgr in instance.delete https://review.openstack.org/443764 | 11:04 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transform missing delete notifications https://review.openstack.org/410297 | 11:04 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send soft_delete from context manager https://review.openstack.org/476459 | 11:04 |
| *** sulo has left #openstack-nova | 11:04 | |
| *** Tom_ has quit IRC | 11:06 | |
| *** vladikr has joined #openstack-nova | 11:06 | |
| *** nicolasbock has joined #openstack-nova | 11:07 | |
| gibi | bauzas: could you make a quick look at https://review.openstack.org/#/c/504930/ ? it is a single line of test case improvement that removes an test instability. | 11:09 |
| *** vladikr has quit IRC | 11:10 | |
| *** vladikr has joined #openstack-nova | 11:11 | |
| *** armax has quit IRC | 11:12 | |
| *** moshele has quit IRC | 11:12 | |
| *** liverpooler has quit IRC | 11:13 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova master: nova-manage db archive_deleted_rows is not multi-cell aware https://review.openstack.org/507486 | 11:14 |
| *** alexchadin has quit IRC | 11:14 | |
| *** yassine has joined #openstack-nova | 11:15 | |
| *** vladikr has quit IRC | 11:18 | |
| *** ralonsoh_ has joined #openstack-nova | 11:21 | |
| openstackgerrit | Matthew Booth proposed openstack/nova master: libvirt: Don't VIR_MIGRATE_NON_SHARED_INC without migrate_disks https://review.openstack.org/507202 | 11:21 |
| openstackgerrit | Matthew Booth proposed openstack/nova master: python3: Specify unicode explicitly calling etree.tostring() https://review.openstack.org/507488 | 11:21 |
| mdbooth | stephenfin: You were looking at ^^^ | 11:22 |
| mdbooth | There was a python3 problem in the first version of that patch. Turns out it's a general problem with etree.tostring() which other tests were ignoring by mocking out :/ | 11:22 |
| mdbooth | Hence the prior patch to fix all etree.tostring() calls | 11:23 |
| *** ralonsoh has quit IRC | 11:24 | |
| *** smatzek has joined #openstack-nova | 11:26 | |
| *** TuanLA has quit IRC | 11:29 | |
| *** alexchadin has joined #openstack-nova | 11:30 | |
| *** trinaths has quit IRC | 11:30 | |
| *** trinaths has joined #openstack-nova | 11:31 | |
| *** thorst has quit IRC | 11:31 | |
| mdbooth | gibi: Before I rebase it, any chance you could promote your +1 here to a +2: https://review.openstack.org/#/c/479802/8 ? | 11:33 |
| mdbooth | Then I'll rebase in a bit after it merges | 11:33 |
| *** zhurong has quit IRC | 11:35 | |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Ensure errors_out_migration errors out migration https://review.openstack.org/479802 | 11:37 |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Use Migration object in ComputeManagerMigrationTestCase https://review.openstack.org/502126 | 11:37 |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Automatically revert resize which fails on destination https://review.openstack.org/462521 | 11:37 |
| mdbooth | Actually meh it was a clean rebase anyway. Should keep the existing +2. | 11:37 |
| *** yangyapeng has joined #openstack-nova | 11:38 | |
| *** zhurong has joined #openstack-nova | 11:39 | |
| *** markvoelker has joined #openstack-nova | 11:39 | |
| *** acormier has joined #openstack-nova | 11:43 | |
| *** acormier has quit IRC | 11:47 | |
| *** tylerderosagrund has joined #openstack-nova | 11:49 | |
| *** vladikr has joined #openstack-nova | 11:55 | |
| openstackgerrit | Lajos Katona proposed openstack/nova master: Extend ServerMovingTests with custom resources https://review.openstack.org/497399 | 11:56 |
| *** moshele has joined #openstack-nova | 11:59 | |
| *** gabor_antal has joined #openstack-nova | 12:00 | |
| *** trinaths1 has joined #openstack-nova | 12:02 | |
| *** trinaths has quit IRC | 12:02 | |
| *** moshele has quit IRC | 12:03 | |
| *** jpena is now known as jpena|lunch | 12:03 | |
| *** tiendc has quit IRC | 12:04 | |
| *** trinaths has joined #openstack-nova | 12:04 | |
| *** moshele has joined #openstack-nova | 12:05 | |
| *** wlfightup has quit IRC | 12:05 | |
| *** trinaths1 has quit IRC | 12:07 | |
| *** dave-mcc_ has joined #openstack-nova | 12:09 | |
| *** moshele has quit IRC | 12:11 | |
| *** MVenesio has joined #openstack-nova | 12:11 | |
| *** thorst has joined #openstack-nova | 12:11 | |
| *** markvoelker has quit IRC | 12:12 | |
| openstackgerrit | Alex Xu proposed openstack/nova-specs master: Add trait support in the allocation candidates API https://review.openstack.org/497713 | 12:13 |
| bauzas | gibi: sorry was outside, but back now, +W'd | 12:14 |
| *** edmondsw has joined #openstack-nova | 12:14 | |
| *** dtantsur|lunch is now known as dtantsur | 12:15 | |
| gibi | bauzas: thanks | 12:16 |
| *** litao__ has quit IRC | 12:16 | |
| gibi | mdbooth: I'm on a meeting but opened the review so I will check it | 12:16 |
| efried | ratailor Did you get your questions answered about the osc microversion? | 12:17 |
| *** manasm has joined #openstack-nova | 12:17 | |
| ratailor | efried, not yet. could you help ? | 12:17 |
| efried | ratailor Your observation is correct, and by design. As for the reasoning behind it... that's complicated. | 12:18 |
| efried | ratailor But if you want a specific microversion in osc, you can ask for it with an env var, CLI opt, or (I think) conf var. | 12:18 |
| bauzas | efried: if the microversion is not asking for a new attribute :p | 12:19 |
| efried | Yeah, there's that :) | 12:19 |
| mdbooth | efried: OSC will happily send invalid requests because it doesn't understand those microversions, though. | 12:19 |
| manasm | bauzas: running into an issue with resize on an upgraded env from Ocata to Pike, the resize fails at following call in scheduler_utils.setup_instance_group method - | 12:20 |
| mdbooth | At that point, the user might as well just open telnet and type the http themselves :) | 12:20 |
| efried | ratailor A lot of it has to do with the fact, as bauzas and mdbooth are pointing out, that osc has not maintained functional parity with microversions. | 12:20 |
| manasm | request_spec.instance_group.hosts = list(group_info.hosts) | 12:20 |
| *** moshele has joined #openstack-nova | 12:20 | |
| *** sree has joined #openstack-nova | 12:20 | |
| manasm | bauzas: this is because the instance_group is set to None in the request_spec. | 12:20 |
| manasm | Is there a known issue around that? | 12:21 |
| bauzas | manasm: do you have an open bug for that? | 12:21 |
| * mdbooth really wishes CI would short-circuit when it fails. | 12:21 | |
| manasm | not yet | 12:21 |
| ratailor | efried, To make it as stable as possible. right ? | 12:21 |
| ratailor | efried, and backward compatible. | 12:21 |
| efried | ratailor The reasoning may not be quite that high-minded :) | 12:21 |
| ratailor | efried, cool | 12:21 |
| mdbooth | This is a data corruptor with a simple fix: https://review.openstack.org/#/c/507202/ | 12:22 |
| bauzas | manasm: AFAIR, we create a new InstanceGroup object when looking at the primitive dicts | 12:22 |
| ratailor | efried, Thanks! | 12:22 |
| efried | ratailor If you want more history/explanation on the osc side, find dtroyer. I don't remember offhand which channel is the accepted one for talking about the CLI, but I know he's in #openstack-dev | 12:22 |
| mdbooth | Affects libvirt live migration with attached volumes | 12:22 |
| efried | ratailor Possibly #openstack-sdks | 12:23 |
| ratailor | efried, sure, Thanks! | 12:23 |
| *** gbarros has joined #openstack-nova | 12:24 | |
| efried | ratailor yw, good luck. | 12:24 |
| ratailor | efried, actually I was searching him only, but didn't found in core openstack channels. | 12:24 |
| ratailor | efried, Thanks for your help! | 12:24 |
| *** tylerderosagrund has quit IRC | 12:24 | |
| efried | It may be a tad early for him to be on just yet, but he's at least got bouncers in those two channels. | 12:24 |
| manasm | bauzas: yes, we have the new InstanceGroup object but the we are trying to assign it to request_spec.instance_group which is None. | 12:25 |
| bauzas | efried: ratailor: FWIW, I already provided a point for the Sydney Forum about OSC vs. our CLI in https://etherpad.openstack.org/p/SYD-nova-brainstorming L35 | 12:25 |
| bauzas | I'd love to hear operators about what they use and how | 12:26 |
| ratailor | bauzas, ++1 | 12:27 |
| *** acormier has joined #openstack-nova | 12:27 | |
| bauzas | manasm: you said when you resize? | 12:28 |
| bauzas | manasm: so we're getting the ReqSpec from the DB first | 12:28 |
| bauzas | if there was no group yet for that instance, then the ReqSpec.instance_group field could be None | 12:28 |
| manasm | yes, btw we get the object from db using - objects.InstanceGroup.get_by_instance_uuid(context, instance_uuid), and if it is None then we don't hit the issue. | 12:28 |
| *** udesale has quit IRC | 12:29 | |
| *** armax has joined #openstack-nova | 12:30 | |
| *** yufei has quit IRC | 12:31 | |
| *** acormier has quit IRC | 12:31 | |
| bauzas | manasm: wait, I need to look at the code for resize | 12:32 |
| bauzas | do you have a stacktrace to share? | 12:32 |
| *** markvoelker has joined #openstack-nova | 12:32 | |
| *** pchavva has joined #openstack-nova | 12:34 | |
| *** hemna__ has joined #openstack-nova | 12:34 | |
| *** manasm has quit IRC | 12:36 | |
| *** trinaths has quit IRC | 12:36 | |
| *** armax has quit IRC | 12:37 | |
| *** yamamoto has quit IRC | 12:38 | |
| *** trinaths has joined #openstack-nova | 12:38 | |
| kashyap | mdbooth: FWIW, looks good. Compact summary, and a nice fix: https://review.openstack.org/#/c/507202/2 | 12:38 |
| mdbooth | It breaks some tempest tests, though :/ | 12:39 |
| mdbooth | Probably means I didn't think of something. | 12:40 |
| mdbooth | It does look obvious, though. | 12:40 |
| stephenfin | mdbooth: Reviewed both. Tidy job | 12:40 |
| * mdbooth hasn't investigated the tempest failures yet, though. | 12:40 | |
| kashyap | mdbooth: Wonder if you want to link to to the virDomainMigrateFlags page: https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainMigrateFlags | 12:40 |
| kashyap | Which talks about WTF actually the flag VIR_MIGRATE_NON_SHARED_INC is | 12:40 |
| kashyap | If you think it's not needed, disregard me | 12:41 |
| kashyap | Maybe not required. Those who want to know about it, a simple look up will fetch it | 12:42 |
| kashyap | BTW, the "tempest-dsvm-full-xenial" succeeded | 12:44 |
| *** liverpooler has joined #openstack-nova | 12:45 | |
| *** zhurong has quit IRC | 12:46 | |
| *** jaypipes has joined #openstack-nova | 12:49 | |
| *** lyan has joined #openstack-nova | 12:49 | |
| *** artom has joined #openstack-nova | 12:50 | |
| *** artom has quit IRC | 12:51 | |
| kashyap | Ah, it's the "gate-grenade-dsvm-neutron-ubuntu-xenial" that failed. | 12:51 |
| *** artom has joined #openstack-nova | 12:51 | |
| mdbooth | kashyap: Yep, in the block migration tests :) | 12:52 |
| *** artom has quit IRC | 12:52 | |
| mdbooth | I suspect the failure there is probably related ;) | 12:52 |
| *** bnemec has joined #openstack-nova | 12:52 | |
| mdbooth | kashyap: If you have time to investigate those failures, btw, I have to do something else right now. Otherwise I'll get to them later. | 12:52 |
| *** artom has joined #openstack-nova | 12:52 | |
| kashyap | mdbooth: I'm looking at the log - http://logs.openstack.org/02/507202/2/check/gate-tempest-dsvm-py35-ubuntu-xenial/8485b63/console.html | 12:52 |
| kashyap | I'll note on the review if I learn something new from the log | 12:53 |
| openstackgerrit | Yikun Jiang proposed openstack/nova master: Update Instance action's updated_at when action event updated. https://review.openstack.org/507473 | 12:53 |
| * kashyap above meant: 'gate-grenade-dsvm-neutron-multinode-live-migration-nv' | 12:54 | |
| *** gbarros has quit IRC | 12:54 | |
| kashyap | So the right log is - http://logs.openstack.org/02/507202/2/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/b210147/console.html | 12:54 |
| *** yamamoto has joined #openstack-nova | 12:55 | |
| openstackgerrit | konstantin proposed openstack/nova master: switch from filesystem to disk for parallels containers https://review.openstack.org/506687 | 12:55 |
| openstackgerrit | konstantin proposed openstack/nova master: don't add device address if there is no any units https://review.openstack.org/506686 | 12:55 |
| *** bhagyashri_s has quit IRC | 12:55 | |
| *** pooja_jadhav has quit IRC | 12:56 | |
| *** neha_alhat has quit IRC | 12:56 | |
| openstackgerrit | Yikun Jiang proposed openstack/nova master: Update Instance action's updated_at when action event updated. https://review.openstack.org/507473 | 12:57 |
| *** erlon has joined #openstack-nova | 12:57 | |
| *** esberglu has joined #openstack-nova | 12:57 | |
| *** sree has quit IRC | 12:58 | |
| *** sree has joined #openstack-nova | 12:58 | |
| *** sree has quit IRC | 12:59 | |
| *** sree has joined #openstack-nova | 12:59 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: nova.utils.get_ksa_adapter() https://review.openstack.org/488137 | 12:59 |
| *** Dinesh_Bhor has quit IRC | 12:59 | |
| *** sree has quit IRC | 13:00 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Use ksa adapter for placement conf & requests https://review.openstack.org/492247 | 13:00 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-vif stable/ocata: Updated from global requirements https://review.openstack.org/490256 | 13:00 |
| *** sree has joined #openstack-nova | 13:00 | |
| *** jpena|lunch is now known as jpena | 13:01 | |
| *** alexchadin has quit IRC | 13:01 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Get auth from context for glance endpoint https://review.openstack.org/490057 | 13:02 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Get auth from context for glance endpoint https://review.openstack.org/490057 | 13:02 |
| openstackgerrit | Sean Dague proposed openstack/nova master: Break out BasicTestCase https://review.openstack.org/507253 | 13:06 |
| openstackgerrit | Sean Dague proposed openstack/nova master: Don't use mock.patch.stopall https://review.openstack.org/507527 | 13:06 |
| openstackgerrit | Sean Dague proposed openstack/nova master: Remove REQUIRES_LOCKING as nothing needs process locking in the tests https://review.openstack.org/507528 | 13:06 |
| openstackgerrit | Sean Dague proposed openstack/nova master: WIP: demonstrate no use of external locking https://review.openstack.org/507529 | 13:06 |
| *** pooja_jadhav has joined #openstack-nova | 13:06 | |
| *** yamamoto has quit IRC | 13:06 | |
| *** bhagyashris has joined #openstack-nova | 13:07 | |
| *** neha_alhat has joined #openstack-nova | 13:07 | |
| *** lucasxu has joined #openstack-nova | 13:07 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Don't fix protocol-less glance api_servers anymore https://review.openstack.org/505317 | 13:08 |
| *** Dinesh_Bhor has joined #openstack-nova | 13:08 | |
| *** ratailor has quit IRC | 13:10 | |
| *** yingjun has joined #openstack-nova | 13:11 | |
| *** mriedem has joined #openstack-nova | 13:11 | |
| openstackgerrit | Yikun Jiang proposed openstack/nova master: Update Instance action's updated_at when action event updated. https://review.openstack.org/507473 | 13:12 |
| sdague | gibi: so, interesting fact from this morning, I'm pretty convinced we don't need any of the REQUIRES_LOCKING code | 13:16 |
| *** gbarros has joined #openstack-nova | 13:16 | |
| sdague | gibi: also, if you are able to take a look at the qemu 2.10 support patch, that would be cool - https://review.openstack.org/#/c/505673/ | 13:20 |
| sdague | or bauzas | 13:20 |
| *** shaner has quit IRC | 13:21 | |
| bauzas | ack | 13:23 |
| *** baoli has joined #openstack-nova | 13:23 | |
| *** smatzek has quit IRC | 13:27 | |
| *** udesale has joined #openstack-nova | 13:27 | |
| *** cleong has joined #openstack-nova | 13:29 | |
| *** jangutter has joined #openstack-nova | 13:32 | |
| *** felipemonteiro has joined #openstack-nova | 13:32 | |
| *** avolkov has quit IRC | 13:33 | |
| *** felipemonteiro_ has joined #openstack-nova | 13:34 | |
| *** manasm has joined #openstack-nova | 13:35 | |
| mdbooth | stephenfin: Eurgh. So it seems that in python2.7 libvirt binding's migrateToURI3() barfs if destination_xml is unicode. | 13:36 |
| *** felipemonteiro has quit IRC | 13:38 | |
| mdbooth | stephenfin: think I'm going to leave that etree.tostring() patch always producing unicode, but I'm going to add a 2/3 hack next to the migrateToURI3 which handles the difference, because IMHO the libvirt bindings should accept a unicode string there, and it's just saner. | 13:38 |
| kashyap | Matt, hmm, the assertThat() mismatch seems to be related to ID: | 13:39 |
| kashyap | 2017-09-26 12:26:17.199293 | 2017-09-26 12:26:17.198 | reference = u'ubuntu-xenial-2-node-rax-ord-11137317-923993' | 13:39 |
| kashyap | 2017-09-26 12:26:17.200739 | 2017-09-26 12:26:17.200 | actual = u'ubuntu-xenial-2-node-rax-ord-11137317' | 13:39 |
| mdbooth | kashyap: Yes, that's just asserting that the migration happened | 13:39 |
| mdbooth | i.e. assert id == the destination | 13:39 |
| *** moshele has quit IRC | 13:39 | |
| mdbooth | I hate python3 | 13:40 |
| mdbooth | And python2 | 13:40 |
| mdbooth | But the 2->3 split especially | 13:40 |
| kashyap | Ah, I see. (about the assert ID) | 13:40 |
| *** armax has joined #openstack-nova | 13:41 | |
| *** burt has joined #openstack-nova | 13:44 | |
| *** yingjun has quit IRC | 13:45 | |
| *** avolkov has joined #openstack-nova | 13:46 | |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Add Selection objects https://review.openstack.org/499239 | 13:49 |
| sdague | mriedem: I was diving through thinking more about https://review.openstack.org/#/c/507239/ last night, why do you think that external locking is required there? Because it should blow up if not provided but is needed | 13:51 |
| mriedem | sdague: i thought that also locked those tests to run serially | 13:52 |
| mriedem | maybe i should be using https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/fixture/lockutils.py#L22 | 13:52 |
| mriedem | ^ is actually what i started using | 13:52 |
| sdague | mriedem: serial to what? | 13:52 |
| openstackgerrit | konstantin proposed openstack/nova master: switch from filesystem to disk for parallels containers https://review.openstack.org/506687 | 13:52 |
| openstackgerrit | konstantin proposed openstack/nova master: don't add device address if there is no any units https://review.openstack.org/506686 | 13:52 |
| *** mnestratov has joined #openstack-nova | 13:53 | |
| *** cfriesen_ has joined #openstack-nova | 13:53 | |
| mriedem | sdague: to other tests | 13:53 |
| mriedem | like, things that would be mocking those nova.rpc globals | 13:53 |
| *** tidwellr has joined #openstack-nova | 13:53 | |
| sdague | all the tests in a worker run serially | 13:53 |
| *** eantyshev has joined #openstack-nova | 13:53 | |
| sdague | and the tests between workers are in completely different processes | 13:53 |
| sdague | with no shared state | 13:54 |
| eantyshev | mikal: Hello! Regarding your review https://review.openstack.org/#/c/492325 | 13:54 |
| *** trinaths has left #openstack-nova | 13:54 | |
| sdague | The reason we had REQUIRES_LOCKING at all was because oslo required a directory name or it exploded | 13:54 |
| sdague | but we feed those all temp directories anyway, they never cross lock between workers | 13:55 |
| *** belmoreira has joined #openstack-nova | 13:55 | |
| eantyshev | it fails on parallels virt_type, and I'd like to update it for you, don't you mind? | 13:56 |
| sdague | but I actually don't think that default behavior holds any more, and we can probably fully delete that variable anyway, as it definitely confuses people as to what it does | 13:56 |
| sdague | mriedem: https://review.openstack.org/#/c/507253/ - I was experimenting this morning | 13:56 |
| *** acormier has joined #openstack-nova | 13:57 | |
| *** eharney has joined #openstack-nova | 13:58 | |
| *** armax has quit IRC | 13:58 | |
| mriedem | avolkov: you might like to take a crack at this https://bugs.launchpad.net/nova/+bug/1719460 | 13:59 |
| openstack | Launchpad bug 1719460 in OpenStack Compute (nova) "(perf) Unnecessarily joining instance.services when listing instances regardless of microversion" [Medium,Triaged] | 13:59 |
| mriedem | should be pretty simple | 13:59 |
| *** awaugama has joined #openstack-nova | 14:01 | |
| *** shaner has joined #openstack-nova | 14:01 | |
| *** smatzek has joined #openstack-nova | 14:01 | |
| *** belmoreira has quit IRC | 14:01 | |
| *** artom has quit IRC | 14:02 | |
| *** crushil has joined #openstack-nova | 14:02 | |
| *** artom has joined #openstack-nova | 14:02 | |
| dansmith | mriedem: any outcome from your testing yesterday? | 14:03 |
| * gibi reads back | 14:03 | |
| mriedem | dansmith: i've got the clean slate, just getting setup to start the 2nd scenario with the 500 ACTIVE and 500 ERROR instances | 14:03 |
| mriedem | for the cell0 and cell1 listing | 14:03 |
| dansmith | okay | 14:03 |
| *** rmart04 has joined #openstack-nova | 14:03 | |
| mriedem | going to need to do that flavor thing because otherwise it's a 60 second rpc timeout per call to select_destinations | 14:04 |
| *** belmoreira has joined #openstack-nova | 14:04 | |
| dansmith | yeah | 14:04 |
| *** rmart04 has quit IRC | 14:04 | |
| gibi | mriedem, sdague: this also means that the problems we see with the rpc tests in bug 1685333 is not beacuase of the lack of locking | 14:06 |
| openstack | bug 1685333 in OpenStack Compute (nova) "Fatal Python error: Cannot recover from stack overflow. - in py35 unit test job" [High,Confirmed] https://launchpad.net/bugs/1685333 | 14:06 |
| sdague | mriedem: is there a reset on rpc variables that is needed that's not happening? | 14:06 |
| mriedem | sdague: the TestRPC class does a reset per test method | 14:07 |
| *** yamamoto has joined #openstack-nova | 14:07 | |
| *** armax has joined #openstack-nova | 14:07 | |
| sdague | gibi: I don't see how it could be. It might be a deadlock | 14:07 |
| mriedem | dansmith: also came across this last night https://bugs.launchpad.net/nova/+bug/1719487 | 14:07 |
| openstack | Launchpad bug 1719487 in OpenStack Compute (nova) "nova-manage db archive_deleted_rows is not multi-cell aware" [Wishlist,Triaged] - Assigned to Zhenyu Zheng (zhengzhenyu) | 14:07 |
| sdague | the biggest issue though is it doesn't have the timeout bits in place, so it's hard to see what's going on | 14:07 |
| sdague | I think if we trigger the timeout we get a stack trace | 14:08 |
| gibi | sdague: that would be nice | 14:08 |
| dansmith | mriedem: meh | 14:08 |
| mriedem | meh?! | 14:08 |
| *** rmart04 has joined #openstack-nova | 14:08 | |
| *** Tom__ has joined #openstack-nova | 14:08 | |
| mriedem | it's wishlist, sure | 14:08 |
| dansmith | MEH | 14:09 |
| mriedem | gdi | 14:09 |
| gibi | sdague, mriedem: we are at the start of the cycle so I'm brave enough to try to remove the whole locking code and see what happens | 14:09 |
| mriedem | dansmith: oh yeah, also came across this last night https://review.openstack.org/#/c/502236/ | 14:09 |
| *** rmart04 has quit IRC | 14:09 | |
| mriedem | derp | 14:09 |
| dansmith | ack yeah | 14:11 |
| mriedem | gibi: you can be brave locally to start :) | 14:11 |
| sdague | gibi: yeh, well we should at least get the test_rpc under timeout control, regardless of the rest of it | 14:11 |
| *** Tom__ has quit IRC | 14:13 | |
| gibi | mriedem: I can definitly do that | 14:13 |
| avolkov | mriedem: ack | 14:13 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Make TestRPC inherit from the base nova TestCase https://review.openstack.org/507239 | 14:13 |
| mriedem | ^ removes the lock thing | 14:14 |
| mriedem | so we'll have the timeout fixture | 14:14 |
| *** yamamoto has quit IRC | 14:14 | |
| gibi | sdague: agree. I found one more place where we use the testtools.TestCase directly. I left a comment in your review | 14:15 |
| sdague | gibi: in the fixtures? | 14:15 |
| gibi | sdague: here https://github.com/openstack/nova/blob/62c4535a85f7d37f1c9da1e8a747f25ec63dc785/nova/tests/unit/api/openstack/test_requestlog.py#L38 | 14:16 |
| sdague | ah, cool, good catch | 14:16 |
| mriedem | i thought ^ was intentional | 14:16 |
| mriedem | for the placement split or something | 14:16 |
| gibi | sdague: I think fixtures are OK to derive from testtools.TestCase as we use fixtures like mixins | 14:17 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/pike: Fix --max-count handling for nova-manage cell_v2 map_instances https://review.openstack.org/507552 | 14:17 |
| sdague | gibi: yeh, some of the more advanced ones should see the timeout | 14:17 |
| *** baoli has quit IRC | 14:17 | |
| sdague | but I think that's follow on | 14:17 |
| sdague | mriedem: it's a good question | 14:17 |
| *** baoli has joined #openstack-nova | 14:17 | |
| gibi | sdague, mriedem: at least this request_log test should be also under timeout control | 14:18 |
| sdague | gibi: so, I'd actually rather handle nova/tests/unit/api/openstack/test_requestlog.py as follow on, because those actually do most of the fixture setup (except the timeout one) manually | 14:18 |
| sdague | so it's going to be a bit more extensive change there | 14:18 |
| sdague | I do agree that we should get that under timeout control | 14:18 |
| sdague | but test_rpc is failing a lot now | 14:19 |
| gibi | sdague: I'm OK with that approach. Then I'm +2 on your patch introducing BasicTestCase | 14:19 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Fix --max-count handling for nova-manage cell_v2 map_instances https://review.openstack.org/507556 | 14:19 |
| openstackgerrit | Matt Riedemann proposed openstack/nova stable/newton: Fix --max-count handling for nova-manage cell_v2 map_instances https://review.openstack.org/507557 | 14:20 |
| *** rmart04 has joined #openstack-nova | 14:20 | |
| *** rmart04 has quit IRC | 14:20 | |
| manasm | bauzas: here is the exception I saw with the resize -2017-09-22 07:49:16.377 13573 ERROR nova.api.openstack.extensions File "/usr/lib/python2.7/site-packages/nova/scheduler/utils.py", line 567, in setup_instance_group | 14:20 |
| manasm | 2017-09-22 07:49:16.377 13573 ERROR nova.api.openstack.extensions request_spec.instance_group.hosts = list(group_info.hosts) | 14:20 |
| manasm | 2017-09-22 07:49:16.377 13573 ERROR nova.api.openstack.extensions | 14:20 |
| manasm | 2017-09-22 07:49:16.377 13573 ERROR nova.api.openstack.extensions AttributeError: 'NoneType' object has no attribute 'hosts' | 14:21 |
| manasm | 2017-09-22 07:49:16.377 13573 ERROR nova.api.openstack.extensions | 14:21 |
| jaypipes | mriedem, dansmith, gibi, sdague, bauzas: any of you noticed weird glitches in the new Gerrit web UI where the screen blinks and flashes when you open up long in-page comments? | 14:21 |
| dansmith | no | 14:21 |
| jaypipes | hmmm | 14:21 |
| *** eantyshev has left #openstack-nova | 14:21 | |
| gibi | at least not yet | 14:21 |
| jaypipes | it's a good thing I don't have Tourettes. | 14:21 |
| jaypipes | or epilepsy I gues | 14:21 |
| *** felipemonteiro_ has quit IRC | 14:22 | |
| jaypipes | efried: around? want to chat about "trait inheritance"... | 14:22 |
| efried | jaypipes I thought you'd never ask :* | 14:23 |
| jaypipes | lol | 14:23 |
| efried | jaypipes I have also experienced the gerrit UI glitchiness. | 14:23 |
| jaypipes | efried: oh, good (or bad...) at least I'm not the only one | 14:23 |
| efried | So yeah, trait inheritance... | 14:23 |
| efried | Did you see my long-winded comment with example based on (or at least attributed to) your response to my response etc. etc.? | 14:24 |
| jaypipes | efried: yeah, so it's absolutely correct that whatever is constructing the provider tree will need to attach traits at the appropriate provider leel | 14:24 |
| jaypipes | level | 14:24 |
| efried | Yuh. And the spec (ultimately the docs) will need to dictate what level(s) is/are "appropriate". | 14:24 |
| *** sree has quit IRC | 14:24 | |
| efried | Because the code is gonna hafta do some work to percolate 'em around, if that's supported. | 14:25 |
| jaypipes | efried: no, there's no percolating around... | 14:25 |
| sdague | jaypipes: url? | 14:26 |
| efried | jaypipes sdague Talking about this 'un: https://review.openstack.org/#/c/497713/6/specs/queens/approved/add-trait-support-in-allocation-candidates.rst@42 | 14:26 |
| jaypipes | sdague: are you talking about the gerrit thing or the nested providers thing? :) | 14:27 |
| efried | (oh, sdague unless you were... yeah...) | 14:27 |
| sdague | jaypipes: gerrit thing | 14:27 |
| jaypipes | sdague: mostly seen it happen on specs with long (>8 replies) inline comment "threads" | 14:27 |
| jaypipes | sdague: next time it happens I'll ping you a link | 14:27 |
| efried | For me, the gerrit thing is intermittent, happens when I'm expanding comments on a long page with lots of comments | 14:27 |
| jaypipes | ya | 14:27 |
| sdague | gerrit sends back a lot of ajax calls to get all those bits | 14:28 |
| efried | But not reproducible, cause I pop up to the review and back down and do the same thing and it doesn't happen the second time. | 14:28 |
| sdague | if it's gone slow, or your connection is weird, it might take a while for them to pile in and render | 14:28 |
| efried | I don't think it's ajax. Seems like client-side js focus() calls. | 14:28 |
| *** belmoreira has quit IRC | 14:28 | |
| jaypipes | sdague: nah, it's more like a loop in the UI that happens. | 14:28 |
| jaypipes | sdague: ya, what efried said :) | 14:28 |
| sdague | jaypipes: well, web console in chrome might help explain things | 14:29 |
| jaypipes | like it can't decide which comment to align to the top of the screen canvas | 14:29 |
| *** zhouyaguo has quit IRC | 14:29 | |
| jaypipes | sdague: when it happens again I'll ping ya | 14:29 |
| efried | I noticed focus bugs before the upgrade too, usually when composing a comment on a long page, it would jump around (shoving my comment box off the visible screen) | 14:29 |
| jaypipes | efried: yeah, that's happened for a long time | 14:29 |
| sdague | note, we also inject a lot of our own custom client side js to do the CI rollup, so it's entirely possible that is related to the issue | 14:30 |
| *** lucasxu has quit IRC | 14:30 | |
| sdague | regardless seeing if you can get an inspect console on the issue would be handy | 14:31 |
| *** Eran_Kuris has quit IRC | 14:32 | |
| jaypipes | sdague: will do | 14:32 |
| jaypipes | efried: you want me to response here or on the review? | 14:32 |
| efried | jaypipes Okay, so in the example in the comment I linked above: does that work as stated? | 14:32 |
| *** tetsuro has joined #openstack-nova | 14:33 | |
| efried | jaypipes Wouldn't mind discussing a bit, then putting a summary in the review. Just to make sure we're talking about the same thing, save time. | 14:33 |
| jaypipes | efried: your statement there, right after "How about" is spot on. | 14:33 |
| jaypipes | efried: this is excellent: | 14:34 |
| jaypipes | "With nested resource providers, traits defined on a parent RP are assumed to belong to all its child (descendant) RPs. However, traits defined on a child RP do not apply to the parent (ancestor) RPs. There is no implied sharing of traits within aggregates." | 14:34 |
| jaypipes | efried: even more explicit would be pointing out that aggregates don't actually have *any* traits associated to themselves at all (there's no aggregate_metadata table like there is in Nova) | 14:34 |
| jaypipes | efried: only resource providers have traits associated with them. | 14:34 |
| efried | Rightright, point being that RP1 doesn't inherit any traits from RP2 just because they're in the same aggregate. | 14:35 |
| jaypipes | efried: correct. it's worth spelling that out. aggregates are only grouping mechanisms, nothing more. | 14:35 |
| efried | jaypipes Okay, cool. So traits are inherited in NRPs, downwards but not upwards. And the example below that sentence would work as described. I guess the implementation details aren't important, but I'm a bit curious how it would work if you didn't actually internally copy the traits from the parent to its children. | 14:36 |
| *** coreywright has quit IRC | 14:37 | |
| jaypipes | efried: don't worry about the implementation details of the queries at this point. | 14:37 |
| efried | jaypipes Roger that. So okay, it sounds like we're in agreement. Thanks for the talk. | 14:38 |
| jaypipes | efried: just typing up in the review... gimme a few | 14:39 |
| efried | ++ | 14:39 |
| *** ratailor has joined #openstack-nova | 14:43 | |
| jaypipes | alex_xu: not sure if you're still up, but if you can make the few requested changes on https://review.openstack.org/#/c/497713/ from me and efried, I'm ready to +2 that one. | 14:43 |
| ratailor | sdague, you around ? | 14:43 |
| openstackgerrit | Matthew Booth proposed openstack/nova master: Make etree.tostring() emit unicode everywhere https://review.openstack.org/507488 | 14:44 |
| openstackgerrit | Matthew Booth proposed openstack/nova master: libvirt: Don't VIR_MIGRATE_NON_SHARED_INC without migrate_disks https://review.openstack.org/507202 | 14:44 |
| ratailor | sdague could you have a look at https://review.openstack.org/#/c/504885/ | 14:44 |
| *** lbragstad has quit IRC | 14:44 | |
| *** cdent has joined #openstack-nova | 14:45 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Moving more utils to ServerResourceAllocationTestBase https://review.openstack.org/499539 | 14:46 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: factor out compute service start in ServerMovingTest https://review.openstack.org/503037 | 14:46 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test resource allocation during soft delete https://review.openstack.org/495159 | 14:46 |
| *** moshele has joined #openstack-nova | 14:46 | |
| *** mdnadeem has quit IRC | 14:46 | |
| openstackgerrit | Moshe Levi proposed openstack/nova master: Don't overwrite binding-profile https://review.openstack.org/505613 | 14:46 |
| *** hongbin has joined #openstack-nova | 14:47 | |
| gibi | sdague: I'm looking at https://review.openstack.org/#/c/505673 (qemu 2.10 support). Do we have a way to unit test the change you made in the libvirt/driver.py ? | 14:48 |
| *** ratailor has quit IRC | 14:50 | |
| *** coreywright has joined #openstack-nova | 14:51 | |
| *** ratailor has joined #openstack-nova | 14:51 | |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: Change 'InstancePCIRequest' spec field https://review.openstack.org/449257 | 14:52 |
| openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova master: Read Neutron port 'binding_profile' during boot https://review.openstack.org/507481 | 14:52 |
| *** ralonsoh_ is now known as ralonsoh | 14:52 | |
| *** lbragstad has joined #openstack-nova | 14:54 | |
| gibi | if somebody want some easy patches to review then I'd like to suggest a test improvement series starts https://review.openstack.org/#/c/499539 and mriedem already +2 on it. | 14:54 |
| bauzas | jaypipes: I did noticed | 14:55 |
| *** eharney has quit IRC | 14:56 | |
| bauzas | jaypipes: Firefox FTW | 14:56 |
| *** sshwarts has quit IRC | 14:56 | |
| jaypipes | bauzas: I'm on firefox... | 14:56 |
| bauzas | when I say FTW, it's sarcastic | 14:56 |
| bauzas | so yeah, I'm hitted too | 14:56 |
| bauzas | I suspected a French regulation cause, but looks like it's not :p | 14:57 |
| *** gyee has joined #openstack-nova | 14:57 | |
| mriedem | gibi: good point, there is no unit test coverage of the libvirt driver change or the images change to use the new flag | 14:58 |
| *** gyee has quit IRC | 14:58 | |
| mriedem | gibi: -1 that sonofa | 14:58 |
| *** gyee has joined #openstack-nova | 14:58 | |
| gibi | mriedem: I think the image part is covered by https://review.openstack.org/#/c/505673/5/nova/tests/unit/virt/libvirt/test_utils.py@181 | 14:59 |
| bauzas | mriedem: not sure I love assigning a new module attribute for the QEMU version, but I guess it's probably the only easy way | 15:00 |
| *** manasm has quit IRC | 15:00 | |
| bauzas | mriedem: I'm thinking of cases where you don't reload your module and you're doomed | 15:00 |
| *** Oku_OS is now known as Oku_OS-away | 15:00 | |
| mriedem | oh i see | 15:00 |
| bauzas | but I guess upgrading qemu would be part of a nova minor upgrade as well, so that would require restarting n-cpu as well | 15:01 |
| mriedem | bauzas: jamespage went an alternate route where the version was passed down through the many plumbing layers, and it was quickly unmanageable | 15:01 |
| mriedem | another alternative is handling the error, parsing the output and retrying with the flag, which also sucks | 15:01 |
| bauzas | mriedem: yeah, I was thinking the image module wasn't written like that, but unfortunately it is | 15:01 |
| *** lajoskatona has left #openstack-nova | 15:01 | |
| bauzas | it's just an helper module | 15:02 |
| bauzas | so no way to pass a flag but just adding a global var which sucks in my mind but which sucks less than the other possibilities you envisaged | 15:02 |
| *** moshele has quit IRC | 15:05 | |
| *** slaweq_ has joined #openstack-nova | 15:10 | |
| mriedem | weird, seeing this in the logs when creating 500 servers using the fake driver | 15:10 |
| mriedem | Sep 26 15:09:38 devstack nova-compute[30351]: DEBUG nova.compute.resource_tracker [None req-f0c0e899-18e6-47f9-b13b-34829646d07e demo demo] Instance 62604071-77c2-46e1-9f57-d7192edc3f82 has been deleted (perhaps locally). Deleting allocations that remained for this instance against this compute host: {u'resources': {u'VCPU': 1, u'MEMORY_MB': 512, u'DISK_GB': 1}}. {{(pid=30351) _remove_deleted_instances_allocations /opt/stack/ | 15:10 |
| mriedem | /nova/compute/resource_tracker.py:1171}} | 15:10 |
| mriedem | Sep 26 15:09:38 devstack nova-compute[30351]: INFO nova.scheduler.client.report [None req-f0c0e899-18e6-47f9-b13b-34829646d07e demo demo] Deleted allocation for instance 62604071-77c2-46e1-9f57-d7192edc3f82 | 15:10 |
| *** eharney has joined #openstack-nova | 15:11 | |
| mriedem | doesn't really matter for what i'm testing, but it's odd | 15:11 |
| *** lucasxu has joined #openstack-nova | 15:13 | |
| *** penick has joined #openstack-nova | 15:15 | |
| *** gbarros has quit IRC | 15:15 | |
| *** felipemonteiro has joined #openstack-nova | 15:15 | |
| mriedem | stephenfin: what's your actual -1 on this now? https://review.openstack.org/#/c/457532/ | 15:15 |
| mriedem | i'd like to see this get in so i can use it in devstack | 15:15 |
| mriedem | dtroyer told us to make the command names "openstack resource provider <action>" | 15:15 |
| mriedem | to be consistent with everything else in osc | 15:15 |
| mriedem | being new to osc, i'm going to follow his recommendations | 15:15 |
| *** rnoriega_ is now known as rnoriega | 15:16 | |
| stephenfin | mriedem: There isn't one. Removed now | 15:16 |
| mriedem | thanks | 15:16 |
| mriedem | and yeah, it's a bit large, but it lays the base crud ops for RPs | 15:16 |
| mriedem | plus the common framework stuff | 15:16 |
| *** felipemonteiro_ has joined #openstack-nova | 15:16 | |
| *** andreas_s has quit IRC | 15:18 | |
| *** brault has quit IRC | 15:19 | |
| *** felipemonteiro has quit IRC | 15:20 | |
| *** brault has joined #openstack-nova | 15:20 | |
| *** chyka has joined #openstack-nova | 15:21 | |
| *** felipemonteiro_ has quit IRC | 15:21 | |
| cdent | jaypipes: one of the other things I did almost immediately after you asked was limiting aggregate checks: https://review.openstack.org/#/c/489633/ | 15:21 |
| *** tonyb has quit IRC | 15:22 | |
| sdague | gibi: it's tested in images.py | 15:23 |
| sdague | mriedem / gibi: https://review.openstack.org/#/c/505673/5/nova/tests/unit/virt/libvirt/test_utils.py | 15:23 |
| jaypipes | cdent: cool, will check shortly. | 15:23 |
| mriedem | sdague: i think he meant there is no unit test for the driver code setting the variable in https://review.openstack.org/#/c/505673/5/nova/tests/unit/virt/libvirt/test_driver.py | 15:23 |
| gibi | sdague: that test sets the images.QEMU_VERSION directly, but you have some code in the driver.py that sets images.QEMU_VERSION | 15:23 |
| *** tonyb has joined #openstack-nova | 15:24 | |
| mriedem | like test_next_min_qemu_version_ok | 15:24 |
| openstackgerrit | Chris Dent proposed openstack/nova master: [placement] gabbi tests for shared custom resource class https://review.openstack.org/485209 | 15:24 |
| mriedem | gibi: you can -1 and i can add the test later | 15:25 |
| sdague | mriedem: he already did | 15:25 |
| mriedem | ok | 15:25 |
| gibi | :) | 15:26 |
| mriedem | dansmith: you said os reboot and restack would cleanup devstack? | 15:27 |
| dansmith | mriedem: reboot and stack | 15:27 |
| mriedem | ok | 15:27 |
| *** gbarros has joined #openstack-nova | 15:28 | |
| *** lyan has quit IRC | 15:32 | |
| openstackgerrit | Evgeny Antyshev proposed openstack/nova master: Add ploop procedures to privsep.libvirt https://review.openstack.org/507569 | 15:32 |
| *** sree has joined #openstack-nova | 15:32 | |
| *** psachin has quit IRC | 15:32 | |
| *** lyan has joined #openstack-nova | 15:32 | |
| *** gbarros has quit IRC | 15:34 | |
| *** lyan has quit IRC | 15:34 | |
| *** lyan has joined #openstack-nova | 15:34 | |
| *** mingyu has quit IRC | 15:34 | |
| *** gbarros has joined #openstack-nova | 15:36 | |
| *** mingyu has joined #openstack-nova | 15:36 | |
| *** sree has quit IRC | 15:37 | |
| *** gbarros has quit IRC | 15:37 | |
| *** jdillaman has joined #openstack-nova | 15:38 | |
| *** armax has quit IRC | 15:38 | |
| *** gbarros has joined #openstack-nova | 15:40 | |
| openstackgerrit | Sean Dague proposed openstack/nova master: Support qemu >= 2.10 https://review.openstack.org/505673 | 15:40 |
| sdague | gibi / mriedem unit test added | 15:40 |
| gibi | sdague: looking | 15:41 |
| *** felipemonteiro has joined #openstack-nova | 15:42 | |
| gibi | sdague: thanks, looks good, +2 | 15:45 |
| *** kristian__ has joined #openstack-nova | 15:45 | |
| sdague | mriedem: ok, you next, reminding you of the "be faster and merge things" principle from the PTG :) | 15:45 |
| dansmith | sdague: he's currently working on being fast and merge things on my instance list patch | 15:47 |
| dansmith | sdague: but I'll look at yours so he can continue to be fast and merge on my patch | 15:47 |
| sdague | :) | 15:47 |
| sdague | it's pretty small and should be straight forward | 15:47 |
| *** kristian__ has quit IRC | 15:48 | |
| * sdague reruns tests locally to figure out how the REQUIRED_LOCKING https://review.openstack.org/#/c/507528/ passed locally before | 15:48 | |
| *** kristian__ has joined #openstack-nova | 15:48 | |
| mriedem | sdague: i shouldn't approve my own change | 15:50 |
| sdague | mriedem: it's barely your change any more | 15:51 |
| *** manasm has joined #openstack-nova | 15:52 | |
| *** tssurya has quit IRC | 15:53 | |
| *** sbezverk has quit IRC | 16:00 | |
| *** jistr is now known as jistr|mtg | 16:01 | |
| gibi | I've just realized that I have to migrate the notification burndown chart from openshift 2 to 3 until end of September. This will be joyful | 16:02 |
| mdbooth | stephenfin: Are the changes to https://review.openstack.org/#/c/507488/ evil? | 16:02 |
| *** jdillaman has quit IRC | 16:02 | |
| mdbooth | Thinking specifically of the 'if six.py2:' in libvirt/guest.py | 16:03 |
| mdbooth | I still think it's saner to have unicode everywhere, and convert to something else at the point of use | 16:03 |
| dansmith | cdent: are you working your way up that migration uuid series? if so, I'll hold off pushing that fix you just identified until you have a chance to nit out on anything else | 16:04 |
| stephenfin | mdbooth: No, I'd probably do the same thing | 16:04 |
| dansmith | meaning, you can identify more nit-ish things and I can fix them, vs. just pushing for nits later | 16:04 |
| * stephenfin awaits 2020 and the death of Python 2.7 eagerly | 16:04 | |
| cdent | dansmith: yup (on phone at the moment though) | 16:04 |
| dansmith | cdent: okay, np | 16:05 |
| *** rcernin has quit IRC | 16:05 | |
| *** ratailor has quit IRC | 16:06 | |
| stephenfin | mdbooth: though I would like to see one of the tests slightly reworked, if that makes sense to you? | 16:06 |
| mdbooth | stephenfin: Sure thing. I'll take a look in the morning. | 16:07 |
| openstackgerrit | Merged openstack/nova master: Remove SCREEN_LOGDIR from devstack install setting https://review.openstack.org/507425 | 16:07 |
| stephenfin | (y) | 16:07 |
| *** udesale has quit IRC | 16:08 | |
| *** Apoorva has joined #openstack-nova | 16:09 | |
| *** mvk has quit IRC | 16:11 | |
| *** moshele has joined #openstack-nova | 16:15 | |
| *** armax has joined #openstack-nova | 16:15 | |
| *** yassine has quit IRC | 16:16 | |
| *** jistr|mtg is now known as jistr | 16:18 | |
| *** markmc has quit IRC | 16:20 | |
| *** armax has quit IRC | 16:20 | |
| bauzas | edleafe: jaypipes: cdent: I thought we would be talking of how we would lead reschedules in https://review.openstack.org/#/c/498830/ | 16:20 |
| *** markmc has joined #openstack-nova | 16:21 | |
| bauzas | edleafe: jaypipes: cdent: I see a couple of comments in that spec review, but have you settled down on discussing reschedules as being out of scope for that spec? | 16:21 |
| dansmith | cdent: ah I see you have +1s on most of the rest of the set anyway, so I'll just push | 16:21 |
| cdent | dansmith: i’m in the midst of the last one now | 16:22 |
| dansmith | cdent: ah okay then I'll wait | 16:22 |
| cdent | so give me a couple of minutes (phone call was shorter than expected) | 16:22 |
| dansmith | yeah not trying to rush you | 16:22 |
| openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: config drive https://review.openstack.org/409404 | 16:23 |
| dansmith | I just want to push that review button and get that squirt of dopamine you know :P | 16:23 |
| edleafe | bauzas: reschedules will be a different spec | 16:24 |
| bauzas | edleafe: so the spec is litterally just for mentioning which object the scheduler will return to conductor ? | 16:24 |
| rybridges | Hey guys. I am using the ocata release and am wondering if there is any way to print the user data associated with an instance that i own from the cli with the openstack client | 16:25 |
| bauzas | edleafe: looks uber too much | 16:25 |
| bauzas | I mean, super heavy | 16:25 |
| edleafe | bauzas: since this will be sent over RPC, we needed agreement on it so that we don't find ourselves changing it later | 16:26 |
| bauzas | I'd be up concentrating our minds on how we plan to pass that object | 16:26 |
| bauzas | edleafe: we did a couple of RPC changes that didn't require a spec fortunately | 16:26 |
| bauzas | but I leave the mic to mriedem | 16:26 |
| edleafe | bauzas: the idea is to get it close to correct before we make the change | 16:26 |
| dansmith | bauzas: specs are cheap | 16:26 |
| dansmith | if edleafe wants separate specs, I don't think there's a problem | 16:27 |
| edleafe | bauzas: and given the amount of discussion on the Selection object spec, I'd say it was a good thing to do | 16:27 |
| *** yufei has joined #openstack-nova | 16:27 | |
| dansmith | we should focus on getting the work done and not the process | 16:27 |
| bauzas | dansmith: well, I'd rather then look at code, but okay :) | 16:27 |
| bauzas | yeah that | 16:27 |
| edleafe | bauzas: the work is being done in parallel | 16:27 |
| *** kristian__ has quit IRC | 16:27 | |
| *** r-daneel has joined #openstack-nova | 16:27 | |
| bauzas | edleafe: well, okay | 16:28 |
| *** kristian__ has joined #openstack-nova | 16:28 | |
| bauzas | edleafe: the thing is, if you want a spec, fine with me, but then precise the scope | 16:28 |
| *** yufei has left #openstack-nova | 16:28 | |
| bauzas | since it was a work item, I was expecting more | 16:28 |
| dansmith | mriedem: so I was just looking at this for evac and live migration.. this method _moves_ allocations to the destination, not copies AFAICT: https://github.com/openstack/nova/blob/master/nova/scheduler/utils.py#L222-L224 | 16:29 |
| dansmith | mriedem: is that right? | 16:29 |
| bauzas | if the spec isn't targeting to mention how reschedules would be done, fair enough but just make sure you clearly scope that | 16:29 |
| cdent | dansmith: dobne | 16:29 |
| cdent | done! | 16:29 |
| dansmith | cdent: yes the last one is less done | 16:30 |
| * cdent nods | 16:30 | |
| mriedem | dansmith: copies | 16:32 |
| *** kristian__ has quit IRC | 16:32 | |
| dansmith | mriedem: oh does claim_resources() do the doubling thing? | 16:33 |
| mriedem | no | 16:33 |
| mriedem | it takes the allocations for the instance on the source node, and makes those same allocations for the instance on the dest node | 16:33 |
| dansmith | which will erase the source allocation | 16:33 |
| mriedem | it's basically what the scheduler would do, | 16:33 |
| dansmith | because... only one consumer | 16:33 |
| mriedem | oh it calls claim_resources, | 16:33 |
| mriedem | so yeah it doubles | 16:33 |
| mriedem | this is the thing where force=True | 16:34 |
| mriedem | so we don't call the scheduler to double the allocs | 16:34 |
| mriedem | and i said i wanted to move back into the scheduler, but we'd need a skip_filters flag in select_destinations | 16:34 |
| dansmith | okay I didn't think claim_resources was the doubling one, but maybe so, I'll dig a bit | 16:34 |
| *** jdillaman has joined #openstack-nova | 16:34 | |
| mriedem | claim_resources calls the double stuff method | 16:34 |
| dansmith | okay | 16:34 |
| dansmith | cdent: can you look at my comment on the DRY thing and see if you buy what I'm sellin' ? | 16:35 |
| cdent | dansmith: I will buy that with an entire whole dollar, if you comment the plan | 16:38 |
| dansmith | ack | 16:38 |
| dansmith | cdent: you saw the "when we have an atomic operation we should remove this" right? | 16:40 |
| *** jangutter has quit IRC | 16:40 | |
| cdent | yes, but (unless I missed it) there’s no “this dupe with that other thing but we don’t care cuz” | 16:42 |
| dansmith | I will add more words | 16:42 |
| cdent | I’ll still buy it for a dollar even if you don’t | 16:42 |
| *** dtantsur is now known as dtantsur|afk | 16:42 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make allocation cleanup honor new by-migration rules https://review.openstack.org/498948 | 16:45 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Pre-create migration object https://review.openstack.org/498950 | 16:45 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Revert allocations by migration uuid https://review.openstack.org/498949 | 16:45 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Refactor resource tracker to account for migration allocations https://review.openstack.org/506419 | 16:45 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make migration uuid hold allocations for migrating instances https://review.openstack.org/506420 | 16:45 |
| *** yufei has joined #openstack-nova | 16:50 | |
| *** yufei has quit IRC | 16:51 | |
| openstackgerrit | Sean Dague proposed openstack/nova master: Move ploop commands to privsep. https://review.openstack.org/492325 | 16:51 |
| *** claudiub has quit IRC | 16:52 | |
| openstackgerrit | Sean Dague proposed openstack/nova master: Move ploop commands to privsep. https://review.openstack.org/492325 | 16:54 |
| *** derekh has quit IRC | 16:54 | |
| *** trinaths has joined #openstack-nova | 16:56 | |
| *** trinaths has left #openstack-nova | 16:56 | |
| *** trinaths1 has joined #openstack-nova | 16:57 | |
| *** trinaths1 has left #openstack-nova | 16:57 | |
| mriedem | notifications meeting in openstack-meeting-4 in 2 minutes | 16:58 |
| *** cdent has quit IRC | 16:58 | |
| gibi | ... and now it is started | 17:00 |
| *** baoli has quit IRC | 17:01 | |
| *** slaweq_ has quit IRC | 17:02 | |
| *** baoli has joined #openstack-nova | 17:02 | |
| *** abalutoiu has joined #openstack-nova | 17:07 | |
| *** rcernin has joined #openstack-nova | 17:11 | |
| *** manasm has quit IRC | 17:16 | |
| *** Swami has joined #openstack-nova | 17:23 | |
| openstackgerrit | Moshe Levi proposed openstack/nova master: Don't overwrite binding-profile https://review.openstack.org/505613 | 17:27 |
| *** crushil has quit IRC | 17:27 | |
| *** crushil has joined #openstack-nova | 17:27 | |
| *** kristian__ has joined #openstack-nova | 17:29 | |
| *** moshele has quit IRC | 17:32 | |
| openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: config drive https://review.openstack.org/409404 | 17:32 |
| *** jpena is now known as jpena|off | 17:32 | |
| *** kristian__ has quit IRC | 17:34 | |
| *** abalutoiu has quit IRC | 17:36 | |
| *** gszasz has quit IRC | 17:36 | |
| *** tesseract has quit IRC | 17:37 | |
| *** ragiman has quit IRC | 17:38 | |
| mriedem | dansmith: aha, i think i'm hitting issues in devstack where placement isn't getting cleaned up for instances that get 'local' deleted in the api | 17:38 |
| mriedem | not totally sure yet, but failing to burst 500 new instances, hitting NoValidHost | 17:38 |
| mriedem | and i assume it's placement b/c it's not the scheduler filters | 17:38 |
| *** gbarros has quit IRC | 17:38 | |
| dansmith | mriedem: and why do you have locally-deleted instances for this test? | 17:39 |
| melwitt | for local deletes, allocations aren't cleaned up till the compute host heals it | 17:39 |
| dansmith | right, what melwitt said | 17:39 |
| mriedem | mysql> select count(id) from consumers; | 17:39 |
| mriedem | +-----------+ | 17:39 |
| mriedem | | count(id) | | 17:39 |
| mriedem | +-----------+ | 17:39 |
| mriedem | | 2002 | | 17:39 |
| mriedem | +-----------+ | 17:39 |
| mriedem | 1 row in set (0.01 sec) | 17:39 |
| *** cdent has joined #openstack-nova | 17:39 | |
| mriedem | melwitt: there is no compute for these | 17:40 |
| mriedem | they failed during scheduling | 17:40 |
| mriedem | although yeah why would placement have allocations for these... | 17:40 |
| mriedem | wtf | 17:40 |
| mriedem | stack@devstack:~$ nova list | grep -c ERROR | 17:40 |
| mriedem | 1000 | 17:40 |
| melwitt | oh, hm | 17:40 |
| mriedem | so i've got 1000 instances in ERROR state, and 2002 consumers in the api db | 17:40 |
| *** artom has quit IRC | 17:41 | |
| melwitt | allocations are written at claim time? | 17:41 |
| mriedem | from the scheduler yeah | 17:41 |
| *** kristian__ has joined #openstack-nova | 17:41 | |
| melwitt | so that would explain the ones you do have. but I guess your point is why are there more allocation consumers than non error instances | 17:42 |
| *** kristia__ has joined #openstack-nova | 17:42 | |
| mriedem | that's because i've deleted 1000 over time | 17:42 |
| mriedem | i was hitting messaging timeouts between conductor and the scheduler earlier today, so had 500 in error which i needed to be active, so deleted all of those, restarted conductor and scheduler, and was able to create a single instance | 17:43 |
| mriedem | so tried with 500 more again | 17:43 |
| mriedem | and hit novalidhost on all of those | 17:43 |
| openstackgerrit | Merged openstack/nova master: cleanup test-requirements https://review.openstack.org/507063 | 17:44 |
| *** kristian__ has quit IRC | 17:45 | |
| *** gabor_antal has quit IRC | 17:46 | |
| *** jmlowe has quit IRC | 17:46 | |
| *** lbragstad has quit IRC | 17:47 | |
| *** gabor_antal has joined #openstack-nova | 17:48 | |
| *** moshele has joined #openstack-nova | 17:48 | |
| *** ralonsoh has quit IRC | 17:51 | |
| *** lbragstad has joined #openstack-nova | 17:51 | |
| *** ijw has joined #openstack-nova | 17:53 | |
| *** kristia__ has quit IRC | 17:54 | |
| *** ijw has quit IRC | 17:55 | |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make live migration hold resources with a migration allocation https://review.openstack.org/507638 | 17:55 |
| dansmith | jaypipes: cdent: ^ quick stab at the live migrate version of this | 17:55 |
| dansmith | it's probably rough at this point, but worth a look I think | 17:56 |
| *** vvargaszte has joined #openstack-nova | 17:58 | |
| *** ijw has joined #openstack-nova | 17:58 | |
| *** ijw has quit IRC | 17:58 | |
| *** ijw has joined #openstack-nova | 17:58 | |
| *** baoli has quit IRC | 18:00 | |
| *** baoli has joined #openstack-nova | 18:01 | |
| *** Apoorva_ has joined #openstack-nova | 18:02 | |
| *** vvargaszte has quit IRC | 18:04 | |
| *** Apoorva has quit IRC | 18:06 | |
| *** kristian__ has joined #openstack-nova | 18:08 | |
| *** Apoorva_ has quit IRC | 18:08 | |
| *** kristia__ has joined #openstack-nova | 18:08 | |
| *** Apoorva has joined #openstack-nova | 18:08 | |
| *** moshele has quit IRC | 18:09 | |
| *** kristi___ has joined #openstack-nova | 18:11 | |
| *** lucasxu has quit IRC | 18:11 | |
| *** dave-mcc_ is now known as dave-mccowan | 18:12 | |
| *** kristian__ has quit IRC | 18:12 | |
| *** kristia__ has quit IRC | 18:14 | |
| *** moshele has joined #openstack-nova | 18:17 | |
| *** jmlowe has joined #openstack-nova | 18:18 | |
| *** moshele has quit IRC | 18:20 | |
| *** gbarros has joined #openstack-nova | 18:21 | |
| *** xyang1 has joined #openstack-nova | 18:24 | |
| -openstackstatus- NOTICE: The infra team is continuing work to bring Zuul v3 online; expect service disruptions and please see https://docs.openstack.org/infra/manual/zuulv3.html for more information. | 18:24 | |
| *** slaweq_ has joined #openstack-nova | 18:27 | |
| *** xyang1 has quit IRC | 18:27 | |
| *** moshele has joined #openstack-nova | 18:31 | |
| cdent | dansmith: haven’t had a chance to give it a proper look, but saw a weird when skimming the live migrate thing | 18:32 |
| dansmith | lol | 18:32 |
| *** techstepper has joined #openstack-nova | 18:32 | |
| *** Apoorva has quit IRC | 18:33 | |
| dansmith | it's returning True-ish which is what I wanted for the functional tests | 18:33 |
| dansmith | so.. working as designed? :) | 18:33 |
| dansmith | s/returning/being/ | 18:33 |
| *** Apoorva has joined #openstack-nova | 18:33 | |
| openstackgerrit | Merged openstack/nova master: Set the Pike release version for scheduler RPC https://review.openstack.org/507245 | 18:34 |
| cdent | go python! | 18:34 |
| mriedem | wtf, so i can't create multiple instances, i get novalidhost, but i can create one at a time | 18:38 |
| melwitt | are you using multi-create? | 18:38 |
| mriedem | yeah | 18:38 |
| mriedem | wasn't a problem yesterday | 18:38 |
| melwitt | oh | 18:38 |
| mriedem | but i had a bit of a cleaner env yesterday | 18:38 |
| *** gabor_antal has quit IRC | 18:39 | |
| *** gabor_antal has joined #openstack-nova | 18:39 | |
| melwitt | multi-create will reject you if any one of min_count can't be accommodated. so one at a time would work if you're in that situation, if some/most of them fit | 18:39 |
| mriedem | yesterday i created 100, then like 200, then 500 more or something | 18:40 |
| mriedem | eventually got to 1000 | 18:40 |
| mriedem | i can just restack this env, but it makes me worry that we aren't properly cleaning up allocations somewhere | 18:41 |
| melwitt | yeah | 18:42 |
| melwitt | did you say yesterday you don't have computes, or something like that? I just wonder what happens with FakeDriver, if it somehow doesn't call the healing allocations code | 18:43 |
| mriedem | we don't heal since pike | 18:43 |
| mriedem | if you don't have computes < pike, we don't heal | 18:43 |
| mriedem | this is just a single compute, single node devstack | 18:44 |
| mriedem | with the fake driver and noop quota | 18:44 |
| melwitt | oh wait, sorry I was thinking of local delete | 18:44 |
| melwitt | I was trying to think if with FakeDriver, does the code that deletes allocations when an instance is deleted, run | 18:44 |
| melwitt | or if that even matters | 18:45 |
| *** vvargaszte has joined #openstack-nova | 18:45 | |
| mriedem | the compute manager cleans up allocations when an instance is deleted | 18:45 |
| dansmith | mriedem: we heal for deletes | 18:45 |
| dansmith | mriedem: did you archive them after local delete before you started up? | 18:45 |
| mriedem | i've been archiving yeah | 18:46 |
| melwitt | I'm not sure whether he had local deletes | 18:46 |
| dansmith | so that's why | 18:46 |
| dansmith | I thought he did | 18:46 |
| melwitt | I thought he did too but I'm getting a little confused | 18:46 |
| *** slaweq_ has quit IRC | 18:46 | |
| mriedem | yesterday i didn't have any instances in ERROR state, so they were all in the cell | 18:46 |
| mriedem | i deleted all of those and then archived cell0 and cell1 | 18:47 |
| *** moshele has quit IRC | 18:47 | |
| dansmith | deleted them locally? | 18:47 |
| mriedem | today i've been trying to get 500 ERROR during scheduling, and 500 ACTIVE | 18:47 |
| mriedem | based on the flavor i use | 18:47 |
| mriedem | dansmith: deleted via the api | 18:47 |
| *** slaweq_ has joined #openstack-nova | 18:47 | |
| dansmith | mriedem: with compute down or no? | 18:47 |
| mriedem | remember me complaining about how long that was taking yesterday? | 18:47 |
| mriedem | no | 18:47 |
| mriedem | took 2+ hours to delete 1000 ACTIVE instances | 18:47 |
| dansmith | I do, but I didn't remember all your details | 18:48 |
| mriedem | yeah i basically trying to get back to clean state before starting today | 18:48 |
| mriedem | so was archiving the db's last night | 18:48 |
| mriedem | btw, bauzas pointed this out before, but we log this way too many times | 18:49 |
| mriedem | Sep 26 18:44:37 devstack nova-compute[30351]: DEBUG nova.compute.resource_tracker [None req-992d494e-d328-4204-bcfe-80d926cf0a65 demo demo] We're on a Pike compute host in a deployment with all Pike compute hosts. Skipping auto-correction of allocations. {{(pid=30351) _update_usage_from_instance /opt/stack/nova/nova/compute/resource_tracker.py:1071}} | 18:49 |
| dansmith | mriedem: unrelated, see this: http://status.openstack.org/openstack-health/#/test/nova.tests.functional.test_servers.ServersTestV219.test_description_errors?duration=P3M | 18:52 |
| *** slaweq_ has quit IRC | 18:52 | |
| dansmith | mriedem: I think this test is occasionally taking up to 240s locally when it should be about 8s | 18:52 |
| mriedem | jesus | 18:53 |
| *** lucasxu has joined #openstack-nova | 18:53 | |
| dansmith | and I think it's because it creates a server that it never cleans up and then abruptly exits where we take down conductor before the compute service finishes waiting on a call or something | 18:53 |
| dansmith | so I have a patch to just make it clean up the server and I _think_ it's working | 18:53 |
| mriedem | the one weird spike in august is, weird | 18:53 |
| mriedem | https://bugs.launchpad.net/nova/+bug/1719714 | 18:53 |
| openstack | Launchpad bug 1719714 in OpenStack Compute (nova) "Excessive logging of "We're on a Pike compute host in a deployment with all Pike compute hosts."" [Medium,Confirmed] | 18:53 |
| dansmith | mriedem: it would have just been ordering reasons | 18:54 |
| dansmith | mriedem: note the rising tail at present too | 18:54 |
| *** mnestratov has quit IRC | 18:57 | |
| *** baoli has quit IRC | 18:57 | |
| *** baoli has joined #openstack-nova | 18:59 | |
| *** ianw|pto is now known as ianw | 19:01 | |
| *** egonzalez has joined #openstack-nova | 19:02 | |
| *** ijw has quit IRC | 19:03 | |
| *** mvk has joined #openstack-nova | 19:06 | |
| *** tbachman has quit IRC | 19:08 | |
| *** jaosorior has quit IRC | 19:09 | |
| *** moshele has joined #openstack-nova | 19:10 | |
| *** sahid has quit IRC | 19:11 | |
| *** elod has quit IRC | 19:12 | |
| mriedem | alright i'm just going to restack | 19:18 |
| mriedem | nuts to this | 19:18 |
| *** tbachman has joined #openstack-nova | 19:19 | |
| *** sree has joined #openstack-nova | 19:25 | |
| mriedem | dansmith: jaypipes: bauzas: https://review.openstack.org/#/c/498947/6 | 19:28 |
| mriedem | that test_servers thing is wrong | 19:28 |
| *** ijw has joined #openstack-nova | 19:28 | |
| *** vvargaszte has quit IRC | 19:28 | |
| openstackgerrit | Matthew Treinish proposed openstack/nova master: Add slowest command to tox.ini https://review.openstack.org/507657 | 19:29 |
| mtreinish | dansmith: ^^^ | 19:29 |
| mriedem | there are 2 tests for failures during evacaute on the dest | 19:29 |
| mriedem | 1. test_evacuate_claim_on_dest_fails - that is testing when the claim fails with ComputeResourcesUnavailable | 19:29 |
| mriedem | 2. test_evacuate_rebuild_on_dest_fails - that is testing when the claim is successful but the driver.rebuild method raises some exception | 19:29 |
| dansmith | mtreinish: sweet | 19:29 |
| jaypipes | mriedem: sorry, I disagree with you. | 19:30 |
| mriedem | i wrote those tests | 19:30 |
| *** sree has quit IRC | 19:30 | |
| mriedem | so please explain how i'm wrong that they are now made redundant in that change | 19:30 |
| jaypipes | mriedem: that test raising TestingException was not useful. Because TestingException isn't what is ever raised by any code. | 19:30 |
| mriedem | it's simulating the virt driver raising the error during rebuild | 19:30 |
| mriedem | AFTER the successful claim | 19:31 |
| mriedem | it could be ProcessExecutionError | 19:31 |
| mriedem | from driver.spawn() | 19:31 |
| mriedem | if you like | 19:31 |
| *** slaweq_ has joined #openstack-nova | 19:31 | |
| mriedem | these 2 tests are testing very specific failures | 19:31 |
| dansmith | mriedem: right but we don't run the claim teardown code in that case | 19:31 |
| mriedem | dansmith: correct, which is why we run the allocation cleanup manually | 19:32 |
| mriedem | and that's what that is testing | 19:32 |
| *** vladikr has quit IRC | 19:32 | |
| mriedem | the test you changed isn't meant to test drop_move_claim | 19:32 |
| mriedem | the docstring explains that | 19:32 |
| jaypipes | mriedem: if the point of the test (as is in that docstring) is to ensure allocations are cleaned up after a failed rebuild, then the test should raise the exception that would be raised *after* a claim has been made for the new resources. | 19:32 |
| dansmith | jaypipes: he's saying another one does that | 19:32 |
| mriedem | jaypipes: you realize the virt drivers can raise any kinds of crazy shit right? | 19:33 |
| dansmith | mriedem: so in this case you want the test to validate that the allocations _don't_ get cleaned up is that right? | 19:33 |
| *** ijw has quit IRC | 19:33 | |
| jaypipes | mriedem: Matt, I'm trying to be civil. | 19:33 |
| mriedem | https://review.openstack.org/#/c/499877/ | 19:34 |
| *** rcernin has quit IRC | 19:34 | |
| dansmith | this is what it's testing: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2800-L2827 | 19:35 |
| dansmith | the except exception case of that | 19:35 |
| mriedem | so ^ is testing that drop_move_claim removes the allocation when the claim was successful but the virt driver raised some exception | 19:35 |
| *** kristi___ has quit IRC | 19:35 | |
| jaypipes | mriedem: OK, I see that now. | 19:35 |
| *** ijw has joined #openstack-nova | 19:35 | |
| mriedem | https://review.openstack.org/#/c/499874/ added the other test | 19:36 |
| mriedem | that was a recreate test for a bug | 19:36 |
| mriedem | fixed in https://review.openstack.org/#/c/499878/ | 19:36 |
| dansmith | mriedem: we get it | 19:37 |
| dansmith | mriedem: can you answer my question above about what you want it to do? | 19:37 |
| mriedem | the test should go back to whatever it was testing | 19:39 |
| mriedem | which is the case that the claim passes, but the virt driver raises | 19:39 |
| mriedem | so we'd remove the allocation via drop_move_claim before | 19:39 |
| dansmith | right, but you assert some behavior that happens inside drop_move_claim | 19:40 |
| dansmith | which no longer happens | 19:40 |
| *** ijw has quit IRC | 19:40 | |
| mriedem | then that drop_move_claim behavior has to be replayed elsewhere i guess | 19:40 |
| jaypipes | what if the it's a same-host rebuild? :( | 19:40 |
| mriedem | there is no claim for a same host rebuild | 19:40 |
| jaypipes | k | 19:40 |
| mriedem | so you wouldn't hit ComputeResourcesUnavailable | 19:40 |
| dansmith | exactly, but you could hit other exceptions | 19:41 |
| jaypipes | mriedem: but you *would* hit the TestingException "crazy shit" | 19:41 |
| jaypipes | mriedem: and you're asserting that we'd delete the allocation against the instance in that case, right? | 19:41 |
| mriedem | that doesn't have anything to do with dropping an allocation though | 19:41 |
| mriedem | no | 19:41 |
| jaypipes | mriedem: oh, sorry, you're asserting that the *update_available_resource()* call would clean up allocations for a failed build? | 19:42 |
| jaypipes | rebuild. | 19:42 |
| mriedem | no | 19:42 |
| dansmith | no | 19:42 |
| jaypipes | guh | 19:42 |
| dansmith | the test _is_ asserting that the dest host's allocation was cleaned up by drop_move_claim | 19:42 |
| mriedem | we don't ever want to remove allocations for a *rebuild* | 19:42 |
| mriedem | the tests are specifically for evacuate | 19:42 |
| mriedem | where the scheduler creates allocations on the dest host | 19:43 |
| mriedem | we fail the evacuate on the dest host, so we need to remove those allocatoins created by the scheduler | 19:43 |
| *** armax has joined #openstack-nova | 19:43 | |
| dansmith | there's a specific reason why I made this change, | 19:44 |
| dansmith | and I talked it through with jaypipes which is why I made this | 19:44 |
| mriedem | i'm sorry for being grouchy about this, | 19:44 |
| mriedem | but i've spent the better part of the last 6 weeks fixing these allocation bugs, | 19:44 |
| dansmith | so I'll have to go re-load all my context on this before I can really think about it | 19:44 |
| mriedem | so being told i don't understand the test pisses me off | 19:44 |
| jaypipes | mriedem: understood. and you're saying that you want the drop_move_claim() to remove those resources when ComputeResourcesUnvailable is raised but you want update_available_resource() to delete the allocations when a virt driver exception is raised? | 19:44 |
| *** Apoorva has quit IRC | 19:44 | |
| dansmith | jaypipes: no | 19:44 |
| *** vladikr has joined #openstack-nova | 19:45 | |
| dansmith | mriedem: without offending your test sensibilities, you see this right? https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2812 | 19:45 |
| dansmith | that's what *should* have been deleting the target's allocation | 19:45 |
| dansmith | only in the unavailable case | 19:46 |
| dansmith | but in reality, we were always doing it for the other cases as well | 19:46 |
| mriedem | how? | 19:47 |
| dansmith | how? because we always ran drop_move_claim | 19:47 |
| dansmith | reset the test to where it was and run it with the oddball exception and we'll assert that the dest host claim is zero, but it no longer is after this change | 19:48 |
| dansmith | this: https://pastebin.com/fMUmmgMC | 19:48 |
| mriedem | i don't know if we're talking about the same thing, | 19:49 |
| mriedem | i wrote https://review.openstack.org/#/c/499877/ to show that we didn't need to manually remove allocations on the dest when the driver failed | 19:49 |
| mriedem | because of drop_move_claim | 19:49 |
| mriedem | the other test was to show that we needed to add https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2812 | 19:49 |
| mriedem | when the claim itself fails | 19:49 |
| mriedem | raising ComputeResourcesUnavailable | 19:49 |
| dansmith | right, but that doesn't make sense right? | 19:49 |
| dansmith | if we fail for some driver reason, we're now "on" that dest host and should be able to run a same-host rebuild on it | 19:50 |
| dansmith | which won't re-claim for us | 19:50 |
| mriedem | no, we're not on that host | 19:50 |
| mriedem | if driver.spawn fails, we're not on that host | 19:50 |
| mriedem | the instance is only on the dest host if we get here https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2836 | 19:50 |
| mriedem | which doesn't happen if the driver fails | 19:50 |
| *** tbachman has quit IRC | 19:50 | |
| dansmith | why is this here? https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2812 | 19:50 |
| mriedem | see https://review.openstack.org/#/c/499878/ | 19:51 |
| mriedem | plus the comment above it | 19:51 |
| *** awaugama has quit IRC | 19:52 | |
| mriedem | so i don't know what else is going on in this patch, i didn't get that far, i saw the commit message and change to the test and wanted to bring that up since it's approved | 19:54 |
| dansmith | mriedem: okay yeah I really thought that the rt claim would set host and node | 19:54 |
| dansmith | mriedem: you better -2 that or something so it doesn't merge | 19:55 |
| mriedem | i don't think that works | 19:55 |
| dansmith | mriedem: -2 will make it not merge I think, it just won't kick it | 19:55 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Move allocation manipulation out of drop_move_claim() https://review.openstack.org/498947 | 19:55 |
| openstackgerrit | Dan Smith proposed openstack/nova master: BUMPMove allocation manipulation out of drop_move_claim() https://review.openstack.org/498947 | 19:55 |
| mriedem | i know a new commit will do it | 19:55 |
| mriedem | heh | 19:55 |
| mikal | I have a cold, hold me | 19:55 |
| mriedem | gdi mikal | 19:56 |
| mriedem | you've stepped into the wrong room at the wrong time | 19:56 |
| mriedem | way out of line donny | 19:56 |
| *** jmlowe has quit IRC | 19:56 | |
| jaypipes | you're outta your element, mikal | 19:56 |
| mikal | jaypipes: that's always been true though | 19:56 |
| jaypipes | :P | 19:56 |
| mriedem | mikal, btw, given your plentiful rackspaceness, do you know if rax ever used this thing http://lists.openstack.org/pipermail/openstack-operators/2017-September/014267.html ? | 19:57 |
| mikal | mriedem: my name isn't donny? | 19:57 |
| mriedem | gdi mikal | 19:57 |
| mriedem | https://www.youtube.com/watch?v=AS8X2Qp_6aA | 19:57 |
| mriedem | i'm walter in this scenario | 19:57 |
| *** lucasxu has quit IRC | 19:57 | |
| mriedem | filled with rage | 19:57 |
| mikal | mriedem: so, Rackspace is definitely running code newer than kilo, so if they haven't noticed they don't need it? | 19:57 |
| mikal | mriedem: private cloud didn't use it, public might but johnthetubaguy would know more about that | 19:58 |
| mikal | mriedem: ahhh, ok, I shall study before next time | 19:58 |
| mriedem | mikal: anyone that was relying on it and has newer than kilo, and didn't notice, then yeah i guess we don't need it | 19:58 |
| mriedem | that's the assertion in the ML thread and commit to remove it anyway | 19:59 |
| *** baoli has quit IRC | 20:00 | |
| melwitt | does anything ever set 'group_members' in filter_properties? I'm not finding anything https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L205 | 20:03 |
| *** baoli has joined #openstack-nova | 20:04 | |
| sdague | mikal: I retool your ploop patch with the fixes from the virtuozo folks | 20:04 |
| melwitt | this line is making reschedule fail with "'NoneType' object is not iterable" after a late-affinity-check failure. I don't see how reschedule after late check ever could work | 20:04 |
| *** pcaruana has quit IRC | 20:05 | |
| * mtreinish laughs everytime he sees ploop | 20:06 | |
| mriedem | melwitt: it could be a case of group_members being set on the request spec initially, and then it's transformed into the primitive filter_properties stuff which doesn't include the group_members? | 20:08 |
| mriedem | i've seen some wonky stuff with how the request spec transforms to/from the legacy filter props | 20:08 |
| mikal | sdague: ta, looking at it now | 20:09 |
| mriedem | melwitt: see _to_legacy_group_info ? | 20:09 |
| mriedem | it sets group_updated=True but doesn't include group_members | 20:09 |
| melwitt | mriedem: I don't see that RequestSpec has any group_members in it. I grepped for "group_members" in nova and found nothing that ever sets it | 20:09 |
| *** dave-mcc_ has joined #openstack-nova | 20:09 | |
| melwitt | yeah, I see that. that's the only thing that looks like it could be related | 20:10 |
| mikal | sdague: looks like there is a rebase error there though? The console pty stuff is now in that patch. | 20:10 |
| mtreinish | dansmith: http://logs.openstack.org/57/507657/1/check/gate-nova-python27-ubuntu-xenial/55aebe1/console.html#_2017-09-26_19_42_08_172719 | 20:10 |
| mikal | sdague: I shall rectify | 20:10 |
| melwitt | mriedem: late affinity check reschedule just looks totally broken unless I'm blind. which may very well be the case | 20:10 |
| mriedem | melwitt: this added it https://review.openstack.org/#/c/148277/ | 20:11 |
| melwitt | thanks | 20:11 |
| *** dave-mccowan has quit IRC | 20:12 | |
| mriedem | https://review.openstack.org/#/c/148277/64/nova/scheduler/utils.py@349 | 20:12 |
| melwitt | looks like that line in scheduler/utils that sets it is gone now | 20:12 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Add alternate hosts https://review.openstack.org/486215 | 20:12 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Add Selection objects https://review.openstack.org/499239 | 20:12 |
| openstackgerrit | Ed Leafe proposed openstack/nova master: Return Selection objects from the scheduler driver https://review.openstack.org/495854 | 20:12 |
| melwitt | trying to find what removed it | 20:13 |
| mriedem | melwitt: https://review.openstack.org/#/c/469037/ | 20:13 |
| mriedem | pike ^ | 20:13 |
| *** egonzalez has quit IRC | 20:13 | |
| mriedem | https://review.openstack.org/#/c/469037/6/nova/scheduler/utils.py | 20:13 |
| melwitt | thanks | 20:13 |
| melwitt | okay, so the conductor logic is still relying on stuff being in filter_properties via RequestSpec.from_primitives | 20:14 |
| mriedem | looks like it, left some comments in https://review.openstack.org/#/c/469037/6/nova/objects/request_spec.py | 20:15 |
| mriedem | likely need a functional regression test to show the failure | 20:15 |
| mriedem | then fix on top and backport both to pike | 20:15 |
| mriedem | melwitt: do you have a bug for this? | 20:16 |
| melwitt | mriedem: no, I can open one. wanted to sanity check with yall first | 20:17 |
| mriedem | someone was saying they hit this exact same thing this morning to bauzas | 20:17 |
| *** Apoorva has joined #openstack-nova | 20:18 | |
| melwitt | hah, what a coinky dink | 20:18 |
| mriedem | please keep that sailor talk for at home | 20:19 |
| melwitt | aye aye sir! | 20:20 |
| melwitt | matey | 20:20 |
| melwitt | oh, that's pirate. nvm | 20:20 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Move allocation manipulation out of drop_move_claim() https://review.openstack.org/498947 | 20:20 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make allocation cleanup honor new by-migration rules https://review.openstack.org/498948 | 20:20 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Pre-create migration object https://review.openstack.org/498950 | 20:20 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Revert allocations by migration uuid https://review.openstack.org/498949 | 20:20 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Refactor resource tracker to account for migration allocations https://review.openstack.org/506419 | 20:20 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make migration uuid hold allocations for migrating instances https://review.openstack.org/506420 | 20:20 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make live migration hold resources with a migration allocation https://review.openstack.org/507638 | 20:20 |
| mriedem | dansmith: what's the word | 20:21 |
| melwitt | bird bird bird bababir bird's the word | 20:22 |
| *** dave-mccowan has joined #openstack-nova | 20:23 | |
| *** liverpooler has quit IRC | 20:23 | |
| *** belmoreira has joined #openstack-nova | 20:25 | |
| *** Tom_ has joined #openstack-nova | 20:25 | |
| *** dave-mcc_ has quit IRC | 20:25 | |
| *** moshele has quit IRC | 20:25 | |
| *** smatzek has quit IRC | 20:29 | |
| *** Tom_ has quit IRC | 20:29 | |
| melwitt | mriedem: https://bugs.launchpad.net/nova/+bug/1719730 | 20:30 |
| openstack | Launchpad bug 1719730 in OpenStack Compute (nova) "Reschedule after the late affinity check fails with "'NoneType' object is not iterable"" [Undecided,New] | 20:30 |
| *** jmlowe has joined #openstack-nova | 20:30 | |
| * efried thought "Grease" is the word | 20:31 | |
| *** ltomasbo has quit IRC | 20:31 | |
| *** jpena|off has quit IRC | 20:32 | |
| efried | mriedem sdague https://review.openstack.org/#/c/488137/ should be ready again | 20:32 |
| melwitt | heh | 20:32 |
| *** cleong has quit IRC | 20:33 | |
| *** ltomasbo has joined #openstack-nova | 20:35 | |
| *** baoli_ has joined #openstack-nova | 20:35 | |
| *** jpena|off has joined #openstack-nova | 20:35 | |
| *** ijw has joined #openstack-nova | 20:36 | |
| *** felipemonteiro has quit IRC | 20:36 | |
| *** baoli has quit IRC | 20:36 | |
| *** sahid has joined #openstack-nova | 20:37 | |
| *** techstepper has quit IRC | 20:39 | |
| *** penick has quit IRC | 20:39 | |
| *** crushil has quit IRC | 20:42 | |
| *** ijw has quit IRC | 20:43 | |
| *** penick has joined #openstack-nova | 20:44 | |
| *** pchavva has quit IRC | 20:46 | |
| *** yikun_jiang has quit IRC | 20:47 | |
| *** yikun_jiang has joined #openstack-nova | 20:47 | |
| *** armax has quit IRC | 20:50 | |
| *** armax has joined #openstack-nova | 20:50 | |
| *** armax has quit IRC | 20:50 | |
| *** armax has joined #openstack-nova | 20:51 | |
| *** armax has quit IRC | 20:51 | |
| *** armax has joined #openstack-nova | 20:52 | |
| *** armax has quit IRC | 20:52 | |
| *** armax has joined #openstack-nova | 20:52 | |
| *** armax has quit IRC | 20:53 | |
| *** itlinux has joined #openstack-nova | 20:53 | |
| *** ijw has joined #openstack-nova | 20:54 | |
| *** kfarr has joined #openstack-nova | 20:56 | |
| *** kfarr has quit IRC | 20:57 | |
| *** hemna__ has quit IRC | 21:00 | |
| *** mnestratov has joined #openstack-nova | 21:00 | |
| *** sahid has quit IRC | 21:03 | |
| *** smatzek has joined #openstack-nova | 21:04 | |
| *** jmlowe has quit IRC | 21:07 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Add recreate test for live migrate rollback not cleaning up dest allocs https://review.openstack.org/507677 | 21:07 |
| mriedem | dansmith: ^ thus begins another round of these | 21:07 |
| *** baoli_ has quit IRC | 21:07 | |
| *** jmlowe has joined #openstack-nova | 21:09 | |
| openstackgerrit | Eric Berglund proposed openstack/nova master: PowerVM Driver: config drive https://review.openstack.org/409404 | 21:11 |
| *** belmoreira has quit IRC | 21:12 | |
| *** armax has joined #openstack-nova | 21:12 | |
| *** MVenesio has quit IRC | 21:14 | |
| *** slaweq_ has quit IRC | 21:14 | |
| *** MVenesio has joined #openstack-nova | 21:15 | |
| *** thorst has quit IRC | 21:16 | |
| *** thorst has joined #openstack-nova | 21:16 | |
| *** jmlowe has quit IRC | 21:18 | |
| *** MVenesio has quit IRC | 21:19 | |
| *** belmoreira has joined #openstack-nova | 21:19 | |
| *** vladikr has quit IRC | 21:20 | |
| *** thorst has quit IRC | 21:21 | |
| *** belmoreira has quit IRC | 21:22 | |
| *** tidwellr has quit IRC | 21:27 | |
| *** tbachman has joined #openstack-nova | 21:28 | |
| *** pino has joined #openstack-nova | 21:30 | |
| *** eharney has quit IRC | 21:30 | |
| pino | Hi Folks, I'm just getting started on a project that would provide an alternative to using key-pairs for instances: ssh certificates. This requires injecting into the instance (before startup) a host certificate, a user CA public key, and authorized principals file(s); then modifying sshd_config to use them. What's the right way to hook into the co | 21:36 |
| pino | mpute instance lifecycle? | 21:36 |
| pino | I'm just experimenting, but I'm wondering if this would be best built as part of Nova itself, or separately hook into the lifecycle. | 21:37 |
| *** mnestratov has quit IRC | 21:38 | |
| jaypipes | pino: definitely not part of Nova itself, no. apart from writing files to a config drive, Nova doesn't mess with the VM. | 21:38 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove dest node allocations during live migration rollback https://review.openstack.org/507687 | 21:39 |
| pino | jaypipes: ok, fair enough... but shouldn't support for ssh certificates be modelled similar to keypair support? | 21:39 |
| jaypipes | pino: honestly, I'm not sure what the diff is between a key pair, with the private part of the pair downloaded to the user and the public part laid down on the VM config drive, and the SSH certificates thing you're describing. | 21:41 |
| pino | And in terms of doing it outside of Nova, do you agree Nova notifications are not the right mechanism? The injection of various files, plus modification of the sshd_config must be done before first boot. Any advice about where, and how to do the hook? | 21:41 |
| jaypipes | pino: I'm not an expert in ssh stuff, apologies. | 21:41 |
| pino | jaypipes: I probably gave too much detail. I'm just looking for some hints about how I can hook into the startup workflow and block it until I've configured the VMs SSH the way I want it. | 21:43 |
| *** r-daneel has quit IRC | 21:43 | |
| *** armax has quit IRC | 21:43 | |
| jaypipes | pino: I think cloud-init is more what you are looking for? | 21:44 |
| *** claudiub has joined #openstack-nova | 21:44 | |
| mriedem | pino: https://docs.openstack.org/nova/latest/user/vendordata.html | 21:45 |
| mriedem | setup an external rest service that provides metadata to the guest when it's created | 21:45 |
| mriedem | example https://github.com/openstack/novajoin | 21:46 |
| *** edmondsw has quit IRC | 21:46 | |
| penick | pino: I use SSH CA in my environment, maybe I can help? | 21:47 |
| pino | mriedem: I saw that but wasn't sure it was the right approach. I'll take a closer look, thanks for the example. | 21:47 |
| penick | I think I see what you're trying to do, and I think what you're probably going to want is to build a small webservice to create and sign SSH certificates, then tie that in with the nova vendordata stuff to get injected into the instance on boot | 21:48 |
| mriedem | it's a wild penick | 21:48 |
| mriedem | i wonder what the keyword is here | 21:48 |
| penick | I identify as feral | 21:48 |
| pino | jaypipes: I'm looking at cloud-init too... but I want my setup script to run without the user's help (they shouldn't have to do any setup). | 21:48 |
| *** ijw has quit IRC | 21:49 | |
| pino | penick: that makes perfect sense. | 21:49 |
| *** edmondsw_ has joined #openstack-nova | 21:49 | |
| pino | Ok, so I have a few topics/approaches to study. Thanks! | 21:50 |
| *** ijw has joined #openstack-nova | 21:50 | |
| penick | np :) | 21:51 |
| *** edmondsw_ has quit IRC | 21:54 | |
| *** yamahata has joined #openstack-nova | 21:54 | |
| *** cdent has quit IRC | 21:55 | |
| *** yassine has joined #openstack-nova | 21:58 | |
| *** esberglu has quit IRC | 21:58 | |
| *** penick has quit IRC | 22:01 | |
| *** pino has quit IRC | 22:03 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Use ksa adapter for keystone conf & requests https://review.openstack.org/507693 | 22:10 |
| *** esberglu has joined #openstack-nova | 22:12 | |
| *** sdague has quit IRC | 22:13 | |
| *** burt has quit IRC | 22:15 | |
| *** jaypipes has quit IRC | 22:16 | |
| *** esberglu has quit IRC | 22:16 | |
| *** yassine has quit IRC | 22:17 | |
| *** acormier has quit IRC | 22:20 | |
| *** jmlowe has joined #openstack-nova | 22:24 | |
| *** Sukhdev has joined #openstack-nova | 22:24 | |
| rybridges | Hey guys, I have a question. I am trying to inject some default user data into every instance while it is provisioning at this location -> https://github.com/openstack/nova/blob/stable/ocata/nova/compute/api.py#L1011 (I am adding an internal patch for this) In my patch, I create the user data, merge it with any existing user data on the instance, and then try to write the new user data to the | 22:26 |
| rybridges | database. I am stuck getting it to write into the database. instance.save( | 22:26 |
| *** smatzek has quit IRC | 22:27 | |
| rybridges | instance.save() is throwing stack traces. so I am trying to use the update_instance() method defined in the API class defined here https://github.com/openstack/nova/blob/stable/ocata/nova/compute/api.py#L2622 | 22:27 |
| rybridges | but that does not seem to actually be saving the user data in the instance for some reason | 22:27 |
| rybridges | it calls build_req.save() in the update_instance() method | 22:27 |
| *** lyan has quit IRC | 22:29 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove dest node allocations during live migration rollback https://review.openstack.org/507687 | 22:32 |
| melwitt | rybridges: doing it that way is a bad idea IMHO. if you're looking to have default data injected into every instance, you should look into the vendordata stuff that was linked earlier | 22:32 |
| *** dave-mccowan has quit IRC | 22:36 | |
| rybridges | so its not default perse, it will actually change based on some parameters. i just figured it would be easier for people to understand my problem if i said default | 22:36 |
| *** itlinux has quit IRC | 22:37 | |
| rybridges | i dont like the vendordata stuff because it requires us to write an external webservice which complicates our deployment | 22:37 |
| rybridges | i would rather just make a small patch which hits an entry point and injects the user data into the instance | 22:37 |
| rybridges | it is much simpler and easier to debug/work with | 22:37 |
| rybridges | i feel like it should not be this difficult.. | 22:38 |
| rybridges | to just save the instance and get the user data written to the db | 22:38 |
| mriedem | you know what's going to complicate your deployment? | 22:39 |
| mriedem | constantly rebasing your fork, and when we change the internals that it depends on | 22:39 |
| *** lbragstad has quit IRC | 22:40 | |
| rybridges | we plan on adding a vendordata service eventually | 22:40 |
| rybridges | also, the rebase is basically nothing | 22:41 |
| rybridges | my patch is 3 lines | 22:41 |
| melwitt | +1. I've been there before (patching nova) and would not recommend it | 22:41 |
| rybridges | because i use an entry point that just passes locals to another function defined in a separate package | 22:41 |
| *** moshele has joined #openstack-nova | 22:41 | |
| rybridges | what i am trying to do is very simple | 22:42 |
| rybridges | i just want to get a poc working right now | 22:42 |
| mriedem | dansmith: i joked about this, but somone made it a reality http://forumtopics.openstack.org/cfp/details/6 | 22:42 |
| mriedem | new devstack setup, still can't create 500 vms at once, they all go to NoValidHost, so maybe i didn't try to create this many at once yesterday - i wonder if i'm making placement bomb out, and the scheduler is rolling everything back | 22:45 |
| rybridges | is user data immutable now? similar to how provision_updated_at is immutable in ironic | 22:47 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Move allocation manipulation out of drop_move_claim() https://review.openstack.org/498947 | 22:48 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make allocation cleanup honor new by-migration rules https://review.openstack.org/498948 | 22:48 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Pre-create migration object https://review.openstack.org/498950 | 22:48 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Revert allocations by migration uuid https://review.openstack.org/498949 | 22:48 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Refactor resource tracker to account for migration allocations https://review.openstack.org/506419 | 22:48 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make migration uuid hold allocations for migrating instances https://review.openstack.org/506420 | 22:48 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Make live migration hold resources with a migration allocation https://review.openstack.org/507638 | 22:48 |
| melwitt | mriedem: did the scheduler logs offer any clues? | 22:50 |
| mriedem | i think those have wrapped by now | 22:51 |
| mriedem | i can create in chunks of 100 just fine | 22:51 |
| *** yassine has joined #openstack-nova | 22:51 | |
| melwitt | okay. was just curious | 22:52 |
| mriedem | now i've got 100 ACTIVE instances, with 100 consumers in the api db and 300 allocations, | 22:52 |
| mriedem | which makes sense b/c 1 cpu, 1 ram, 1 disk allocation per instance | 22:52 |
| mriedem | 500 in the nova_cell0 db | 22:52 |
| mriedem | aha | 22:53 |
| mriedem | ubuntu@devstack:~$ sudo journalctl -a -u devstack@n-sch.service | grep Unable | 22:53 |
| mriedem | Sep 26 22:28:37 devstack nova-scheduler[2951]: WARNING nova.scheduler.client.report [None req-af92d5f2-4c99-4231-966e-939e1da04239 demo admin] Unable to submit allocation for instance 5f9f4f7d-8a2f-4fb8-b30a-024ed2e8e49d (409 {"errors": [{"status": 409, "request_id": "req-cda80554-6083-45b0-87bf-9e9c9924213f", "detail": "There was a conflict when trying to complete your request.\n\n Inventory changed while attempting to alloc | 22:53 |
| mriedem | Another thread concurrently updated the data. Please retry your update ", "title": "Conflict"}]}) | 22:53 |
| mriedem | Sep 26 22:28:37 devstack nova-scheduler[2951]: DEBUG nova.scheduler.filter_scheduler [None req-af92d5f2-4c99-4231-966e-939e1da04239 demo admin] Unable to successfully claim against any host. {{(pid=2951) _schedule /opt/stack/nova/nova/scheduler/filter_scheduler.py:221}} | 22:53 |
| mriedem | ubuntu@devstack:~$ | 22:53 |
| mriedem | and then that removes all allocations for all instances | 22:54 |
| mriedem | dansmith: melwitt: ^ | 22:54 |
| mriedem | so yeah that's my failure here | 22:54 |
| melwitt | um, so is that a new scheduling race condition that has to be resolved with reschedules? to replace the old claim race? | 22:55 |
| mriedem | it does retry http://paste.openstack.org/show/621996/ | 22:55 |
| mriedem | we do a retry in the scheduler | 22:55 |
| melwitt | yeah, but I thought after "claims in the scheduler" we don't have concurrent request race problems that get kicked out to be retried | 22:56 |
| mriedem | i don't know why that's logged 6 times | 22:56 |
| dansmith | I do | 22:56 |
| mriedem | heh | 22:56 |
| dansmith | because you're scheduling so many things to one compute, and it only retries a certain number of times | 22:56 |
| mriedem | do tell :) | 22:56 |
| mriedem | i'm not surprised it's hitting a conflict | 22:56 |
| mriedem | but why is that logged 3 times? | 22:57 |
| dansmith | melwitt: we still have concurrent updates that we have to retry | 22:57 |
| mriedem | 6 i mean | 22:57 |
| mriedem | https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L1007 | 22:57 |
| mriedem | because ^ we retry 3 times | 22:57 |
| mriedem | you know, hitting a conflict that we have to retry once in 500 instances with a single compute, is pretty good | 22:57 |
| mriedem | although i'm not sure which of the 500 this is that falied | 22:57 |
| melwitt | so this is like the claim race except worse in that the things should have succeeded but can't. maybe it won't happen in real life because by the time that retry limit would be hit, the compute host would already be rejecting claims | 22:57 |
| dansmith | well, actually.. are you doing one boot there or is there anything else going on? | 22:58 |
| mriedem | single boot, --min-count 500 | 22:58 |
| mriedem | only thing that could be changing inventory is the RT? | 22:58 |
| *** claudiub has quit IRC | 22:58 | |
| melwitt | oh, okay so all or none. so nvm what I said | 22:58 |
| dansmith | so we should be only making one call to scheduler I guess | 22:58 |
| mriedem | hmm, so before claims in the scheduler, if you multi-create, wouldn't we only fail some of these after claims in the compute and reschedules? | 23:00 |
| mriedem | so chances are you'd have some/most active, but others in error after reschedules? | 23:00 |
| mriedem | now it's all or none | 23:00 |
| dansmith | all or none for the claim process | 23:00 |
| dansmith | you can still fail for other reasons | 23:00 |
| *** chyka has quit IRC | 23:01 | |
| mriedem | filters i suppose yeah | 23:01 |
| dansmith | but yeah, people really hate this behavior that we tell them they can boot 500 things and later fail 20% of them because we suck | 23:01 |
| dansmith | this is much better | 23:01 |
| dansmith | so I guess my thinking about why these are changing was because of concurrent requests, but if you're just doing one boot, I'm not sure | 23:02 |
| melwitt | yeah, that's what I was trying to think in real life the retries would have a much better chance of succeeding bc not so many concentrated on one host, right | 23:02 |
| dansmith | maybe the compute is tickling the inventory in some way, such that during 500 of them it gets changed | 23:02 |
| melwitt | because the claim would detect the host full and then it would move on to another host | 23:03 |
| dansmith | melwitt: yeah, I'm pretty confident it's related to the having of one compute and 500 instances | 23:03 |
| melwitt | the thing I'm wondering is if the host list order is consistent, could this happen in real life because they'll all try the first host in the list first. but as things are claimed, that host will fill up and then no longer be considered | 23:04 |
| *** smatzek has joined #openstack-nova | 23:04 | |
| mriedem | melwitt: i'm wondering the same | 23:04 |
| *** thorst has joined #openstack-nova | 23:04 | |
| mriedem | if we're not shuffling the hosts a bit | 23:04 |
| melwitt | so won't get bombarded that badly? | 23:04 |
| dansmith | mriedem: I want to know what the instance uuid is for each of those lines | 23:05 |
| *** yamamoto has joined #openstack-nova | 23:05 | |
| dansmith | mriedem: like maybe it retried once for a few instances, then three times for one and bailed the whole process | 23:06 |
| dansmith | because any one of them gets to three retries and it should fail and stop the num_instances loop I think | 23:06 |
| mriedem | 5f9f4f7d-8a2f-4fb8-b30a-024ed2e8e49d | 23:06 |
| mriedem | oh...http://paste.openstack.org/show/621996/ | 23:07 |
| mriedem | we don't have the instance uuid in that message | 23:07 |
| dansmith | ...that's what I'm asking for yeah | 23:07 |
| mriedem | ok, adding | 23:07 |
| mriedem | that explains the 6 log messages | 23:07 |
| *** thorst has quit IRC | 23:08 | |
| dansmith | while you're re-testing, I'd like to suggest that we not hold up the instance list stuff on this if it takes too much longer | 23:08 |
| dansmith | we can revert it if there's a real performance regression pretty easily, and we're delaying soak-time for any non-perf-related bugs we might be able to resolve just from our own infra workload | 23:09 |
| mriedem | well, i wasted most of my day on this 500 novalidhost thing, | 23:09 |
| mriedem | now i know i can create in 100 chunks | 23:09 |
| dansmith | and we know we're not regressing the performance of the infra jobs that have run against this | 23:09 |
| mriedem | so i'm at the point that i've got 500 in cell0 and 500 in cell1, and going to get numbers on those | 23:09 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Log instance uuid when retrying claims in the scheduler https://review.openstack.org/507705 | 23:12 |
| openstackgerrit | Moshe Levi proposed openstack/nova master: Don't overwrite binding-profile https://review.openstack.org/505613 | 23:16 |
| *** hongbin has quit IRC | 23:22 | |
| openstackgerrit | Ed Leafe proposed openstack/nova-specs master: Return Selection Objects https://review.openstack.org/498830 | 23:24 |
| dansmith | I'm actually not sure why we'd be hitting concurrent updates during an allocation event, | 23:25 |
| dansmith | given that we're not passing rp_generation | 23:25 |
| *** sree has joined #openstack-nova | 23:26 | |
| dansmith | either the allocation fits or doesn't | 23:26 |
| melwitt | hm, yeah | 23:27 |
| dansmith | it must be that placement doesn't hide transaction commits from us | 23:27 |
| dansmith | based on the commit that added it | 23:27 |
| *** chyka has joined #openstack-nova | 23:28 | |
| dansmith | which would just be "tons of churn on one provider" as the reason | 23:28 |
| melwitt | what does that mean? placement not hiding transaction commits | 23:29 |
| *** sree has quit IRC | 23:30 | |
| dansmith | an allocation will increment the generation of the provider on the placement side | 23:31 |
| dansmith | and a transaction will abort if we try to do it at the same time as something else | 23:31 |
| melwitt | ah, right. that's what I was thinking of | 23:31 |
| dansmith | placement should really retry those for us server-side I would think, for an allocation type request | 23:31 |
| melwitt | I couldn't remember where the RP generation was related there | 23:31 |
| *** chyka has quit IRC | 23:32 | |
| dansmith | doesn't matter, but I would think it would be better, especially given the advice for that error is "try exactly the same thing again" | 23:32 |
| melwitt | yeah, I was thinking the same | 23:32 |
| mriedem | this was the error from the server | 23:32 |
| mriedem | There was a conflict when trying to complete your request.\n\n Inventory changed while attempting to allocate: Another thread concurrently updated the data. Please retry your update | 23:32 |
| dansmith | right | 23:32 |
| mriedem | i'm not sure why inventory would change | 23:32 |
| mriedem | that's static in this case | 23:32 |
| *** smatzek has quit IRC | 23:32 | |
| dansmith | any allocation will change the generation | 23:33 |
| mriedem | the update_available_resource periodic will post inventory, but only if it changes | 23:33 |
| dansmith | so any two allocations can conflict | 23:33 |
| melwitt | I'm +1 on the idea of retrying server-side | 23:33 |
| *** takashin has joined #openstack-nova | 23:34 | |
| melwitt | I'm a little worried how many conflicts can we get in real life by ppl trying to create 500 servers and the scheduler is doing a "pack" pattern | 23:34 |
| melwitt | as far as how to tune how many retries | 23:35 |
| melwitt | to allow | 23:35 |
| dansmith | this is also a highly synthetic scenario with a "virt driver" that doesn't get looked at much.. it could be doing something to re-stab inventory for no reason or something | 23:35 |
| mriedem | for each instance, we go through the filters | 23:36 |
| *** acormier has joined #openstack-nova | 23:36 | |
| mriedem | and doesn't the scheduler have some kind of tracking on the HostState objects themselves for chosen hosts? | 23:36 |
| edleafe | mriedem: that error message is poorly worded. It should be something like "available inventory has changed" | 23:36 |
| melwitt | oh, the "inventory changed" yeah, I don't know anything about that. so that means it wasn't an allocation writing conflict? | 23:36 |
| dansmith | melwitt: it's al related | 23:36 |
| dansmith | *all | 23:36 |
| dansmith | edleafe: that's not really accurate, AFACT, since allocations will cause the generation to increase | 23:37 |
| dansmith | edleafe: and thus it could be nothing changed with inventory to cause that | 23:37 |
| edleafe | dansmith: uh, that's why I added "available". | 23:37 |
| edleafe | dansmith: IOW, some inventory has been allocated | 23:37 |
| mriedem | this is basically the inventory reporting for the fake driver https://github.com/openstack/nova/blob/master/nova/virt/fake.py#L111 | 23:37 |
| edleafe | and changed the generation | 23:37 |
| dansmith | edleafe: okay, I wouldn't word it that way for clarity, but okay :) | 23:38 |
| edleafe | I wouldn't word it that way either, but I was guessing the author's intent | 23:38 |
| edleafe | *cough* cdent *cough* | 23:39 |
| dansmith | mriedem: that's the info that we use to generate it, yes | 23:39 |
| melwitt | yeah, if it can happen without inventory (total possible capacity) changing, then that error message is confusing to me | 23:39 |
| edleafe | melwitt: yeah, it's worded very poorly | 23:40 |
| dansmith | mriedem: I'd look to see if the compute is hitting placement /inventory ever after the first go, and maybe check the nothing-changed short-circuit to make sure we're never going through it | 23:40 |
| *** acormier has quit IRC | 23:40 | |
| mriedem | the inventory nothing changed? | 23:40 |
| melwitt | I dunno, I think I know just enough for it to be confusing. for an end user, it might not be confusing | 23:40 |
| dansmith | mriedem: https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L571-L572 | 23:40 |
| *** NightKhaos has quit IRC | 23:40 | |
| mriedem | melwitt: end user won't see it, they'll see NoValidHost on 500 instances | 23:40 |
| mriedem | the operator will see it | 23:41 |
| melwitt | good point | 23:41 |
| dansmith | mriedem: or just look at placement logs to see if inventory is hit any time after compute startup | 23:41 |
| *** smatzek has joined #openstack-nova | 23:41 | |
| *** smatzek_ has joined #openstack-nova | 23:43 | |
| *** Swami has quit IRC | 23:46 | |
| mriedem | gdi, how do i regex search with grep | 23:46 |
| mriedem | sudo journalctl -a -u devstack@placement-api.service | grep '.*PUT.*\/inventories.*' | 23:47 |
| *** smatzek has quit IRC | 23:47 | |
| *** NightKhaos has joined #openstack-nova | 23:47 | |
| dansmith | PUT.*invent should be all you need | 23:48 |
| *** penick has joined #openstack-nova | 23:48 | |
| mriedem | doesn't work | 23:49 |
| mriedem | ah, well, | 23:51 |
| mriedem | PUT.*alloc works | 23:51 |
| mriedem | so it probably just wrapped | 23:51 |
| mriedem | and it's not updating inventory, as it shouldn't | 23:51 |
| mriedem | got my 1000 instances now, so will do the test stuff once i'm done with dinner | 23:52 |
| dansmith | so I'd also check to make sure compute isn't doing the ocata healing during boot or something like that | 23:53 |
| *** penick has quit IRC | 23:53 | |
| *** chyka has joined #openstack-nova | 23:54 | |
| *** penick has joined #openstack-nova | 23:55 | |
| takashin | Spec cores, could you review https://review.openstack.org/#/c/489029/ ? It got one +2. | 23:57 |
| *** vladikr has joined #openstack-nova | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!