| *** haplo37_ has joined #openstack-nova | 00:00 | |
| *** rama_y has quit IRC | 00:01 | |
| *** mlavalle has quit IRC | 00:01 | |
| ryan_____ | auggy: Thanks! I am pretty sure that is what I am looking for | 00:03 |
|---|---|---|
| *** suresh12 has joined #openstack-nova | 00:04 | |
| auggy | jlk: well i poked around a little to see if anything stood out and i found this bit of code that does cache cleanup - https://github.com/openstack/nova/blob/master/nova/virt/libvirt/imagecache.py#L381 | 00:07 |
| *** ducttape_ has joined #openstack-nova | 00:07 | |
| auggy | ooh here's the base file handler bit: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/imagecache.py#L381 | 00:09 |
| auggy | anyways, assuming i'm not missing something entirely, maybe the answer to your question is buried in that code | 00:11 |
| auggy | oh yeah here's how it determines "in use" images - https://github.com/openstack/nova/blob/master/nova/virt/libvirt/imagecache.py#L381 | 00:12 |
| auggy | hey stewie925! can you provide the full command you are trying to run? | 00:12 |
| *** diogogmt has joined #openstack-nova | 00:12 | |
| *** penick has quit IRC | 00:13 | |
| *** penick has joined #openstack-nova | 00:14 | |
| jlk | auggy: thanks for digging! I'mr eading now | 00:15 |
| *** diana-test has quit IRC | 00:15 | |
| *** sdake_ has quit IRC | 00:16 | |
| jlk | Turns out the file I was hoping could be purged is being held open by qemu though | 00:16 |
| auggy | ahhh | 00:16 |
| auggy | yeah i was gonna say it looks like qemu-img is doing the actual conversion | 00:16 |
| auggy | but idk how nova/glance/whoever is interfacing with that exactly | 00:16 |
| *** ducttape_ has quit IRC | 00:17 | |
| auggy | there might be some aussies lingering around that might know.. like mikal ?? | 00:17 |
| *** jamesdenton has joined #openstack-nova | 00:17 | |
| auggy | oh yeah dansmith might be around too | 00:17 |
| *** Apoorva_ has quit IRC | 00:19 | |
| *** Apoorva has joined #openstack-nova | 00:19 | |
| auggy | oh and i almost forgot melwitt is in west coast time too, maybe she has an idea? | 00:19 |
| *** hieulq_ has joined #openstack-nova | 00:25 | |
| auggy | wow apparently i fail at copy-paste, i just pasted the same 3 links above ahaha | 00:25 |
| auggy | determining in use images - https://github.com/openstack/nova/blob/master/nova/virt/libvirt/imagecache.py#L158 | 00:26 |
| auggy | and this was the part that handles checks for the base image - https://github.com/openstack/nova/blob/master/nova/virt/libvirt/imagecache.py#L314 | 00:27 |
| *** tjones has left #openstack-nova | 00:29 | |
| *** diana-test has joined #openstack-nova | 00:29 | |
| *** mnestratov has quit IRC | 00:30 | |
| *** hieulq__ has joined #openstack-nova | 00:31 | |
| *** mtanino has quit IRC | 00:32 | |
| *** penick has quit IRC | 00:33 | |
| *** hieulq_ has quit IRC | 00:34 | |
| *** stewie925 has quit IRC | 00:35 | |
| jlk | nod | 00:39 |
| *** jvgrant_ has joined #openstack-nova | 00:39 | |
| *** claudiub has quit IRC | 00:40 | |
| *** nic has quit IRC | 00:41 | |
| *** jvgrant has quit IRC | 00:43 | |
| *** hoangcx has joined #openstack-nova | 00:48 | |
| *** browne has quit IRC | 00:52 | |
| *** david-lyle has quit IRC | 00:52 | |
| auggy | hrm.. now i have a question... | 00:53 |
| *** sdake has joined #openstack-nova | 00:53 | |
| jlk | heh I have many | 00:54 |
| auggy | i just set up nova on a clean vm to verify something and the test_rescue_config_drive fails because i don't have genisoimage installed | 00:54 |
| jlk | What it looks like is the snapshot image lands in _base, and becomes the basis for a copy-on-write setup | 00:54 |
| auggy | this is a unit test, shouldn't that call to be mocked? | 00:54 |
| jlk | but the customer's instance has apparently filled the filesystem, so all the stuff has been copied | 00:54 |
| auggy | that call to the mkiso_cmd or whatever | 00:55 |
| auggy | oh wow | 00:55 |
| jlk | so there's a 300G snapshot sitting in _base, and 300G instance disk sitting in instances/<uuid> | 00:55 |
| auggy | oh man | 00:55 |
| jlk | so that particular instance is consuming 600G of HV disk space | 00:55 |
| jlk | it's windows so I wonder if NTFS is conflicting with copy on write? | 00:57 |
| *** thorst has quit IRC | 00:58 | |
| jlk | oh maybe it's because we have "preallocate_images = space" set | 00:59 |
| *** gouthamr has quit IRC | 00:59 | |
| auggy | yeah i just hack on the code, i don't actually know how to *use* nova haha ;) | 01:00 |
| openstackgerrit | STEW TY proposed openstack/nova: Transform instance.soft_delete notifications https://review.openstack.org/382126 | 01:00 |
| *** phuongnh has joined #openstack-nova | 01:00 | |
| *** haplo37 has quit IRC | 01:02 | |
| jlk | hrm. | 01:02 |
| *** thorst has joined #openstack-nova | 01:02 | |
| *** karlamrhein has quit IRC | 01:03 | |
| *** esberglu has quit IRC | 01:03 | |
| *** Sukhdev has quit IRC | 01:04 | |
| auggy | i think i answered my own question but i'll have to ping mdbooth tomorrow when he's around | 01:04 |
| *** haplo37 has joined #openstack-nova | 01:05 | |
| *** Apoorva_ has joined #openstack-nova | 01:05 | |
| *** abalutoiu has quit IRC | 01:05 | |
| auggy | basically other places that call the mkisoimg_cmd are mocking genisoimage | 01:05 |
| auggy | i mean in the unit tests, so i just need to find out if there was a reason why it wasn't mocked for this test or if it was just overlooked | 01:06 |
| auggy | ironically i just triaged a bug where someone was complaining about us having a genisoimage dependency and suggested that we actually didn't | 01:06 |
| auggy | hahah | 01:06 |
| *** raunak has quit IRC | 01:07 | |
| jlk | lolol | 01:07 |
| jlk | the holes we dive into | 01:08 |
| *** Apoorva has quit IRC | 01:08 | |
| *** thorst has quit IRC | 01:08 | |
| *** baoli has quit IRC | 01:09 | |
| *** Apoorva_ has quit IRC | 01:10 | |
| *** agireud has quit IRC | 01:10 | |
| *** jamesdenton has quit IRC | 01:11 | |
| auggy | i know this was totally random | 01:12 |
| auggy | it just so happens i had some issues running some tests on my regular dev environment so spun up a vanilla one | 01:13 |
| * auggy really should do that more often | 01:13 | |
| auggy | this test was added in june | 01:13 |
| *** karlamrhein has joined #openstack-nova | 01:13 | |
| *** thorst has joined #openstack-nova | 01:15 | |
| *** ducttape_ has joined #openstack-nova | 01:17 | |
| *** rfolco has joined #openstack-nova | 01:18 | |
| *** jamesdenton has joined #openstack-nova | 01:19 | |
| *** thorst has quit IRC | 01:20 | |
| *** baoli has joined #openstack-nova | 01:20 | |
| *** tuanluong has joined #openstack-nova | 01:21 | |
| *** ducttape_ has quit IRC | 01:24 | |
| *** rama_y has joined #openstack-nova | 01:24 | |
| *** thorst has joined #openstack-nova | 01:26 | |
| jlk | hrm, can't manage to get Nova to not attach a process to the file in _base/ | 01:26 |
| *** baoli has quit IRC | 01:27 | |
| *** thorst has quit IRC | 01:29 | |
| auggy | redirect with treats? | 01:31 |
| * auggy is so unhelpful | 01:31 | |
| auggy | i think i'm gonna wander off for the evening, one of these days i'll actually do the work i set out to do instead of getting distracted by all these silly bugs | 01:32 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-vif: Updated from global requirements https://review.openstack.org/377527 | 01:34 |
| *** hieulq__ has quit IRC | 01:43 | |
| *** brian_price has quit IRC | 01:45 | |
| *** kaisers has joined #openstack-nova | 01:46 | |
| *** gyee has quit IRC | 01:46 | |
| *** bdeschenes has quit IRC | 01:48 | |
| *** suresh12 has quit IRC | 01:48 | |
| *** rama_y has quit IRC | 01:50 | |
| *** ryan_____ has quit IRC | 01:50 | |
| *** woodster_ has quit IRC | 01:50 | |
| *** ducttape_ has joined #openstack-nova | 01:52 | |
| *** kaisers has quit IRC | 01:55 | |
| *** tiendc has joined #openstack-nova | 01:58 | |
| *** mtanino has joined #openstack-nova | 02:01 | |
| *** jwcroppe has joined #openstack-nova | 02:04 | |
| *** rrecio has quit IRC | 02:15 | |
| *** rrecio has joined #openstack-nova | 02:15 | |
| *** browne has joined #openstack-nova | 02:17 | |
| *** unicell1 has quit IRC | 02:17 | |
| *** sdake has quit IRC | 02:18 | |
| *** amotoki has joined #openstack-nova | 02:19 | |
| *** sdake has joined #openstack-nova | 02:20 | |
| *** amotoki has quit IRC | 02:31 | |
| *** raunak has joined #openstack-nova | 02:37 | |
| *** haplo37 has quit IRC | 02:39 | |
| *** Hosam_ has quit IRC | 02:40 | |
| *** Hosam has joined #openstack-nova | 02:41 | |
| *** amotoki has joined #openstack-nova | 02:43 | |
| *** haplo37 has joined #openstack-nova | 02:44 | |
| *** yamahata has quit IRC | 02:45 | |
| *** Hosam has quit IRC | 02:46 | |
| *** thorst has joined #openstack-nova | 02:50 | |
| *** diogogmt has quit IRC | 02:54 | |
| *** thorst has quit IRC | 02:54 | |
| *** thorst has joined #openstack-nova | 02:54 | |
| *** thorst has quit IRC | 02:55 | |
| *** ducttape_ has quit IRC | 02:57 | |
| openstackgerrit | Nguyen Hung Phuong proposed openstack/nova: Remove mox from nova/tests/unit/test_configdrive2.py https://review.openstack.org/322754 | 02:59 |
| *** amotoki_ has joined #openstack-nova | 03:04 | |
| *** thorst has joined #openstack-nova | 03:05 | |
| *** raunak has quit IRC | 03:06 | |
| *** amotoki has quit IRC | 03:07 | |
| *** Sukhdev has joined #openstack-nova | 03:09 | |
| *** thorst has quit IRC | 03:11 | |
| *** Sukhdev has quit IRC | 03:13 | |
| *** agireud has joined #openstack-nova | 03:14 | |
| *** rrecio has quit IRC | 03:20 | |
| *** Shashi has quit IRC | 03:23 | |
| *** raunak has joined #openstack-nova | 03:24 | |
| *** browne has quit IRC | 03:27 | |
| *** adisky has joined #openstack-nova | 03:33 | |
| *** amotoki has joined #openstack-nova | 03:35 | |
| *** amotoki has quit IRC | 03:36 | |
| *** amotoki_ has quit IRC | 03:36 | |
| *** tongli has quit IRC | 03:38 | |
| *** kaisers has joined #openstack-nova | 03:40 | |
| *** sdake has quit IRC | 03:43 | |
| *** sdake has joined #openstack-nova | 03:43 | |
| *** kaisers has quit IRC | 03:45 | |
| *** links has joined #openstack-nova | 03:45 | |
| *** markvoelker has quit IRC | 03:48 | |
| *** vishwanathj has quit IRC | 03:50 | |
| *** dikonoor has joined #openstack-nova | 03:55 | |
| *** suresh12 has joined #openstack-nova | 03:56 | |
| *** ducttape_ has joined #openstack-nova | 03:57 | |
| *** ducttape_ has quit IRC | 04:02 | |
| *** amotoki has joined #openstack-nova | 04:06 | |
| *** code-R has joined #openstack-nova | 04:14 | |
| *** suresh12_ has joined #openstack-nova | 04:19 | |
| *** amotoki_ has joined #openstack-nova | 04:19 | |
| *** suresh12 has quit IRC | 04:20 | |
| *** suresh12 has joined #openstack-nova | 04:21 | |
| *** amotoki has quit IRC | 04:23 | |
| *** sudipto has joined #openstack-nova | 04:23 | |
| *** sudipto_ has joined #openstack-nova | 04:23 | |
| *** suresh12_ has quit IRC | 04:23 | |
| *** ManishD has joined #openstack-nova | 04:24 | |
| *** sdake has quit IRC | 04:25 | |
| *** mingyu has joined #openstack-nova | 04:32 | |
| *** mingyu_ has joined #openstack-nova | 04:34 | |
| *** mingyu has quit IRC | 04:34 | |
| *** haplo37 has quit IRC | 04:35 | |
| *** mingyu_ has quit IRC | 04:36 | |
| *** mingyu has joined #openstack-nova | 04:37 | |
| *** suresh12_ has joined #openstack-nova | 04:37 | |
| *** suresh12 has quit IRC | 04:40 | |
| *** haplo37 has joined #openstack-nova | 04:40 | |
| *** suresh12 has joined #openstack-nova | 04:40 | |
| *** unicell has joined #openstack-nova | 04:40 | |
| *** pbandark has joined #openstack-nova | 04:41 | |
| *** mingyu has quit IRC | 04:41 | |
| *** salv-orlando has joined #openstack-nova | 04:41 | |
| *** sridharg has joined #openstack-nova | 04:42 | |
| *** suresh12_ has quit IRC | 04:42 | |
| *** coolsvap has joined #openstack-nova | 04:44 | |
| *** unicell has quit IRC | 04:45 | |
| *** GB21 has joined #openstack-nova | 04:46 | |
| *** markvoelker has joined #openstack-nova | 04:49 | |
| *** unicell has joined #openstack-nova | 04:49 | |
| *** Shashi has joined #openstack-nova | 04:51 | |
| *** suresh12 has quit IRC | 04:51 | |
| *** sdake has joined #openstack-nova | 04:51 | |
| *** armax has quit IRC | 04:51 | |
| *** HenryG has quit IRC | 04:51 | |
| *** suresh12 has joined #openstack-nova | 04:52 | |
| *** HenryG has joined #openstack-nova | 04:52 | |
| *** unicell1 has joined #openstack-nova | 04:53 | |
| *** markvoelker has quit IRC | 04:54 | |
| *** unicell has quit IRC | 04:54 | |
| *** jamesdenton has quit IRC | 04:55 | |
| *** priya__ has quit IRC | 04:58 | |
| *** ducttape_ has joined #openstack-nova | 04:59 | |
| *** amotoki_ has quit IRC | 04:59 | |
| *** narayrak has joined #openstack-nova | 05:02 | |
| *** ducttape_ has quit IRC | 05:04 | |
| *** ociuhandu has joined #openstack-nova | 05:05 | |
| *** jaosorior has joined #openstack-nova | 05:07 | |
| *** prometheanfire has joined #openstack-nova | 05:08 | |
| prometheanfire | migrations are failing on postgres for m->n | 05:08 |
| *** jwcroppe has quit IRC | 05:09 | |
| prometheanfire | https://gist.github.com/prometheanfire/98689217594ffa210cab83a0ac5ff588 | 05:09 |
| prometheanfire | also, nova doesn't work on py34 still | 05:10 |
| prometheanfire | https://gist.github.com/prometheanfire/e25b5f53f3247aca94ec14b67832da1c | 05:10 |
| prometheanfire | personlly, I blame mikal for all this | 05:10 |
| prometheanfire | mikal: you might want to look at the first one though | 05:10 |
| prometheanfire | might be showstopper for newton | 05:11 |
| prometheanfire | keystone had a similiar problem last cycle | 05:11 |
| *** thorst has joined #openstack-nova | 05:11 | |
| prometheanfire | I kinda wonder if it's just because I never used aggregates | 05:12 |
| *** psachin has joined #openstack-nova | 05:13 | |
| *** psachin has joined #openstack-nova | 05:14 | |
| *** thorst has quit IRC | 05:16 | |
| *** penick has joined #openstack-nova | 05:18 | |
| *** clarkb has quit IRC | 05:18 | |
| *** agireud has quit IRC | 05:19 | |
| *** prateek has joined #openstack-nova | 05:20 | |
| *** code-R has quit IRC | 05:22 | |
| prometheanfire | mikal: https://bugs.launchpad.net/nova/+bug/1630446 for the migration fail | 05:28 |
| openstack | Launchpad bug 1630446 in OpenStack Compute (nova) "postgresql online_data_migrations fail for m->n" [Undecided,New] | 05:28 |
| *** kaisers has joined #openstack-nova | 05:29 | |
| *** slaweq_ has joined #openstack-nova | 05:29 | |
| *** akshai has joined #openstack-nova | 05:32 | |
| *** kaisers has quit IRC | 05:33 | |
| *** moshele has joined #openstack-nova | 05:33 | |
| *** sudswas__ has joined #openstack-nova | 05:34 | |
| *** jaosorior has quit IRC | 05:35 | |
| prometheanfire | mikal: and here's a post migration failure when doing openstack server list https://bugs.launchpad.net/nova/+bug/1630448 | 05:35 |
| openstack | Launchpad bug 1630448 in OpenStack Compute (nova) "postgres newton post upgrade failure DBAPIError exception wrapped from (psycopg2.ProgrammingError) column build_requests.instance_uuid does not exist" [Undecided,New] | 05:35 |
| *** jaosorior has joined #openstack-nova | 05:35 | |
| *** sudipto has quit IRC | 05:36 | |
| *** kaisers has joined #openstack-nova | 05:36 | |
| *** sudipto has joined #openstack-nova | 05:37 | |
| *** sudipto_ has quit IRC | 05:37 | |
| *** raunak has quit IRC | 05:37 | |
| *** agireud has joined #openstack-nova | 05:38 | |
| *** GB21 has quit IRC | 05:38 | |
| prometheanfire | and the py3 failhttps://bugs.launchpad.net/nova/+bug/1630449 | 05:38 |
| openstack | Launchpad bug 1630449 in OpenStack Compute (nova) "nova-api - newton - python34 - TypeError: memoryview: str object does not have the buffer interface" [Undecided,New] | 05:38 |
| prometheanfire | bah | 05:38 |
| prometheanfire | and the py3 fail https://bugs.launchpad.net/nova/+bug/1630449 | 05:38 |
| *** bks has joined #openstack-nova | 05:40 | |
| *** suresh12 has quit IRC | 05:41 | |
| *** unicell1 has quit IRC | 05:42 | |
| *** unicell has joined #openstack-nova | 05:43 | |
| *** amotoki has joined #openstack-nova | 05:43 | |
| *** sridharg has quit IRC | 05:44 | |
| *** sridharg has joined #openstack-nova | 05:45 | |
| *** markvoelker has joined #openstack-nova | 05:50 | |
| *** hemanthm|bbl is now known as hemanthm|away | 05:50 | |
| *** GB21 has joined #openstack-nova | 05:50 | |
| *** lpetrut has joined #openstack-nova | 05:51 | |
| *** ekuris has joined #openstack-nova | 05:52 | |
| *** mingyu has joined #openstack-nova | 05:52 | |
| *** markvoelker has quit IRC | 05:55 | |
| *** akshai has quit IRC | 05:56 | |
| *** amotoki has quit IRC | 05:57 | |
| *** jwcroppe has joined #openstack-nova | 05:58 | |
| *** suresh12 has joined #openstack-nova | 05:59 | |
| *** ducttape_ has joined #openstack-nova | 05:59 | |
| *** salv-orlando has quit IRC | 06:01 | |
| *** jwcroppe has quit IRC | 06:04 | |
| *** ducttape_ has quit IRC | 06:04 | |
| prometheanfire | anyway, sleeping, hope this didn't break nova :| | 06:05 |
| *** mjura has joined #openstack-nova | 06:06 | |
| *** david-lyle has joined #openstack-nova | 06:08 | |
| *** sandanar has joined #openstack-nova | 06:11 | |
| *** ratailor has joined #openstack-nova | 06:11 | |
| *** mjura has quit IRC | 06:12 | |
| *** mjura has joined #openstack-nova | 06:12 | |
| *** paul-carlton2 has joined #openstack-nova | 06:12 | |
| *** Matti_ has quit IRC | 06:16 | |
| prometheanfire | mail sent to list | 06:18 |
| *** nkrinner_afk is now known as nkrinner | 06:19 | |
| prometheanfire | dansmith: ^ think I found a bug for m->n | 06:22 |
| openstackgerrit | Béla Vancsics proposed openstack/nova: Transform instance.shelve_offload notifications https://review.openstack.org/377239 | 06:22 |
| * tonyb waits for mail to arrive | 06:23 | |
| prometheanfire | tonyb: getting something? | 06:23 |
| tonyb | prometheanfire: a message about how m -> n db migrations don't work | 06:24 |
| prometheanfire | ya | 06:25 |
| prometheanfire | light on details because I have few | 06:25 |
| *** alexchadin has joined #openstack-nova | 06:25 | |
| openstackgerrit | Ildiko Vancsa proposed openstack/nova: WIP: Allow multi-attach in compute api https://review.openstack.org/271047 | 06:25 |
| openstackgerrit | Ildiko Vancsa proposed openstack/nova: libvirt: Allow multiple volume attachments https://review.openstack.org/267587 | 06:25 |
| *** Manish has joined #openstack-nova | 06:25 | |
| *** ManishD has quit IRC | 06:28 | |
| *** andreas_s has joined #openstack-nova | 06:33 | |
| *** unicell has quit IRC | 06:33 | |
| *** unicell has joined #openstack-nova | 06:34 | |
| prometheanfire | well, nn for now then, I'll be back in 6-7 hours | 06:35 |
| *** salv-orlando has joined #openstack-nova | 06:36 | |
| *** salv-orlando has quit IRC | 06:40 | |
| kashyap | auggy: Yes, that article is pretty old, and that part of code has changed completely. I'll add a note at the top about this. | 06:41 |
| *** lpetrut has quit IRC | 06:43 | |
| *** szirtesit has joined #openstack-nova | 06:45 | |
| *** pcaruana has joined #openstack-nova | 06:46 | |
| *** liverpooler has joined #openstack-nova | 06:46 | |
| *** markvoelker has joined #openstack-nova | 06:51 | |
| *** rcernin has joined #openstack-nova | 06:53 | |
| *** code-R has joined #openstack-nova | 06:54 | |
| *** hieulq has quit IRC | 06:55 | |
| *** code-R_ has joined #openstack-nova | 06:56 | |
| *** markvoelker has quit IRC | 06:56 | |
| *** jwcroppe has joined #openstack-nova | 06:56 | |
| *** salv-orlando has joined #openstack-nova | 06:56 | |
| *** yamahata has joined #openstack-nova | 06:57 | |
| *** hieulq has joined #openstack-nova | 06:58 | |
| *** hieulq has quit IRC | 06:59 | |
| *** code-R has quit IRC | 06:59 | |
| *** ducttape_ has joined #openstack-nova | 07:00 | |
| *** jwcroppe has quit IRC | 07:02 | |
| *** david-lyle has quit IRC | 07:02 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova: config options: improve libvirt imagecache https://review.openstack.org/378497 | 07:03 |
| *** hieulq has joined #openstack-nova | 07:05 | |
| *** ducttape_ has quit IRC | 07:05 | |
| *** hieulq has quit IRC | 07:06 | |
| *** gibi_ has quit IRC | 07:06 | |
| *** tesseract- has joined #openstack-nova | 07:07 | |
| *** suresh12 has quit IRC | 07:08 | |
| *** haplo37 has quit IRC | 07:09 | |
| *** matrohon has joined #openstack-nova | 07:09 | |
| *** hieulq has joined #openstack-nova | 07:11 | |
| *** rcernin has quit IRC | 07:12 | |
| *** rcernin has joined #openstack-nova | 07:12 | |
| *** gibi has joined #openstack-nova | 07:13 | |
| *** haplo37 has joined #openstack-nova | 07:16 | |
| *** amotoki has joined #openstack-nova | 07:20 | |
| *** ihrachys has joined #openstack-nova | 07:21 | |
| *** hogepodge has quit IRC | 07:21 | |
| *** Manish has quit IRC | 07:22 | |
| *** oanson has joined #openstack-nova | 07:27 | |
| *** amotoki has quit IRC | 07:32 | |
| *** claudiub has joined #openstack-nova | 07:37 | |
| openstackgerrit | Béla Vancsics proposed openstack/nova: Increase the possible size of the flavor extra_spec value https://review.openstack.org/382179 | 07:41 |
| *** abalutoiu has joined #openstack-nova | 07:41 | |
| *** amotoki has joined #openstack-nova | 07:43 | |
| *** markus_z has joined #openstack-nova | 07:43 | |
| *** GB21 has quit IRC | 07:49 | |
| openstackgerrit | Sergey Nikitin proposed openstack/nova-specs: PCI NUMA Policies https://review.openstack.org/361140 | 07:49 |
| *** markvoelker has joined #openstack-nova | 07:51 | |
| *** tuanluong has quit IRC | 07:54 | |
| *** salv-orl_ has joined #openstack-nova | 07:54 | |
| openstackgerrit | Allen proposed openstack/nova: Remove the duplicated test function https://review.openstack.org/382185 | 07:55 |
| *** markvoelker has quit IRC | 07:56 | |
| *** amotoki has quit IRC | 07:56 | |
| *** salv-orlando has quit IRC | 07:57 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** pilgrimstack has joined #openstack-nova | 08:00 | |
| *** zzzeek has joined #openstack-nova | 08:00 | |
| *** ducttape_ has joined #openstack-nova | 08:01 | |
| *** paul-carlton2 has quit IRC | 08:02 | |
| *** ducttape_ has quit IRC | 08:06 | |
| *** roeyc has joined #openstack-nova | 08:09 | |
| *** hogepodge has joined #openstack-nova | 08:11 | |
| *** zul has quit IRC | 08:11 | |
| *** duonghq has joined #openstack-nova | 08:14 | |
| *** zenoway has joined #openstack-nova | 08:15 | |
| *** EinstCrazy has joined #openstack-nova | 08:16 | |
| *** mgoddard has joined #openstack-nova | 08:16 | |
| *** amotoki has joined #openstack-nova | 08:17 | |
| *** pbandark has quit IRC | 08:17 | |
| *** paul-carlton2 has joined #openstack-nova | 08:24 | |
| *** pbandark has joined #openstack-nova | 08:24 | |
| *** zul has joined #openstack-nova | 08:24 | |
| *** yassine has joined #openstack-nova | 08:35 | |
| *** yassine is now known as Guest86252 | 08:35 | |
| *** GB21 has joined #openstack-nova | 08:36 | |
| *** jwcroppe has joined #openstack-nova | 08:42 | |
| *** derekh has joined #openstack-nova | 08:46 | |
| *** suresh12 has joined #openstack-nova | 08:48 | |
| *** salv-orl_ has quit IRC | 08:48 | |
| *** zenoway has quit IRC | 08:50 | |
| *** jaosorior is now known as jaosorior_lunch | 08:50 | |
| *** zenoway has joined #openstack-nova | 08:51 | |
| openstackgerrit | Timofey Durakov proposed openstack/nova: Change log level to debug for migrations pairing https://review.openstack.org/382195 | 08:51 |
| *** jwcroppe has quit IRC | 08:51 | |
| *** markvoelker has joined #openstack-nova | 08:52 | |
| *** suresh12 has quit IRC | 08:53 | |
| *** zenoway has quit IRC | 08:53 | |
| *** zenoway has joined #openstack-nova | 08:54 | |
| *** mkoderer has joined #openstack-nova | 08:54 | |
| *** roeyc has quit IRC | 08:55 | |
| *** roeyc has joined #openstack-nova | 08:56 | |
| *** eglynn has joined #openstack-nova | 08:56 | |
| *** eglynn has quit IRC | 08:57 | |
| *** markvoelker has quit IRC | 08:57 | |
| *** zenoway has quit IRC | 08:58 | |
| *** CristinaPauna has quit IRC | 08:59 | |
| *** sdake has quit IRC | 08:59 | |
| *** amotoki has quit IRC | 09:00 | |
| *** lpetrut has joined #openstack-nova | 09:00 | |
| *** lpetrut has quit IRC | 09:01 | |
| *** zenoway has joined #openstack-nova | 09:02 | |
| *** lpetrut has joined #openstack-nova | 09:03 | |
| *** alexchadin has quit IRC | 09:06 | |
| *** alexchadin has joined #openstack-nova | 09:06 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova: config options: improve libvirt imagecache https://review.openstack.org/378497 | 09:08 |
| *** sridharg has quit IRC | 09:10 | |
| *** CristinaPauna has joined #openstack-nova | 09:10 | |
| *** alexchadin has quit IRC | 09:11 | |
| *** thorst has joined #openstack-nova | 09:13 | |
| *** zenoway has quit IRC | 09:16 | |
| *** josecastroleon has joined #openstack-nova | 09:16 | |
| *** zenoway has joined #openstack-nova | 09:16 | |
| *** TonyXu has joined #openstack-nova | 09:18 | |
| *** thorst has quit IRC | 09:19 | |
| *** EinstCrazy has quit IRC | 09:23 | |
| *** cdent has joined #openstack-nova | 09:25 | |
| *** sahid has joined #openstack-nova | 09:27 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova: conf: Attempt to resolve TODOs in scheduler.py https://review.openstack.org/374647 | 09:27 |
| openstackgerrit | Stephen Finucane proposed openstack/nova: conf: Group scheduler options https://review.openstack.org/349959 | 09:27 |
| openstackgerrit | Stephen Finucane proposed openstack/nova: conf: Deprecate 'scheduler_json_file_location' https://review.openstack.org/374121 | 09:27 |
| sfinucan | raj_singh: Could you look at this again? The release note issue was with the dependency instead https://review.openstack.org/#/c/374121/ | 09:28 |
| *** amotoki has joined #openstack-nova | 09:31 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova: nova-manage: Use consistent prints for errors https://review.openstack.org/289309 | 09:31 |
| *** amotoki has quit IRC | 09:31 | |
| *** amotoki has joined #openstack-nova | 09:31 | |
| *** EinstCrazy has joined #openstack-nova | 09:32 | |
| *** hferenc has joined #openstack-nova | 09:34 | |
| *** EinstCrazy has quit IRC | 09:34 | |
| *** omnipresent has joined #openstack-nova | 09:37 | |
| *** zenoway has quit IRC | 09:38 | |
| *** zenoway has joined #openstack-nova | 09:40 | |
| *** omnipresent has quit IRC | 09:43 | |
| *** jaosorior_lunch is now known as jaosorior | 09:46 | |
| *** code-R_ has quit IRC | 09:46 | |
| *** alexchadin has joined #openstack-nova | 09:47 | |
| *** ihrachys has quit IRC | 09:48 | |
| *** haplo37 has quit IRC | 09:50 | |
| *** pbandark has quit IRC | 09:52 | |
| bauzas | alaski: once you're up, trick bug I'm not opiniated https://bugs.launchpad.net/nova/+bug/1630454 | 09:53 |
| openstack | Launchpad bug 1630454 in OpenStack Compute (nova) "shelve offload does not reduce core and RAM quota" [Undecided,New] | 09:53 |
| *** markvoelker has joined #openstack-nova | 09:53 | |
| *** pbandark has joined #openstack-nova | 09:54 | |
| *** diga has joined #openstack-nova | 09:55 | |
| *** haplo37 has joined #openstack-nova | 09:56 | |
| *** code-R has joined #openstack-nova | 09:57 | |
| *** markvoelker has quit IRC | 09:58 | |
| *** gszasz has joined #openstack-nova | 09:58 | |
| *** roeyc has quit IRC | 10:00 | |
| *** code-R has quit IRC | 10:02 | |
| *** code-R has joined #openstack-nova | 10:03 | |
| *** omnipresent has joined #openstack-nova | 10:08 | |
| *** zenoway has quit IRC | 10:09 | |
| *** mvk has quit IRC | 10:09 | |
| *** hieulq has quit IRC | 10:13 | |
| *** code-R has quit IRC | 10:13 | |
| *** sdague has joined #openstack-nova | 10:15 | |
| *** code-R has joined #openstack-nova | 10:18 | |
| *** salv-orlando has joined #openstack-nova | 10:19 | |
| *** ociuhandu has quit IRC | 10:20 | |
| *** mgoddard_ has joined #openstack-nova | 10:22 | |
| *** cdent has quit IRC | 10:23 | |
| *** dikonoor has quit IRC | 10:23 | |
| *** mgoddard has quit IRC | 10:25 | |
| *** salv-orlando has quit IRC | 10:25 | |
| openstackgerrit | Ildiko Vancsa proposed openstack/nova: Remove check_attach https://review.openstack.org/335358 | 10:26 |
| *** mingyu has quit IRC | 10:27 | |
| *** code-R has quit IRC | 10:28 | |
| *** GB21 has quit IRC | 10:30 | |
| *** hoangcx has quit IRC | 10:31 | |
| *** code-R has joined #openstack-nova | 10:31 | |
| *** jwcroppe has joined #openstack-nova | 10:32 | |
| *** code-R has quit IRC | 10:33 | |
| *** alexchadin has quit IRC | 10:34 | |
| *** alexchadin has joined #openstack-nova | 10:38 | |
| *** Shashi has quit IRC | 10:38 | |
| *** duonghq has quit IRC | 10:38 | |
| *** ihrachys has joined #openstack-nova | 10:38 | |
| *** jwcroppe has quit IRC | 10:38 | |
| *** GB21 has joined #openstack-nova | 10:39 | |
| *** mingyu has joined #openstack-nova | 10:40 | |
| *** tbachman has quit IRC | 10:40 | |
| *** code-R has joined #openstack-nova | 10:40 | |
| *** abalutoiu has quit IRC | 10:41 | |
| *** mvk has joined #openstack-nova | 10:41 | |
| *** mingyu has quit IRC | 10:43 | |
| *** alexchadin has quit IRC | 10:46 | |
| *** alexchadin has joined #openstack-nova | 10:46 | |
| *** roeyc has joined #openstack-nova | 10:47 | |
| *** alexchadin has quit IRC | 10:51 | |
| *** alexchadin has joined #openstack-nova | 10:51 | |
| *** alexchadin has quit IRC | 10:52 | |
| *** code-R has quit IRC | 10:52 | |
| *** alexchadin has joined #openstack-nova | 10:52 | |
| *** rtheis has joined #openstack-nova | 10:54 | |
| *** markvoelker has joined #openstack-nova | 10:54 | |
| *** omnipresent has quit IRC | 10:55 | |
| *** zenoway has joined #openstack-nova | 10:56 | |
| *** code-R has joined #openstack-nova | 10:58 | |
| *** markvoelker has quit IRC | 10:59 | |
| *** zenoway has quit IRC | 11:00 | |
| *** code-R has quit IRC | 11:01 | |
| *** abalutoiu has joined #openstack-nova | 11:02 | |
| *** tiendc has quit IRC | 11:04 | |
| *** bdeschenes has joined #openstack-nova | 11:19 | |
| *** marksei has joined #openstack-nova | 11:21 | |
| *** salv-orlando has joined #openstack-nova | 11:21 | |
| *** bvanhav has joined #openstack-nova | 11:23 | |
| *** salv-orlando has quit IRC | 11:26 | |
| *** code-R has joined #openstack-nova | 11:28 | |
| *** alexchadin has quit IRC | 11:29 | |
| *** alexchadin has joined #openstack-nova | 11:29 | |
| *** code-R_ has joined #openstack-nova | 11:30 | |
| *** salv-orlando has joined #openstack-nova | 11:30 | |
| *** cdent has joined #openstack-nova | 11:30 | |
| *** coolsvap is now known as coolsvap_ | 11:30 | |
| *** code-R has quit IRC | 11:33 | |
| *** bdeschenes has quit IRC | 11:39 | |
| *** mgoddard_ has quit IRC | 11:44 | |
| *** bear3bonds has joined #openstack-nova | 11:46 | |
| *** bear3bonds has left #openstack-nova | 11:46 | |
| *** cdent has quit IRC | 11:48 | |
| *** cdent has joined #openstack-nova | 11:49 | |
| *** thorst has joined #openstack-nova | 11:51 | |
| *** tbachman has joined #openstack-nova | 11:53 | |
| *** markvoelker has joined #openstack-nova | 11:54 | |
| *** bdeschenes has joined #openstack-nova | 11:57 | |
| *** jlanoux has joined #openstack-nova | 11:58 | |
| *** Hosam has joined #openstack-nova | 11:59 | |
| *** markvoelker has quit IRC | 11:59 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Reserve NUMA nodes with PCI devices attached https://review.openstack.org/364468 | 12:01 |
| *** pjm6_ has joined #openstack-nova | 12:01 | |
| *** Hosam has quit IRC | 12:03 | |
| *** pjm6 has quit IRC | 12:04 | |
| *** bdeschenes has quit IRC | 12:04 | |
| *** cdelatte has joined #openstack-nova | 12:04 | |
| *** alexchadin has quit IRC | 12:04 | |
| openstackgerrit | Pavel Kholkin proposed openstack/nova-specs: Strict isolation of group of hosts for image and flavor https://review.openstack.org/381912 | 12:05 |
| *** sudipto has quit IRC | 12:06 | |
| *** sudswas__ has quit IRC | 12:06 | |
| *** ducttape_ has joined #openstack-nova | 12:08 | |
| *** ociuhandu has joined #openstack-nova | 12:09 | |
| *** ratailor has quit IRC | 12:10 | |
| *** code-R_ has quit IRC | 12:11 | |
| *** coolsvap_ is now known as coolsvap | 12:13 | |
| *** code-R has joined #openstack-nova | 12:14 | |
| *** gouthamr has joined #openstack-nova | 12:16 | |
| *** phuongnh has quit IRC | 12:17 | |
| *** sdake has joined #openstack-nova | 12:19 | |
| *** maticue has joined #openstack-nova | 12:20 | |
| *** mah has joined #openstack-nova | 12:20 | |
| mah | Hi all, How to let openstack use Nova access API v2.0 instead of v2.1 ? | 12:21 |
| *** pradk has joined #openstack-nova | 12:21 | |
| bauzas | mah: see http://developer.openstack.org/api-guide/compute/microversions.html | 12:23 |
| bauzas | mah: v2.1 is bacwards compatible with v2.0 | 12:23 |
| bauzas | backwards* | 12:23 |
| prometheanfire | rpodolyaka, bauzas thanks for pointing out my stupid, it was a late night... | 12:24 |
| bauzas | mah: you just need to modify your compute endpoint but in case your client doesn't ask for a microversion, you'll get v2.1 which is identical to v2.0 | 12:24 |
| mah | Yes but I do not know why in my application using jcloud library I got error | 12:24 |
| bauzas | prometheanfire: np | 12:24 |
| rpodolyaka | prometheanfire: no problem! :) | 12:24 |
| mah | I go error when I create server | 12:25 |
| mah | got* | 12:25 |
| mah | using POST | 12:25 |
| bauzas | mah: which Nova version ? newton ? | 12:25 |
| mah | but using GET is fine | 12:25 |
| mah | Mitaka | 12:25 |
| mah | TripleO | 12:25 |
| bauzas | mah: that's possibly because the only change between v2.1 and v2.0 is that the input is verified | 12:26 |
| bauzas | mah: see http://developer.openstack.org/api-guide/compute/microversions.html#client-interaction , do you ask for a microversion ? | 12:26 |
| mah | no | 12:27 |
| bauzas | mah: if not, which resource call are you asking for POST? | 12:27 |
| bauzas | POST /servers ? | 12:27 |
| mah | the one used for creating servers | 12:27 |
| mah | yes | 12:27 |
| bauzas | mah: please provide me the n-api ERROR log using paste.openstack.org | 12:27 |
| *** haplo37 has quit IRC | 12:28 | |
| *** ducttape_ has quit IRC | 12:29 | |
| *** cdent has quit IRC | 12:29 | |
| *** krtaylor has quit IRC | 12:30 | |
| *** haplo37 has joined #openstack-nova | 12:30 | |
| *** mgoddard has joined #openstack-nova | 12:30 | |
| mah | http://paste.openstack.org/show/584443/ | 12:31 |
| openstackgerrit | Pavel Kholkin proposed openstack/nova: [WIP] scheduler: strictly isolate group of hosts for image/flavor https://review.openstack.org/370458 | 12:34 |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Add extra-specs-image-meta-conflicts spec https://review.openstack.org/382382 | 12:34 |
| bauzas | mah: I'd prefer the n-api corresponding log | 12:35 |
| mah | ok | 12:35 |
| bauzas | mah: here, that doesn't show that Nova has a problem, just that your app has one :) | 12:35 |
| bauzas | mah: mitaka, you said ? | 12:36 |
| johnthetubaguy | sounds like jclouds isn't sending a valid request | 12:36 |
| * bauzas looking at api-paste.ini | 12:36 | |
| bauzas | johnthetubaguy: that's my understanding | 12:36 |
| bauzas | probably due to v2.1 enforcing the user input validation more than v2.0 was used to | 12:36 |
| kashyap | mdbooth: When you're about, I'm making some changes to this, as best as I can - https://review.openstack.org/#/c/290510/13 | 12:37 |
| bauzas | johnthetubaguy: in mitaka, we keeped the v2.0 legacy path but we changed api-paste to point /v2.0 to the v2.1 codepath, right? | 12:37 |
| mdbooth | kashyap: looking | 12:37 |
| kashyap | mdbooth: If you have time, perhaps I could use your help. | 12:37 |
| kashyap | mdbooth: Haven't uploaded yet, fixing PEP8 stuff | 12:37 |
| bauzas | johnthetubaguy: that's only in this cycle that we dropped the old v2.0 path, right? | 12:38 |
| mdbooth | kashyap: Ok, ping me when you've pushed it. | 12:38 |
| kashyap | I haven't addressed every comment yet, mostly the changes in driver.py & fs.py, so far. | 12:38 |
| kashyap | mdbooth: Sure, thanks for the quick response! | 12:38 |
| *** cleong has joined #openstack-nova | 12:38 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova-specs: Virtual instance rescue with stable disk devices https://review.openstack.org/382388 | 12:39 |
| mdbooth | lyarwood: Are you real? | 12:41 |
| mdbooth | lyarwood: If so, wonder if you're interested in this I posted while you were away: http://lists.openstack.org/pipermail/openstack-dev/2016-September/104617.html | 12:42 |
| mah | I could not find error in the nova-api.log | 12:42 |
| lyarwood | mdbooth: yeah and yeah I was going to ask about that ;) | 12:43 |
| mdbooth | lyarwood: In the context of stable rescue, I think this would turn into: add a new disk and configure the instance to boot from it. | 12:43 |
| mdbooth | With some complication for config disks | 12:43 |
| openstackgerrit | Feodor Tersin proposed openstack/nova: Fix cold migration with qcow2 ephemeral disks https://review.openstack.org/346263 | 12:43 |
| bauzas | mah: could you please make sure you're sending the exact request body as said in http://developer.openstack.org/api-ref-compute-v2.1.html#createServer ? | 12:44 |
| *** markvoelker has joined #openstack-nova | 12:44 | |
| lyarwood | mdbooth: yeah, we still need to allow users to select either a stable or unstable rescue right? | 12:44 |
| *** zul has quit IRC | 12:44 | |
| mdbooth | lyarwood: Do we really? | 12:44 |
| *** abalutoiu has quit IRC | 12:44 | |
| *** baoli has joined #openstack-nova | 12:44 | |
| mdbooth | lyarwood: Ask Jony Ive what he thinks about user choice. | 12:45 |
| mdbooth | Sir Jony Ive. | 12:45 |
| mah | I found it | 12:45 |
| bauzas | mah: first of all, which URL endpoint are you hitting ? /v2.1 or /v2.0 ? | 12:45 |
| lyarwood | mdbooth: haha, yeah point | 12:45 |
| mah | v2.1 | 12:45 |
| lyarwood | mdbooth: I just assumed we would want to keep unstable rescues around | 12:45 |
| bauzas | mah: could you please lemme know the full URL (of course, not the FQDN, just the path) | 12:46 |
| mah | http://paste.openstack.org/show/584445/ here is the error at n-api | 12:46 |
| lyarwood | mdbooth: there's no spec for your libvirt suggestions at the moment right? | 12:46 |
| mdbooth | lyarwood: Nope | 12:46 |
| mdbooth | lyarwood: However, I believe it would replace persistent instance metadata, and solve more problems. | 12:47 |
| johnthetubaguy | bauzas: we didn't drop it this cycle, /2.0 still has no hard validation | 12:47 |
| mdbooth | (But we still need to refactor the storage code) | 12:47 |
| bauzas | mah: I'm sorry but http://paste.openstack.org/show/584445/ really looks like jclouds just kicked the connection while just uploading at the same time | 12:47 |
| johnthetubaguy | bauzas: now its possible the deployer has not exposed that end point in the service catalog, which is the bit that just dawned on me | 12:47 |
| sdague | johnthetubaguy: is there an API meeting today? alex_xu is out for the week, right? | 12:48 |
| bauzas | johnthetubaguy: okay, I was unsure, we relaxed the validation then in mitaka | 12:48 |
| mdbooth | lyarwood: If you fancied taking it on, we've also had reports of customer surprise due to inconsistent device ordering. | 12:48 |
| bauzas | johnthetubaguy: I remember your changes | 12:48 |
| * mdbooth doesn't like surprising customers. | 12:48 | |
| bauzas | johnthetubaguy: here, the user is using jclouds pointing to the /v2.1 path | 12:48 |
| lyarwood | mdbooth: yeah the MS issues re disks going offline? | 12:48 |
| mdbooth | lyarwood: Amongst others. | 12:48 |
| bauzas | without microversions I suppose | 12:48 |
| johnthetubaguy | sdague: unsure, I can't remember what we said last week | 12:48 |
| mah | but I tested the same application using jcloud for openstack mitaka with nova access endpoint with v2 , it works fine | 12:49 |
| *** zul has joined #openstack-nova | 12:49 | |
| mah | so may be there is a difference in the Json body that should be used | 12:49 |
| mah | or difference in the response between v2 and v2.1 ? | 12:49 |
| bauzas | mah: that's two codepaths sharing the same routing | 12:49 |
| johnthetubaguy | bauzas: oh, I see you are kinda asking the reverse, not sure when we added the relax path for v2.0, but I think it was before that was the default paste | 12:50 |
| mah | So then why it works fine when it uses /v2 and not working with /v2.1 ? | 12:50 |
| bauzas | mah: that's stange if you say that /v2.0 doesn't tell you the same, because it looks like the http negociation failed before we went to the WSGI app | 12:50 |
| mah | and that happens with Post only | 12:50 |
| mah | but it works fine with /v2.1 when it uses GET request | 12:51 |
| johnthetubaguy | mah: all valid 2.0 requests should work just fine with 2.1, and get the same response in both cases, the difference is when you send an invalid request | 12:51 |
| sdague | mah: can you get a raw wire dump there? | 12:51 |
| sdague | because what this really looks like is that content-length is getting sent by jclouds, and then it's not sending the right amount, and webob is freaking out | 12:51 |
| sdague | this is pretty low level explosion in webob deserialization | 12:52 |
| bauzas | yeah that's my understanding | 12:52 |
| *** jaypipes has joined #openstack-nova | 12:52 | |
| mah | so how to solve that issue ? | 12:52 |
| *** kylek3h has joined #openstack-nova | 12:52 | |
| bauzas | nothing related to the nova wsgi app I guess | 12:52 |
| sdague | mah: well, I think we need to figure out what the actual raw HTTP request looks like | 12:52 |
| sdague | mah: or if you have a reproduce scenario that we could run on a devstack to expose it | 12:53 |
| bauzas | mah: are you sure you tested the same cloud, both hitting /v2.1 and /v2.0 with different results ? | 12:53 |
| sdague | like a small jclouds based program | 12:53 |
| johnthetubaguy | oh, its a 500 error | 12:53 |
| bauzas | johnthetubaguy: that's a very low-level 500 :) | 12:53 |
| johnthetubaguy | ack | 12:53 |
| bauzas | johnthetubaguy: http://paste.openstack.org/show/584445/ is the trace | 12:54 |
| mah | yes i tried with packstack mitaka ,, and its endpoint for Nova is /v2 | 12:54 |
| bauzas | I need to drop off for 30 mins or so | 12:54 |
| mah | now I am doing that for OPNFV APEx which is based on TripleO Mitaka .. the nova enpoint is /v2.1 | 12:54 |
| bauzas | mah: ah, so different clouds? | 12:54 |
| mah | it is 500 error | 12:54 |
| mah | yes | 12:55 |
| *** tblakes has joined #openstack-nova | 12:55 | |
| bauzas | mah: I'd suggest you to directly hit the /v2.0 endpoint on the TripleO based cloud | 12:55 |
| bauzas | mah: and see if that's reproducable | 12:55 |
| bauzas | mah: because I'd bet you'd see that too with /v2.0 | 12:55 |
| sdague | mah: or, if you can build a reproduce for devstack, we can run with debug here. The biggest issue is getting some kind of reproduce for us. | 12:56 |
| bauzas | ++ | 12:56 |
| * bauzas bbiab | 12:56 | |
| mah | so how to activate /v2.0 instead of /v2.1 without messing up anything ? | 12:56 |
| *** GB21 has quit IRC | 12:56 | |
| *** prometheanfire has left #openstack-nova | 12:57 | |
| *** josecastroleon has quit IRC | 12:57 | |
| *** bvanhav has quit IRC | 12:58 | |
| sdague | mah: I don't the exact steps in a trippleo environment. The crux of it is to change the compute endpoint definition to point to 2.0 | 12:58 |
| *** abalutoiu has joined #openstack-nova | 12:59 | |
| *** diga has quit IRC | 12:59 | |
| *** edmondsw has joined #openstack-nova | 12:59 | |
| *** pradk has quit IRC | 12:59 | |
| *** jheroux has joined #openstack-nova | 12:59 | |
| mah | ok | 12:59 |
| sdague | https://github.com/openstack-dev/devstack/blob/563d3777823fa4ee113ac930c4066c3f8d5f8373/lib/nova#L411-L425 is where we do that kind of thing in devstack, which you could use as a roadmap | 12:59 |
| *** bks has quit IRC | 13:00 | |
| mah | there is no command like keystone-endopoint create / delete ? | 13:00 |
| *** ducttape_ has joined #openstack-nova | 13:00 | |
| sdague | mah: https://github.com/openstack-dev/devstack/blob/563d3777823fa4ee113ac930c4066c3f8d5f8373/functions-common#L1001-L1002 | 13:02 |
| *** jichen has joined #openstack-nova | 13:03 | |
| sdague | openstack endpoint create | 13:03 |
| sdague | is what you are looking for | 13:03 |
| sdague | but I don't think there is an update, so you are going to have to delete the existing compute one first | 13:03 |
| *** prateek has quit IRC | 13:03 | |
| mah | yes | 13:03 |
| mah | will the deletion of the existing one mess up things ? | 13:04 |
| sdague | mah: it will globally effect everything, so make sure you are doing this in a test env | 13:04 |
| mah | mmm | 13:04 |
| mah | so I guess I can avoid that now | 13:05 |
| sdague | once there is a compute endpoint again, it should be fine. But then again, you also shouldn't be getting this error | 13:05 |
| *** baoli_ has joined #openstack-nova | 13:05 | |
| mah | what do you see the problem came from ? | 13:05 |
| *** ekuris has quit IRC | 13:05 | |
| openstackgerrit | Hussain Chachuliya proposed openstack/nova: Replace uuid4() with uuidsentinel https://review.openstack.org/377660 | 13:05 |
| openstackgerrit | Hussain Chachuliya proposed openstack/nova: Replace uuid4() with uuidsentinel https://review.openstack.org/377661 | 13:05 |
| openstackgerrit | Hussain Chachuliya proposed openstack/nova: Replace uuid4() with uuidsentinel https://review.openstack.org/377663 | 13:05 |
| openstackgerrit | Hussain Chachuliya proposed openstack/nova: Replace uuid4() with uuidsentinel https://review.openstack.org/377667 | 13:05 |
| openstackgerrit | Hussain Chachuliya proposed openstack/nova: hacking: Use uuidutils or uuidsentinel to generate UUID https://review.openstack.org/382401 | 13:05 |
| sdague | mah: the webob error makes this seem like it is a deep python library issue | 13:05 |
| mah | in jcloud or in openstack ? | 13:06 |
| sdague | mah: either you'll need to debug on your side, or create a reproduce scenario we can run in devstack to understand this one more | 13:06 |
| *** sudswas__ has joined #openstack-nova | 13:06 | |
| *** sudipto has joined #openstack-nova | 13:06 | |
| sdague | mah: I really don't know at this point, there isn't enough details | 13:06 |
| sdague | and I need to pop off for a bit here, back post breakfast | 13:07 |
| mah | the reporoduce scenario is to create an instance in openstack | 13:07 |
| sdague | mah: it's not | 13:07 |
| sdague | because we create 100+ instances of openstack during every gate run | 13:07 |
| sdague | and never get this issue | 13:07 |
| mah | but did you tested it on TripleO ? | 13:08 |
| *** baoli has quit IRC | 13:08 | |
| mah | Ok ttyl | 13:08 |
| *** mdrabe has joined #openstack-nova | 13:09 | |
| *** hshiina has joined #openstack-nova | 13:09 | |
| *** roeyc has quit IRC | 13:10 | |
| *** ducttape_ has quit IRC | 13:11 | |
| johnthetubaguy | mah: did you try create an instance with jclouds for a vanilla devstack setup of stable/mitaka? that would be a good set of repo steps | 13:12 |
| mah | I tried it with Packstack stable/mitaka | 13:12 |
| *** cdelatte has quit IRC | 13:12 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova: Add swap volume notifications (start, end) https://review.openstack.org/250283 | 13:13 |
| *** xyang1 has joined #openstack-nova | 13:14 | |
| *** bvanhav has joined #openstack-nova | 13:14 | |
| *** EinstCrazy has joined #openstack-nova | 13:15 | |
| *** hemanthm|away is now known as hemanthm | 13:15 | |
| *** EinstCrazy has quit IRC | 13:15 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova-specs: Add swap volume notifications https://review.openstack.org/340715 | 13:16 |
| *** sudipto has quit IRC | 13:16 | |
| *** sudswas__ has quit IRC | 13:16 | |
| openstackgerrit | Julian proposed openstack/nova: Transform instance.power_off notifications https://review.openstack.org/330761 | 13:16 |
| *** kevin_ has quit IRC | 13:17 | |
| openstackgerrit | Takashi NATSUME proposed openstack/nova: Add swap volume notifications (error) https://review.openstack.org/328055 | 13:18 |
| *** cdelatte has joined #openstack-nova | 13:18 | |
| *** roeyc has joined #openstack-nova | 13:19 | |
| *** cdelatte has quit IRC | 13:19 | |
| alaski | bauzas: commented on the shelve/quota bug | 13:24 |
| *** Matti_ has joined #openstack-nova | 13:26 | |
| *** tongli has joined #openstack-nova | 13:28 | |
| *** kevin_ has joined #openstack-nova | 13:29 | |
| *** zhenguo has joined #openstack-nova | 13:32 | |
| *** akshai has joined #openstack-nova | 13:34 | |
| *** cmartin has joined #openstack-nova | 13:34 | |
| openstackgerrit | John Garbutt proposed openstack/nova-specs: WIP: API: rework security group APIs https://review.openstack.org/382414 | 13:34 |
| *** jaosorior has quit IRC | 13:36 | |
| *** jaosorior has joined #openstack-nova | 13:36 | |
| *** mriedem has joined #openstack-nova | 13:37 | |
| *** tbachman has quit IRC | 13:39 | |
| *** esberglu has joined #openstack-nova | 13:39 | |
| *** bnemec has joined #openstack-nova | 13:40 | |
| *** marksei has quit IRC | 13:41 | |
| *** hieulq__ has joined #openstack-nova | 13:43 | |
| *** mgoddard_ has joined #openstack-nova | 13:45 | |
| *** abalutoiu has quit IRC | 13:45 | |
| *** moshele has quit IRC | 13:46 | |
| paul-carlton2 | hi, I'm seeing a bug in evacuate, when an instance with sriov port is evacuated it is allocated a new device id on the target and updates neutron accordingly but the network info used by nova is not updated, see https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2800-L2810 | 13:47 |
| *** ayoung has quit IRC | 13:47 | |
| paul-carlton2 | Anyone any ideas what the right method to call is to refresh the network_info? | 13:48 |
| *** mgoddard has quit IRC | 13:48 | |
| *** TonyXu has quit IRC | 13:49 | |
| bauzas | mah: AFAIK, TripleO creates a legacy compute endpoint, just look at the service catalog | 13:49 |
| raj_singh | sfinucan: Done | 13:50 |
| *** dave-mccowan has joined #openstack-nova | 13:50 | |
| openstackgerrit | Julian proposed openstack/nova: Transform instance.power_off notifications https://review.openstack.org/330761 | 13:50 |
| sfinucan | raj_singh: Thanks :) | 13:50 |
| *** mgoddard has joined #openstack-nova | 13:52 | |
| bauzas | alaski: thanks for your comment, I think I provided the same comment on Twitter | 13:53 |
| *** mgoddard_ has quit IRC | 13:53 | |
| *** mlavalle has joined #openstack-nova | 13:53 | |
| mah | do you mean the ones in keystone ? | 13:53 |
| bauzas | alaski: I think it's rather a design discussion or a behaviour discussion, not really a bug | 13:53 |
| alaski | bauzas: yep | 13:53 |
| bauzas | mah: yup, the keystone service catalog entries about all the service types | 13:54 |
| mah | yes I found it | 13:54 |
| *** salv-orl_ has joined #openstack-nova | 13:54 | |
| bauzas | mah: computev2 maybe ? | 13:54 |
| mah | but when I create a new endpoint I do not found it in the list | 13:54 |
| mah | Compute Service V2.1 | 13:55 |
| bauzas | alaski: okay, I'll put the bug as a Wishlist category | 13:55 |
| *** eliqiao has quit IRC | 13:55 | |
| *** yamahata has quit IRC | 13:55 | |
| *** alex_xu has quit IRC | 13:55 | |
| *** yamahata has joined #openstack-nova | 13:55 | |
| mah | can you check here please http://paste.openstack.org/show/584459/ ? | 13:56 |
| *** roeyc has quit IRC | 13:56 | |
| *** salv-orlando has quit IRC | 13:57 | |
| mriedem | mdbooth: still seeing swap volume fail even with your libvirt job completion fix http://logs.openstack.org/73/374373/2/check/gate-tempest-dsvm-full-ubuntu-xenial/149fe3e/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-10-04_15_54_52_078 | 13:57 |
| mdbooth | mriedem: Looking | 13:57 |
| raj_singh | sdague: Keystone spec to allow expired user token is up now https://review.openstack.org/#/c/381361/ | 13:58 |
| sahid | btw mriedem, you might be interested by ACK that one https://review.openstack.org/#/c/378560/ | 13:58 |
| mdbooth | mriedem: Which patch is that against? | 13:58 |
| * mdbooth doesn't know how to wrangle the url | 13:58 | |
| mriedem | mdbooth: https://review.openstack.org/#/c/378746/ | 13:58 |
| mriedem | oh | 13:58 |
| mdbooth | Thanks | 13:58 |
| mriedem | mdbooth: this https://review.openstack.org/#/c/374373/ | 13:59 |
| mriedem | that's the devstack change that enables the swap volume test | 13:59 |
| openstackgerrit | Julian proposed openstack/nova: Transform instance.power_off notifications https://review.openstack.org/330761 | 14:00 |
| openstackgerrit | Julian proposed openstack/nova: Transform instance.snapshot notifications https://review.openstack.org/330128 | 14:01 |
| mdbooth | mriedem: I wonder if that's the bug referred to in alex_xu's NOTE | 14:01 |
| mdbooth | kashyap: ^^^ | 14:01 |
| kashyap | mdbooth: Which comment are you referencing? | 14:02 |
| kashyap | Just catching up scroll | 14:02 |
| mdbooth | kashyap: http://logs.openstack.org/73/374373/2/check/gate-tempest-dsvm-full-ubuntu-xenial/149fe3e/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-10-04_15_54_52_078 | 14:02 |
| *** alex_xu has joined #openstack-nova | 14:02 | |
| mriedem | mdbooth: alex's NOTE? | 14:02 |
| * kashyap clicks | 14:03 | |
| *** eharney has joined #openstack-nova | 14:03 | |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova: libvirt: fix detach volume race condition https://review.openstack.org/290510 | 14:03 |
| kashyap | mriedem: Where is the note? | 14:03 |
| mdbooth | alex_xu mriedem: So, IIUC (not a certainty!), that bug is saying that we can't do the resize immediately after the abort_job(pivot), because the abort is async, but shouldn't be | 14:03 |
| mdbooth | kashyap: driver.py, line 1193 | 14:03 |
| kashyap | Hmm, the dreaded "block copy still active" error | 14:03 |
| mdbooth | However, it seems that we've got a call path via write_instance_config() which is failing because there's an active block job | 14:04 |
| bauzas | mah: are you using a templated service catalog or a DB-backed service catalog ? I'd consider TripleO using the latter (hopefully) | 14:04 |
| kashyap | mriedem: mdbooth Yes | 14:04 |
| mriedem | oh i see https://github.com/openstack/nova/blob/0f4bd241665c287e49f2d30ca79be96298217b7e/nova/virt/libvirt/driver.py#L1190 | 14:04 |
| mdbooth | If resize_to is Falsey here, we won't have done the second wait (required because of the libvirt bug) | 14:04 |
| kashyap | But the referenced bug is fixed, IIRC | 14:04 |
| *** TonyXu has joined #openstack-nova | 14:04 | |
| mdbooth | But we're still poking something which expects it to have finished | 14:04 |
| mah | I think DB-backed service catalog, because what is inside this template not used it is temporary | 14:05 |
| kashyap | (Fixed in 1.2.7 & above, though.) | 14:05 |
| mdbooth | I wonder if we should pull that wait out of the Falsey test | 14:05 |
| openstackgerrit | Vasyl Saienko proposed openstack/nova: WIP: Add portgroups support to ironic virt driver https://review.openstack.org/382438 | 14:05 |
| mdbooth | mriedem: Right, also what kashyap says, this was fixed in upstream libvirt a while back | 14:05 |
| *** pradk has joined #openstack-nova | 14:05 | |
| mriedem | so that finally had me wondering the other day | 14:05 |
| mdbooth | mriedem: That's my initial working theory, anyway | 14:05 |
| mriedem | if we failed on sometihng else and hit the finally, which fails, | 14:05 |
| mriedem | then the failure from the finally block is going to overwrite the original error right? | 14:06 |
| *** tbachman has joined #openstack-nova | 14:06 | |
| mdbooth | mriedem: You've exceeded my knowledge of python edge cases :) | 14:06 |
| mriedem | btw, | 14:06 |
| mriedem | https://github.com/openstack/nova/blob/0f4bd241665c287e49f2d30ca79be96298217b7e/nova/compute/manager.py#L4930 | 14:07 |
| mriedem | in this scenario resize_to will be false | 14:07 |
| mriedem | because tempest is creating 2 basically identical volumes | 14:07 |
| mdbooth | mriedem: Ah... you're saying that something in the try block failed, but we can't see it because we failed again | 14:07 |
| mriedem | mdbooth: yeah | 14:07 |
| mdbooth | Hmm, I wonder if we'd see any evidence of that anywhere | 14:07 |
| mriedem | so i might wrap that in a try/except of it's own | 14:07 |
| mriedem | or excutils.save_and_reraise_exception i mean | 14:08 |
| mdbooth | mriedem: Gimme an hour to dig in these logs, but yeah, save_and_reraise sounds sensible | 14:08 |
| *** pbandark has quit IRC | 14:08 | |
| mriedem | ok i'll push a change quick | 14:08 |
| kashyap | mriedem: Were you referring to this 'finally' block? | 14:08 |
| kashyap | finally: | 14:08 |
| kashyap | self._host.write_instance_config(xml) | 14:08 |
| mriedem | and make the devstack patch dependent on it | 14:08 |
| mriedem | kashyap: yes | 14:08 |
| mriedem | that's what's failing in the test | 14:08 |
| mriedem | but i'm wondering if something else is failing before the finally | 14:08 |
| mdbooth | mriedem: Is it a deterministic failure, btw? | 14:09 |
| sdague | mriedem: https://review.openstack.org/#/c/374373/ still seems to be failing. Real or artifact? | 14:09 |
| mriedem | mdbooth: nope | 14:09 |
| mdbooth | eurgh | 14:09 |
| mriedem | sdague: that's what we're talking about | 14:09 |
| sdague | mriedem: gotcha | 14:09 |
| mdbooth | mriedem: However, also what I said about resize_to | 14:09 |
| kashyap | mriedem: mdbooth: Slight tangent - That "finally" is actually a (necessary) hack. Why? | 14:09 |
| mriedem | mdbooth: sure, but we don't actually get into that resize_to block | 14:09 |
| mdbooth | if resize_to is falsey we'll carry on without waiting for the abort to complete | 14:09 |
| kashyap | Up above, on lines 1178 & 1179, we're undefining the guest config | 14:09 |
| mriedem | mdbooth: i could add a debug log statement in the resize_to block to show if we're doing it | 14:09 |
| kashyap | if guest.has_persistent_configuration(): | 14:10 |
| kashyap | guest.delete_configuration() | 14:10 |
| * mdbooth was going to look for the libvirt logs | 14:10 | |
| kashyap | That is also a current "trick" we're using because, copy jobs are not yet re-startable. | 14:10 |
| mriedem | resize_to in this case is 0 i think | 14:10 |
| mdbooth | mriedem: In that case, I suspect that's just a bug | 14:10 |
| mdbooth | Well, a failure to workaround a known bug :) | 14:11 |
| mriedem | this is the resize_to logic in the compute manager https://github.com/openstack/nova/blob/0f4bd241665c287e49f2d30ca79be96298217b7e/nova/compute/manager.py#L4927-L4931 | 14:11 |
| mriedem | note that we've never tested this code before :) | 14:11 |
| mriedem | so i'm sure it's buggy | 14:11 |
| *** dims has quit IRC | 14:11 | |
| mdbooth | mriedem: Are we resizing during the swap operation? | 14:13 |
| mriedem | mdbooth: no | 14:13 |
| mriedem | the volumes are the same size | 14:13 |
| mriedem | at least in the tempest test | 14:13 |
| mdbooth | mriedem: Yeah, that just looks like a straight bug to me | 14:14 |
| mdbooth | mriedem: Gimme a sec I'll knock up a patch to test | 14:14 |
| *** links has quit IRC | 14:14 | |
| mriedem | mdbooth: ah so you're saying we need a wait for the abort_job to finish? | 14:15 |
| mdbooth | I think so, yes | 14:15 |
| mriedem | ok | 14:15 |
| *** jaosorior has quit IRC | 14:16 | |
| *** eliqiao has joined #openstack-nova | 14:16 | |
| kashyap | mriedem: By default, the copy job keeps syncing source & destination. | 14:16 |
| kashyap | There are two possible actions: | 14:16 |
| mriedem | mdbooth: you want to add a debug log statement in the case that we get into that resize_to block and log what the resize_to value is? and also wrap self._host.write_instance_config(xml) in the finally block in an excutils.save_and_reraise_exception? | 14:16 |
| kashyap | (1) Gracefully abort the sync | 14:17 |
| mdbooth | mriedem: Sure. Just going to throw something over the fence first though, no tests. | 14:17 |
| kashyap | (2) "Pivot" == Which implies abort, plus, point live guest to the new copy. | 14:17 |
| *** dims has joined #openstack-nova | 14:17 | |
| mdbooth | kashyap: So, if the 'pivot' is still ongoing, any idea why defineXML() would fail? | 14:18 |
| openstackgerrit | Matthew Booth proposed openstack/nova: libvirt: Wait for abort_job to complete when not resizing https://review.openstack.org/382449 | 14:18 |
| mriedem | mdbooth: write it against this bug https://bugs.launchpad.net/nova/+bug/1630600 | 14:19 |
| openstack | Launchpad bug 1630600 in OpenStack Compute (nova) "swap volume intermittently fails with "libvirtError: block copy still active: domain has active block job"" [Undecided,New] | 14:19 |
| *** jianghuaw has quit IRC | 14:19 | |
| *** TravT has joined #openstack-nova | 14:19 | |
| mdbooth | mriedem: So ^^^ just moves the wait | 14:19 |
| *** annegentle has joined #openstack-nova | 14:19 | |
| mdbooth | mriedem: Sure thing. I'll add the other things now too. Can you try running the test against ^^^ in the meantime? | 14:19 |
| *** lamt has joined #openstack-nova | 14:20 | |
| kashyap | mdbooth: Hmm, not sure. The defineXML() should occur post-pivot. | 14:20 |
| *** TravT has quit IRC | 14:20 | |
| mdbooth | kashyap: Right, but if resize_to is Falsey here, then we won't have waited for it to complete | 14:20 |
| mriedem | mdbooth: yup | 14:20 |
| mdbooth | Which would make it race. | 14:20 |
| *** TravT has joined #openstack-nova | 14:20 | |
| *** BobBall has quit IRC | 14:21 | |
| mriedem | sdague: welcome back btw | 14:21 |
| kashyap | mdbooth: How exactly are we getting the 'resize_to'? | 14:21 |
| mriedem | kashyap: see above | 14:22 |
| mriedem | kashyap: https://github.com/openstack/nova/blob/0f4bd241665c287e49f2d30ca79be96298217b7e/nova/compute/manager.py#L4927-L4931 | 14:22 |
| mriedem | kashyap: for the tempest test that comes to 0 | 14:22 |
| mdbooth | kashyap: mriedem posted it above. It should be set by compute manager if we're swapping to a volume of a different size. | 14:22 |
| mriedem | b/c the volumes are the same size | 14:22 |
| kashyap | mriedem: Oh, you've pointed it out above, looking. (/me still hungry, haven't had lunch) | 14:22 |
| * mriedem hasn't had coffee | 14:22 | |
| mdbooth | kashyap: ! | 14:22 |
| *** ducttape_ has joined #openstack-nova | 14:23 | |
| kashyap | mriedem: Insane, you breathe fire anyway | 14:23 |
| mriedem | well i'm working on it | 14:23 |
| *** Matti__ has joined #openstack-nova | 14:23 | |
| kashyap | Coffee or not, doesn't matter :P | 14:23 |
| *** eliqiao has quit IRC | 14:25 | |
| *** psachin has quit IRC | 14:28 | |
| *** eliqiao has joined #openstack-nova | 14:28 | |
| mdbooth | mriedem: Incidentally, is it safe/desirable to log connection_info? | 14:28 |
| gibi | jaypipes: I created the burndown chart for the notification transformation as you requested. https://vntburndown-gibi.rhcloud.com/index.html | 14:29 |
| mdbooth | safe: does it contain credentials? | 14:29 |
| mdbooth | desirable: is it useful? | 14:29 |
| mdbooth | At DEBUG, obviously | 14:29 |
| mriedem | mdbooth: not safe | 14:29 |
| mriedem | it has creds | 14:29 |
| mriedem | so you'd have to use strutils.mask_password | 14:29 |
| mdbooth | mriedem: k, thanks | 14:29 |
| sdague | mriedem: thanks | 14:29 |
| mdbooth | Got it | 14:30 |
| *** hussain_c has joined #openstack-nova | 14:30 | |
| mriedem | sdague: e-r graph is on a cron right? | 14:30 |
| *** gszasz has quit IRC | 14:30 | |
| sdague | mriedem: yes | 14:30 |
| mriedem | ok | 14:30 |
| sdague | oh, it broke, eh? | 14:30 |
| sdague | so, the problem with that is all the logs are inexcessible to all the devs | 14:31 |
| *** rodrigods has quit IRC | 14:32 | |
| *** rodrigods has joined #openstack-nova | 14:32 | |
| *** adisky has quit IRC | 14:32 | |
| *** akshai has quit IRC | 14:33 | |
| *** gszasz has joined #openstack-nova | 14:33 | |
| mriedem | sdague: ah there we go it just refreshed http://status.openstack.org//elastic-recheck/index.html | 14:34 |
| mriedem | anteaya: ^ | 14:35 |
| *** baoli_ has quit IRC | 14:35 | |
| *** moshele has joined #openstack-nova | 14:35 | |
| anteaya | mriedem: thanks, and yay! | 14:35 |
| *** hieulq__ has quit IRC | 14:35 | |
| mriedem | sdague: yeah, clarkb gave a paste but i was able to recreate the failure locally yesterday | 14:35 |
| mriedem | sdague: so i just self-approved the workaround https://review.openstack.org/#/c/382049/ | 14:35 |
| mriedem | not sure that's the right long term solution but it'd been busted since 9/29 | 14:35 |
| sdague | mriedem: oh, right, so something is failing on the ts parsing | 14:36 |
| mriedem | the timestamp value was too big so ES didn't send it back | 14:36 |
| mriedem | the trace is in the bug report | 14:36 |
| mriedem | so the data list was [] and we assumed it had a value | 14:36 |
| mriedem | so IndexError | 14:36 |
| sdague | gotcha | 14:36 |
| kashyap | mriedem: Ah, this race is only uncovered now because _swap_volume() Tempest test is enabled... | 14:37 |
| *** moshele has quit IRC | 14:38 | |
| *** Matti__ has quit IRC | 14:38 | |
| *** esberglu has quit IRC | 14:38 | |
| *** edmondsw has quit IRC | 14:38 | |
| *** cleong has quit IRC | 14:38 | |
| *** maticue has quit IRC | 14:38 | |
| *** pjm6_ has quit IRC | 14:38 | |
| *** mkoderer has quit IRC | 14:38 | |
| *** hogepodge has quit IRC | 14:38 | |
| *** tesseract- has quit IRC | 14:38 | |
| *** mjura has quit IRC | 14:38 | |
| *** sandanar has quit IRC | 14:38 | |
| *** narayrak has quit IRC | 14:38 | |
| *** rfolco has quit IRC | 14:38 | |
| *** diana-test has quit IRC | 14:38 | |
| *** jdillaman1 has quit IRC | 14:38 | |
| *** ducnc has quit IRC | 14:38 | |
| *** jed56 has quit IRC | 14:38 | |
| *** artom has quit IRC | 14:38 | |
| *** clenimar has quit IRC | 14:38 | |
| *** davidlenwell has quit IRC | 14:38 | |
| *** gabor_antal has quit IRC | 14:38 | |
| *** ejat has quit IRC | 14:38 | |
| *** mugsie has quit IRC | 14:38 | |
| *** anthonyper has quit IRC | 14:38 | |
| *** kwathore has quit IRC | 14:38 | |
| *** mugsie has joined #openstack-nova | 14:39 | |
| *** pjm6 has joined #openstack-nova | 14:39 | |
| *** esberglu has joined #openstack-nova | 14:39 | |
| *** Matti__ has joined #openstack-nova | 14:39 | |
| *** edmondsw has joined #openstack-nova | 14:39 | |
| *** rfolco has joined #openstack-nova | 14:39 | |
| *** kwathore has joined #openstack-nova | 14:39 | |
| *** mjura has joined #openstack-nova | 14:39 | |
| *** diana-test has joined #openstack-nova | 14:39 | |
| *** maticue has joined #openstack-nova | 14:39 | |
| *** artom has joined #openstack-nova | 14:39 | |
| *** gabor_antal has joined #openstack-nova | 14:39 | |
| *** clenimar has joined #openstack-nova | 14:39 | |
| *** jdillaman1 has joined #openstack-nova | 14:39 | |
| *** sandanar has joined #openstack-nova | 14:39 | |
| *** narayrak has joined #openstack-nova | 14:39 | |
| *** anthonyper has joined #openstack-nova | 14:39 | |
| *** hogepodge has joined #openstack-nova | 14:39 | |
| *** ducnc has joined #openstack-nova | 14:39 | |
| *** bnemec has quit IRC | 14:39 | |
| *** tesseract- has joined #openstack-nova | 14:39 | |
| *** dharinic|pto is now known as dharinic | 14:39 | |
| *** ejat has joined #openstack-nova | 14:40 | |
| *** ejat has joined #openstack-nova | 14:40 | |
| *** davidlenwell has joined #openstack-nova | 14:40 | |
| *** alexchad_ has joined #openstack-nova | 14:40 | |
| *** mkoderer has joined #openstack-nova | 14:40 | |
| mriedem | kashyap: yes | 14:40 |
| *** burgerk has joined #openstack-nova | 14:40 | |
| mriedem | i've always hated swap_volume b/c it's not tested | 14:40 |
| mriedem | so was loathe to approve changes to it | 14:40 |
| kashyap | Yeah, I was just about to complain | 14:40 |
| kashyap | It's riddled with caveats & hedges that you have to carefully go through | 14:41 |
| *** akshai has joined #openstack-nova | 14:41 | |
| *** DuncanT has quit IRC | 14:41 | |
| *** cleong has joined #openstack-nova | 14:42 | |
| pkoniszewski | mriedem: hi, I just proposed a chain to implement test that will LM VMs back and forth, https://review.openstack.org/#/c/382456/ -> https://review.openstack.org/#/c/382451/ -> https://review.openstack.org/#/c/379638/ | 14:42 |
| *** akshai has quit IRC | 14:42 | |
| *** artom has quit IRC | 14:42 | |
| mriedem | pkoniszewski: yeah did you see my -1 on the devstack change? | 14:43 |
| mriedem | oh i see you updated it | 14:43 |
| jaypipes | gibi: awesome, thanks! | 14:43 |
| mriedem | cool | 14:43 |
| *** jed56 has joined #openstack-nova | 14:43 | |
| pkoniszewski | mriedem: yeah, did wrong dependency chain | 14:44 |
| *** bnemec has joined #openstack-nova | 14:44 | |
| *** akshai has joined #openstack-nova | 14:44 | |
| mdbooth | Question for anybody: calling save_and_reraise_exception() from a finally block where we may or may not be in exception context. | 14:44 |
| mdbooth | Looks to me like that will fail. | 14:44 |
| *** penick has quit IRC | 14:44 | |
| mdbooth | I can obviously hack round this in any number of ways, but is anybody aware of a quick, clean way? | 14:45 |
| mdbooth | This can't be the first time somebody has wanted to do this | 14:45 |
| *** david-lyle has joined #openstack-nova | 14:45 | |
| mriedem | pkoniszewski: i think the dep chain is still wrong | 14:45 |
| mriedem | d-g > devstack > tempest | 14:45 |
| mriedem | mdbooth: maybe just use a simple try/except in the finally block | 14:46 |
| mriedem | although, if that fails we want to reraise | 14:46 |
| mdbooth | That's one way, but I still need to save the exception myself | 14:46 |
| mdbooth | right... | 14:46 |
| mriedem | but we want to raise the original exception if there was one | 14:46 |
| mriedem | hmm | 14:46 |
| mdbooth | I can do it, just wondering if there's an existing pattern to follow | 14:47 |
| mriedem | mdbooth: so, | 14:47 |
| *** kfarr has joined #openstack-nova | 14:47 | |
| mriedem | well, | 14:47 |
| *** DuncanT has joined #openstack-nova | 14:47 | |
| mdbooth | Ah, wait | 14:47 |
| mdbooth | if exc_type is not None... | 14:47 |
| mriedem | was going to say, change the finally to an except block and use the save_and_reraise_exception in there | 14:47 |
| mriedem | i assume the context manager just noops? | 14:47 |
| mdbooth | I think it might just work anyway | 14:47 |
| *** artom has joined #openstack-nova | 14:48 | |
| mriedem | mdbooth: https://github.com/openstack/oslo.utils/blob/3.16.0/oslo_utils/excutils.py#L194 | 14:48 |
| *** Swami has joined #openstack-nova | 14:48 | |
| mriedem | i think it'll fail | 14:48 |
| mdbooth | mriedem: I think I can pass (reraise=False) | 14:49 |
| mdbooth | Err, then it will never reraise | 14:49 |
| mdbooth | That's undesirable | 14:49 |
| *** yamahata has quit IRC | 14:49 | |
| mriedem | right | 14:49 |
| mriedem | so, | 14:49 |
| mriedem | however about we just have both an except and finally? | 14:49 |
| *** yamahata has joined #openstack-nova | 14:49 | |
| mriedem | does calling that method even matter if we fail? | 14:50 |
| mdbooth | Yes | 14:50 |
| mdbooth | So, you can't rebase a persistent domain | 14:50 |
| mdbooth | So we temporarily undefine the domain | 14:50 |
| mdbooth | We need it to be persisten again afterwards, even if it fails | 14:51 |
| * mdbooth notes that this is also undesirable | 14:51 | |
| mriedem | mdbooth: use sys.exc_info()? | 14:51 |
| mdbooth | Something to do with the semantics of rebase not being restartable iirc. | 14:51 |
| pkoniszewski | mriedem: hope this one is correct, tempest change is the first patch | 14:51 |
| mdbooth | mriedem: I'll hack something up :) | 14:52 |
| mriedem | exc_info = sys.exc_info() | 14:52 |
| mriedem | six.reraise(*exc_info) | 14:52 |
| mriedem | reraise if the exc_info tuple values are not None | 14:53 |
| *** david-lyle has quit IRC | 14:54 | |
| *** lpetrut has quit IRC | 14:55 | |
| *** david-lyle has joined #openstack-nova | 15:00 | |
| *** Matti__ has quit IRC | 15:00 | |
| *** Matti_ has quit IRC | 15:00 | |
| *** gszasz has quit IRC | 15:00 | |
| *** jistr is now known as jistr|call | 15:01 | |
| mriedem | pkoniszewski: raj_singh: the tempest config option isn't defined in this change.. https://review.openstack.org/#/c/379638/ | 15:02 |
| *** claudiub has quit IRC | 15:03 | |
| mriedem | mdbooth: btw, the devstack patch to test swap volume is updated with a depends on your fix | 15:04 |
| mdbooth | mriedem: Cool. Are you going to spam it with rechecks a few times? | 15:05 |
| *** jlanoux has quit IRC | 15:05 | |
| openstackgerrit | Sergey Nikitin proposed openstack/nova-specs: Restore V3 Diagnostics info https://review.openstack.org/357884 | 15:05 |
| mdbooth | I'm going to update it, but the core change will remain the same. | 15:05 |
| *** mjura has quit IRC | 15:05 | |
| openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova: libvirt: virtlogd: use virtlogd for char devices https://review.openstack.org/323765 | 15:09 |
| openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova: libvirt: remove py26 compat code in "get_console_output" https://review.openstack.org/334480 | 15:09 |
| *** Matti__ has joined #openstack-nova | 15:10 | |
| *** oanson has quit IRC | 15:11 | |
| *** cdent has joined #openstack-nova | 15:11 | |
| *** rcernin has quit IRC | 15:12 | |
| mdbooth | mriedem: Can you confirm, btw, that the host this is running on has libvirt <1.2.7 ? | 15:12 |
| sfinucan | wznoinsk: Does the Intel PCI CI do SR-IOV testing? | 15:12 |
| sfinucan | If not, what does it do? :) | 15:12 |
| *** stewie925 has joined #openstack-nova | 15:12 | |
| *** gszasz has joined #openstack-nova | 15:12 | |
| mdbooth | Because if it's libvirt >= 1.2.7 this should be fixed, and we might be looking in the wrong place. | 15:12 |
| *** pcaruana has quit IRC | 15:13 | |
| kashyap | mdbooth: Yes, but I presume we're using 1.2.5? I'm also curious what comes iwth 1.2.7 | 15:13 |
| *** jistr|call is now known as jistr | 15:13 | |
| mdbooth | https://bugzilla.redhat.com/show_bug.cgi?id=1119173 | 15:13 |
| openstack | bugzilla.redhat.com bug 1119173 in libvirt "The default behavor of abort block job with pivot flag isn't sync" [Unspecified,Closed: nextrelease] - Assigned to eblake | 15:13 |
| mriedem | mdbooth: host would be 1.3.1 i think | 15:14 |
| mriedem | mdbooth: yeah it's a xenial job so it's libvirt 1.3.1 http://logs.openstack.org/73/374373/2/check/gate-tempest-dsvm-full-ubuntu-xenial/149fe3e/logs/dpkg-l.txt.gz | 15:14 |
| *** salv-orl_ has quit IRC | 15:15 | |
| mdbooth | mriedem: In that case we might have the wrong root cause. | 15:15 |
| *** hshiina has quit IRC | 15:15 | |
| mdbooth | mriedem: Meh, I still think this is a bug. Lets get it out of the way and see what's left. | 15:15 |
| *** ijw has joined #openstack-nova | 15:16 | |
| *** armax has joined #openstack-nova | 15:17 | |
| mdbooth | Of course, it could just change the timing and fix it by coincidence :/ | 15:17 |
| *** hieulq__ has joined #openstack-nova | 15:18 | |
| mdbooth | mriedem: In that case, I suspect your exception in the try block theory. | 15:19 |
| kashyap | mdbooth: Still, as you noted -- if the definXML() is issued _while_ the pivot is happening the race could still be there | 15:19 |
| *** rrecio has joined #openstack-nova | 15:20 | |
| *** rrecio has quit IRC | 15:20 | |
| kashyap | And, heh, yes, it appears "fix it by coincidence" is how _swap_volume() manages to stays alive :-) | 15:21 |
| *** rrecio has joined #openstack-nova | 15:21 | |
| *** Matti__ has quit IRC | 15:21 | |
| *** diogogmt has joined #openstack-nova | 15:22 | |
| wznoinsk | sfinucan: it does use SRIOV nic for passthru: http://52.27.155.124/pci/295315/13/console_status.log.gz | 15:22 |
| wznoinsk | if that's what you're asking | 15:23 |
| *** tesseract- has quit IRC | 15:24 | |
| *** code-R has quit IRC | 15:25 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Reserve NUMA nodes with PCI devices attached https://review.openstack.org/364468 | 15:25 |
| sfinucan | wznoinsk: Sure is | 15:25 |
| *** code-R has joined #openstack-nova | 15:25 | |
| sfinucan | and these are direct passthrough only, i.e. neutron isn't used? | 15:26 |
| *** david-lyle has quit IRC | 15:26 | |
| openstackgerrit | Matthew Booth proposed openstack/nova: libvirt: Wait for abort_job to complete when not resizing https://review.openstack.org/382449 | 15:27 |
| *** andreas_s has quit IRC | 15:27 | |
| wznoinsk | sfinucan: correct, no neutron: http://52.27.155.124/pci/295315/13/screen-logs/, it's 'normal' vnic_type | 15:29 |
| *** bvanhav has quit IRC | 15:31 | |
| *** vishwanathj has joined #openstack-nova | 15:31 | |
| *** david-lyle has joined #openstack-nova | 15:32 | |
| *** peter-hamilton has joined #openstack-nova | 15:33 | |
| *** lucasagomes is now known as lucas-hungry | 15:35 | |
| *** eglynn has joined #openstack-nova | 15:36 | |
| *** jaosorior has joined #openstack-nova | 15:37 | |
| *** Matti__ has joined #openstack-nova | 15:37 | |
| *** jlanoux has joined #openstack-nova | 15:37 | |
| *** gouthamr has quit IRC | 15:38 | |
| *** Matti_ has joined #openstack-nova | 15:38 | |
| *** sudipto has joined #openstack-nova | 15:38 | |
| *** sudswas__ has joined #openstack-nova | 15:38 | |
| *** baoli has joined #openstack-nova | 15:39 | |
| wznoinsk | sfinucan: actually sorry, got confused, it does pass the VFs as hostdev'ices http://52.27.155.124/pci/295315/13/screen-logs/n-cpu.log.2016-09-02-204034.gz (look for address bus="0x04"), matches bus of the VFs listed here http://52.27.155.124/pci/295315/13/console_status.log.gz | 15:40 |
| *** rama_y has joined #openstack-nova | 15:40 | |
| *** zenoway has joined #openstack-nova | 15:41 | |
| *** pbandark has joined #openstack-nova | 15:41 | |
| *** Matti__ has quit IRC | 15:41 | |
| *** paul-carlton2 has quit IRC | 15:42 | |
| *** jlanoux_ has joined #openstack-nova | 15:42 | |
| *** zhenguo has quit IRC | 15:43 | |
| raj_singh | mriedem: https://review.openstack.org/#/c/379638 | 15:43 |
| jlk | mdbooth: it's been suggested that you might have run across something I'm seeing, where an instance is booted from a snapshot (the snapshot is quite large) and seems to immediately copy-on-write the entire space of the snapshot, leaving us with an instance that is consuming 2x the on-disk size as the flavor states, and having both the file in _base and in instances/<UUID> held open by libvirt. Have you ran into anything like this? | 15:44 |
| mdbooth | jlk: Is this the same bug as with shelve? | 15:45 |
| jlk | I don't know what that bug is. | 15:45 |
| *** jlanoux has quit IRC | 15:45 | |
| *** salv-orlando has joined #openstack-nova | 15:45 | |
| jlk | I'm actually not even sure this is a "bug" per se, or just a quirk of this customer's snapshot contents and instance contents (it's windows) | 15:45 |
| *** gouthamr has joined #openstack-nova | 15:45 | |
| mdbooth | jlk: This sounds like a bug I reported against shelve. | 15:45 |
| mdbooth | jlk: I don't have the cycles to double check right now, but iirc... | 15:46 |
| mriedem | raj_singh: commented | 15:46 |
| jlk | I'm happy to plow through it | 15:46 |
| mdbooth | If you're using qcow2, we download the snapshot from glance | 15:47 |
| *** bvanhav has joined #openstack-nova | 15:47 | |
| mdbooth | The snapshot is flattened, i.e. it contains both the original image and the overlay (modifications) | 15:47 |
| mdbooth | So it's the same size as the image | 15:47 |
| *** Swami has quit IRC | 15:47 | |
| mdbooth | For shelve, at least, this is written to _base, and we create a new overlay on top of that in the instance directory | 15:48 |
| raj_singh | mriedem: Thx, uploading another patch set | 15:48 |
| mdbooth | So you've immediately duplicated your original image data | 15:48 |
| jlk | nod | 15:49 |
| mdbooth | And over time you'll duplicate it again in the instance dir as the disk diverges from the original. | 15:49 |
| jlk | what I've tested on a linux image, it appears with our setting we download the snapshot to _base, fallocate it to the size of the flavor disk, then create an overlay file in the instance path that is tiny to start with. | 15:49 |
| *** hieulq__ has quit IRC | 15:49 | |
| mdbooth | If it's the same bug with snapshot, that's going to be pretty hard to solve, though | 15:49 |
| mdbooth | It's architectural. | 15:49 |
| mdbooth | The fallocate is irrelevant, but yeah. | 15:50 |
| mdbooth | We need to store different data in glance. | 15:50 |
| jlk | I only mention it because in my test case, the VM flavor was 80G, but the image itself was 1.2g | 15:50 |
| mdbooth | Not a one-liner :) | 15:50 |
| jlk | (snapshot) | 15:50 |
| jlk | but I think in my customer's case, the snapshot image is a full 300~ gigs | 15:51 |
| mdbooth | That code is so stupid... | 15:51 |
| *** annegentle has quit IRC | 15:51 | |
| mdbooth | s/stupid/crufty/ | 15:51 |
| jlk | it's twisty | 15:51 |
| mdbooth | Yup, *very* hard to understand fully | 15:51 |
| mdbooth | My personal working set frequently isn't quite big enough, and bits fall out the end while I'm still loading them at the beginning. | 15:52 |
| *** annegentle has joined #openstack-nova | 15:52 | |
| jlk | lol | 15:53 |
| mdbooth | jlk: So the fallocate is likely a separate bug, tbh. | 15:53 |
| *** gouthamr has quit IRC | 15:53 | |
| *** david-lyle has quit IRC | 15:53 | |
| *** sudipto has quit IRC | 15:53 | |
| *** sudswas__ has quit IRC | 15:53 | |
| jlk | it's a config item we have turned on | 15:53 |
| mdbooth | I can't think why we'd fallocate anything in the image cache. | 15:53 |
| jlk | allocate_image = space | 15:54 |
| jlk | but yeah I don't know why it allocates that in the _base path | 15:54 |
| *** gouthamr has joined #openstack-nova | 15:54 | |
| *** Matti__ has joined #openstack-nova | 15:54 | |
| *** sandanar has quit IRC | 15:54 | |
| jlk | since who knows what other flavors will boot from that cache | 15:54 |
| *** matrohon has quit IRC | 15:54 | |
| *** kfarr_ has joined #openstack-nova | 15:54 | |
| jlk | our goal with that was to make sure that an instance's flavor's disk reservation is accounted for, and couldn't get overcommitted and run out when one instance starts writing a lot | 15:55 |
| openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova-specs: libvirt: Move spec "libvirt-virtlogd" from Newton to Ocata https://review.openstack.org/382526 | 15:55 |
| *** ducttape_ has quit IRC | 15:56 | |
| *** kfarr has quit IRC | 15:56 | |
| *** code-R_ has joined #openstack-nova | 15:57 | |
| *** mdrabe has quit IRC | 15:57 | |
| jlk | I just wish I knew what was going on with the customer's instance so that the copy on write "savings" weren't lost. | 15:57 |
| openstackgerrit | Markus Zoeller (markus_z) proposed openstack/nova-specs: libvirt: Move spec "libvirt-virtlogd" from Newton to Ocata https://review.openstack.org/382526 | 15:58 |
| *** Sukhdev_ has joined #openstack-nova | 15:59 | |
| *** ducttape_ has joined #openstack-nova | 15:59 | |
| jaypipes | mriedem, sdague: have you guys noticed a particular issue in check/gate jobs about secgroups/networks/subnets already in use? I'm looking in logstash e-r with the query message:"An object with that identifier already exists" and seeing hundreds of results, but I see no bug in launchpad for it. | 16:00 |
| *** code-R has quit IRC | 16:00 | |
| *** claudiub has joined #openstack-nova | 16:00 | |
| *** markus_z has quit IRC | 16:00 | |
| *** kfarr_ has quit IRC | 16:00 | |
| *** rrecio_ has joined #openstack-nova | 16:00 | |
| *** kfarr has joined #openstack-nova | 16:01 | |
| *** mdrabe has joined #openstack-nova | 16:01 | |
| mdbooth | kashyap mriedem: I've been meditating on those logs and came across this: | 16:01 |
| mdbooth | 2016-10-04 15:54:52.076+0000: 30555: error : qemuDomainBlockPivot:16137 : block | 16:02 |
| mdbooth | copy still active: disk 'vdb' not ready for pivot yet | 16:02 |
| *** ducttape_ has quit IRC | 16:02 | |
| mdbooth | I wonder if we've got a race calling abort_job() | 16:02 |
| mdbooth | Immediately before that, I see: | 16:02 |
| mdbooth | 2016-10-04 15:54:52.075+0000: 30550: info : qemuMonitorIOProcess:423 : | 16:03 |
| mdbooth | QEMU_MONITOR_IO_PROCESS: mon=0x7fa43000ee60 buf={"return": [{"io-status": "ok", | 16:03 |
| mdbooth | "device": "drive-virtio-disk1", "busy": true, "len": 1073741824, "offset": | 16:03 |
| mdbooth | 1073741824, "paused": false, "speed": 0, "ready": false, "type": "mirror"}], | 16:03 |
| mdbooth | "id": "libvirt-34"} | 16:03 |
| mdbooth | So immediately before that, BlockJobInfo has return cur==end | 16:03 |
| mdbooth | signalling the mirror phase is complete | 16:03 |
| mdbooth | But then we fail to pivot... | 16:03 |
| *** rrecio has quit IRC | 16:04 | |
| mriedem | jaypipes: link? | 16:04 |
| *** stewie925 has quit IRC | 16:04 | |
| mriedem | mdbooth: so we thought we were complete but the disk isn't ready for pivot yet | 16:05 |
| mriedem | that's fun | 16:05 |
| mdbooth | mriedem: CAVEAT: I'm not familiar with this functionality :) | 16:05 |
| mdbooth | However, that's how it looks | 16:05 |
| jaypipes | mriedem: here's an example of the failure: http://logs.openstack.org/00/382000/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/07e5243/console.html | 16:06 |
| *** Sukhdev_ has quit IRC | 16:06 | |
| mriedem | mdbooth: and we don't get that libvirtError b/c we're swallowing the original exception | 16:06 |
| mdbooth | Just reading the BlockJobAbort api | 16:06 |
| *** baoli has quit IRC | 16:06 | |
| mdbooth | mriedem: boom! | 16:06 |
| mriedem | fun | 16:06 |
| jaypipes | mriedem: I don't know how to get the URL for an e-r query. If I change the query in logstash/kibana, the URL doesn't change :( | 16:06 |
| mriedem | seems like a libvirt bug | 16:06 |
| mdbooth | Either that, or we're abusing the api | 16:06 |
| mdbooth | But at first glance, yeah | 16:06 |
| mriedem | jaypipes: so this? http://logs.openstack.org/00/382000/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/07e5243/logs/screen-n-api.txt.gz?level=TRACE#_2016-10-05_14_35_04_333 | 16:07 |
| mriedem | deadlock city http://logs.openstack.org/00/382000/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/07e5243/logs/screen-q-svc.txt.gz?level=TRACE | 16:08 |
| *** suresh12 has joined #openstack-nova | 16:09 | |
| jaypipes | mriedem: ah, good spot. is this something you've seen a lot recently? | 16:09 |
| mriedem | maybe | 16:09 |
| *** code-R_ has quit IRC | 16:09 | |
| mriedem | i was looking for something a week or two ago like this and got lost in the neutron logs | 16:09 |
| *** Apoorva has joined #openstack-nova | 16:10 | |
| mriedem | well let me fire up logstash | 16:10 |
| *** mvk has quit IRC | 16:11 | |
| *** ssl329 has joined #openstack-nova | 16:11 | |
| mriedem | jaypipes: btw the conflicts in http://logs.openstack.org/00/382000/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/07e5243/console.html are just failures on teardown | 16:12 |
| mriedem | teardown vomiting because something went crazy in the test | 16:12 |
| mriedem | checking syslog and dstat around 14:35:04.333 | 16:13 |
| mriedem | fwiw it doesn't look like a recent regression | 16:14 |
| *** Apoorva has quit IRC | 16:14 | |
| ssl329 | not sure if this is the right channel to ask.. I'm testing nova+ironic, I use "nove boot" to create baremetal instance. i tried to pass files over to the baremetal instance using "nova boot" with "--file" option, however, it doesn't seem to works (no file is sent)... is this supported? of any additional parameters that I need to change at nova.conf/ironic.conf? | 16:14 |
| mriedem | it's barfing since we're doing our super expensive proxy call to build the list of security groups when doing a server show | 16:14 |
| *** ihrachys has quit IRC | 16:15 | |
| *** suresh12_ has joined #openstack-nova | 16:17 | |
| *** suresh12 has quit IRC | 16:17 | |
| mdbooth | mriedem: I have to head out. Sounds like we need to do some more liasing with the libvirt folks, which is kashyap's forte. I'll catch up with him in the morning. | 16:17 |
| mriedem | jaypipes: hmm, logstash is showing 9/27 | 16:17 |
| mriedem | when it starts | 16:17 |
| mriedem | which is < 10 days of the logs we keep | 16:17 |
| mriedem | jaypipes: i wonder if the TLS stuff went in on 9/27 | 16:18 |
| mriedem | mtreinish: ^ do you remember? | 16:18 |
| * kashyap reads scrollback | 16:18 | |
| *** haplo37_ has quit IRC | 16:18 | |
| mriedem | yeah https://review.openstack.org/#/c/375776/ | 16:18 |
| mdbooth | kashyap: Looks like BlockJobInfo returned cur==end, then the pivot failed with: | 16:18 |
| mtreinish | mriedem: that sounds about the right timing | 16:18 |
| mriedem | mtreinish: jaypipes: ^ | 16:18 |
| mdbooth | 2016-10-04 15:54:52.076+0000: 30555: error : qemuDomainBlockPivot:16137 : block | 16:18 |
| mdbooth | copy still active: disk 'vdb' not ready for pivot yet | 16:18 |
| mdbooth | Look in the libvirt logs for the job mriedem originally linked. | 16:19 |
| mdbooth | You can see the BlockJobInfo just before it | 16:19 |
| mdbooth | I just read the docs for rebase(), which suggests we're correctly assuming we can pivot at this stage | 16:19 |
| kashyap | mdbooth: Yeah, will do. I actually have to go bring some groceries befor the store is closed. Be back in a bit | 16:19 |
| kashyap | mdbooth: Yes, once cur == end | 16:19 |
| openstackgerrit | Drew Thorstensen (thorst) proposed openstack/nova-specs: PowerVM Initial Driver Work https://review.openstack.org/381772 | 16:20 |
| mdbooth | But I'm wondering if there might be a race in there, perhaps if something is still writing to the source? | 16:20 |
| kashyap | Pivot is definitely a valid operation | 16:20 |
| * kashyap looks for 'len' / 'offset' values in the logs | 16:20 | |
| kashyap | mdbooth: "ready": false -- means the operation is still in progress | 16:20 |
| mdbooth | kashyap: You can see it almost immediately above | 16:20 |
| *** haplo37_ has joined #openstack-nova | 16:21 | |
| *** nic has joined #openstack-nova | 16:21 | |
| mdbooth | kashyap: Ooh, that's interesting. We're not looking for that at all. | 16:21 |
| kashyap | mdbooth: Right...is there an event? /me looks | 16:21 |
| mdbooth | "io-status": "ok", | 16:21 |
| mdbooth | "device": "drive-virtio-disk1", "busy": true, "len": 1073741824, "offset": | 16:21 |
| mdbooth | 1073741824, "paused": false, "speed": 0, "ready": false, "type": "mirror" | 16:21 |
| mdbooth | There's no event that I can see | 16:21 |
| kashyap | mdbooth: Can you quickly paste the log URL please? | 16:22 |
| mdbooth | Why why would len==offset and ready==false? | 16:22 |
| mdbooth | http://logs.openstack.org/73/374373/2/check/gate-tempest-dsvm-full-ubuntu-xenial/149fe3e/logs/libvirt/libvirtd.txt.gz | 16:22 |
| mdbooth | I downloaded them. | 16:22 |
| kashyap | mdbooth: If len == offset, and if we get BLOCK_JOB_COMPLETED event, then we're good. | 16:22 |
| * kashyap brings an interesting commit message for mdbooth | 16:22 | |
| mdbooth | We get len == offset, but *no* BLOCK_JOB_COMPLETED | 16:23 |
| mriedem | jaypipes: https://bugs.launchpad.net/nova/+bug/1630664 | 16:23 |
| openstack | Launchpad bug 1630664 in OpenStack Compute (nova) "Intermittent failure in n-api connecting to neutron to list ports after TLS was enabled in CI" [Undecided,New] | 16:23 |
| mdbooth | kashyap: Anyway, I had to leave 5 mins ago. Catch up in the morning. | 16:23 |
| kashyap | mdbooth: Here's something you'll surely not find boring -- http://libvirt.org/git/?p=libvirt.git;a=commit;h=eae5924 | 16:23 |
| *** rrecio has joined #openstack-nova | 16:23 | |
| kashyap | Sure. I too have to run in a few, urgent errands. But will check in a bit | 16:24 |
| mdbooth | kashyap: That looks *exactly* like what we're seeing here | 16:24 |
| kashyap | "Few parts of the code looked at the current progress of and assumed that a two phase blockjob is in the _READY state as soon as the progress reached 100% (info.cur == info.end). In current versions of qemu this assumption is invalid and qemu exposes a new flag 'ready' in the query-block-jobs output that is set to true if the job is actually finished." | 16:24 |
| mdbooth | Eurgh, we should really move to processing those events :/ | 16:25 |
| kashyap | mdbooth: :-) | 16:25 |
| * mdbooth really really runs this time | 16:25 | |
| kashyap | `git tag --contains eae5924` says | 16:25 |
| mdbooth | laters | 16:25 |
| kashyap | We need at least 1.2.18 | 16:25 |
| kashyap | (To take advantage of that "ready: true) | 16:25 |
| *** woodster_ has joined #openstack-nova | 16:26 | |
| *** rrecio_ has quit IRC | 16:26 | |
| *** nkrinner is now known as nkrinner_afk | 16:27 | |
| *** ayoung has joined #openstack-nova | 16:28 | |
| *** ducttape_ has joined #openstack-nova | 16:31 | |
| *** stewie925 has joined #openstack-nova | 16:31 | |
| *** mdrabe has quit IRC | 16:31 | |
| *** mdrabe has joined #openstack-nova | 16:31 | |
| *** unicell1 has joined #openstack-nova | 16:32 | |
| *** raunak has joined #openstack-nova | 16:33 | |
| *** raunak has quit IRC | 16:33 | |
| *** ociuhandu has quit IRC | 16:33 | |
| *** unicell has quit IRC | 16:33 | |
| *** gus_ is now known as gus | 16:33 | |
| *** raunak has joined #openstack-nova | 16:34 | |
| *** lucas-hungry is now known as lucasagomes | 16:35 | |
| *** ducttape_ has quit IRC | 16:35 | |
| *** sahid has quit IRC | 16:36 | |
| *** suresh12_ has quit IRC | 16:37 | |
| openstackgerrit | STEW TY proposed openstack/nova: Transform instance.resume notifications https://review.openstack.org/375784 | 16:38 |
| *** jlanoux_ has quit IRC | 16:39 | |
| *** annegentle has quit IRC | 16:41 | |
| *** jaosorior has quit IRC | 16:42 | |
| openstackgerrit | Drew Thorstensen (thorst) proposed openstack/nova-specs: PowerVM Initial Driver Work https://review.openstack.org/381772 | 16:43 |
| *** yamahata has quit IRC | 16:44 | |
| *** suresh12 has joined #openstack-nova | 16:44 | |
| *** krypto has joined #openstack-nova | 16:45 | |
| *** unicell1 has quit IRC | 16:46 | |
| *** claudiub has quit IRC | 16:47 | |
| openstackgerrit | STEW TY proposed openstack/nova: Transform instance.unshelve notifications https://review.openstack.org/382565 | 16:49 |
| *** hehx has quit IRC | 16:49 | |
| *** Matti__ has quit IRC | 16:50 | |
| *** Matti_ has quit IRC | 16:50 | |
| *** david-lyle has joined #openstack-nova | 16:52 | |
| *** Apoorva has joined #openstack-nova | 16:53 | |
| *** yonglihe has quit IRC | 16:55 | |
| *** lucasagomes is now known as lucas-brb | 16:55 | |
| openstackgerrit | Andrew Laski proposed openstack/nova: Remove unnecessary credential sanitation for logging https://review.openstack.org/382571 | 16:56 |
| *** gszasz has quit IRC | 16:59 | |
| *** narayrak has quit IRC | 16:59 | |
| *** tejasvi has joined #openstack-nova | 17:00 | |
| mriedem | jaypipes: sdague: johnthetubaguy: https://hangouts.google.com/hangouts/_/calendar/bWF0dHJpZWQyMkBnbWFpbC5jb20.5hegb2ek1bfjdh34e985i7lndg?authuser=0 | 17:00 |
| *** derekh has quit IRC | 17:01 | |
| jaypipes | mriedem: ty sir. | 17:01 |
| johnthetubaguy | mriedem: be with you in a sec, finishing up another meeting | 17:01 |
| *** hehx has joined #openstack-nova | 17:01 | |
| *** tblakes has quit IRC | 17:04 | |
| *** jed56 has quit IRC | 17:05 | |
| *** tjones has joined #openstack-nova | 17:05 | |
| *** kfarr has quit IRC | 17:06 | |
| *** tejasvi has left #openstack-nova | 17:06 | |
| *** paul-carlton2 has joined #openstack-nova | 17:09 | |
| *** mkoderer has quit IRC | 17:09 | |
| *** lucas-brb is now known as lucasagomes | 17:11 | |
| *** AJaeger has joined #openstack-nova | 17:11 | |
| AJaeger | nova team, could you review https://review.openstack.org/379962 , please? This fixes an integration test of nova and olso that runs as periodic job every day. | 17:12 |
| AJaeger | If there's no interest in that job, tell me and we stop it... | 17:12 |
| *** alexchad_ has quit IRC | 17:12 | |
| oomichi | AJaeger: thanks, +2 | 17:14 |
| *** suresh12 has quit IRC | 17:15 | |
| *** baoli has joined #openstack-nova | 17:18 | |
| *** amotoki has quit IRC | 17:21 | |
| AJaeger | thanks, oomichi. | 17:21 |
| *** unicell has joined #openstack-nova | 17:21 | |
| *** yamahata has joined #openstack-nova | 17:22 | |
| *** mgoddard has quit IRC | 17:23 | |
| *** rrecio has quit IRC | 17:24 | |
| *** rrecio has joined #openstack-nova | 17:25 | |
| *** mvk has joined #openstack-nova | 17:25 | |
| *** cleong has quit IRC | 17:27 | |
| *** claudiub has joined #openstack-nova | 17:27 | |
| AJaeger | thanks, melwitt ! | 17:28 |
| *** AJaeger has left #openstack-nova | 17:28 | |
| *** rrecio has quit IRC | 17:29 | |
| *** browne has joined #openstack-nova | 17:30 | |
| *** Swami has joined #openstack-nova | 17:30 | |
| openstackgerrit | greghaynes proposed openstack/nova: Make api-paste.ini part of python module https://review.openstack.org/382582 | 17:30 |
| *** pbandark has quit IRC | 17:30 | |
| openstackgerrit | greghaynes proposed openstack/nova: Make api-paste.ini part of python module https://review.openstack.org/382582 | 17:31 |
| *** rrecio has joined #openstack-nova | 17:31 | |
| *** ducttape_ has joined #openstack-nova | 17:31 | |
| *** ihrachys has joined #openstack-nova | 17:32 | |
| *** yamahata has quit IRC | 17:34 | |
| *** jichen_ has joined #openstack-nova | 17:34 | |
| *** Matti_ has joined #openstack-nova | 17:35 | |
| *** hemanthm is now known as hemanthm|lunch | 17:36 | |
| *** jichen has quit IRC | 17:36 | |
| *** jichen_ is now known as jichen | 17:36 | |
| *** ducttape_ has quit IRC | 17:36 | |
| *** lpetrut has joined #openstack-nova | 17:37 | |
| *** dikonoor has joined #openstack-nova | 17:37 | |
| *** gouthamr has quit IRC | 17:41 | |
| *** ociuhandu has joined #openstack-nova | 17:42 | |
| *** haplo37 has quit IRC | 17:43 | |
| *** haplo37 has joined #openstack-nova | 17:44 | |
| *** artom has quit IRC | 17:45 | |
| *** karthiks has quit IRC | 17:46 | |
| *** lpetrut has quit IRC | 17:46 | |
| *** artom has joined #openstack-nova | 17:46 | |
| openstackgerrit | Divya K Konoor proposed openstack/nova-specs: Allow admin users to pass key_name corresponding to another user and deploy a VM successfully https://review.openstack.org/382588 | 17:53 |
| *** cmartin has quit IRC | 17:54 | |
| *** moshele has joined #openstack-nova | 17:54 | |
| *** ihrachys has quit IRC | 17:54 | |
| *** suresh12 has joined #openstack-nova | 17:57 | |
| *** hemanthm|lunch is now known as hemanthm | 17:58 | |
| *** Apoorva_ has joined #openstack-nova | 17:59 | |
| *** zenoway has quit IRC | 18:00 | |
| *** cmartin has joined #openstack-nova | 18:00 | |
| *** zenoway has joined #openstack-nova | 18:01 | |
| *** prometheanfire has joined #openstack-nova | 18:01 | |
| prometheanfire | Data migrations for PciDevice are not safe, likely because not all services that access the DB directly are updated to the latest version | 18:01 |
| *** lpetrut has joined #openstack-nova | 18:01 | |
| prometheanfire | I'm not sure why that is showing up when everything is updated | 18:02 |
| *** ihrachys has joined #openstack-nova | 18:02 | |
| *** Apoorva has quit IRC | 18:02 | |
| *** bvanhav has quit IRC | 18:03 | |
| *** yamahata has joined #openstack-nova | 18:04 | |
| dansmith | prometheanfire: it means you have some service records that are still old | 18:04 |
| *** moshele has quit IRC | 18:04 | |
| dansmith | prometheanfire: do you have some services marked as down? | 18:04 |
| *** mtanino has quit IRC | 18:04 | |
| *** zenoway has quit IRC | 18:05 | |
| prometheanfire | nope | 18:05 |
| prometheanfire | checked that | 18:06 |
| prometheanfire | none that are missing either | 18:06 |
| *** gouthamr has joined #openstack-nova | 18:06 | |
| dansmith | prometheanfire: can you look at the versions in the services table? | 18:07 |
| prometheanfire | ya, gimme a sec | 18:07 |
| openstackgerrit | Divya K Konoor proposed openstack/nova-specs: Allow admin users to pass key_name corresponding to another user and deploy a VM successfully https://review.openstack.org/382588 | 18:09 |
| prometheanfire | I do have two services not at version 15, one is nova-network, which is deleted and the other is nova-compute on one of my upgraded nodes | 18:10 |
| dansmith | so that's why | 18:10 |
| prometheanfire | ya | 18:10 |
| prometheanfire | I don't know what I need to do to correct nova-compute on the upgraded node | 18:10 |
| prometheanfire | simply starting the service didn't work? | 18:11 |
| dansmith | prometheanfire: starting the service will do it | 18:11 |
| *** slaweq_ has quit IRC | 18:11 | |
| prometheanfire | well, it's started and running | 18:11 |
| prometheanfire | and still the old version | 18:11 |
| * prometheanfire goes to figure out why the update package didn't install | 18:12 | |
| dansmith | :) | 18:12 |
| *** ssl329 has quit IRC | 18:12 | |
| *** tblakes has joined #openstack-nova | 18:13 | |
| dansmith | prometheanfire: the deleted one *should* be ignored. If not, please do let me know as that'd be a bug | 18:13 |
| prometheanfire | k | 18:14 |
| * prometheanfire probably has the oldest postgres backed cloud, so, useful for testing long term cleanup and stuff | 18:15 | |
| prometheanfire | may not be the oldest, but grizzly at least | 18:15 |
| *** slaweq_ has joined #openstack-nova | 18:15 | |
| openstackgerrit | Divya K Konoor proposed openstack/nova-specs: Allow admin users to pass key_name corresponding to another user and deploy a VM successfully https://review.openstack.org/382588 | 18:16 |
| *** thorst is now known as thorst_afk | 18:17 | |
| *** ducttape_ has joined #openstack-nova | 18:18 | |
| mriedem | jaypipes: you might like to read #openstack-meeting right now | 18:19 |
| mriedem | dansmith: ^ | 18:19 |
| openstackgerrit | Divya K Konoor proposed openstack/nova-specs: Allow admin users to pass key_name corresponding to another user and deploy a VM successfully https://review.openstack.org/382588 | 18:21 |
| *** salv-orlando has quit IRC | 18:22 | |
| *** bvanhav has joined #openstack-nova | 18:22 | |
| *** moshele has joined #openstack-nova | 18:22 | |
| *** slaweq_ has quit IRC | 18:23 | |
| *** Matti_ has quit IRC | 18:23 | |
| *** david-lyle has quit IRC | 18:25 | |
| openstackgerrit | Vasyl Saienko proposed openstack/nova: Testing DROP SERVICE_HOST patch https://review.openstack.org/343569 | 18:29 |
| cburgess | What is gluon? | 18:29 |
| raj_singh | cburgess: https://wiki.openstack.org/wiki/Gluon | 18:30 |
| openstackgerrit | Chris Dent proposed openstack/nova: A demo script for updating resource inventory https://review.openstack.org/382613 | 18:30 |
| mriedem | cburgess: also https://www.openstack.org/videos/video/gluon-an-enabler-for-nfv | 18:30 |
| cburgess | thanks | 18:31 |
| *** campee has joined #openstack-nova | 18:31 | |
| dansmith | cburgess: it's like when you have two things that aren't compatible or related, but you glue one onto the other via abusing some interface of one to accomplish your goal | 18:33 |
| dansmith | think of it as "glue-on" and it makes more sense | 18:34 |
| * dansmith runs | 18:34 | |
| cburgess | dansmith You really are grumpy old man. :) | 18:34 |
| dansmith | heh | 18:34 |
| *** dikonoor has quit IRC | 18:34 | |
| *** code-R has joined #openstack-nova | 18:35 | |
| *** annegentle has joined #openstack-nova | 18:36 | |
| auggy | i'm amazed it isn't written in perl! | 18:37 |
| *** code-R_ has joined #openstack-nova | 18:38 | |
| cburgess | Ouch burn | 18:39 |
| auggy | i didn't mean it as a burn! | 18:39 |
| * auggy <3's perl | 18:40 | |
| auggy | i've written a lot of code to "glue" api's together in perl, it's pretty good at that | 18:40 |
| cburgess | Yeah I've done that too | 18:41 |
| cburgess | I wrote a whole CMS in perl once. | 18:41 |
| auggy | omg ME TOO | 18:41 |
| cburgess | Well I worked on one that was written in perl. | 18:41 |
| *** code-R has quit IRC | 18:41 | |
| cburgess | I didn't actually write it. | 18:41 |
| auggy | mine glued together oracle, filemaker pro, and mysql | 18:41 |
| cburgess | rmk did | 18:41 |
| auggy | that was a thing for awhile i think, people wanted to roll their own CMS | 18:42 |
| *** TravT has quit IRC | 18:43 | |
| auggy | oh whoops i was thinking CRM | 18:43 |
| jlk | CMS, CRM, CMDB, they're all mostly the same.. | 18:43 |
| auggy | that's what i did in perl | 18:43 |
| auggy | hahaha | 18:43 |
| auggy | CMDB is just horrible | 18:43 |
| cdent | Sometimes I miss perl, but then I don't. | 18:45 |
| *** claudiub has quit IRC | 18:45 | |
| *** thorst_afk is now known as thorst_ | 18:45 | |
| *** narayrak has joined #openstack-nova | 18:46 | |
| *** eglynn has quit IRC | 18:49 | |
| *** narayrak has quit IRC | 18:51 | |
| *** ducttape_ has quit IRC | 18:51 | |
| *** narayrak has joined #openstack-nova | 18:51 | |
| *** Sukhdev has joined #openstack-nova | 18:51 | |
| *** ducttape_ has joined #openstack-nova | 18:52 | |
| *** haplo37_ has quit IRC | 18:53 | |
| *** annegentle has quit IRC | 18:53 | |
| *** haplo37_ has joined #openstack-nova | 18:56 | |
| *** code-R_ has quit IRC | 18:56 | |
| *** code-R has joined #openstack-nova | 18:56 | |
| *** Apoorva_ has quit IRC | 18:58 | |
| *** Apoorva has joined #openstack-nova | 18:59 | |
| *** TravT has joined #openstack-nova | 19:01 | |
| *** matrohon has joined #openstack-nova | 19:02 | |
| *** TravT has quit IRC | 19:03 | |
| *** ijw has quit IRC | 19:03 | |
| *** marst has joined #openstack-nova | 19:04 | |
| *** TravT has joined #openstack-nova | 19:04 | |
| bauzas | dansmith: mriedem: so kinda weirdo bug https://bugs.launchpad.net/nova/+bug/1630161 | 19:06 |
| openstack | Launchpad bug 1630161 in python-novaclient "nova image-list is deprecated, but it should work even now" [Undecided,New] | 19:06 |
| *** raunak has quit IRC | 19:07 | |
| mriedem | they have to use <2.36 | 19:08 |
| *** lamt has quit IRC | 19:08 | |
| *** aleph1 has joined #openstack-nova | 19:08 | |
| *** cdent has quit IRC | 19:09 | |
| mriedem | bauzas: i've marked it invalid | 19:09 |
| bauzas | k | 19:09 |
| *** Matti_ has joined #openstack-nova | 19:09 | |
| prometheanfire | dansmith: looks like it's a bug | 19:10 |
| dansmith | prometheanfire: meaning it's considering that deleted service record? | 19:11 |
| bauzas | mriedem: soooo, just tbc, we still support the proxy calls but only if the user asks for <2.36, right? | 19:11 |
| mriedem | yes | 19:11 |
| prometheanfire | ya | 19:11 |
| dansmith | prometheanfire: have you confirmed by purging that one deleted record and seen it continue? | 19:12 |
| bauzas | mriedem: okay, I remember our midcycle, I was unknowing the implementation | 19:12 |
| prometheanfire | dansmith: no, I didn't know if it would be better to do that or to wait so I can test a fix | 19:12 |
| mriedem | bauzas: we didn't do all of the work to make the image proxy CLIs work because they were already deprecated | 19:12 |
| dansmith | prometheanfire: well, I was going to say grab a snapshot and then we can run against the snapshot, but.. | 19:12 |
| prometheanfire | lol | 19:12 |
| dansmith | prometheanfire: if you file a bug I'll write you up a fix and you can test it live if you want | 19:12 |
| prometheanfire | I'm fine with that | 19:12 |
| mriedem | whereas the network proxy CLIs weren't deprecated, so we bent over backward to make those fallback to 2.35 | 19:12 |
| prometheanfire | I have a before-migration snapshot | 19:13 |
| prometheanfire | I suppose I could make another, but reverting would really suck | 19:13 |
| mriedem | bauzas: https://github.com/openstack/python-novaclient/blob/6.0.0/novaclient/v2/images.py#L123 | 19:13 |
| dansmith | prometheanfire: not to revert, just for testing | 19:13 |
| prometheanfire | k | 19:13 |
| bauzas | mriedem: looking | 19:13 |
| dansmith | prometheanfire: but either way | 19:13 |
| mriedem | btw, i plan on doing a massive purge of novaclient stuff at some point when i get the time | 19:14 |
| dansmith | prometheanfire: this code is gone in ocata so this will be newton-only | 19:14 |
| *** bvanhav has quit IRC | 19:14 | |
| *** ihrachys has quit IRC | 19:15 | |
| *** jamesdenton has joined #openstack-nova | 19:15 | |
| bauzas | mriedem: okay, so to clarify, given nova image-list was already deprecated, it's fine to just have the exception, instead of asking 2.35, right? | 19:15 |
| bauzas | I suppose the warning can be confusing tho | 19:16 |
| prometheanfire | ya | 19:17 |
| prometheanfire | and formatting didn't work :( https://bugs.launchpad.net/nova/+bug/1630732 | 19:17 |
| openstack | Launchpad bug 1630732 in OpenStack Compute (nova) "online_data_migrations does not ignore deleted services" [Undecided,New] | 19:17 |
| mriedem | bauzas: yeah | 19:17 |
| mriedem | we're not going to fallback to passing 2.35 on the image CLIs because they were already deprecated | 19:18 |
| mriedem | bauzas: i agree it's slightly confusing | 19:18 |
| auggy | bauzas: yeah i saw that bug too and was wondering about it, thanks for bringing it up | 19:18 |
| mriedem | bauzas: but, it's also going away in 1 week | 19:18 |
| mriedem | and the image CLIs have been deprecated for awhiel | 19:18 |
| prometheanfire | dansmith: # nova-manage db online_data_migrations | 19:18 |
| prometheanfire | Running batches of 50 until complete | 19:18 |
| prometheanfire | 2016-10-05 14:11:00.578 2197 ERROR nova.db.sqlalchemy.api [req-60487805-6488-428e-b364-8eb6a1b620aa - - - - -] Data migrations for PciDevice are not safe, likely because not all services that access the DB directly are updated to the latest version | 19:18 |
| mriedem | so my sympathy is real low | 19:18 |
| prometheanfire | # select * from services; | 19:18 |
| prometheanfire | created_at | updated_at | deleted_at | id | host | binary | topic | report_count | disabled | deleted | disabled_reason | last_seen_up | forced_down | version | 19:18 |
| prometheanfire | ----------------------------+----------------------------+----------------------------+-- --+-----------------------------+--------------------+-------------+--------------+----------+---------+-----------------+----------------------------+-------------+--------- | 19:18 |
| mriedem | prometheanfire: paste buddy | 19:18 |
| prometheanfire | 2016-03-28 21:54:50.624939 | 2016-10-05 05:04:27.88604 | | 8 | 127.0.0.1 | nova-osapi_compute | | 0 | f | 0 | | | f | 15 | 19:18 |
| prometheanfire | 2014-04-01 02:25:19.568992 | 2016-10-05 19:10:36.775756 | | 1 | master.openstack.mthode.org | nova-scheduler | scheduler | 6508816 | f | 0 | | 2016-10-05 19:10:36.775144 | f | 15 | 19:18 |
| prometheanfire | 2014-09-22 22:48:42.410217 | 2016-10-05 19:10:38.957666 | | 5 | master.openstack.mthode.org | nova-cert | cert | 5880779 | f | 0 | | 2016-10-05 19:10:38.957042 | f | 15 | 19:19 |
| prometheanfire | 2014-04-01 03:24:01.305666 | 2014-04-08 16:17:50.835133 | 2015-08-03 06:30:12.744504 | 4 | master.openstack.mthode.org | nova-network | network | 65072 | f | 4 | | | f | 0 | 19:19 |
| jlk | gaaaah | 19:19 |
| prometheanfire | 2016-03-28 21:54:51.04119 | 2016-10-05 05:06:33.459861 | | 9 | 127.0.0.1 | nova-metadata | | 0 | f | 0 | | | f | 15 | 19:19 |
| prometheanfire | 2014-04-01 02:54:42.114856 | 2016-10-05 19:10:43.925853 | | 3 | master.openstack.mthode.org | nova-conductor | conductor | 6515405 | f | 0 | | 2016-10-05 19:10:43.924894 | f | 15 | 19:19 |
| prometheanfire | 2014-04-01 02:54:22.058152 | 2016-10-05 19:10:35.282541 | | 2 | master.openstack.mthode.org | nova-consoleauth | consoleauth | 6513588 | f | 0 | | 2016-10-05 19:10:35.281509 | f | 15 | 19:19 |
| prometheanfire | 2014-09-23 02:52:56.806792 | 2016-10-05 19:10:44.616719 | | 6 | node03.openstack.mthode.org | nova-compute | compute | 5329359 | f | 0 | | 2016-10-05 19:10:15.192567 | f | 15 | 19:19 |
| prometheanfire | sorry... | 19:19 |
| prometheanfire | yep | 19:19 |
| prometheanfire | dansmith: https://gist.github.com/prometheanfire/ac6081f6cccf1118dfd6cfc51b2193cb as an example | 19:19 |
| bauzas | mriedem: fair enough :) | 19:20 |
| dansmith | prometheanfire: file a bug please | 19:20 |
| prometheanfire | I did | 19:20 |
| prometheanfire | dansmith: https://bugs.launchpad.net/nova/+bug/1630732 | 19:21 |
| openstack | Launchpad bug 1630732 in OpenStack Compute (nova) "online_data_migrations does not ignore deleted services" [Undecided,New] | 19:21 |
| dansmith | prometheanfire: I'm trying to write a test to poke it and not seeing it, but just started | 19:21 |
| dansmith | prometheanfire: ah, thanks | 19:21 |
| prometheanfire | :D | 19:21 |
| *** moshele has quit IRC | 19:21 | |
| auggy | mriedem: bauzas: what's the "right" way we are recommending people to do instead of "nova image-list"? is it an osc call? | 19:21 |
| mriedem | auggy: it's in the warning message | 19:21 |
| bauzas | auggy: see the warning :) | 19:21 |
| auggy | ok kewl | 19:21 |
| auggy | haha yeah i should read more ;D | 19:21 |
| auggy | thanks :D | 19:22 |
| dansmith | ah, I see | 19:22 |
| *** david-lyle has joined #openstack-nova | 19:23 | |
| *** cmartin has quit IRC | 19:27 | |
| *** paul-carlton2 has quit IRC | 19:28 | |
| *** code-R has quit IRC | 19:29 | |
| *** Hosam has joined #openstack-nova | 19:31 | |
| *** Hosam_ has joined #openstack-nova | 19:32 | |
| *** nic has left #openstack-nova | 19:33 | |
| *** nic has joined #openstack-nova | 19:33 | |
| kashyap | mriedem: Okay, I have been up for the last hour or so chatting w/ upstream QEMU folks. After all of which, the conclusion is: libvirt should be taught to mangle the cur/end values (in the virDomainGetBlockJobInfo() API) when QEMU does not yet report a flag. | 19:33 |
| kashyap | Anyway, it might be too much info for here. I'll also write an email to openstack-dev list, about its Nova interaction, & my notes from the log. | 19:34 |
| kashyap | [Just to refresh your memory, this is re that _swap_volume() Big Swamp) | 19:34 |
| dansmith | prometheanfire: so, um... network services aren't considered in that check | 19:34 |
| dansmith | prometheanfire: only api and conductor | 19:34 |
| dansmith | ohhh | 19:35 |
| dansmith | cripes | 19:35 |
| *** Hosam has quit IRC | 19:36 | |
| *** ishant|away has quit IRC | 19:37 | |
| prometheanfire | dansmith: woo? | 19:37 |
| *** claudiub has joined #openstack-nova | 19:37 | |
| dansmith | prometheanfire: well, woo because I think I know why, but not woo because this has been broken since day one and deserves no wooing | 19:38 |
| *** ishant|away has joined #openstack-nova | 19:39 | |
| prometheanfire | heh | 19:40 |
| * artom woos dansmith | 19:40 | |
| mriedem | kashyap: ok | 19:41 |
| mriedem | you'll have to dumb it down for me | 19:41 |
| kashyap | mriedem: For Nova, the current workaround seems to be that we have to 'wait' a little longer. | 19:43 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova: Send vif-plugged to dest host if evacuating https://review.openstack.org/371048 | 19:43 |
| kashyap | However, if _know_ that QEMU is new enough to send an event, waiting for the event is more reliable than polling. | 19:43 |
| kashyap | Anyway, I'll write whatever I can to the list in the morning, or if I can finish it tonight. Once I file that libvirt bug | 19:43 |
| *** sdake has quit IRC | 19:45 | |
| *** sdake has joined #openstack-nova | 19:46 | |
| *** annegentle has joined #openstack-nova | 19:49 | |
| jaypipes | dansmith: would be cool to get your eyeballs on the https://review.openstack.org/#/q/topic:bp/custom-resource-classes series if you have some free time. | 19:49 |
| dansmith | prometheanfire: https://review.openstack.org/382639 | 19:50 |
| dansmith | prometheanfire: pretty stupid :/ | 19:50 |
| dansmith | jaypipes: will try but today is full | 19:51 |
| *** ducttape_ has quit IRC | 19:51 | |
| openstackgerrit | Andrew Laski proposed openstack/nova: Replace admin check with policy check in placement API https://review.openstack.org/382640 | 19:52 |
| alaski | dansmith: same mistake I made | 19:52 |
| dansmith | alaski: yeah :/ | 19:52 |
| *** krypto has quit IRC | 19:53 | |
| *** salv-orlando has joined #openstack-nova | 19:53 | |
| jaypipes | dansmith: not a prob, any time would be cool. | 19:53 |
| *** salv-orl_ has joined #openstack-nova | 19:54 | |
| *** cmartin has joined #openstack-nova | 19:54 | |
| *** Matti_ has quit IRC | 19:57 | |
| *** Apoorva has quit IRC | 19:58 | |
| *** salv-orlando has quit IRC | 19:58 | |
| mriedem | dansmith: are we considering that for a newton rc3? | 20:01 |
| mriedem | looks latent | 20:01 |
| dansmith | mriedem: it's really important because we removed that code in ocata, requiring people to have run it :/ | 20:02 |
| dansmith | so, I think it could go post-rc3 if you want, but.. it's a pretty big deal | 20:02 |
| *** lucasagomes is now known as lucas-afk | 20:02 | |
| prometheanfire | dansmith: testing in a sec | 20:03 |
| *** ijw has joined #openstack-nova | 20:03 | |
| dansmith | prometheanfire: thanks.. super happy you tested this so early :D | 20:03 |
| mriedem | dhellmann: dims: how terrible would it be if we had to do a newton rc3? | 20:03 |
| auggy | i have a dumb question, why didn't any of the gate jobs pick it up? do we have a possible coverage gap elsewhere too? | 20:04 |
| mriedem | auggy: we don't test pci devices in the gate | 20:04 |
| prometheanfire | dansmith: I try to package openstack about a week before actual release | 20:04 |
| dims | mriedem : what review/bug? just want to peek | 20:04 |
| prometheanfire | and I try to test what I package :D | 20:04 |
| auggy | mriedem: ok thx | 20:04 |
| mriedem | dims: https://review.openstack.org/#/c/382639/1 | 20:04 |
| mriedem | auggy: we have 3rd party ci for pci, but it doesn't test grenade | 20:04 |
| dansmith | auggy: mriedem well, we don't need pci devices for this, but it's also graceful so people can do it when they can, so we didn't gate on it fully | 20:04 |
| prometheanfire | dansmith: if it makes you feel better keystone had the same thing happen to them for mitaka | 20:04 |
| stevemar | it's true | 20:05 |
| dansmith | prometheanfire: no, it does not make me feel better that we're no better than keystone :P | 20:05 |
| dhellmann | mriedem : that looks like a bug blocking some upgrades? | 20:05 |
| mriedem | dhellmann: yes | 20:05 |
| mriedem | it's latent, but we dropped that code in ocata already so it's required to have this working before ocata | 20:05 |
| dansmith | dhellmann: anyone with pci devices wouldn't be able to upgrade from what is currently newton rc2 to ocata | 20:05 |
| *** TravT has quit IRC | 20:06 | |
| dansmith | dhellmann: we coudl backport and require people to be running the latest stable, which they should anyway | 20:06 |
| dansmith | and have had to do before | 20:06 |
| dhellmann | we're < 24 hrs from "final", and you can do a patch release monday. does that work? | 20:06 |
| dims | mriedem : did you just mark the bug as won't fix? https://bugs.launchpad.net/nova/+bug/1630732 | 20:06 |
| openstack | Launchpad bug 1630732 in OpenStack Compute (nova) newton "online_data_migrations does not ignore deleted services" [High,In progress] - Assigned to Dan Smith (danms) | 20:06 |
| * dhellmann looks at what else is in stable/newton for nova | 20:06 | |
| mriedem | note that people skip releases on upgrade now too... | 20:06 |
| mriedem | dims: won't fix for master | 20:06 |
| dims | ah | 20:06 |
| prometheanfire | stevemar: didn't know you were around :P | 20:07 |
| dansmith | mriedem: but they shouldn't, so.. I don't care :) | 20:07 |
| prometheanfire | dansmith: that fixed it | 20:07 |
| *** bvanhav has joined #openstack-nova | 20:07 | |
| dansmith | prometheanfire: sweet | 20:07 |
| stevemar | prometheanfire: i'm always around, and mostly round | 20:07 |
| mriedem | dansmith: yeah i know | 20:07 |
| prometheanfire | acked the review | 20:07 |
| dhellmann | mriedem : does this bug block anyone from upgrading to newton? | 20:07 |
| dansmith | dhellmann: no | 20:07 |
| stevemar | sounds like it could be fixed in a patch release? | 20:08 |
| dhellmann | ok, with my ultra-cautious hat on I suggest not doing an rc3 and releasing this as a patch early next week | 20:08 |
| dansmith | dhellmann: I agree | 20:08 |
| dims | dansmith : even if the online migration is stuck things would still work? | 20:08 |
| dhellmann | does anyone feel strongly that a patch release is going to cause trouble? | 20:08 |
| dansmith | dims: no | 20:08 |
| mriedem | i'm holding all other newton backports out, so we could slip this in a quick patch release | 20:08 |
| *** ducttape_ has joined #openstack-nova | 20:08 | |
| prometheanfire | 14.0.1? | 20:08 |
| dansmith | dims: if you have pci devices, ocata will refuse to upgrade if you missed this | 20:08 |
| dhellmann | mriedem : you could even include several items, if this isn't going to break until someone tries to go newton-ocata | 20:09 |
| *** ociuhandu has quit IRC | 20:09 | |
| dansmith | dims: like, the first db schema migration will bail and mock you via twitter | 20:09 |
| dims | ah! | 20:09 |
| dhellmann | wow, harsh :-) | 20:09 |
| dansmith | dhellmann: we don't fsck around | 20:09 |
| *** ijw has quit IRC | 20:10 | |
| dhellmann | indeed | 20:10 |
| dhellmann | it sounds like we've settled on a patch release, then? | 20:10 |
| prometheanfire | sgtm, I already have the code :P | 20:10 |
| dansmith | hah | 20:10 |
| *** matrohon has quit IRC | 20:11 | |
| *** Hosam has joined #openstack-nova | 20:11 | |
| dims | dansmith : dhellmann : is there a way to publish work arounds if any? | 20:12 |
| *** suresh12 has quit IRC | 20:12 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova: Send vif-plugged to dest host if evacuating https://review.openstack.org/371048 | 20:12 |
| dansmith | mriedem: actually, this is the one case I couldn't enforce because it's so fuzzy, | 20:13 |
| dansmith | mriedem: so they'll not hit a warning, they will just lose some parenting relationships in their pci devices | 20:14 |
| dansmith | because the pci code is so terrible | 20:14 |
| *** Hosam_ has quit IRC | 20:14 | |
| mriedem | dims: i dont think so | 20:14 |
| mriedem | prometheanfire: btw your gist in the bug report is a 404 | 20:15 |
| prometheanfire | ya, the docs are tied to the repo | 20:15 |
| prometheanfire | mriedem: I know, I removed it on purpose, supose I should have edited it... | 20:15 |
| mriedem | so can we get a paste of the failure? | 20:15 |
| mriedem | oh | 20:15 |
| *** lpetrut has quit IRC | 20:16 | |
| *** Apoorva has joined #openstack-nova | 20:16 | |
| diana | dansmith: I have a few questions about this spec if you have time: https://blueprints.launchpad.net/nova/+spec/paginate-simple-tenant-usage | 20:16 |
| prometheanfire | mriedem: sorry, don't like leaving info up is all, all it was was a select * from services; | 20:16 |
| prometheanfire | showing the deleted entry | 20:17 |
| dansmith | prometheanfire: we just need an example of the message | 20:17 |
| dansmith | not the dump | 20:17 |
| dansmith | which should be in the gatelogs, I can dig one up | 20:17 |
| dansmith | diana: kinda in the middle of some stuff atm | 20:17 |
| prometheanfire | 2016-10-05 00:08:30.989 14688 ERROR nova.db.sqlalchemy.api [req-47dc2ac8-3c1f-4841-bf44-7a082fc3a3ef - - - - -] Data migrations for PciDevice are not safe, likely because not all services that access the DB directly are updated to the latest version | 20:17 |
| diana | dansmith: okie dokie, I'll circle back tomorrow. thx | 20:17 |
| prometheanfire | it's just the one line | 20:17 |
| dansmith | prometheanfire: yeah that | 20:18 |
| dansmith | mriedem: ^ | 20:18 |
| *** Pavo has joined #openstack-nova | 20:19 | |
| openstackgerrit | Andrew Laski proposed openstack/nova: Replace admin check with policy check in placement API https://review.openstack.org/382640 | 20:21 |
| *** raunak has joined #openstack-nova | 20:23 | |
| *** Pavo has quit IRC | 20:27 | |
| *** Pavo has joined #openstack-nova | 20:27 | |
| *** ociuhandu has joined #openstack-nova | 20:28 | |
| mriedem | got it | 20:29 |
| *** TravT has joined #openstack-nova | 20:31 | |
| *** TravT has quit IRC | 20:34 | |
| *** penick has joined #openstack-nova | 20:35 | |
| *** cmartin has quit IRC | 20:36 | |
| *** bvanhav has quit IRC | 20:36 | |
| *** jichen has quit IRC | 20:39 | |
| mriedem | mdbooth: will you still be around in barcelona on friday at 11am? | 20:39 |
| *** peter-hamilton has quit IRC | 20:40 | |
| *** devananda is now known as devananda|afk | 20:43 | |
| mriedem | mikal: you'll be in barcelona right? | 20:47 |
| *** suresh12 has joined #openstack-nova | 20:50 | |
| *** rtheis has quit IRC | 20:51 | |
| mriedem | mikal: because i put your name as chair for this session https://www.openstack.org/summit/barcelona-2016/summit-schedule/events/16936/nova-completing-vendordata-v2 | 20:51 |
| mikal | mriedem: yep, I will be | 20:52 |
| mikal | mriedem: Is there really enough content for an entire session in that? | 20:52 |
| auggy | mriedem: oh thanks for updating the data migrations pci bug, i wasn't sure how to point it at the right project... "Nominate for series"? is that the right clicky thing? | 20:54 |
| *** lamt has joined #openstack-nova | 20:55 | |
| sdague | mikal: the thinking was that that, plus the token expiration effort puts together a more general keystone / nova session | 20:56 |
| mriedem | auggy: yes | 20:56 |
| sdague | and it would be nice to close on the vendordata thing and get it landed early in the cycle, as it lets us gut hooks | 20:56 |
| mriedem | mikal: if it's all unicorns and rainbows we can end it early | 20:57 |
| mriedem | but we didn't come out of the nova/keystone session like that in austin | 20:57 |
| mikal | sdague: well, this cycle is only 14 seconds long, so what does "early" mean this time? | 20:57 |
| sdague | 13 | 20:57 |
| sdague | seconds | 20:57 |
| dansmith | heh | 20:57 |
| sdague | hop to it | 20:58 |
| sdague | typey typey | 20:58 |
| mriedem | early means by jesus's birthday | 20:58 |
| mriedem | happy birthday jesus, btw we're removing the ability to use your hooks | 20:58 |
| mikal | sdague: I don't think you understand how much work related napping I have scheduled this quarter... | 20:58 |
| mriedem | if we say that vendordata is a new project, and needs a PTL, we can trick tonyb into doing it probably | 21:00 |
| sdague | "it's a trap!" | 21:00 |
| prometheanfire | tonyb: don't do it | 21:01 |
| *** Matti_ has joined #openstack-nova | 21:02 | |
| *** ociuhandu has quit IRC | 21:04 | |
| *** ijw has joined #openstack-nova | 21:07 | |
| *** eharney has quit IRC | 21:12 | |
| *** ayoung has quit IRC | 21:12 | |
| *** ijw has quit IRC | 21:12 | |
| *** BobBall has joined #openstack-nova | 21:12 | |
| *** ociuhandu has joined #openstack-nova | 21:13 | |
| *** BobBall is now known as Guest60855 | 21:13 | |
| *** whenry has joined #openstack-nova | 21:13 | |
| *** Guest60855 has quit IRC | 21:17 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova: WIP: Add error handling for delete-volume API https://review.openstack.org/382660 | 21:18 |
| *** lpetrut has joined #openstack-nova | 21:21 | |
| *** ociuhandu has quit IRC | 21:22 | |
| *** tblakes has quit IRC | 21:22 | |
| *** haplo37_ has quit IRC | 21:24 | |
| *** thorst_ has quit IRC | 21:26 | |
| *** haplo37_ has joined #openstack-nova | 21:26 | |
| *** thorst_ has joined #openstack-nova | 21:26 | |
| *** marst has quit IRC | 21:28 | |
| *** thorst_ has quit IRC | 21:31 | |
| *** suresh12 has quit IRC | 21:33 | |
| dims | mikal : how many crowns for tonyb ? :) | 21:33 |
| *** suresh12 has joined #openstack-nova | 21:36 | |
| *** slaweq_ has joined #openstack-nova | 21:38 | |
| mriedem | i guess that depends on how many sweets he eats and how often he brushes his teeth | 21:38 |
| *** mriedem has quit IRC | 21:38 | |
| *** bnemec has quit IRC | 21:39 | |
| dims | hahaha good one | 21:39 |
| *** kaisers has quit IRC | 21:42 | |
| *** ducttape_ has quit IRC | 21:44 | |
| *** ducttape_ has joined #openstack-nova | 21:44 | |
| *** xyang1 has quit IRC | 21:44 | |
| openstackgerrit | Ed Leafe proposed openstack/nova: Changed the name of the standard resource classes https://review.openstack.org/382669 | 21:46 |
| *** akshai has quit IRC | 21:46 | |
| *** mtanino has joined #openstack-nova | 21:46 | |
| *** _ducttape_ has joined #openstack-nova | 21:47 | |
| *** narayrak has quit IRC | 21:48 | |
| *** marst has joined #openstack-nova | 21:48 | |
| *** ducttape_ has quit IRC | 21:49 | |
| *** suresh12 has quit IRC | 21:51 | |
| *** jheroux has quit IRC | 21:52 | |
| *** suresh12 has joined #openstack-nova | 21:52 | |
| *** _ducttape_ has quit IRC | 21:52 | |
| *** edmondsw has quit IRC | 21:53 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova: Send vif-plugged to dest host if evacuating https://review.openstack.org/371048 | 21:57 |
| *** lpetrut has quit IRC | 21:57 | |
| *** raunak has quit IRC | 21:58 | |
| *** lamt has quit IRC | 21:59 | |
| tonyb | moar PTLs! | 21:59 |
| tonyb | I need to be nova PTL and on the TC and then I can lord it over mikal ;P | 21:59 |
| *** penick has quit IRC | 22:01 | |
| *** BobBall has joined #openstack-nova | 22:01 | |
| *** BobBall is now known as Guest5533 | 22:01 | |
| *** lamt has joined #openstack-nova | 22:04 | |
| *** whenry has quit IRC | 22:04 | |
| *** Apoorva_ has joined #openstack-nova | 22:04 | |
| *** penick has joined #openstack-nova | 22:04 | |
| *** zul has quit IRC | 22:07 | |
| *** mdrabe has quit IRC | 22:07 | |
| *** rrecio has quit IRC | 22:08 | |
| *** Apoorva has quit IRC | 22:08 | |
| *** ijw has joined #openstack-nova | 22:08 | |
| *** gabor_antal_ has joined #openstack-nova | 22:09 | |
| *** gabor_antal has quit IRC | 22:09 | |
| *** Guest86252 has quit IRC | 22:11 | |
| *** jamesdenton has quit IRC | 22:12 | |
| *** ijw has quit IRC | 22:13 | |
| *** esberglu has quit IRC | 22:15 | |
| *** slaweq_ has quit IRC | 22:16 | |
| *** vishwanathj has quit IRC | 22:18 | |
| *** Guest5533 has quit IRC | 22:18 | |
| *** TravT has joined #openstack-nova | 22:19 | |
| *** raunak has joined #openstack-nova | 22:19 | |
| *** gabor_antal_ has quit IRC | 22:19 | |
| *** gabor_antal has joined #openstack-nova | 22:21 | |
| *** suresh12 has quit IRC | 22:23 | |
| *** kaisers has joined #openstack-nova | 22:24 | |
| *** Pavo has quit IRC | 22:27 | |
| *** Pavo has joined #openstack-nova | 22:28 | |
| *** TravT has quit IRC | 22:28 | |
| *** kaisers has quit IRC | 22:28 | |
| *** TravT has joined #openstack-nova | 22:29 | |
| *** suresh12 has joined #openstack-nova | 22:32 | |
| *** burgerk has quit IRC | 22:33 | |
| *** BobBall_ has joined #openstack-nova | 22:33 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack/nova: Add error handling for delete-volume API https://review.openstack.org/382660 | 22:39 |
| *** BobBall_ has quit IRC | 22:40 | |
| *** gouthamr has quit IRC | 22:41 | |
| *** hamzy has quit IRC | 22:42 | |
| *** mlavalle has quit IRC | 22:44 | |
| *** lamt has quit IRC | 22:44 | |
| *** esberglu has joined #openstack-nova | 22:48 | |
| *** esberglu has quit IRC | 22:48 | |
| *** Pavo has quit IRC | 22:49 | |
| *** BobBall_ has joined #openstack-nova | 22:49 | |
| *** aleph1 has quit IRC | 22:50 | |
| *** Pavo has joined #openstack-nova | 22:51 | |
| *** BobBall_ has quit IRC | 22:54 | |
| *** ducttape_ has joined #openstack-nova | 22:55 | |
| *** yamahata has quit IRC | 22:57 | |
| prometheanfire | tonyb: I'd vote for that | 22:57 |
| tonyb | prometheanfire: ;P | 22:57 |
| *** ducttape_ has quit IRC | 23:00 | |
| *** ducttape_ has joined #openstack-nova | 23:01 | |
| *** coreywright has quit IRC | 23:02 | |
| *** diogogmt has quit IRC | 23:05 | |
| *** yamahata has joined #openstack-nova | 23:05 | |
| *** Apoorva_ has quit IRC | 23:06 | |
| *** Apoorva has joined #openstack-nova | 23:07 | |
| *** salv-orl_ has quit IRC | 23:07 | |
| *** vladikr has quit IRC | 23:08 | |
| *** coreywright has joined #openstack-nova | 23:08 | |
| *** jamielennox is now known as jamielennox|away | 23:09 | |
| *** ijw has joined #openstack-nova | 23:10 | |
| *** penick has quit IRC | 23:15 | |
| *** aleph1 has joined #openstack-nova | 23:15 | |
| *** ijw has quit IRC | 23:15 | |
| *** sdague has quit IRC | 23:17 | |
| *** Sukhdev has quit IRC | 23:17 | |
| *** pradk has quit IRC | 23:18 | |
| *** Sukhdev has joined #openstack-nova | 23:19 | |
| *** markvoelker has quit IRC | 23:22 | |
| *** lamt has joined #openstack-nova | 23:24 | |
| *** logan- has quit IRC | 23:28 | |
| *** TravT has quit IRC | 23:32 | |
| *** jamesdenton has joined #openstack-nova | 23:35 | |
| *** TravT has joined #openstack-nova | 23:35 | |
| *** TravT has quit IRC | 23:36 | |
| *** logan- has joined #openstack-nova | 23:37 | |
| *** TravT has joined #openstack-nova | 23:40 | |
| *** hamzy has joined #openstack-nova | 23:44 | |
| *** Swami has quit IRC | 23:46 | |
| *** david-lyle has quit IRC | 23:47 | |
| *** nic has quit IRC | 23:48 | |
| *** sdake has quit IRC | 23:48 | |
| *** mriedem has joined #openstack-nova | 23:50 | |
| *** gouthamr has joined #openstack-nova | 23:57 | |
| *** jamesdenton has quit IRC | 23:58 | |
| *** jamesden_ has joined #openstack-nova | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!