| *** bbowen has joined #openstack-nova | 00:00 | |
| *** slaweq has joined #openstack-nova | 00:10 | |
| *** slaweq has quit IRC | 00:15 | |
| *** ociuhandu has joined #openstack-nova | 00:19 | |
| *** slaweq has joined #openstack-nova | 00:21 | |
| *** tosky has quit IRC | 00:26 | |
| *** ociuhandu has quit IRC | 00:26 | |
| *** spsurya has joined #openstack-nova | 01:28 | |
| *** ociuhandu has joined #openstack-nova | 01:29 | |
| *** ociuhandu has quit IRC | 01:34 | |
| *** dave-mccowan has joined #openstack-nova | 01:46 | |
| *** ociuhandu has joined #openstack-nova | 01:59 | |
| *** chenhaw has joined #openstack-nova | 02:00 | |
| *** slaweq has quit IRC | 02:02 | |
| *** ociuhandu has quit IRC | 02:03 | |
| *** ociuhandu has joined #openstack-nova | 02:04 | |
| *** chenhaw has quit IRC | 02:06 | |
| *** ociuhandu has quit IRC | 02:09 | |
| *** chenhaw has joined #openstack-nova | 02:21 | |
| *** yaawang has quit IRC | 02:24 | |
| *** yaawang has joined #openstack-nova | 02:33 | |
| *** ociuhandu has joined #openstack-nova | 02:51 | |
| *** ociuhandu has quit IRC | 02:53 | |
| *** ociuhandu has joined #openstack-nova | 02:55 | |
| *** ociuhandu has quit IRC | 03:03 | |
| *** mkrai has joined #openstack-nova | 03:26 | |
| *** ricolin has joined #openstack-nova | 03:47 | |
| *** davee_ has joined #openstack-nova | 04:03 | |
| *** ociuhandu has joined #openstack-nova | 04:03 | |
| *** davee___ has quit IRC | 04:04 | |
| *** bhagyashris has joined #openstack-nova | 04:04 | |
| *** ociuhandu has quit IRC | 04:08 | |
| *** udesale has joined #openstack-nova | 04:37 | |
| *** dave-mccowan has quit IRC | 04:40 | |
| *** awalende has joined #openstack-nova | 05:18 | |
| *** lbragstad_ has joined #openstack-nova | 05:21 | |
| *** awalende has quit IRC | 05:23 | |
| *** lbragstad has quit IRC | 05:24 | |
| *** ociuhandu has joined #openstack-nova | 05:30 | |
| *** ociuhandu has quit IRC | 05:36 | |
| *** mkrai has quit IRC | 05:38 | |
| *** links has joined #openstack-nova | 05:39 | |
| *** mkrai has joined #openstack-nova | 05:41 | |
| *** Luzi has joined #openstack-nova | 06:06 | |
| *** huaqiang has quit IRC | 06:27 | |
| *** avolkov has joined #openstack-nova | 06:29 | |
| *** pcaruana has joined #openstack-nova | 06:32 | |
| *** mkrai has quit IRC | 06:38 | |
| *** ralonsoh has joined #openstack-nova | 06:51 | |
| *** belmoreira has joined #openstack-nova | 06:54 | |
| *** ircuser-1 has quit IRC | 06:55 | |
| *** belmoreira has quit IRC | 06:56 | |
| *** dpawlik has joined #openstack-nova | 07:00 | |
| *** ociuhandu has joined #openstack-nova | 07:01 | |
| *** dpawlik has quit IRC | 07:05 | |
| *** ociuhandu has quit IRC | 07:05 | |
| *** slaweq has joined #openstack-nova | 07:08 | |
| *** abaindur has quit IRC | 07:08 | |
| *** tkajinam has quit IRC | 07:11 | |
| *** tkajinam has joined #openstack-nova | 07:12 | |
| *** dpawlik has joined #openstack-nova | 07:15 | |
| *** mkrai has joined #openstack-nova | 07:23 | |
| *** slaweq has quit IRC | 07:37 | |
| *** slaweq has joined #openstack-nova | 07:40 | |
| *** abaindur has joined #openstack-nova | 07:43 | |
| *** abaindur has quit IRC | 07:53 | |
| *** damien_r has joined #openstack-nova | 08:04 | |
| *** jangutter has joined #openstack-nova | 08:09 | |
| *** awalende has joined #openstack-nova | 08:14 | |
| *** tesseract has joined #openstack-nova | 08:14 | |
| *** awalende has quit IRC | 08:18 | |
| *** tkajinam has quit IRC | 08:25 | |
| *** tkajinam has joined #openstack-nova | 08:26 | |
| *** tosky has joined #openstack-nova | 08:31 | |
| *** tssurya has joined #openstack-nova | 08:31 | |
| *** rpittau|afk is now known as rpittau | 08:31 | |
| *** ccamacho has joined #openstack-nova | 08:34 | |
| *** yedongcan has joined #openstack-nova | 08:34 | |
| *** tkajinam has quit IRC | 08:39 | |
| *** maciejjozefczyk has quit IRC | 08:46 | |
| *** ttx has quit IRC | 08:50 | |
| *** ttx has joined #openstack-nova | 08:50 | |
| *** dtantsur|afk is now known as dtantsur | 08:52 | |
| *** ociuhandu has joined #openstack-nova | 08:55 | |
| *** ociuhandu has quit IRC | 08:56 | |
| *** ociuhandu has joined #openstack-nova | 08:57 | |
| *** trident has quit IRC | 09:07 | |
| *** trident has joined #openstack-nova | 09:09 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Use virDomainBlockCopy to swap volumes with >= 5.10.0 https://review.opendev.org/696834 | 09:10 |
|---|---|---|
| *** ganso has quit IRC | 09:23 | |
| *** ganso has joined #openstack-nova | 09:23 | |
| *** ganso has quit IRC | 09:24 | |
| *** ganso has joined #openstack-nova | 09:24 | |
| *** derekh has joined #openstack-nova | 09:35 | |
| *** damien_r has quit IRC | 09:38 | |
| *** yedongcan has quit IRC | 09:49 | |
| *** damien_r has joined #openstack-nova | 09:50 | |
| *** awalende has joined #openstack-nova | 09:52 | |
| *** yaawang has quit IRC | 09:53 | |
| openstackgerrit | Dirk Mueller proposed openstack/nova master: Remove rootwrap filters for nova network https://review.opendev.org/696844 | 09:59 |
| *** ociuhandu has quit IRC | 10:04 | |
| *** ivve has joined #openstack-nova | 10:08 | |
| openstackgerrit | Guo Jingyu proposed openstack/nova master: WIP:Drop unreliable host_aggregate_map https://review.opendev.org/696851 | 10:11 |
| *** yaawang has joined #openstack-nova | 10:12 | |
| *** lpetrut has joined #openstack-nova | 10:13 | |
| *** mkrai has quit IRC | 10:24 | |
| *** tosky_ has joined #openstack-nova | 10:26 | |
| *** ociuhandu has joined #openstack-nova | 10:28 | |
| *** tosky has quit IRC | 10:28 | |
| openstackgerrit | Guo Jingyu proposed openstack/nova-specs master: Proposal for a safer noVNC console with password authentication https://review.opendev.org/623120 | 10:36 |
| *** tosky_ is now known as tosky | 10:38 | |
| *** mkrai has joined #openstack-nova | 10:45 | |
| *** ralonsoh has quit IRC | 11:01 | |
| *** ralonsoh has joined #openstack-nova | 11:03 | |
| *** mkrai has quit IRC | 11:05 | |
| *** chenhaw has quit IRC | 11:06 | |
| *** mkrai has joined #openstack-nova | 11:09 | |
| stephenfin | sean-k-mooney: Hit https://review.opendev.org/#/c/689861/ and it's predecessor, FYI. Sorry for the delay | 11:12 |
| *** ociuhandu has quit IRC | 11:16 | |
| *** ociuhandu has joined #openstack-nova | 11:17 | |
| *** ociuhandu has quit IRC | 11:26 | |
| *** ociuhandu has joined #openstack-nova | 11:26 | |
| *** ivve has quit IRC | 11:26 | |
| *** tssurya has quit IRC | 11:29 | |
| *** mkrai has quit IRC | 11:32 | |
| *** ociuhandu has quit IRC | 11:43 | |
| *** ociuhandu has joined #openstack-nova | 11:45 | |
| sean-k-mooney | ah cool ill address the comments on that today | 11:46 |
| sean-k-mooney | thanks | 11:46 |
| sean-k-mooney | stephenfin: by the way you realise that i did the test refactor before you started yours right. and that i cant depend you your refactor because i need to backport this which is why i orginally asked you to base your refactor on top of that series | 11:48 |
| sean-k-mooney | i can drop the new base class and go back to the other _wait_for_state_chage but i can use any of your refactoring or ill have to undo all of it when i backport | 11:49 |
| *** ociuhandu has quit IRC | 11:50 | |
| *** dtantsur is now known as dtantsur|afk | 11:54 | |
| *** mkrai has joined #openstack-nova | 11:57 | |
| sean-k-mooney | * i can't use | 12:00 |
| *** martinkennelly has joined #openstack-nova | 12:16 | |
| *** ivve has joined #openstack-nova | 12:19 | |
| *** ociuhandu has joined #openstack-nova | 12:24 | |
| *** ociuhandu has quit IRC | 12:29 | |
| stephenfin | sean-k-mooney: Yup, but I don't think it's a massive deal that '_wait_for_state_change' behaves oddly on stable branches, given that there's plenty of tests using it | 12:31 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: Revert "nova shared storage: rbd is always shared storage" https://review.opendev.org/682523 | 12:33 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Rename _is_storage_shared_with to _is_path_shared_with https://review.opendev.org/693337 | 12:33 |
| *** udesale has quit IRC | 12:36 | |
| *** udesale has joined #openstack-nova | 12:37 | |
| sean-k-mooney | stephenfin: ya im using the fake notifier too so i can also use the notifer to wait for the completeion events too so its fine ill rework them later today | 12:39 |
| sean-k-mooney | gibi: johnthetubaguy do either of ye feel comfrotable reviewing pci code | 12:49 |
| stephenfin | sean-k-mooney: Can I say no to that? ^ :P | 12:59 |
| *** ygk_12345 has joined #openstack-nova | 13:02 | |
| sean-k-mooney | you can but you already reviewed the patch and said yes :) | 13:07 |
| sean-k-mooney | also i said comfortabley not do you like reviewing pci stuff :) | 13:07 |
| *** nweinber has joined #openstack-nova | 13:08 | |
| sean-k-mooney | i also feel like i missed a chance to use "no take backsies" but that might date me a little | 13:11 |
| *** ccamacho has quit IRC | 13:12 | |
| *** ccamacho has joined #openstack-nova | 13:12 | |
| *** efried_pto is now known as efried | 13:13 | |
| *** ccamacho has quit IRC | 13:13 | |
| *** ccamacho has joined #openstack-nova | 13:13 | |
| openstackgerrit | Eric Fried proposed openstack/os-traits master: Add 'TYPE_PLOOP' image type. https://review.opendev.org/696435 | 13:14 |
| efried | stephenfin: I fixed that link ^ -- do you want to confirm real quick before I send it? | 13:15 |
| *** dave-mccowan has joined #openstack-nova | 13:16 | |
| *** dave-mccowan has quit IRC | 13:21 | |
| sean-k-mooney | the ploop image type is only used by parallel/openvz containers rights | 13:24 |
| efried | No idea, but it's on the list, so we should have a trait for it. | 13:25 |
| efried | (and by "no idea", I mean your sentence has at least three words I don't know) | 13:25 |
| sean-k-mooney | ya it is https://wiki.openvz.org/Ploop/format | 13:25 |
| sean-k-mooney | and yes we should have a trait for it | 13:26 |
| efried | sounds like a euphemism for a particular type of turd to me <shrug> | 13:26 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Process requested_resources in ResourceRequest init https://review.opendev.org/696354 | 13:28 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Reusable RequestGroup.add_{resource|trait} https://review.opendev.org/696380 | 13:28 |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Use string suffixes and provider mappings https://review.opendev.org/696418 | 13:28 |
| efried | stephenfin, gibi: fixed the unit test on the bottom one ^ | 13:28 |
| *** ociuhandu has joined #openstack-nova | 13:30 | |
| openstackgerrit | Merged openstack/os-traits master: Add 'TYPE_PLOOP' image type. https://review.opendev.org/696435 | 13:34 |
| *** ygk_12345 has quit IRC | 13:34 | |
| *** mkrai has quit IRC | 13:41 | |
| *** mkrai has joined #openstack-nova | 13:41 | |
| *** ociuhandu has quit IRC | 13:55 | |
| *** ociuhandu has joined #openstack-nova | 13:55 | |
| *** jroll has quit IRC | 13:57 | |
| *** jroll has joined #openstack-nova | 13:59 | |
| *** tkajinam has joined #openstack-nova | 14:06 | |
| *** ociuhandu has quit IRC | 14:10 | |
| *** mkrai has quit IRC | 14:10 | |
| *** haleyb|away is now known as haleyb | 14:17 | |
| *** bhagyashris has quit IRC | 14:20 | |
| *** ociuhandu has joined #openstack-nova | 14:22 | |
| *** lbragstad_ is now known as lbragstad | 14:22 | |
| *** eharney has joined #openstack-nova | 14:23 | |
| gibi | efried: I'm +2 on the bottom patch | 14:23 |
| efried | thanks gibi | 14:23 |
| gibi | thanks for starting this work | 14:23 |
| gibi | you are solving some of my todos :) | 14:23 |
| efried | does the strategy seem sane to you wrt the requester_id <=> group ID? | 14:23 |
| gibi | efried: yeah, that step is what I had in my mind | 14:24 |
| efried | great | 14:24 |
| gibi | efried: I'm a bit affraid the replacement of fill_provider_mapping with the mapping from the a_c response. Theortically it should work I just hope there will not be too much complication | 14:25 |
| efried | we'll see :) | 14:25 |
| efried | gibi: I think the most complicated bit might be revert_resize | 14:26 |
| gibi | yes, that is the type of surgery that better do that plan | 14:26 |
| efried | By that time the original mappings might be long gone | 14:26 |
| efried | ...unless we persist them in the RequestSpec...? | 14:26 |
| gibi | efried: in case of revert_resize we can either start using multiple port binding for resize which is hard but nice, or persist the mapping somewhere | 14:26 |
| gibi | if we go to the later then I would persist it to the Migration object | 14:27 |
| gibi | as the migration object has the proper lifecycle for this type of data | 14:27 |
| gibi | the RequestSpec lives longer | 14:27 |
| *** munimeha1 has joined #openstack-nova | 14:35 | |
| *** tbachman has joined #openstack-nova | 14:36 | |
| *** awalende_ has joined #openstack-nova | 14:40 | |
| sean-k-mooney | gibi: i would like to move to useing multiple port bindings for resize for what its worth | 14:41 |
| sean-k-mooney | gibi: we will be using it for cross cell resize | 14:41 |
| sean-k-mooney | so i think it makes sense to just use it always for resize/cold migrate | 14:41 |
| *** awalende has quit IRC | 14:44 | |
| *** awalende has joined #openstack-nova | 14:44 | |
| *** awalende_ has quit IRC | 14:44 | |
| *** awalende has quit IRC | 14:45 | |
| sean-k-mooney | efried: gibi when ye have time could yet take a look at https://review.opendev.org/#/c/674072/ and maybe https://review.opendev.org/#/c/695118/ | 14:46 |
| sean-k-mooney | ill be going on vacation in 2 weeks but i have a 4 patches i really want to get merged if i can before i do those are two of them and im resping the other two now | 14:47 |
| *** mriedem has joined #openstack-nova | 14:48 | |
| efried | stephenfin, lyarwood: thoughts on bug tagging here? https://review.opendev.org/#/c/682594/ | 14:49 |
| efried | sean-k-mooney: that might be a bit out of my bailiwick, but I'll give it a try. | 14:51 |
| stephenfin | looking | 14:53 |
| lyarwood | efried: yeah that's fair, I can also address the nits before we merge this afternoon | 14:53 |
| efried | okay | 14:53 |
| lyarwood | efried: I think I was trying to avoid any suggestion that this was the only fix required here | 14:53 |
| stephenfin | Your call. I could go either way. I'm guessing it was left that way since mriedem has a better albeit not backportable fix | 14:53 |
| stephenfin | yup | 14:54 |
| efried | right, I'm looking at it from the perspective of stable, where if we don't Closes-Bug on this patch, there will never be a "fix". | 14:54 |
| efried | I guess if you really want, we could keep this one Related-Bug and switch it to Closes-Bug on the backports | 14:54 |
| efried | can't believe there's not a precedent for something like this. | 14:55 |
| *** mkrai has joined #openstack-nova | 14:55 | |
| lyarwood | Yeah not that I recall but I'd be okay with that from a stable point of view | 14:56 |
| lyarwood | related on master, closes on stable that is | 14:56 |
| sean-k-mooney | well does it actully fix the bug | 14:56 |
| sean-k-mooney | or just make it less bad | 14:56 |
| sean-k-mooney | ye are talking about https://bugs.launchpad.net/nova/+bug/1844296 right | 14:56 |
| openstack | Launchpad bug 1844296 in OpenStack Compute (nova) "Stale BDM records remain in the DB after n-api to n-cpu RPC timeouts during reserve_block_device_name" [Undecided,In progress] - Assigned to Lee Yarwood (lyarwood) | 14:56 |
| sean-k-mooney | and this patch https://review.opendev.org/#/c/682594/ | 14:57 |
| lyarwood | sean-k-mooney: yeah, it cleans up after one type of failure, it doesn't stop the failure from happening. | 14:57 |
| sean-k-mooney | ah i see | 14:57 |
| sean-k-mooney | so its treating the symtoms not the cause | 14:57 |
| lyarwood | sean-k-mooney: yeah and very specific symptoms at that. | 14:58 |
| *** tkajinam has quit IRC | 14:58 | |
| sean-k-mooney | you could make it partial-bug if you plan to try and fix it someother way on master in a follow up patch | 14:58 |
| lyarwood | ah yeah I forgot about partial | 14:59 |
| lyarwood | that might do | 14:59 |
| sean-k-mooney | related bug makes sense too but i can see why you might what cloeses bug for stabel if you dont think any complete fix woudl be backportable | 14:59 |
| sean-k-mooney | Partial-Bug i think is the most correct but that wont mark it as "fixed" on stable | 15:00 |
| mriedem | see my comments from PS4 | 15:00 |
| mriedem | 1. Can we sort out the various bugs I've mentioned to see which ones this fixes and then duplicate whatever is a duplicate and decide on one (probably the oldest) bug for the same issue. | 15:00 |
| mriedem | 2. I don't trust unit tests for this type of stuff since it involves (a) more than one service and (b) the database. Can we write a functional test to recreate the bug and then assert the fix resolves it? | 15:00 |
| mriedem | i believe i sorted out the bazillion related bugs here https://review.opendev.org/#/c/692940/ | 15:01 |
| lyarwood | gah, I'm not sure how I missed that, sorry mriedem | 15:02 |
| mriedem | probably distracted by all of the comments inline about races and locks and such that might mean this doesn't fix a bug | 15:03 |
| openstackgerrit | Dan Smith proposed openstack/nova master: Add a way to exit early from a wait_for_instance_event() https://review.opendev.org/695985 | 15:04 |
| mriedem | if you want i could write the functional test if you don't want to bother with that | 15:10 |
| mriedem | lyarwood: since the bdm.create is the last thing that happens on the compute, in what case would this cleanup catch and actually clean anything up? | 15:11 |
| mriedem | like just the split second that we actually created the record and then got a messaging timeout? | 15:11 |
| lyarwood | mriedem: happy to write the func test | 15:13 |
| lyarwood | mriedem: and yeah but that's asssuming that is a split second and n-cpu isn't stuck somehow | 15:13 |
| lyarwood | mriedem: as mdbooth suggested | 15:13 |
| lyarwood | mriedem: FWIW we have seen this downstream at least, so either way it's possible to hit and reasonable to cleanup | 15:14 |
| mriedem | sure, so if n-cpu is stuck on let's say getting the instance uuid lock, or doing something in the driver like _get_device_name_for_instance, we wouldn't create the bdm in that case anyway | 15:14 |
| mriedem | where/how was it triggered downstream? | 15:14 |
| *** ociuhandu has quit IRC | 15:14 | |
| mriedem | slow compute and the api timed out? | 15:14 |
| mriedem | because if so, the change to make reserve_block_device_name using the long_rpc_timeout will fix that | 15:15 |
| *** ociuhandu has joined #openstack-nova | 15:15 | |
| lyarwood | mriedem: https://bugzilla.redhat.com/show_bug.cgi?id=1752734 - I didn't get a root cause out of this as the env wasn't logging in DEBUG unfortunatley | 15:15 |
| openstack | bugzilla.redhat.com bug 1752734 in openstack-nova "Invalid bdm record remains when reserve_block_device_name rpc call times out" [High,On_dev] - Assigned to lyarwood | 15:15 |
| lyarwood | mriedem: but we did end up with stale BDM records in the DB after seeing timeouts | 15:16 |
| sean-k-mooney | lyarwood: ok so there is a customer case attached to that so its not a ci issue | 15:16 |
| mriedem | 3. nova-api again requests to nova-compute to do attaching operation 4. nova-compute updates bdm with retrieved connection info 5. if some error like timeout detected between 3-4, remove bdm record created at 2 | 15:17 |
| mriedem | so that's not the same thing | 15:17 |
| mriedem | that's (1) reserve_block_device_name worked and created a bdm and then (2) when the api cast to attach_volume something timed out and the bdm was orphaned | 15:18 |
| *** links has quit IRC | 15:18 | |
| lyarwood | mriedem: yeah I think that description is wrong, there's reserve_block_device_name failures documented in the next private comment of the bug | 15:19 |
| lyarwood | sean-k-mooney: ^ can confirm that, however I don't think we can share the actual logs here. | 15:20 |
| mriedem | even so, the change to make reserve_block_device_name using the long_rpc_timeout will fix that | 15:20 |
| mriedem | but i don't think that long rpc timeout goes back to queens | 15:20 |
| sean-k-mooney | am i was reading it and im not sure the comment need to be private. we cant share the raw logs but i do not see antynig in comment 1 that is really an issue | 15:21 |
| *** ociuhandu has quit IRC | 15:21 | |
| mriedem | if it's just a messagingtimeout i can understand that | 15:21 |
| sean-k-mooney | MessagingTimeout: Timed out waiting for a reply to message ID fa3d55d2d8c248ba82a13c940350ea9c | 15:21 |
| lyarwood | super secret volume uuids | 15:21 |
| lyarwood | ;) | 15:21 |
| sean-k-mooney | so the long rpc might fix it ya | 15:22 |
| mriedem | it's a 60 second timeout by default with a blocking api - if the compute is slow or we're waiting on the instance uuid lock (maybe because a periodic is doing something with the instance at the time of attach), we could reasonably timeout | 15:22 |
| mriedem | https://review.opendev.org/#/c/566696/ only goes back to rocky though, so you can't get the long_rpc_timeout fix to queens | 15:22 |
| mriedem | so no use in saying that's the backportable fix | 15:22 |
| *** dpawlik has quit IRC | 15:22 | |
| mriedem | anyway, it just seems like a really small window where the fix in the api would handle it since the bdm create is the last thing to happen - if we timeout before that we still orphan the bdm record | 15:23 |
| mriedem | and i know i'm contradicting my "what's the harm?" replies to mdbooth in the comments :) | 15:23 |
| mriedem | the better longer term fix is the wip i posted, but that's not backportable | 15:24 |
| lyarwood | yeah agreed on that part, we could backport long_rpc_timeout downstream *cough* | 15:24 |
| lyarwood | but yeah | 15:24 |
| mriedem | also your bug is definitely a duplicate of https://bugs.launchpad.net/nova/+bug/1425352 | 15:25 |
| openstack | Launchpad bug 1425352 in OpenStack Compute (nova) "A volume remains attached and cannot be detached after attaching it fails" [Low,Confirmed] | 15:25 |
| * sean-k-mooney may have been looking at the patch to see if we could | 15:25 | |
| sean-k-mooney | lyarwood: the issue would be the oslo.messaging min version bump | 15:25 |
| mriedem | sean-k-mooney: correct we're not backporting that to queens-em upstream | 15:26 |
| lyarwood | okay that's likely not an issue downstram | 15:27 |
| lyarwood | downstream* | 15:27 |
| sean-k-mooney | i dont know | 15:27 |
| sean-k-mooney | i would not nessisarliy be comfrotable with that | 15:27 |
| sean-k-mooney | but we could explore it as an option | 15:27 |
| sean-k-mooney | it would depend on what version of oslo.messaging we are actully using downstream | 15:28 |
| lyarwood | >= 5.29.0 | 15:28 |
| lyarwood | sean-k-mooney: when did this come in? | 15:28 |
| sean-k-mooney | 6.3.0 | 15:28 |
| lyarwood | ah | 15:29 |
| lyarwood | that would be an issue | 15:29 |
| sean-k-mooney | https://github.com/openstack/requirements/blob/stable/queens/upper-constraints.txt#L145 | 15:29 |
| sean-k-mooney | so queens was capped a 5.35.6 | 15:29 |
| lyarwood | yeah guess what we use downstream | 15:29 |
| sean-k-mooney | so 6.3.0 proably will break things | 15:29 |
| lyarwood | okay that's dream is dead | 15:29 |
| lyarwood | that* | 15:29 |
| sean-k-mooney | is it higher then the upper-constraints for queens? cause i hate when we do that | 15:30 |
| lyarwood | no it's not | 15:30 |
| sean-k-mooney | cool | 15:30 |
| mriedem | you could create your own workaround option as a timeout for this specific call that overrides the default rpc_response_timeout just for your queens-only change and drop it for long_rpc_timeout when you start using that for this call from upstream | 15:30 |
| *** ociuhandu has joined #openstack-nova | 15:32 | |
| sean-k-mooney | ya is there an advantage to not useing the same config option | 15:32 |
| sean-k-mooney | is it jsut to make it clear it applies to just this one call | 15:33 |
| mriedem | lyarwood: i left some comments in the patch to try and summarize irc discussion and duplicated the bug to the existing one for the same issue. i'd hold off on writing a functional test until more people weigh in on what the correct direction is here given backports and such. | 15:37 |
| *** artom has joined #openstack-nova | 15:37 | |
| gibi | sean-k-mooney: could you quickly respin https://review.opendev.org/#/c/695118/ to fix stephenfin nits in the conf doc and the release notes? The change looks good to me too but I guess you want to backpor it so a follow up won't help | 15:38 |
| *** Luzi has quit IRC | 15:38 | |
| *** ociuhandu has quit IRC | 15:38 | |
| sean-k-mooney | yep i can do that in 5 mins thanks for taking a look | 15:38 |
| *** ociuhandu has joined #openstack-nova | 15:39 | |
| gibi | sean-k-mooney: thanks. Ping me and I will +2 | 15:39 |
| mriedem | would be nice if any of the blizzard people would ack that change since they reported it | 15:39 |
| *** ociuhandu has quit IRC | 15:44 | |
| mriedem | stephenfin: tbc i'm waiting on vmware ci results on https://review.opendev.org/#/c/696503/ before moving it forward | 15:45 |
| mriedem | if that ci is busted (let's say we don't get results within the next 24 hours or something) then i'm cool with moving forward since if they can't maintain a working ci then we can't maintain their driver for them | 15:45 |
| stephenfin | coolness | 15:45 |
| *** ociuhandu has joined #openstack-nova | 15:48 | |
| *** dpawlik has joined #openstack-nova | 15:51 | |
| sean-k-mooney | ... the release notes tox env takes forever to run | 15:55 |
| *** dpawlik has quit IRC | 15:56 | |
| *** bhagyashris has joined #openstack-nova | 15:57 | |
| *** macz has joined #openstack-nova | 15:58 | |
| *** eharney has quit IRC | 15:59 | |
| openstackgerrit | sean mooney proposed openstack/nova master: add [libvirt]/max_queues config option https://review.opendev.org/695118 | 15:59 |
| sean-k-mooney | gibi: stephenfin ^ nits addressed no other changes | 15:59 |
| gibi | sean-k-mooney: looking... | 15:59 |
| *** ociuhandu has quit IRC | 16:01 | |
| gibi | eandersson: the bug is coming from blizzard ^^ could you check the bugfix from your perspective? | 16:01 |
| sean-k-mooney | i will be filing and fixing a related bug for vhost-user seperatly but that should cover the case in the current bug if not let me know. | 16:02 |
| sean-k-mooney | actully thinking about that i wonder if i should also add extra code to handel updating this on live migration? | 16:08 |
| sean-k-mooney | i dont think we have code for the queue lenghts on live migration so i kind of feel a seperate patch to handel both queue lenght and max queues would make sense since that will need object changes and wont be backportable where as this simple fix is | 16:11 |
| sean-k-mooney | gibi: stephenfin do you think that would be good to do as a followup? looking at the migrate data object we dont pass any of that info today | 16:13 |
| gibi | sean-k-mooney: do you say that during live migration the server will use the source host config regarding the number of queues? | 16:14 |
| sean-k-mooney | yes and rx/tx queue lenghts | 16:15 |
| *** ociuhandu has joined #openstack-nova | 16:15 | |
| sean-k-mooney | we have no support for updating this at all | 16:15 |
| sean-k-mooney | gibi: cold migrate will be fine and a hard reboot would fix it after a live migrate | 16:16 |
| sean-k-mooney | it would only be an issue if you went form a new host with a higher limit to an old host with a smaller one | 16:16 |
| sean-k-mooney | the bug in this case was we limit the queues to 8 eventhogh the kernel support 256 so in generally it wont be an issue | 16:17 |
| sean-k-mooney | but for correct ness we should be using the destination value | 16:17 |
| gibi | sean-k-mooney: yeah, I think that the fix to use the dest value make sense | 16:18 |
| sean-k-mooney | it should be a seperate patch however right | 16:18 |
| sean-k-mooney | so we can backport this one | 16:18 |
| sean-k-mooney | the fix for dest value would need an object change | 16:18 |
| *** ociuhandu has quit IRC | 16:19 | |
| sean-k-mooney | assuming livirt allows it at all. it might just say no an reject the live migration | 16:20 |
| *** mkrai has quit IRC | 16:20 | |
| sean-k-mooney | gibi: ill file a bug and we can figure out what to do with it later | 16:21 |
| gibi | sean-k-mooney: ack | 16:21 |
| *** mlavalle has joined #openstack-nova | 16:24 | |
| *** rouk has joined #openstack-nova | 16:26 | |
| *** ivve has quit IRC | 16:26 | |
| efried | stephenfin: can I get a quick re+A on https://review.opendev.org/#/c/696354/ ? (Updated UT) | 16:27 |
| *** gyee has joined #openstack-nova | 16:29 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Enable cross-cell resize in the nova-multi-cell job https://review.opendev.org/656656 | 16:32 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Flesh out docs for cross-cell resize/cold migrate https://review.opendev.org/696212 | 16:32 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: Simplify FinishResizeAtDestTask event handling https://review.opendev.org/695337 | 16:32 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add negative test to delete server during cross-cell resize claim https://review.opendev.org/688832 | 16:32 |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Implement reschedule logic for cross-cell resize/migrate https://review.opendev.org/696213 | 16:32 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Clarify configuration steps for PF devices https://review.opendev.org/694522 | 16:32 |
| stephenfin | Sure, done | 16:33 |
| stephenfin | efried: Some real nice docs there ^ :) | 16:33 |
| efried | thanks, and noted. | 16:35 |
| dansmith | efried: did you see my comment about the event getting 422? | 16:36 |
| *** lbragsta_ has joined #openstack-nova | 16:36 | |
| efried | not yet | 16:36 |
| *** gyee has quit IRC | 16:36 | |
| dansmith | okay I don't want sundar to be held up on a decision there or anything | 16:36 |
| efried | reading now | 16:37 |
| *** rpittau is now known as rpittau|afk | 16:38 | |
| efried | dansmith: oh, that's a good point. If cyborg gets this particular 422 it should mean the bind finished before we even got to the host, so the poll inside wait_for_instance_event will trigger and prevent us from waiting (for an event that won't come) | 16:38 |
| dansmith | efried: right, if it gets a 404 then it should fail the bind, but if it gets 422, it shouldn't freak out | 16:39 |
| efried | ++ | 16:39 |
| dansmith | in fact, 422 confirms, in a weird way, that the instance is still there, just not on a host yet | 16:39 |
| dansmith | so if you could chuck that ++ into the review so sundar isn't waiting that would be good | 16:40 |
| *** udesale has quit IRC | 16:41 | |
| efried | dansmith: done | 16:43 |
| dansmith | efried: spanks | 16:43 |
| efried | comfortable and slimming | 16:44 |
| dansmith | efried: have you chatted with him to know that he's responning? | 16:44 |
| dansmith | *respinning | 16:44 |
| efried | haltingly | 16:44 |
| dansmith | haltingly? | 16:44 |
| efried | think voicemail tag | 16:44 |
| efried | but with slack | 16:44 |
| dansmith | I, uh, will trust you | 16:46 |
| *** bhagyashris has quit IRC | 16:46 | |
| stephenfin | dansmith: Can I remove remotable methods on objects if there are no callers left in-tree | 16:49 |
| *** bhagyashris has joined #openstack-nova | 16:50 | |
| stephenfin | Or are they versioned since they're RPC'y | 16:50 |
| dansmith | stephenfin: not according to the letter of the law | 16:50 |
| dansmith | and yes, they're RPC by virtue of being remotable | 16:50 |
| dansmith | stephenfin: I would just replace their bodies with raise NotImplementedError() | 16:50 |
| stephenfin | Yeah, this is for Network.(create|destroy|associate|disassociate) | 16:51 |
| *** gyee has joined #openstack-nova | 16:51 | |
| stephenfin | I'll do just that | 16:51 |
| dansmith | aye | 16:51 |
| *** trident has quit IRC | 16:51 | |
| *** trident has joined #openstack-nova | 16:53 | |
| *** df_sbr has joined #openstack-nova | 16:54 | |
| *** damien_r has quit IRC | 16:55 | |
| *** Sundar has joined #openstack-nova | 16:55 | |
| *** awalende has joined #openstack-nova | 16:56 | |
| *** eharney has joined #openstack-nova | 16:57 | |
| efried | stephenfin: +2 on that docs patch, but a pile of nits if you're so inclined. | 16:59 |
| *** awalende has quit IRC | 16:59 | |
| efried | gibi or stephenfin: are you on board with https://review.opendev.org/#/c/695985/ (ability to cancel instance events)? | 17:00 |
| melwitt | stephenfin: did you see my note here that other quotas related to nova-network will be able to be removed? https://review.opendev.org/#/c/686812/7/nova/api/openstack/compute/quota_classes.py@38 | 17:01 |
| stephenfin | That's way too much context to load up at 5pm /o\ I can hit it in the morning though | 17:01 |
| efried | ack, thx | 17:01 |
| stephenfin | melwitt: Sure did. It's on the list of stuff still to clean up | 17:02 |
| melwitt | stephenfin: k, coolness | 17:02 |
| Sundar | Responded to the discussion on exit_wait_early for instance events in https://review.opendev.org/#/c/631244 | 17:10 |
| Sundar | efried: stephenfin: ^ relates to https://review.opendev.org/#/c/695985/ | 17:12 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: libvirt: Add a rbd_connect_timeout configurable https://review.opendev.org/669167 | 17:12 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/rocky: libvirt: Add a rbd_connect_timeout configurable https://review.opendev.org/669168 | 17:12 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/queens: libvirt: Add a rbd_connect_timeout configurable https://review.opendev.org/669169 | 17:12 |
| lyarwood | melwitt: ^ reopened and rebased | 17:12 |
| melwitt | lyarwood: ack thanks | 17:16 |
| *** KeithMnemonic has joined #openstack-nova | 17:18 | |
| mriedem | lyarwood: did anyone that reported that issue actually say the fix resolved it for them? | 17:19 |
| lyarwood | mriedem: we've just sent a test build out to a stable/queens user | 17:19 |
| lyarwood | mriedem: it's going to fix their issues but will at least confirm that their network is borked | 17:20 |
| *** mdbooth has joined #openstack-nova | 17:28 | |
| efried | Sundar: back atcha. To be clear, I'm not seeing a problem; are you? | 17:31 |
| *** macz has quit IRC | 17:32 | |
| Sundar | efried: The notification from Cyborg will fail in many/most cases; ignoring that error could be considered a hack, rather than a solution. After all, folks looking at the n-api logs will see an error. Are we ok with that? | 17:33 |
| sean-k-mooney | gibi: this is the bug for the issue with live migrating https://bugs.launchpad.net/nova/+bug/1854844. ill try to think about ways to adress this in a backportable way. i think i know of one way but this might just be something we have to fix on master only | 17:33 |
| openstack | Launchpad bug 1854844 in OpenStack Compute (nova) "libvirt: tx/rx queue lenght and max queues are not updtated on live migration" [Low,Triaged] - Assigned to sean mooney (sean-k-mooney) | 17:33 |
| efried | Sundar: Only if they're looking at debug logs :) | 17:34 |
| efried | The cyborg logs should of course explain what's happening as well. | 17:35 |
| efried | And yes, I'm fine with this. | 17:35 |
| Sundar | efried: From Cyborg's POV, the notification after binding would be best-effort: ignore any failures and complete the binding. The Cyborg logs will indicate as much. | 17:37 |
| sean-k-mooney | gibi: we might be able to store the queue paramater in the port binding profile which is an unversioned dict of strigns filed and was intend to store hypervior specific info for a port. | 17:38 |
| efried | Sundar: Not *any* failures. | 17:39 |
| Sundar | If we're all ok with that, I'm good. But there are a few nuances to consider. First, the function to get the resolved ARQs is now defined in https://review.opendev.org/631245, while it needs to be used in the previous patch https://review.opendev.org/631244. | 17:39 |
| sean-k-mooney | gibi: the use of multiple port bindings for live migration and the rx/tx queue size options were both intoduced in rocky so it should be backportable to all affected version if i do it right | 17:39 |
| efried | Sundar: As dansmith suggested (oh, maybe it was in IRC, not in the patch) you should still fail and clean up the binding if you get e.g. a 404. | 17:40 |
| efried | meaning the instance was deleted by the time you went to emit the notification. | 17:40 |
| sean-k-mooney | gibi: well assuming the call order works out ill follow up with this later | 17:40 |
| efried | So, to be clear, the 422 is a special case, with a prominent NOTE and a nice debug log stating that the binding completed before the instance landed on the host, so it's acceptable that the notification is dropped. | 17:41 |
| efried | Sundar: refactoring method X from patch Z into patch Y because that's where it's now going to be used, yeah, that's SOP. | 17:42 |
| Sundar | efried: There could be a bunch of different failure modes: cannot locate n-api service, located but it timed out?, etc. We need to distinguish that category from 'called n-api and it returned an error'. In the latter case, is there granularity of error codes for why it failed? | 17:42 |
| efried | Sundar: Yes, the 422 is a special case; any other error is an error. | 17:42 |
| *** jangutter has quit IRC | 17:43 | |
| Sundar | Need to ceck if 422 is used for any other error case. | 17:43 |
| Sundar | *check | 17:43 |
| *** lbragsta_ has quit IRC | 17:44 | |
| Sundar | Re. the patch series, I think we need to squash https://review.opendev.org/631245 into the previous patch https://review.opendev.org/631244. Because once the get_resolved_arqs() moves to previous patch, the only things left behind are the virt driver changes. It would arguably be easier to understand those changes together with the changes to get | 17:47 |
| Sundar | the arqs in compute/manager.py. | 17:47 |
| mriedem | nice https://docs.openstack.org/nova/latest/admin/#overview - "nova-api Receives XML requests" | 17:50 |
| *** macz has joined #openstack-nova | 17:52 | |
| *** dpawlik has joined #openstack-nova | 17:52 | |
| *** bhagyashris has quit IRC | 17:55 | |
| melwitt | stephenfin: I noticed a new recent comment on this bug https://bugs.launchpad.net/nova/+bug/1289064 saying, maybe it can be closed bc numa aware lm was merged? seems so to me but wanted to check with you | 17:56 |
| openstack | Launchpad bug 1289064 in OpenStack Compute (nova) "live migration of instance should claim resources on target compute node" [Medium,In progress] - Assigned to Artom Lifshitz (notartom) | 17:56 |
| Sundar | Only one use of 422 error code in create: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/server_external_events.py . So, we are probably good. | 17:56 |
| efried | yup (I checked that earlier) | 17:56 |
| artom | melwitt, stephenfin, oh, well - not entirely - only NUMA LM does claims | 17:57 |
| *** dpawlik has quit IRC | 17:57 | |
| artom | Then again, only NUMA LM has non-placement resources, so maybe yes? | 17:57 |
| artom | SRIOV is handled as well, albeit separately | 17:57 |
| melwitt | er, or artom, sorry ^ (I pinged stephen bc his patches about rejecting lm requests that have numa topology were merged as Related-Bug) | 17:58 |
| melwitt | hm, ok. I don't really understand what that means so I dunno either | 17:59 |
| sean-k-mooney | melwitt: ya so stephens patch just reject live migration if the guest has a numa toplogy at the api | 17:59 |
| sean-k-mooney | sriov live migration directly allocates and claims the devices during live migration | 18:00 |
| *** derekh has quit IRC | 18:00 | |
| sean-k-mooney | numa live migration uses a move claim to claim resouces | 18:00 |
| sean-k-mooney | (cpus and mempages/hugepages) | 18:00 |
| artom | sean-k-mooney, so, could we say that all non-placement resources are not accounted correctly during live migration? | 18:02 |
| sean-k-mooney | no | 18:02 |
| sean-k-mooney | i dont think this is an issue on master | 18:02 |
| artom | Err, *are accounted | 18:02 |
| eandersson | Looks good gibi | 18:02 |
| eandersson | btw mriedem did you see the minor bug I mentioned over the weekend | 18:03 |
| sean-k-mooney | artom: right non-placement rescoues (cpu,mempages,sriov devices) should be accounted for in train+ and we blocked numa/sriov migration before that retoactivly | 18:03 |
| sean-k-mooney | artom: melwitt so we might just be able to close https://bugs.launchpad.net/nova/+bug/1289064 | 18:04 |
| openstack | Launchpad bug 1289064 in OpenStack Compute (nova) "live migration of instance should claim resources on target compute node" [Medium,In progress] - Assigned to Artom Lifshitz (notartom) | 18:04 |
| *** factor has quit IRC | 18:04 | |
| eandersson | https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2075 <-- eventlet.Timeout (based on BaseException, not Exception) may be raised here causing an UnboundError. | 18:04 |
| mriedem | eandersson: nope | 18:05 |
| melwitt | sean-k-mooney: if there's not anything more to do there, I'd think it would be nice to close it | 18:06 |
| sean-k-mooney | well there are still cases where live migration is not supported but on master i think we account for all resouces correctly in the cases where live migration is supported | 18:07 |
| mriedem | eandersson: open a bug | 18:07 |
| artom | sean-k-mooney, melwitt, so.... fix release 'Train'? | 18:07 |
| dansmith | efried: Sundar: I was on a call during your discussion, but...we're all good now? | 18:08 |
| sean-k-mooney | artom: ya i think so | 18:08 |
| * artom does the needful | 18:08 | |
| melwitt | yeah, I can't remember how/if there's a way to mark it as fixed in the past. but I don't think that matters too much | 18:09 |
| artom | Done. | 18:09 |
| sean-k-mooney | you can add the sepcific release series to the bug | 18:09 |
| sean-k-mooney | i left a comment in the bug too | 18:09 |
| * artom couldn't, maybe I need more lp powers | 18:09 | |
| eandersson | mriedem https://bugs.launchpad.net/nova/+bug/1854848 | 18:10 |
| openstack | Launchpad bug 1854848 in OpenStack Compute (nova) "build_and_run_instance can error out with an UnboundException" [Undecided,New] | 18:10 |
| Sundar | efried: I think so, if you and all are ok with whatever I said above -- squishing the patches, etc. ignoring 422 alone seems fine. if n-api is unreachable or times out, that will still be an error that fails the binding. | 18:10 |
| sean-k-mooney | artom: added train | 18:10 |
| eandersson | It's a minor issue, but caused VMs to get stuck in BUILD... fyi was caused by selinux not being configured properly. | 18:10 |
| sean-k-mooney | artom: melwitt there is a "Target to series" button you can use | 18:11 |
| sean-k-mooney | you need to be in the nova bug team on LP to see/use it | 18:11 |
| *** ivve has joined #openstack-nova | 18:11 | |
| melwitt | yeah, I have that, but I don't think that will stop it from saying "released in <U version number>" whenever it's released next | 18:12 |
| melwitt | it'll still count it as landing "today" first and then "backported" to train. I don't think it matters, just that's what the automation is gonna do when it comments later | 18:12 |
| sean-k-mooney | ya proably | 18:13 |
| sean-k-mooney | i dont think we have automation tieded to that out side of the completed specs thing but that uses blueprints not bugs | 18:14 |
| sean-k-mooney | artom: if you are triaging nova bugs upstream you should join https://launchpad.net/~nova-bugs its an open team so you will be automatically approved | 18:16 |
| artom | sean-k-mooney, can't say I'm triaging dilligently | 18:16 |
| sean-k-mooney | well you know where to find it if you change your mind | 18:17 |
| melwitt | sean-k-mooney: I mean the thing that will make a comment on the bug that says "this bug was fixed in version x.x.x.x" that will be technically wrong because it's gonna say a U version. again, I don't think it matters, just that's what it's going to say on the comment | 18:17 |
| sean-k-mooney | 00. | 18:18 |
| sean-k-mooney | 00.1 | 18:18 |
| sean-k-mooney | droped phone | 18:18 |
| sean-k-mooney | am isnt that done by the release tooling | 18:18 |
| sean-k-mooney | not launchpad | 18:18 |
| melwitt | it is yeah | 18:19 |
| sean-k-mooney | but ya i guess in this case its not going to be set since artoms change did not have closes bug | 18:19 |
| sean-k-mooney | anyway it should be fine | 18:20 |
| melwitt | I don't think it matters, I think it just does a sweep through things that are Fix Released and makes a comment with the fixed in version | 18:20 |
| *** ricolin has quit IRC | 18:22 | |
| *** macz has quit IRC | 18:23 | |
| *** spsurya has quit IRC | 18:25 | |
| *** macz has joined #openstack-nova | 18:27 | |
| *** macz has quit IRC | 18:27 | |
| efried | dansmith: Yes, thanks. | 18:27 |
| *** macz has joined #openstack-nova | 18:28 | |
| dansmith | efried: so there'll need to be a change to cyborg for the 422 thing presumably | 18:30 |
| efried | yes | 18:30 |
| dansmith | efried: Sundar: is there a testing patch somewhere in the series that wires this all up with a fake cyborg driver? | 18:31 |
| dansmith | I assume that would be the one that needs to depends-on that cyborg change | 18:31 |
| efried | the CI job is working last I checked | 18:31 |
| sean-k-mooney | there is a patch to add a tempest job againd nova nad i think the job is now running on all cyborg patches | 18:31 |
| dansmith | there are a lot of patches that have various confused histories | 18:31 |
| efried | dansmith: https://review.opendev.org/#/c/670999/20 | 18:32 |
| efried | passing atm | 18:32 |
| dansmith | ack just found that | 18:32 |
| sean-k-mooney | same was going to link it | 18:32 |
| efried | there was a time when I looked and it wasn't doing anything, but I thought the last time I checked it was actually running through the spawn flow with a real (pseudo) device | 18:33 |
| sean-k-mooney | it is doing two things | 18:33 |
| efried | still very basic though | 18:33 |
| efried | green path only | 18:33 |
| sean-k-mooney | its cratein a device profile and then a super minima test of booting a vm | 18:33 |
| sean-k-mooney | althogh that one is not correct | 18:33 |
| sean-k-mooney | its booting a vm but not waiting for ti to go active before deleting it | 18:34 |
| sean-k-mooney | at least the last time i looked but everythin is in place to add proper lifecycle tests | 18:34 |
| dansmith | seems like a lot of vm_state=error in the compute logs for that, I'll have to track it down closer to see if it's related | 18:34 |
| sean-k-mooney | https://opendev.org/openstack/cyborg-tempest-plugin/src/branch/master/cyborg_tempest_plugin/tests/scenario/test_accelerator_basic_ops.py#L45-L60 | 18:35 |
| dansmith | does that even wait for it to become running? | 18:35 |
| sean-k-mooney | no which i mentioned before | 18:36 |
| sean-k-mooney | to the cyborg folks | 18:36 |
| sean-k-mooney | create server does https://opendev.org/openstack/cyborg-tempest-plugin/src/branch/master/cyborg_tempest_plugin/tests/scenario/manager.py#L245-L257 | 18:36 |
| dansmith | heh okay | 18:36 |
| dansmith | oh okay | 18:36 |
| dansmith | also, the name of the test makes it hard to track down those instances in the logs | 18:36 |
| dansmith | would be better if it was called test_server_ops_with_accel or something | 18:36 |
| sean-k-mooney | ya with that said its an easy change as is extenidn it to do more really tests | 18:37 |
| Sundar | sean-k-mooney: The tempest test waits till the server becomes active: https://review.opendev.org/#/c/667231/11/cyborg_tempest_plugin/tests/scenario/manager.py@155 | 18:37 |
| dansmith | I don't see that the cyborg instance ever made it to the compute node | 18:39 |
| sean-k-mooney | oh you are using the common tempest function ok | 18:39 |
| *** jmlowe has quit IRC | 18:39 | |
| Sundar | dansmith: The tempest CI test in cyborg-tempest-plugin is called test_server_basic_ops '' https://review.opendev.org/#/c/667231/11/cyborg_tempest_plugin/tests/scenario/test_accelerator_basic_ops.py@45 | 18:40 |
| dansmith | Sundar: right, which makes it hard to distinguish from the other test_server_basic_ops | 18:40 |
| dansmith | I don't see it actually running though: https://8664d62e69400bd89796-9cb1d5e035819d8d5535734f80756cd4.ssl.cf1.rackcdn.com/670999/20/check/tempest-integrated-compute/cdfc846/testr_results.html.gz | 18:40 |
| sean-k-mooney | Sundar: that is an existing class name in tempest | 18:40 |
| dansmith | oh wait, that's the wrong log, ignore me | 18:41 |
| sean-k-mooney | although the module path should be different | 18:41 |
| Sundar | If you just want to mention 'accel' somewhere in the name, that's not a problem. Will do. | 18:41 |
| dansmith | sean-k-mooney: right but the test name is what drives the tempest tenant name I think | 18:42 |
| sean-k-mooney | https://2371b0492dbe3a0c56c0-5469ab4f5c2741453cb8b95135b2c449.ssl.cf2.rackcdn.com/670999/20/check/cyborg-tempest/2124885/testr_results.html.gz | 18:42 |
| sean-k-mooney | that is the result you were looking for | 18:42 |
| dansmith | but they're hardcoding the server name, which helps | 18:42 |
| dansmith | yeah I know | 18:42 |
| dansmith | so for this fake test, we don't actually provide any device to libvirt to attach to the guest right: | 18:42 |
| dansmith | ? | 18:43 |
| Sundar | That's right | 18:43 |
| sean-k-mooney | correct | 18:43 |
| sean-k-mooney | the arq has type fake and its ignored | 18:43 |
| Sundar | The path till creating/binding.getting ARQs is the same as for a real driver | 18:43 |
| dansmith | will there be a real job running on intel ci or something that can actually do this? | 18:43 |
| dansmith | Sundar: sure, it just doesn't cover actually producing a device and getting libvirt to attach it | 18:44 |
| Sundar | dansmith: Yes, we are planning a 3rd party CI for Intel FPGAs | 18:44 |
| sean-k-mooney | we could maybe use some other virtual device for testing in a more robust way too | 18:44 |
| dansmith | sean-k-mooney: yeah I was going to ask if we could pass some harmless pci device through, but wasn't sure if we can do that in the gate without nested | 18:45 |
| Sundar | sean-k-mooney: We have plans to enhance the fake driver to simulate programming -- it will just respons to the programming API within Cyborg with a success/failure | 18:45 |
| *** igordc has joined #openstack-nova | 18:45 | |
| sean-k-mooney | dansmith: i was thinking of creating a file on disk and passing it to the guest | 18:45 |
| dansmith | Sundar: yeah, that's good, I just want to see the actual "here's the pci device" stuff tested | 18:45 |
| Sundar | That would cover more paths, including Glance interaction within Cyborg. But it would still not attach a PCI device to the VM | 18:45 |
| *** martinkennelly has quit IRC | 18:45 | |
| dansmith | sean-k-mooney: how does a file on disk simulate a pci device/ | 18:46 |
| dansmith | okay anyway, this log looks good for what we have now and can test now, so that's cool | 18:46 |
| dansmith | I'll want to look at this one Sundar retools the current patches per the pending feedback | 18:46 |
| sean-k-mooney | it does not but it will simulate genarting xml. i was really thinking of a lvm driver for local caching so we could have cyborg provide local storage to a vm. | 18:47 |
| dansmith | Sundar: FWIW, I have this week and next week before I turn to a pumpkin for the rest of the year | 18:47 |
| sean-k-mooney | e.g. have a driver that could be useful in producion but has no hardware depency | 18:47 |
| Sundar | dansmith: I am going to squash https://review.opendev.org/631245 into https://review.opendev.org/631244, for the reasons I mentioned above | 18:47 |
| dansmith | sean-k-mooney: is that a thing? I thought we were focused on pci devices for accelerators right now | 18:47 |
| sean-k-mooney | we are but form a livirt driver point of there is very littel difference between generating the xml for a pci device vs a block device | 18:48 |
| dansmith | Sundar: okay I skimmed over that.. IMHO, keeping the new event definition in its own patch makes sense just because of how much api stuff has to change | 18:48 |
| dansmith | sean-k-mooney: seems fairly different to me.. I understand it's just a chunk of xml, but it's not the same kind of xml we really care about | 18:49 |
| Sundar | dansmith: Agreed. The event definition patch https://review.opendev.org/692707 will stay as is. | 18:49 |
| sean-k-mooney | if we just want a way to create a device (lvm volume) program it (dd image to volume) and validate it in the guset it was a way to do it. | 18:49 |
| dansmith | sean-k-mooney: it's something, it's just not enough | 18:49 |
| dansmith | Sundar: oh sorry I was looking at the wrong two tabs, lemme re-read | 18:49 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Request and store the decrypted path of an attached encrypted volume https://review.opendev.org/696693 | 18:50 |
| sean-k-mooney | ya i was also looking at ways to fake pci devices in linux and there are some ways but its not trivial | 18:50 |
| Sundar | sean-k-mooney: The code paths within the virt driver are quite different for block devices and PCI devices. So, how does this help? | 18:50 |
| dansmith | Sundar: 631234 all but goes away after you address my feedback | 18:51 |
| dansmith | Sundar: so assuming you mean just bringing forward the cyborg client method, then sure | 18:51 |
| dansmith | Sundar: it would be attaching a block device as an accelerator, so no typical nova block device handling for it | 18:51 |
| sean-k-mooney | its a more complete end to end test that is all. also im not sure how https://review.opendev.org/#/c/689070/ is going to progress and its another way to enable local caching | 18:51 |
| dansmith | sean-k-mooney: I don't think cyborg for block devices is a good way to address the local caching thing | 18:52 |
| dansmith | sean-k-mooney: how is that not going to confuse people? | 18:52 |
| sean-k-mooney | well given samsung are now supporting sriov on there new ssds it might be but sure ill let you get back to it | 18:52 |
| dansmith | well, I guess we can have that discussion, but I definitely don't want to merge that with this effort at the moment | 18:53 |
| dansmith | and since PCI is different enough, I still want to see PCI working for this | 18:53 |
| sean-k-mooney | oh sure. its totally seperate | 18:53 |
| sean-k-mooney | ya | 18:53 |
| sean-k-mooney | i just have not figured out how to fake it yet | 18:53 |
| sean-k-mooney | the kernel netdevsim module was the close i came up with in the past | 18:54 |
| dansmith | yeah, well, as long as we'll have intel ci so we can see it working, that's something | 18:54 |
| Sundar | dansmith: 631234 is a typo, right? Quoting myself from above: I think we need to squash https://review.opendev.org/631245 into the previous patch https://review.opendev.org/631244. Because once the get_resolved_arqs() moves to previous patch, the only things left behind are the virt driver changes. It would arguably be easier to understand those | 18:54 |
| Sundar | changes together with the changes to get the arqs in compute/manager.py. | 18:54 |
| dansmith | Sundar: I meant 631245 yes, sorry | 18:54 |
| sean-k-mooney | ya. the fake driver should allow us to test the api flow. the intel ci should validate it works with real hardware | 18:55 |
| Sundar | Cool, np | 18:55 |
| dansmith | Sundar: the virt changes will be insulated from the cyborg client method is my point | 18:55 |
| *** jmlowe has joined #openstack-nova | 18:55 | |
| dansmith | Sundar: if I were you I would have one patch that adds the support into the manager, all the build_resources changes to capture the ARQs and pass them to the virt driver, | 18:56 |
| dansmith | Sundar: then a separate patch to implement things for libvirt | 18:56 |
| Sundar | dansmith: Sure, will try to get back ASAP with the changes | 18:56 |
| Sundar | Oh, separate patch for libvirt? | 18:56 |
| dansmith | yes definitely | 18:56 |
| Sundar | That would be 63124 today | 18:56 |
| Sundar | *631245 | 18:56 |
| dansmith | Sundar: so when hyperv people come along, we point them at the libvirt patch and say "implement this part for your driver and you will be good"... separate from the plumbing in compute to wire everything up | 18:57 |
| Sundar | I see, fine with that too | 18:58 |
| dansmith | Sundar: in the compute manager patch, you just assume that any virt driver below you will ignore the ARQs you pass to it, which will be true of libvirt, until the next patch in the series where that becomes untrue for libvirt, but remains true for all the others | 18:58 |
| openstackgerrit | Merged openstack/nova stable/pike: Add regression recreate test for bug 1830747 https://review.opendev.org/663124 | 18:59 |
| openstack | bug 1830747 in OpenStack Compute (nova) pike "Error 500 trying to migrate an instance after wrong request_spec" [High,In progress] https://launchpad.net/bugs/1830747 - Assigned to Matt Riedemann (mriedem) | 18:59 |
| openstackgerrit | Merged openstack/nova stable/pike: Workaround missing RequestSpec.instance_group.uuid https://review.opendev.org/663125 | 18:59 |
| Sundar | dansmith: In the compute manager patch, the signature for libvirt driver's spawn() will have tio change to include the arqs as a parameter. | 18:59 |
| dansmith | Sundar: does this method go away if you pass the stuff from compute manager to virt? https://review.opendev.org/#/c/631245/39/nova/accelerator/cyborg.py | 18:59 |
| dansmith | Sundar: that would leave the virt patch you have as just the libvirt config bits | 18:59 |
| dansmith | Sundar: correct, spawn() will need to take the ARQs, just like it takes network and block device info | 19:00 |
| *** lpetrut has quit IRC | 19:00 | |
| *** macz has quit IRC | 19:01 | |
| dansmith | Sundar: https://github.com/openstack/nova/blob/56d3cd7aa7f4d3be01fd2a5c10903fb548c49458/nova/compute/manager.py#L2491-L2499 | 19:01 |
| *** dpawlik has joined #openstack-nova | 19:01 | |
| dansmith | Sundar: just like block and network, you should pull out arqs from resources and pass to spawn | 19:02 |
| dansmith | arguably we could just be passing in resources to spawn to avoid splitting it out, but let's not go down that hole in this set | 19:02 |
| dansmith | Sundar: and since I will have a strong opinion on that later, let me go ahead and say, it should be called: | 19:03 |
| dansmith | accel_info = resources['accel_info'] | 19:03 |
| dansmith | :) | 19:03 |
| Sundar | dansmith: That method would remain, albeit it would move to the compute manager patch. So that the waiting logic can call it once to decide whether to quit early. The virt driver patch 631245 will slim down to just the libvirt driver changes. The change in the signature for virt driver base class and the libvirt driver (to include arqs as a | 19:03 |
| Sundar | parameter) will now be in the compute manager patch. | 19:03 |
| dansmith | Sundar: ah right, forgot that would be the check, so yes, move it thusly | 19:03 |
| Sundar | dansmith: What you said about resources['accel...'] is exactly what I said in the gerrit review :) | 19:04 |
| dansmith | ack | 19:04 |
| dansmith | Sundar: except not the same names | 19:04 |
| dansmith | Sundar: I was giving you a freebie pre -1 on the *name* :) | 19:04 |
| Sundar | What name? | 19:05 |
| dansmith | accel_info instead of accelerator_requests | 19:05 |
| Sundar | I see. To match network_info etc. | 19:05 |
| dansmith | yes | 19:05 |
| dansmith | because I have issues. my therapist and I are working on them. | 19:05 |
| Sundar | Ok, I'm not too hung up on names | 19:05 |
| dansmith | good :) | 19:05 |
| *** dpawlik has quit IRC | 19:06 | |
| Sundar | sean-k-mooney: still intrigued by your idea for a fake device, block or PCI. Do you have a way to simulate a PCI device? It can be *quite* difficult AFAICS with linux PCI subsystem. | 19:08 |
| dansmith | so I was thinking about something else, like grab the last usb host and pass that through | 19:08 |
| dansmith | Sundar: I would focus on getting the patches back into order, and let sean-k-mooney pull something out of his butt for faking the PCI stuff... :) | 19:09 |
| Sundar | Yup, that's what I plan to do :) but a 'real fake driver' (heck, I'll grab the award for today's oxymoron) would be very useful for the future. | 19:10 |
| dansmith | he pulled off some fancy stuff for the numa live migration stuff, so we should see if he can do it again here :) | 19:10 |
| sean-k-mooney | Sundar: the closest i have come up with is the netdevsim kernel module wich does create sriov capable fake pci device for testing and developing the kernel ebpf code | 19:10 |
| sean-k-mooney | Sundar: how ever i dont think you can actully pass those device to a qemu instace. i have not tried however | 19:10 |
| sean-k-mooney | i did try to use it for sriov testing in the past but i could not get it to work | 19:11 |
| sean-k-mooney | i was able to create the fake device but i could not assing the vfs the pf might have worked but we dont really want to have to compile kernel module in the gate if we can avoid it | 19:11 |
| Sundar | sean-k-mooney: It would be useful to have a libvirt patch where the domain XML for hostdev can have an attribute that indicates 'do not pass this to qemu'. | 19:12 |
| Sundar | That would be useful outside of OpenStack too, I would guess. | 19:12 |
| dansmith | sean-k-mooney: that's why i was thinking pick something on the host we could pass through, like one of those USB hosts | 19:12 |
| dansmith | Sundar: can't imagine the libvirt folks would go for that | 19:12 |
| sean-k-mooney | dansmith: maybe ya | 19:12 |
| sean-k-mooney | there was talk and patchs for adding the abilty to emulate sriov device in qemu | 19:14 |
| sean-k-mooney | if that ever gets merged upstream we might be able to use that | 19:14 |
| sean-k-mooney | but for now it think focusing on getting the patches in order is better | 19:14 |
| sean-k-mooney | if i come up with a way to fake sriov/pci deivces ill let ye know | 19:15 |
| Sundar | We're all agreed on that :). Time for me to get back to work. | 19:15 |
| sean-k-mooney | one thing i have been meaning to test for a while is if i can use a neutron port as a pci device for a guest | 19:16 |
| sean-k-mooney | they dont support vfs put i might be able to do a PF passthough | 19:16 |
| sean-k-mooney | im pretty sure we are missing a few things to make that work however espacally in the upstream ci. | 19:18 |
| *** macz has joined #openstack-nova | 19:33 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Use Placement 1.34 (string suffixes & mappings) https://review.opendev.org/696418 | 19:38 |
| openstackgerrit | Eric Fried proposed openstack/nova master: WIP: Tie requester_id to RequestGroup suffix https://review.opendev.org/696946 | 19:38 |
| efried | gibi: 696418 ^ now just does the 1.34 cutover, will continue with the rest in subsequent patches. | 19:40 |
| *** dpawlik has joined #openstack-nova | 19:41 | |
| *** jangutter has joined #openstack-nova | 19:44 | |
| *** dpawlik has quit IRC | 19:46 | |
| *** jmlowe has quit IRC | 19:48 | |
| *** jangutter has quit IRC | 19:48 | |
| *** tesseract has quit IRC | 19:49 | |
| openstackgerrit | Merged openstack/nova master: Process requested_resources in ResourceRequest init https://review.opendev.org/696354 | 19:51 |
| *** avolkov_ has joined #openstack-nova | 19:54 | |
| *** donnyd_ has joined #openstack-nova | 19:54 | |
| *** jrosser_ has joined #openstack-nova | 19:54 | |
| *** amodi_ has joined #openstack-nova | 19:55 | |
| *** tbachman_ has joined #openstack-nova | 19:56 | |
| *** tinwood_ has joined #openstack-nova | 19:56 | |
| *** mugsie_ has joined #openstack-nova | 19:56 | |
| *** StevenK_ has joined #openstack-nova | 19:57 | |
| *** macz has quit IRC | 20:02 | |
| *** tbachman has quit IRC | 20:02 | |
| *** ralonsoh has quit IRC | 20:02 | |
| *** pcaruana has quit IRC | 20:02 | |
| *** avolkov has quit IRC | 20:02 | |
| *** brault has quit IRC | 20:02 | |
| *** gryf has quit IRC | 20:02 | |
| *** amodi has quit IRC | 20:02 | |
| *** Roamer` has quit IRC | 20:02 | |
| *** gibi has quit IRC | 20:02 | |
| *** Jeffrey4l_ has quit IRC | 20:02 | |
| *** jkulik has quit IRC | 20:02 | |
| *** tinwood has quit IRC | 20:02 | |
| *** mugsie has quit IRC | 20:02 | |
| *** StevenK has quit IRC | 20:02 | |
| *** johanssone has quit IRC | 20:02 | |
| *** jrosser has quit IRC | 20:02 | |
| *** donnyd has quit IRC | 20:02 | |
| *** rabel has quit IRC | 20:02 | |
| *** fnordahl has quit IRC | 20:02 | |
| *** cmurphy has quit IRC | 20:02 | |
| *** dr_gogeta86 has quit IRC | 20:02 | |
| *** avolkov_ is now known as avolkov | 20:02 | |
| *** tbachman_ is now known as tbachman | 20:02 | |
| *** donnyd_ is now known as donnyd | 20:02 | |
| *** jrosser_ is now known as jrosser | 20:02 | |
| *** Jeffrey4l has joined #openstack-nova | 20:02 | |
| *** johanssone has joined #openstack-nova | 20:03 | |
| efried | dansmith: is nullable=True (e.g. [1]) made moot by having a default and using obj_set_defaults() in __init__ (e.g. [2])? | 20:04 |
| efried | [1] https://github.com/openstack/nova/blob/f1382651dc3b7b945b69a2af0bc05aa472f26b28/nova/objects/request_spec.py#L1061 | 20:04 |
| efried | [2] https://github.com/openstack/nova/blob/f1382651dc3b7b945b69a2af0bc05aa472f26b28/nova/objects/request_spec.py#L1071 | 20:04 |
| *** ralonsoh has joined #openstack-nova | 20:04 | |
| dansmith | no | 20:04 |
| dansmith | efried: nullable=True *only* means "can be set to None" | 20:04 |
| efried | ah fusk, will I *ever* get that right? | 20:04 |
| dansmith | probably not | 20:05 |
| efried | I thought it meant "can be absent" (in terms of `in`) | 20:05 |
| dansmith | no, | 20:05 |
| dansmith | all fields can be absent all the time | 20:05 |
| dansmith | also unrelated to defaults | 20:05 |
| *** pcaruana has joined #openstack-nova | 20:05 | |
| efried | dansmith: oh; so does obj_set_defaults() mean I don't have to do the `in` check? | 20:05 |
| dansmith | no | 20:05 |
| dansmith | if you haven't set defaults on the object, then 'in' will fail if the field is unset | 20:06 |
| dansmith | you're connecting three things which are unrelated | 20:06 |
| dansmith | __contains__ means nothing more than "is it set right now" | 20:06 |
| dansmith | nullable means "can be set to none" | 20:06 |
| *** cmurphy has joined #openstack-nova | 20:07 | |
| dansmith | and default= means "if they call obj_set_defaults(), set this field to this value" | 20:07 |
| efried | right, so forget nullable; I'm really just confirming whether gibi is correct here https://review.opendev.org/#/c/696418/1/nova/scheduler/utils.py@327 | 20:07 |
| dansmith | technically he is not right | 20:07 |
| dansmith | practically, he is. | 20:07 |
| efried | because the obj could be loaded from cold storage | 20:07 |
| dansmith | right | 20:07 |
| efried | where it was unset | 20:07 |
| dansmith | or you could del obj['field'] | 20:08 |
| efried | but in this case the only way it got to cold storage was from having been previously __init__()ed, which set the defaults | 20:08 |
| dansmith | the init hack might actually result in it being set when you load it from the db, but unintentionally, which is why I have campaigned against *ever* doing that | 20:08 |
| efried | oh, unless it got stored/restored at an earlier version, I guess. | 20:08 |
| dansmith | we don't check for every field in every object always because it's pointless.. in most cases we know which fields are always set | 20:09 |
| *** gyee has quit IRC | 20:09 | |
| dansmith | if there is any question, then you must check, but if there's not then you don'[t | 20:09 |
| efried | so the check is not truly redundant, but not worth keeping in this case. | 20:10 |
| dansmith | so the init hack probably means you dont need to check (hence "practically") but... the init hack is not something we should be doing, IMHO, so.. | 20:10 |
| dansmith | right | 20:10 |
| dansmith | I mean, | 20:10 |
| openstackgerrit | Merged openstack/nova master: Reusable RequestGroup.add_{resource|trait} https://review.opendev.org/696380 | 20:10 |
| dansmith | you can answer the "worth keeping" part, as I dont' have context | 20:10 |
| dansmith | but I assume you're saying it has always been set somewhere, so probably safe | 20:10 |
| dansmith | which, if true, is in keeping with our practices | 20:11 |
| efried | I'm glad I asked. | 20:11 |
| efried | thanks | 20:11 |
| dansmith | yarp | 20:13 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/train: compute: Use long_rpc_timeout in reserve_block_device_name https://review.opendev.org/696953 | 20:13 |
| *** gryf has joined #openstack-nova | 20:13 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova master: api-guide: flesh out the server actions section https://review.opendev.org/696954 | 20:14 |
| mriedem | dansmith: melwitt: i wrote up a doc on using the instance actions API/commands ^ | 20:14 |
| *** jmlowe has joined #openstack-nova | 20:14 | |
| melwitt | f yeah | 20:15 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/stein: compute: Use long_rpc_timeout in reserve_block_device_name https://review.opendev.org/696955 | 20:15 |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/rocky: compute: Use long_rpc_timeout in reserve_block_device_name https://review.opendev.org/696956 | 20:16 |
| *** munimeha1 has quit IRC | 20:20 | |
| *** ralonsoh has quit IRC | 20:20 | |
| melwitt | mriedem: I thought you might appreciate this video https://twitter.com/Fobwashed/status/1200701358623473664 | 20:24 |
| efried | I am clearly missing a huge amount of context and will never get that video. | 20:26 |
| dansmith | same | 20:26 |
| mriedem | i don't actually play games made after 98 so while i get it, ... | 20:26 |
| mriedem | it's a joke about a shitty first person adventure game | 20:26 |
| dansmith | people who live in their parents' basements are laughing hysterically right now though, I'm sure | 20:27 |
| melwitt | really? aw man | 20:27 |
| dansmith | (and melwitt apparently :P) | 20:27 |
| melwitt | my favorite game in the series was morrowind. I started playing skyrim but lost interest | 20:28 |
| dansmith | never heard of either | 20:28 |
| mriedem | the pan flute is a nice touch | 20:28 |
| dansmith | damn kids. | 20:28 |
| melwitt | yeah, I guess only rpg fans might know it. though skyrim got pretty universally popular (to my surprise) | 20:29 |
| *** macz has joined #openstack-nova | 20:31 | |
| *** gyee has joined #openstack-nova | 20:36 | |
| *** tbachman has quit IRC | 20:50 | |
| *** tbachman has joined #openstack-nova | 20:55 | |
| mriedem | oh i am an rpg fan and have the saving throws to prove it | 20:55 |
| efried | "saving throws" sounds like a judo thing to me | 20:56 |
| *** amodi_ is now known as amodi | 20:57 | |
| efried | like "sacrifice throws" https://www.youtube.com/watch?v=XHuqFd1BHbE | 20:58 |
| mriedem | there is an equivalent amount of BO smell | 21:00 |
| mriedem | i'm not sure you can be turned to stone by a cockatrice in judo though | 21:01 |
| melwitt | lol ......and that's why I thought there was a chance you'd appreciate the video | 21:02 |
| *** eharney has quit IRC | 21:06 | |
| *** damien_r has joined #openstack-nova | 21:10 | |
| *** damien_r has quit IRC | 21:14 | |
| *** tbachman has quit IRC | 21:19 | |
| *** Sundar has quit IRC | 21:25 | |
| *** tbachman has joined #openstack-nova | 21:27 | |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider Config File: YAML file loading and schema validation https://review.opendev.org/673341 | 21:40 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider Config File: Function to further validate and retrieve configs https://review.opendev.org/676029 | 21:40 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider Config File: Functions to merge provider configs to provider tree https://review.opendev.org/676522 | 21:40 |
| openstackgerrit | Dustin Cowles proposed openstack/nova master: WIP: Provider Config File: Enable loading and merging of provider configs https://review.opendev.org/693460 | 21:40 |
| *** dpawlik has joined #openstack-nova | 21:43 | |
| *** dpawlik has quit IRC | 21:48 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Tie requester_id to RequestGroup suffix https://review.opendev.org/696946 | 21:48 |
| *** nweinber has quit IRC | 21:48 | |
| *** slaweq has quit IRC | 21:48 | |
| *** StevenK_ is now known as StevenK | 21:49 | |
| *** fnordahl has joined #openstack-nova | 21:56 | |
| *** rcernin has joined #openstack-nova | 21:57 | |
| *** mugsie_ is now known as mugsie | 21:59 | |
| *** rcernin has quit IRC | 21:59 | |
| *** factor has joined #openstack-nova | 22:08 | |
| *** slaweq has joined #openstack-nova | 22:09 | |
| *** gshippey has quit IRC | 22:10 | |
| openstackgerrit | Merged openstack/nova master: api-guide: flesh out the server actions section https://review.opendev.org/696954 | 22:11 |
| *** pcaruana has quit IRC | 22:11 | |
| *** tbachman has quit IRC | 22:12 | |
| *** slaweq has quit IRC | 22:13 | |
| *** eharney has joined #openstack-nova | 22:13 | |
| *** abaindur has joined #openstack-nova | 22:16 | |
| *** dpawlik has joined #openstack-nova | 22:22 | |
| *** abaindur has quit IRC | 22:24 | |
| *** abaindur has joined #openstack-nova | 22:24 | |
| *** dpawlik has quit IRC | 22:26 | |
| *** rcernin has joined #openstack-nova | 22:57 | |
| *** avolkov has quit IRC | 23:08 | |
| *** tkajinam has joined #openstack-nova | 23:09 | |
| *** tbachman has joined #openstack-nova | 23:19 | |
| *** mlavalle has quit IRC | 23:30 | |
| openstackgerrit | Eric Fried proposed openstack/nova master: Use Placement 1.34 (string suffixes & mappings) https://review.opendev.org/696418 | 23:41 |
| openstackgerrit | Eric Fried proposed openstack/nova master: Tie requester_id to RequestGroup suffix https://review.opendev.org/696946 | 23:41 |
| *** tosky has quit IRC | 23:41 | |
| *** mriedem has quit IRC | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!