*** betherly has quit IRC | 00:01 | |
*** mmethot_ has joined #openstack-nova | 00:09 | |
*** logan- has quit IRC | 00:12 | |
*** mmethot has quit IRC | 00:13 | |
*** logan- has joined #openstack-nova | 00:13 | |
*** betherly has joined #openstack-nova | 00:17 | |
*** ivve has quit IRC | 00:21 | |
*** betherly has quit IRC | 00:22 | |
*** markvoelker has joined #openstack-nova | 00:23 | |
*** tetsuro has joined #openstack-nova | 00:32 | |
*** hongbin has joined #openstack-nova | 00:34 | |
*** tetsuro has quit IRC | 00:57 | |
*** ricolin has joined #openstack-nova | 01:02 | |
*** tetsuro has joined #openstack-nova | 01:02 | |
*** mkrai has joined #openstack-nova | 01:12 | |
*** igordc has quit IRC | 01:13 | |
*** igordc has joined #openstack-nova | 01:15 | |
*** ricolin_ has joined #openstack-nova | 01:17 | |
*** betherly has joined #openstack-nova | 01:19 | |
*** tbachman has quit IRC | 01:19 | |
*** ricolin has quit IRC | 01:20 | |
*** igordc has quit IRC | 01:23 | |
*** betherly has quit IRC | 01:24 | |
*** pcaruana has quit IRC | 01:26 | |
*** tetsuro has quit IRC | 01:28 | |
*** gyee has quit IRC | 01:33 | |
*** markvoelker has quit IRC | 01:39 | |
*** markvoelker has joined #openstack-nova | 01:40 | |
*** betherly has joined #openstack-nova | 01:48 | |
*** betherly has quit IRC | 01:53 | |
openstackgerrit | Li Liu proposed openstack/nova master: WIP: Create and bind Cyborg ARQs. https://review.opendev.org/631244 | 02:09 |
---|---|---|
openstackgerrit | Eric Fried proposed openstack/nova master: trivial: Remove unused '_instance_to_allocations_dict' function https://review.opendev.org/674893 | 02:11 |
*** mkrai has quit IRC | 02:19 | |
*** _mlavalle_1 has joined #openstack-nova | 02:24 | |
*** mkrai has joined #openstack-nova | 02:25 | |
*** mlavalle has quit IRC | 02:26 | |
*** ricolin_ is now known as ricolin | 02:27 | |
*** huaqiang has joined #openstack-nova | 02:27 | |
*** betherly has joined #openstack-nova | 02:29 | |
*** _mlavalle_2 has joined #openstack-nova | 02:32 | |
*** betherly has quit IRC | 02:34 | |
*** _mlavalle_1 has quit IRC | 02:35 | |
*** BjoernT has quit IRC | 02:43 | |
*** _mlavalle3 has joined #openstack-nova | 02:43 | |
*** _mlavalle_2 has quit IRC | 02:44 | |
*** igordc has joined #openstack-nova | 02:49 | |
*** betherly has joined #openstack-nova | 02:50 | |
*** BjoernT has joined #openstack-nova | 02:52 | |
*** ircuser-1 has quit IRC | 02:52 | |
*** huaqiang has quit IRC | 02:52 | |
*** betherly has quit IRC | 02:55 | |
fungi | i directly reenqueued 674821,1 back into the gate after that last failure, so no need to recheck it | 02:56 |
fungi | i'd promote it but the only thing ahead is a cinder change which is eta 28 minutes to merge anyway | 02:56 |
openstackgerrit | ZhengMa proposed openstack/nova master: Check container_format on instance creation https://review.opendev.org/673407 | 02:56 |
*** tetsuro has joined #openstack-nova | 02:58 | |
openstackgerrit | ZhengMa proposed openstack/nova master: Check container_format on instance creation https://review.opendev.org/673407 | 02:59 |
*** _mlavalle3 has quit IRC | 02:59 | |
*** _mlavalle3 has joined #openstack-nova | 03:00 | |
fungi | the failure was a TimeoutException in test_walk_versions on py36, for the record | 03:01 |
openstackgerrit | Li Liu proposed openstack/nova master: WIP: Create and bind Cyborg ARQs. https://review.opendev.org/631244 | 03:12 |
*** markvoelker has quit IRC | 03:14 | |
*** huaqiang has joined #openstack-nova | 03:17 | |
*** whoami-rajat has joined #openstack-nova | 03:19 | |
*** tetsuro has quit IRC | 03:29 | |
*** psachin has joined #openstack-nova | 03:36 | |
openstackgerrit | YAMAMOTO Takashi proposed openstack/nova master: Revert "Revert resize: wait for events according to hybrid plug" https://review.opendev.org/675021 | 03:36 |
*** markvoelker has joined #openstack-nova | 03:38 | |
*** betherly has joined #openstack-nova | 03:42 | |
*** markvoelker has quit IRC | 03:43 | |
*** pcaruana has joined #openstack-nova | 03:44 | |
*** betherly has quit IRC | 03:47 | |
*** hongbin has quit IRC | 03:49 | |
openstackgerrit | Li Liu proposed openstack/nova master: WIP: Create and bind Cyborg ARQs. https://review.opendev.org/631244 | 03:53 |
*** tetsuro has joined #openstack-nova | 04:03 | |
*** udesale has joined #openstack-nova | 04:04 | |
*** ratailor has joined #openstack-nova | 04:08 | |
openstackgerrit | Li Liu proposed openstack/nova master: WIP: Added Cyborg External Events https://review.opendev.org/675022 | 04:11 |
*** BjoernT has quit IRC | 04:12 | |
openstackgerrit | Brin Zhang proposed openstack/python-novaclient master: Microversion 2.76: Add user_id/project_id to migration-list API https://review.opendev.org/675023 | 04:14 |
openstackgerrit | Alex Xu proposed openstack/nova-specs master: Update the virtual persistent memory spec based on discussion https://review.opendev.org/675024 | 04:15 |
openstackgerrit | Jing Zhang proposed openstack/nova master: Remove the current host from cold migration target host list https://review.opendev.org/675025 | 04:17 |
*** bhagyashris has joined #openstack-nova | 04:18 | |
*** jaosorior has quit IRC | 04:22 | |
*** markvoelker has joined #openstack-nova | 04:24 | |
*** markvoelker has quit IRC | 04:38 | |
*** betherly has joined #openstack-nova | 05:05 | |
*** abaindur has joined #openstack-nova | 05:07 | |
*** icarusfactor has joined #openstack-nova | 05:08 | |
*** factor has quit IRC | 05:08 | |
*** betherly has quit IRC | 05:10 | |
*** abaindur has quit IRC | 05:11 | |
*** abaindur has joined #openstack-nova | 05:11 | |
*** Luzi has joined #openstack-nova | 05:19 | |
openstackgerrit | Merged openstack/nova master: Replace non-nova server fault message https://review.opendev.org/674821 | 05:20 |
*** betherly has joined #openstack-nova | 05:26 | |
*** tetsuro has quit IRC | 05:30 | |
*** igordc has quit IRC | 05:31 | |
*** betherly has quit IRC | 05:31 | |
openstackgerrit | ZhengMa proposed openstack/nova master: Check container_format on instance creation https://review.opendev.org/673407 | 05:40 |
*** takashin has quit IRC | 05:47 | |
*** ociuhandu has joined #openstack-nova | 05:49 | |
*** ociuhandu has quit IRC | 05:53 | |
*** dpawlik has joined #openstack-nova | 05:56 | |
*** icarusfactor has quit IRC | 06:01 | |
*** dpawlik has quit IRC | 06:01 | |
*** factor has joined #openstack-nova | 06:02 | |
*** markvoelker has joined #openstack-nova | 06:18 | |
*** betherly has joined #openstack-nova | 06:18 | |
*** markvoelker has quit IRC | 06:22 | |
*** betherly has quit IRC | 06:23 | |
*** tetsuro has joined #openstack-nova | 06:24 | |
*** udesale has quit IRC | 06:24 | |
*** udesale has joined #openstack-nova | 06:24 | |
*** maciejjozefczyk has joined #openstack-nova | 06:25 | |
*** udesale has quit IRC | 06:27 | |
*** udesale has joined #openstack-nova | 06:27 | |
*** jaosorior has joined #openstack-nova | 06:28 | |
*** ccamacho has joined #openstack-nova | 06:45 | |
*** dpawlik has joined #openstack-nova | 06:46 | |
*** slaweq has joined #openstack-nova | 06:48 | |
*** betherly has joined #openstack-nova | 06:50 | |
*** betherly has quit IRC | 06:54 | |
openstackgerrit | Merged openstack/nova master: trivial: Remove unused '_instance_to_allocations_dict' function https://review.opendev.org/674893 | 07:01 |
*** rcernin has quit IRC | 07:03 | |
*** rpittau|afk is now known as rpittau | 07:07 | |
*** tssurya has joined #openstack-nova | 07:08 | |
*** huaqiang has quit IRC | 07:10 | |
*** huaqiang has joined #openstack-nova | 07:10 | |
*** udesale has quit IRC | 07:10 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix non-existent method of Mock https://review.opendev.org/675041 | 07:14 |
*** takashin has joined #openstack-nova | 07:18 | |
*** tesseract has joined #openstack-nova | 07:26 | |
*** ralonsoh has joined #openstack-nova | 07:29 | |
*** ivve has joined #openstack-nova | 07:34 | |
*** xek has joined #openstack-nova | 07:34 | |
*** lpetrut has joined #openstack-nova | 07:38 | |
*** jistr is now known as jistr|afk | 07:42 | |
*** helenafm has joined #openstack-nova | 07:48 | |
*** spsurya has joined #openstack-nova | 07:50 | |
*** ralonsoh has quit IRC | 07:54 | |
brinzhang | takashi: Why abandon this patch https://review.opendev.org/#/c/450119? | 07:55 |
*** huaqiang has quit IRC | 07:56 | |
brinzhang | takashi: I have a bp, it's codes need to add parameters to support for migration-list command, but there isnot the ``openstack server migration-list`` CLI | 07:57 |
*** udesale has joined #openstack-nova | 08:00 | |
brinzhang | takashin: :) | 08:01 |
takashin | brinzhang: Hi | 08:02 |
brinzhang | https://review.opendev.org/#/c/450119 this patch, looks good, but why are you abandon? | 08:02 |
takashin | brinzhang: There was a subsequnt BP. It was the function to abort cold migration. | 08:03 |
*** cgoncalves has quit IRC | 08:04 | |
takashin | brinzhang: But in community discussin, it is not necessary to abort cold migration. So https://review.opendev.org/#/c/450119 was abondaned. | 08:04 |
*** cgoncalves has joined #openstack-nova | 08:04 | |
brinzhang | takashin: Now in the openstackclient, there is not has the 'opesntack server migration-list', I was reviewed this patch, found you added it | 08:05 |
*** abaindur has quit IRC | 08:05 | |
*** jangutter has joined #openstack-nova | 08:05 | |
brinzhang | takashin: so I asked for that, if so, I will push a new patch for the migration-list | 08:06 |
takashin | brinzhang: You can take over it. | 08:06 |
*** threestrands has quit IRC | 08:06 | |
brinzhang | takashin: cool, I will have a try, and remove the cold migration deal, and add my requirement. | 08:08 |
*** mauro|call has quit IRC | 08:13 | |
*** mauro|call has joined #openstack-nova | 08:14 | |
*** panda has quit IRC | 08:22 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: Use os-brick locking for volume attach and detach https://review.opendev.org/614190 | 08:23 |
*** panda has joined #openstack-nova | 08:23 | |
*** cdent has joined #openstack-nova | 08:23 | |
*** ralonsoh has joined #openstack-nova | 08:26 | |
*** derekh has joined #openstack-nova | 08:29 | |
*** takashin has left #openstack-nova | 08:30 | |
*** tetsuro has quit IRC | 08:35 | |
*** ralonsoh has quit IRC | 08:35 | |
*** tetsuro has joined #openstack-nova | 08:36 | |
*** huaqiang has joined #openstack-nova | 08:37 | |
*** betherly has joined #openstack-nova | 08:45 | |
*** ralonsoh has joined #openstack-nova | 08:47 | |
*** markvoelker has joined #openstack-nova | 08:50 | |
*** markvoelker has quit IRC | 08:55 | |
*** mauro|call has quit IRC | 08:58 | |
*** mauro|call has joined #openstack-nova | 09:00 | |
*** tkajinam has quit IRC | 09:02 | |
*** ociuhandu has joined #openstack-nova | 09:03 | |
*** ociuhandu has quit IRC | 09:13 | |
*** mauro|call is now known as takamatsu | 09:14 | |
*** ociuhandu has joined #openstack-nova | 09:16 | |
*** tetsuro has quit IRC | 09:19 | |
openstackgerrit | Chris Dent proposed openstack/nova master: Add a document that describes profiling eventlet services https://review.opendev.org/674865 | 09:22 |
*** ociuhandu has quit IRC | 09:25 | |
bhagyashris | stephenfin: Hi , Good morning :) | 09:28 |
stephenfin | bhagyashris: Evening o/ | 09:29 |
*** ociuhandu has joined #openstack-nova | 09:29 | |
bhagyashris | stephenfin: I saw you have uploaded patches including the upgrade changes, But I found few issues after applying all the patches. #link: http://paste.openstack.org/show/755606/ | 09:29 |
bhagyashris | stephenfin: noted issues in above #link path | 09:30 |
*** mkrai has quit IRC | 09:33 | |
bhagyashris | stephenfin: As per discussion and mentioned in spec we are not going to support both the PCPU and VCPU in single boot request so IMO we will need to add check some where to reject such request with proper error message to user. | 09:34 |
stephenfin | bhagyashris: Yeah, I should have a WIP flag on that. There's a few corner cases I still need to cover including that one | 09:34 |
stephenfin | You're correct about the pinning though. I guess that's because we need to set the 'cpu_policy' field on the 'InstanceNUMATopology' object | 09:35 |
*** mkrai has joined #openstack-nova | 09:36 | |
bhagyashris | stephenfin:yeah | 09:36 |
bhagyashris | stephenfin: And also I would like to know about are we going tp supporting new syntax of flavor extra specs like resources:PCPU and resources:VCPU in Train? | 09:37 |
bhagyashris | S/tp/to | 09:38 |
stephenfin | That would require changes to the InstanceNUMATopology object, but like we discussed previously, they don't need to be done yet | 09:39 |
bhagyashris | stephenfin: if yes then in that case we will also need to take care the first issue that I have mentioned above ... gust xml will give wrong result if user will create the instance using flavor extra specs resources:PCPU=1 | 09:40 |
stephenfin | Yup. Indeed. That needs to be done regardless. | 09:41 |
bhagyashris | stephenfin: OK | 09:42 |
bhagyashris | stephenfin: and one more thing I would like to ask about the translation of flavor extra specs and image meta data .. are you going to use my change https://review.opendev.org/#/c/672223/1 or you are going to continue on your change? | 09:42 |
stephenfin | I think I'm going to keep going with https://review.opendev.org/#/c/671801/. I don't like the idea of rewriting the flavor behind the users back and this change doesn't require that. I might change my mind again, but I'll go with this for now | 09:44 |
bhagyashris | stephenfin: OK No problem. | 09:45 |
stephenfin | bhagyashris: Thanks for testing this stuff, btw. Really good to have as many eyes on it as possible, as we're seeing :) | 09:45 |
bhagyashris | stephenfin: If you are OK? then I will take up adding the functional test cases and will fix the existing test that you have added for https://review.opendev.org/#/c/674895/2 | 09:47 |
stephenfin | bhagyashris: Yes, please. I kept the Author the same on that. I'm going to be working on the previous two patches and adding support for reporting the 'HW_CPU_HYPERTHREADING' trait today | 09:48 |
bhagyashris | stephenfin: OK then I will take up this task and will upload the patch soon | 09:48 |
*** jistr|afk is now known as jistr | 09:54 | |
*** sapd1_x has joined #openstack-nova | 09:54 | |
*** ricolin_ has joined #openstack-nova | 10:01 | |
bhagyashris | stephenfin: Thank you :) | 10:01 |
*** bhagyashris has quit IRC | 10:03 | |
*** ricolin has quit IRC | 10:03 | |
*** spatel has joined #openstack-nova | 10:27 | |
*** helenafm has quit IRC | 10:29 | |
*** udesale has quit IRC | 10:31 | |
*** spatel has quit IRC | 10:31 | |
*** mvkr has joined #openstack-nova | 10:36 | |
*** sean-k-mooney has quit IRC | 10:52 | |
*** jangutter_ has joined #openstack-nova | 10:55 | |
*** jangutter has quit IRC | 10:58 | |
*** ociuhandu has quit IRC | 11:07 | |
*** mkrai has quit IRC | 11:08 | |
*** jangutter_ has quit IRC | 11:08 | |
*** sean-k-mooney has joined #openstack-nova | 11:09 | |
*** jangutter has joined #openstack-nova | 11:12 | |
*** jangutter has quit IRC | 11:14 | |
*** jangutter has joined #openstack-nova | 11:38 | |
*** jangutter has quit IRC | 11:40 | |
sean-k-mooney | stephenfin: gibi alex_xu https://review.opendev.org/#/c/621646/23 has been around for a while and it hasnt gotten any core review. i have review this several times at this point, would ye be able to take a look? | 11:40 |
*** boxiang has quit IRC | 11:42 | |
*** boxiang has joined #openstack-nova | 11:42 | |
*** jangutter has joined #openstack-nova | 11:43 | |
*** jangutter has quit IRC | 11:45 | |
gibi | sean-k-mooney: I opened it now on a tab but I cannot promies to reach there today | 11:48 |
sean-k-mooney | gibi: no worries i was pingged to review it yesterday so i did it this morning | 11:48 |
*** markvoelker has joined #openstack-nova | 11:49 | |
sean-k-mooney | i just wanted to flag it to some people since it was opened back in december and they have been rebasing it since | 11:49 |
*** mkrai has joined #openstack-nova | 11:58 | |
*** mgariepy has joined #openstack-nova | 11:58 | |
*** jangutter has joined #openstack-nova | 11:59 | |
alex_xu | sean-k-mooney: I added it to my review list, will reach it when I do review work this week | 12:03 |
sean-k-mooney | alex_xu: cool thanks | 12:05 |
*** ratailor has quit IRC | 12:05 | |
*** shilpasd has joined #openstack-nova | 12:06 | |
stephenfin | sean-k-mooney: sure | 12:07 |
*** ociuhandu has joined #openstack-nova | 12:09 | |
sean-k-mooney | speaking of reveiw where are we at with https://review.opendev.org/#/c/668656/7/specs/train/approved/use-pcpu-vcpu-in-one-instance.rst | 12:10 |
sean-k-mooney | oh wrong version | 12:11 |
*** markvoelker has quit IRC | 12:11 | |
sean-k-mooney | all the discusstion seams to be on 7 but the patch is on version 11 | 12:11 |
*** ociuhandu has quit IRC | 12:14 | |
sean-k-mooney | i think the table for the setting in the image vs flavor is wrong by the way | 12:14 |
*** markvoelker has joined #openstack-nova | 12:16 | |
*** ociuhandu has joined #openstack-nova | 12:19 | |
*** ociuhandu has quit IRC | 12:22 | |
*** ociuhandu has joined #openstack-nova | 12:23 | |
openstackgerrit | Alex Xu proposed openstack/nova master: Add functional tests for resize with VGPU https://review.opendev.org/675106 | 12:26 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: scheduler: Flatten 'ResourceRequest.from_extra_specs', 'from_image_props' https://review.opendev.org/674894 | 12:27 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | 12:27 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | 12:27 |
*** kaisers has joined #openstack-nova | 12:28 | |
*** jistr is now known as jistr|call | 12:37 | |
*** ociuhandu has quit IRC | 12:38 | |
*** ociuhandu has joined #openstack-nova | 12:39 | |
alex_xu | sean-k-mooney: dansmith also has a point about whether we should let the user specific which vcpu is dedicated | 12:40 |
*** helenafm has joined #openstack-nova | 12:42 | |
openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Nova object changes for forbidden aggregates request filter https://review.opendev.org/671072 | 12:43 |
openstackgerrit | Shilpa Devharakar proposed openstack/nova master: DB API changes to get non-matching aggregates from metadata https://review.opendev.org/671074 | 12:45 |
*** zbr has quit IRC | 12:46 | |
openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Add a new request filter to forbidden aggregates https://review.opendev.org/671075 | 12:46 |
*** ociuhandu has quit IRC | 12:46 | |
sean-k-mooney | alex_xu: dansmith the reason for being able to define which cpus are dedicated is the same for why we allow it for realtime cpus. we want to be able to say that core x is low performace and can be used for the os/manament interfaces vs core y is high perfomace and can be used for the application | 12:47 |
openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Enable request filter forbidden_aggregates https://review.opendev.org/667952 | 12:48 |
*** ociuhandu has joined #openstack-nova | 12:48 | |
sean-k-mooney | alex_xu: dansmith: so yes we do need to allow the user to pass a mask in my opipion and i dont think nova should do any magic to try and calualte it. | 12:48 |
*** ricolin_ is now known as ricolin | 12:48 | |
alex_xu | sean-k-mooney: hint by you, if I want the realtime cpu to be dedicated, then I still need a way to define that. | 12:48 |
sean-k-mooney | if the user has multiple numa nodes and/or a specific cpu toplogy then they shoudl understand that mapping | 12:49 |
sean-k-mooney | well that is simple. use the same mask for both | 12:49 |
alex_xu | yea, otherwise, we will have a magic to mapping realtime vcpu to dedicated vcpu inside nova | 12:49 |
sean-k-mooney | but really i dont think we should allow realtime core to not be dedicated | 12:49 |
alex_xu | ah, that make sense. but it still will be a magic | 12:50 |
sean-k-mooney | if you are declaring a core as realtime it does not make sense for it to be multiplex to a shared host core that other vms can execute on as that will violate any realtiem logic | 12:51 |
sean-k-mooney | well we just and the bitmasks | 12:51 |
sean-k-mooney | actuuly xor the bits and check the result is 0 | 12:51 |
sean-k-mooney | actully no that is also wrong | 12:52 |
sean-k-mooney | but you get the idea we just need to validate each bit set in the realtime mask is set in the dedicated mask | 12:53 |
alex_xu | yea | 12:53 |
alex_xu | i got it | 12:53 |
sean-k-mooney | to dansmith point however i do belive we discusssed the idea of not exposing a mask in the past and instead doing what he say and just having tally counts of how many of each type you want | 12:55 |
sean-k-mooney | then evenly distubting them over the numa nodes and ensureing within a numa node the shared cpus are first followed by the pinned ones | 12:55 |
sean-k-mooney | that fits the paradime where core 0 is often reserved for the os | 12:56 |
sean-k-mooney | and in a multi numa works the first core on each numa node is often reserved for the os | 12:56 |
sean-k-mooney | but that does seam like magic that would easily break | 12:57 |
alex_xu | good to know | 13:00 |
* alex_xu has too much fun with functional test and vgpu and resize, can't open the eye | 13:01 | |
sean-k-mooney | alex_xu: i saw the irc ping for the patch. what tempeted you to work on that | 13:01 |
sean-k-mooney | also what time zone are you in. i assume its eitehr quite late or very early for you | 13:02 |
*** zbr has joined #openstack-nova | 13:03 | |
alex_xu | sean-k-mooney: hah | 13:05 |
*** mriedem has joined #openstack-nova | 13:10 | |
*** udesale has joined #openstack-nova | 13:19 | |
openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Enable request filter isolate_aggregates https://review.opendev.org/667952 | 13:19 |
openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: docs: clarify nova migration-list --host option https://review.opendev.org/674950 | 13:23 |
openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: WIP: Add --migration-type and --source-compute to migration-list https://review.opendev.org/675117 | 13:23 |
mriedem | lyarwood: as you're probably aware once these land https://review.opendev.org/#/q/I5e0a43ec59341c9ac62f89105ddf82c4a014df81 we'll be doing stable branch releases | 13:28 |
sean-k-mooney | mriedem: those are the cve backports right | 13:29 |
mriedem | yes | 13:29 |
*** mkrai has quit IRC | 13:29 | |
mriedem | dansmith: can you get this this stein backport? https://review.opendev.org/#/c/672723/ | 13:29 |
*** mkrai has joined #openstack-nova | 13:29 | |
mriedem | melwitt: can you get these stein backports for a series you're familiar with https://review.opendev.org/#/q/topic:bug/1764556+(status:open+OR+status:merged)+branch:stable/stein | 13:30 |
lyarwood | mriedem: ack yup | 13:30 |
dansmith | mriedem: jes | 13:30 |
mriedem | thanks | 13:31 |
*** mkrai has quit IRC | 13:35 | |
*** BjoernT_ has joined #openstack-nova | 13:36 | |
mnaser | can i get a second pair of eyes | 13:42 |
mnaser | i think there is a nova bug here that we uncovered in OSA | 13:42 |
mnaser | https://logs.opendev.org/33/660333/10/check/openstack-ansible-deploy-aio_metal-ubuntu-bionic/97d8bc3/logs/host/nova-compute.service.journal-23-20-40.log.txt.gz#_Aug_06_22_51_25 | 13:42 |
mnaser | nova-compute magically trying to access db? | 13:42 |
mnaser | (and raising an exception because of that) | 13:43 |
*** jaosorior has quit IRC | 13:44 | |
*** factor has quit IRC | 13:44 | |
*** factor has joined #openstack-nova | 13:45 | |
yonglihe | <sean-k-mooney: Hope you have time take a look the Orphan clean up patch, which i revised, but busy on another patch last week. thanks. https://review.opendev.org/#/c/627765/ | 13:45 |
sean-k-mooney | am i can make time. i saw melwitt had a question when i looked last week about shutdown_unkown. was that resolved | 13:47 |
mnaser | im trying to do git blame on github and its timing out | 13:47 |
mnaser | boo | 13:47 |
mriedem | mnaser: hmm, yeah compute is calling compute.API.stop to stop the instance, which is hitting some code in the compute rpcapi client router to determine the minimum compute version across all cells which it shouldn't need to be doing, b/c you're in a cell | 13:47 |
mnaser | thats what i figured based on the traceback but i havent done my share of git blame yet to see when this might be started | 13:48 |
mnaser | https://github.com/openstack/nova/blame/master/nova/compute/manager.py crashes github | 13:48 |
*** ociuhandu has quit IRC | 13:48 | |
mriedem | mnaser: https://review.opendev.org/#/c/649197/ | 13:48 |
sean-k-mooney | mnaser: ya that and the libvirt dirver are two of the files better blamed on the terminal | 13:48 |
*** ociuhandu has joined #openstack-nova | 13:49 | |
mriedem | i want to say we should be hitting that when compute starts up though, or used to? | 13:49 |
mriedem | mnaser: do you have the api db configured in nova.conf that you're computes are using? | 13:49 |
mnaser | mriedem: in this case i believe its an aio so yes it will be there | 13:50 |
mnaser | in an actual deployment, osa doesnt do this | 13:50 |
mriedem | even if it's a single node you can run nova-compute with a different config file | 13:50 |
mriedem | w/o the db creds | 13:50 |
mriedem | that's what devstack does | 13:50 |
dansmith | mnaser: is the api db connection set? | 13:50 |
dansmith | I think that's how it determines if it should use the all-cells or this-cell method | 13:50 |
mnaser | yeah.. it would be a very 'CI' specific thing if we used 2 different configs, unless we always used 2 configs, but yeah | 13:51 |
mnaser | if we have to roll out nova-compute.conf and nova.conf -- that's fine, we'll fix that | 13:51 |
sean-k-mooney | mnaser: well always using two config makes sense in a way | 13:51 |
dansmith | mnaser: was that aimed at me? | 13:51 |
sean-k-mooney | it is nice to be able to seperate compute only things for contoler only things | 13:52 |
mnaser | dansmith: just in general, im wondering if we're doing things wrong (i.e. nova operates with the assumption that api db _should not_ be set in nova-compute) | 13:52 |
*** mkrai has joined #openstack-nova | 13:52 | |
mriedem | i've got to run my kid somewhere, bbiab | 13:52 |
dansmith | mnaser: the assumption is that computes should not be talking to any db, and thus shouldn't have creds set at all | 13:52 |
dansmith | mnaser: like, the creds shouldn't even need to be on those systems | 13:52 |
sean-k-mooney | the compute talkign directly to a db was a legacy feature correect | 13:53 |
sean-k-mooney | its inteded to go via the conductor | 13:53 |
mnaser | right, in a 'production' deployment of OSA (where compute != control), the value is not set, but in our aio, the logic is compute is inside the controller group so it gets added | 13:53 |
mnaser | i agree with you that it should not be set | 13:53 |
dansmith | mnaser: okay, but it should have a different config which matches what a compute looks like non-aio | 13:53 |
mnaser | https://github.com/openstack/openstack-ansible-os_nova/blob/master/templates/nova.conf.j2#L196-L212 | 13:54 |
mnaser | so we only set these if we're running against a host that is not part of nova-compute group, but in this case in aio, every host is so it gets included | 13:54 |
dansmith | this is obviously failing because of a somewhat fragile assumption about what having api db creds set means, but.. this is why it's good to use realistic configs even in an aio situation | 13:54 |
sean-k-mooney | mnaser: do ye share config files between services? | 13:55 |
mnaser | sean-k-mooney: yes, but its templated to be different depending where it ends up getting deployed | 13:55 |
sean-k-mooney | e.g. i know kolla ansible generate a different nova.conf per nova service even on the same host | 13:55 |
mnaser | yes we do that too, but they all end up at /etc/nova/nova.conf | 13:56 |
sean-k-mooney | mnaser: right that is different then kolla ansible | 13:56 |
sean-k-mooney | right within the contianer | 13:56 |
sean-k-mooney | but different paths in the host right | 13:56 |
mnaser | nah its always at /etc/nova/nova.conf here | 13:56 |
*** tesseract has quit IRC | 13:56 | |
sean-k-mooney | so its alway one file on the host? | 13:56 |
mnaser | yep | 13:57 |
mnaser | i mean for me its fine, if from a nova perspective, the assumption that api_db shouldnt be defined, i can fix it inside osa, but if you think that assumption isn't right, i can also fix it inside nova | 13:57 |
*** tesseract has joined #openstack-nova | 13:57 | |
sean-k-mooney | ach well that explains why this does not work then | 13:57 |
*** whoami-rajat has quit IRC | 13:57 | |
sean-k-mooney | well the assumtion is no db will be defiend in the compute agent's nova.conf | 13:57 |
sean-k-mooney | all db acess form the comptue agent is expected to go via the cell conductor | 13:58 |
mnaser | yep, i agree with that | 13:58 |
dansmith | mnaser: I think you should change OSA so that AIO and non-AIO are effectively the same by using separate nova confs per service | 13:59 |
alex_xu | sean-k-mooney: can we have hw_cpu_policy as mixed, then have hw:cpu_dedicated_mask? | 14:00 |
*** tesseract has quit IRC | 14:00 | |
mnaser | dansmith: makes sense, i'll have us roll out config to nova-$service.conf inside OSA and i think that should clear this up | 14:00 |
mnaser | thanks for the help and heads up :>? | 14:00 |
dansmith | mnaser: cool | 14:00 |
*** tbachman has joined #openstack-nova | 14:00 | |
sean-k-mooney | for what its worth kolla-ansible still use one template but generates multiple files and this is how they guard the db section | 14:01 |
sean-k-mooney | https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova/templates/nova.conf.j2#L156-L166 | 14:01 |
*** tesseract has joined #openstack-nova | 14:01 | |
mnaser | yeah im thinking that probably is in line of what we'll end u pdoing | 14:02 |
sean-k-mooney | it makes the condional simpler as most are based on the service name instaead of the group you are in or if a service is enabled | 14:03 |
mnaser | yeah that does sound easier than a bunch of complicated jinja2 | 14:03 |
sean-k-mooney | but it requires the template to be evaulated and generated seperatly per service | 14:03 |
mnaser | yeah that does increase the # of times we have to do it but eh | 14:03 |
*** ociuhandu has quit IRC | 14:03 | |
mnaser | if its the right thing(tm) then i rather do that than a few less tasks | 14:04 |
sean-k-mooney | ya it synergies really nicely with kollas config overide mechanisum | 14:04 |
sean-k-mooney | perhaps it could be broken out at somepoint and shared between kolla-ansible, tripleo and osa | 14:05 |
yonglihe | sean-k-mooney: i reply melwitt but not sure is that what he concern. but thanks your review. | 14:05 |
*** ociuhandu has joined #openstack-nova | 14:06 | |
*** slaweq has quit IRC | 14:07 | |
*** slaweq has joined #openstack-nova | 14:08 | |
*** noonedeadpunk has joined #openstack-nova | 14:09 | |
*** betherly has quit IRC | 14:10 | |
*** ociuhandu has quit IRC | 14:11 | |
*** mvkr has quit IRC | 14:12 | |
*** dpawlik has quit IRC | 14:15 | |
dansmith | sean-k-mooney: you did not just say "it synergies" ... | 14:20 |
*** ociuhandu has joined #openstack-nova | 14:20 | |
*** spatel has joined #openstack-nova | 14:21 | |
sean-k-mooney | for once im using it an example of its usage in a non marketing bs context | 14:21 |
dansmith | heh | 14:22 |
sean-k-mooney | the config file per service per host design works realy well with the config override system kolla-ansible has. i think it was the config override desigin goals that acttuly motivated that approch initally so i guess its not suprising they work well together | 14:24 |
*** ociuhandu has quit IRC | 14:25 | |
*** mkrai has quit IRC | 14:26 | |
*** betherly has joined #openstack-nova | 14:29 | |
*** Luzi has quit IRC | 14:30 | |
noonedeadpunk | And does that make sense to create nova-conductor.conf and place db credentials only there? | 14:36 |
*** jistr|call is now known as jistr | 14:36 | |
dansmith | noonedeadpunk: devstack does a nova-cell1.conf for all the non-compute services that need to talk to the database, but yeah | 14:36 |
dansmith | nova.conf has everything, and is used by the control services, api, etc. nova-cell1 is no api, just cell db creds, and nova-compute.conf is the bare minimum for compute | 14:37 |
stephenfin | efried, gibi: What's the expected behavior here? http://paste.openstack.org/show/755623/ | 14:37 |
stephenfin | Maybe dansmith and sean-k-mooney too, idk ^ | 14:37 |
noonedeadpunk | dansmith: thanks | 14:37 |
sean-k-mooney | stephenfin: for legacy reasons that shoudl be 2 | 14:38 |
*** tbachman has quit IRC | 14:38 | |
sean-k-mooney | e.g. we have alway allowed that | 14:38 |
sean-k-mooney | but i dont like that you can change it | 14:38 |
*** betherly has quit IRC | 14:39 | |
sean-k-mooney | if we block it it could break existing flavors | 14:39 |
sean-k-mooney | so there would be an upgrade impact | 14:39 |
dansmith | sean-k-mooney: why for legacy reasons? | 14:39 |
mriedem | technically devstack has several conf files for nova because of the superconductor mode | 14:39 |
sean-k-mooney | dansmith: we decided to allow operators to specify resouce request in the flavor extra specs | 14:40 |
sean-k-mooney | that is the sole legacy use of this | 14:40 |
dansmith | sean-k-mooney: I'm not sure what you mean | 14:40 |
dansmith | resource requests take precedence over the base flavor values | 14:40 |
dansmith | that's new behavior | 14:40 |
dansmith | and critical for things like ironic | 14:40 |
sean-k-mooney | dansmith: no its not | 14:40 |
mriedem | yes...it is | 14:41 |
dansmith | where we "display" a value to the user via the flavor, but ask for something different from placement based on the overrides | 14:41 |
mriedem | as of pike | 14:41 |
sean-k-mooney | resouce request have always had precedece over the flavor value | 14:41 |
dansmith | sean-k-mooney: not what? | 14:41 |
mriedem | sean-k-mooney: you just said the same thing as dan | 14:41 |
stephenfin | there's a lot of drivers using the flavor.vcpus field | 14:41 |
dansmith | sean-k-mooney: right, which is the new behavior | 14:41 |
sean-k-mooney | dansmith: i was saying its not new it has been that way since we added resouces: in the flavor extra spec | 14:42 |
mriedem | anything new in nova in <3 years is "new" | 14:42 |
mriedem | :) | 14:42 |
sean-k-mooney | hehe :) | 14:42 |
dansmith | and new since the introduction of that thing | 14:42 |
stephenfin | including the libvirt driver | 14:42 |
mriedem | stephenfin: yes, and it's going to be a big problem for that pcpu stuff | 14:42 |
stephenfin | I mean, couldn't you request 'resources:VCPU=1' and 'flavor.vcpus=1000' ? | 14:43 |
sean-k-mooney | mriedem: not really | 14:43 |
sean-k-mooney | unless you ask for less | 14:43 |
stephenfin | and placement would be screwwwwed | 14:43 |
mriedem | you could add a property to the Flavor object that proxies the extra spec override when getting Flavor.vcpus | 14:43 |
sean-k-mooney | stephenfin: we allow it but i agree its not ideal | 14:43 |
stephenfin | especially if placement is thing that enforces proxies now | 14:43 |
sean-k-mooney | one of the origial usecase was to allow yo to do flavor.vcpu=4 resouce:vcpu=0 for ironic but we ended up going a different route | 14:44 |
stephenfin | sean-k-mooney: It feels slightly DDoS'y, at least if placement does quotas. I guess the thing that would prevent that is that flavor creation is admin-only but still | 14:44 |
*** ociuhandu has joined #openstack-nova | 14:45 | |
sean-k-mooney | e.g. so we could discribe the resouce an ironic flavor would have and then set the value to 0 and use the custome_resouce_class=1 internally | 14:45 |
*** altlogbot_3 has quit IRC | 14:46 | |
efried | # If any of the three standard resources above was explicitly given in | 14:46 |
efried | # the extra_specs - in any group - we need to replace it, or delete it | 14:46 |
efried | # if it was given as zero. | 14:46 |
dansmith | sean-k-mooney: what "different route" did we go? | 14:46 |
mriedem | stephenfin: read this for context https://docs.openstack.org/ironic/stein/install/configure-nova-flavors.html | 14:46 |
efried | regardless of the reason, we definitely coded it so that extra specs would override base amounts. | 14:46 |
sean-k-mooney | dansmith: we went with setting the flavor vlaues to 0 and just useing the custom resouce class | 14:46 |
dansmith | sean-k-mooney: no we didn't | 14:46 |
dansmith | sean-k-mooney: see that doc mriedem just posted | 14:47 |
mriedem | i think sean-k-mooney means the flavor values via the extra spec, | 14:47 |
mriedem | not the actual top-level flavor properties | 14:47 |
*** altlogbot_2 has joined #openstack-nova | 14:47 | |
dansmith | I don't think he does :) | 14:47 |
*** betherly has joined #openstack-nova | 14:47 | |
stephenfin | sean-k-mooney: We don't touch the flavor properties. We just ignore them | 14:47 |
dansmith | because he described what we *are* doing as the original plan that we abandoned | 14:47 |
*** mkrai has joined #openstack-nova | 14:47 | |
mriedem | i'm not sure you can even create a flavor with 0 vcpus or ram | 14:47 |
* stephenfin wonders what hell hole he's wandered into with this PCPU stuff... | 14:48 | |
mriedem | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/flavor_manage.py#L33 | 14:48 |
sean-k-mooney | no https://docs.openstack.org/ironic/stein/install/configure-nova-flavors.html is exactly the use case i though this was for but i though we ended up not doing it | 14:48 |
sean-k-mooney | ok good to know that actully was done | 14:48 |
mriedem | stephenfin: you didn't realize this 6 months ago? | 14:48 |
efried | What we need is to start over with a flavorsv2 | 14:48 |
efried | How long could it possibly take to deprecate and remove cellsv1, I mean flavorsv1? | 14:49 |
dansmith | efried: not helping | 14:49 |
stephenfin | mriedem: I knew it was bad but not this bad /o\ I'd estimate a solid 80% of the code I've written has been tech debt-related | 14:49 |
mriedem | the base flavor values for vcpu/ram/disk are *everywhere* and in *everything* in nova, as far as resource accounting is concerned | 14:50 |
mriedem | so changing the meaning of one of those has huge implications | 14:50 |
dansmith | yup | 14:50 |
sean-k-mooney | ya | 14:50 |
mriedem | the only one that was ever sort of special was disk for bfv | 14:50 |
mriedem | which dansmith finally fixed in i think rocky | 14:50 |
mriedem | by fixed i mean don't claim disk usage if you're bfv | 14:51 |
mriedem | and don't report to the hypervisors API that we're consuming disk for a bfv instance | 14:51 |
mriedem | ec | 14:51 |
mriedem | *etc | 14:51 |
sean-k-mooney | i personally am not sure why we still allow bfv with root_gb !=0 i get the upgrade impact of that but it would be nice to fully lock that down | 14:51 |
sean-k-mooney | but that is a differnet hill to die on | 14:52 |
sean-k-mooney | anyway for the vcpu stuff we never generate xml from the resouce:* values | 14:52 |
cdent | so many hills. so many deaths. | 14:52 |
sean-k-mooney | we only use thos for placmenet | 14:52 |
mriedem | unless i'm mistaken, | 14:53 |
mriedem | until this pcpu hullabaloo, | 14:53 |
mriedem | the only time you'd ever override vcpu/ram/disk with extra specs is for ironic flavors | 14:53 |
mriedem | for which the driver doesn't look at those anymore anyway | 14:53 |
sean-k-mooney | no wiht the pcpu stuff we still dont plan to generate the xml form resouce: we plan to use the extra_specs | 14:53 |
mriedem | resource: *is* extra specs | 14:53 |
stephenfin | Yeah, my life has been made significantly harder by having to care about 'resources:VCPU' and 'resources:PCPU' | 14:54 |
sean-k-mooney | but yes i think you are right the only time you would override them is ironic | 14:54 |
* mriedem remembers the ironic driver is still busted a bit on stable... | 14:54 | |
stephenfin | I'm seriously considering a big "this isn't doing anything unless you're ironic" warning and moving on | 14:54 |
stephenfin | efried: ^ | 14:54 |
mriedem | https://review.opendev.org/#/c/637217/ | 14:54 |
stephenfin | There's edge case after edge case | 14:55 |
sean-k-mooney | mriedem: it is but i mean we have hw:cpu_policy to determin if we translate flavor.vcpu to resouce:vcpu or resouces:pcpu | 14:55 |
sean-k-mooney | stephenfin: the proably is how do you know its an ironic flavor | 14:55 |
mriedem | sean-k-mooney: if you're talking about things in the plan in the spec, i haven't read it so don't know | 14:55 |
efried | stephenfin: what specifically? resources:VCPU/PCPU in extra specs? | 14:56 |
sean-k-mooney | mriedem: we knew trying to supprot resouces: for influncing the xml generation and assignment woudl be a paid soe we are not supporting it | 14:56 |
stephenfin | mriedem: tl;dr: if you specify 'hw:cpu_policy=dedicated', you now get 'PCPU' inventory, but we need to care about 'resources:PCPU' in addition | 14:56 |
stephenfin | efried: yup | 14:56 |
sean-k-mooney | *pain | 14:56 |
sean-k-mooney | efried: i would expand it to all resocues generated form flavor.* but the problem is we have no way to know if its an ironic flavor or not | 14:57 |
mriedem | umm, couldn't you just bork if you have hw:cpu_policy=dedicated and resources:VCPU? | 14:57 |
sean-k-mooney | so we cant | 14:57 |
*** factor has quit IRC | 14:57 | |
sean-k-mooney | we would still request resocue:PCU=flavor.vcpus in addtion to resources:VCPU | 14:57 |
mriedem | technically any ironic flavor is hw:cpu_policy=dedicated by default yeah? | 14:57 |
stephenfin | mriedem: Yup, and that's what I'm doing (ditto for explicit PCPU too). The top issue is flavor.vcpus != resources:PCPU | 14:58 |
mriedem | i mean you do'nt specify that, it's just implied b/c it's baremetal | 14:58 |
sean-k-mooney | stephenfin: there is one case where its not equal by design | 14:58 |
mriedem | you know, mogan wouldn't have this problem.... | 14:58 |
* mriedem ducks | 14:58 | |
*** factor has joined #openstack-nova | 14:58 | |
sean-k-mooney | which hw:emulator_threads_policy=isolate | 14:58 |
stephenfin | Also, 'hw:emulator_threads_policy=isolate' means "consume an extra core and put emulator threads there", so there it's (flavor.vcpus + 1) != resources:PCPU | 14:59 |
stephenfin | sean-k-mooney: jinx | 14:59 |
sean-k-mooney | mriedem: for ironic all cpus will still be reported using VCPU i think since we dont plan to update the driver | 14:59 |
sean-k-mooney | at least not unless someone asks | 14:59 |
sean-k-mooney | stephenfin: yes | 15:00 |
mriedem | mnaser: dansmith: looking at this trace again https://logs.opendev.org/33/660333/10/check/openstack-ansible-deploy-aio_metal-ubuntu-bionic/97d8bc3/logs/host/nova-compute.service.journal-23-20-40.log.txt.gz#_Aug_06_22_51_25 and where we hit it https://github.com/openstack/nova/blob/master/nova/compute/rpcapi.py#L422 , | 15:00 |
stephenfin | So this is all a massive PITA. TBH, I'd rather just say "don't use 'hw:cpu_policy' or any of those extra specs with 'resources:(P|V)CPU' and kick that can down the road | 15:00 |
stephenfin | efried: Could we do that for now? ^ | 15:00 |
sean-k-mooney | which if we kill that option entrily and just always map the emulator thread to the shared set in the futre will make your life simpeler | 15:00 |
mriedem | what do you think about adding a try/except for DBNotAllowed to just log something useful and re-raise, like "this service does not allow direct database access, you should run without the [api_database]/connection configured" | 15:00 |
stephenfin | sean-k-mooney: I'd be happy to do that but I need a cycle to do it | 15:01 |
stephenfin | Because we break everything otherwise, assuming the operator doesn't have cpu_shared_set configured everywhere | 15:01 |
sean-k-mooney | ya | 15:01 |
mriedem | sean-k-mooney: reported....where? | 15:01 |
mriedem | sean-k-mooney: for ironic the only inventory on the node should be the custom resource class reported by the driver's update_provider_tree method, which gets the resource class off the ironic node resource | 15:02 |
sean-k-mooney | mriedem: in placement | 15:02 |
sean-k-mooney | oh right | 15:02 |
mriedem | and the flavor should only consume, in placement, those custom resource class allocatoins | 15:02 |
sean-k-mooney | ya i forgot | 15:02 |
mriedem | that's why the vcpu/ram/disk are all zero'ed out | 15:02 |
mriedem | the values for those in the flavor are just cosmetic | 15:02 |
sean-k-mooney | so no change for ironic and hw:cpu_policy will have no effect | 15:02 |
sean-k-mooney | it should not be used but its meaning less | 15:03 |
mriedem | mnaser: if you have a bug for that osa issue i could tack on a nova logging patch to maybe make that more clear if someone hits that misconfig | 15:03 |
*** factor has quit IRC | 15:03 | |
*** tbachman has joined #openstack-nova | 15:03 | |
melwitt | mriedem: will do | 15:04 |
mnaser | mriedem: i think noonedeadpunk will likely look into fixing it so might be good if he created a bug to track it ^ | 15:05 |
efried | stephenfin: I would be fine limiting the use of resources:(P|V)CPU as strictly as possible without breaking backward compat. | 15:05 |
efried | but resources$S:* will be a different story. | 15:05 |
stephenfin | efried: What would the reason for 'resources$S:PCPU' be? | 15:05 |
stephenfin | I mean, without NUMA at least | 15:05 |
stephenfin | *NUMA in placement | 15:06 |
dansmith | mriedem: sure | 15:06 |
efried | stephenfin: Exactly NUMA in placement. | 15:07 |
efried | "will be" future tense intended | 15:07 |
sean-k-mooney | efried: why not block resources$S:* for pcus and vcpus | 15:07 |
efried | for now, yes, we certainly can ^ | 15:08 |
stephenfin | Why not forever? | 15:08 |
sean-k-mooney | we shoudl only support using the hw:numa_cpu.0 values | 15:08 |
sean-k-mooney | efried: e.g. we already have syntax for specify the cpu to numa mappings | 15:08 |
efried | stephenfin: pretty sure there will be affinity use cases where we'll need that level of control. | 15:08 |
stephenfin | Given that we have ways to request all these things with extra specs | 15:08 |
efried | including things like accelerators? | 15:09 |
efried | and VGPUs? | 15:09 |
efried | And bandwidth? | 15:09 |
efried | esp. that last one | 15:09 |
efried | because we need to be able to do affinity by subtree matching | 15:09 |
sean-k-mooney | if we also support mixed cpus in instance we will jsut combien the mask with the hw:numa_cpu.X=Y status | 15:09 |
sean-k-mooney | *spec | 15:09 |
noonedeadpunk | mnaser: I guess writing a patch might be quicker that filing in a bug... | 15:09 |
efried | ...when the requests come from different thingies | 15:09 |
huaqiang | sean-k-mooney: morning | 15:10 |
stephenfin | efried: 'hw:numa_affinity:$RESOURCE_CLASS:$POLICY' or something similar? | 15:10 |
stephenfin | e.g. 'hw:numa_affinity:VCPU:prefer' | 15:11 |
stephenfin | *VGPU | 15:11 |
gibi | I don't think we ever want to request bandwidth via flavor exta_spec | 15:11 |
efried | exactly gibi | 15:11 |
efried | but we still need to be able to express (NUMA) affinity | 15:12 |
stephenfin | We're not. We're just saying what the affinity of whatever the bandwidth resource class is | 15:12 |
stephenfin | Assuming we can know that ahead of time (idk) | 15:12 |
efried | by tying together the port bw request group suffix and the NUMA request group suffix | 15:12 |
efried | there's going to be a coupling problem for sure | 15:13 |
efried | like what we have for group_policy, only much worse. | 15:13 |
*** jaosorior has joined #openstack-nova | 15:14 | |
gibi | if nova start using the port uuid as the port suffix for the bandwidth request then a single use flavor can be created where the port group can be referenced for affinity. But this sounds pretty horrific | 15:17 |
gibi | s/port suffix/group suffix/ | 15:17 |
efried | agree, I don't think we want the suffix to be a port UUID, do we? | 15:18 |
gibi | efried: currently each port's resource request is understood as a separate group | 15:18 |
gibi | efried: so a natural suffix for the group could be the port uuid | 15:18 |
sean-k-mooney | stephenfin: why woudl we need hw:numa_affinity:VCPU:prefer | 15:19 |
*** ircuser-1 has joined #openstack-nova | 15:19 | |
stephenfin | sean-k-mooney: *VGPU | 15:19 |
sean-k-mooney | oh for graphics ya | 15:19 |
stephenfin | Just some way to define NUMA affinity for individual resource class types | 15:20 |
sean-k-mooney | cpus and memory are are affinites by adding hw:numa_node or one of the option that imples it so i was confused | 15:20 |
stephenfin | It doesn't allow you to define the same for individual instances, but that's slightly insane | 15:20 |
stephenfin | Yup, my mistake | 15:20 |
sean-k-mooney | well you would be surprised what we have approved in the past and never implmeneted. | 15:21 |
sean-k-mooney | stephenfin: https://github.com/openstack/nova-specs/blob/master/specs/newton/approved/virt-image-props-boot-override.rst | 15:22 |
efried | gibi: I guess it's not really relevant until we can tie in whatever piece of hardware is associated with the NUMA node. RP for a NIC PF, kind of thing. Cause today you don't request network VF resources or whatever, right? | 15:23 |
gibi | efried: right. we don't request VF resource from neutron via the resource request. | 15:24 |
gibi | efried: you are also correct that when VF will be requested from neturon along with the bandwidth then expressing NUMA affinity will be an issue to be solved | 15:25 |
efried | gibi: Won't the majority of affinity use cases involve only one NUMA node? | 15:25 |
sean-k-mooney | we request vf internally in nova currently based on the vif type | 15:26 |
efried | sean-k-mooney: yes but VF not tracked by placement yet | 15:26 |
efried | is my point | 15:26 |
sean-k-mooney | but sriov vf will be owned and created by nova in placmenet in the future | 15:26 |
sean-k-mooney | not by neutron | 15:26 |
efried | right | 15:26 |
efried | oh | 15:26 |
efried | oh? | 15:26 |
efried | That will make it tougher | 15:27 |
*** slaweq_ has joined #openstack-nova | 15:27 | |
sean-k-mooney | yes all pci decice handeling is currently done by nova | 15:27 |
efried | to affine the bw resources with the vf resources | 15:27 |
sean-k-mooney | neutron does not have a whitelist of device that can be used | 15:27 |
efried | what if it was a VIF resource? | 15:27 |
efried | Anyway, if we had a one-NUMA-node use case, we could handle this fairly well. Even multiple NUMA nodes if they're symmetrical. | 15:28 |
sean-k-mooney | we discussed adding vif ressource that would be owned by the neutron backend too | 15:28 |
efried | A case where I want $this specific VF/bw associated with $big_numa and $that specific VF/bw associated with $small_numa that things get really hairy. | 15:28 |
*** huaqiang has quit IRC | 15:28 | |
*** slaweq has quit IRC | 15:29 | |
sean-k-mooney | well nomally its not symetric | 15:29 |
*** huaqiang has joined #openstack-nova | 15:29 | |
sean-k-mooney | idealy it would be | 15:29 |
sean-k-mooney | but its quite common to only have nics attached to numa 0 | 15:29 |
efried | It's not? I thought that was the point of saying hw:numa_nodes=2 | 15:29 |
sean-k-mooney | no | 15:29 |
efried | Sorry, I mean if the CPU/mem resources in the NUMA nodes are symmetric | 15:29 |
efried | I don't mean "same number of VFs per NUMA node" | 15:30 |
sean-k-mooney | hw:numa_nodes=2 is refering to the guest numa toplogy | 15:30 |
*** factor has joined #openstack-nova | 15:30 | |
sean-k-mooney | oh by default yest | 15:30 |
sean-k-mooney | *yes | 15:30 |
efried | when we do that we're dividing evenly, yah? | 15:30 |
sean-k-mooney | you can change that with hw:numa_mem.0=512 hw:numa_mem.1=1536 | 15:31 |
huaqiang | sean-k-mooney: hi | 15:31 |
efried | sean-k-mooney: Yes, exactly, that's my point: those cases are rarer, right? | 15:31 |
sean-k-mooney | efried: yes if you jsut do hw:numa_nodes it devices cpus and ram evenly | 15:31 |
sean-k-mooney | we also check its evenly deivisable | 15:31 |
efried | sean-k-mooney: Do we currently support VF NUMA affinity? | 15:31 |
sean-k-mooney | efried: yes | 15:31 |
sean-k-mooney | for 4+ years | 15:32 |
sean-k-mooney | we require it in the case of neutron sriov which is what im trying to chagne | 15:32 |
sean-k-mooney | with https://review.opendev.org/#/c/674072/ | 15:32 |
sean-k-mooney | gibi: ^ i would like your input on that by the way | 15:33 |
sean-k-mooney | huaqiang: hi | 15:33 |
efried | sean-k-mooney: even in the case of lopsided NUMA nodes? You can say which one you want your VF associated with? | 15:33 |
sean-k-mooney | efried: no | 15:33 |
sean-k-mooney | you cannot specify that at all | 15:33 |
efried | okay, then we're fine. | 15:34 |
efried | we continue to say you can't specify that. | 15:34 |
sean-k-mooney | we guarentee it will come form one of the numa ndoes the vm is mapped too | 15:34 |
sean-k-mooney | but we give no garuntee it on a specific guest numa node | 15:34 |
*** _mlavalle3 has quit IRC | 15:34 | |
gibi | efried: if nova keeps handling the VFs then nova can do a VF-NUMA affinity. Bandwidth is already tied to VF today in case of SRIOV. | 15:34 |
*** mlavalle has joined #openstack-nova | 15:35 | |
sean-k-mooney | currently its always preseneted as if its attache to guest numa node 0 regradelss of what host numa node it came from | 15:35 |
efried | and we can give the requests for NUMA nodes and VFs generic known suffixes, and use same_subtree to tie them together. | 15:35 |
efried | perfect | 15:35 |
sean-k-mooney | i also want to chagne that but not in a way tha will impact placmenet or shcudleing in any way | 15:35 |
efried | Yeah, let's not give any more granular control over that | 15:35 |
efried | cause then we would have to support it via placement :) | 15:35 |
efried | Okay, it's the middle of the night for huaqiang, let's wrap up and yield the floor... | 15:36 |
sean-k-mooney | efried: what i want to add i the use of a livirt feature that allows mupliple pci roots in the guest | 15:36 |
huaqiang | sean-k-mooney: got the comments for the 'using PCPU VCPU in one instance', thanks | 15:36 |
sean-k-mooney | that will allwo us to attach the pci device to the virtual numa node that maps to the host numa node the deive is on | 15:36 |
huaqiang | but I checked the existing logic for 'hw:cpu_policy=dedicated' and 'hw_cpu_policy=shared', the final policy is 'dedicated' | 15:37 |
sean-k-mooney | can you show me that in code | 15:38 |
huaqiang | sure | 15:38 |
sean-k-mooney | its this https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1327-L1338 | 15:39 |
sean-k-mooney | ok we do perfer the flavor over the image in that case | 15:39 |
huaqiang | you find it! | 15:40 |
sean-k-mooney | ya i knew what code shoudl be enforcing it | 15:40 |
sean-k-mooney | its incositent with how we handel hw:cpu_threads_policy | 15:40 |
sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1370-L1377 | 15:40 |
sean-k-mooney | and numa topligies https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1290 | 15:41 |
sean-k-mooney | had thought we had made it an error for all conflitc at this point | 15:41 |
sean-k-mooney | we can chagne that without at least 1 cycle to deprecate the old behavior so i guess we have to stick with the current behavior | 15:42 |
*** sapd1_x has quit IRC | 15:42 | |
sean-k-mooney | huaqiang: it would be good to add a warning here https://github.com/openstack/nova/blob/master/nova/virt/hardware.py#L1328 as i dont think that is what people expect | 15:42 |
huaqiang | so you mean the flavor first behavior | 15:43 |
sean-k-mooney | yes | 15:43 |
sean-k-mooney | we started moving to raising exception if they conflict for most of these value a whiel ago | 15:43 |
sean-k-mooney | at least when adding new values | 15:43 |
sean-k-mooney | i guess we did not change older values proably due to upgrade concerns | 15:44 |
huaqiang | so for the new 'mixed' policy, we need consider conflict as invalid | 15:45 |
*** helenafm has quit IRC | 15:45 | |
huaqiang | but for 'dedicated' and 'shared' conflicts we follow the rule that flavor first | 15:45 |
sean-k-mooney | well we have a choice | 15:45 |
sean-k-mooney | my perference would be to consider all conflcit of hw:cpu_policy and hw_cpu_policy to be invalid and raise an exception | 15:46 |
sean-k-mooney | but since we cant do that htis cycle | 15:46 |
sean-k-mooney | its better to be consistent i think | 15:46 |
sean-k-mooney | it would be nice to issue a warning if we detect a conflict however | 15:47 |
huaqiang | I am good for both | 15:47 |
huaqiang | ok | 15:47 |
huaqiang | another thing i don't know if you noticed that | 15:48 |
huaqiang | hw:cpu_realtime exists | 15:48 |
huaqiang | but there is no 'hw_cpu_realtime' in image properties | 15:48 |
huaqiang | this is different with the mask for dedicated cpus | 15:48 |
huaqiang | Alex mentioned this difference, and cpu realtime mask has a more loose rules | 15:49 |
*** icarusfactor has joined #openstack-nova | 15:50 | |
sean-k-mooney | yes we did not want to allow users to request realtime guest via the image | 15:50 |
sean-k-mooney | but we did allow the mask i belive | 15:50 |
huaqiang | mask is allowed from both image and flavor | 15:50 |
sean-k-mooney | so you can only enabel realtime cpus via the flavor but both the flavor and image can set the mask | 15:51 |
huaqiang | that is the truth | 15:51 |
sean-k-mooney | we have allowed image to enable cpu pinning provided the the flavor has not forbiding it by setting hw:cpu_policy=shared | 15:52 |
*** factor has quit IRC | 15:52 | |
sean-k-mooney | so i think its fine wto allow hw_cpu_policy=mixed give hw_cpu_policy=dedicated is alredy allowed | 15:52 |
sean-k-mooney | similarly i think having the mask in the image and flavor is ok | 15:52 |
mkrai | Hi Nova team, I am working on implementing the resize API for baremetals and need some help in the design. I tried to put an initial design here https://etherpad.openstack.org/p/nova-resize-ironic-api | 15:53 |
sean-k-mooney | but im sure someone else has an opipion on that ^ its what the spec currently say however | 15:53 |
gibi | sean-k-mooney: left comment in https://review.opendev.org/#/c/674072 about the notification tests | 15:54 |
huaqiang | yes, dansmith is questioned on the masks | 15:54 |
sean-k-mooney | gibi: thanks i can never rememebr where that is | 15:54 |
*** icarusfactor has quit IRC | 15:55 | |
mkrai | efried, sean-k-mooney mriedem alex_xu could you please take a look? | 15:55 |
sean-k-mooney | im fixing other typos but was about to spend the next hour finding it | 15:55 |
efried | mkrai: Hi there. I recall mriedem had some ideas about that design. dansmith could probably provide some good input as well. To be clear, this is going to be a U release thing, you're just looking for initial design help? | 15:55 |
*** whoami-rajat has joined #openstack-nova | 15:55 | |
sean-k-mooney | mkrai: yes i have it open | 15:55 |
gibi | sean-k-mooney: we might improve the test to dump the file path when the assert fails | 15:55 |
*** mkrai_ has joined #openstack-nova | 15:55 | |
mkrai | efried, Yes! | 15:55 |
mkrai | sean-k-mooney, Thanks! | 15:56 |
huaqiang | dansmith: hello | 15:56 |
*** ralonsoh has quit IRC | 15:57 | |
*** ralonsoh has joined #openstack-nova | 15:57 | |
sean-k-mooney | efried: mkrai could start on any ironic internal prep work that was need this cycle but the ironic driver changes would be next cycle i guess | 15:58 |
*** tesseract has quit IRC | 15:58 | |
mriedem | i love being continually distracted about asking to review specs and plans and code for things that do not even have a chance for train | 15:59 |
mkrai | sean-k-mooney, I have already posted spec for the ironic reconfigure API that will be needed by the resize API | 15:59 |
sean-k-mooney | gibi: are you ok with https://review.opendev.org/#/c/674072 adressing bug: #179520 as a bugfix to the original spec | 15:59 |
openstack | bug 179468 in python-cups (Ubuntu) "duplicate for #179520 applet.py crashed with TypeError in on_job_hold_activate()" [High,Fix released] https://launchpad.net/bugs/179468 - Assigned to Pascal De Vuyst (pascal-devuyst) | 15:59 |
mriedem | meanwhile my mega cross-cell resize series is not getting review but has been up for 2 cycles | 15:59 |
mkrai | sean-k-mooney, https://review.opendev.org/#/c/672252/ | 16:00 |
mkrai | I will add this link to the etherpad | 16:00 |
efried | mriedem: I'm going out on a limb and saying distraction is not the most significant factor in lack of reviews on cross-cell resize. | 16:00 |
efried | At least for me that's true | 16:00 |
gibi | sean-k-mooney: yeah I'm OK with that being a bugfix. Maybe a link to the original spec / bp in the commit message would help | 16:01 |
sean-k-mooney | mriedem: ^ that question is also to use as well are you ok with https://review.opendev.org/#/c/674072 adressing the feature gap in the orginal spec. | 16:01 |
efried | "I'm askeird" is the major reason | 16:01 |
mriedem | efried: i'm not saying it's a walk in the park | 16:01 |
sean-k-mooney | gibi: sure ill add one | 16:01 |
efried | fwiw, I have been intending to wade in | 16:01 |
mriedem | that's why i had integration testing of cross-cell resize working before putting out a review guide | 16:01 |
efried | but procrastinating successfully so far | 16:01 |
mriedem | which is more than i can say for a lot of the features we merge... | 16:02 |
efried | yeah yeah | 16:02 |
mriedem | i'm also a bit disappointed that cern hasn't been more involved | 16:02 |
mriedem | but they seem...distracted lately as well | 16:02 |
*** tesseract has joined #openstack-nova | 16:02 | |
mriedem | sean-k-mooney: idk even know what https://review.opendev.org/#/c/674072 is about | 16:03 |
mriedem | but it looks like it's not fun | 16:03 |
sean-k-mooney | we only allow you to specify the numa affinity policy for pci device in teh pci alias. the spec origially had flavor and image properites for it | 16:04 |
sean-k-mooney | neutron sriov dose not use pci alias | 16:04 |
mriedem | which original spec | 16:04 |
mriedem | ? | 16:04 |
sean-k-mooney | so we entirely broke the original neuton port usecase. | 16:04 |
sean-k-mooney | one sec ill get it | 16:04 |
mriedem | "lack of a feature is not a bug" | 16:05 |
sean-k-mooney | https://github.com/openstack/nova-specs/blob/master/specs/queens/implemented/share-pci-between-numa-nodes.rst | 16:05 |
sean-k-mooney | https://github.com/openstack/nova-specs/commit/74b47ac2f968b13b7d15987063d0f52ecba02a7e#diff-b5d95cd01ee920f26b55b45f2777b7aa is wher whe chaned it | 16:05 |
*** gyee has joined #openstack-nova | 16:05 | |
mriedem | ok i'm pretty sure i didn't review any of this https://review.opendev.org/#/q/topic:bp/share-pci-between-numa-nodes+(status:open+OR+status:merged) | 16:06 |
mriedem | looks like it was jay, sylvain and gibi | 16:06 |
mriedem | i do, however, always enjoy the late "oh btw this thing we landed 3 releases ago didn't do what it was supposed to but our QE team just got around to testing it" | 16:06 |
sean-k-mooney | mriedem: that is true which is why i want non redhat core to basicaly decide if this is valid to adress as a bugfix as redhat folk have a conflict of interest | 16:06 |
sean-k-mooney | well we knew it was broken for a while but planned to fix it in U | 16:07 |
*** igordc has joined #openstack-nova | 16:07 | |
sean-k-mooney | we have a custom request to support this that came in last week e.g. after spec freeze | 16:07 |
mriedem | so you want to slide it in as a bug fix? | 16:08 |
sean-k-mooney | so this is why im asking is this something we can address upstream in Train | 16:08 |
mriedem | you know i'm not going to let you backport this right? | 16:08 |
*** maciejjozefczyk has quit IRC | 16:09 | |
sean-k-mooney | well we do actully have a bug for it but i also added the wrong one | 16:09 |
sean-k-mooney | mriedem: and yes i know | 16:09 |
mriedem | this smells like a blueprint to me, not a bug | 16:09 |
*** tssurya has quit IRC | 16:10 | |
sean-k-mooney | given we are pass m2 is this something that people would be willing to consider of train as a blueprint | 16:10 |
sean-k-mooney | i am happy to file one and ask for a blueprint exception | 16:10 |
mriedem | christ | 16:11 |
mriedem | tell you what, if you can lean on dansmith and he can lean on me then we can talk | 16:11 |
mriedem | but i just -1ed this as needing a blueprint | 16:11 |
openstackgerrit | Merged openstack/nova stable/stein: Replace non-nova server fault message https://review.opendev.org/674828 | 16:11 |
sean-k-mooney | mriedem: i am also ok with punting it but that just forces me to consider downstrema only way to adres this. | 16:11 |
dansmith | wut | 16:11 |
openstackgerrit | Merged openstack/nova stable/stein: Revert "[libvirt] Filter hypervisor_type by virt_type" https://review.opendev.org/672723 | 16:11 |
*** mkrai_ has quit IRC | 16:11 | |
mriedem | dansmith: "we forgot to do a thing in queens" https://review.opendev.org/#/c/674072/ | 16:12 |
*** efried is now known as efried_rollin | 16:12 | |
dansmith | mriedem: if a config flag to disable certain processor flags for the biggest SEV in decades is a feature, then ffs, this surely is | 16:13 |
sean-k-mooney | this is the orignal bug report by the way https://bugs.launchpad.net/nova/+bug/1795920 | 16:13 |
openstack | Launchpad bug 1795920 in OpenStack Compute (nova) "SR-IOV shared PCI numa not working " [Undecided,Confirmed] | 16:13 |
melwitt | since everybody's in a good mood this morning, think it's a good time to ask for reviews on multi-cell archive_deleted_rows https://review.opendev.org/507486 | 16:13 |
*** shilpasd has quit IRC | 16:15 | |
*** betherly has quit IRC | 16:18 | |
mriedem | RAGE | 16:20 |
stephenfin | Terrible game, that | 16:20 |
mriedem | oh right you wanted to backport that too... :) | 16:21 |
*** maciejjozefczyk has joined #openstack-nova | 16:22 | |
*** ociuhandu has quit IRC | 16:22 | |
*** cdent has quit IRC | 16:22 | |
*** ociuhandu has joined #openstack-nova | 16:22 | |
*** ricolin_ has joined #openstack-nova | 16:23 | |
*** ivve has quit IRC | 16:23 | |
huaqiang | dansmith: about the evenly distributing PCPUs/VCPUs in instance, I have replied in v7 | 16:23 |
dansmith | huaqiang: yeah, I know, and so have others | 16:24 |
huaqiang | I need your comment | 16:24 |
dansmith | huaqiang: I don't like that we're just letting the user define the orientation of every last electron instead of being abstract | 16:24 |
dansmith | but that's just my opinion | 16:24 |
dansmith | it's clear that others feel it's very important to be able to arrange all the electrons | 16:24 |
sean-k-mooney | dansmith: i dont know if you saw my conversation with alex_xu this morning | 16:26 |
*** ricolin has quit IRC | 16:26 | |
huaqiang | I just think if the cpu maks is provided, administrator can have more choice | 16:26 |
dansmith | I *saw* it, but didn't pay close attention | 16:26 |
sean-k-mooney | you are right that we prviosuly discussed an alternitive | 16:26 |
melwitt | lol arranging electrons? *slowly backs away* | 16:26 |
dansmith | huaqiang: yeah I know.. everyone wants full control of everything | 16:26 |
sean-k-mooney | e.g. a tally count of how many pcpu and vcpu | 16:26 |
sean-k-mooney | and then a conention that per numa node we place the numa local vcpu first followed by the numa local pcus | 16:27 |
dansmith | sean-k-mooney: seems like arranging the dedicated cpus first and the non-dedicates ones after would still give you all the control you need, since you can already control numa nodes, memory assignment, etc | 16:27 |
sean-k-mooney | this make the core 0 is for the os eveythring else if for the app case just work | 16:27 |
dansmith | but I know I should just forget about this ever trying to be anything other than manual bitmasks for everything | 16:27 |
sean-k-mooney | dansmith: you want the non dedicate ones first | 16:27 |
sean-k-mooney | there are some things in the kernel you cant move off core 0 | 16:28 |
*** betherly has joined #openstack-nova | 16:28 | |
sean-k-mooney | at least not eaisly | 16:28 |
melwitt | mriedem: I think I asked about backporting --before, not this :) but yeah, both are things that would help with operability for queens *emoji with eyelashes* | 16:28 |
dansmith | whatever | 16:28 |
sean-k-mooney | dansmith: well it is a valid alternitve im not totally against | 16:28 |
*** ociuhandu has quit IRC | 16:28 | |
*** markvoelker has quit IRC | 16:28 | |
sean-k-mooney | but if we were to go that route i think we would need time to think about it and spec out | 16:28 |
sean-k-mooney | meaning this defintly would be moved to U | 16:29 |
sean-k-mooney | the mask is consitent with realtiem cpus btu i get the concern over complexity | 16:29 |
*** ricolin_ is now known as ricolin | 16:29 | |
*** psachin has quit IRC | 16:30 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add useful error log when _determine_version_cap raises DBNotAllowed https://review.opendev.org/675148 | 16:31 |
mriedem | mnaser: noonedeadpunk: ^ did an OSA nova bug get filed? | 16:31 |
mriedem | if so i can tack it on there | 16:31 |
huaqiang | dansmith: got. thanks | 16:32 |
mriedem | melwitt: oh right yeah | 16:32 |
mriedem | is queens the major RHOSP release all of the customers are on now because that seems to be where all of the recent customer stuff is coming from | 16:32 |
*** betherly has quit IRC | 16:32 | |
sean-k-mooney | mriedem: yes | 16:33 |
mriedem | non-LTS RHOSP i should say | 16:33 |
melwitt | that is correct, heh | 16:33 |
sean-k-mooney | queens is osp 13. osp 10(netwon) is EOL in december so everyone is moving to 13. | 16:33 |
melwitt | we're maybe probably gonna backport both of these I think. because RDO cloud and/or our CI runs on queens and not having these tools has been pain for those running the clusters | 16:34 |
melwitt | *backport downstream, at least | 16:34 |
*** factor has joined #openstack-nova | 16:35 | |
mnaser | mriedem: i think noonedeadpunk did not do one so perhaps OSA instead can tack on | 16:35 |
mriedem | super. i'm now tasked internally with helping to deploy and operate a distribution based on mitaka, so things are really moving forward in the world. | 16:35 |
mnaser | lololol | 16:35 |
melwitt | oooof, damn | 16:36 |
mriedem | and the only red hat recruiter emails i get are for consulting gigs :/ \o/ | 16:36 |
mriedem | need to fire up the old beat farmer dream | 16:37 |
mriedem | at least that way i can get in on some sweet sweet government subsidies | 16:37 |
melwitt | lol xD | 16:37 |
*** markvoelker has joined #openstack-nova | 16:38 | |
* mriedem starts stable/stein release procedures | 16:39 | |
*** tesseract has quit IRC | 16:39 | |
*** ricolin has quit IRC | 16:40 | |
*** lpetrut has quit IRC | 16:41 | |
*** markvoelker has quit IRC | 16:42 | |
*** markvoelker has joined #openstack-nova | 16:44 | |
*** ociuhandu has joined #openstack-nova | 16:44 | |
mriedem | stein release wip proposed https://review.opendev.org/#/c/675152/ | 16:46 |
mriedem | artom: sean-k-mooney: ^ has the ovs hybrid plug revert resize thing in it too | 16:47 |
noonedeadpunk | mriedem: not yet:( | 16:47 |
noonedeadpunk | I'll fill in then right now | 16:47 |
*** mkrai has quit IRC | 16:48 | |
mriedem | melwitt: thanks for hitting those backports | 16:48 |
melwitt | deps on route to the gate | 16:48 |
melwitt | np | 16:48 |
mriedem | you can bug me about the all cells archive thing, i've been meaning to get to it, just sidetracked | 16:48 |
melwitt | *en route (how embarrassing) | 16:48 |
melwitt | yesss, thanks | 16:49 |
sean-k-mooney | oh ok cool for tansparancey there is this revert fo that https://review.opendev.org/#/c/675021/ but artom and i both belive the issue that they were trying to fix is unrelated | 16:49 |
*** ociuhandu has quit IRC | 16:50 | |
artom | mriedem, it's already downstream, but thanks :) | 16:50 |
artom | sean-k-mooney, yeah, he's barking up the wrong tree with that as far as I'm concerned | 16:50 |
*** igordc has quit IRC | 16:51 | |
sean-k-mooney | the timelines dont match up as you said | 16:51 |
sean-k-mooney | we could ad special case support for midonet trivially but there is somthing else going on there | 16:52 |
artom | Their errors are the mm executable returning an error, nothing to do with events | 16:52 |
sean-k-mooney | ah ok i didnt look at the compute log until now | 16:53 |
noonedeadpunk | mriedem: ok, seems that you've already linked patch to some bug | 16:53 |
artom | sean-k-mooney, he gave me a good night scare though, I got the gerrit review email right before bed | 16:55 |
*** rpittau is now known as rpittau|afk | 16:56 | |
sean-k-mooney | ya im sure the heart rate spike didnt help you drifting off to sleep | 16:56 |
mriedem | noonedeadpunk: yeah and old bug | 16:57 |
mriedem | *an | 16:57 |
mriedem | sean-k-mooney: were you going to update this? https://review.opendev.org/#/c/647735/ | 16:57 |
*** maciejjozefczyk has quit IRC | 16:57 | |
mriedem | i'd like to make sure we don't lose track of documenting the features we add | 16:58 |
sean-k-mooney | yes | 16:58 |
sean-k-mooney | i also have to document the vpmu image property | 16:58 |
mriedem | oh i know | 16:58 |
mriedem | was just going through https://review.opendev.org/#/c/671338/ | 16:58 |
*** derekh has quit IRC | 16:59 | |
sean-k-mooney | ah right but yes ill update both. im almost finish fixing some other typos in an os-vif patch so i might as well update the glance patchs before i start anything else | 16:59 |
sean-k-mooney | want me to add you to them | 17:00 |
mriedem | umm | 17:00 |
mriedem | you can if you want | 17:01 |
*** ociuhandu has joined #openstack-nova | 17:01 | |
sean-k-mooney | i can just update the nova patches with the link | 17:01 |
*** betherly has joined #openstack-nova | 17:03 | |
noonedeadpunk | mriedem: https://bugs.launchpad.net/nova/+bug/1839360 | 17:07 |
openstack | Launchpad bug 1839360 in OpenStack Compute (nova) "nova-compute fails with DBNotAllowed error" [Undecided,New] | 17:07 |
*** betherly has quit IRC | 17:08 | |
*** igordc has joined #openstack-nova | 17:10 | |
*** artom has quit IRC | 17:13 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add useful error log when _determine_version_cap raises DBNotAllowed https://review.opendev.org/675148 | 17:13 |
mriedem | noonedeadpunk: done thanks | 17:13 |
*** udesale has quit IRC | 17:14 | |
* mriedem gets lunch | 17:15 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | 17:16 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | 17:16 |
* stephenfin goes home | 17:16 | |
*** brinzhang_ has joined #openstack-nova | 17:16 | |
stephenfin | sean-k-mooney: If you've a chance tomorrow, I think https://review.opendev.org/671801 is mostly in good shape now. That was a rough one | 17:16 |
sean-k-mooney | cool how is the series as a whole | 17:16 |
stephenfin | I need to test it on real hardware but I think it's just the reshape that still needs some work | 17:17 |
sean-k-mooney | i need to resinstall my sriov systems i think tomorowo but is your stuff ready to test after i do that | 17:17 |
stephenfin | For a new deployment, yes. Upgrades are still WIP due to reshape | 17:17 |
*** altlogbot_2 has quit IRC | 17:17 | |
sean-k-mooney | well its not jsut upgrades right | 17:18 |
stephenfin | How so? | 17:18 |
sean-k-mooney | its going form not defiing the config to defining them | 17:18 |
stephenfin | What is? | 17:18 |
sean-k-mooney | the reshape | 17:18 |
stephenfin | (my head is pretty much fried at this point) | 17:19 |
stephenfin | Ah. Um, ask me tomorrow? :) | 17:19 |
*** brinzhang has quit IRC | 17:19 | |
sean-k-mooney | ok | 17:19 |
sean-k-mooney | go clear your head. im going to start dinner. ill have the os-vif patch done soone by the way just on the release note. thanks for taking a look yesterday | 17:20 |
stephenfin | no problem | 17:22 |
* stephenfin really goes home | 17:23 | |
*** altlogbot_1 has joined #openstack-nova | 17:23 | |
*** betherly has joined #openstack-nova | 17:24 | |
*** betherly has quit IRC | 17:29 | |
*** mgariepy has quit IRC | 17:29 | |
*** ociuhandu has quit IRC | 17:29 | |
*** mgariepy has joined #openstack-nova | 17:30 | |
*** altlogbot_1 has quit IRC | 17:31 | |
*** altlogbot_2 has joined #openstack-nova | 17:35 | |
*** ociuhandu has joined #openstack-nova | 17:39 | |
*** ociuhandu has quit IRC | 17:43 | |
*** betherly has joined #openstack-nova | 17:44 | |
*** betherly has quit IRC | 17:49 | |
*** dpawlik has joined #openstack-nova | 17:50 | |
*** spatel has quit IRC | 17:57 | |
openstackgerrit | sean mooney proposed openstack/os-vif master: only disable mac ageing for ovs hybrid plug https://review.opendev.org/672834 | 18:03 |
*** betherly has joined #openstack-nova | 18:04 | |
*** betherly has quit IRC | 18:09 | |
melwitt | hm, looks like hyperv ci is failing with 'sudo: /usr/sbin/mke2fs: command not found' all the time as of late | 18:16 |
melwitt | http://cloudbase-ci.com/nova/674828/1/tempest/subunit.html.gz | 18:17 |
*** artom has joined #openstack-nova | 18:17 | |
mriedem | melwitt: https://github.com/openstack/tempest/commit/887f99a288c1d2bef45356c916b317d37828e6ac | 18:20 |
melwitt | ahh | 18:20 |
mriedem | looks like hyperv ci needs to blacklist that test | 18:20 |
mriedem | likely need to email their ci group | 18:20 |
dansmith | do they not use a cirros gues? | 18:20 |
dansmith | *guest | 18:20 |
mriedem | no idea | 18:21 |
dansmith | I would think they did, and that this test should be fine | 18:21 |
sean-k-mooney | they might be using a windows nano server image? | 18:21 |
dansmith | there's also a failure to resolve thing in that error log | 18:21 |
melwitt | yeah, I didn't understand that part bc I don't see that "hostname" as part of the command line | 18:22 |
*** spsurya has quit IRC | 18:22 | |
dansmith | yeah | 18:22 |
sean-k-mooney | looking at the logs melwitt linked they apper to be useing ubuntu as the ssh user name | 18:22 |
dansmith | maybe a bug in some fake sudo wrapper they use or something | 18:23 |
sean-k-mooney | so i assume its a ubuntu guest image | 18:23 |
mriedem | it's ubuntu 16 | 18:23 |
mriedem | name | ubuntu-16 | 18:23 |
mriedem | openstack --os-cloud=devstack-admin --os-region-name=RegionOne image create ubuntu-16 --public --container-format=bare --disk-format vhdx | 18:23 |
mriedem | whatever this is ubuntu-16.04-minimal-cloudimg-amd64-sync.vhdx | 18:23 |
dansmith | wonder why that and not cirros | 18:24 |
mriedem | not sure | 18:24 |
dansmith | so it's possible that their image is missing mke2fs I guess | 18:24 |
mriedem | they have their own cloud-init thing | 18:24 |
mriedem | for hyperv | 18:24 |
sean-k-mooney | dansmith: perhaps there is a dirver issue with hyperv | 18:24 |
dansmith | for windows guests, right? | 18:24 |
sean-k-mooney | that said cirros | 18:25 |
sean-k-mooney | uses teh ubunut kernel | 18:25 |
sean-k-mooney | so proably not | 18:25 |
mriedem | anywho, here is the email info https://wiki.openstack.org/wiki/ThirdPartySystems/Cloudbase_Compute_Hyper-V_CI | 18:25 |
*** betherly has joined #openstack-nova | 18:25 | |
sean-k-mooney | mriedem: cloudbase-init | 18:25 |
sean-k-mooney | i think that is only for windows guests | 18:25 |
dansmith | sean-k-mooney: maybe, but there's not much reason for that (i.e. perf for tests) unless it's physically required for disk access or something | 18:25 |
sean-k-mooney | hyperv uses parvitualised devices | 18:26 |
dansmith | I guess maybe they only expose PV devices and need a real kernel | 18:26 |
dansmith | okay | 18:26 |
sean-k-mooney | so if you do lspci by default you will have no devices | 18:26 |
sean-k-mooney | its weird | 18:26 |
dansmith | okay, well, that's why probably | 18:26 |
dansmith | but that opens them up to this kind of thing | 18:26 |
dansmith | would think it'd be worth just working on getting that stuff into cirros or building their own cirros, | 18:27 |
dansmith | but alas | 18:27 |
sean-k-mooney | but what i dont get is cirros is built with the ubuntu kernel | 18:27 |
sean-k-mooney | so i dont think it buys them much | 18:27 |
sean-k-mooney | maybe they could use cirros 4.x | 18:27 |
dansmith | it doesn't have a full complement of modules thjough, AFAIK | 18:27 |
dansmith | anyway, this is clearly a thing the hyperv people can fix if they want :) | 18:28 |
sean-k-mooney | ya that is true. i have hyperv on my windows install but i have never tried cirros with it | 18:28 |
sean-k-mooney | well in this case tehy just need to install mke2fs | 18:28 |
sean-k-mooney | in the guest image | 18:29 |
*** betherly has quit IRC | 18:30 | |
melwitt | I'll try to email their list https://wiki.openstack.org/wiki/ThirdPartySystems/Hyper-V_CI | 18:30 |
melwitt | also looks like it's ok on the master branch. I'm seeing that failure on older branches | 18:38 |
*** betherly has joined #openstack-nova | 18:45 | |
*** mvkr has joined #openstack-nova | 18:47 | |
*** BjoernT has joined #openstack-nova | 18:48 | |
melwitt | email bounced :( | 18:49 |
*** betherly has quit IRC | 18:50 | |
*** BjoernT_ has quit IRC | 18:50 | |
melwitt | I forwarded to ociuhandu in case he is involved with the CI | 18:58 |
*** tbachman has quit IRC | 18:58 | |
*** tbachman has joined #openstack-nova | 19:00 | |
*** abaindur has joined #openstack-nova | 19:00 | |
*** mgariepy has quit IRC | 19:13 | |
*** mgariepy has joined #openstack-nova | 19:13 | |
*** mgariepy has quit IRC | 19:14 | |
*** mgariepy has joined #openstack-nova | 19:15 | |
*** BjoernT_ has joined #openstack-nova | 19:15 | |
*** mgariepy has quit IRC | 19:16 | |
*** BjoernT has quit IRC | 19:17 | |
*** mgariepy has joined #openstack-nova | 19:18 | |
*** mgariepy has quit IRC | 19:19 | |
*** mgariepy has joined #openstack-nova | 19:20 | |
*** rouk has joined #openstack-nova | 19:22 | |
*** mgariepy has quit IRC | 19:33 | |
*** dave-mccowan has joined #openstack-nova | 19:33 | |
*** mgariepy has joined #openstack-nova | 19:34 | |
*** lpetrut has joined #openstack-nova | 19:35 | |
*** vesper11 has quit IRC | 19:37 | |
*** tbachman has quit IRC | 19:45 | |
*** betherly has joined #openstack-nova | 19:46 | |
*** betherly has quit IRC | 19:51 | |
*** dpawlik has quit IRC | 19:53 | |
*** betherly has joined #openstack-nova | 19:55 | |
*** efried_rollin is now known as efried | 19:59 | |
*** betherly has quit IRC | 20:00 | |
*** markvoelker has quit IRC | 20:01 | |
*** lpetrut has quit IRC | 20:01 | |
*** factor has quit IRC | 20:03 | |
*** markvoelker has joined #openstack-nova | 20:03 | |
*** factor has joined #openstack-nova | 20:03 | |
*** factor has quit IRC | 20:05 | |
*** tbachman has joined #openstack-nova | 20:05 | |
*** factor has joined #openstack-nova | 20:05 | |
*** ivve has joined #openstack-nova | 20:15 | |
*** betherly has joined #openstack-nova | 20:16 | |
*** whoami-rajat has quit IRC | 20:17 | |
*** betherly has quit IRC | 20:21 | |
openstackgerrit | Sundar Nadathur proposed openstack/nova master: ksa auth conf and client for Cyborg access https://review.opendev.org/631242 | 20:26 |
openstackgerrit | Sundar Nadathur proposed openstack/nova master: Refactor some methods for reuse by Cyborg-related code. https://review.opendev.org/673734 | 20:26 |
openstackgerrit | Sundar Nadathur proposed openstack/nova master: WIP: Add Cyborg device profile groups to request spec. https://review.opendev.org/631243 | 20:26 |
openstackgerrit | Sundar Nadathur proposed openstack/nova master: WIP: Create and bind Cyborg ARQs. https://review.opendev.org/631244 | 20:26 |
openstackgerrit | Sundar Nadathur proposed openstack/nova master: WIP: Get resolved Cyborg ARQs and add PCI BDFs to VM's domain XML. https://review.opendev.org/631245 | 20:26 |
openstackgerrit | Sundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted. https://review.opendev.org/673735 | 20:26 |
*** factor has quit IRC | 20:32 | |
*** factor has joined #openstack-nova | 20:32 | |
*** factor has quit IRC | 20:35 | |
*** factor has joined #openstack-nova | 20:36 | |
openstackgerrit | Merged openstack/nova stable/stein: Add functional recreate test for bug 1764556 https://review.opendev.org/673812 | 20:39 |
openstack | bug 1764556 in OpenStack Compute (nova) stein ""nova list" fails with exception.ServiceNotFound if service is deleted and has no UUID" [Medium,In progress] https://launchpad.net/bugs/1764556 - Assigned to Matt Riedemann (mriedem) | 20:39 |
openstackgerrit | Merged openstack/nova stable/stein: Add functional regression test for bug 1778305 https://review.opendev.org/673813 | 20:39 |
openstack | bug 1778305 in OpenStack Compute (nova) stein "Nova may erronously look up service version of a deleted service, when hostname have been reused" [Medium,In progress] https://launchpad.net/bugs/1778305 - Assigned to Matt Riedemann (mriedem) | 20:39 |
openstackgerrit | Merged openstack/nova master: tox: Keeping going with docs https://review.opendev.org/670332 | 20:39 |
*** ralonsoh has quit IRC | 20:58 | |
melwitt | mriedem: merge conflict :( https://review.opendev.org/673814 | 20:59 |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add new default roles in os-services API policies https://review.opendev.org/648480 | 21:03 |
openstackgerrit | Matt Riedemann proposed openstack/nova stable/stein: Don't generate service UUID for deleted services https://review.opendev.org/673814 | 21:04 |
mriedem | with what? rebase was simple | 21:04 |
*** slaweq_ has quit IRC | 21:06 | |
*** BjoernT_ has quit IRC | 21:08 | |
melwitt | I dunno, I didn't pull it down | 21:10 |
*** slaweq_ has joined #openstack-nova | 21:11 | |
mriedem | melwitt: https://review.opendev.org/#/c/507486/65 | 21:14 |
mriedem | also had some comments in the nova-next patch on top, which dansmith might have thoughts on | 21:15 |
melwitt | thanks! | 21:16 |
*** slaweq_ has quit IRC | 21:17 | |
mriedem | melwitt: there are a few things in there i commented on which are separate issues, which i can push some patches to address so this doesn't get more cluttered up | 21:20 |
melwitt | ok, that sounds great if you're up for it. else I can break things out | 21:21 |
melwitt | I see there's a lot to go through, I didn't look in detail yet | 21:22 |
*** xek has quit IRC | 21:22 | |
*** markvoelker has quit IRC | 21:23 | |
mriedem | nothing really big or complicated, | 21:26 |
mriedem | i think the functional test could be simplified, | 21:26 |
mriedem | and if separated from ServersTestBase it could be less dependent on some of the weirdness in that class | 21:26 |
*** vesper11 has joined #openstack-nova | 21:26 | |
* melwitt looks wistfully at ServersTestBase | 21:27 | |
*** macz has joined #openstack-nova | 21:28 | |
*** tjgresha has quit IRC | 21:28 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Don't mention CONF.api_database.connection in user-facing messages/docs https://review.opendev.org/675205 | 21:31 |
*** tjgresha has joined #openstack-nova | 21:31 | |
*** tjgresha has quit IRC | 21:32 | |
*** factor has quit IRC | 21:35 | |
*** factor has joined #openstack-nova | 21:35 | |
*** markvoelker has joined #openstack-nova | 21:38 | |
*** tbachman has quit IRC | 21:42 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Clean up docstrings for archive_deleted_rows https://review.opendev.org/675209 | 21:43 |
*** mchlumsky has quit IRC | 21:44 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Re-use DB engine connection during archive_deleted_rows https://review.opendev.org/675218 | 21:49 |
mriedem | melwitt: ok i think that's all the separate nit stuff | 21:49 |
melwitt | thanx | 21:49 |
*** markvoelker has quit IRC | 21:50 | |
*** tbachman has joined #openstack-nova | 22:08 | |
*** slaweq_ has joined #openstack-nova | 22:11 | |
*** mlavalle has quit IRC | 22:13 | |
*** rcernin has joined #openstack-nova | 22:15 | |
*** slaweq_ has quit IRC | 22:16 | |
*** markvoelker has joined #openstack-nova | 22:19 | |
mriedem | duck | 22:22 |
mriedem | and | 22:22 |
mriedem | cover | 22:22 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.opendev.org/614036 | 22:23 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Instance.hidden field https://review.opendev.org/631123 | 22:23 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add TargetDBSetupTask https://review.opendev.org/627892 | 22:23 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask https://review.opendev.org/631581 | 22:23 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask https://review.opendev.org/633853 | 22:23 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method https://review.opendev.org/633293 | 22:23 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask https://review.opendev.org/627890 | 22:23 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.opendev.org/634832 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova.compute.utils.delete_image https://review.opendev.org/637605 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.opendev.org/627891 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Refactor ComputeManager.remove_volume_connection https://review.opendev.org/642183 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add power_on kwarg to ComputeDriver.spawn() method https://review.opendev.org/642590 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add finish_snapshot_based_resize_at_dest compute method https://review.opendev.org/635080 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add FinishResizeAtDestTask https://review.opendev.org/635646 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add Destination.allow_cross_cell_move field https://review.opendev.org/614035 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute CrossCellMigrationTask from MigrationTask https://review.opendev.org/635668 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Plumb allow_cross_cell_resize into compute API resize() https://review.opendev.org/635684 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Filter duplicates from compute API get_migrations_sorted() https://review.opendev.org/636224 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional test for anti-affinity cross-cell migration https://review.opendev.org/661859 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Support cross-cell moves in external_instance_event https://review.opendev.org/658478 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cross-cell resize policy rule and enable in API https://review.opendev.org/638269 | 22:24 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Enable cross-cell resize in the nova-multi-cell job https://review.opendev.org/656656 | 22:24 |
*** mriedem has quit IRC | 22:24 | |
*** brinzhang has joined #openstack-nova | 22:31 | |
*** brinzhang_ has quit IRC | 22:34 | |
*** markvoelker has quit IRC | 22:38 | |
*** ivve has quit IRC | 22:45 | |
*** markvoelker has joined #openstack-nova | 22:53 | |
*** macz has quit IRC | 22:53 | |
openstackgerrit | Eric Fried proposed openstack/nova master: Use :oslo.config:* in nova-manage doc https://review.opendev.org/675224 | 22:58 |
*** markvoelker has quit IRC | 23:08 | |
openstackgerrit | Dustin Cowles proposed openstack/nova master: Provider config file schema and loader https://review.opendev.org/673341 | 23:09 |
*** slaweq_ has joined #openstack-nova | 23:11 | |
*** betherly has joined #openstack-nova | 23:15 | |
*** slaweq_ has quit IRC | 23:16 | |
*** betherly has quit IRC | 23:20 | |
*** hoonetorg has quit IRC | 23:26 | |
*** takashin has joined #openstack-nova | 23:32 | |
*** igordc has quit IRC | 23:35 | |
*** betherly has joined #openstack-nova | 23:37 | |
*** markvoelker has joined #openstack-nova | 23:38 | |
*** hoonetorg has joined #openstack-nova | 23:40 | |
openstackgerrit | Takashi NATSUME proposed openstack/nova master: Fix non-existent method of Mock https://review.opendev.org/675041 | 23:40 |
*** markvoelker has quit IRC | 23:43 | |
*** betherly has quit IRC | 23:45 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!