| *** igordc has joined #openstack-nova | 00:02 | |
| *** mlavalle has quit IRC | 00:09 | |
| *** ociuhandu has joined #openstack-nova | 00:10 | |
| *** ociuhandu has quit IRC | 00:20 | |
| *** igordc has quit IRC | 00:34 | |
| *** lbragstad has joined #openstack-nova | 00:37 | |
| *** ircuser-1 has joined #openstack-nova | 00:49 | |
| songwenping_ | Hi, gibi. | 00:58 |
|---|---|---|
| songwenping_ | i have a issue with filter_scheduler with accelerators. pls see the detail at https://etherpad.opendev.org/p/filter_scheduler_issue_with_accelerators | 01:00 |
| songwenping_ | the relate fix patch is https://review.opendev.org/#/c/722651/ | 01:01 |
| songwenping_ | the relate bug reported is https://bugs.launchpad.net/nova/+bug/1874664. pls help me have a look. | 01:02 |
| openstack | Launchpad bug 1874664 in OpenStack Compute (nova) "Boot more than one instances failed with accelerators in its flavor" [Medium,Confirmed] - Assigned to Wenping Song (wenping1) | 01:02 |
| *** songwenping__ has joined #openstack-nova | 01:16 | |
| *** songwenping_ has quit IRC | 01:19 | |
| *** Liang__ has joined #openstack-nova | 01:19 | |
| *** songwenping_ has joined #openstack-nova | 01:21 | |
| *** songwenping__ has quit IRC | 01:24 | |
| *** yaawang has quit IRC | 01:29 | |
| *** yaawang has joined #openstack-nova | 01:29 | |
| *** ociuhandu has joined #openstack-nova | 01:57 | |
| *** ociuhandu has quit IRC | 02:10 | |
| *** yaawang has quit IRC | 02:14 | |
| *** yaawang has joined #openstack-nova | 02:15 | |
| *** artom has quit IRC | 02:26 | |
| *** mkrai has joined #openstack-nova | 02:29 | |
| *** yaawang has quit IRC | 02:32 | |
| *** yaawang has joined #openstack-nova | 02:33 | |
| *** brinzhang has quit IRC | 02:55 | |
| *** brinzhang has joined #openstack-nova | 02:55 | |
| *** ociuhandu has joined #openstack-nova | 02:57 | |
| *** jangutter has joined #openstack-nova | 03:00 | |
| *** jangutter_ has quit IRC | 03:01 | |
| *** ociuhandu has quit IRC | 03:05 | |
| *** huaqiang has joined #openstack-nova | 03:12 | |
| *** threestrands has joined #openstack-nova | 03:12 | |
| *** psachin has joined #openstack-nova | 03:18 | |
| *** ociuhandu has joined #openstack-nova | 03:53 | |
| *** ociuhandu has quit IRC | 04:00 | |
| *** brinzhang_ has joined #openstack-nova | 04:02 | |
| *** ratailor has joined #openstack-nova | 04:13 | |
| *** songwenping__ has joined #openstack-nova | 04:20 | |
| *** brinzhang has quit IRC | 04:23 | |
| *** songwenping_ has quit IRC | 04:23 | |
| *** brinzhang has joined #openstack-nova | 04:24 | |
| *** ircuser-1 has quit IRC | 04:33 | |
| *** evrardjp has quit IRC | 04:35 | |
| *** evrardjp has joined #openstack-nova | 04:35 | |
| *** brinzhang_ has quit IRC | 04:40 | |
| *** brinzhang_ has joined #openstack-nova | 04:41 | |
| *** vishalmanchanda has joined #openstack-nova | 04:57 | |
| openstackgerrit | Arthur Dayne proposed openstack/nova-specs master: new a test https://review.opendev.org/723794 | 05:02 |
| *** songwenping_ has joined #openstack-nova | 05:21 | |
| *** brinzhang_ has quit IRC | 05:24 | |
| *** brinzhang_ has joined #openstack-nova | 05:24 | |
| *** songwenping__ has quit IRC | 05:24 | |
| *** brinzhang_ has quit IRC | 05:25 | |
| *** brinzhang has quit IRC | 05:26 | |
| *** brinzhang_ has joined #openstack-nova | 05:26 | |
| *** brinzhang has joined #openstack-nova | 05:26 | |
| *** ociuhandu has joined #openstack-nova | 05:37 | |
| *** links has joined #openstack-nova | 05:38 | |
| *** ociuhandu has quit IRC | 05:50 | |
| *** ociuhandu has joined #openstack-nova | 06:00 | |
| *** mkrai has quit IRC | 06:05 | |
| *** dpawlik has joined #openstack-nova | 06:06 | |
| *** mkrai has joined #openstack-nova | 06:06 | |
| *** songwenping__ has joined #openstack-nova | 06:13 | |
| *** ociuhandu has quit IRC | 06:15 | |
| *** ociuhandu has joined #openstack-nova | 06:15 | |
| *** songwenping_ has quit IRC | 06:17 | |
| *** damien_r has joined #openstack-nova | 06:28 | |
| *** ociuhandu has quit IRC | 06:30 | |
| *** damien_r has quit IRC | 06:33 | |
| *** udesale has joined #openstack-nova | 06:37 | |
| *** ttsiouts has joined #openstack-nova | 06:39 | |
| *** ociuhandu has joined #openstack-nova | 06:49 | |
| *** ttsiouts has quit IRC | 06:53 | |
| *** slaweq has joined #openstack-nova | 06:53 | |
| *** nightmare_unreal has joined #openstack-nova | 06:54 | |
| *** ttsiouts has joined #openstack-nova | 06:55 | |
| *** maciejjozefczyk has joined #openstack-nova | 06:56 | |
| gibi | good morning | 07:02 |
| gibi | songwenping__: ack, I will look at it shortly | 07:02 |
| songwenping__ | thanks, gibi. | 07:03 |
| *** damien_r has joined #openstack-nova | 07:04 | |
| *** ccamacho has joined #openstack-nova | 07:10 | |
| bauzas | good morning Nova | 07:11 |
| aarents | good morning | 07:13 |
| bauzas | fwiw, I'm a bit on and off this week due to children vacations at home | 07:14 |
| bauzas | fortunately, it's after RC1 :) | 07:15 |
| gibi | bauzas: a quick question. If we have a bug in the cyborg integration code https://bugs.launchpad.net/nova/+bug/1874664 and that code was added in Ussuri then Am I correct that such bug is an RC2 candidate? | 07:18 |
| openstack | Launchpad bug 1874664 in OpenStack Compute (nova) "Boot more than one instances failed with accelerators in its flavor" [Medium,Confirmed] - Assigned to Wenping Song (wenping1) | 07:18 |
| * bauzas clicks | 07:18 | |
| bauzas | gibi: MHO is that I don't think it's an urgent regression | 07:19 |
| bauzas | gibi: given we weren't supporting Cyborg by Train | 07:20 |
| gibi | bauzas: so it is a regression but it targets a small portion of a user base and it definetly does not break upgrade | 07:20 |
| bauzas | gibi: do we have already a fix ? | 07:20 |
| gibi | bauzas: there is a patch https://review.opendev.org/#/c/722651/ | 07:20 |
| gibi | but I haven't looked it yet how complex it is | 07:21 |
| bauzas | gibi: if not, we could just ask brinzhang to modify the documentation to say that Nova won't support more than one instance for Cyborg in Ussuri | 07:21 |
| bauzas | and then we could backport the fix to a Ussuri later .z | 07:21 |
| gibi | bauzas: good point. if the fix is problematic in RC timeframe then we can do a documentation patch | 07:21 |
| gibi | bauzas: thanks for the consultation | 07:22 |
| bauzas | gibi: anyway, it's just my opinion | 07:23 |
| bauzas | brinzhang: are you around ? | 07:23 |
| AJaeger | morning, any nova core available to review two tiny cleanups related to Babel/translations, please? https://review.opendev.org/#/c/723206/2 and https://review.opendev.org/#/c/720725/1 ? | 07:25 |
| *** ociuhandu has quit IRC | 07:26 | |
| bauzas | AJaeger: ack, will look | 07:26 |
| brinzhang | bauzas, gibi: ack | 07:26 |
| brinzhang | bauzas, gibi: I can add that in accelerator support docs, to limit mutil create instances | 07:27 |
| *** tesseract has joined #openstack-nova | 07:27 | |
| brinzhang | songwenping__ post the etherpad, I think that true, and he has some idea to fix this issue. | 07:28 |
| brinzhang | I agree with bauzas, we can try to fix this issue, than consider to backport to U later ^ | 07:29 |
| AJaeger | thanks, bauzas | 07:29 |
| *** songwenping_ has joined #openstack-nova | 07:29 | |
| brinzhang | bauzas: That I just need to add the limit to https://docs.openstack.org/api-guide/compute/accelerator-support.html | 07:30 |
| *** brinzhang has quit IRC | 07:30 | |
| bauzas | brinzhang: yeah, I just provided a bug comment https://bugs.launchpad.net/nova/+bug/1874664 | 07:30 |
| openstack | Launchpad bug 1874664 in OpenStack Compute (nova) "Boot more than one instances failed with accelerators in its flavor" [Medium,Confirmed] - Assigned to Wenping Song (wenping1) | 07:30 |
| *** brinzhang has joined #openstack-nova | 07:30 | |
| *** songwenping_ has quit IRC | 07:31 | |
| bauzas | brinzhang: fwiw, we did it like this for https://docs.openstack.org/nova/latest/admin/virtual-gpu.html | 07:31 |
| *** songwenping_ has joined #openstack-nova | 07:31 | |
| bauzas | err, sorry this https://docs.openstack.org/nova/latest/admin/virtual-gpu.html#caveats | 07:31 |
| *** ociuhandu has joined #openstack-nova | 07:32 | |
| *** songwenping__ has quit IRC | 07:33 | |
| *** rpittau|afk is now known as rpittau | 07:34 | |
| bauzas | AJaeger: sent to the gate, thanks for clarifying | 07:35 |
| AJaeger | thanks, bauzas | 07:36 |
| *** tosky has joined #openstack-nova | 07:36 | |
| AJaeger | could somebody import translations, please? https://review.opendev.org/722644 | 07:38 |
| gibi | AJaeger: does this transaltion patch needs to be backported to stable/ussuri too? | 07:39 |
| AJaeger | gibi: there will be a separte one proposed if available | 07:41 |
| AJaeger | gibi: in other words: no | 07:41 |
| AJaeger | translations are special | 07:41 |
| AJaeger | gibi: and release notes are only translated on master | 07:42 |
| gibi | AJaeger: thanks. | 07:42 |
| AJaeger | thanks as well, gibi ;) | 07:43 |
| gibi | AJaeger: does it mean that if we have a reno update merged after this point then we need to trigger a re-translation of the reno somehow? | 07:43 |
| *** ttsiouts has quit IRC | 07:44 | |
| *** ociuhandu has quit IRC | 07:45 | |
| *** ociuhandu has joined #openstack-nova | 07:45 | |
| gibi | AJaeger: I mean we need to add at leat an upgrade warning to the ussuri reno due to a bug discovered after RC1 | 07:46 |
| *** jraju__ has joined #openstack-nova | 07:46 | |
| *** links has quit IRC | 07:47 | |
| *** ttsiouts has joined #openstack-nova | 07:47 | |
| *** mkrai has quit IRC | 07:51 | |
| *** ralonsoh has joined #openstack-nova | 07:52 | |
| *** brinzhang_ has quit IRC | 07:55 | |
| AJaeger | gibi: that happens automatically - after each merge, any new strings are send to the translation server and the translators translate at their leisure | 07:57 |
| AJaeger | gibi: and if nobody translates, it shows up in English, so will be there for sure | 07:58 |
| gibi | AJaeger: cool thanks | 07:58 |
| AJaeger | gibi: so, no worries, continue adding releasenotes ;) | 07:58 |
| *** ttsiouts has quit IRC | 08:03 | |
| *** brinzhang_ has joined #openstack-nova | 08:06 | |
| *** songwenping__ has joined #openstack-nova | 08:06 | |
| *** songwenping_ has quit IRC | 08:09 | |
| *** brinzhang has quit IRC | 08:09 | |
| *** martinkennelly has joined #openstack-nova | 08:19 | |
| *** mkrai has joined #openstack-nova | 08:22 | |
| *** ttsiouts has joined #openstack-nova | 08:28 | |
| *** happyhemant has joined #openstack-nova | 08:30 | |
| *** derekh has joined #openstack-nova | 08:32 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/nova stable/ussuri: Imported Translations from Zanata https://review.opendev.org/723160 | 08:35 |
| *** udesale has quit IRC | 08:36 | |
| *** udesale has joined #openstack-nova | 08:36 | |
| gibi | elod: ^^ | 08:43 |
| *** priteau has joined #openstack-nova | 08:53 | |
| *** jraju__ has quit IRC | 08:53 | |
| *** jraju__ has joined #openstack-nova | 08:57 | |
| *** xek has joined #openstack-nova | 08:58 | |
| gibi | zigo, gmann, dansmith, stephenfin: replied in https://bugs.launchpad.net/nova/+bug/1875418 let me know if you disagree or if my proposal is not feasible | 08:59 |
| openstack | Launchpad bug 1875418 in OpenStack Compute (nova) "Generated policy.json in Ussuri is broken by default" [Undecided,In progress] - Assigned to Ghanshyam Mann (ghanshyammann) | 08:59 |
| gibi | or if you have any other proposal as a way forward | 08:59 |
| zigo | reading ... | 09:01 |
| *** links has joined #openstack-nova | 09:02 | |
| *** jraju__ has quit IRC | 09:02 | |
| zigo | gibi: I very much agree with all you wrote, thanks you ! | 09:03 |
| zigo | That's exactly what I would like to happen. | 09:04 |
| -openstackstatus- NOTICE: Zuul is currently failing testing, please refrain from recheck and submitting of new changes until this is solved. | 09:04 | |
| *** ChanServ changes topic to "Zuul is currently failing testing, please refrain from recheck and submitting of new changes until this is solved." | 09:04 | |
| gibi | zigo: let's see what others think about the feasibility | 09:04 |
| *** rcernin has quit IRC | 09:04 | |
| gibi | as I have close to zero knowledge about the generator | 09:04 |
| *** avolkov has joined #openstack-nova | 09:07 | |
| *** songwenping_ has joined #openstack-nova | 09:12 | |
| *** dtantsur|afk is now known as dtantsur | 09:14 | |
| -openstackstatus- NOTICE: Zuul is currently failing all testing, please refrain from approving, rechecking or submitting of new changes until this is solved. | 09:14 | |
| *** ChanServ changes topic to "Zuul is currently failing all testing, please refrain from approving, rechecking or submitting of new changes until this is solved." | 09:15 | |
| *** songwenping__ has quit IRC | 09:15 | |
| gibi | bauzas: how the vpgus are avoiding the problem what songwenping_ described for accelerators? | 09:18 |
| gibi | songwenping_: do I understand correctly that your bug appers when you create two instances with one singe POST /servers request? | 09:21 |
| bauzas | gibi: which problem, sorry ? | 09:21 |
| songwenping_ | yes, gibi. | 09:21 |
| bauzas | gibi: do I have a bug to lookup ? | 09:22 |
| gibi | bauzas: same bug you looked at this morning https://bugs.launchpad.net/nova/+bug/1874664 | 09:23 |
| openstack | Launchpad bug 1874664 in OpenStack Compute (nova) "Boot more than one instances failed with accelerators in its flavor" [Medium,Confirmed] - Assigned to Wenping Song (wenping1) | 09:23 |
| gibi | songwenping_: thanks | 09:23 |
| bauzas | hah | 09:23 |
| bauzas | gibi: for vGPUs, we only ask for one RC | 09:24 |
| bauzas | eg. resources:VGPU=2 | 09:24 |
| bauzas | so, when calling Placement, it returns some allocations but all the VGPU ones are in the same RP | 09:25 |
| gibi | bauzas: in case of instance multi create you need two separate allocation of one VGPUs | 09:25 |
| gibi | bauzas: I mean instance multi create with min=2 max=2 | 09:26 |
| bauzas | ah this | 09:26 |
| bauzas | gibi: actually https://bugs.launchpad.net/nova/+bug/1780225 :-) | 09:27 |
| openstack | Launchpad bug 1780225 in OpenStack Compute (nova) "Libvirt error when using --max > 1 with vGPU" [High,Confirmed] - Assigned to Sylvain Bauza (sylvain-bauza) | 09:27 |
| gibi | bauzas: ohh but that leads to a different error in libvirt instead of placement | 09:27 |
| bauzas | gibi: I should be testing it | 09:28 |
| bauzas | lemme try to use the functional test | 09:28 |
| lyarwood | does anyone have an example in api-ref where we list some pre-conditions to calling an API? Looking at the evacuate section I wanted to add a pre-condition around the host being fenced and service being reported as down/forced-down. | 09:29 |
| lyarwood | ah found some for reboot nvm | 09:30 |
| gibi | bauzas: can we have two PGPU RPs providing the same vgpu types under the same compute RP? If yes then I think the instance multicreate can fail due to running out of VGPU resource on the PGPU in the first allocation candidate | 09:32 |
| gibi | and that would be similar to what songwenping_ reported for accelerators | 09:32 |
| bauzas | gibi: yes of course | 09:32 |
| bauzas | gibi: what do you want me to test ? | 09:33 |
| bauzas | 2 RPs with the same vgpu type, each them having, say, 8 vGPUs | 09:34 |
| bauzas | then asking for 2 instances for 2 vGPUs | 09:34 |
| bauzas | gibi: works with you ? | 09:34 |
| gibi | bauzas: give me a sec | 09:34 |
| bauzas | gibi: fwiw I'm just trying to reproduce https://bugs.launchpad.net/nova/+bug/1780225 with the functional tests | 09:35 |
| openstack | Launchpad bug 1780225 in OpenStack Compute (nova) "Libvirt error when using --max > 1 with vGPU" [High,Confirmed] - Assigned to Sylvain Bauza (sylvain-bauza) | 09:35 |
| *** sapd1_x has quit IRC | 09:35 | |
| gibi | bauzas: you could try that case when ask for 2 instance with 2 vGPUs each, then the compute has two PGPUs 2 VGPU each | 09:37 |
| aarents | lyarwood: just FYI http://paste.openstack.org/show/792735/ | 09:38 |
| gibi | I now assume that the multi create will try to allocate the resources in placement for the second intance from the same PGPU as for the first instance due to https://github.com/openstack/nova/blob/master/nova/scheduler/filter_scheduler.py#L238 | 09:39 |
| gibi | bauzas: as we are never considering the second allocation candidate from the a host | 09:39 |
| gibi | s/considering/consider/ | 09:39 |
| bauzas | gibi: okay, lemme try to do it | 09:39 |
| gibi | OK | 09:39 |
| gibi | bauzas: in general I agree that the songwenping_'s bug is not an RC2 candidate. Let's just document it | 09:40 |
| bauzas | gibi: /me just looks how to simulate multi create with functests | 09:40 |
| gibi | bauzas: test_create_multiple_servers | 09:41 |
| gibi | bauzas: https://github.com/openstack/nova/blob/be9afebd6a6c400d2c7f7a13dfe25c8c641c1baa/nova/tests/functional/test_servers.py#L589 | 09:41 |
| bauzas | thanks, I know we had it | 09:41 |
| bauzas | but I was trying to find the test :) | 09:41 |
| bauzas | test_servers <3 | 09:42 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Add evacuation pre-conditions around the src host https://review.opendev.org/723853 | 09:42 |
| lyarwood | aarents: ack thanks | 09:42 |
| lyarwood | aarents: so it's always adding that additional block | 09:43 |
| lyarwood | aarents: ah sorry you snipped the output | 09:45 |
| lyarwood | aarents: so sometimes it did? | 09:45 |
| aarents | lyarwood: not always.. that's odd, in the paste only 7% of times and it depend of size you fallocate | 09:45 |
| aarents | lyarwood: yep | 09:45 |
| lyarwood | aarents: right my bad | 09:45 |
| lyarwood | aarents: yeah that's weird | 09:45 |
| gibi | brinzhang, songwenping_ : so I assume that some of you will propose patach that documents the limitation about multi create with accelerators | 09:47 |
| bauzas | grrrr, /me raises hand at tox.ini when you change dependencies and then you only have DSL 7Mbps bandwidth | 09:47 |
| gibi | brinzhang_, songwenping_: let me know if you need help | 09:47 |
| *** sapd1_x has joined #openstack-nova | 09:51 | |
| *** tkajinam has quit IRC | 09:54 | |
| *** ttsiouts has quit IRC | 09:55 | |
| bauzas | gibi: interesting, when trying to multi create 2 instances asking for 1 vGPU each, I get no exceptions but it tells me 4 vGPUs instead of 2 | 09:56 |
| *** brinzhang has joined #openstack-nova | 09:56 | |
| bauzas | (one compute, 2 pGPUs with a capacity of 8 each) | 09:57 |
| brinzhang | gibi: later, we will propose a patch, to note the multi create in https://docs.openstack.org/api-guide/compute/accelerator-support.html | 09:57 |
| *** ttsiouts has joined #openstack-nova | 09:58 | |
| bauzas | gibi: ahah, no, that's just my test which is wrong, we create 2 mdevs :p | 09:58 |
| *** brinzhang_ has quit IRC | 10:00 | |
| bauzas | gibi: so, I can't reproduce https://bugs.launchpad.net/nova/+bug/1780225 with my functional test | 10:00 |
| openstack | Launchpad bug 1780225 in OpenStack Compute (nova) "Libvirt error when using --max > 1 with vGPU" [High,Confirmed] - Assigned to Sylvain Bauza (sylvain-bauza) | 10:00 |
| bauzas | given it was for Rocky, I think it was fixed by the fact that in Stein we have a RP per pGPU | 10:05 |
| *** Liang__ has quit IRC | 10:10 | |
| *** rpittau is now known as rpittau|bbl | 10:10 | |
| gibi | brinzhang: ack, thanks a lot | 10:13 |
| brinzhang | gibi: np | 10:14 |
| gibi | bauzas: do you try that with the libvirt driver or with the fake driver? I think the placement issue can be recreated with the fake driver | 10:15 |
| bauzas | gibi: i use the fakelibvirt driver | 10:15 |
| bauzas | not the fake driver itself | 10:15 |
| gibi | I see | 10:15 |
| gibi | anyhow it is not burning hot issue right now and songwenping_ PoC seems to be a step in the good direction | 10:16 |
| gibi | as soon as we have a func recreate for the accelerator case we can adapt that for the VGPU case as well (with the fake driver) | 10:17 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: WIP: Test multi create with vGPUs https://review.opendev.org/723858 | 10:24 |
| bauzas | gibi: ^ | 10:24 |
| gibi | bauzas: ack, I will look | 10:24 |
| bauzas | gibi: I'll provide a new revision for it with testing what happens when you ask for 2 vGPUs in multicreate but each pGPU can only create one | 10:24 |
| kashyap | lyarwood: Hi, I'm just digging into the Q35 failure with 'virt-preview': seems like the logs are already gone | 10:25 |
| gibi | bauzas: OK | 10:25 |
| bauzas | gibi: tbc, placement doesn't support sharding resources over RPs but here this is not the issue | 10:26 |
| kashyap | lyarwood: I take it that you haven't had a chance to look at them I just did a 'recheck' for it to reun | 10:26 |
| kashyap | s/reun/rerun/ | 10:26 |
| gibi | bauzas: yeah, if we have two instance requesting one VGPU each then we never allocate two VGPUs in the same placement request so no sharding is needed from placement | 10:27 |
| *** jaosorior has joined #openstack-nova | 10:30 | |
| *** dasp has quit IRC | 10:32 | |
| lyarwood | kashyap: I haven't sorry | 10:34 |
| bauzas | gibi: interesting, we don't have the problem for multi-create with vGPUs filling up capacity | 10:35 |
| *** dasp has joined #openstack-nova | 10:35 | |
| * bauzas just uploads the new revision | 10:35 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: Test multi create with vGPUs https://review.opendev.org/723858 | 10:39 |
| bauzas | gibi: see above, I created two instances with 8 vGPUs each | 10:39 |
| kashyap | yarwood: No problem; I'll dig in | 10:39 |
| bauzas | oh snap | 10:42 |
| * bauzas facepalms | 10:43 | |
| bauzas | (I just ran the older test and now the new one) | 10:43 |
| bauzas | ah, reproduced | 10:45 |
| *** brinzhang_ has joined #openstack-nova | 10:48 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova master: Test multi create with vGPUs https://review.opendev.org/723858 | 11:04 |
| bauzas | brinzhang: songwenping_: gibi: confirmed the issue for vGPUs | 11:05 |
| bauzas | see the above patch https://review.opendev.org/723858 | 11:05 |
| bauzas | so the problem is not related to cyborg but rather for all nested resource providers | 11:05 |
| bauzas | anyway, not a RC regression given we had the same issue in Train for vGPUs (and possibly bandwidth-aware instances,wdyt gibi ?) | 11:06 |
| * bauzas goes lunching | 11:06 | |
| *** brtknr has quit IRC | 11:07 | |
| *** ociuhandu has quit IRC | 11:11 | |
| *** brtknr has joined #openstack-nova | 11:15 | |
| gibi | bauzas: thanks for the reporoduction | 11:16 |
| *** ociuhandu has joined #openstack-nova | 11:19 | |
| *** brtknr has quit IRC | 11:23 | |
| *** tosky has quit IRC | 11:23 | |
| brinzhang_ | gibi: my partner proposed a bug, see https://bugs.launchpad.net/nova/+bug/1875624 | 11:25 |
| openstack | Launchpad bug 1875624 in OpenStack Compute (nova) "the vms can not be force deleted when vm_status is soft-delete and task-state=deleting" [Undecided,New] - Assigned to xuyuanhao (thourch) | 11:25 |
| brinzhang_ | gibi: can you check | 11:25 |
| gibi | brinzhang_: will check soon | 11:25 |
| brinzhang_ | gibi: thanks | 11:25 |
| *** iurygregory has quit IRC | 11:26 | |
| *** iurygregory has joined #openstack-nova | 11:26 | |
| *** brtknr has joined #openstack-nova | 11:28 | |
| *** tkajinam has joined #openstack-nova | 11:34 | |
| gibi | brinzhang_: confirmed the bug. | 11:40 |
| *** ociuhandu has quit IRC | 11:42 | |
| *** ociuhandu has joined #openstack-nova | 11:44 | |
| *** ttsiouts has quit IRC | 11:45 | |
| *** mgariepy has joined #openstack-nova | 11:54 | |
| *** tosky has joined #openstack-nova | 11:54 | |
| *** ociuhandu has quit IRC | 11:55 | |
| *** ociuhandu has joined #openstack-nova | 11:57 | |
| *** nweinber has joined #openstack-nova | 11:58 | |
| *** belmoreira has joined #openstack-nova | 11:58 | |
| *** dklyle has quit IRC | 12:00 | |
| *** mgariepy has quit IRC | 12:02 | |
| *** raildo has joined #openstack-nova | 12:03 | |
| *** rpittau|bbl is now known as rpittau | 12:08 | |
| *** ociuhandu has quit IRC | 12:11 | |
| *** derekh has quit IRC | 12:13 | |
| *** mgariepy has joined #openstack-nova | 12:16 | |
| *** songwenping__ has joined #openstack-nova | 12:17 | |
| *** happyhemant has quit IRC | 12:20 | |
| *** songwenping_ has quit IRC | 12:20 | |
| *** ociuhandu has joined #openstack-nova | 12:21 | |
| *** ChanServ changes topic to "Current runways: https://etherpad.openstack.org/p/nova-runways-ussuri -- This channel is for Nova development. For support of Nova deployments, please use #openstack." | 12:27 | |
| -openstackstatus- NOTICE: Zuul has been restarted, all events are lost, recheck or re-approve any changes submitted since 9:50 UTC. | 12:27 | |
| bauzas | gibi: brinzhang: FWIW, I think we should change the title of bug 1874664 to make it clear it's for all nested Resource Providers | 12:30 |
| openstack | bug 1874664 in OpenStack Compute (nova) "Boot more than one instances failed with accelerators in its flavor" [Medium,Confirmed] https://launchpad.net/bugs/1874664 - Assigned to Wenping Song (wenping1) | 12:30 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Add nested resource providers limit for multi create https://review.opendev.org/723884 | 12:30 |
| gibi | bauzas: I agree | 12:31 |
| *** artom has joined #openstack-nova | 12:32 | |
| bauzas | changed https://bugs.launchpad.net/nova/+bug/1874664 | 12:32 |
| openstack | Launchpad bug 1874664 in OpenStack Compute (nova) "Instance multi-create doesn't support available resources spread between children RPs" [Medium,Confirmed] - Assigned to Wenping Song (wenping1) | 12:32 |
| bauzas | gibi: have you tested it for bandwidth-aware instances ? | 12:33 |
| bauzas | actually, that's traits, right? | 12:33 |
| gibi | bauzas: multi create with neutron ports are not supported even without bandwidth | 12:34 |
| gibi | multi create with neutron net is supported without bandwidth | 12:34 |
| bauzas | ack ok | 12:35 |
| bauzas | fwiw, multi-create works with vGPUs if one RP has all the capacity for all the instances | 12:35 |
| brinzhang_ | bauzas, gibi: I am now confused (dazzled) ^^ | 12:35 |
| bauzas | and I guess it's the same for cyborg resources | 12:36 |
| *** dpawlik has quit IRC | 12:36 | |
| gibi | brinzhang_, bauzas: I have to jump on a call, sorry. I will read back later | 12:36 |
| *** songwenping__ has quit IRC | 12:36 | |
| brinzhang_ | bauzas: you mean change "Add" to "All"? | 12:36 |
| brinzhang_ | bauzas: if possiable, you can edit that patch, or you can leave comments inline, I have to go home now, it's too later for me, I am sorry | 12:38 |
| brinzhang_ | gibi, bauzas: thanks ^^ | 12:38 |
| *** dpawlik has joined #openstack-nova | 12:40 | |
| bauzas | brinzhang: see the bug description modification https://bugs.launchpad.net/nova/+bug/1874664 | 12:40 |
| openstack | Launchpad bug 1874664 in OpenStack Compute (nova) "Instance multi-create doesn't support available resources spread between children RPs" [Medium,Confirmed] - Assigned to Wenping Song (wenping1) | 12:40 |
| brinzhang_ | bauzas: I think that use you write in bug description in my patch | 12:47 |
| brinzhang_ | it looks better | 12:47 |
| brinzhang_ | an easy to understand | 12:47 |
| *** jsuchome has joined #openstack-nova | 12:47 | |
| brinzhang_ | s/an/and | 12:47 |
| *** alistarle has joined #openstack-nova | 12:48 | |
| *** ratailor has quit IRC | 12:48 | |
| AJaeger | The nova ussuri translations are at https://review.opendev.org/723160, any stable nova core to import them, please? | 13:01 |
| *** dklyle has joined #openstack-nova | 13:02 | |
| nightmare_unreal | can someone review this : https://review.opendev.org/#/c/715395/ Thanks | 13:02 |
| *** alistarle has quit IRC | 13:05 | |
| *** vesper11 has quit IRC | 13:06 | |
| *** vesper has joined #openstack-nova | 13:06 | |
| *** mkrai has quit IRC | 13:11 | |
| elod | AJaeger: about https://review.opendev.org/723160 : if I understand correctly it is safe to merge now and don't really need any extra review. Am I right? | 13:18 |
| *** derekh has joined #openstack-nova | 13:18 | |
| AJaeger | elod: it's safe to merge now and most projects have single core review. | 13:20 |
| AJaeger | elod: let me grab you a link for the safe... | 13:20 |
| AJaeger | elod: http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014437.html has "For projects with translations, watch for any translation patches coming through and merge them quickly" | 13:21 |
| elod | AJaeger: thanks! reading | 13:21 |
| elod | ohh, the release countdown mail, i see, thanks | 13:23 |
| AJaeger | yep, that one | 13:23 |
| elod | AJaeger: approved | 13:24 |
| *** eharney has quit IRC | 13:25 | |
| artom | stephenfin, any chance I could get you to revisit https://review.opendev.org/#/c/687404/20 ? | 13:30 |
| *** ttsiouts has joined #openstack-nova | 13:32 | |
| *** eharney has joined #openstack-nova | 13:34 | |
| AJaeger | thanks, elod | 13:41 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/nova stable/train: Add default cpu model for AArch64 https://review.opendev.org/723900 | 13:43 |
| *** psachin has quit IRC | 13:46 | |
| elod | well, thanks for calling my attention to that patch. I've already had a look at it, but saw that zuul was in bad shape so waited for zuul to get back to normal | 13:48 |
| stephenfin | artom: sure | 13:53 |
| *** ttsiouts has quit IRC | 13:56 | |
| *** hamzy has joined #openstack-nova | 14:00 | |
| * gibi jumps from one call to another | 14:02 | |
| * kashyap waits for gibi to report "video fatique" :D | 14:04 | |
| kashyap | s/fatique/fatigue/ | 14:04 |
| *** mriedem has joined #openstack-nova | 14:12 | |
| *** ttsiouts has joined #openstack-nova | 14:12 | |
| * gibi has covid fatigue | 14:16 | |
| artom | stephenfin, thanks :) | 14:17 |
| gmann | gibi: thanks, i added comment. i think changing 'oslopolicy-sample-generator' depends how we change it, say adding deprecated rules by default or based on request. because there might be operator who are using this tool for no-deprecated-rules usage | 14:21 |
| *** ttsiouts has quit IRC | 14:22 | |
| gmann | which was the only way to move to new defaults and stop old token to pass via deprecated rule until we introduced new flag in oslo.policy 'oslo_policy.enforce_new_defaults' | 14:22 |
| stephenfin | dansmith: Now that we've branched, could you take a look at https://review.opendev.org/#/c/537414/ and https://review.opendev.org/#/c/530905/ again? | 14:22 |
| gmann | may be bnemec and stephenfin can input how we can accommodate both use case of 'oslopolicy-sample-generator' here. | 14:23 |
| gmann | i mean adding deprecated based on request can be done but if it solve the zigo case. | 14:23 |
| dansmith | stephenfin: ack | 14:23 |
| stephenfin | dansmith: ta | 14:23 |
| dansmith | gmann: I thought it was said that the generator wasn't going to get that new mode | 14:23 |
| dansmith | or at least, not in time | 14:24 |
| gmann | dansmith: yeah that was my understanding but gibi opinion is to do that if we can. | 14:25 |
| bnemec | I'm not a big fan of adding a feature to support an anti-pattern in deploying OpenStack. | 14:25 |
| stephenfin | Could we just handwrite a policy.json file and include it in our sdist for this release? | 14:26 |
| dansmith | gmann: the problem I see is that the oslo tool will not do that by default, but our default in nova is to need the deprecated rules, | 14:26 |
| stephenfin | zigo could consume that instead of using the oslopolicy tool | 14:26 |
| gmann | we have to carefully things of way for operator to move to new defaults was only overwrite the rule in policy file which is this case we taken as broken | 14:26 |
| dansmith | so we're requiring a lot of people to know to disable one default and override another, to get a consistent set of defaults that will work | 14:26 |
| artom | stephenfin, thanks :) | 14:27 |
| gmann | exactly | 14:27 |
| artom | dansmith, https://review.opendev.org/#/c/672595/73 pretty please? When your queue gets to it | 14:27 |
| *** udesale_ has joined #openstack-nova | 14:27 | |
| dansmith | stephenfin: I think that's more obscure for people that aren't already looking for a sample file, but maybe easier to get themselves out of the hole once they realize they've deployed and screwed themselves up | 14:27 |
| * artom wonders if the NUMA LM func tests can beat the device tagging tempest tests record | 14:27 | |
| dansmith | artom: ack | 14:28 |
| *** udesale has quit IRC | 14:29 | |
| gmann | gibi: I will update the patch for reno comments and will keep bug open. and we can discuss the best possible approach about policy file usage in cross project sessions in PTG what bnemec added in oslo etherpad and i linked in nova ptg ethrepad too. | 14:31 |
| gmann | gibi: zigo that works for you ^^ ? | 14:32 |
| *** rpittau is now known as rpittau|brb | 14:32 | |
| *** links has quit IRC | 14:34 | |
| dansmith | stephenfin: oh right, yeah I'm not going to approve those completely inconsistent style things in the middle of a file, but I'm sure someone else will | 14:35 |
| stephenfin | huh? | 14:36 |
| dansmith | the "black says this ugly style is cool, so I'll just break from the rest of nova conventions here for the new code I'm adding" thing in your tests | 14:37 |
| stephenfin | um, those are entirely new tests? | 14:37 |
| dansmith | in a file with a style, in a project with a style | 14:37 |
| stephenfin | really? | 14:38 |
| dansmith | you're not even consistent within those tests | 14:38 |
| stephenfin | we're inconsistent all over the place | 14:39 |
| stephenfin | you're not going to hammer me for style, surely? | 14:39 |
| *** bauzas has quit IRC | 14:43 | |
| artom | stephenfin, dansmith, hey, I may be late to the party, but those .contains("nova_object.name") searches... | 14:43 |
| *** bauzas has joined #openstack-nova | 14:43 | |
| artom | This kind of substring search is notoriously slow in SQL, no? | 14:43 |
| artom | For the compute nodes, it's probably fine, since there aren't that many of them, but for instances... | 14:43 |
| dansmith | artom: they're not indexed, so yes | 14:43 |
| artom | So we're just accepting that? | 14:45 |
| artom | I dunno if that's been talked about before, as I said, I'm late to the party | 14:45 |
| stephenfin | I don't think we've a choice. There's no other heuristic we can use to differentiate the legacy entries from the non-legacy ones | 14:46 |
| dansmith | I guess if we did it in python, despite being slower, it's load on the machine running the nova-manage, instead of the DB server itself, which is probably better | 14:46 |
| *** bauzas has quit IRC | 14:47 | |
| dansmith | although maybe we'd have no filter at that point and basically return all the instances | 14:47 |
| dansmith | so yeah I dunno | 14:47 |
| *** mlavalle has joined #openstack-nova | 14:47 | |
| *** bauzas has joined #openstack-nova | 14:48 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Clarify the policy new defaults upgrade notes https://review.opendev.org/723645 | 14:49 |
| gmann | dansmith: gibi zigo updated , please check - https://review.opendev.org/#/c/723645/ | 14:49 |
| artom | dansmith, stephenfin, I guess there's no way to do "pagination" when selecting instance_extra? | 14:50 |
| dansmith | artom: what would that help? | 14:50 |
| artom | dansmith, select all instance_extra in steps of 100 or whatever, do the filtering in Python, find the instance UUIDs that way, then with those go back to the database? | 14:51 |
| artom | Apparently you can do stuff with LIMIT: https://stackoverflow.com/questions/3799193/mysql-data-best-way-to-implement-paging | 14:52 |
| dansmith | artom: but we'd still end up selecting every instance out of the database every time right? | 14:53 |
| dansmith | every instance, in groups of 100 | 14:53 |
| artom | Yeah, but at least it's spread out over time? | 14:53 |
| artom | You're not filtering on all X thousands instances in a single query | 14:53 |
| dansmith | artom: well, the migrations are supposed to be idempotent, so ideally you can run this over and over and have less and less impact | 14:53 |
| gmann | melwitt: all setup for devstack and grenade now. we can approve this - https://review.opendev.org/#/c/704364/5 | 14:54 |
| gmann | and then backport | 14:54 |
| dansmith | artom: I think I'd rather the string filter in the query than just continually select every instance out of the database on every run, only to find that none are needed | 14:54 |
| gmann | *done now | 14:54 |
| dansmith | artom: I'd have to iterate every instance in the DB every time you asked me "are you done?" | 14:54 |
| artom | dansmith, but isn't that what the string filter in the query is doing anyways, behind the scenes? | 14:55 |
| artom | Looking at every instance in the database? | 14:55 |
| dansmith | artom: no, it's looking at every value in a table | 14:55 |
| dansmith | which is massively less expensive than selecting multiple tables, feeding those to python, constructing tons and tons of objects, for us to go through and do...that same string comparison | 14:56 |
| artom | dansmith, right sorry - but what's what I was suggesting we do in python - select instance_uuid, numa_topology from instance_extra <in steps of 100>, fitler in python, use those instance_uuids for the actual update | 14:56 |
| *** rpittau|brb is now known as rpittau | 14:57 | |
| artom | But maybe ^^^ is impossible with our models/sqlalchemy? | 14:57 |
| dansmith | you mean a direct query and not an ORM one? that's better, but you're still sending the results of all those queries across the network every time | 14:57 |
| dansmith | no, we can do it without an ORM query, we do that in places | 14:57 |
| *** ociuhandu has quit IRC | 14:57 | |
| artom | dansmith, yeah :/ | 14:58 |
| artom | I guess we'd need metrics of how long the string filter would take, as a function of # of instances | 14:59 |
| artom | To see if we're making a mountain our of a molehill | 14:59 |
| dansmith | it'll definitely be faster for the db server to do it, the only benefit I can see is you offload the cpu processing to another machine, but you pay the penalty of increasing latency and definitely network traffic | 14:59 |
| dansmith | besides, stephenfin will want to nuke this migration in a cycle or two anyway, so it won't be run forever | 15:00 |
| *** ociuhandu has joined #openstack-nova | 15:00 | |
| gibi | away | 15:01 |
| * gibi is reading back | 15:01 | |
| artom | dansmith, I suppose asking CERN to test this for us is not an option ;) | 15:02 |
| dansmith | artom: sure you can ask them for their opinion on the pain | 15:03 |
| dansmith | I dunno what the alternative is though | 15:03 |
| artom | Yeah, valid point | 15:03 |
| dansmith | especially in cern's case, the impact of sending their massive amount of data over the network is not likely better | 15:03 |
| dansmith | and, they have upgrade windows where they can handle this kind of thing | 15:03 |
| dansmith | artom: well, one alternative is to avoid doing this, which is valid | 15:04 |
| dansmith | artom: this is cleanup that we don't need to do, other than to address stephenfin's OCD | 15:04 |
| artom | I tend to like stephenfin OCD ;) | 15:04 |
| dansmith | which while this seems like a good one to do, because it's data at rest, it's not *really* costing us much to maintain the compatibility | 15:04 |
| artom | dansmith, I don't suppose doing it on the fly is an option? | 15:05 |
| *** ociuhandu has quit IRC | 15:05 | |
| dansmith | artom: we're already doing it on the fly, that's my point | 15:05 |
| artom | Like, every time we call those legacy dict methods, update the DB? | 15:05 |
| *** ociuhandu has joined #openstack-nova | 15:05 | |
| artom | Seriously? | 15:05 |
| artom | Yeeeaaahhh | 15:05 |
| artom | I have to say, that sways me the other way around | 15:05 |
| dansmith | we're doing it on read, and if we save it for some reason (like a migration) then we update it | 15:05 |
| dansmith | we could do update-on-read too, which we've done in other places | 15:06 |
| artom | Update on read would be nice | 15:06 |
| dansmith | I'd be cool with that | 15:06 |
| dansmith | despite this looking nicer, it really isn't *necessary* to update this.. these compat routines have been here for ages | 15:06 |
| artom | I'd be less worried if the hit to the DB was less | 15:07 |
| artom | If jaypipes was still around he could set us straight | 15:07 |
| artom | Maybe I'm paranoid for nothing | 15:07 |
| dansmith | yeah, I think it's a good insight.. I honestly hadn't really considered it, but I think you've got a good point | 15:07 |
| dansmith | if it was something we really had to do, then I'd be for this approach, let the DB server do the hard bit, but... | 15:08 |
| dansmith | the other thing about this, is for instances created since like 2018, this is just pain for no reason | 15:08 |
| dansmith | or whenever we moved to the new format | 15:08 |
| dansmith | this is really just looking for super ancient instances that have been nursed for a long time, | 15:08 |
| dansmith | which definitely puts it into perspective for me | 15:09 |
| stephenfin | 2014 | 15:09 |
| dansmith | hah | 15:09 |
| artom | stephenfin, your call I guess - looks like update on read might be a more acceptable solution, though with it we'd never be 100% sure we got all of those ancient instances | 15:12 |
| stephenfin | And this isn't OCD. I've to maintain the hardware.py module and uglier corners of libvirt driver code, and I'm continuously trying to burn down tech debt inflicted upon me by others to make that job easier | 15:12 |
| *** udesale_ has quit IRC | 15:13 | |
| openstackgerrit | Merged openstack/nova master: Imported Translations from Zanata https://review.opendev.org/722644 | 15:13 |
| stephenfin | artom: personally, I was for deleting all the compat code to handle instances that hadn't been _touched_ since 2014, but dansmith said that wasn't good enough so I did this | 15:14 |
| stephenfin | and have been working on it for two years | 15:14 |
| artom | stephenfin, sorry dude, didn't mean to sabotage it :( | 15:14 |
| artom | But it would have been dishonest to silence my concern | 15:14 |
| artom | belmoreira, around? We're having a DB performance discussion, and figured CERN's scale might shed some light | 15:15 |
| *** tkajinam has quit IRC | 15:17 | |
| belmoreira | artom I just need to leave now. If you can I will be around tomorrow | 15:17 |
| artom | belmoreira, ok, I'll ping you again earlier tomorrow morning (I'm on EDT) | 15:17 |
| artom | I'm trying to read up on LIKE performance, which I *assume* is what sqlalchemy boils down .contains() to... | 15:20 |
| dansmith | like is a little more expensive than a strstr() I think, although it might collapse to that if mysql notices there are no pattern characters | 15:21 |
| *** gyee has joined #openstack-nova | 15:21 | |
| *** mgariepy has quit IRC | 15:24 | |
| gibi | dansmith, gmann, stephenfin, zigo: OK so I see that the change in the policy generator is not feasible before ussuri GA and might also seemed as a step backward to have that change in V. Then are we suggesting to zigo that his use case is not really valid / supported in Ussuri and in Victoria? | 15:25 |
| gibi | or what is the way forward top of the reno update? | 15:25 |
| *** mgariepy has joined #openstack-nova | 15:26 | |
| dansmith | gibi: no, I think zigo's case is very valid, and common | 15:26 |
| dansmith | gibi: I think we've screwed up here, and I think all I've said is that I don't think we have a lot of good options | 15:26 |
| gmann | but should we consider the "new policy file generated without deprecated rules to switch to new defaults" are not valid ? | 15:27 |
| gmann | because in both cases, policy file can be used that way | 15:28 |
| gibi | dansmith: thanks. I feel that we have no good options | 15:28 |
| dansmith | indeed :/ | 15:28 |
| gibi | gmann: do you mean that use the new generated file and configure nova and keystone with scopes? | 15:29 |
| gibi | gmann: I think that is a valid use case | 15:29 |
| dansmith | we can't do that in the upgrade I think | 15:29 |
| dansmith | because people with non-scoped current policy files will break | 15:29 |
| gmann | yes. and in past also if we deprecate any policy rule and operator want to move to new defalt, policy overwrite is the option they had | 15:30 |
| gibi | dansmith: true, not for the upgrade, but for the new deployments | 15:30 |
| dansmith | gibi: only distros have the ability to do something different for upgrade vs. new I think | 15:30 |
| dansmith | gibi: we the nova project have to assume upgrade | 15:30 |
| gmann | yeah | 15:30 |
| dansmith | I think what we can do is set up for the most default case, which is where we are now, and accept that the case where you're using the generation tool is going to break | 15:30 |
| dansmith | it's likely common, but less common than all the other cases I think | 15:31 |
| dansmith | accept, and document/warn about the potential problem I mean | 15:32 |
| openstackgerrit | Merged openstack/nova stable/ussuri: Imported Translations from Zanata https://review.opendev.org/723160 | 15:32 |
| gibi | OK, I see. thanks. I will summarize it in the bug | 15:32 |
| gmann | dansmith: i agree it might be common to re-generate policy file and end up no deprecated rule but is not that wrong usage ? and some point we have to tell them its not right one | 15:32 |
| dansmith | gmann: no, I don't think it's wrong.. it's not what we want people to do, but what we want them to do isn't very convenient or user-friendly, which is why I think it's likely common | 15:33 |
| *** mgariepy has quit IRC | 15:33 | |
| dansmith | gibi: also the other action item is to switch to yaml policy by default going forward I think, and encourage people to move to that | 15:33 |
| gmann | and it is like it was not reported before when few policy default were changed. i am not sure if new default were superset of old so that same problem would occur | 15:33 |
| gmann | dansmith: +1 on switch yaml. | 15:34 |
| gmann | and this bug - https://bugs.launchpad.net/oslo.policy/+bug/1853170 | 15:34 |
| openstack | Launchpad bug 1853170 in oslo.policy "Need documentation on recommended operator workflow for deprecated policies" [High,Triaged] | 15:34 |
| gibi | dansmith: good point about yaml, adding that to the PTG etherpad | 15:34 |
| dansmith | gmann: this we can all agree on :P | 15:34 |
| gmann | we need some consistent usage guide also | 15:34 |
| gmann | gibi: its there, in cross project section | 15:35 |
| gibi | gmann: even the yaml usage? | 15:35 |
| *** AJaeger has left #openstack-nova | 15:35 | |
| gmann | gibi: ah i thought it is written in description but i missed. please add | 15:36 |
| gibi | done :) | 15:36 |
| gmann | thanks | 15:36 |
| gmann | dansmith: gibi new flag aded in ussuri (https://bugs.launchpad.net/nova/+bug/1875418) to switch to new default instead of overwriting the policy file can improve the usage at some extend. | 15:38 |
| openstack | Launchpad bug 1875418 in OpenStack Compute (nova) "Generated policy.json in Ussuri is broken by default" [High,In progress] - Assigned to Ghanshyam Mann (ghanshyammann) | 15:38 |
| gmann | sorry, this flag - 'oslo_policy.enforce_new_defaults' | 15:38 |
| dansmith | gmann: this is separate from the full switch to only scoped policies right | 15:39 |
| dansmith | ? | 15:39 |
| gmann | dansmith: yes. | 15:39 |
| dansmith | so, I think maybe we should just go ahead and shut the door in V on using the old stuff | 15:39 |
| dansmith | because this is now a failure on the upgrade "policy" we should just hard pivot over to the new stuff in V, | 15:40 |
| dansmith | apologize for the short notice, and move on | 15:40 |
| dansmith | and maybe our U renos need to state that.. "might as well go ahead and convert in U to avoid this again in V" | 15:40 |
| *** belmoreira has quit IRC | 15:40 | |
| bauzas | gibi: others: procedural question but https://bugs.launchpad.net/nova/+bug/1875418 should have a ussuri-rc-candidate tag or not ? | 15:41 |
| openstack | Launchpad bug 1875418 in OpenStack Compute (nova) "Generated policy.json in Ussuri is broken by default" [High,In progress] - Assigned to Ghanshyam Mann (ghanshyammann) | 15:41 |
| gmann | dansmith: yeah mentioning V upgrade in reno is good idea. but removing old stuff in V might be difficult now as we have conveyed the old support till W in few wanrings and original reno | 15:42 |
| gibi | bauzas: as far as I see we will keep the bug open after Ussuri so it should not be tagged | 15:42 |
| dansmith | gmann: we still have time to correct those warnings though right? | 15:42 |
| gmann | dansmith: humm, we can do and bakport. but i am thinking if that is too early for people not re-generating the file. i mean they need to adopt new scope token which is very new things. | 15:43 |
| gibi | I've updated the policy bug with the current agreement above. I have to leave for today, will read back tomorrow | 15:43 |
| bauzas | honestly, this bug scares me | 15:43 |
| bauzas | couldn't we just tell that we won't support new policies until V ? | 15:44 |
| dansmith | gmann: well, it just feels like extending this any longer than necessary is worse than concentrating al the pain | 15:44 |
| dansmith | bauzas: we've already broken a set of users though, | 15:44 |
| bauzas | chances are that operators wouldn't read relnotes | 15:44 |
| bauzas | dansmith: :( | 15:44 |
| dansmith | so I was going to say instead of breaking 25% of them now, and then 75% of them later, we should just get it over with | 15:45 |
| bauzas | like, "take a pill, and suffer in silence" ? :) | 15:45 |
| dansmith | or "rip off the bandage" | 15:45 |
| bauzas | I guess we can't obviously make it a pre-upgrade check | 15:46 |
| gmann | or we add the oslo guide on best handle the deprecated rule in file - https://bugs.launchpad.net/oslo.policy/+bug/1853170 | 15:46 |
| openstack | Launchpad bug 1853170 in oslo.policy "Need documentation on recommended operator workflow for deprecated policies" [High,Triaged] | 15:46 |
| bauzas | this would require us to write some Train patch and a release | 15:46 |
| *** mgariepy has joined #openstack-nova | 15:46 | |
| bauzas | but this would be nice, they'd get the warnings before upgrading to U | 15:47 |
| dansmith | bauzas: we could definitely do some nova-status guessing, yeah | 15:47 |
| dansmith | bauzas: not a train patch, a U patch.. nova-status from U helps warn about things from T->U | 15:48 |
| bauzas | dansmith: but this would be a pre-Victoria check, right ? (unless we backport to Train the patch itself) | 15:48 |
| openstackgerrit | Merged openstack/nova master: Switch to TOX_CONSTRAINTS_FILE https://review.opendev.org/722814 | 15:48 |
| openstackgerrit | Merged openstack/nova master: Test multi create with vGPUs https://review.opendev.org/723858 | 15:48 |
| bauzas | mmmm, amiwrong ? | 15:48 |
| dansmith | bauzas: depends on which bit you're talking about warning for | 15:48 |
| openstackgerrit | Merged openstack/nova master: Update contributor guide for Victoria https://review.opendev.org/722647 | 15:48 |
| dansmith | bauzas: the stuff we're breaking in U would be a nova-status U patch to warn the T people | 15:48 |
| bauzas | this would require a U install somewhere but okay | 15:49 |
| dansmith | that's the way nova-status works | 15:49 |
| bauzas | I'm then confused | 15:49 |
| bauzas | then, it's all good | 15:49 |
| bauzas | let's make it a nova-status upgrade check and yell something is wrong | 15:49 |
| bauzas | double this with relnotes | 15:50 |
| bauzas | and gosh saves the rest | 15:50 |
| *** brinzhang_ has quit IRC | 15:50 | |
| bauzas | gmann: ^ | 15:50 |
| *** brinzhang_ has joined #openstack-nova | 15:50 | |
| jsuchome | efried, bauzas, dansmith: Hi, could we please get https://review.opendev.org/#/c/572805/ reopen _again_ ? | 15:51 |
| gmann | how we will differentiate the upgrade with re-generated file vs new deployment/upgrade moving to new system | 15:51 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Add online migration for legacy NUMA objects https://review.opendev.org/537414 | 15:51 |
| bauzas | gmann: greenfields don't run the nova-status check | 15:51 |
| dansmith | ah, yeah I guess gmann has a point | 15:52 |
| bauzas | make it a flag (c) | 15:52 |
| efried | gmann: restored | 15:52 |
| dansmith | jsuchome: are you going to work on it yourself? | 15:53 |
| jsuchome | unless tobiash has time to pick it up ... | 15:53 |
| stephenfin | artom: I switched that to startswith which is as optimal as I can get, aside from enumerating every possible legacy NUMA topology configuration :) | 15:53 |
| artom | stephenfin, hehe | 15:54 |
| gmann | I still feel (dansmith might be angry on me saying this again and again ) this usage of re-generated file is wrong thought common and telling them to use it in right way (with consistent guide on oslo side) can correct the things for long term too. | 15:54 |
| gmann | *though | 15:54 |
| dansmith | gmann: the right way is to switch to yaml, fully commented :) | 15:54 |
| bauzas | gmann: the ship has sailed. | 15:55 |
| dansmith | bauzas: agree :) | 15:56 |
| gmann | yeah, yaml and they can un-comment the rule with new values if they want. Honestly satying, that the way i was thinking people using policy file after policy-in-code | 15:56 |
| gmann | humm | 15:56 |
| bauzas | gmann: can you please clarify the problem you have for distinguishing the different upgrade cases ? | 15:56 |
| bauzas | I see three cases | 15:57 |
| bauzas | 1/ Ussuri greenfields | 15:57 |
| jsuchome | dansmith: so last time it was tobiash, let's wait if we wants to continue and if not, I would start myself | 15:57 |
| bauzas | 2/ T->U and no policy.json changes | 15:57 |
| bauzas | 3/ T->U and specific policy.json | 15:57 |
| artom | stephenfin, yeah, I left a follow-up comment. I guess I just don't know enough in this case, and what I've been able to learn isn't enough | 15:57 |
| bauzas | gmann: amirite ? | 15:57 |
| gmann | bauzas: policy file with new defaults only (no deprecated old default) cover two case 1. this broken case 2. people move to new defaults intentionally. | 15:57 |
| dansmith | jsuchome: okay | 15:57 |
| gmann | bauzas: 3rd one has ^^ above cases | 15:58 |
| bauzas | gmann: sure, but that's not the case you wanna warn, right? | 15:58 |
| tobiash | jsuchome: feel free to take it, atm I unfortunately I don't have time to work on it | 15:58 |
| stephenfin | zzzeek: I know you're busy, but any chance you'd be able to shine a light on https://review.opendev.org/#/c/537414/ at some point? | 15:58 |
| stephenfin | zzzeek: please and thank you :) | 15:58 |
| *** klindgren has quit IRC | 15:59 | |
| *** lpetrut has joined #openstack-nova | 15:59 | |
| gmann | bauzas: yeah, so we just say in nova-status what we are saying in reno ? | 15:59 |
| *** klindgren has joined #openstack-nova | 15:59 | |
| gmann | i mean new reno - https://review.opendev.org/#/c/723645/ | 15:59 |
| * bauzas looks | 16:00 | |
| bauzas | gmann: but basically, yeah, code in python what you write in restructuredtext | 16:00 |
| bauzas | because parsers aren't good with english syntax | 16:00 |
| zzzeek | stephenfin: whats the quesiton, is contains() or startswith() faster? | 16:01 |
| stephenfin | zzzeek: Yes, if there isn't an index on the column | 16:01 |
| zzzeek | stephenfin: both contains() / startswith() are based on LIKE which is going to need a table scan | 16:01 |
| artom | zzzeek, and more generally, if we're stuck doing text filtering on a column without an index, what would be the lest sucky way? | 16:02 |
| artom | *least | 16:02 |
| bauzas | gmann: I briefly looked at the note | 16:02 |
| zzzeek | I don't see the db/model files here that we're talking about I just see objects files, is there a SQL query in those ? | 16:02 |
| zzzeek | and is the issue that you don't wnat to add a new column with the indexed information you need ? | 16:02 |
| bauzas | gmann: and unless I'm wrong, I don't see a technical problem for providing a nova-status check command verifying this | 16:02 |
| stephenfin | zzzeek: sec, lemme drag up the model | 16:02 |
| zzzeek | oh i see it | 16:03 |
| zzzeek | compute_nodes = (context.session.query(models.ComputeNode) | 16:03 |
| zzzeek | .filter(models.ComputeNode.numa_topology != null()) | 16:03 |
| zzzeek | .filter(~models.ComputeNode.numa_topology.startswith( | 16:03 |
| zzzeek | '{"nova_object.name"')) | 16:03 |
| zzzeek | that ? yeah that's not great :) | 16:03 |
| zzzeek | depends on number of rows | 16:03 |
| zzzeek | < 1000, no problem | 16:03 |
| bauzas | gmann: ie. look at the file, and if you find both old-style and new-style, yell it | 16:03 |
| artom | zzzeek, well, all instances | 16:03 |
| artom | So multiple thousands, maybe into the millions for large deployments | 16:03 |
| zzzeek | including old ones that are offline ? | 16:03 |
| zzzeek | artom: yeah that's not going to be fast | 16:03 |
| stephenfin | artom: not deleted instances | 16:03 |
| stephenfin | tbc | 16:04 |
| zzzeek | you want to get the # of rows as low as possible before applying a like on it | 16:04 |
| artom | stephenfin, does that filtering happen before or after the string filter? | 16:04 |
| gmann | bauzas: ok, let me add in nova-status also | 16:04 |
| bauzas | gmann: if you wanna some examples, go over there https://github.com/openstack/nova/blob/master/nova/cmd/status.py#L364 | 16:04 |
| zzzeek | there are other ways to do this, mysql/mariadb might ahve a MATCH operator but that requires changes | 16:04 |
| stephenfin | I actually don't know /o\ Does filter ordering matter? | 16:04 |
| artom | zzzeek, yeah, IIUC MATCH needs full text index, no? | 16:05 |
| zzzeek | but ideally you'd have the information you need captured in some dedicated column, or if for example the "numa_topology" were a forieng key to some collection table, you could limit the rows on that side to a list of posible foreign key ids, that kind of thing | 16:05 |
| zzzeek | artom: yes | 16:05 |
| gmann | bauzas: sure, thanks | 16:06 |
| bauzas | anyway, /me calls it a day | 16:06 |
| stephenfin | zzzeek: Unfortunately this is a JSON blob rather than an actual table. No foreign keys here | 16:07 |
| zzzeek | so as far as LIKE '%foo%' vs. LIKE 'foo%', I don't know the details of what mysql/mariadb would do, we would imagine the second is "faster" but i dont know that it can use indexes for that | 16:07 |
| stephenfin | and we're trying to move from one format of JSON blob to another | 16:07 |
| artom | zzzeek, we can't play around with the schema for this, we need to work with the columns/indeces that we have | 16:07 |
| artom | Or don't have :( | 16:07 |
| zzzeek | stephenfin: Mysql /mariadb have JSON operators now, do those apply here? | 16:08 |
| zzzeek | e.g. you can extract values from a json structure | 16:08 |
| stephenfin | do those apply if the column type is just TEXT? | 16:08 |
| zzzeek | stephenfin: yes there is no JSON type on both backends, there's ...some kind of keyword on one of htem | 16:08 |
| dansmith | zzzeek: is that really faster than a string comparison? I would expect that would require reading the whole text, more cpu to parse, and then run the comparison | 16:09 |
| zzzeek | mysql has a native JSON mariasdb does not but the json operators should work on either | 16:09 |
| zzzeek | dansmith: I dunno, would need to read some docs | 16:09 |
| dansmith | I would be absolutely blown away if so :) | 16:09 |
| zzzeek | https://mariadb.com/kb/en/json_value/ | 16:10 |
| zzzeek | dansmith: Postgresql JSON type probably builds some index | 16:10 |
| artom | stephenfin, I don't support looking at updated_date is enough? | 16:10 |
| artom | *suppose | 16:10 |
| zzzeek | Mysqls' might as well | 16:10 |
| artom | Or can help in any way? | 16:10 |
| zzzeek | mariadb, not too sure | 16:10 |
| dansmith | zzzeek: sure, if the column type is json then maybe | 16:10 |
| stephenfin | artom: I considered that, but I don't know when someone upgraded | 16:10 |
| stephenfin | so if ~in theory~ someone was on Juno for ages - say, until 2016, and eventually decided to upgrade, and are now upgrading that same cloud to Victoria | 16:11 |
| artom | stephenfin, actually - what about a two step approach? 1. the update on read thing we talked about earlier 2. let it sit for a cycle or two, then do a query with udpated_at earlier than time spent sitting | 16:11 |
| zzzeek | dansmith: oh here's an intersting approahc, uisng a generated column to index parts of the json document when they are inserted | 16:12 |
| zzzeek | https://www.compose.com/articles/mysql-for-json-generated-columns-and-indexing/ | 16:12 |
| dansmith | zzzeek: that doesn't help this case | 16:12 |
| stephenfin | and are moving the instance for the first time since before that upgrade off of Juno :) | 16:12 |
| zzzeek | dansmith: brcause....table cannot be changed, right? | 16:12 |
| dansmith | zzzeek: we're specifically looking at old data yeahg | 16:12 |
| *** brinzhang_ has quit IRC | 16:12 | |
| zzzeek | dansmith: old data can be copied into...new tables and columns? | 16:12 |
| *** brinzhang_ has joined #openstack-nova | 16:13 | |
| dansmith | zzzeek: no, we might as well convert it if we're going to read it | 16:13 |
| zzzeek | dansmith: yup. if you have a big old JSON blob and you want to pull things out of it and performance is an issue then you would need to put this data into some indexable format somewhere else | 16:13 |
| *** ociuhandu has quit IRC | 16:13 | |
| zzzeek | stephenfin: ^^^ | 16:14 |
| dansmith | artom: updated_at is on the instance_extra row, right? so it doesn't really mean we've updated any one or all columns necessarily | 16:14 |
| stephenfin | zzzeek: okay, thanks for the input :) | 16:15 |
| artom | dansmith, hrmm, yeah - I guess I was assuming "update on read" would trigger for any read of any column of that row | 16:16 |
| artom | Not just instance_extra.numa_topology reads | 16:16 |
| dansmith | instance_extra was really supposed to be individually queried and updated, although I dunno how much that really happens.. it was mostly for lazy-loadable blobby things originally | 16:16 |
| artom | Is this is the first time we're hitting this problem? | 16:17 |
| artom | Apparently so - at least using .contains() | 16:17 |
| *** rpittau is now known as rpittau|afk | 16:18 | |
| * artom -> lunch | 16:19 | |
| *** ociuhandu has joined #openstack-nova | 16:20 | |
| *** ociuhandu has quit IRC | 16:25 | |
| zigo | If you guys think we don't have enough time before the final release (which I can agree on), why not just delay this for 21.0.1 or something? | 16:29 |
| zigo | My biggest concern is not being able to actually "see" what's set by what we have in defaults. | 16:30 |
| zigo | Yes, it's documented, but it's not the same. | 16:30 |
| zigo | I'll try using policy.yaml though ... :P | 16:30 |
| dansmith | zigo: delay "this" meaning the policy stuff in general? | 16:30 |
| zigo | Yeah. | 16:31 |
| dansmith | zigo: it was a large number of patches.. reverting it now would be .. huge. | 16:31 |
| zigo | dansmith: I thought it'd be just a simple patch to the oslo-policy-sample-generator ... :P | 16:32 |
| zigo | Maybe I'm being naive. | 16:32 |
| dansmith | zigo: the change in question was on the nova side, not oslo.. we'd need new code on the oslo side to allow generating the policy file with the now-deprecated original defaults | 16:35 |
| *** evrardjp has quit IRC | 16:35 | |
| *** evrardjp has joined #openstack-nova | 16:35 | |
| zigo | Now I see what you guys were talking about for the yaml thing: it's by default generated with everything commented out ... | 16:37 |
| zigo | (I just tried...) | 16:37 |
| zigo | That's probably nicer indeed, and probably good enough for operators, and also maybe more easy to have a policy.d | 16:37 |
| zigo | Though the last time I tried, it wouldn't load ... :/ | 16:37 |
| * zigo tries again this time. | 16:37 | |
| *** ociuhandu has joined #openstack-nova | 16:38 | |
| *** dtantsur is now known as dtantsur|afk | 16:40 | |
| zigo | Looks like it works as one would expect... :) | 16:41 |
| jsuchome | dansmith: ok, so it seems it's up to me, so if you could reopen and reassign it ... thanks | 16:41 |
| dansmith | jsuchome: all you need to do is propose the changes yourself | 16:42 |
| jsuchome | so should I just cherry pick to my branch? | 16:43 |
| *** sapd1_x has quit IRC | 16:43 | |
| zigo | All this would be great if the Debian infrastructure wasn't completely down today ... :/ | 16:44 |
| dansmith | jsuchome: just grab the latest version of that patch into your tree, make changes, git review. | 16:44 |
| *** lyarwood has quit IRC | 16:45 | |
| *** vishalmanchanda has quit IRC | 16:45 | |
| *** maciejjozefczyk has quit IRC | 16:50 | |
| *** lpetrut has quit IRC | 16:51 | |
| *** derekh has quit IRC | 16:52 | |
| *** hoonetorg has joined #openstack-nova | 17:00 | |
| *** sapd1_x has joined #openstack-nova | 17:11 | |
| *** nightmare_unreal has quit IRC | 17:25 | |
| artom | dansmith, answers provided: https://review.opendev.org/#/c/672595/73 | 17:33 |
| artom | (Hopefully) | 17:33 |
| gmann | zigo: nice. +1 thanks for checking. | 17:33 |
| *** martinkennelly has quit IRC | 17:37 | |
| *** martinkennelly has joined #openstack-nova | 17:38 | |
| artom | \o/ | 17:43 |
| artom | I have a 1:1 in 15 minutes, otherwise it'd be beer time | 17:44 |
| *** ociuhandu has quit IRC | 17:44 | |
| *** ociuhandu has joined #openstack-nova | 17:46 | |
| *** dpawlik has quit IRC | 17:48 | |
| *** ociuhandu has quit IRC | 17:51 | |
| gmann | zigo: and you generated with same tool right ? | 17:53 |
| *** tesseract has quit IRC | 17:53 | |
| zigo | gmann: Yeah, just --format yaml. Now, I'll have to set policy.yaml as default in nova.conf | 17:54 |
| zigo | Though as I wrote earlier, Debian @UBC is down, so can't do anything right now ... :( | 17:54 |
| *** lee1 has joined #openstack-nova | 17:54 | |
| zigo | No Git to play with. | 17:55 |
| gmann | +1 | 17:55 |
| *** gryf has quit IRC | 17:56 | |
| zigo | Also, this clashes with puppet-openstack which uses .json. | 17:57 |
| zigo | It's going to be fun time to fix too ... | 17:57 |
| *** alistarle has joined #openstack-nova | 18:06 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Clarify the policy new defaults upgrade notes https://review.opendev.org/723645 | 18:08 |
| openstackgerrit | Merged openstack/nova master: Remove Babel requirement https://review.opendev.org/720725 | 18:11 |
| openstackgerrit | Merged openstack/nova master: Remove translation sections from setup.cfg https://review.opendev.org/723206 | 18:11 |
| *** alistarle has quit IRC | 18:15 | |
| *** sapd1_x has quit IRC | 18:17 | |
| *** ociuhandu has joined #openstack-nova | 18:23 | |
| *** takamatsu has quit IRC | 18:26 | |
| *** sapd1_x has joined #openstack-nova | 18:30 | |
| *** brinzhang has quit IRC | 18:34 | |
| *** brinzhang has joined #openstack-nova | 18:34 | |
| *** sapd1_x has quit IRC | 18:44 | |
| *** ralonsoh has quit IRC | 18:51 | |
| *** priteau has quit IRC | 18:53 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova stable/queens: DNM: Partial cherry-pick of report client changes https://review.opendev.org/723750 | 18:57 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova stable/queens: DNM: Add a placement audit command https://review.opendev.org/723751 | 18:57 |
| *** ociuhandu has quit IRC | 18:57 | |
| *** jangutter has quit IRC | 19:04 | |
| *** jangutter has joined #openstack-nova | 19:04 | |
| *** ociuhandu has joined #openstack-nova | 19:13 | |
| *** xek has quit IRC | 19:13 | |
| *** lee1 is now known as lyarwood | 19:24 | |
| *** ccamacho has quit IRC | 19:42 | |
| *** jsuchome has quit IRC | 19:47 | |
| *** JamesBenson has quit IRC | 20:02 | |
| *** JamesBenson has joined #openstack-nova | 20:07 | |
| *** threestrands has quit IRC | 20:09 | |
| *** rchurch has joined #openstack-nova | 20:16 | |
| *** xek has joined #openstack-nova | 20:24 | |
| melwitt | gmann: I thought you wanted to wait for these first? https://review.opendev.org/#/q/topic:qa-ussuri-release+status:open | 20:31 |
| gmann | melwitt: devstack and grenade setup mainly which are merged. devstack-gate changes is for legacy jobs which this patch replacing | 20:32 |
| melwitt | oh ok | 20:32 |
| *** dustinc has joined #openstack-nova | 20:34 | |
| *** raildo_ has joined #openstack-nova | 20:37 | |
| sean-k-mooney | am https://bugs.launchpad.net/nova/+bug/1875418 have we ever support a policy.yaml? | 20:39 |
| openstack | Launchpad bug 1875418 in OpenStack Compute (nova) "Generated policy.json in Ussuri is broken by default" [High,In progress] - Assigned to Ghanshyam Mann (ghanshyammann) | 20:39 |
| *** raildo has quit IRC | 20:39 | |
| sean-k-mooney | i tought we only supported policy files in json format | 20:39 |
| sean-k-mooney | i ask because teh " [puppet][packaging] Switching to policy.yaml (over policy.json)" thread seams to imply that is a thing | 20:41 |
| sean-k-mooney | but i did not think we supported a policy.yaml file instead of policy.json | 20:41 |
| *** martinkennelly has quit IRC | 20:42 | |
| sean-k-mooney | huh i guess its a thing https://docs.openstack.org/oslo.policy/latest/admin/policy-yaml-file.html | 20:42 |
| bnemec | We've been recommending yaml since policy in code happened. | 20:43 |
| bnemec | Apparently we need to work on our PR though. :-) | 20:44 |
| gmann | yeah, and not much attention on that and this issue came up now | 20:45 |
| sean-k-mooney | ya i mean yaml is probably nice to work with in terms of generating and reading the file and form a python point of view it makes little difference as we will jsut load it into a python dict in etierh case then process it | 20:45 |
| sean-k-mooney | but i never heard that we added supprot for parseing yaml files instead | 20:45 |
| sean-k-mooney | bnemec: it might be something that woudl be worth make a comuntiy goal | 20:46 |
| sean-k-mooney | to get everyone to move to policy.yaml espeically the deployment tools | 20:46 |
| openstackgerrit | Merged openstack/nova master: Functional tests for NUMA live migration https://review.opendev.org/672595 | 20:46 |
| gmann | it is supported in oslo side right | 20:46 |
| bnemec | sean-k-mooney: It was: https://governance.openstack.org/tc/goals/selected/queens/policy-in-code.html | 20:46 |
| sean-k-mooney | although i guest they should not be setting one by default | 20:46 |
| bnemec | Oh, deployment tools. | 20:46 |
| sean-k-mooney | bnemec: well plocy in code was | 20:47 |
| *** nweinber has quit IRC | 20:47 | |
| sean-k-mooney | bnemec: i was more thinking of makeing sure our docs advise to use policy.yaml files | 20:47 |
| sean-k-mooney | and any deployment tools that currently supprot customising policy.json use policy.yaml | 20:48 |
| bnemec | They do: https://docs.openstack.org/oslo.policy/latest/admin/policy-json-file.html ;-) | 20:48 |
| bnemec | But yeah, clearly we need to push it harder. | 20:48 |
| sean-k-mooney | bnemec: do the nova docs? | 20:48 |
| gmann | bnemec: and that happen when things get broken otherwise its less chance people read doc or be in sync on new things | 20:49 |
| bnemec | That I don't know. The only in-project policy doc I'm aware of is the policy sphinx plugin. | 20:49 |
| sean-k-mooney | https://docs.openstack.org/nova/rocky/configuration/policy.html i gues it just list the polices | 20:49 |
| sean-k-mooney | and the sample kind of looks like yaml https://docs.openstack.org/nova/rocky/configuration/sample-policy.html | 20:49 |
| gmann | sean-k-mooney: the file form mentioned is yaml one -https://docs.openstack.org/nova/latest/configuration/sample-policy.html | 20:50 |
| sean-k-mooney | ok it is a yaml file | 20:50 |
| bnemec | Yep, that's the yaml sample policy output. | 20:50 |
| sean-k-mooney | ya you can download it https://docs.openstack.org/nova/rocky/_downloads/nova.policy.yaml.sample | 20:50 |
| sean-k-mooney | ok im just used to people saying policy.json the whole time | 20:50 |
| sean-k-mooney | we still refer to it as policy.json here https://docs.openstack.org/nova/rocky/configuration/ | 20:52 |
| *** jmlowe has joined #openstack-nova | 20:54 | |
| bnemec | Yeah, and unfortunately policy.json is still the default name in oslo.policy. We've had discussions about making it look for both json and yaml, but it has security implications if we guess wrong. | 20:54 |
| *** xek has quit IRC | 20:55 | |
| sean-k-mooney | so does it only look for one of them | 20:55 |
| gmann | first it look for json and then yaml, if i am not wrong | 20:56 |
| sean-k-mooney | ok so if you have both it will use the json file | 20:56 |
| sean-k-mooney | or will it load the default form code then load the json then the yaml | 20:56 |
| sean-k-mooney | both would be vaild but im not really sure which would be more suprising | 20:57 |
| sean-k-mooney | i kind of would expect the second option with an error if there was a conflit between the json and yaml file | 20:57 |
| sean-k-mooney | our just error if both were there | 20:57 |
| sean-k-mooney | that will be less surprising i guess | 20:58 |
| sean-k-mooney | having multiple souces of policy is a beacon for bugs | 20:58 |
| bnemec | It only looks for policy.json. It will parse files in either format in the order gmann said. | 20:59 |
| bnemec | So if you say policy_file=policy.something it will first attempt to parse it as json, and if that fails try again as yaml. | 20:59 |
| sean-k-mooney | ok so to use https://docs.openstack.org/oslo.policy/latest/admin/policy-yaml-file.html you need to chagne the policy_file in the main config file to point to policy.yaml | 21:00 |
| gmann | yeah and default are loaded only if the registered rule is not in file | 21:00 |
| bnemec | Right | 21:00 |
| sean-k-mooney | ok that is not obvious form the docs but i can see why you might have chosen that design | 21:01 |
| sean-k-mooney | well good to know | 21:01 |
| gmann | bnemec: i think we can change the default value to policy.yaml and start error if not present and no fallback to policy.json ? | 21:01 |
| gmann | i mean in Victoria | 21:01 |
| bnemec | Worth noting that it can be overridden by individual projects. I believe cinder has done that. | 21:01 |
| bnemec | I don't know how much pain it caused their users when they switched, or if they were lucky enough to start out with it that way. | 21:01 |
| sean-k-mooney | bnemec: right oslo.config support progomatic overrides | 21:01 |
| sean-k-mooney | so we could do set_default in nova to point it to a different default for nova | 21:02 |
| bnemec | gmann: But we can't error on a missing policy file because that's perfectly valid. | 21:02 |
| gmann | bnemec: ah, that's right. | 21:03 |
| sean-k-mooney | like this right https://docs.openstack.org/oslo.config/4.0.0/faq.html#why-are-configuration-options-not-part-of-a-library-s-api | 21:03 |
| bnemec | sean-k-mooney: There's an api provided for it: https://github.com/openstack/oslo.policy/blob/master/oslo_policy/opts.py#L121 | 21:03 |
| bnemec | Generally speaking, consumers shouldn't mess with library opts directly. | 21:03 |
| sean-k-mooney | yep i just linked to the docs for it | 21:03 |
| bnemec | Yes, exactly. | 21:04 |
| bnemec | async communication. :-) | 21:04 |
| sean-k-mooney | and ya i agree because it make debuging it a pain for the lib maintainer | 21:04 |
| bnemec | And it will break if we ever rename the opt, even with deprecation. | 21:04 |
| bnemec | Because the in-code references don't have deprecation logic. | 21:04 |
| sean-k-mooney | you could proably make that work bust defineing an atribute that was initalsed to the other one | 21:05 |
| sean-k-mooney | but also good to know | 21:05 |
| sean-k-mooney | im not sure we actully ever do this in nova | 21:05 |
| sean-k-mooney | or in any other code i have looked at | 21:06 |
| sean-k-mooney | hum actully we are for things in our own config.... im going to pretend i didnt see that and move on | 21:09 |
| *** JamesBenson has quit IRC | 21:24 | |
| *** raildo_ has quit IRC | 21:26 | |
| *** bbowen has quit IRC | 21:29 | |
| *** bbowen has joined #openstack-nova | 21:34 | |
| *** ociuhandu has quit IRC | 21:34 | |
| *** ociuhandu has joined #openstack-nova | 21:34 | |
| *** JamesBenson has joined #openstack-nova | 21:38 | |
| *** slaweq has quit IRC | 21:41 | |
| *** ociuhandu has quit IRC | 21:41 | |
| *** mriedem has left #openstack-nova | 21:56 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add nova-status upgrade check and reno for policy new defaults https://review.opendev.org/723645 | 22:08 |
| *** JamesBenson has quit IRC | 22:15 | |
| openstackgerrit | Merged openstack/nova master: zuul: Switch to the Zuulv3 grenade job https://review.opendev.org/704364 | 22:20 |
| openstackgerrit | sean mooney proposed openstack/nova master: silence amqp heartbeat warning https://review.opendev.org/724188 | 22:28 |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova stable/ussuri: zuul: Switch to the Zuulv3 grenade job https://review.opendev.org/724189 | 22:29 |
| gmann | lyarwood: melwitt backported to ussuri ^^ to have single grenade job running in ussuri as grenade zuulv3 job merged in ussuri | 22:30 |
| sean-k-mooney | melwitt: im not sure if https://review.opendev.org/#/c/724188/1/nova/config.py will work but assuming it does it would be good to get your input on if we should drop that log message as the patch currently does or just reduce the log level to debug | 22:31 |
| *** rcernin has joined #openstack-nova | 22:33 | |
| gmann | dansmith: bauzas gibi added the upgrade check also for policy stuff - https://review.opendev.org/#/c/723645/ | 22:35 |
| *** ociuhandu has joined #openstack-nova | 23:04 | |
| *** JamesBenson has joined #openstack-nova | 23:06 | |
| *** JamesBenson has quit IRC | 23:11 | |
| *** ociuhandu has quit IRC | 23:17 | |
| *** avolkov has quit IRC | 23:27 | |
| *** tosky has quit IRC | 23:39 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!