abaindur | so for pre-copy, you are suggesting to try memcache for keystone tokens? | 00:00 |
---|---|---|
sean-k-mooney | ya but im not saying dont try enableing post-copy | 00:00 |
sean-k-mooney | im just sugging that memcache might impove the performace enough to not need it | 00:00 |
sean-k-mooney | i woudl test both | 00:00 |
sean-k-mooney | abaindur: how did you deploy openstack by the way | 00:01 |
abaindur | thanks, yea we'll probably try both options then to see which leads to least downtime | 00:01 |
sean-k-mooney | for both options is basically just a compute agent restart and cofig update | 00:01 |
sean-k-mooney | abaindur: at least in the case of https://bugs.launchpad.net/nova/+bug/1836642 we were seeing edgcaces wher ethe metadta api respoce could take upwards of 12 - 16 seconds and that went away when we enabeld the cache | 00:03 |
openstack | Launchpad bug 1836642 in neutron "Metadata responses are very slow sometimes" [High,Incomplete] - Assigned to Slawek Kaplonski (slaweq) | 00:03 |
sean-k-mooney | abaindur: although to be clear we enabel both the cache section and the memcahced_server in the keystone_authoken section | 00:05 |
sean-k-mooney | so all of the highlight section is how we adressed the isseu https://zuul.opendev.org/t/openstack/build/dcde79801a624c25b195a46ead7af562/log/controller/logs/etc/nova/nova-cpu_conf.txt#62-68 | 00:05 |
*** bbowen has quit IRC | 00:08 | |
sean-k-mooney | abaindur: https://docs.openstack.org/keystonemiddleware/latest/middlewarearchitecture.html#improving-response-time might be of interst | 00:09 |
*** gyee has quit IRC | 00:11 | |
sean-k-mooney | i think that might only be relevent to the apis service however and might not apply to when we use the keystone auth client to talk to otehr apis but im not sure baout that. anyway its late here so im going to go o/ | 00:11 |
abaindur | sean-k-mooney: oh wait, we have memcached_servers set in keystone_authtoken | 00:11 |
abaindur | but we dont have a [cache] section with the other configs | 00:12 |
sean-k-mooney | ah the cache section is novas own internal cache config | 00:12 |
*** ttsiouts has quit IRC | 00:12 | |
sean-k-mooney | so in the metadata api case it is what allows cache copies fo the metadata to be shareed between processes | 00:13 |
sean-k-mooney | i think your best path forward in that cae is likely your original one of enabling post copy. you might also want o consider auto convergence | 00:14 |
*** factor has joined #openstack-nova | 00:20 | |
*** tetsuro has joined #openstack-nova | 00:23 | |
abaindur | ok thanks for the help! will update here later if i find out more | 00:27 |
openstackgerrit | Merged openstack/nova master: Allocate mdevs when resizing or reverting resize https://review.opendev.org/712741 | 00:42 |
*** ociuhandu has joined #openstack-nova | 00:43 | |
*** ociuhandu has quit IRC | 00:49 | |
*** mkrai_ has quit IRC | 00:50 | |
*** bbowen has joined #openstack-nova | 00:50 | |
*** mlavalle has quit IRC | 01:03 | |
*** Liang__ has joined #openstack-nova | 01:03 | |
*** ttsiouts has joined #openstack-nova | 01:20 | |
*** zhanglong has joined #openstack-nova | 01:27 | |
openstackgerrit | Merged openstack/nova stable/train: Reject boot request for unsupported images https://review.opendev.org/708577 | 01:50 |
*** ttsiouts has quit IRC | 01:55 | |
*** mkrai has joined #openstack-nova | 02:11 | |
*** francoisp has quit IRC | 02:15 | |
*** ttsiouts has joined #openstack-nova | 02:31 | |
*** ttsiouts has quit IRC | 02:36 | |
*** factor has quit IRC | 02:49 | |
*** hongbin has joined #openstack-nova | 03:00 | |
*** ttsiouts has joined #openstack-nova | 03:04 | |
*** factor has joined #openstack-nova | 03:15 | |
*** psachin has joined #openstack-nova | 03:25 | |
*** ttsiouts has quit IRC | 03:38 | |
*** tetsuro has quit IRC | 04:00 | |
*** ociuhandu has joined #openstack-nova | 04:19 | |
*** ociuhandu has quit IRC | 04:24 | |
*** evrardjp has quit IRC | 04:35 | |
*** evrardjp has joined #openstack-nova | 04:35 | |
*** hongbin has quit IRC | 04:35 | |
*** zhanglong has quit IRC | 04:48 | |
*** ttsiouts has joined #openstack-nova | 04:49 | |
*** ratailor has joined #openstack-nova | 04:50 | |
*** zhanglong has joined #openstack-nova | 04:52 | |
*** tetsuro has joined #openstack-nova | 04:59 | |
*** vishalmanchanda has joined #openstack-nova | 05:09 | |
*** udesale has joined #openstack-nova | 05:10 | |
*** ratailor_ has joined #openstack-nova | 05:19 | |
*** ratailor has quit IRC | 05:21 | |
*** udesale_ has joined #openstack-nova | 05:21 | |
*** ttsiouts has quit IRC | 05:23 | |
*** udesale has quit IRC | 05:24 | |
*** avolkov has joined #openstack-nova | 05:55 | |
*** ttsiouts has joined #openstack-nova | 05:58 | |
*** ratailor_ has quit IRC | 06:02 | |
*** ratailor has joined #openstack-nova | 06:02 | |
*** ttsiouts has quit IRC | 06:02 | |
*** ratailor has quit IRC | 06:04 | |
*** ratailor has joined #openstack-nova | 06:05 | |
*** dpawlik has joined #openstack-nova | 06:08 | |
*** udesale_ has quit IRC | 06:14 | |
*** ttsiouts has joined #openstack-nova | 06:28 | |
*** dklyle has quit IRC | 06:32 | |
*** breizhkoala has joined #openstack-nova | 06:36 | |
*** udesale has joined #openstack-nova | 06:37 | |
gibi | stephenfin, gmann, artom: I'm +2 on the policy code changes. | 06:38 |
*** udesale has quit IRC | 06:43 | |
*** ociuhandu has joined #openstack-nova | 06:48 | |
gibi | I'm tempted to squash the doc follow up into the original doc patch for more readable diff but meh I wait for others to show up | 06:48 |
*** tetsuro has quit IRC | 06:51 | |
bauzas | gibi: gmann: fwiw, I'll leave the prelude section change pretty small for the policy changes, but you are free to amend it for clarifying what we finally got for policy changes | 06:55 |
bauzas | hopefully, will be posted later this morning | 06:55 |
gibi | bauzas: thanks | 06:55 |
bauzas | (and good morning Nova, sorry for my manners) | 06:55 |
bauzas | gibi: tbh, I'm mostly paraphrasing the cycle highlights with a couple of additions (the BFV instance rescue and the vGPU multiple types support) | 06:57 |
bauzas | gibi: worth mentioning others ? | 06:57 |
gibi | I think that is plenty | 06:58 |
*** ociuhandu has quit IRC | 06:58 | |
*** ociuhandu has joined #openstack-nova | 06:58 | |
bauzas | gibi: well, I'm not afraid, that will still be an order of magnitude less than the rest of the relnotes | 06:58 |
bauzas | https://blueprints.launchpad.net/nova/+spec/remove-xvpvncproxy and https://blueprints.launchpad.net/nova/+spec/remove-nova-network-ussuri should also be commented I tihnk | 06:59 |
bauzas | gibi: fwiw, this is how it looks at the moment without a prelude section https://docs.openstack.org/releasenotes/nova/unreleased.html | 06:59 |
bauzas | good luck with this. | 06:59 |
bauzas | anyway, late breakfast, kids are up | 07:00 |
gibi | hm, based on that I even highlight the fact that we dropped supporting python 2.7 | 07:01 |
*** ttsiouts has quit IRC | 07:03 | |
*** rpittau|afk is now known as rpittau | 07:03 | |
*** nightmare_unreal has joined #openstack-nova | 07:03 | |
*** tetsuro has joined #openstack-nova | 07:04 | |
*** maciejjozefczyk has joined #openstack-nova | 07:04 | |
*** ttsiouts has joined #openstack-nova | 07:07 | |
*** slaweq_ has joined #openstack-nova | 07:11 | |
*** tesseract has joined #openstack-nova | 07:13 | |
gibi | sean-k-mooney: I guess you have SRIOV env at hand. Could you check if the bug https://bugs.launchpad.net/nova/+bug/1851545 is valid? | 07:25 |
openstack | Launchpad bug 1851545 in OpenStack Compute (nova) "Port update exception on nova unshelve for instance with PCI devices (part 2)" [Undecided,New] | 07:25 |
gibi | for me it would take a day or so to set one up | 07:26 |
*** ociuhandu has quit IRC | 07:26 | |
bauzas | gibi: good point, okay, let's wrap this up and see what people think | 07:37 |
gibi | bauzas: yep, lets do the review on gerrit :) | 07:38 |
*** ttsiouts has quit IRC | 07:39 | |
*** ttsiouts has joined #openstack-nova | 07:40 | |
*** tosky has joined #openstack-nova | 07:44 | |
*** ccamacho has joined #openstack-nova | 07:47 | |
*** xek has joined #openstack-nova | 07:52 | |
*** threestrands has quit IRC | 07:55 | |
*** ralonsoh has joined #openstack-nova | 07:57 | |
elod | zigo melwitt : about https://bugs.launchpad.net/nova/+bug/1788014 , I realized that it needs maybe some refactoring... then I forgot about looking at it. :/ Will do today. I'll need some thorough review for that i guess. | 07:59 |
openstack | Launchpad bug 1788014 in OpenStack Compute (nova) rocky "when live migration fails due to a internal error rollback is not handeled correctly." [Medium,In progress] - Assigned to Elod Illes (elod-illes) | 07:59 |
zigo | elod: Thanks ! | 08:00 |
*** jangutter has joined #openstack-nova | 08:00 | |
*** ociuhandu has joined #openstack-nova | 08:01 | |
*** lennyb has joined #openstack-nova | 08:03 | |
*** KeithMnemonic has quit IRC | 08:05 | |
bauzas | gosh, the tox target for relnotes takes a while recalculating all the relnotes... | 08:06 |
*** tetsuro_ has joined #openstack-nova | 08:08 | |
*** martinkennelly has joined #openstack-nova | 08:09 | |
*** tetsuro has quit IRC | 08:11 | |
*** ttsiouts has quit IRC | 08:14 | |
*** links has joined #openstack-nova | 08:16 | |
*** udesale has joined #openstack-nova | 08:16 | |
*** jangutter has quit IRC | 08:19 | |
*** jangutter has joined #openstack-nova | 08:19 | |
*** ttsiouts has joined #openstack-nova | 08:21 | |
*** udesale_ has joined #openstack-nova | 08:22 | |
*** udesale has quit IRC | 08:22 | |
*** udesale has joined #openstack-nova | 08:22 | |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Ussuri 21.0.0 prelude section https://review.opendev.org/721548 | 08:38 |
bauzas | wow, it took a while but I'm done | 08:38 |
openstackgerrit | Merged openstack/nova master: Add new default roles in quota class policies https://review.opendev.org/719100 | 08:39 |
bauzas | gibi: stephenfin: dansmith: melwitt: alex_xu: gmann: others, the prelude section patch is up for reviews https://review.opendev.org/721548 | 08:39 |
bauzas | lyarwood: too ^ | 08:40 |
* lyarwood clicks | 08:41 | |
*** belmoreira has joined #openstack-nova | 08:41 | |
bauzas | gibi: can I hold the bugs lock ? | 08:41 |
gibi | bauzas: I'm looking at https://bugs.launchpad.net/nova/+bug/1831315 right now but after that I have to jump to your prelude patch so you can have the lock | 08:42 |
openstack | Launchpad bug 1831315 in OpenStack Compute (nova) "nova-manage cell_v2 discover_hosts fails for IPv6 - ValueError: invalid literal for int() with base 10 - db connection URI gets its brackets eaten" [Undecided,New] - Assigned to Harald Jensås (harald-jensas) | 08:42 |
bauzas | gibi: you don't order bugs by Age, right? | 08:43 |
*** priteau has joined #openstack-nova | 08:45 | |
gibi | bauzas: I skip those that I cannot do anyithing with | 08:47 |
bauzas | ok, I'll try to look at them and ask for more if needed | 08:47 |
* bauzas does this every week for downstream bugs | 08:47 | |
gibi | thanks | 08:47 |
bauzas | gibi: for example, https://bugs.launchpad.net/nova/+bug/1860312 looks unrelated to Nova | 09:00 |
openstack | Launchpad bug 1860312 in OpenStack Compute (nova) "compute service failed to delete" [Undecided,New] | 09:00 |
gibi | bauzas: is it so? | 09:00 |
*** dtantsur|afk is now known as dtantsur | 09:00 | |
gibi | bauzas: for me it seems like we somehow manage to allow a db corruption | 09:01 |
gibi | "When I try to delete the old compute service(status down), it failed." | 09:01 |
gibi | sure I don't know how the deployment ended up in "ComputeHostNotFound: Compute host mgt-slave03 could not be found " state | 09:02 |
gibi | but I feel that nova should support deleting old unused compute services | 09:02 |
gibi | my problem is that I have no way to try to recreate the bug without building a kubenetes + ironic env | 09:05 |
*** zhanglong has quit IRC | 09:07 | |
*** vishalmanchanda has quit IRC | 09:09 | |
kashyap | lyarwood: So ... one test failure seems legit here: https://review.opendev.org/#/c/708701/4 | 09:11 |
kashyap | lyarwood: You know the reason once you see the failing test: test_stable_device_rescue_cdrom_ide() + Q35 :) | 09:12 |
lyarwood | I've not had a look yet no | 09:16 |
lyarwood | I assume with that test that q35 doesn't have an IDE bus? | 09:16 |
lyarwood | iirc that's the case right | 09:16 |
lyarwood | I can add a configurable in tempest for that | 09:17 |
*** happyhemant has joined #openstack-nova | 09:17 | |
lyarwood | the config drive issue is still there as well btw | 09:17 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Add stable device rescue docs https://review.opendev.org/700837 | 09:18 |
kashyap | lyarwood: Yeah, the issue is indeed 'q35' has no IDE bus | 09:18 |
kashyap | lyarwood: Sorry, which config drive issue? | 09:18 |
lyarwood | kashyap: see test_server_basic_ops etc, blkid -L config-2 -o device', exit status: 2 | 09:19 |
lyarwood | ah wait | 09:19 |
lyarwood | that's because that's a cdrom | 09:19 |
lyarwood | over SCSI? | 09:20 |
kashyap | lyarwood: You mean over IDE, instead of SATA (or 'virtio')? | 09:20 |
lyarwood | kashyap: we don't have IDE, I can't recall if we default to SCSI in that case or SATA | 09:21 |
kashyap | I'm not sure yet ... I'll see what my non-configurable one-liner yields: https://review.opendev.org/#/c/721555/ | 09:21 |
lyarwood | kashyap: nack, IDE + cdrom is still a valid test case prior to q35 IMHO | 09:21 |
kashyap | lyarwood: Don't you recall the patch you made? :) We default to SATA with Q35: https://review.opendev.org/#/c/663011/ | 09:21 |
kashyap | lyarwood: Oh, sure; it still _is_ a valid test for PC | 09:22 |
lyarwood | ah right | 09:22 |
kashyap | (I'm just seeing what this not-to-be-merged yields. And a config tunable is perhaps the way) | 09:22 |
lyarwood | so are you pulling that in anywhere to test against 135? | 09:23 |
kashyap | lyarwood: Sorry, 135? | 09:24 |
lyarwood | q35 sorry | 09:24 |
bauzas | gibi: the problem is that ironic still sees some node | 09:24 |
bauzas | gibi: while nova doesn't | 09:25 |
kashyap | lyarwood: Err, I need to make that dependent against your 'q35' job, isn' it? | 09:25 |
kashyap | Let me go do that | 09:25 |
bauzas | gibi: at least we can ask the reporter to tell us whether nova still sees it => Incomplete | 09:26 |
lyarwood | kashyap: make my q35 change depend on your tempest change | 09:26 |
kashyap | lyarwood: Yeah. Not sure what I'm thinking | 09:27 |
kashyap | Doing ... | 09:27 |
gibi | bauzas: OK, I was not able to formulate any valid question to the reporter. I glad you could | 09:27 |
bauzas | gibi: I'm just doing it | 09:27 |
* lyarwood rebuilds his bionic q35 env | 09:27 | |
kashyap | lyarwood: Can I remove the Depends-On in your 'q35' CI change against the 'blkid -L' change? | 09:30 |
kashyap | lyarwood: I'll remove it for now, so as to test one thing at a time -- okay? | 09:31 |
lyarwood | kashyap: well it's seperate | 09:32 |
lyarwood | kashyap: ah wait no it isn't | 09:32 |
lyarwood | kashyap: it's in openstack/tempest so yeah feel free to | 09:32 |
lyarwood | kashyap: btw I'm planing on writing a spec for the q35 default change next week | 09:33 |
kashyap | Okido; I'll add a note in the commit with a link to tit. | 09:33 |
kashyap | lyarwood: FWIW, I already have it in place somewhere | 09:33 |
kashyap | One is this BP: https://blueprints.launchpad.net/nova/+spec/gracefully-handle-qemu-machine-types | 09:34 |
kashyap | And a spec should also be around ... | 09:34 |
kashyap | lyarwood: Here's the spec started 8 months ago: https://review.opendev.org/#/c/631154/ | 09:35 |
*** sapd1 has joined #openstack-nova | 09:35 | |
lyarwood | kk I'll repropose that for V | 09:35 |
kashyap | lyarwood: I was meaning to send a note to the upstream list as a heads-up. | 09:37 |
kashyap | (I have a draft e-mail, will send it sometime this week. Giving a heads-up to prepare ... as this is a virtual 'motherboard' change) | 09:37 |
openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/708701 | 09:39 |
lyarwood | kashyap: lets get the spec reproposed, updated and on the agenda for PTG first | 09:41 |
* kashyap nods | 09:41 | |
lyarwood | kashyap: I'm not going to be around for PTG btw would you be okay leading any sessions/discussions on the ML? | 09:42 |
bauzas | gibi: FYI I think I found the problem but before triaging it to Confirmed, I need to ask the reporter to tell us whether I'm right https://bugs.launchpad.net/nova/+bug/1860312/comments/1 | 09:42 |
openstack | Launchpad bug 1860312 in OpenStack Compute (nova) "compute service failed to delete" [Undecided,Incomplete] | 09:42 |
kashyap | lyarwood: Yeah, I was intending to that. As I've been delaying this work for various reasons | 09:42 |
gibi | bauzas: cool | 09:43 |
gibi | bauzas: also I left some comments on the prelude. good job! | 09:44 |
bauzas | gibi: ack | 09:44 |
gibi | stephenfin: could you re-review stable device rescue doc patch? https://review.opendev.org/#/c/700837 | 09:45 |
bauzas | I wish I could easily find bug duplicates with Launchpad, because I'm pretty sure that bug 1860312 is not the only one for the same issue | 09:45 |
openstack | bug 1860312 in OpenStack Compute (nova) "compute service failed to delete" [Undecided,Incomplete] https://launchpad.net/bugs/1860312 | 09:45 |
stephenfin | sure, just finished gmann doc change | 09:45 |
stephenfin | *gmann's | 09:45 |
gibi | stephenfin: that would have been my other request :) | 09:45 |
* gibi goes and read the policy doc pathces | 09:45 | |
bauzas | gibi: bingo : https://bugs.launchpad.net/nova/+bug/1853009 | 09:46 |
openstack | Launchpad bug 1853009 in OpenStack Compute (nova) "Ironic node rebalance race can lead to missing compute nodes in DB" [Undecided,In progress] - Assigned to Mark Goddard (mgoddard) | 09:46 |
mgoddard | bauzas, gibi: would love some reviews on those patches :) | 09:47 |
bauzas | mgoddard: absolutely, will do | 09:48 |
mgoddard | I think some were failing unit tests, I can tidy up if someone plans to look | 09:48 |
bauzas | mgoddard: how many times ironic can rebalance ? | 09:48 |
*** sapd1_x has joined #openstack-nova | 09:48 | |
bauzas | because I think that can be a problem for a shit ton of deployers using ironic | 09:48 |
mgoddard | bauzas: advance warning: prepare a cup of coffee, or three. Nasty race conditions ahead | 09:48 |
mgoddard | bauzas: unlimited rebalancing | 09:49 |
stephenfin | lyarwood, gibi: done https://review.opendev.org/#/c/700837/ | 09:49 |
bauzas | in particular when it goes on top on something abstract API like k8s... | 09:49 |
mgoddard | bauzas: the hash ring just consists of the active nova computes | 09:49 |
bauzas | mgoddard: just tbc, ironic does rebalance if the things below it are shuffled ? | 09:49 |
gibi | bauzas: ack about the duplicates | 09:50 |
gibi | stephenfin: thanks | 09:50 |
bauzas | in our case, because pods changed or whatever, right? | 09:50 |
lyarwood | stephenfin: ah thanks, ./me respins | 09:50 |
mgoddard | bauzas: there are two hash rings - one in nova, one in ironic conductor. You only need to care about the one in nova | 09:50 |
bauzas | mgoddard: you do confirm that https://bugs.launchpad.net/nova/+bug/1860312 sounds dup of https://bugs.launchpad.net/nova/+bug/1853009 ? | 09:50 |
openstack | Launchpad bug 1860312 in OpenStack Compute (nova) "compute service failed to delete" [Undecided,Incomplete] | 09:50 |
openstack | Launchpad bug 1853009 in OpenStack Compute (nova) "Ironic node rebalance race can lead to missing compute nodes in DB" [Undecided,In progress] - Assigned to Mark Goddard (mgoddard) | 09:50 |
mgoddard | bauzas: if it's sitting on top of a wobbly k8s jelly, with containers coming and going, it's quite likely :) will look in more detail | 09:51 |
bauzas | mgoddard: I'd dare say there are two hard problems in CS | 09:51 |
bauzas | hash rings come with the first hard problme | 09:52 |
bauzas | mgoddard: that's exactly what the reporter says " | 09:52 |
bauzas | I deployed openstack with openstack-helm on kubernetes.When one of the nova-compute service(driver=ironic replica of the deployment is 1) breakdown.It may be scheduled to another node by kubernetes.When I try to delete the old compute service(status down), it failed." | 09:52 |
mgoddard | bauzas: although in this case I think it's a different problem. They're trying to delete the compute service, not the compute node | 09:53 |
mgoddard | bauzas: I'd guess it's because they're using the service ID rather than the UUID | 09:53 |
mgoddard | ID no longer seems to work | 09:54 |
mgoddard | need to specify an API version to get the UUID in the list | 09:54 |
bauzas | mgoddard: it's a side effect | 09:55 |
bauzas | mgoddard: since the virt driver stopped reporting the node, then the RT deleted the compute node record | 09:55 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Add stable device rescue docs https://review.opendev.org/700837 | 09:55 |
lyarwood | stephenfin: ^ thanks btw | 09:56 |
mgoddard | bauzas: but the service should still exist, even without a node | 09:57 |
bauzas | mgoddard: we're making different assumptions in different pieces of code | 09:57 |
mgoddard | bauzas: I haven't seen that happen before | 09:57 |
bauzas | mgoddard: in the RT, we assume that it's reasonable to not have a CN record when the virt driver tells you no about it | 09:58 |
bauzas | mgoddard: but on the other hand, we assume that compute nodes exist if you delete a compute service | 09:58 |
mgoddard | bauzas: if that is true, should be easy to reproduce? | 10:00 |
mgoddard | and probably easy to test my hypothesis also | 10:00 |
*** Liang__ has quit IRC | 10:03 | |
bauzas | mgoddard: I've asked the reporter to come back with details | 10:03 |
openstackgerrit | Merged openstack/nova master: Use oslo policy flag to disable default change warning instead of all https://review.opendev.org/717884 | 10:04 |
bauzas | I could have some OSP crazy env that would help me to verify this hypothesis but to be frank I don't have the energy for it :) | 10:04 |
bauzas | mgoddard: like I said in my last comment of bug 1860312, I'd rather spend my energy on reviewing your change | 10:05 |
openstack | bug 1860312 in OpenStack Compute (nova) "compute service failed to delete" [Undecided,Incomplete] https://launchpad.net/bugs/1860312 | 10:05 |
bauzas | mgoddard: actually, the bug is pretty nasty and has a lot of other related bugs, do you think you'd be able to give it a good shape soon ? | 10:08 |
bauzas | to give *the change a good shape soon ? | 10:08 |
bauzas | gibi: FYI, ^ | 10:10 |
bauzas | we're too close to RC1 to consider those bugs as rc-potential but they're definitely worth backportable down the road to all our stable branches | 10:11 |
bauzas | actually, mriedem thought the same https://bugs.launchpad.net/nova/+bug/1841481 | 10:12 |
openstack | Launchpad bug 1841481 in OpenStack Compute (nova) "Race during ironic re-balance corrupts local RT ProviderTree and compute_nodes cache" [Medium,In progress] - Assigned to Matt Riedemann (mriedem) | 10:12 |
gibi | bauzas: ack, I agree that we should not try to blow up RC1 with this fix. | 10:12 |
bauzas | mgoddard: sorry for interupting you again, but we have around 4 open bugs for probably the same issue(s) or related | 10:12 |
bauzas | mgoddard: could we consider https://bugs.launchpad.net/nova/+bug/1853009 as the primary one ? | 10:12 |
openstack | Launchpad bug 1853009 in OpenStack Compute (nova) "Ironic node rebalance race can lead to missing compute nodes in DB" [High,In progress] - Assigned to Mark Goddard (mgoddard) | 10:13 |
* bauzas will mark all the stable branches as potential targets for the fix | 10:13 | |
mgoddard | bauzas: I'll get the patches into shape today | 10:14 |
bauzas | mgoddard: as I said, we're too short in time to deliver them for RC1, but I think this should be addressed as quicky as possible after the victoria branch | 10:15 |
mgoddard | bauzas: ack | 10:15 |
*** tetsuro_ has quit IRC | 10:17 | |
bauzas | mgoddard: fwiw, starting to review https://review.opendev.org/#/c/695012/4/nova/tests/functional/regressions/test_bug_1853009.py | 10:22 |
mgoddard | bauzas: cool. I don't expect the code to change significantly, just need to adapt to upstream changes since proposed | 10:23 |
*** ttsiouts has quit IRC | 10:26 | |
*** sapd1_y has joined #openstack-nova | 10:26 | |
*** ttsiouts has joined #openstack-nova | 10:28 | |
*** sapd1_x has quit IRC | 10:30 | |
*** rpittau is now known as rpittau|bbl | 10:31 | |
*** ttsiouts has quit IRC | 10:33 | |
*** ttsiouts has joined #openstack-nova | 10:37 | |
gibi | bauzas: what do you think about https://bugs.launchpad.net/nova/+bug/1872385 it is currently marked as rc potential but it the fix is not ready. I can take the fix over this afternoon and make it complete or we can drop the rc potential tag as you also noted that this is not a clear regression for Ussuri as we have child RPs for couple of releases now | 10:39 |
openstack | Launchpad bug 1872385 in OpenStack Compute (nova) "delete sub resource provider in tree" [High,In progress] - Assigned to Wenping Song (wenping1) | 10:39 |
* gibi switch to lunch operations | 10:40 | |
*** slaweq_ is now known as slaweq | 10:43 | |
sean-k-mooney | gibi: i dont have it deploy but i have worked with dvd on https://bugs.launchpad.net/nova/+bug/1851545 downstream https://bugzilla.redhat.com/show_bug.cgi?id=1767797 | 10:47 |
openstack | Launchpad bug 1851545 in OpenStack Compute (nova) "Port update exception on nova unshelve for instance with PCI devices (part 2)" [Undecided,New] | 10:47 |
openstack | bugzilla.redhat.com bug 1767797 in openstack-nova "When unshelving an SR-IOV instance, the binding profile isn't reclaimed or rescheduled, and this might cause PCI-PT conflicts" [High,New] - Assigned to nova-maint | 10:47 |
sean-k-mooney | gibi: it looks like all my comments downstream are private but yes this is broken on queens i memtioned this seperatly before. we are not updating the port binding profile before generating the xml on unshelve | 10:48 |
sean-k-mooney | im not sure if it affect all release but its broken on queens | 10:48 |
*** sapd1 has quit IRC | 10:50 | |
sean-k-mooney | gibi: ill traige the upstream bug. i can proably try and replicate it later i just have not had time to work on it since it was first reported but now might be as good a time as any one i finish rebasing my current patches | 10:50 |
*** mkrai has quit IRC | 11:02 | |
*** mkrai has joined #openstack-nova | 11:03 | |
*** udesale has quit IRC | 11:09 | |
gibi | sean-k-mooney: thanks a lot! | 11:11 |
*** derekh has joined #openstack-nova | 11:15 | |
openstackgerrit | Merged openstack/nova master: Add missing white spaces between words in log messages https://review.opendev.org/720112 | 11:29 |
*** ociuhandu has quit IRC | 11:34 | |
*** ociuhandu has joined #openstack-nova | 11:35 | |
*** ociuhandu has quit IRC | 11:40 | |
*** priteau has quit IRC | 11:44 | |
*** nweinber has joined #openstack-nova | 11:52 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: Revert "Temporarily skip TestNovaMigrationsMySQL" https://review.opendev.org/718630 | 12:03 |
*** ratailor has quit IRC | 12:06 | |
gibi | bauzas and others: I have to jump on a meeting marathon for the next 2 and a half hours | 12:15 |
*** links has quit IRC | 12:17 | |
*** links has joined #openstack-nova | 12:18 | |
openstackgerrit | Merged openstack/nova-specs master: Adds spec infrastructure for Victoria https://review.opendev.org/710023 | 12:19 |
*** ttsiouts has quit IRC | 12:25 | |
*** brinzhang has quit IRC | 12:30 | |
*** dtantsur is now known as dtantsur|brb | 12:30 | |
*** rpittau|bbl is now known as rpittau | 12:31 | |
*** derekh has quit IRC | 12:33 | |
*** ttsiouts has joined #openstack-nova | 12:39 | |
*** derekh has joined #openstack-nova | 12:41 | |
*** mriedem has joined #openstack-nova | 12:44 | |
*** _kukacz has joined #openstack-nova | 12:45 | |
*** ociuhandu has joined #openstack-nova | 12:48 | |
bauzas | gibi: ack thanks, and I'll review https://review.opendev.org/#/c/719163/ | 12:48 |
*** _kukacz has quit IRC | 12:51 | |
*** _kukacz has joined #openstack-nova | 12:52 | |
*** psachin has quit IRC | 12:53 | |
*** ociuhandu has quit IRC | 12:58 | |
*** ociuhandu has joined #openstack-nova | 13:00 | |
*** ociuhandu has quit IRC | 13:01 | |
*** ociuhandu has joined #openstack-nova | 13:02 | |
openstackgerrit | Merged openstack/nova master: docs: Add stable device rescue docs https://review.opendev.org/700837 | 13:05 |
*** francoisp has joined #openstack-nova | 13:12 | |
*** rcernin has quit IRC | 13:14 | |
kashyap | lyarwood: Since I've written it partially, and been vacillating on this for a while, I'll repropose this spec for V, as that is on TODO "forever" anyway. I know you've already got plenty on your plate | 13:16 |
*** zhanglong has joined #openstack-nova | 13:16 | |
kashyap | lyarwood: And I'll add you to the 'collaborators' list, that sounds okay? | 13:17 |
lyarwood | ack sure | 13:17 |
kashyap | lyarwood: Appreciate the virtual kick in the *** | 13:20 |
kashyap | A real one will also be accepted when circumstances allow in-person gathering ;-) | 13:20 |
gmann | gibi: stephenfin thanks. let me squash both policy doc patches. | 13:22 |
*** toabctl has quit IRC | 13:25 | |
*** toabctl has joined #openstack-nova | 13:27 | |
*** zhanglong has quit IRC | 13:28 | |
*** zhanglong has joined #openstack-nova | 13:28 | |
*** spatel has joined #openstack-nova | 13:35 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/708701 | 13:43 |
*** lbragstad has joined #openstack-nova | 13:43 | |
*** KeithMnemonic has joined #openstack-nova | 13:44 | |
lyarwood | kashyap: ^ added a virt-preview job, I think swap volume might only work once we switch to block copy | 13:45 |
*** lbragstad_ has quit IRC | 13:45 | |
lyarwood | kashyap: with q35 | 13:45 |
* kashyap clicks | 13:45 | |
kashyap | lyarwood: Ah, yes. Indeed that's true | 13:46 |
lyarwood | that said I can't reproduce this outside of the tempest test | 13:49 |
* kashyap nods; will investigate later. Sorry, dealing with some emergency | 13:54 | |
lyarwood | yup np | 13:55 |
*** mlavalle has joined #openstack-nova | 13:58 | |
*** lbragstad_ has joined #openstack-nova | 14:04 | |
*** lbragstad has quit IRC | 14:06 | |
*** sapd1 has joined #openstack-nova | 14:07 | |
*** zhanglong has quit IRC | 14:18 | |
openstackgerrit | Mark Goddard proposed openstack/nova master: Add functional regression test for bug 1853009 https://review.opendev.org/695012 | 14:20 |
openstack | bug 1853009 in OpenStack Compute (nova) ussuri "Ironic node rebalance race can lead to missing compute nodes in DB" [High,In progress] https://launchpad.net/bugs/1853009 - Assigned to Mark Goddard (mgoddard) | 14:20 |
openstackgerrit | Mark Goddard proposed openstack/nova master: Clear rebalanced compute nodes from resource tracker https://review.opendev.org/695187 | 14:20 |
openstackgerrit | Mark Goddard proposed openstack/nova master: Invalidate provider tree when compute node disappears https://review.opendev.org/695188 | 14:20 |
openstackgerrit | Mark Goddard proposed openstack/nova master: Prevent deletion of a compute node belonging to another host https://review.opendev.org/694802 | 14:20 |
openstackgerrit | Mark Goddard proposed openstack/nova master: Fix inactive session error in compute node creation https://review.opendev.org/695189 | 14:20 |
mgoddard | bauzas: ^ | 14:20 |
mgoddard | unit test fix was simple, missing a fair=True argument to the synchronized decorator in RT, added upstream | 14:21 |
bauzas | mgoddard: ack, will review it later today | 14:25 |
mgoddard | thanks! | 14:25 |
bauzas | stephenfin: can you please help me understand what you mean by https://review.opendev.org/#/c/721548/1/releasenotes/notes/ussuri-prelude-4b96f1244cefcdf4.yaml@12 ? | 14:26 |
bauzas | stephenfin: looking at https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#hyperlinks I can't find what you mean | 14:27 |
stephenfin | bauzas: | 14:27 |
stephenfin | `Hello, world`__ | 14:27 |
stephenfin | 14:27 | |
stephenfin | __ https://example.com | 14:27 |
stephenfin | Or grep for '__' in releasenotes/notes | 14:27 |
bauzas | okay, but that's not something related to sphinx rST right? | 14:27 |
stephenfin | It's a general rST thing, not specific to Sphinx | 14:28 |
* bauzas trying to be a better documentator :) | 14:28 | |
stephenfin | bauzas: https://docutils.sourceforge.io/docs/user/rst/quickref.html#indirect-hyperlink-targets | 14:28 |
bauzas | ok, /me goes looking at https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html | 14:28 |
bauzas | heh, jinxed | 14:29 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP: nova-next: Start testing the 'q35' machine type https://review.opendev.org/708701 | 14:29 |
*** sapd1 has quit IRC | 14:29 | |
bauzas | ah, OK, I understand better : "The reference name of the reference is not used to match the reference to its target. Instead, the order of anonymous hyperlink references and targets within the document is significant: the first anonymous reference will link to the first anonymous target. The number of anonymous hyperlink references in a document must match the number of anonymous targets. " | 14:30 |
stephenfin | Yup. Named links must be unique in their document so they chance of having conflicts in release notes is reasonably high, given the volume of them we have | 14:31 |
stephenfin | *so the chance | 14:32 |
*** _erlon_ has quit IRC | 14:40 | |
*** _erlon_ has joined #openstack-nova | 14:41 | |
*** sapd1 has joined #openstack-nova | 14:41 | |
*** dtantsur|brb is now known as dtantsur | 14:47 | |
stephenfin | dansmith: Am I wrong, or should this not happen? https://bugs.launchpad.net/nova/+bug/1868033 | 14:47 |
openstack | Launchpad bug 1868033 in OpenStack Compute (nova) "Booting instance with pci_device fails during rocky->stein live upgrade" [High,Confirmed] - Assigned to Stephen Finucane (stephenfinucane) | 14:47 |
stephenfin | Because of https://github.com/openstack/oslo.versionedobjects/blob/master/oslo_versionedobjects/base.py#L555-L564 | 14:47 |
dansmith | stephenfin: well, he's saying that it _is_ trying to backport, but it's not backporting far enough, which makes it sound like the manifest being sent it wrong or something | 14:50 |
dansmith | which really shouldn't happen | 14:50 |
*** dklyle has joined #openstack-nova | 14:50 | |
dansmith | he's wrong to equate the upgrade_level to anything with objects though -- they are unrelated | 14:50 |
stephenfin | yeah, that patch isn't a suitable fix | 14:50 |
dansmith | stephenfin: I wonder if it's because that object isn't calling super() | 14:53 |
dansmith | and its obj_make_compat() is only handling the sub-object backporting, which shouldn't be needed anymore anyway, so maybe this one got skipped when those were culled? | 14:53 |
stephenfin | Ah, good spot. That's likely it | 14:55 |
stephenfin | the superclass handles ListOfObjectField and ObjectField fields explicitly | 14:55 |
dansmith | right, I'm having to reboot my dev machine, | 14:56 |
dansmith | but I think you'll find most list objects don't have an explicit backport routine anymore | 14:56 |
dansmith | yeah, confirmed, but let me look up the culling patch | 14:57 |
dansmith | ah, I think I see | 14:59 |
dansmith | we used to use that obj_relationships mapping, and obj_make_compatible was still centralized based on that map for lists | 15:00 |
dansmith | but this one didn't use that, so it was overriding from the beginning and thus didn't get caught in the grep | 15:00 |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Ussuri 21.0.0 prelude section https://review.opendev.org/721548 | 15:02 |
dansmith | stephenfin: I think we need to get sam to test the proper fix..happy to do that unless you've already started | 15:02 |
stephenfin | dansmith: Nope, go for it | 15:02 |
*** mgariepy has quit IRC | 15:04 | |
*** kevinbenton has quit IRC | 15:05 | |
bauzas | gibi: stephenfin: if you don't mind, as I said in a comment, I don't want to depend on https://review.opendev.org/#/c/720129/ | 15:05 |
*** kevinbenton has joined #openstack-nova | 15:06 | |
bauzas | we had a terrible experience once in Newton when we merged the prelude reno file *after* we branched to Ocata (IIRC) | 15:06 |
bauzas | because the prelude wasn't seen in 14.0.0 then | 15:06 |
bauzas | yeah, I remembered correctly : https://docs.openstack.org/releasenotes/nova/newton.html#release-notes-14-0-1-stable-newton | 15:07 |
bauzas | to avoid this, you need to first merge a prelude reno file in the master, then once you branch, you can backport a fix for the file | 15:08 |
gibi | bauzas: OK, then lets have a preliminary prelude merged soon, and fix it when the doc is available | 15:13 |
*** mgariepy has joined #openstack-nova | 15:17 | |
openstackgerrit | Dan Smith proposed openstack/nova master: Remove stale nested backport from InstancePCIRequests https://review.opendev.org/721667 | 15:20 |
*** lbragstad_ is now known as lbragstad | 15:21 | |
gibi | bauzas: I still not get where we use aggregate metadata for image pre-chacing feature. I think we simply use aggregates but not aggregate metadata | 15:27 |
bauzas | gibi: okay, that's maybe me | 15:27 |
bauzas | I may have misunderstood the API version | 15:27 |
bauzas | gibi: OK, you're right, will respin | 15:30 |
bauzas | https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/image-precache-support.html#os-aggregates | 15:31 |
gibi | thanks | 15:33 |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Ussuri 21.0.0 prelude section https://review.opendev.org/721548 | 15:36 |
gibi | bauzas: +2 ^^ | 15:39 |
bauzas | ack, thanks | 15:39 |
stephenfin | bauzas: think you missed something | 15:39 |
bauzas | we should hold a bit until a handful of cores and contributors have reviewed it I think | 15:39 |
bauzas | ahah, see | 15:39 |
bauzas | stephenfin: probably, tell me what :) | 15:40 |
stephenfin | I left a comment :) | 15:40 |
*** tkajinam has quit IRC | 15:40 | |
gibi | OK I will only vote +1 until most of the cores leave some feedback on it | 15:41 |
stephenfin | bauzas: fwiw, because you're not using Sphinx'isms there, you can use a tool like rst.ninjs.org for rendering | 15:41 |
*** udesale_ has quit IRC | 15:41 | |
stephenfin | (so long as you remove the yaml stuff and dedent things, obviously) | 15:41 |
bauzas | stephenfin: you can test locally with the releasenotes target | 15:44 |
bauzas | stephenfin: that's what I did and it worked | 15:44 |
bauzas | stephenfin: also replied | 15:44 |
bauzas | with the releasenotes tox* target, my bad | 15:44 |
bauzas | (but remember, you need to commit first your changes in order for reno to work | 15:45 |
stephenfin | Ah, you're right. My mistake. | 15:45 |
bauzas | <scratch, scratch> => git commit => tox -ereleasenotes (and take a coffee cup, it'll take a while to generate) | 15:45 |
stephenfin | I've another point left about putting the URLs between the bullet points too /o\ | 15:45 |
stephenfin | They render really weirdly | 15:46 |
stephenfin | Because it's rendering as multiple <ul>s instead of one <ul> with multiple <li>s | 15:46 |
bauzas | stephenfin: weirdo, look at what the tox target generates : http://paste.openstack.org/raw/792482/ | 15:47 |
bauzas | (open it as an HTML file) | 15:48 |
stephenfin | yeah, see how there are multiple <ul> tags? | 15:48 |
stephenfin | there should only be one | 15:48 |
bauzas | ok, I can fix this, I see your point | 15:50 |
stephenfin | bauzas: actually, you can do this | 15:50 |
stephenfin | http://rst.ninjs.org/#LSBgSGVsbG9gX18KCiAgLi4gX186IFRlc3QKCi0gV29ybGQKLSBUZXN0 | 15:50 |
stephenfin | just indent the links | 15:50 |
bauzas | yeah I was thinking on indenting, not dedenting | 15:50 |
bauzas | hence the <ul> tags | 15:51 |
stephenfin | cool | 15:51 |
bauzas | if I wasn't using the bullets but an ordered list, we would get 1. 1. 1. | 15:51 |
stephenfin | exactly | 15:51 |
bauzas | just verifying that's the only asked nit besides your rewordings | 15:52 |
*** gyee has joined #openstack-nova | 15:52 | |
*** tbachman has quit IRC | 15:53 | |
bauzas | stephenfin: b/c I don't wanna get a stylistic -1 again, you okay with me saying : to configure `read-only` access | 15:56 |
*** tbachman has joined #openstack-nova | 15:56 | |
bauzas | and not : to configure "read-only" access | 15:56 |
bauzas | (using italics instead of brakets) | 15:56 |
stephenfin | *read-only* would be better | 15:56 |
bauzas | bold, dude ? | 15:56 |
bauzas | man, you're gross | 15:56 |
stephenfin | nope, that's **read-only** | 15:57 |
* bauzas facepalms | 15:57 | |
stephenfin | this ain't markdown, my man | 15:57 |
* bauzas learned German, English and a bit of Spanish, but apologies for having missed rST | 15:57 | |
stephenfin | :P | 15:57 |
stephenfin | single backticks mean default role | 15:57 |
stephenfin | which is configurable | 15:57 |
stephenfin | in most environment it defaults to italics | 15:58 |
bauzas | ah-ha I see | 15:58 |
* stephenfin learned that the hard way writing docs for Sphinx itself :( | 15:58 | |
bauzas | there is a good reason why it's named **Sphinx** AFAICT | 15:58 |
bauzas | they could have subtitled it 'old greek' | 15:59 |
bauzas | (and while we speak, it's building relnotes locally... | 16:00 |
*** ttsiouts has quit IRC | 16:04 | |
bauzas | (still building) | 16:05 |
bauzas | hold my beer | 16:05 |
*** mkrai has quit IRC | 16:05 | |
*** ttsiouts has joined #openstack-nova | 16:05 | |
bauzas | mgoddard: I haven't forgotten you, but my time was on working for release notes | 16:06 |
stephenfin | we're going to need to start compressing release notes again at some point | 16:06 |
stephenfin | like we did for e.g. mitaka | 16:06 |
mgoddard | bauzas: I've been waiting long enough, I can wait a little longer :) | 16:06 |
bauzas | stephenfin: that's not the problem | 16:06 |
bauzas | stephenfin: it does scan all my local branches | 16:06 |
bauzas | this is what takes a while | 16:06 |
bauzas | I should drop the local stable branches, that's it | 16:07 |
stephenfin | nah, it'll drag them in from the remote | 16:08 |
stephenfin | iirc | 16:08 |
bauzas | maybe, that's been a while I haven't really looked deep at the reno code itself | 16:09 |
bauzas | (Ocata I think) | 16:09 |
*** mkrai has joined #openstack-nova | 16:10 | |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Ussuri 21.0.0 prelude section https://review.opendev.org/721548 | 16:14 |
*** _kukacz has quit IRC | 16:14 | |
bauzas | wow | 16:14 |
bauzas | 5 mins to upload, thanks Netflix | 16:14 |
*** ociuhandu has quit IRC | 16:15 | |
*** mkrai has quit IRC | 16:15 | |
*** ociuhandu has joined #openstack-nova | 16:17 | |
*** ociuhandu has quit IRC | 16:20 | |
*** ociuhandu has joined #openstack-nova | 16:21 | |
*** yaawang_ has joined #openstack-nova | 16:32 | |
openstackgerrit | Elod Illes proposed openstack/nova stable/rocky: WIP: libvirt: check job status for VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED event https://review.opendev.org/711233 | 16:32 |
*** jraju__ has joined #openstack-nova | 16:32 | |
*** links has quit IRC | 16:33 | |
*** yaawang has quit IRC | 16:33 | |
*** evrardjp has quit IRC | 16:35 | |
*** evrardjp has joined #openstack-nova | 16:35 | |
*** lseki has quit IRC | 16:38 | |
*** happyhemant has quit IRC | 16:38 | |
*** rpittau is now known as rpittau|afk | 16:39 | |
*** masayukig has quit IRC | 16:39 | |
*** nightmare_unreal has quit IRC | 16:40 | |
*** happyhemant has joined #openstack-nova | 16:40 | |
*** jrosser has quit IRC | 16:41 | |
*** klindgren has quit IRC | 16:41 | |
*** klindgren has joined #openstack-nova | 16:41 | |
*** mnaser has quit IRC | 16:41 | |
*** lseki has joined #openstack-nova | 16:42 | |
*** maciejjozefczyk has quit IRC | 16:42 | |
*** bcm has quit IRC | 16:42 | |
*** TheJulia has quit IRC | 16:42 | |
*** csatari has quit IRC | 16:43 | |
*** masayukig has joined #openstack-nova | 16:43 | |
*** jrosser has joined #openstack-nova | 16:43 | |
*** bcm has joined #openstack-nova | 16:44 | |
*** TheJulia has joined #openstack-nova | 16:44 | |
*** nightmare_unreal has joined #openstack-nova | 16:44 | |
*** mnaser has joined #openstack-nova | 16:44 | |
*** csatari has joined #openstack-nova | 16:45 | |
*** vdrok has quit IRC | 16:45 | |
*** mnaser has quit IRC | 16:46 | |
*** mnaser has joined #openstack-nova | 16:47 | |
*** vdrok has joined #openstack-nova | 16:47 | |
*** derekh has quit IRC | 16:48 | |
*** mnaser has quit IRC | 16:49 | |
*** mnaser has joined #openstack-nova | 16:49 | |
*** jraju__ has quit IRC | 16:57 | |
*** dtantsur is now known as dtantsur|afk | 16:59 | |
*** ajitha has joined #openstack-nova | 17:01 | |
ajitha | hi all, we have running a openstack CI for cinder.. with ubuntu 18.04 and py3.6... After this commit https://opendev.org/openstack/nova/commit/f1e448d6729f674101b34b08e3c5e95fd8f75770 , our CI/CD pipeline is getting break in devstack with following error "pip._vendor.pkg_resources.ContextualVersionConflict: (oslo.policy 3.0.3 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('oslo.policy>=3.1.0'), {'nova'}).." | 17:05 |
ajitha | how to suppress this from local.conf? | 17:06 |
ajitha | or any method to send suppress_deprecation_warnings=True from local.conf | 17:07 |
stephenfin | ajitha: Are you using system packages? | 17:08 |
stephenfin | Python packages, that is | 17:09 |
ajitha | i am using python packages mostly...you are asking for oslo.policy? | 17:09 |
stephenfin | Yeah, is this just a bare Ubuntu 18.04 host with DevStack on it? | 17:10 |
stephenfin | I'm checking whether you've installed oslo.policy from pip or apt | 17:10 |
ajitha | yes bare ubuntu 18.04 with devstack | 17:11 |
ajitha | from pip only not from apt | 17:11 |
stephenfin | okay, can you paste the full traceback (paste.openstack.org) | 17:11 |
stephenfin | ? | 17:12 |
ajitha | sure | 17:12 |
ajitha | thanks | 17:12 |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Fix the followup comment of policy doc https://review.opendev.org/721322 | 17:13 |
*** nightmare_unreal has quit IRC | 17:13 | |
ajitha | stephenfin: http://paste.openstack.org/show/792492/ | 17:21 |
ajitha | paste.openstack didnt allow to paste the complete log.. | 17:22 |
gmann | stephenfin: fixed the comment in this to compare those. merging the same in original patch now https://review.opendev.org/721322 | 17:22 |
stephenfin | ajitha: I suspect you're hitting https://github.com/pypa/pip/issues/988 | 17:25 |
stephenfin | and unfortunately I don't have any easy answers, you need something to install oslo.config >= 3.1.0 before 3.0.3 gets installed | 17:26 |
ajitha | manual devstack execution works fine with an error throwing as ERROR: nova 20.1.0.dev1681 has requirement oslo.policy>=3.1.0, you'll have oslo-policy 3.0.3 which is incompatible... but it skips that and devstack gets completed. and while executing devstack-gate jenkins job, devstack gets error at the above point | 17:26 |
stephenfin | We're not hitting that in the nova gate so I suspect it's something to do with a plugin that's installing a package that depends on oslo.config < 3.1.0 | 17:27 |
stephenfin | a plugin in your CI environment, that is | 17:27 |
stephenfin | the folks on #openstack-infra might be more help; they're well used to dealing with pip's brokenness /o\ | 17:28 |
ajitha | thank you. | 17:29 |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add docs and releasenotes for BP policy-defaults-refresh https://review.opendev.org/720129 | 17:29 |
ajitha | it works with py3.6 too?? | 17:29 |
stephenfin | the gate? yeah, I don't think this is a py3.6 issue | 17:30 |
stephenfin | it's a pip issue, I'd say | 17:30 |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Add docs and releasenotes for BP policy-defaults-refresh https://review.opendev.org/720129 | 17:30 |
ajitha | stephenfin : thanks. The error is throwing in the manual devstack execution(./stack.sh) but it skips and complete the devstack.. In case of devstack-gate, it just stops at that point | 17:32 |
*** ociuhandu has quit IRC | 17:34 | |
*** ociuhandu has joined #openstack-nova | 17:35 | |
*** ociuhandu has quit IRC | 17:40 | |
*** tesseract has quit IRC | 17:59 | |
*** hamzy__ has quit IRC | 17:59 | |
*** sapd1 has quit IRC | 18:05 | |
gmann | stephenfin: gibi this is ready now - https://review.opendev.org/#/c/720129/11 | 18:15 |
*** ralonsoh has quit IRC | 18:16 | |
*** ociuhandu has joined #openstack-nova | 18:24 | |
*** gmann is now known as gmann_lunch | 18:31 | |
*** ociuhandu has quit IRC | 18:44 | |
*** ociuhandu has joined #openstack-nova | 18:45 | |
*** ociuhandu has quit IRC | 18:45 | |
*** ociuhandu has joined #openstack-nova | 18:45 | |
*** jangutter has quit IRC | 18:49 | |
*** gmann_lunch is now known as gmann | 18:51 | |
*** belmoreira has quit IRC | 18:53 | |
*** jangutter has joined #openstack-nova | 18:59 | |
*** ociuhandu has quit IRC | 19:11 | |
*** ociuhandu has joined #openstack-nova | 19:11 | |
*** slaweq has quit IRC | 19:14 | |
*** ociuhandu has quit IRC | 19:17 | |
*** slaweq has joined #openstack-nova | 19:19 | |
*** ttsiouts has quit IRC | 19:21 | |
*** ttsiouts has joined #openstack-nova | 19:25 | |
*** grandchild has joined #openstack-nova | 19:32 | |
*** ccamacho has quit IRC | 19:34 | |
*** breizhkoala has quit IRC | 19:35 | |
*** martinkennelly has quit IRC | 19:36 | |
*** ociuhandu has joined #openstack-nova | 19:48 | |
*** ociuhandu has quit IRC | 19:53 | |
*** ajitha has quit IRC | 20:02 | |
*** mgariepy has quit IRC | 20:09 | |
*** mgariepy has joined #openstack-nova | 20:12 | |
*** xek has quit IRC | 20:30 | |
*** nweinber has quit IRC | 20:39 | |
*** ociuhandu has joined #openstack-nova | 21:03 | |
*** ociuhandu has quit IRC | 21:08 | |
*** dlbewley has joined #openstack-nova | 21:10 | |
*** bbowen has quit IRC | 21:15 | |
*** bbowen has joined #openstack-nova | 21:16 | |
*** spatel has quit IRC | 21:30 | |
*** dpawlik has quit IRC | 21:34 | |
*** sean-k-mooney has quit IRC | 21:40 | |
*** slaweq has quit IRC | 21:42 | |
*** sean-k-mooney has joined #openstack-nova | 21:43 | |
*** mriedem has left #openstack-nova | 21:45 | |
*** ttsiouts has quit IRC | 21:46 | |
*** ttsiouts has joined #openstack-nova | 22:02 | |
*** rcernin has joined #openstack-nova | 22:17 | |
*** ttsiouts has quit IRC | 22:46 | |
*** tkajinam has joined #openstack-nova | 22:53 | |
*** spatel has joined #openstack-nova | 22:53 | |
*** tosky has quit IRC | 23:03 | |
*** avolkov has quit IRC | 23:25 | |
*** ociuhandu has joined #openstack-nova | 23:25 | |
*** ociuhandu has quit IRC | 23:30 | |
*** grandchild has quit IRC | 23:36 | |
*** mlavalle has quit IRC | 23:37 | |
*** tetsuro has joined #openstack-nova | 23:42 | |
*** tetsuro has quit IRC | 23:44 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!