| opendevreview | Merged openstack/nova master: Define new functional test tox env for placement gate to run https://review.opendev.org/c/openstack/nova/+/813679 | 00:55 |
|---|---|---|
| opendevreview | Steve Baker proposed openstack/nova master: Remain in DELETING for ironic CLEANING, CLEANWAIT https://review.opendev.org/c/openstack/nova/+/813729 | 02:01 |
| opendevreview | Merged openstack/placement master: Use 'functional-without-sample-db-tests' tox env for placement nova job https://review.opendev.org/c/openstack/placement/+/813680 | 02:03 |
| opendevreview | melanie witt proposed openstack/nova master: Handle urlencoded lists when filtering servers by tags https://review.opendev.org/c/openstack/nova/+/813736 | 03:29 |
| *** bhagyashris is now known as bhagyashris|rover | 05:21 | |
| bauzas | good morning Nova | 06:54 |
| gibi | bauzas: o/ | 06:55 |
| gibi | gmann: thanks for unblocking the placement gate with https://review.opendev.org/c/openstack/placement/+/813680 I have concerns that we only solved the local problem and not the root of the problem in zuul. I do think that tox_extra_args is defined to pass extra args to tox, and the test case filter regexp is such extra args | 06:56 |
| gibi | it is not even a problem that zuul passes that args to tox when printing the config as tox accepts the extra args there too, but the problem is that it is passed without proper quoting | 06:57 |
| gibi | so whoever will use tox_extra_args for anything in zuul that needs quoting will hit the same issue as we hit | 06:57 |
| bauzas | can't disagree with gibi here | 07:02 |
| bauzas | but at least we unblocked the periodic run | 07:02 |
| gibi | bauzas: yeah, I'm OK to unblock the gate then fix the real problem | 07:03 |
| gibi | that is a good order | 07:03 |
| gibi | bauzas: if you have time for a smallish workaround fix to review that would be appreciated https://review.opendev.org/c/openstack/nova/+/813419 | 07:07 |
| bauzas | gibi: sure, today I'm trying to dedicate time for upstream reviews and bug triage | 07:08 |
| gibi | thanks | 07:08 |
| alecorps7 | hello | 07:17 |
| bauzas | gibi: -1 for a relnote missing but see my comments https://review.opendev.org/c/openstack/nova/+/813419 | 07:38 |
| gibi | bauzas: thanks, I will respin | 07:39 |
| gibi | bauzas: regarding a better solution I discussed that with sean-k-mooney | 07:39 |
| gibi | bauzas: we need more info from neutron to decide when to wait for a plug event from neutron | 07:39 |
| bauzas | gibi: we can discuss this next week with the neutron folks | 07:40 |
| gibi | bauzas: we can try yes | 07:40 |
| bauzas | gibi: what I'd like is that *all* neutron backends should provide an event | 07:40 |
| bauzas | nova shouldn't know which backend neutron uses | 07:40 |
| gibi | bauzas: yes that would be also an option but I think that is a harder one | 07:40 |
| bauzas | at least for an event | 07:41 |
| bauzas | I don't understand why it'd be hard for neutron to provide an event for plugs | 07:41 |
| bauzas | again, having events be different between neutron backends looks bad to me | 07:42 |
| bauzas | gibi: just added a point about this in https://etherpad.opendev.org/p/nova-yoga-ptg L105 | 07:44 |
| gibi | bauzas: thanks I will try to fill in the details htere | 07:49 |
| gibi | sean-k-mooney: ^^ | 07:49 |
| gibi | fy | 07:49 |
| gibi | fyi | 07:49 |
| * bauzas wonders whether our CI should suffer from the OVH outage they have atm | 08:20 | |
| frickler | bauzas: lots of jobs with POST_FAILURE for failing log upload, but they seem to be back now, so safe to recheck hopefully | 08:42 |
| bauzas | frickler: yeah the outage is now done | 08:44 |
| bauzas | they isolated the faulty DC | 08:44 |
| bauzas | thread in French https://twitter.com/olesovhcom/status/1448196879020433409 | 08:45 |
| opendevreview | Elod Illes proposed openstack/nova stable/ussuri: Revert "[stable-only] Set lower-constraints job as non-voting" https://review.opendev.org/c/openstack/nova/+/813784 | 08:47 |
| sean-k-mooney | bauzas: nova really really really need to know what what backend neutron uses | 09:45 |
| sean-k-mooney | bauzas: or they need to tell use exactly when they will send the envents for that backend opacly | 09:46 |
| sean-k-mooney | bauzas: not all backend can send plug time events because they dont all have a way to send info too neutron about when the port is created and its finsihed wiring it up | 09:47 |
| sean-k-mooney | bauzas: at present ovn cannot do that although the ml2 dirver can be modifed to eventually do that | 09:47 |
| sean-k-mooney | bauzas: for odl they had to create a websocket on the netorn server and modify odl to seend events to that websocket to implemente plug time events whihc invovled a lot fo code development in odl | 09:48 |
| sean-k-mooney | bauzas: contrail still has not implemnted multiple port binding gettingthem to send plug time events likely will never happen | 09:48 |
| sean-k-mooney | linux bridge can send pulg time event but because it currently pool for interface chcnages insted of opening a netlink socket and reciving event it can miss the removal and addtion of interface on hard reboot | 09:49 |
| bauzas | sean-k-mooney: sorry but for me, events are like notifications | 09:50 |
| bauzas | sean-k-mooney: it's kinda a public API | 09:50 |
| sean-k-mooney | bauzas: its more a contract | 09:51 |
| sean-k-mooney | bauzas: i proposed that neutron tell use when they send events or normalise to a common contract back in train and neutron did not agree to either | 09:52 |
| opendevreview | Alexey Stupnikov proposed openstack/nova master: Rollback problematic port bindings on source host only https://review.opendev.org/c/openstack/nova/+/813659 | 09:52 |
| sean-k-mooney | bauzas: you can see in the first version fo thsi we had a network_events section https://review.opendev.org/c/openstack/neutron-specs/+/645173/1/specs/train/port-binding-extended-information.rst#141 | 09:55 |
| sean-k-mooney | bauzas: this is the orginal etherpad i wrote with rodolfo https://etherpad.opendev.org/p/portbinding-records | 09:58 |
| sean-k-mooney | it was coverd durign the train corss project session https://etherpad.opendev.org/p/ptg-train-xproj-nova-neutron | 09:59 |
| *** mdbooth9 is now known as mdbooth | 10:00 | |
| sean-k-mooney | bauzas: i am fine with finding a better solution to what i proposed in thte past however nova must know the behaivor of the neutron backend to operat correctly | 10:00 |
| sean-k-mooney | what we are doing today is complex and very error prone and i spend far too much of my time currently fixing bugs that are cause by not knowing when events will be sent | 10:01 |
| bauzas | sean-k-mooney: that's my concern | 10:06 |
| bauzas | the more nova needs to know about neutron, the more issues we could get | 10:06 |
| bauzas | as operators need to set different options per service | 10:07 |
| bauzas | and they can miss some related options | 10:07 |
| sean-k-mooney | bauzas: i have been trying to get neuton to expose the info we need for litrally year at this point with the goal of ensureing that no config option are need in nova | 10:07 |
| sean-k-mooney | this has been an ongoing battle since before placment or os-vif was a thing | 10:08 |
| sean-k-mooney | so i agree | 10:08 |
| sean-k-mooney | we shoudl not need operators to care or set anything | 10:08 |
| sean-k-mooney | but to do that we must know either what contract the neutron backend has abstractly or we need to embed that knolage in nova and just know what backend it is | 10:09 |
| sean-k-mooney | currently we try tro guess based on the vif_type and some other partmeter in the port binding_details field | 10:09 |
| sean-k-mooney | but we dont really have the info we need | 10:09 |
| bauzas | sean-k-mooney: yeah hence the need of a nova-neutron discussion at the PTG | 10:11 |
| sean-k-mooney | https://github.com/openstack/nova/blob/7b063e4d0518af3e57872bc0288a94edcd33c19d/nova/network/model.py#L560-L579 this is what i have tried to do lately since the last time they rejected adding the events | 10:11 |
| sean-k-mooney | with has_bind_time basically being https://github.com/openstack/nova/blob/7b063e4d0518af3e57872bc0288a94edcd33c19d/nova/network/model.py#L481-L490 | 10:12 |
| sean-k-mooney | bauzas: im happy for there to be a PTG discussion on this since i planned to bring this up anyway as part fo the ovn migration disucssion | 10:13 |
| bauzas | cool | 10:13 |
| sean-k-mooney | bauzas: but i just dont have high confidence that anything will be done about it | 10:13 |
| sean-k-mooney | its at least the 4th time we will have talked about it | 10:14 |
| bauzas | it's a yet again "Dare to Care" discussion, heh | 10:14 |
| sean-k-mooney | if we really want to fix this perhaps we (nova pepole) might need to go implement it in neuton | 10:15 |
| bauzas | well | 10:15 |
| bauzas | first, let's see what's coming | 10:15 |
| sean-k-mooney | for reasons i know that our neturon folk at redhat are proposing adding plug time event support to ovn | 10:16 |
| sean-k-mooney | reason being live migration | 10:16 |
| sean-k-mooney | but to actully fix ovn live migration we need to actully change ovn too | 10:16 |
| sean-k-mooney | the way ovn is currently desigined its not possibel to have 0 down time live migration with libvirt | 10:17 |
| lpetrut | hi, could you please take another look over the gmr patch? https://review.opendev.org/c/openstack/nova/+/810922 | 10:26 |
| sean-k-mooney | sure | 10:27 |
| lpetrut | thanks | 10:27 |
| bauzas | lpetrut: excellent catch | 10:31 |
| bauzas | lpetrut: could you please fill a bug against gmr not working properly due to uswsgi | 10:31 |
| bauzas | I'd like this to be documented in the yoga relnotes if we merge it | 10:32 |
| sean-k-mooney | bauzas: its not really a GMR bug | 10:32 |
| sean-k-mooney | you already found that we can pass signals to the python app if we configure uwsgi correctly | 10:33 |
| sean-k-mooney | but by default it will trap the sig_usr2 | 10:33 |
| lpetrut | I've added a Nova release note. indeed, it doesn't seem like an oslo.reports bug, if needed I can file a bug against nova. last time, the consensus was that it's a minor feature that doesn't require a blueprint | 10:43 |
| *** mdbooth3 is now known as mdbooth | 10:44 | |
| opendevreview | Lucian Petrut proposed openstack/nova master: api: enable oslo.reports when using uwsgi https://review.opendev.org/c/openstack/nova/+/810922 | 10:53 |
| gibi | sean-k-mooney: I made a bit of progress witht the unshelve functional test. There is a reschedule happening, but it feels like it is on an instance from another test case?! see my last comment with logs in https://review.opendev.org/c/openstack/nova/+/813674 | 11:26 |
| gibi | sean-k-mooney: the subunit file has a lot more information that what is visible from the job-output.txt | 11:33 |
| gibi | it can be extracted to individual tests and it shows that the instance uuid logged in our failed test is actually mentioned in another test case log as well | 11:34 |
| sean-k-mooney | gibi: dod upi see https://review.opendev.org/c/openstack/nova/+/813695 | 11:34 |
| sean-k-mooney | ah you did | 11:35 |
| sean-k-mooney | gibi: i feel like when we are waiting for the virsion notificiaotn in that case we need to wait for one related to the vm we are unshlving | 11:35 |
| sean-k-mooney | but i tought the notifier was per test instnace | 11:36 |
| gibi | sean-k-mooney: the notifier should be unique per test case too | 11:36 |
| gibi | https://paste.opendev.org/show/809961/ | 11:36 |
| gibi | see this paste | 11:36 |
| gibi | this mentions the same uuid in two test case logs | 11:36 |
| sean-k-mooney | the fact that there are 4 in my case feels supiocisly because 4 tests run when i filter | 11:36 |
| sean-k-mooney | i.e. if i fileter by test_unshelve_offloaded_server_with_qos_port_pci_update_fails | 11:36 |
| sean-k-mooney | it runs 4 versions of that test | 11:37 |
| songwenping_ | Hi,team, when i once put two nodes in one aggregate, nova sheduler only update one node to the aggregate, if i use the other node to create vm, there are no valid host failed, the log is AZFilter return 0 hosts. | 11:37 |
| gibi | sean-k-mooney: yepp there are 4 versions | 11:37 |
| gibi | sean-k-mooney: if you see the 4 test case interacts via the notifier that is also a problem | 11:37 |
| gibi | is should not | 11:37 |
| sean-k-mooney | gibi: so im not sure that they do but i could print all of the notificaiotn object i guess | 11:37 |
| sean-k-mooney | see what they are | 11:38 |
| gibi | yeah that could help | 11:38 |
| songwenping_ | sean-k-mooney, gibi: when does nova-scheduler update host aggregate map, change host from aggregate? | 11:44 |
| gibi | sean-k-mooney: I checked 3 reproduction form the gate, it is always nova.tests.functional.test_servers.ServersTestV219.test_description_errors test case logs that mentions the same instance uuid as the failed unshelve test case | 11:46 |
| gibi | songwenping_: I don't know without looking into the code, sorry | 11:46 |
| sean-k-mooney | gibi: interesting so to repoduce this we shoudl run both of those tests | 11:47 |
| sean-k-mooney | it does sound like we are sharing global state some how | 11:47 |
| gibi | sean-k-mooney: yeah, it is alway 61 seconds after the succesfull run of .ServersTestV219.test_description_errors that the unshelve test fails | 11:48 |
| gibi | that sounds like a 60 sec timout on an RPC | 11:48 |
| songwenping_ | thanks gibi. :( | 11:49 |
| gibi | songwenping_: sorry I knee deep in someting else at the moment | 11:49 |
| sean-k-mooney | songwenping_: nova does not move host between aggreates. you have to use the api to do that | 11:50 |
| gibi | sean-k-mooney: so my assumption is that the test_description_errors case does not end cleanly | 11:50 |
| sean-k-mooney | songwenping_: i dont think we cache that infor in the scheduler its not part of the host state objects | 11:50 |
| sean-k-mooney | songwenping_: so once its commited to the db i think the schduler will see it | 11:50 |
| songwenping_ | sean-k-mooney: nova-scheduler update part of these hosts. | 11:52 |
| sean-k-mooney | songwenping_: im not sure what you mean by that | 11:54 |
| sean-k-mooney | the nova scedular nerver modfies aggreates or compute nodes | 11:54 |
| sean-k-mooney | it just makes claims in plcement and selesct the destionation for instances | 11:55 |
| sean-k-mooney | aggreate membership si entrily mange extrenally by the nova-api | 11:55 |
| songwenping_ | sean-k-mooney: nova-scheduler update its host_aggregate_map attribute | 11:55 |
| songwenping_ | nova-api change the aggregate's host | 11:56 |
| songwenping_ | let me debug on my env first. | 11:56 |
| sean-k-mooney | what release of openstack are you using | 11:58 |
| songwenping_ | R | 11:58 |
| sean-k-mooney | rocky | 11:58 |
| songwenping_ | yes | 11:58 |
| sean-k-mooney | im not sure that exist in the host manager anymore | 11:58 |
| sean-k-mooney | ok it does | 11:59 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/scheduler/host_manager.py#L353 | 11:59 |
| sean-k-mooney | its plural host_aggregates_map not host_aggregate_map | 12:00 |
| songwenping_ | yes sorry | 12:00 |
| sean-k-mooney | so its called here https://github.com/openstack/nova/blob/7b063e4d0518af3e57872bc0288a94edcd33c19d/nova/scheduler/manager.py#L662-L670 | 12:01 |
| sean-k-mooney | which is called in a number of places in tghe compute api https://github.com/openstack/nova/blob/7b063e4d0518af3e57872bc0288a94edcd33c19d/nova/compute/api.py#L6231 | 12:02 |
| sean-k-mooney | this is where we update it when you add a host https://github.com/openstack/nova/blob/7b063e4d0518af3e57872bc0288a94edcd33c19d/nova/compute/api.py#L6414 | 12:02 |
| sean-k-mooney | songwenping_: so we update it after we update it in the db but before we update it in placment | 12:04 |
| sean-k-mooney | songwenping_: so if you are using placment for aggres there is a very short period of time where the host is a member of an aggreate but placment doe snot know yet | 12:04 |
| sean-k-mooney | however all of this happens before the api request returns | 12:04 |
| songwenping_ | yeah, but the host param seems a single once | 12:05 |
| sean-k-mooney | the invocation of the schduler metohd is a cast | 12:06 |
| sean-k-mooney | so it might take a while after we retrun for that to propageate and the schdulers to update | 12:06 |
| songwenping_ | sean-k-moony: not exactly, we wait a long time. | 12:10 |
| sean-k-mooney | songwenping_: what is the actul problem you are seeing | 12:12 |
| sean-k-mooney | have you filed a bug describing it with logs | 12:12 |
| songwenping_ | not yet, i am not sure | 12:12 |
| sean-k-mooney | i currently on a call so i dont really have time to help debug it now | 12:12 |
| songwenping_ | no hurry. | 12:13 |
| songwenping_ | i am debugging now. | 12:14 |
| gibi | sean-k-mooney: seems test_description_errors can fail and leave running greenthreads behind: https://paste.opendev.org/show/809966/ | 12:48 |
| gibi | I mean can produce an error without the testcase failing | 12:49 |
| gibi | then leaking running greanthreads | 12:49 |
| sean-k-mooney | i see | 12:49 |
| sean-k-mooney | ok and i guess that can somehow impact other tests | 12:49 |
| gibi | yeah it is still strange how these tests interact. there is a global state somewhere | 12:50 |
| sean-k-mooney | could there somehow be sharing betwen the fake_impl for oslo.messaging | 12:50 |
| sean-k-mooney | or the db fixture. it seams like somehost when those finally time out it trigres the db to be cleaned up | 12:51 |
| sean-k-mooney | or i guess the notifcation fixture but really im just speculating | 12:53 |
| gibi | I dig forward.. | 12:53 |
| sean-k-mooney | at least we have something to investigate | 12:53 |
| gibi | yeah | 12:54 |
| opendevreview | Federico Ressi proposed openstack/nova master: Debug Nova APIs call failures https://review.opendev.org/c/openstack/nova/+/806683 | 13:03 |
| opendevreview | Federico Ressi proposed openstack/nova master: Check Nova project changes with Tobiko scenario test cases https://review.opendev.org/c/openstack/nova/+/806853 | 13:04 |
| kashyap | sean-k-mooney: I haven't checked the specs and the Git logs yet - do you recall what's the status of vDPA in upstream Nova? | 13:18 |
| sean-k-mooney | we have partial support | 13:19 |
| sean-k-mooney | we do not support any move operations | 13:19 |
| sean-k-mooney | and we only support it for netwok devices | 13:19 |
| kashyap | Ah, right - no move support | 13:19 |
| sean-k-mooney | with ovs hardware offload | 13:20 |
| sean-k-mooney | e.g. we do not support vdpa in legacy mode only in switchdev mode and the sriovnic_agent only support legacy mode | 13:21 |
| sean-k-mooney | so you cant use vdpa with openstack without ovs as the contol plane using tcflower to instll flow into the nic | 13:21 |
| sean-k-mooney | kashyap: was there a resaond you wanted to know | 13:21 |
| sean-k-mooney | for our product it will be tech previwe in 17 until we can actully support at least some of the move ops | 13:22 |
| kashyap | sean-k-mooney: Not for product - I was curious from an upstream PoV | 13:22 |
| sean-k-mooney | i may add more support this cycle but hardware has been a limiting factor to working on this | 13:23 |
| kashyap | The reason was just I was reading about vDPA and it reminded me of it | 13:23 |
| sean-k-mooney | ack | 13:23 |
| *** bhagyashris|rover is now known as bhagyashris | 13:39 | |
| sean-k-mooney | does this look familar to anyone | 14:23 |
| sean-k-mooney | https://paste.opendev.org/show/809977/ | 14:23 |
| sean-k-mooney | ] GET /placement/resource_providers?in_tree=4838f356-678a-477c-a2b8-7a743ad0842c => generated 162 bytes in 5 msecs (HTTP/1.1 404) | 14:23 |
| sean-k-mooney | but in the debug logs the path is wrong | 14:24 |
| sean-k-mooney | 92.168.1.10 "GET /placemen//resource_providers?in_tree=4838f356-678a-477c-a2b8-7a743ad0842c" status: 404 | 14:24 |
| sean-k-mooney | the apach2 config just has ProxyPass "/placement" "unix:/var/run/uwsgi/placement-api.socket|uwsgi://uwsgi-uds-placement-api/" retry=0 | 14:25 |
| sean-k-mooney | this is causeing "/usr/local/bin/nova-status --config-file /etc/nova/nova.conf upgrade check" to fail which breaks devstack | 14:26 |
| alecorps | hello | 14:26 |
| alecorps | is someone can have a look to the PR 808791 ? | 14:27 |
| sean-k-mooney | PR as in pull request | 14:28 |
| gibi | sean-k-mooney: you have wrong config in you apache proxy | 14:28 |
| gibi | sean-k-mooney: in devstack | 14:28 |
| gibi | sean-k-mooney: searching for the commit... | 14:28 |
| gibi | sean-k-mooney: https://review.opendev.org/c/openstack/devstack/+/811303 | 14:28 |
| sean-k-mooney | gibi: i guess i need to pull devstack? | 14:28 |
| gibi | probably yes | 14:28 |
| gibi | you need the above fix is you have a new enough apache | 14:29 |
| sean-k-mooney | oh ok | 14:29 |
| sean-k-mooney | it removed the / on the socket | 14:29 |
| sean-k-mooney | alecorps: what is PR 808791 | 14:30 |
| alecorps | https://review.opendev.org/c/openstack/nova/+/808791 | 14:30 |
| alecorps | yes pull request :) | 14:31 |
| alecorps | sorry | 14:31 |
| sean-k-mooney | alecorps: ok that is not a pull request | 14:31 |
| sean-k-mooney | its a gerrit reivew | 14:31 |
| alecorps | yes sorry | 14:31 |
| sean-k-mooney | they are diffenet code review systems we used ot somtiem get PRs to the github mirror | 14:31 |
| sean-k-mooney | ok so that is a new feature | 14:32 |
| sean-k-mooney | dos it have a approve spec or blueprint | 14:32 |
| sean-k-mooney | https://blueprints.launchpad.net/nova/+spec/vmware-fcd | 14:33 |
| sean-k-mooney | ok so that is not approved currently | 14:33 |
| sean-k-mooney | so ideally we need to review that at the next team meeting and either approve as a specless blueprint or request a spec if its invalice | 14:34 |
| alecorps | ok, do i need to be there ? | 14:34 |
| sean-k-mooney | it looks likke its self contained to the vmware code | 14:34 |
| sean-k-mooney | it just need to be put on the adgenda | 14:34 |
| sean-k-mooney | hovever we might not have next weeks meeting because of the ptg | 14:35 |
| sean-k-mooney | so the meeting after | 14:35 |
| alecorps | ok cool | 14:35 |
| sean-k-mooney | i can take a look at it in the mean time but procetully its blocked until the paper work is done | 14:35 |
| sean-k-mooney | you could just add it to the ptg adgenda too | 14:35 |
| alecorps | understood, thanks | 14:35 |
| alecorps | I iwll add it in the agenda | 14:37 |
| alecorps | i added it in open discussion on the wiki | 14:47 |
| alecorps | let me know if it's ok for you ? thanks | 14:47 |
| sean-k-mooney | yep that shoudl be fine | 14:49 |
| sean-k-mooney | if your going to attend the PTG next week you could also add it to https://etherpad.opendev.org/p/nova-yoga-ptg | 14:50 |
| sean-k-mooney | otherwise we will likely cover it tuseday week as i expect next weeks meeing is cancled for the ptg | 14:50 |
| gmann | gibi: ack, as i have removed the use of tox_extra_args now but yes in future if we use then this is something to note down. | 14:53 |
| alecorps | thanks Sean, not sure for ptg, i will try | 14:54 |
| opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Move the implemented specs for the xena release https://review.opendev.org/c/openstack/nova-specs/+/812248 | 15:38 |
| bauzas | gibi: stephenfin (if you're around) : approval for the xena implemented specs would be appreciated ^ | 15:39 |
| bauzas | i just rebased my own change into brinzhang's change as he was missing a few bits | 15:39 |
| stephenfin | bauzas: done | 15:57 |
| bauzas | stephenfin: ta | 15:58 |
| bauzas | sean-k-mooney: I moved your PTG point about rbac for novaclient during the rbac popup team meeting session we plan | 15:59 |
| bauzas | but we can put it off this slot and just discuss between us | 15:59 |
| sean-k-mooney | ok | 15:59 |
| bauzas | or we can leave it there during the meeting and we don't have time to go thru it, we can try to find another time | 16:00 |
| sean-k-mooney | well basically i think we shoudl be deprecating the novaclinet cli so i dont think we shoudl add the project id passthough feature to novaclinet for rbac | 16:00 |
| bauzas | sean-k-mooney: sorry, I messed up your colors by pasting, btw. | 16:00 |
| sean-k-mooney | its fine | 16:00 |
| sean-k-mooney | move things as you see fit | 16:00 |
| bauzas | I'm just doing a first round | 16:04 |
| bauzas | probably something like "paperwork nova", then "general nova", then others | 16:05 |
| * bauzas stops for the day, see ya | 16:16 | |
| opendevreview | Merged openstack/nova-specs master: Move the implemented specs for the xena release https://review.opendev.org/c/openstack/nova-specs/+/812248 | 16:22 |
| stephenfin | melwitt: good spot on https://review.opendev.org/c/openstack/nova/+/812144 | 16:26 |
| stephenfin | dansmith: So nova-compute doesn't access the DB and iirc, you're expected to upgrade all services at once, right? i.e. the N and N-1 in a single deployment only applies to nova-compute N-1 | 16:31 |
| sean-k-mooney | stephenfin: nova-compute only accesses the db via the conductor | 16:31 |
| dansmith | stephenfin: you're expected to upgrade all the non-compute services at once, yes, largely because of schema but not only for that | 16:31 |
| stephenfin | Okay, so that being the case, why do we insist that DB columns are removed in a later cycle than the corresponding SQLAlchemy model fields? | 16:32 |
| opendevreview | Merged openstack/nova-specs master: Re-propose Unified Limits in Nova https://review.opendev.org/c/openstack/nova-specs/+/809020 | 16:32 |
| dansmith | so you can apply new schema before you roll any of that new code | 16:32 |
| stephenfin | ah | 16:32 |
| * stephenfin puts that in the docs | 16:33 | |
| dansmith | schema apply being potentially very expensive, rewriting tables, etc | 16:33 |
| dansmith | it's in the upgrade doc somewhere, I just linked it the other day | 16:33 |
| stephenfin | yup, and having to come first for the additive stuff | 16:33 |
| dansmith | ideally if you've rolled the schema, then "upgrading" is just starting new containers, which can be pretty quick | 16:33 |
| gibi | gmann: ack, I will file a bug to zuul when I have time about the quoting issue | 16:37 |
| sean-k-mooney | provieded we have only made aditive changes in principal for something like an FFU we can fully upgrade the db schema while running n-3 compute nodes and then do all the online migration when we bounce the containers. | 16:37 |
| sean-k-mooney | the queens to train ffu in oo however stop on each release to do the db sync for some reaons | 16:38 |
| sean-k-mooney | so i dont think they have ever actuly done that in practice where tehy did the db sync rom the targent n release while the n-3 contianer where running | 16:39 |
| melwitt | stephenfin: thanks for the detailed explanation about the auto-generation stuff! | 16:40 |
| stephenfin | nw, it's *very* cool, if you ask me | 16:40 |
| stephenfin | zzzeek++ | 16:40 |
| gibi | sean-k-mooney: I manage to make a stable local reproduction for the unshelve func test bug https://bugs.launchpad.net/nova/+bug/1946339/comments/6 | 16:54 |
| gibi | now I just have to debug it to find the leaking global state between the tests | 16:55 |
| sean-k-mooney | that great | 16:58 |
| opendevreview | sean mooney proposed openstack/nova master: [WIP] adress intermitent failure of functional tests https://review.opendev.org/c/openstack/nova/+/813695 | 16:58 |
| sean-k-mooney | i just fixed the pep8 issues with ^ | 16:58 |
| sean-k-mooney | gibi: also thanks for the devstack tip it fixed my placment issue | 16:59 |
| sean-k-mooney | gibi: could it be form this | 17:02 |
| sean-k-mooney | https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/impl_fake.py#L148-L159 | 17:02 |
| sean-k-mooney | gibi: could we be reusing the same exchange between tests | 17:03 |
| sean-k-mooney | if we initalise the fake messaign drver without passing a unique exchange name per test | 17:04 |
| sean-k-mooney | gibi: tox allows use to group test per class correct | 17:04 |
| sean-k-mooney | coudl you try that with your reopducecer and see it that helps | 17:04 |
| sean-k-mooney | gibi: we do try and cleanup the exchanges https://github.com/openstack/nova/blob/7b063e4d0518af3e57872bc0288a94edcd33c19d/nova/tests/fixtures/nova.py#L741-L744 | 17:11 |
| sean-k-mooney | but that does not reset self._default_exchange | 17:13 |
| sean-k-mooney | i guess that is not required | 17:13 |
| sean-k-mooney | self._exchanges.setdefault(name, FakeExchange(name)) shoudl still be a new exchange | 17:14 |
| sean-k-mooney | gibi: just looking at test_description_errors | 17:52 |
| sean-k-mooney | the create_server funciton its calling i think is expected to wait for it to be active | 17:52 |
| sean-k-mooney | oh | 17:52 |
| sean-k-mooney | it not using the one form the integrated helper | 17:53 |
| sean-k-mooney | so ya your right its not waiting | 17:53 |
| gibi | sean-k-mooney: sorry I had to go offline | 17:58 |
| gibi | will read back tomorrow | 17:58 |
| opendevreview | Julia Kreger proposed openstack/nova master: WIP Ironic - Handle instance host on rebalance https://review.opendev.org/c/openstack/nova/+/813897 | 21:36 |
| melwitt | stephenfin: I dunno if you noticed this too but the arm64 non-voting jobs started failing often recently https://zuul.openstack.org/builds?job_name=openstack-tox-py38-arm64&job_name=openstack-tox-py39-arm64+%28non-voting%29&project=openstack%2Fnova and the timing coincided with when a few of the db-related test patches landed. is there any chance it's related? | 21:53 |
| melwitt | this one in particular https://review.opendev.org/c/openstack/nova/+/810291 | 21:54 |
| melwitt | the other two that merged at the same time were https://review.opendev.org/c/openstack/nova/+/810856 and https://review.opendev.org/c/openstack/nova/+/810857 which I thought aren't likely to be related... linking them too just in case | 21:55 |
| -opendevstatus- NOTICE: Both Gerrit and Zuul services are being restarted briefly for minor updates, and should return to service momentarily; all previously running builds will be reenqueued once Zuul is fully started again | 22:50 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!