| gibi | sean-k-mooney: as we really like shelve/unshelve to clean up messy situations this bugfix probably very useful https://review.opendev.org/q/topic:bug%252F2025480 | 10:01 |
|---|---|---|
| kevko | Hi everyone, i would like to ask if someone can approve this blueprint proposed by my colleaguage Viktor Krivak, we already have a code implemented but waiting for specs approval , so we can send patches to gerrit...could someone look into please ? https://review.opendev.org/c/openstack/nova-specs/+/890779 , thank you very much | 10:12 |
| kevko | ^^ it's solution for az-affinity and anti-az-affinity which can be also used by octavia to reach better high-availability as amphoras will be spread across AZs | 10:13 |
| gibi | kevko: we reached nova spec freeze for 2023.2 (bobcat) on 6th of July https://releases.openstack.org/bobcat/schedule.html#b-nova-spec-freeze | 10:14 |
| gibi | this is why you don't get attention on the spec | 10:14 |
| kevko | hmm, does it mean that this can be approved for caracal only right ? | 10:14 |
| gibi | kevko: yes | 10:15 |
| gibi | kevko: you can see the caracal schedule here https://releases.openstack.org/caracal/schedule.html | 10:15 |
| kevko | hmm, can we push patchsets for this to gerrit and mark as DNM for example, we would like to see zull results etc ... | 10:16 |
| gibi | kevko: yes, you can do that | 10:16 |
| kevko | (because i read somewhere that it's "forbidden" to push the code if spec is not approved ... so i am better asking ) | 10:16 |
| gibi | kevko: that will help the spec reviewers as well to see not just the spec proposal but the resulting implementation | 10:16 |
| gibi | kevko: if you can point me to the place where it was written then I will be happy to fix that | 10:17 |
| kevko | let me find it | 10:18 |
| kevko | https://github.com/openstack/nova-specs/blob/master/specs/2023.2-template.rst | 10:35 |
| kevko | Gibi, so question is when we can except that new directory for caracal release will be available for specs repo to move spec for caracal ? | 10:39 |
| kevko | Gibi, can we create directory for 2024.1 specs now ? | 10:45 |
| gibi | kevko: which part of the spec template suggested that you should not push code for review? | 11:36 |
| gibi | I see | 11:36 |
| gibi | "You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged." | 11:37 |
| kevko | yep ..maybe misunderstood .. | 11:37 |
| gibi | kevko: it is basically a warning that we might ask you during the spec review to change the direction of the spec and therefore you might need to rewrite the code | 11:39 |
| gibi | on the directory question. Technically we could propose the directory right now | 11:40 |
| gibi | kevko: you can propose a similar patch to https://review.opendev.org/c/openstack/nova-specs/+/872068 to create the 2024.1 directory | 11:46 |
| kevko | gibi Thank you, I really appreciate it :), then if the spec will be in right directory, can we expect that someone will check the spec even if now it's bobcat cycle ? or we will wait until december ? | 11:49 |
| opendevreview | Michal Arbet proposed openstack/nova-specs master: Create specs directory for 2024.1 Caracal https://review.opendev.org/c/openstack/nova-specs/+/891468 | 12:17 |
| opendevreview | Michal Arbet proposed openstack/nova-specs master: Add availability zone-aware affinity filter to the scheduler https://review.opendev.org/c/openstack/nova-specs/+/890779 | 12:20 |
| gibi | kevko: to be honest right now is vacation period, then we will be busy finishing the content for the bobcat release. So I would not expect much review on caracal spec until the bobcat release is done in early october. | 12:32 |
| kevko | gibi understand, that's apity | 12:46 |
| kevko | a pity | 12:46 |
| sean-k-mooney | kevko: i would expect it to get attention in mid september | 13:11 |
| sean-k-mooney | it wont be december | 13:11 |
| sean-k-mooney | but we will be busy with release work for then ext 3-4 weeks and thats where most of the attention will be focused until rc1 is released | 13:11 |
| sean-k-mooney | gibi: like i s a strong statement for using shelve to recover but it works surpising well in may siturations | 13:13 |
| kevko | sean-k-mooney: well, it would be nice if you just read it and leave some comment ...but on the other side ...i understand | 13:19 |
| kevko | point is that my colleguage as implementator can work on it ...and it will not work on it until some comment will be left in review :D | 13:20 |
| sean-k-mooney | i can try and take a look in general i dont think we can do what you want however | 13:22 |
| sean-k-mooney | kevko: we cant do az affintiy or anti affintiy at the sechduelr filter level without some majoyr chages as the az is already select before the scheduler runs | 13:22 |
| sean-k-mooney | doing this in the filters has been rejected in the past on 2 ocations for that very reason | 13:23 |
| sean-k-mooney | the az is set in the build request in the api before the rpc to the schduler | 13:23 |
| sean-k-mooney | so unless the spec adresses that it wont work | 13:23 |
| opendevreview | Bence Romsics proposed openstack/nova master: Do not untrack resources of a server being unshelved https://review.opendev.org/c/openstack/nova/+/890406 | 13:29 |
| opendevreview | Dan Smith proposed openstack/nova master: Log excessive lazy-loading behavior https://review.opendev.org/c/openstack/nova/+/891340 | 14:21 |
| opendevreview | Dan Smith proposed openstack/nova master: Avoid lazy-loading in resize and rebuild/evacuate https://review.opendev.org/c/openstack/nova/+/891336 | 14:21 |
| opendevreview | Dan Smith proposed openstack/nova master: Remove a lazy load on every server show https://review.opendev.org/c/openstack/nova/+/891352 | 14:21 |
| gibi | nova meeting will start in 30 mins here. I will be your host stoday in bauzas' absense and I will not try to do a full meeting if not needed :) | 15:31 |
| gibi | #startmeeting nova | 16:00 |
| opendevmeet | Meeting started Tue Aug 15 16:00:41 2023 UTC and is due to finish in 60 minutes. The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'nova' | 16:00 |
| gibi | o/ | 16:00 |
| dansmith | o/ | 16:00 |
| elodilles | o/ | 16:01 |
| gibi | OK, lets get started | 16:02 |
| gibi | this will be a sort meeting | 16:02 |
| gibi | #topic Bugs (stuck/critical) | 16:02 |
| gibi | I don't see any critical bugs | 16:02 |
| gibi | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 40 new untriaged bugs (+0 since the last meeting) | 16:03 |
| gibi | based on the roster elodilles had the baton for a long time | 16:03 |
| elodilles | yepp o:) i've tried to triage some bugs :) | 16:04 |
| gibi | thanks | 16:04 |
| elodilles | np | 16:04 |
| gibi | I think we can let that baton be on PTO until bauzas is back | 16:04 |
| elodilles | ++ | 16:04 |
| gibi | at least I don't know where to assign it | 16:05 |
| gibi | so | 16:05 |
| gibi | #info bug baton is PTO | 16:05 |
| gibi | #info bug baton is on PTO | 16:05 |
| gibi | do we have any bugs that needs discussion? | 16:05 |
| gmann | o/ | 16:06 |
| elodilles | i don't have any to mention (though feel free to look for the bugs if i misjudged any of them) | 16:06 |
| gibi | #topic Gate status | 16:07 |
| dansmith | hoo boy | 16:07 |
| dansmith | things are definitely improving but we still have work to do | 16:07 |
| dansmith | this isn't really gate status, but one thing I've noticed while looking at countless gate logs is that we've had a bunch of lazy-loads sneak in, | 16:08 |
| dansmith | up to five per object one after another for several operations | 16:08 |
| gmann | at least less timeout and oom i think | 16:08 |
| dansmith | I started work on making them visible and have been squashing them here: https://review.opendev.org/q/topic:reduce-lazy-loads | 16:08 |
| gibi | dansmith: nice | 16:09 |
| gibi | is there any specific issue we need to discuss? | 16:09 |
| dansmith | I dunno how others feel, but I think 1 lazy load per object per service should be enough, and any time we need to lazy-load more than one thing, we probably should have pre-queried for the thing | 16:09 |
| gibi | dansmith: in general I agree | 16:09 |
| gibi | especially if we always do the two lazy loads unconditionally | 16:10 |
| dansmith | so while it's not really clearly stability-related, doing this will reduce our load on rabbit, the db, and conductor which I'm sure will help gate performance | 16:10 |
| dansmith | yeah | 16:10 |
| gibi | I agree | 16:10 |
| dansmith | the other gate failure I know of, which I haven't seen much since discussing with melwitt, was the ceph job seeming to hang n-cpu | 16:10 |
| dansmith | I think she was looking at it but I don't know if there was any outcome | 16:10 |
| gibi | OK | 16:11 |
| gibi | thanks for the update | 16:11 |
| dansmith | and then aside from that, plenty of small things but nothing large enough to be specifically noteworthy on its own that I know of | 16:11 |
| gibi | I started to look into https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_1fe/889992/2/gate/nova-tox-functional-py39/1fefb65/testr_results.html on Friday but no real breakthrough yet | 16:12 |
| dansmith | cool | 16:12 |
| gmann | one thing to note, rebuild tests have refactored which was failing more. it work on its own server now and should be less on disturbing the thing on class level server | 16:13 |
| gibi | gmann: thanks for the heads up | 16:13 |
| gmann | and py3.11 testing started, this is fix from frickler for nova https://review.opendev.org/c/openstack/nova/+/891256/2 | 16:14 |
| gmann | and in child patch we can see job is passing | 16:14 |
| gmann | idea is to add py3.11 unit test job as non voting in this cycle and make it voting in next cycle | 16:14 |
| gibi | sounds good | 16:15 |
| gibi | I approved the bindep change now as that was simple enough :) | 16:15 |
| gmann | thanks | 16:16 |
| gibi | anything else about our gate? | 16:16 |
| gmann | nothing else from me | 16:16 |
| gibi | #topic Release Planning | 16:17 |
| gibi | 3 weeks left until Feature Freeze | 16:17 |
| gibi | anything to raise here? | 16:17 |
| gibi | #topic Stable Branches | 16:18 |
| gibi | elodilles: your turn :) | 16:19 |
| elodilles | o7 | 16:19 |
| elodilles | there was not much activity on stable branches | 16:19 |
| elodilles | if the gate issues did not cause any problem on stable gates, then | 16:19 |
| elodilles | #info all stable gates should be OK | 16:19 |
| elodilles | :) | 16:19 |
| elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | 16:19 |
| elodilles | and that's all from me | 16:19 |
| gibi | thanks | 16:20 |
| gibi | others, any comment on stable? | 16:20 |
| gibi | #topic Open discussion | 16:21 |
| gibi | any other business? | 16:21 |
| dansmith | nay | 16:21 |
| sean-k-mooney | i havent really looked at the sabel brances in a a few weeks and no other business form me | 16:22 |
| elodilles | - | 16:22 |
| gibi | OK. Then thanks for joining. :) | 16:22 |
| gibi | #endmeeting | 16:22 |
| opendevmeet | Meeting ended Tue Aug 15 16:22:24 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:22 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2023/nova.2023-08-15-16.00.html | 16:22 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2023/nova.2023-08-15-16.00.txt | 16:22 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2023/nova.2023-08-15-16.00.log.html | 16:22 |
| elodilles | thanks o/ | 16:22 |
| -opendevstatus- NOTICE: Zuul job execution is temporarily paused while we rearrange local storage on the servers | 16:54 | |
| sean-k-mooney | kevko: i left comments on https://review.opendev.org/c/openstack/nova-specs/+/890779 but that approch will not work | 17:01 |
| opendevreview | Dan Smith proposed openstack/nova master: WIP: Avoid lazy-loads on server create https://review.opendev.org/c/openstack/nova/+/891520 | 17:33 |
| opendevreview | Dan Smith proposed openstack/nova master: Avoid lazy-loads on server create https://review.opendev.org/c/openstack/nova/+/891520 | 17:34 |
| -opendevstatus- NOTICE: Zuul job execution has resumed with additional disk space on the servers | 17:44 | |
| dansmith | gibi: another functional failure, but this looks different: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_240/889740/1/check/nova-tox-functional-py39/240dccc/testr_results.html | 18:12 |
| dansmith | surely seems like the 3.9 job is the one we always see trouble with... | 18:12 |
| sean-k-mooney | we coudl just test 3.8 and 3.10 :P | 18:13 |
| sean-k-mooney | oh the db coursor thing | 18:13 |
| sean-k-mooney | so 3.9 is runing on focal with a non default python i.e. that default to 3.8 | 18:15 |
| dansmith | yup | 18:15 |
| sean-k-mooney | so i dont know if there could be a bug in that interperter | 18:15 |
| dansmith | that's why I was pointing it out | 18:16 |
| sean-k-mooney | i dont see this often but we could always try using debian for this job | 18:16 |
| sean-k-mooney | of centos 9 stream | 18:16 |
| sean-k-mooney | that default to 3.9 | 18:16 |
| sean-k-mooney | we woudl jsut need to suppy a diffent nodeset but im not sure if that will help or not | 18:18 |
| sean-k-mooney | am is zuul ok at th moment | 18:19 |
| sean-k-mooney | i pushed a patch almost 2 hours ago and its still queueed to start running | 18:20 |
| dansmith | see the notices above? | 18:21 |
| sean-k-mooney | oh ok | 18:21 |
| sean-k-mooney | so there is proably a backlog | 18:22 |
| sean-k-mooney | ill check tomorrow | 18:22 |
| opendevreview | Dan Smith proposed openstack/nova master: Avoid lazy-loads on server create https://review.opendev.org/c/openstack/nova/+/891520 | 21:19 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!