opendevreview | melanie witt proposed openstack/nova master: imagebackend: Cache the key manager when disk is encrypted https://review.opendev.org/c/openstack/nova/+/826756 | 00:21 |
---|---|---|
opendevreview | melanie witt proposed openstack/nova master: libvirt: Introduce support for qcow2 with LUKS https://review.opendev.org/c/openstack/nova/+/772273 | 00:21 |
opendevreview | wangkuntian proposed openstack/nova master: Modify the command of getting serial console https://review.opendev.org/c/openstack/nova/+/860892 | 03:15 |
*** dasm is now known as dasm|off | 04:20 | |
gibi | o/ | 07:34 |
opendevreview | Sahid Orentino Ferdjaoui proposed openstack/nova-specs master: spec: allowing target state for evacuate https://review.opendev.org/c/openstack/nova-specs/+/857838 | 07:49 |
opendevreview | Sahid Orentino Ferdjaoui proposed openstack/nova-specs master: template: add antelope template https://review.opendev.org/c/openstack/nova-specs/+/860925 | 07:49 |
*** kopecmartin|sick is now known as kopecmartin | 08:08 | |
elodilles | JayF frickler : nova has a topic raised on PTG etherpad about EOL'ing old branches up to at least stein (L163 @ https://etherpad.opendev.org/p/nova-antelope-ptg ) | 08:09 |
opendevreview | Amit Uniyal proposed openstack/nova stable/victoria: For evacuation, ignore if task_state is not None https://review.opendev.org/c/openstack/nova/+/854980 | 09:26 |
opendevreview | Jorge San Emeterio proposed openstack/nova master: Improving logging at '_numa_cells_support_network_metadata'. https://review.opendev.org/c/openstack/nova/+/860930 | 09:33 |
frickler | elodilles: that's a good start, thx for the pointer | 09:39 |
opendevreview | Amit Uniyal proposed openstack/nova stable/victoria: For evacuation, ignore if task_state is not None https://review.opendev.org/c/openstack/nova/+/854980 | 09:50 |
opendevreview | Rajesh Tailor proposed openstack/nova master: Correct wrong reference in docs https://review.opendev.org/c/openstack/nova/+/860936 | 10:24 |
frickler | is fake nova compute testing with devstack still relevant? /me wonders whether to close this bug as obsolete https://bugs.launchpad.net/devstack/+bug/1822908 | 10:46 |
sean-k-mooney | frickler: i know its sometimes used for scale testign but honestly i dont think scale testing and devstack are the right answer | 10:51 |
sean-k-mooney | you woudl be better off using kolla-ansible or one of the other produciton installer that support the fake driver | 10:51 |
sean-k-mooney | im not sure that this is a but by the way when you increase the number of fake compute service you also need to increase the number of fake neutron agents | 10:52 |
sean-k-mooney | it sound like they failed to increease both | 10:52 |
opendevreview | Rajesh Tailor proposed openstack/nova master: Handle InstanceExists exception for duplicate instance https://review.opendev.org/c/openstack/nova/+/860938 | 10:53 |
sean-k-mooney | given matt does not work on openstack anymore they probably wont midn if you close it | 10:53 |
sean-k-mooney | frickler: with that said i dont see a equivlaent for neutron in devstack | 10:55 |
sean-k-mooney | https://github.com/openstack/devstack/blob/a64c843052529cf5c8ac8cf8eb46c59b103b4b47/doc/source/guides/nova.rst | 10:55 |
sean-k-mooney | """https://github.com/openstack/devstack/blob/a64c843052529cf5c8ac8cf8eb46c59b103b4b47/doc/source/guides/nova.rst""" | 10:56 |
sean-k-mooney | kolla take a slightly differnt apptoch https://github.com/openstack/kolla-ansible/blob/a52cf61b2234d2f078dd2893dd37de63e20ea1aa/ansible/roles/neutron/tasks/config-neutron-fake.yml | 10:57 |
sean-k-mooney | running a l2 agent per fake nova compute | 10:57 |
frickler | sean-k-mooney: do you think that then dropping the fake driver support from devstack would be appropriate? the only reference I found outside devstack is https://opendev.org/openstack/nova/src/branch/master/doc/source/contributor/testing/eventlet-profiling.rst#L84 | 11:06 |
frickler | although it might still be useful while developing patches related to it? | 11:07 |
sean-k-mooney | ya not really sure i have never personally fond it that useful | 11:12 |
sean-k-mooney | when evern i do multi node testing i just use libvirt | 11:12 |
sean-k-mooney | i can eaislly run 3 nodes on my laptop | 11:12 |
frickler | hmm, o.k., guess I'll just close the bug as wontfix for now and wait whether that topic comes up again some time | 11:18 |
sean-k-mooney | frickler: we use the fake driver in our funtional tests by the way | 11:19 |
sean-k-mooney | so we do have other ways its tested | 11:20 |
sean-k-mooney | closing it should be ok | 11:20 |
*** dasm|off is now known as dasm | 12:41 | |
opendevreview | Sahid Orentino Ferdjaoui proposed openstack/nova master: compute: enhance compute evacuate instance to support target state https://review.opendev.org/c/openstack/nova/+/858383 | 12:58 |
opendevreview | Sahid Orentino Ferdjaoui proposed openstack/nova master: api: extend evacuate instance to support target state https://review.opendev.org/c/openstack/nova/+/858384 | 12:58 |
opendevreview | Merged openstack/nova stable/victoria: For evacuation, ignore if task_state is not None https://review.opendev.org/c/openstack/nova/+/854980 | 13:06 |
opendevreview | Jorge San Emeterio proposed openstack/nova master: Improving logging at '_numa_cells_support_network_metadata'. https://review.opendev.org/c/openstack/nova/+/860930 | 13:27 |
clarkb | frickler: sean-k-mooney the fake driver was to test thousands of boots all at once | 14:59 |
clarkb | and it caught a number of issues. | 14:59 |
clarkb | (I don't know if it is valuable in devstack anymore, just wanted to call out what its utility was) | 15:00 |
sean-k-mooney | it was used mainly for rally testign to test multipel nodes | 15:03 |
sean-k-mooney | the default config of the fake driver cant supprot 1000s of vms on one host | 15:03 |
sean-k-mooney | im not saying it does not have value we just dont use it often | 15:03 |
sean-k-mooney | we can add it to the ptg adgenda to get more input | 15:04 |
bauzas | reminder : nova meeting in 42 mins here | 15:18 |
bauzas | #startmeeting nova | 16:00 |
opendevmeet | Meeting started Tue Oct 11 16:00:43 2022 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
opendevmeet | The meeting name has been set to 'nova' | 16:00 |
bauzas | heyho | 16:00 |
elodilles | o/ | 16:00 |
bauzas | who's around ? | 16:01 |
bauzas | we can start if needed | 16:02 |
bauzas | hopefully, should be a quick one, as we have the PTG next week | 16:02 |
elodilles | ++ | 16:02 |
bauzas | #topic Bugs (stuck/critical) | 16:02 |
bauzas | #info No Critical bug | 16:02 |
bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 4 new untriaged bugs (-1 since the last meeting) | 16:03 |
gibi | o/ | 16:03 |
dansmith | o/ | 16:03 |
bauzas | #link https://storyboard.openstack.org/#!/project/openstack/placement 26 open stories (+0 since the last meeting) in Storyboard for Placement | 16:03 |
bauzas | elodilles: thanks for having looked at the bugs | 16:03 |
elodilles | np | 16:03 |
bauzas | anything you would want to discuss ? | 16:03 |
bauzas | #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster | 16:03 |
elodilles | maybe one bug | 16:03 |
elodilles | (or two) | 16:03 |
Uggla | */ | 16:04 |
elodilles | related to the well-know volume timeout failure | 16:04 |
elodilles | a new general bug was open: https://bugs.launchpad.net/nova/+bug/1992328 | 16:04 |
elodilles | i know that there are a ton of similar bugs already | 16:04 |
elodilles | some are more specific and some are general | 16:05 |
bauzas | elodilles: do you want to close it as a duplicate ? | 16:05 |
elodilles | if we have the exact same somewhere, then we could | 16:05 |
elodilles | otherwise we can keep it open... | 16:05 |
elodilles | also saw that maybe somewhat coupled with this 30 days old bug: https://bugs.launchpad.net/nova/+bug/1989232 | 16:06 |
elodilles | i mean this is also opened around the same issue | 16:06 |
elodilles | otherwise i didn't have time to go deep into them | 16:07 |
bauzas | well, I dunno | 16:08 |
elodilles | so any idea about what to do with these bugs are welcome | 16:08 |
gibi | these need to be trobule shooted to find the root cause | 16:08 |
gibi | but I don't have time for that | 16:08 |
gibi | timeout is a the visible fault but there is deeper reasons why that happnes | 16:09 |
bauzas | ok, so let's continue to have this bug report then | 16:09 |
bauzas | and if someone wants to fix it, he/she could duplicate it if needed | 16:09 |
elodilles | ack | 16:09 |
bauzas | ok, any other bug report to look at ? | 16:10 |
elodilles | nothing else from me | 16:11 |
bauzas | ok, continuing then | 16:12 |
bauzas | gibi: can you be the bug baton for next week ? | 16:13 |
gibi | lets see | 16:13 |
bauzas | even if next week it will be the PTG ? | 16:13 |
gibi | I will take it | 16:13 |
bauzas | ok | 16:13 |
bauzas | thanks | 16:13 |
bauzas | #info bug baton is being passed to gibi | 16:13 |
bauzas | (sorry, passing it to you, as I was having the baton for 2 weeks :) ) | 16:13 |
bauzas | moving on | 16:14 |
bauzas | #topic Gate status | 16:14 |
bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:14 |
bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status | 16:14 |
bauzas | heh :) | 16:14 |
bauzas | as you see, we have a timeout with the centos9-fips job | 16:14 |
bauzas | but I dunno for how long it was having this timeout | 16:15 |
elodilles | https://zuul.openstack.org/builds?job_name=tempest-centos9-stream-fips&project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly&skip=0 | 16:16 |
elodilles | it seems always? | 16:16 |
bauzas | yes | 16:16 |
bauzas | I guess the owner of the job was adalee, right?. | 16:17 |
bauzas | this is bizarre, I don't see where the job run is timing out | 16:18 |
sean-k-mooney | well the default timeout i think is 2 hours | 16:20 |
sean-k-mooney | so it might just need a little longer sicne the fips job does a reboot | 16:20 |
bauzas | yeah but the job seems to have done | 16:20 |
bauzas | be* done | 16:20 |
bauzas | anyway, nothing urgent | 16:20 |
bauzas | this is just the fact we can't get the result | 16:20 |
bauzas | even if it looks tempest works fine | 16:21 |
sean-k-mooney | its posible it has addtional test in a post playbook | 16:21 |
bauzas | I don't know who wrote the job | 16:21 |
bauzas | but I'll try to find it | 16:21 |
bauzas | should be easy to find | 16:21 |
clarkb | https://zuul.openstack.org/build/5f6e6a2f65ee4a5e90754d07142fab9f/log/job-output.txt#23945-23946 shows where it is timing out | 16:22 |
bauzas | found who added it https://review.opendev.org/c/openstack/nova/+/831844 | 16:23 |
bauzas | thanks clarkb, will look | 16:23 |
sean-k-mooney | ok its tempest full | 16:24 |
sean-k-mooney | so its two tempest runs | 16:24 |
bauzas | | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tempest/playbooks/devstack-tempest.yaml@master] | 16:25 |
sean-k-mooney | the first one completed but the second that runs the slow test and senarior tests timed out | 16:25 |
bauzas | hmpf | 16:25 |
bauzas | yup | 16:25 |
sean-k-mooney | so ya it just need anther say 30 mins added to the tiemout to be safe | 16:25 |
sean-k-mooney | it also could have just been a slow node | 16:26 |
clarkb | and maybe double check logs to see that something didn't get stuck there due to fips | 16:26 |
clarkb | there is a bit of a time delta between the timeout and tempest reporting anything | 16:26 |
clarkb | oh its only 15 seconds nevermind | 16:26 |
sean-k-mooney | yep | 16:26 |
sean-k-mooney | there isnt really a break in the logs | 16:26 |
clarkb | I mathed wrong the first time | 16:26 |
bauzas | yes, this is just a slow test | 16:27 |
bauzas | but should we modify the timeout for that job to be larger ? | 16:27 |
sean-k-mooney | i dont know how much time the reboot adds for fips but 2 hours is proably borderlien | 16:27 |
bauzas | we can DNM this at least | 16:28 |
bauzas | and verify if adding more timeout time helps | 16:28 |
sean-k-mooney | i would add 30mins and monitor it and see roughly how long it takes over the next few weeks | 16:28 |
bauzas | right | 16:28 |
bauzas | I can propose a patch | 16:28 |
sean-k-mooney | ack | 16:28 |
bauzas | #action bauzas to add a 30min more timeout for the centos9-fips periodic job so we will see whether it fixes the timeout | 16:29 |
bauzas | moving on | 16:29 |
bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:29 |
sean-k-mooney | bauzas: its here https://opendev.org/openstack/tempest/src/branch/master/zuul.d/integrated-gate.yaml#L325 just incase your wondering where to do that | 16:29 |
bauzas | ack, thanks but I found it | 16:31 |
bauzas | hence the DNM patch in nova | 16:31 |
bauzas | to test it | 16:31 |
bauzas | with experimental | 16:32 |
bauzas | anyway | 16:32 |
bauzas | nothing to tell more about blind rechecks, I haven't looked at the numbers | 16:32 |
bauzas | moving on | 16:32 |
bauzas | #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures | 16:32 |
bauzas | #topic Release Planning | 16:32 |
bauzas | #link https://releases.openstack.org/zed/schedule.html | 16:32 |
bauzas | #info Zed GA was last week, kudos to the team. | 16:33 |
bauzas | we're officially done with Zed :) | 16:33 |
bauzas | so... | 16:33 |
bauzas | #link https://releases.openstack.org/antelope/schedule.html | 16:33 |
bauzas | #info Antelope-1 is planned in 5 weeks | 16:33 |
bauzas | nothing to tell it, the crux is for the next topic | 16:33 |
bauzas | #topic PTG planning | 16:33 |
bauzas | #link https://etherpad.opendev.org/p/nova-antelope-ptg Antelope PTG etherpad | 16:33 |
bauzas | #link https://ptg.opendev.org/ptg.html PTG schedule | 16:34 |
bauzas | #info Attempted schedule in the etherpad | 16:34 |
bauzas | yeah I did a bit of a topics scrambling | 16:34 |
bauzas | but honestly, we don't have a large agendat *yet* | 16:34 |
bauzas | I'll just move the nova-otherproject topics | 16:35 |
bauzas | for the moment, we have nova-neutron (Thur) and nova-ironic (reminder : *Monday*) | 16:35 |
bauzas | do people want to talk with cinder or other projects ? | 16:35 |
bauzas | I haven't got any ask from the cyborg project | 16:35 |
bauzas | or oslo or QA ? | 16:36 |
bauzas | or manila, or glance... :) | 16:36 |
gibi | nothing on my side | 16:36 |
sean-k-mooney | i think the ironic topic will fill any space we give it | 16:37 |
sean-k-mooney | but im not sure if we will be able to make progress remotely | 16:37 |
bauzas | hah | 16:37 |
bauzas | sean-k-mooney: as I already said a couple of times, I don't think we will find a solution | 16:38 |
bauzas | but, | 16:38 |
bauzas | at least I want our community to understand why we can't just modify an instance .host DB value | 16:38 |
sean-k-mooney | we also shoudl not have the hashring in the driver as we do today | 16:39 |
bauzas | if we're able to explain it even after all the comments dansmith and you did, well, it will be nice then | 16:39 |
sean-k-mooney | i look forward to your summary | 16:40 |
bauzas | anyway, I don't want to discuss this spec by now, we'll have time on Monday :) | 16:40 |
* sean-k-mooney has flushed most of that out of my brain but will re read the spec | 16:40 | |
bauzas | sean-k-mooney: /o\ | 16:40 |
bauzas | but yeah, I'll summarize this session after the PTG :) | 16:40 |
bauzas | moving on | 16:40 |
bauzas | looks we can find other projects to discuss if we want during the PTG anyxway | 16:41 |
bauzas | as a reminder too, we could get pain points or feature requests from operators | 16:41 |
bauzas | #link https://etherpad.opendev.org/p/oct2022-ptg-operator-hour-nova Operator specific etherpad for our nova operator hours | 16:41 |
sean-k-mooney | i hope we do. even if they are ones we have gotten before | 16:41 |
bauzas | #topic Review priorities | 16:42 |
bauzas | #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+(label:Review-Priority%252B1+OR+label:Review-Priority%252B2) | 16:42 |
bauzas | I need to add some changes I have as a review-prio | 16:42 |
bauzas | apart from this, nothing to tell | 16:43 |
bauzas | next topic ? | 16:43 |
bauzas | (we'll discuss the review-prio flag at the PTG anyway) | 16:43 |
bauzas | #topic Stable Branches | 16:43 |
bauzas | elodilles: your dog. | 16:43 |
elodilles | o/ | 16:43 |
elodilles | #info from stable/zed back till stable/ussuri branches' gates should be OK | 16:43 |
elodilles | #info stable/train seems broken again: devstack-gate's test-matrix playbook fails ("No module named yaml") | 16:44 |
elodilles | and it seems we already have a workaround for this: https://review.opendev.org/c/openstack/devstack-gate/+/860961 | 16:44 |
elodilles | and for some information this came with Ansible 6 | 16:44 |
elodilles | the rest are the same: | 16:45 |
elodilles | #info stable/stein (and older) are blocked: grenade and other devstack based jobs fail with the same timeout issue as stable/train was previously | 16:45 |
elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | 16:45 |
elodilles | and that's all | 16:45 |
bauzas | cool thanks | 16:45 |
bauzas | can't wait for next week for discussing about stein :) | 16:46 |
elodilles | stein and older :) | 16:46 |
bauzas | oh you're right btw. | 16:46 |
bauzas | we can now backport again \o/ | 16:46 |
bauzas | given Zed is released | 16:46 |
bauzas | huzzah | 16:46 |
elodilles | \o/ | 16:46 |
gibi | :) | 16:47 |
bauzas | ok, anything to add ? | 16:47 |
bauzas | looks not | 16:47 |
bauzas | #topic Open discussion | 16:47 |
bauzas | just two paperwork items on my side | 16:47 |
bauzas | (bauzas) Cancel next week's meeting due to PTG | 16:47 |
bauzas | I forgot the question mark, but is this really needed ? | 16:48 |
bauzas | anyone having problems about cancelling ? | 16:48 |
bauzas | like, I can only attend by IRC, zoom is terrible to me | 16:48 |
bauzas | or, I can show my face, I just prefer us to continuously discussing over IRC | 16:49 |
bauzas | looks not | 16:49 |
gibi | no need for a meeting next week | 16:49 |
gibi | on IRC | 16:50 |
bauzas | #action bauzas to communicate (email and wiki) on Nov 18 nova meeting be CANCELLED | 16:50 |
bauzas | last item from me | 16:50 |
bauzas | (bauzas) Nov-1 is an holiday here, anyone wants to run the meetingĀ ? | 16:50 |
bauzas | all said ^ | 16:50 |
bauzas | I'm usually on perpetual PTO, but this time I'm asked to be on holiday | 16:51 |
bauzas | I guess most of the Europe is also having holiday at that day, if I trust my company calendar | 16:51 |
bauzas | shall we just cancel it too ? | 16:51 |
bauzas | the fun is, Nov-8 is ALSO an holiday for me the week after | 16:52 |
gibi | I'm of on Nov-1 too | 16:52 |
gibi | off | 16:52 |
elodilles | same for me | 16:52 |
gibi | cancel it | 16:52 |
bauzas | yeah, was just opening the door to anyone fancy running the meeting if he/she wanted | 16:52 |
gibi | I don't see much hands in the air | 16:53 |
gibi | :) | 16:53 |
dansmith | ++ cancel | 16:53 |
bauzas | gibi: on holiday nova-8 as well or not ? | 16:53 |
bauzas | shit | 16:53 |
bauzas | nov-8 | 16:53 |
bauzas | oh nevermind | 16:53 |
bauzas | I'm stupid | 16:54 |
bauzas | I'm not good at knowing our holidays | 16:54 |
bauzas | I confused myself with may | 16:54 |
bauzas | nov-8 is a workday, that's nov-11 which is holiday | 16:54 |
* bauzas feels ashamed | 16:54 | |
bauzas | anyway, sold. | 16:54 |
gibi | I will be in Brno on nov-8 | 16:54 |
gibi | so working but not on upstream nova :) | 16:55 |
bauzas | #undo | 16:55 |
opendevmeet | Removing item from minutes: #action bauzas to communicate (email and wiki) on Nov 18 nova meeting be CANCELLED | 16:55 |
gibi | Nova 18? | 16:55 |
bauzas | #action bauzas to communicate (email and wiki) on Oct 18 nova meeting be CANCELLED | 16:55 |
gibi | Nov 18? | 16:55 |
bauzas | #action bauzas to communicate (email and wiki) on Nov 1st nova meeting be CANCELLED | 16:55 |
gibi | getting better :) | 16:55 |
bauzas | I realized | 16:56 |
elodilles | and Oct 18 | 16:56 |
gibi | oct 18 and nov 1 ACK | 16:56 |
elodilles | sorry, there it is | 16:56 |
elodilles | :X | 16:56 |
bauzas | after this and my confusion on holiday dates, that means it's time for me to end the meeting | 16:56 |
gibi | calendars are hard :D | 16:56 |
bauzas | thanks all and kudos for the hard work you made on Zed, btw. | 16:57 |
bauzas | #endmeeting | 16:57 |
opendevmeet | Meeting ended Tue Oct 11 16:57:20 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:57 |
opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2022/nova.2022-10-11-16.00.html | 16:57 |
opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2022/nova.2022-10-11-16.00.txt | 16:57 |
opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2022/nova.2022-10-11-16.00.log.html | 16:57 |
elodilles | thanks bauzas o/ | 16:57 |
bauzas | looks I'm tired | 16:58 |
elodilles | then it's time to relax with a good french wine? | 16:59 |
gibi | o/ | 17:00 |
bauzas | elodilles: I rather need a beer | 17:02 |
elodilles | i guess that's also an option :) | 17:03 |
opendevreview | Dan Smith proposed openstack/nova master: DNM: Test ceph-multistore with a real image https://review.opendev.org/c/openstack/nova/+/860864 | 18:58 |
opendevreview | Dan Smith proposed openstack/nova master: DNM: Test ceph-multistore with a real image https://review.opendev.org/c/openstack/nova/+/860864 | 21:08 |
atmark | does the nova compute need {cpu,disk,ram}_weight_multiplier option too or just nova scheduler? | 21:53 |
opendevreview | Dan Smith proposed openstack/nova master: DNM: Test ceph-multistore with a real image https://review.opendev.org/c/openstack/nova/+/860864 | 22:01 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!