Tuesday, 2022-10-11

opendevreviewmelanie witt proposed openstack/nova master: imagebackend: Cache the key manager when disk is encrypted  https://review.opendev.org/c/openstack/nova/+/82675600:21
opendevreviewmelanie witt proposed openstack/nova master: libvirt: Introduce support for qcow2 with LUKS  https://review.opendev.org/c/openstack/nova/+/77227300:21
opendevreviewwangkuntian proposed openstack/nova master: Modify the command of getting serial console  https://review.opendev.org/c/openstack/nova/+/86089203:15
*** dasm is now known as dasm|off04:20
gibio/07:34
opendevreviewSahid Orentino Ferdjaoui proposed openstack/nova-specs master: spec: allowing target state for evacuate  https://review.opendev.org/c/openstack/nova-specs/+/85783807:49
opendevreviewSahid Orentino Ferdjaoui proposed openstack/nova-specs master: template: add antelope template  https://review.opendev.org/c/openstack/nova-specs/+/86092507:49
*** kopecmartin|sick is now known as kopecmartin08:08
elodillesJayF frickler : nova has a topic raised on PTG etherpad about EOL'ing old branches up to at least stein (L163 @ https://etherpad.opendev.org/p/nova-antelope-ptg )08:09
opendevreviewAmit Uniyal proposed openstack/nova stable/victoria: For evacuation, ignore if task_state is not None  https://review.opendev.org/c/openstack/nova/+/85498009:26
opendevreviewJorge San Emeterio proposed openstack/nova master: Improving logging at '_numa_cells_support_network_metadata'.  https://review.opendev.org/c/openstack/nova/+/86093009:33
fricklerelodilles: that's a good start, thx for the pointer09:39
opendevreviewAmit Uniyal proposed openstack/nova stable/victoria: For evacuation, ignore if task_state is not None  https://review.opendev.org/c/openstack/nova/+/85498009:50
opendevreviewRajesh Tailor proposed openstack/nova master: Correct wrong reference in docs  https://review.opendev.org/c/openstack/nova/+/86093610:24
frickleris fake nova compute testing with devstack still relevant? /me wonders whether to close this bug as obsolete https://bugs.launchpad.net/devstack/+bug/182290810:46
sean-k-mooneyfrickler: i know its sometimes used for scale testign but honestly i dont think scale testing and devstack are the right answer10:51
sean-k-mooneyyou woudl be better off using kolla-ansible or one of the other produciton installer that support the fake driver10:51
sean-k-mooneyim not sure that this is a but by the way when you increase the number of fake compute service you also need to increase the number of fake neutron agents10:52
sean-k-mooneyit sound like they failed to increease both10:52
opendevreviewRajesh Tailor proposed openstack/nova master: Handle InstanceExists exception for duplicate instance  https://review.opendev.org/c/openstack/nova/+/86093810:53
sean-k-mooneygiven matt does not work on openstack anymore they probably wont midn if you close it10:53
sean-k-mooneyfrickler: with that said i dont see a equivlaent for neutron in devstack10:55
sean-k-mooneyhttps://github.com/openstack/devstack/blob/a64c843052529cf5c8ac8cf8eb46c59b103b4b47/doc/source/guides/nova.rst10:55
sean-k-mooney"""https://github.com/openstack/devstack/blob/a64c843052529cf5c8ac8cf8eb46c59b103b4b47/doc/source/guides/nova.rst"""10:56
sean-k-mooneykolla take a slightly differnt apptoch https://github.com/openstack/kolla-ansible/blob/a52cf61b2234d2f078dd2893dd37de63e20ea1aa/ansible/roles/neutron/tasks/config-neutron-fake.yml10:57
sean-k-mooneyrunning a l2 agent per fake nova compute10:57
fricklersean-k-mooney: do you think that then dropping the fake driver support from devstack would be appropriate? the only reference I found outside devstack is https://opendev.org/openstack/nova/src/branch/master/doc/source/contributor/testing/eventlet-profiling.rst#L8411:06
frickleralthough it might still be useful while developing patches related to it?11:07
sean-k-mooneyya not really sure i have never personally fond it that useful11:12
sean-k-mooneywhen evern i do multi node testing i just use libvirt11:12
sean-k-mooneyi can eaislly run 3 nodes on my laptop 11:12
fricklerhmm, o.k., guess I'll just close the bug as wontfix for now and wait whether that topic comes up again some time11:18
sean-k-mooneyfrickler: we use the fake driver in our funtional tests by the way11:19
sean-k-mooneyso we do have other ways its tested11:20
sean-k-mooneyclosing it should be ok11:20
*** dasm|off is now known as dasm12:41
opendevreviewSahid Orentino Ferdjaoui proposed openstack/nova master: compute: enhance compute evacuate instance to support target state  https://review.opendev.org/c/openstack/nova/+/85838312:58
opendevreviewSahid Orentino Ferdjaoui proposed openstack/nova master: api: extend evacuate instance to support target state  https://review.opendev.org/c/openstack/nova/+/85838412:58
opendevreviewMerged openstack/nova stable/victoria: For evacuation, ignore if task_state is not None  https://review.opendev.org/c/openstack/nova/+/85498013:06
opendevreviewJorge San Emeterio proposed openstack/nova master: Improving logging at '_numa_cells_support_network_metadata'.  https://review.opendev.org/c/openstack/nova/+/86093013:27
clarkbfrickler: sean-k-mooney the fake driver was to test thousands of boots all at once14:59
clarkband it caught a number of issues.14:59
clarkb(I don't know if it is valuable in devstack anymore, just wanted to call out what its utility was)15:00
sean-k-mooneyit was used mainly for rally testign to test multipel nodes15:03
sean-k-mooneythe default config of the fake driver cant supprot 1000s of vms on one host15:03
sean-k-mooneyim not saying it does not have value we just dont use it often15:03
sean-k-mooneywe can add it to the ptg adgenda to get more input15:04
bauzasreminder : nova meeting in 42 mins here15:18
bauzas#startmeeting nova16:00
opendevmeetMeeting started Tue Oct 11 16:00:43 2022 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
opendevmeetThe meeting name has been set to 'nova'16:00
bauzasheyho16:00
elodilleso/16:00
bauzaswho's around ?16:01
bauzaswe can start if needed16:02
bauzashopefully, should be a quick one, as we have the PTG next week16:02
elodilles++16:02
bauzas#topic Bugs (stuck/critical) 16:02
bauzas#info No Critical bug16:02
bauzas#link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 4 new untriaged bugs (-1 since the last meeting)16:03
gibio/16:03
dansmitho/16:03
bauzas#link https://storyboard.openstack.org/#!/project/openstack/placement 26 open stories (+0 since the last meeting) in Storyboard for Placement 16:03
bauzaselodilles: thanks for having looked at the bugs16:03
elodillesnp16:03
bauzasanything you would want to discuss ?16:03
bauzas#info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster16:03
elodillesmaybe one bug16:03
elodilles(or two)16:03
Uggla*/16:04
elodillesrelated to the well-know volume timeout failure16:04
elodillesa new general bug was open: https://bugs.launchpad.net/nova/+bug/199232816:04
elodillesi know that there are a ton of similar bugs already16:04
elodillessome are more specific and some are general16:05
bauzaselodilles: do you want to close it as a duplicate ?16:05
elodillesif we have the exact same somewhere, then we could16:05
elodillesotherwise we can keep it open...16:05
elodillesalso saw that maybe somewhat coupled with this 30 days old bug: https://bugs.launchpad.net/nova/+bug/198923216:06
elodillesi mean this is also opened around the same issue16:06
elodillesotherwise i didn't have time to go deep into them16:07
bauzaswell, I dunno16:08
elodillesso any idea about what to do with these bugs are welcome16:08
gibithese need to be trobule shooted to find the root cause16:08
gibibut I don't have time for that16:08
gibitimeout is a the visible fault but there is deeper reasons why that happnes16:09
bauzasok, so let's continue to have this bug report then 16:09
bauzasand if someone wants to fix it, he/she could duplicate it if needed16:09
elodillesack16:09
bauzasok, any other bug report to look at ?16:10
elodillesnothing else from me16:11
bauzasok, continuing then16:12
bauzasgibi: can you be the bug baton for next week ? 16:13
gibilets see16:13
bauzaseven if next week it will be the PTG ?16:13
gibiI will take it16:13
bauzasok16:13
bauzasthanks16:13
bauzas#info bug baton is being passed to gibi16:13
bauzas(sorry, passing it to you, as I was having the baton for 2 weeks :) )16:13
bauzasmoving on16:14
bauzas#topic Gate status 16:14
bauzas#link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs 16:14
bauzas#link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status16:14
bauzasheh :)16:14
bauzasas you see, we have a timeout with the centos9-fips job16:14
bauzasbut I dunno for how long it was having this timeout16:15
elodilleshttps://zuul.openstack.org/builds?job_name=tempest-centos9-stream-fips&project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly&skip=016:16
elodillesit seems always?16:16
bauzasyes16:16
bauzasI guess the owner of the job was adalee, right?.16:17
bauzasthis is bizarre, I don't see where the job run is timing out16:18
sean-k-mooney well the default timeout i think is 2 hours16:20
sean-k-mooneyso it might just need a little longer sicne the fips job does a reboot16:20
bauzasyeah but the job seems to have done 16:20
bauzasbe* done16:20
bauzasanyway, nothing urgent16:20
bauzasthis is just the fact we can't get the result 16:20
bauzaseven if it looks tempest works fine16:21
sean-k-mooneyits posible it has addtional test in a post playbook16:21
bauzasI don't know who wrote the job16:21
bauzasbut I'll try to find it16:21
bauzasshould be easy to find 16:21
clarkbhttps://zuul.openstack.org/build/5f6e6a2f65ee4a5e90754d07142fab9f/log/job-output.txt#23945-23946 shows where it is timing out16:22
bauzasfound who added it https://review.opendev.org/c/openstack/nova/+/83184416:23
bauzasthanks clarkb, will look16:23
sean-k-mooneyok its tempest full16:24
sean-k-mooneyso its two tempest runs16:24
bauzas| RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tempest/playbooks/devstack-tempest.yaml@master]16:25
sean-k-mooneythe first one completed but the second that runs the slow test and senarior tests timed out16:25
bauzashmpf16:25
bauzasyup16:25
sean-k-mooneyso ya it just need anther say 30 mins added to the  tiemout to be safe16:25
sean-k-mooneyit also could have just been a slow node16:26
clarkband maybe double check logs to see that something didn't get stuck there due to fips16:26
clarkbthere is a bit of a time delta between the timeout and tempest reporting anything16:26
clarkboh its only 15 seconds nevermind16:26
sean-k-mooneyyep16:26
sean-k-mooneythere isnt really a break in the logs16:26
clarkbI mathed wrong the first time16:26
bauzasyes, this is just a slow test16:27
bauzasbut should we modify the timeout for that job to be larger ?16:27
sean-k-mooneyi dont know how much time the reboot adds for fips but 2 hours is proably borderlien16:27
bauzaswe can DNM this at least16:28
bauzasand verify if adding more timeout time helps16:28
sean-k-mooneyi would add 30mins and monitor it and see roughly how long it takes over the next few weeks16:28
bauzasright16:28
bauzasI can propose a patch16:28
sean-k-mooneyack16:28
bauzas#action bauzas to add a 30min more timeout for the centos9-fips periodic job so we will see whether it fixes the timeout16:29
bauzasmoving on16:29
bauzas#info Please look at the gate failures and file a bug report with the gate-failure tag.16:29
sean-k-mooneybauzas: its here https://opendev.org/openstack/tempest/src/branch/master/zuul.d/integrated-gate.yaml#L325 just incase your wondering where to do that16:29
bauzasack, thanks but I found it16:31
bauzashence the DNM patch in nova16:31
bauzasto test it16:31
bauzaswith experimental16:32
bauzasanyway16:32
bauzasnothing to tell more about blind rechecks, I haven't looked at the numbers16:32
bauzasmoving on16:32
bauzas#info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures16:32
bauzas#topic Release Planning 16:32
bauzas#link https://releases.openstack.org/zed/schedule.html16:32
bauzas#info Zed GA was last week, kudos to the team.16:33
bauzaswe're officially done with Zed :)16:33
bauzasso...16:33
bauzas#link https://releases.openstack.org/antelope/schedule.html16:33
bauzas#info Antelope-1 is planned in 5 weeks16:33
bauzasnothing to tell it, the crux is for the next topic16:33
bauzas#topic PTG planning 16:33
bauzas#link https://etherpad.opendev.org/p/nova-antelope-ptg Antelope PTG etherpad16:33
bauzas#link https://ptg.opendev.org/ptg.html PTG schedule16:34
bauzas#info Attempted schedule in the etherpad16:34
bauzasyeah I did a bit of a topics scrambling16:34
bauzasbut honestly, we don't have a large agendat *yet*16:34
bauzasI'll just move the nova-otherproject topics 16:35
bauzasfor the moment, we have nova-neutron (Thur) and nova-ironic (reminder : *Monday*)16:35
bauzasdo people want to talk with cinder or other projects ?16:35
bauzasI haven't got any ask from the cyborg project16:35
bauzasor oslo or QA ?16:36
bauzasor manila, or glance... :)16:36
gibinothing on my side16:36
sean-k-mooneyi think the ironic topic will fill any space we give it16:37
sean-k-mooneybut im not sure if we will be able to make progress remotely16:37
bauzashah16:37
bauzassean-k-mooney: as I already said a couple of times, I don't think we will find a solution16:38
bauzasbut,16:38
bauzasat least I want our community to understand why we can't just modify an instance .host DB value16:38
sean-k-mooneywe also shoudl not have the hashring in the driver as we do today16:39
bauzasif we're able to explain it even after all the comments dansmith and you did, well, it will be nice then16:39
sean-k-mooneyi look forward to your summary16:40
bauzasanyway, I don't want to discuss this spec by now, we'll have time on Monday :)16:40
* sean-k-mooney has flushed most of that out of my brain but will re read the spec16:40
bauzassean-k-mooney: /o\16:40
bauzasbut yeah, I'll summarize this session after the PTG :)16:40
bauzasmoving on16:40
bauzaslooks we can find other projects to discuss if we want during the PTG anyxway16:41
bauzasas a reminder too, we could get pain points or feature requests from operators16:41
bauzas#link https://etherpad.opendev.org/p/oct2022-ptg-operator-hour-nova Operator specific etherpad for our nova operator hours16:41
sean-k-mooneyi hope we do. even if they are ones we have gotten before16:41
bauzas#topic Review priorities 16:42
bauzas#link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+(label:Review-Priority%252B1+OR+label:Review-Priority%252B2) 16:42
bauzasI need to add some changes I have as a review-prio 16:42
bauzasapart from this, nothing to tell16:43
bauzasnext topic ?16:43
bauzas(we'll discuss the review-prio flag at the PTG anyway)16:43
bauzas#topic Stable Branches 16:43
bauzaselodilles: your dog.16:43
elodilleso/16:43
elodilles#info from stable/zed back till stable/ussuri branches' gates should be OK16:43
elodilles#info stable/train seems broken again: devstack-gate's test-matrix playbook fails ("No module named yaml")16:44
elodillesand it seems we already have a workaround for this: https://review.opendev.org/c/openstack/devstack-gate/+/86096116:44
elodillesand for some information this came with Ansible 616:44
elodillesthe rest are the same:16:45
elodilles#info stable/stein (and older) are blocked: grenade and other devstack based jobs fail with the same timeout issue as stable/train was previously16:45
elodilles#info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci16:45
elodillesand that's all16:45
bauzascool thanks16:45
bauzascan't wait for next week for discussing about stein :)16:46
elodillesstein and older :)16:46
bauzasoh you're right btw.16:46
bauzaswe can now backport again \o/16:46
bauzasgiven Zed is released16:46
bauzashuzzah16:46
elodilles\o/16:46
gibi:)16:47
bauzasok, anything to add ?16:47
bauzaslooks not16:47
bauzas#topic Open discussion 16:47
bauzasjust two paperwork items on my side16:47
bauzas(bauzas) Cancel next week's meeting due to PTG16:47
bauzasI forgot the question mark, but is this really needed ?16:48
bauzasanyone having problems about cancelling ?16:48
bauzaslike, I can only attend by IRC, zoom is terrible to me16:48
bauzasor, I can show my face, I just prefer us to continuously discussing over IRC 16:49
bauzaslooks not16:49
gibino need for a meeting next week16:49
gibion IRC16:50
bauzas#action bauzas to communicate (email and wiki) on Nov 18 nova meeting be CANCELLED16:50
bauzaslast item from me16:50
bauzas(bauzas) Nov-1 is an holiday here, anyone wants to run the meetingĀ ?16:50
bauzasall said ^16:50
bauzasI'm usually on perpetual PTO, but this time I'm asked to be on holiday16:51
bauzasI guess most of the Europe is also having holiday at that day, if I trust my company calendar16:51
bauzasshall we just cancel it too ?16:51
bauzasthe fun is, Nov-8 is ALSO an holiday for me the week after16:52
gibiI'm of on Nov-1 too16:52
gibioff16:52
elodillessame for me16:52
gibicancel it16:52
bauzasyeah, was just opening the door to anyone fancy running the meeting if he/she wanted16:52
gibiI don't see much hands in the air16:53
gibi:)16:53
dansmith++ cancel16:53
bauzasgibi: on holiday nova-8 as well or not ?16:53
bauzasshit16:53
bauzasnov-816:53
bauzasoh nevermind16:53
bauzasI'm stupid16:54
bauzasI'm not good at knowing our holidays16:54
bauzasI confused myself with may16:54
bauzasnov-8 is a workday, that's nov-11 which is holiday16:54
* bauzas feels ashamed16:54
bauzasanyway, sold.16:54
gibiI will be in Brno on nov-816:54
gibiso working but not on upstream nova :)16:55
bauzas#undo16:55
opendevmeetRemoving item from minutes: #action bauzas to communicate (email and wiki) on Nov 18 nova meeting be CANCELLED16:55
gibiNova 18?16:55
bauzas#action bauzas to communicate (email and wiki) on Oct 18 nova meeting be CANCELLED16:55
gibiNov 18?16:55
bauzas#action bauzas to communicate (email and wiki) on Nov 1st nova meeting be CANCELLED16:55
gibigetting better :)16:55
bauzasI realized16:56
elodillesand Oct 1816:56
gibioct 18 and nov 1 ACK16:56
elodillessorry, there it is16:56
elodilles:X16:56
bauzasafter this and my confusion on holiday dates, that means it's time for me to end the meeting16:56
gibicalendars are hard :D16:56
bauzasthanks all and kudos for the hard work you made on Zed, btw.16:57
bauzas#endmeeting16:57
opendevmeetMeeting ended Tue Oct 11 16:57:20 2022 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:57
opendevmeetMinutes:        https://meetings.opendev.org/meetings/nova/2022/nova.2022-10-11-16.00.html16:57
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/nova/2022/nova.2022-10-11-16.00.txt16:57
opendevmeetLog:            https://meetings.opendev.org/meetings/nova/2022/nova.2022-10-11-16.00.log.html16:57
elodillesthanks bauzas o/16:57
bauzaslooks I'm tired16:58
elodillesthen it's time to relax with a good french wine?16:59
gibio/17:00
bauzaselodilles: I rather need a beer17:02
elodillesi guess that's also an option :)17:03
opendevreviewDan Smith proposed openstack/nova master: DNM: Test ceph-multistore with a real image  https://review.opendev.org/c/openstack/nova/+/86086418:58
opendevreviewDan Smith proposed openstack/nova master: DNM: Test ceph-multistore with a real image  https://review.opendev.org/c/openstack/nova/+/86086421:08
atmarkdoes the nova compute need {cpu,disk,ram}_weight_multiplier option too or just nova scheduler?21:53
opendevreviewDan Smith proposed openstack/nova master: DNM: Test ceph-multistore with a real image  https://review.opendev.org/c/openstack/nova/+/86086422:01

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!