16:01:07 <Uggla> #startmeeting nova
16:01:07 <opendevmeet> Meeting started Tue May 13 16:01:07 2025 UTC and is due to finish in 60 minutes.  The chair is Uggla. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:07 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:07 <opendevmeet> The meeting name has been set to 'nova'
16:01:14 <elodilles> o/
16:01:18 <fwiesel> o/
16:01:25 <Uggla> Hello everyone
16:01:30 <masahito> o/
16:01:36 <gibi> o/
16:01:39 <Uggla> awaiting a moment for people to join.
16:02:10 <bauzas> o/
16:02:37 <Uggla> #topic Bugs (stuck/critical)
16:02:50 <Uggla> #info https://bugs.launchpad.net/nova/+bug/2110545 nova next is bloqued.
16:03:04 <Uggla> # info https://review.opendev.org/c/openstack/nova/+/949622 proposal to skip this until the bug is fixed.
16:03:29 <gibi> yepp I needed to recheck the disablement https://review.opendev.org/c/openstack/nova/+/949622 as it failed with a kernel panic :/
16:03:29 <Uggla> This issue was discussed before on the channel.
16:04:20 <Uggla> I understood that kernel src vanished or are incorrect and so we can't build the tool.
16:04:42 <bauzas> I'll try to find a solution with my mtty series
16:05:05 <Uggla> the tool is for testing mdev, if I'm not wrong.
16:05:44 <Uggla> gibi, bauzas anything you want to add ?
16:05:49 <gibi> nope
16:06:21 <Uggla> #topic Gate status
16:06:31 <Uggla> #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs
16:06:49 <Uggla> So the bug mentioned before is available ^
16:07:00 <Uggla> #link https://etherpad.opendev.org/p/nova-ci-failures-minimal
16:07:08 <Uggla> #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status
16:07:52 <Uggla> Btw I have checked these periodics they are all ok, despite some of them are a bit flaky.
16:08:02 <Uggla> #info Please look at the gate failures and file a bug report with the gate-failure tag.
16:08:20 <Uggla> #info Please try to provide meaningful comment when you recheck
16:08:47 <Uggla> #topic tempest-with-latest-microversion job status
16:08:54 <Uggla> #link https://zuul.opendev.org/t/openstack/builds?job_name=tempest-with-latest-microversion&skip=0
16:10:02 <gmaan> not much update this week except I made keypair test also passing now
16:10:05 <gmaan> #link https://review.opendev.org/q/topic:%22latest-microversion-testing%22
16:10:38 <Uggla> Hi gmaan, I thought you were not in the call, thx for the update.
16:10:58 <Uggla> gmaan, something else you want to add ?
16:11:15 <gmaan> nothing else from me
16:11:22 <Uggla> thx, so moving on
16:11:28 <sean-k-mooney> o/
16:11:32 <Uggla> #topic Release Planning
16:11:38 <Uggla> #link https://releases.openstack.org/flamingo/schedule.html
16:11:48 <Uggla> #link https://releases.openstack.org/flamingo/schedule.html
16:11:57 <Uggla> #info We reach Flamingo-1 milestone, only 3 weeks before Nova Spec Soft Freeze, do not forget to submit your specs.
16:12:08 <bauzas> milestone 1 if I'm not wrong
16:12:12 <bauzas> heh
16:12:27 <Uggla> ?
16:12:48 <bauzas> just saying where we are
16:13:06 <bauzas> but jinxed
16:13:09 <Uggla> yes  Flamingo-1 milestone ^
16:13:21 <masahito> quick question for the spec freeze. is it deadline to merge a spec or deadline to submit spec?
16:13:31 <elodilles> yepp, this week is Flamingo-1 :]
16:13:49 <sean-k-mooney> milestone 1 is thursday
16:14:05 <sean-k-mooney> so we shoudl be creatign the first release of our libs like os-vif
16:14:15 <sean-k-mooney> if they have not had one already
16:14:38 <elodilles> sean-k-mooney: release patches are already generated where were (functional) code changes
16:14:47 <sean-k-mooney> ack
16:14:56 <Uggla> masahito, soft spec is  deadline to submit spec
16:15:13 <Uggla> spec freeze is deadline to merge the spec
16:15:51 <elodilles> sean-k-mooney: with a quick check i don't see any nova related flamingo-1 release patch ( https://review.opendev.org/q/topic:flamingo-milestone-1 )
16:16:02 <masahito> got it. i understand i need to submit the spec and poc code quickly.
16:16:57 <sean-k-mooney> elodilles: we definally have had some functional changes. althogu i know for os-trait for one we already had at least 1 release
16:17:21 <Uggla> masahito, mainly the code.
16:17:25 <Uggla> oops
16:17:38 <Uggla> sorry mainly the spec
16:17:41 <sean-k-mooney> masahito: you tecnially can submit a spec after the soft freeze but the later you do the more risk peole will run out of time to review
16:18:16 <elodilles> sean-k-mooney: that is 'cycle independent' project, so that one is not closely followed by release team
16:18:29 <masahito> thanks
16:18:45 <sean-k-mooney> elodilles: ack, we can likely move on to the next topic?
16:18:55 <elodilles> sean-k-mooney: ++
16:20:00 <Uggla> sean-k-mooney, elodilles question should I propose a release patch for os-trait ?
16:20:51 <bauzas> that's an independent
16:20:56 <bauzas> no need
16:20:57 <elodilles> Uggla: there is no deadline for os-traits to Flamingo-1, but it can be proposed any time you want
16:21:25 <Uggla> ok thx I just wanted to be sure.
16:21:25 <bauzas> i mean independent release cadence
16:21:50 <Uggla> #topic Review priorities
16:21:59 <Uggla> #link https://etherpad.opendev.org/p/nova-2025.2-status
16:22:00 <elodilles> on the other hand it's good to release when there are merged patches, so that it won't be forgotten during the cycle o:)
16:22:12 <bauzas> Uggla: I could show you up where to find the release models for each of the projects we have in OpenStack
16:22:29 <sean-k-mooney> i know mstill would like to do one for the vdi stuff but we can just do that out of band
16:23:10 <Uggla> Just to let you know the the title of the channel was changed to point to the correct document
16:23:18 <sean-k-mooney> i.e. we have a reqeust to do a release but we dont need to do one proceudrlly on any deadline we just should to unblock them
16:23:58 <sean-k-mooney> Uggla: yes i want to add an opendicssion topic related to that btu we can talk about that when we get there
16:24:37 <Uggla> ok
16:24:44 <Uggla> #topic Stable Branches
16:24:50 <Uggla> time for you elodilles
16:24:57 <elodilles> o7
16:25:04 <elodilles> #info stable branches (stable/2025.1 and stable/2024.*) seem to be in OK state
16:25:10 <elodilles> #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci
16:25:19 <elodilles> that's all from me
16:25:39 <Uggla> cool thx
16:25:42 <Uggla> #topic vmwareapi 3rd-party CI efforts Highlights
16:25:50 <Uggla> fwiesel, anything special ?
16:25:51 <fwiesel> No updates from my side.
16:26:03 <Uggla> thx
16:26:11 <Uggla> #topic Gibi's news about eventlet removal.
16:26:18 <Uggla> #link Series: https://gibizer.github.io/categories/eventlet/
16:26:26 <gibi> latest blog post is https://gibizer.github.io/posts/Eventlet-Removal-The-First-Threading-Bug/
16:26:27 <Uggla> #link reminder patches to review: https://review.opendev.org/q/topic:%22eventlet-removal%22+project:openstack/nova
16:27:02 <gibi> working on cleaning up the nova-scheduler series
16:27:31 <gibi> review is appreciated on https://review.opendev.org/c/openstack/nova/+/948437?usp=search
16:27:42 <gibi> and on https://review.opendev.org/c/openstack/nova/+/948437?usp=search
16:27:50 <gibi> I mean https://review.opendev.org/c/openstack/nova/+/948437?usp=search
16:28:00 <gibi> somebody break my copy paste buffer :)
16:28:03 <bauzas> I so would like to help reviewing
16:28:12 <bauzas> I'll ty to find some time
16:28:16 <gibi> https://review.opendev.org/c/openstack/nova/+/947265
16:28:21 <gibi> https://review.opendev.org/c/openstack/nova/+/948437
16:28:24 <gibi> the two patches to land
16:28:35 <gibi> the long series is not ready but working on it
16:28:36 <sean-k-mooney> 37 is pretty short so i can proably look at that after the meeting
16:28:50 <gibi> that is all from me
16:28:56 <sean-k-mooney> and the hacking one i gues
16:29:02 <gibi> yepp
16:30:01 <Uggla> #topic Open discussion
16:30:21 <fwiesel> That would be my topic, I suppose.
16:30:23 <Uggla> fwiesel, can you go ahead with your topic
16:30:25 <fwiesel> Support for libvirt VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS
16:31:03 <fwiesel> So, since QEMU 2.11 and libvirt 5.2.0 there is support for using multiple threads in the precopy operation.
16:31:03 <sean-k-mooney> oh :) that could be useful
16:31:20 <fwiesel> I was wondering, if I can just to a specless blueprint for that?
16:31:50 <sean-k-mooney> i think that would be supported by our min qemu/libvirt versions
16:32:12 <sean-k-mooney> so it would really just be a config option to allow enabling it
16:32:16 <sean-k-mooney> and perhaps dicussing if it should be on or off by defualt
16:32:29 <gibi> would it be a compute config param to enable? Does it only need to be enabled on the source node?
16:32:41 <Uggla> I think because it require a config option a small spec will be worth it.
16:32:57 <sean-k-mooney> this is really only useful if you have more bandwith aviable on the migration link then a singel core can saturate.
16:33:01 <fwiesel> Ah, so a spec. I  can do that
16:33:18 <fwiesel> Yes, we need roughly 12 threads to saturate our 200GigE links.
16:33:21 <sean-k-mooney> well it could be specless maybe but yes i blive this need to be set on the source
16:33:33 <Uggla> yes so we can bikeshed on the config name. :)
16:33:50 <sean-k-mooney> like the other migration parmaters i dont think it can be changed once the migration is started and that is done form the source node
16:34:39 <Uggla> sean-k-mooney, this is a parameter to pass to migrateToURI3, if I'm not wrong.
16:34:42 <sean-k-mooney> fwiesel: based on that math if you have a 25G link or grather you would benifit form this
16:34:52 <sean-k-mooney> Uggla: yes i belive it is
16:35:17 <gibi> Thanks. I'm OK with a specless but fine with a spec too
16:35:18 <Uggla> sean-k-mooney, I looked quickly before the call.
16:35:46 <sean-k-mooney> i belvie we were asked about this either in the ptg or on irc shourtly before it too
16:35:50 <Uggla> I think the spec is cool to discuss names and defaults
16:36:14 <Uggla> sean-k-mooney, more or less this is from masahito but this is another settigns
16:36:33 <sean-k-mooney> i think it makes sense  to do so sepless or small spec is ok with me
16:36:44 <sean-k-mooney> *specless or a small spec
16:36:55 <sean-k-mooney> we can debate the name in the gerrit review
16:36:57 <fwiesel> The consensus seems then a small spec?
16:37:06 <sean-k-mooney> presumable its a new option in the libvirt section
16:37:15 <bauzas> I'd say that depends on the default value
16:37:32 <sean-k-mooney> like https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_inbound_addr
16:37:36 <bauzas> meaning whether we would have upgrade concerns
16:37:38 <fwiesel> I'd gone with a default that keeps the old settings.
16:37:46 <fwiesel> So, it is opt-in
16:38:03 <sean-k-mooney> if we default to off. i dont think there would be an upgrade concern
16:38:13 <sean-k-mooney> and we can consider changing the default in 2026.2+
16:38:25 <bauzas> yup so then I specless works for me
16:38:27 <sean-k-mooney> we proably shoudl test it in nova-next
16:38:28 <gibi> opt-in works for me
16:38:39 <Uggla> ok so go for spec less
16:38:47 <bauzas> that's an easy enablement knob
16:38:55 <fwiesel> Perfect, saves me some work.
16:39:10 <sean-k-mooney> fwiesel: can you create a bluepirnt for this to tack it and detail that it will be off by default
16:39:29 <sean-k-mooney> unless you already have one?
16:39:37 <fwiesel> sean-k-mooney: will do. I haven't yet.
16:39:43 <Uggla> fwiesel, and ping me, so I would add it to the tracking doc.
16:39:57 <fwiesel> Uggla: Will do
16:40:02 <Uggla> thx
16:40:44 <Uggla> anything else to discuss, if not we can triage some bugs.
16:40:52 <sean-k-mooney> i have one quick item
16:41:06 <Uggla> shoot
16:41:13 <sean-k-mooney> so last week i asked frickler i think to update the channel topic if Uggla aggreed
16:41:32 <sean-k-mooney> which has now been done but they pointed out that we coudl also add ops to the channle https://opendev.org/openstack/project-config/src/commit/50197dce1589385b1256150bcd082093cf7e8898/accessbot/channels.yaml#L239
16:41:38 <sean-k-mooney> nova currently does nto have any
16:41:49 <gibi> lets add some :)
16:41:53 <sean-k-mooney> do we want to do that so we can update the topic ectra our selves
16:41:54 <bauzas> I was thinking Dan was +o
16:42:08 <sean-k-mooney> dan might have it form something else
16:42:17 <sean-k-mooney> but we dont ahve anyone lised in the file today
16:42:25 <Uggla> I looked at it only gmaan if I recall well.
16:42:47 <bauzas> fair
16:42:49 <sean-k-mooney> shall i create a patch to do it? and if so who should we add
16:42:53 <dansmith> I was pretty sure I had ops at one point
16:42:54 <gmaan> I can do
16:43:02 <gmaan> I think I still have ops
16:43:05 <sean-k-mooney> Uggla: im also ok to delgate doign that to someone else
16:43:12 <bauzas> I could add french jokes to the chan title
16:43:45 <bauzas> but hey the PTL should have rights
16:44:21 <gmaan> I have from global ops, ++ to add nova specific also
16:44:26 <Uggla> sean-k-mooney, I can create this patch no worries.
16:44:36 <sean-k-mooney> Uggla: cool that all then
16:44:50 <sean-k-mooney> i just wanted to make sure we could udate the topic when needed
16:44:55 <Uggla> who wants to be ops ?
16:45:08 <bauzas> you should be
16:45:13 <gmaan> do we also need to update topic now or all good for now?
16:45:27 <Uggla> gmaan, topic is ok atm.
16:45:31 <gmaan> k
16:45:38 <sean-k-mooney> gmaan: it should be good now
16:45:51 <sean-k-mooney> the only thing that need to change was the status url
16:46:06 <gmaan> ++
16:46:20 <sean-k-mooney> it was pointing to https://etherpad.opendev.org/p/nova-2025.1-status until last week
16:46:37 <sean-k-mooney> Uggla: that shoudl also proably get added to the nova PTL guide
16:46:43 <sean-k-mooney> as a thing to do at the start of each cycle
16:46:54 <Uggla> sean-k-mooney, ++
16:46:57 <bauzas> yup
16:47:03 <bauzas> good idea
16:47:22 <Uggla> #action Update ptl guide to mention topic update
16:47:23 <sean-k-mooney> we have about 12 minutes left. want to traige a bug or two
16:47:33 <Uggla> yep
16:47:40 <Uggla> #topic Bug scrubbing
16:47:49 <Uggla> #link: https://etherpad.opendev.org/p/nova-bug-selection-for-triaging#L4
16:48:25 <Uggla> First one: https://bugs.launchpad.net/nova/+bug/2106500
16:48:59 <Uggla> I'm not sure we are concerns by this one.
16:49:52 <Uggla> That looks more a packaging pb.
16:50:54 <gibi> graceful shutdown issue maybe
16:51:16 <sean-k-mooney> im not sure that we shold be providing log rotate config
16:51:49 <sean-k-mooney> actully
16:51:51 <sean-k-mooney> https://github.com/openstack/nova/tree/master/etc/nova
16:51:57 <sean-k-mooney> i dont think we ar
16:52:00 <sean-k-mooney> *are
16:52:11 <sean-k-mooney> so this may be a packaging issue yes
16:52:45 <sean-k-mooney> os it proably shoudl be marked invalid in nova but left open for the nova ubuntu package
16:53:13 <Uggla> I propose to answer in that way and set the bug to incomplete. Sounds good ?
16:53:51 <sean-k-mooney> so i would put it to invalid personally
16:54:02 <sean-k-mooney> we do not provide a logroate script in our repo
16:54:09 <sean-k-mooney> i think this is part of the debian package
16:54:10 <fwiesel> Uggla: Here is the blueprint https://blueprints.launchpad.net/nova/+spec/libvirt-migrate-parallel
16:54:19 <Uggla> fwiesel, thx
16:54:29 <Uggla> sean-k-mooney, ok that works for me.
16:54:34 <Uggla> next one:
16:54:46 <Uggla> https://bugs.launchpad.net/nova/+bug/2103413
16:56:47 <gibi> do we officially support 3.13?
16:56:52 <Uggla> We are not supporting 3.13 yet.
16:56:53 <sean-k-mooney> no yet
16:57:04 <sean-k-mooney> its part of the experimantal runtime for this release
16:57:11 <sean-k-mooney> and offically should be supported in 2026.1
16:57:13 <gmaan> not yet but we should start testing it as it can be mandatory for next cycle
16:57:17 <gmaan> yeah
16:57:22 <sean-k-mooney> however ubuntu 25.04 is already using it
16:57:35 <sean-k-mooney> and its an issue for debian 13 also i think
16:57:37 <gibi> it is probably yet another eventlet issue we might or might not able to patch
16:57:47 <sean-k-mooney> eventlet is curently broken on it in general
16:57:56 <Uggla> and also it might be linked to gibi's work on eventlet.
16:58:23 <sean-k-mooney> only by name
16:58:33 <gibi> the eventlet work will not be finished in F so if in G we need to support 3.13 then we are in the pickle
16:58:47 <sean-k-mooney> so if someone knwo how to make it work on 3.13 and can propose a fix
16:59:07 <sean-k-mooney> then we shoudl review it and conside rbackporting to enable debian based distros to supprot epoxy
16:59:16 <sean-k-mooney> but its not offically supproted there so its best effort
16:59:49 <sean-k-mooney> gibi: well i know that eventlet on 3.13 has issues with the threadid
17:00:09 <sean-k-mooney> as in its ment to patch the thread id so that it returns the greenthread id instead
17:00:10 <gibi> this seems to be a GC + eventlet issue
17:00:13 <sean-k-mooney> and thats not workign properly
17:00:21 <gibi> I would not bet on an easy fix
17:00:32 <gibi> not in F.
17:00:41 <gibi> in G we might be able to get rid of eventlet :)
17:00:57 <sean-k-mooney> that one way to fix the issue :)
17:01:08 <gibi> that is the only way I want to bet on :)
17:01:36 <sean-k-mooney> https://github.com/eventlet/eventlet/issues/1030
17:01:49 <sean-k-mooney> so this was the issue i was aware of that was the blocker for nova on 3.13
17:01:56 <sean-k-mooney> you dont actully need nova to trigger it
17:02:03 <sean-k-mooney> but it happens for nova too
17:02:29 <sean-k-mooney> ah and https://github.com/eventlet/eventlet/issues/1032
17:02:34 <sean-k-mooney> is the issue we are talking about
17:03:06 <sean-k-mooney> so we might want to mark it incomplete or invalid and refnrence that https://github.com/eventlet/eventlet/issues/1032
17:03:22 <sean-k-mooney> basicaly eventlet needs to supprot 3.13 before we can
17:03:31 <Uggla> yep I can explain the situation in the ticket.
17:03:35 <gibi> OK
17:03:57 <JayF> Those issues getting fixed were blocked by CI + some administrivia in that repo which I think are resolved now.
17:04:34 <masahito> sorry for the slow reply for the 1st bug.  the attach volume API makes 2 RPC from nova-api to nova-compute, 1st one is call RPC and 2nd one is cast RPC. if the nova-api gets failed between the 2 rpc, garbage block device information remaining.
17:04:54 <masahito> (I think it's okay to talk the details later.)
17:05:24 <Uggla> there is also a status deferred for the ticket
17:06:01 <Uggla> Are you ok to do one more bug ?
17:06:49 <sean-k-mooney> sure
17:06:56 <Uggla> https://bugs.launchpad.net/nova/+bug/2103413
17:07:17 <sean-k-mooney> that seams to be mostly the same thing?
17:07:53 <sean-k-mooney> wiat its exaclty the same bug
17:08:23 <sean-k-mooney> Uggla: did you mean to post a differnt url?
17:08:23 <gibi> then we are done :)
17:08:23 <Uggla> https://bugs.launchpad.net/nova/+bug/2103513
17:08:39 <Uggla> sorry wrong copy pasta
17:09:05 <gibi> I have no clue about TLS spice
17:09:57 <sean-k-mooney> so i dont know if nova actully supprot this yet
17:10:13 <sean-k-mooney> i remember talking about this in the recent spci enabling
17:10:14 <Uggla> what it is on dalmatian
17:10:32 <sean-k-mooney> right but im not sure if that is expected to work even on master
17:10:38 <Uggla> oh
17:10:46 <sean-k-mooney> it might
17:10:56 <sean-k-mooney> but i dont belilve we have tempest testing of it currently
17:11:08 <sean-k-mooney> we only recnetly readded spice testing last cycle
17:11:16 <sean-k-mooney> after not having any tempest testing with it for years
17:11:29 <sean-k-mooney> lets loop back to this again next week
17:11:52 <sean-k-mooney> and maybe ask mikal about it
17:12:02 <Uggla> ok that sounds good.
17:12:33 <Uggla> At least I can right a comment about it.
17:12:44 <Uggla> that's all for today.
17:13:18 <sean-k-mooney> https://review.opendev.org/c/openstack/nova/+/922544
17:13:47 <sean-k-mooney> ok in theory that was added in dalmation
17:13:55 <sean-k-mooney> but perhaps only when usign kerbside
17:14:05 <sean-k-mooney> in anycase we can wrap for today
17:14:25 <Uggla> Thanks all and thanks for the extended time for bug triage.
17:14:33 <Uggla> #endmeeting