| opendevreview | melanie witt proposed openstack/nova master: live migration: Avoid volume rollback mismatches https://review.opendev.org/c/openstack/nova/+/946600 | 01:18 |
|---|---|---|
| melwitt | looks like nova-tox-functional-py39 is failing 100% due to a package dependency conflict https://zuul.opendev.org/t/openstack/builds?job_name=nova-tox-functional-py39&project=openstack/nova | 01:53 |
| melwitt | I also notice cyborg-tempest and barbican-tempest-plugin-simple-crypto started failing recently, 1-2 days ago | 01:54 |
| opendevreview | Ivan Anfimov proposed openstack/nova master: Remove deprecated [workarounds] enable_numa_live_migration https://review.opendev.org/c/openstack/nova/+/905426 | 07:00 |
| gibi | melwitt: the cyborg-tempest failure is caused by cyborg-api not starting: --- no python application found, check your startup logs for errors --- | 07:00 |
| gibi | melwitt: reported bug https://bugs.launchpad.net/openstack-cyborg/+bug/2109583 | 07:06 |
| gibi | melwitt: barbican has very similar failure and similar data of breakdown | 07:07 |
| gibi | so there should be a common cause | 07:07 |
| gibi | filed a barbican bug too https://bugs.launchpad.net/barbican/+bug/2109584 | 07:10 |
| gibi | it all started on 04-27 but I don't see devstack/tempest changes merged around that time | 07:12 |
| gibi | there are requirements changes but nothing suspicious to me | 07:12 |
| gibi | melwitt: but I think the reason functional-py39 fails is that we dropped that python version from global requirements. I will propose a patch to remove the job from nova | 07:15 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Remove python 3.9 support https://review.opendev.org/c/openstack/nova/+/948392 | 07:23 |
| gibi | melwitt, bauzas, sean-k-mooney ^^ | 07:23 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Remove python 3.9 support https://review.opendev.org/c/openstack/nova/+/948392 | 07:24 |
| gibi | this is needed to unblock the nova gate | 07:25 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Remove python 3.9 support https://review.opendev.org/c/openstack/nova/+/948392 | 07:32 |
| frickler | the former looks similar to what ralonsoh found https://bugs.launchpad.net/designate/+bug/2109577 | 07:32 |
| gibi | frickler: indeed | 07:33 |
| gibi | and also started failing on the 27th | 07:34 |
| gibi | so probably the same common but unknown cause | 07:34 |
| frickler | well setuptools==80 released on the 27th, looks very likely | 07:35 |
| gibi | ohh | 07:35 |
| ralonsoh | in designate, the wsgi binary is not being created | 07:38 |
| ralonsoh | but I can´t reproduce it locally | 07:38 |
| gibi | is it the missing pyproject.toml? | 07:39 |
| gibi | 2025-04-29 05:43:24.960850 | controller | DEPRECATION: Legacy editable install of keystone==27.1.0.dev20 from file:///opt/stack/keystone (setup.py develop) is deprecated. pip 25.3 will enforce this behaviour change. A possible replacement is to add a pyproject.toml or enable --use-pep517, and use setuptools >= 64. If the resulting installation is not behaving as expected, try using | 07:39 |
| gibi | --config-settings editable_mode=compat. Please consult the setuptools documentation for more information. Discussion can be found at https://github.com/pypa/pip/issues/11457 | 07:39 |
| gibi | 7010 | 07:39 |
| gibi | https://zuul.opendev.org/t/openstack/build/af7efda68d204f1c99f8eeef77359ebf/log/job-output.txt#7009 | 07:40 |
| gibi | as setuptool 80 changelog says | 07:40 |
| gibi | Develop command no longer uses easy_install, but instead defers execution to pip (which then will re-invoke Setuptools via PEP 517 to build the editable wheel). Most of the options to develop are dropped. This is the final warning before the command is dropped completely in a few months. Use-cases relying on ‘setup.py develop’ should pin to older Setuptools version or migrate to modern build | 07:40 |
| gibi | tooling. (#4955) | 07:40 |
| gibi | https://setuptools.pypa.io/en/stable/history.html | 07:40 |
| gibi | yeah barbican does not have pyproject.toml but nova has | 07:41 |
| gibi | designate neither | 07:42 |
| ralonsoh | neutron doesn't have it | 07:42 |
| ralonsoh | but seems to run fine | 07:42 |
| gibi | maybe neutron does not use setup.py develop | 07:43 |
| gibi | https://review.opendev.org/c/openstack/nova/+/899753 this added pyproject.toml to nova | 07:44 |
| ralonsoh | but, if I'm not wrong, we use editable installs in neutron, at least in neutron-tempest-plugin | 07:45 |
| ralonsoh | maybe I'm wrong | 07:45 |
| johnsom | https://review.opendev.org/c/openstack/designate/+/902846 | 07:56 |
| ralonsoh | johnsom, can you add this info in the designate bug? | 07:57 |
| ralonsoh | johnsom, also this is blocking now the Neutron CI. Could be possible to have it rebased and reviewed? | 07:58 |
| johnsom | Yea, I am about to turn in for the night, but can add a comment | 07:58 |
| johnsom | I would hope so. Maybe also ping in the designate channel | 07:58 |
| ralonsoh | johnsom, thanks a lot! | 07:59 |
| frickler | I can take a look after a bunch of meetings that I've got coming up | 07:59 |
| ralonsoh | gibi, ok, now I realize that Neutron is actually using the wsgi module and no longer using the script. This is why is working | 07:59 |
| gibi | thanks folks | 08:03 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: WIP: allow service to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | 08:46 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Print ThreadPool statistics https://review.opendev.org/c/openstack/nova/+/948340 | 08:46 |
| elodilles | Uggla: just a quick double check: do you want to wait for reviews with the schedule patch (review + freeze days) or is it ready to merge already? https://review.opendev.org/c/openstack/releases/+/948077 | 08:52 |
| gibi | elodilles: Uggla is on PTO the whole week as far as I remember | 09:02 |
| elodilles | ahh, right. | 09:04 |
| elodilles | then I'll wait maybe until the meeting :) | 09:04 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: Add mtty/mdpy support for testing fake mdevs https://review.opendev.org/c/openstack/nova/+/898100 | 09:21 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Add mtty support to nova-next https://review.opendev.org/c/openstack/nova/+/922140 | 09:21 |
| opendevreview | Kamil Sambor proposed openstack/nova master: Replace eventlet.queue.LightQueue with queue.SimpleQueue https://review.opendev.org/c/openstack/nova/+/948417 | 10:05 |
| gibi | sean-k-mooney: bauzas: we need this to unblock the nova gate https://review.opendev.org/c/openstack/nova/+/948392 | 10:57 |
| sean-k-mooney | ... | 10:58 |
| sean-k-mooney | damb i new trhis was going to happen but i hoped we would have a few more months | 10:59 |
| sean-k-mooney | +2... i was hoping to keep watcher compaitble with 3.9 until we have an alternitive but if the requiremetns repo has droped validation fo 3.9 deps we are going to have to do something similar | 11:03 |
| gibi | yeah unfortunately they pulled the trigger | 11:03 |
| gibi | not having a py39 upper constraints causes version conflict in py39 jobs | 11:04 |
| sean-k-mooney | ya so for wathcer ill create a patch to do the same. im not going to bump required_python initally but ill do that in a followup patch | 11:05 |
| sean-k-mooney | that we will have to merge before the end of the cycle | 11:05 |
| sean-k-mooney | our downstream container build does not use the latest version of lib that UC allows so i want to see if it can still build on 3.9 with those deps before i fully block it | 11:06 |
| sean-k-mooney | but that is only a tempory hack. | 11:06 |
| sean-k-mooney | actully by downstrema in this context i just mean the operator jobs whcih are technially upstream using continers built form rdo content | 11:06 |
| sean-k-mooney | we are working on moving those to centos 10 stream instead which will upgrade them to 3.12 | 11:07 |
| sean-k-mooney | i was hoping ot have another month or two to finish that but its fine | 11:07 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: WIP: allow service to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | 11:15 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Print ThreadPool statistics https://review.opendev.org/c/openstack/nova/+/948340 | 11:15 |
| gibi | sean-k-mooney: yeah | 11:16 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Print ThreadPool statistics https://review.opendev.org/c/openstack/nova/+/948340 | 11:29 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: WIP: allow service to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | 11:29 |
| sean-k-mooney | https://review.opendev.org/c/openstack/devstack/+/948408/1/lib/nova#b1118 is fine but it still might be cleaner to have either the nova devstack plugin or the job create a droping file for the systemd serivce in the long term | 11:51 |
| sean-k-mooney | gibi: if we ever land your patch to make the [[post-config|path]] thing work for non exisitent directories | 11:52 |
| sean-k-mooney | we coudl do that via that in the job too | 11:52 |
| gibi | sean-k-mooney: yeah the current devstack patch is a hardocded hack. I planning to have a devstack local conf variable to list the nova services that should run in threading mode then use that to pass the env var to systemd | 11:54 |
| sean-k-mooney | what im kind of hoping we can avoid si a patch to devstack for every binary we need to swap between modes so im hopign we can come up with somethign generic and reusable | 11:54 |
| sean-k-mooney | ya that woudl work. i was thinking more generic have a map of systemd unit name to a list of extr env vars | 11:55 |
| sean-k-mooney | so it can work for any of the services so it woudl work for neutron or glance too | 11:55 |
| sean-k-mooney | but we can keep it nova speicifc not to inflate the scope too much | 11:56 |
| gibi | I will look into the generic mapping, that sounds useful | 12:16 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Print ThreadPool statistics https://review.opendev.org/c/openstack/nova/+/948340 | 12:25 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: WIP: allow service to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | 12:25 |
| sean-k-mooney | by the way could we land https://review.opendev.org/c/openstack/placement/+/945487 i would like to get that off my list of open patches while its not in merge conflict | 12:28 |
| gibi | +2 from me | 12:31 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Add mtty support to nova-next https://review.opendev.org/c/openstack/nova/+/922140 | 12:34 |
| gibi | dansmith: sean-k-mooney: I had some local stress testing of the threaded scheduler. In the happy path scatter-gather worker threads are properly returned to the pool and reused (I scheduled 120 VMs with the pool of 20 threads x 2 worker processes). | 12:35 |
| sean-k-mooney | cool and did that taek a lot of ram and or time or was it reasonable | 12:37 |
| sean-k-mooney | im just askign for a gut feeling by the way | 12:37 |
| sean-k-mooney | i assume that was not one multi create by the way | 12:37 |
| sean-k-mooney | you did parallel requests | 12:38 |
| gibi | the nova-scheduler workers are at 130MB RSS each | 12:40 |
| gibi | I did parallel create requests not a single multicreate | 12:40 |
| *** haleyb|out is now known as haleyb | 13:00 | |
| gibi | aand we have the first CI result where n-sch run with threading https://zuul.opendev.org/t/openstack/build/c2ec6bc4ae4946468d5b2b4f8468d12a/log/controller/logs/screen-n-sch.txt#2855-2856 | 13:04 |
| dansmith | gibi: nice | 13:26 |
| sean-k-mooney | gibi: that all sounds pretty reasonable. out side of a small number of thread local varablie and stack frames we woudl not expect the worker size to grow much as we increase the thread outside of the data needed for the schduler ot schdule | 13:40 |
| sean-k-mooney | so 130MB RSS per worker does not sound bad at all | 13:40 |
| opendevreview | Kamil Sambor proposed openstack/nova master: Replace eventlet semaphores with threading one https://review.opendev.org/c/openstack/nova/+/948437 | 13:43 |
| gibi | the nova-lvm job run 70 worker https://zuul.opendev.org/t/openstack/build/b5d70ad645514181a1fda52bba8187ce/log/controller/logs/screen-n-sch.txt#5099-5100 and n-sch had 120MB RSS https://zuul.opendev.org/t/openstack/build/b5d70ad645514181a1fda52bba8187ce/log/controller/logs/screen-memory_tracker.txt#2507-2508 on the gate | 13:58 |
| gibi | 70 task with 20 worker threads | 13:58 |
| gibi | 2 worker processes | 13:59 |
| sean-k-mooney | how are you calulating the defautl therad pool size by the way. are you jsut levaing futurist decide currently | 14:00 |
| sean-k-mooney | (failures=0, executed=70, runtime=1.90, cancelled=0)> | 14:01 |
| sean-k-mooney | does that mean it ran for 1.9 sconds total? | 14:02 |
| sean-k-mooney | i gues not | 14:02 |
| sean-k-mooney | 0x7cab7c4de4c0 (failures=0, executed=69, runtime=2.33, cancelled=0)> | 14:02 |
| sean-k-mooney | perhaps its since the last tiem you got the stats | 14:02 |
| gibi | futurist decides today | 14:03 |
| gibi | but we can have a config | 14:03 |
| sean-k-mooney | i belvie it default to 2.5*nproc | 14:04 |
| gibi | I can run some tests to see what is the runtime stats there | 14:04 |
| sean-k-mooney | i reused one fo the existing config option but i would proably default this to 4 | 14:04 |
| sean-k-mooney | and allow peopel to overried if needed | 14:04 |
| gibi | https://docs.openstack.org/futurist/latest/reference/index.html#futurist.ExecutorStatistics.runtime doc says total runtime of the pool | 14:04 |
| sean-k-mooney | hum well it went down form 69 to 70 | 14:05 |
| gibi | we have two worker processes so you need to check the pid to see if this is the same worker | 14:06 |
| sean-k-mooney | oh good point | 14:06 |
| sean-k-mooney | are you usign https://docs.openstack.org/futurist/latest/reference/index.html#futurist.periodics.Watcher | 14:06 |
| gibi | futurist creates cpu * 5 threads per executor by default https://github.com/openstack/futurist/blob/master/futurist/_utils.py#L126 | 14:06 |
| sean-k-mooney | gibi: ack thats proably more then we need | 14:06 |
| gibi | I don't use periodics, I log the stats when new work is submitted and if the last report was older than 60 sec | 14:08 |
| gibi | we can change it to periodic | 14:08 |
| gibi | and or we can have a config option for the age | 14:08 |
| gibi | yeah, 20 worker for scatter gather is too much, we can aim for 4 ish, but it depeds on the amount of parallel scheduling we need to serve | 14:09 |
| sean-k-mooney | right but we have 2 ways to scale workers and treads in each workser thread pool. and we can make it configurable | 14:11 |
| sean-k-mooney | well technially 3 you can just run more schdulers | 14:11 |
| sean-k-mooney | i think tying the count to the cpu cores will bit ues down the line espcially if this is deployed on k8s as that could change based on where the pod runs so that why i would prefer to have a small integer default | 14:12 |
| gibi | yeah I will expose a config option with a small default | 14:16 |
| gibi | and document that you can ask for more worker processes as well as a scaling mechanism | 14:17 |
| dansmith | I think it will take some time to figure out what the real default should be | 14:19 |
| dansmith | greenthreads are nice, but they also kinda avoid the resource exhaustion question such that we don't really have a good idea of what it needs to be, IMHO | 14:20 |
| gibi | yepp | 14:54 |
| sean-k-mooney | gibi: https://zuul.opendev.org/t/openstack/buildset/f495c3661ca346a198c80a01387d6ba4 so that not terible for a first run | 14:54 |
| gibi | nova-next is green that is enough for me to declare success | 14:55 |
| sean-k-mooney | im not sure why grenade is unhappy | 14:55 |
| sean-k-mooney | maybe the devstack change | 14:55 |
| gibi | some tempest jobs does not have oslo.service as required project hence the oslo.service depends on is ignored | 14:55 |
| sean-k-mooney | ah ya that too | 14:55 |
| gibi | also nova-multi-cell is green | 14:56 |
| sean-k-mooney | yep most of them are | 14:57 |
| opendevreview | Dan Smith proposed openstack/nova master: WIP Example NVMe cleaning script for one-time-use https://review.opendev.org/c/openstack/nova/+/948446 | 14:57 |
| sean-k-mooney | gibi: the cyborg job is broken because they do not have pyproject.toml supprot so there wsgi script are not gerneated | 14:57 |
| gibi | nova-multi-cell executed the most scatter-gathers and the nova-scheduler RSS is still around 122MB | 14:57 |
| gibi | sean-k-mooney: yeah I ended up at the same conclusion this morning, as well as barbican and designate | 14:58 |
| sean-k-mooney | im fixing that on watcher but that will impact other proejct shortly | 14:58 |
| gibi | I filed bugs for barbican and cyborg, ralonsoh has bug for designate | 14:58 |
| sean-k-mooney | ack there was a pip release 3 days ago | 14:58 |
| gibi | yeah frickler pointed out that setupstools=80 is the root cause | 14:59 |
| sean-k-mooney | well kind of | 14:59 |
| ralonsoh | just for information: https://review.opendev.org/c/openstack/designate/+/902846 | 14:59 |
| ralonsoh | the designate fix (along with another fix for the CI) | 14:59 |
| sean-k-mooney | the actual root causae is we have been relying on a compatibality fallbac that was added 2 years ago and then deprecated for removal a year ago | 15:00 |
| sean-k-mooney | ralonsoh: this is the watcher fix but im respining it now https://review.opendev.org/c/openstack/watcher/+/948438 to fix the mistakes | 15:00 |
| sean-k-mooney | ralonsoh: i was going to defer the removal of the wsgi_script until next cycle | 15:01 |
| ralonsoh | well, now is mandatory I think | 15:02 |
| sean-k-mooney | but i guess if we backported this to epoxy we coudl remove it this cycle maybe | 15:02 |
| sean-k-mooney | no it will still work with older pip | 15:02 |
| sean-k-mooney | on editbale installs | 15:02 |
| sean-k-mooney | ralonsoh: nova still has supprot for them | 15:02 |
| johnsom | I worry about backporting these patches given there is package/deployer impact. I was considering pinning setuptools in 2025.1 | 15:03 |
| sean-k-mooney | if you build the whelie in non editable mode the extenion we have also work i belive | 15:03 |
| sean-k-mooney | johnsom: ya im trying to avoid backporting it too | 15:03 |
| sean-k-mooney | johnsom: that why im intentially not removing the old way fo usign the console scrit in the intial patch | 15:04 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Print ThreadPool statistics https://review.opendev.org/c/openstack/nova/+/948340 | 15:05 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: WIP: allow service to start with threading https://review.opendev.org/c/openstack/nova/+/948311 | 15:05 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: DNM:Run nova-next with n-sch in threading mode https://review.opendev.org/c/openstack/nova/+/948450 | 15:05 |
| johnsom | Well, if we don't pin, it's still likely to break people. That is why I was pushing to do it all at once and get the pain overwith | 15:05 |
| sean-k-mooney | ya its unfortuent we dint get this done in carical liek we orgianlly wanted to do | 15:06 |
| johnsom | +1 | 15:06 |
| sean-k-mooney | in theory if you use an old enough pip/setuptoosl nova still supprot setup.py directly too but nothing tests that | 15:07 |
| johnsom | We plan to land the designate patch today (there is a typo to fix) | 15:07 |
| bauzas | nova meeting in 20 mins here | 15:40 |
| * bauzas will chair the meeting for 20 mins only | 15:41 | |
| haleyb | we noticed a failure in the neutron periodic jobs today with the oslo.utils master branch, but seems more a nova issue - https://bugs.launchpad.net/nova/+bug/2109592 | 15:45 |
| haleyb | i'm assuming that would eventually get caught in an integration job somewhere | 15:46 |
| dansmith | that looks more like an oslo, requirements, etc thing | 15:46 |
| dansmith | (could be nova requirements.txt) | 15:46 |
| sean-k-mooney | ya i was going ot say the same | 15:47 |
| sean-k-mooney | what branch was this on "master" i assume | 15:47 |
| haleyb | dansmith: could be just didn't seem neutron-specific | 15:47 |
| dansmith | haleyb: yes :D | 15:47 |
| frickler | from the timing when this started failing, I was assuming setuptools==80 to be related | 15:47 |
| haleyb | yes, we have a job that uses most (all) of the oslo master branches | 15:47 |
| sean-k-mooney | actully thi si from a neutorn job that is testing with oslo master | 15:47 |
| haleyb | it's a canary job | 15:48 |
| sean-k-mooney | so that may also be a regression in oslo | 15:48 |
| sean-k-mooney | i.e. they moved the module | 15:48 |
| frickler | because from sunday to monday no changes were merged in oslo, neutron or nova | 15:48 |
| gmaan | yeah it is not yet released so will not be visible in other jobs but oslo master one | 15:48 |
| sean-k-mooney | no its where we expect | 15:49 |
| haleyb | we noticed today, started seeing yesterday according to build logs | 15:49 |
| haleyb | https://zuul.openstack.org/builds?job_name=neutron-ovn-tempest-ovs-release-with-oslo-master&job_name=neutron-ovs-tempest-with-oslo-master&skip=0 | 15:49 |
| sean-k-mooney | the module is still in the same place https://github.com/openstack/oslo.utils/blob/master/oslo_utils/imageutils/format_inspector.py | 15:51 |
| sean-k-mooney | so its a little od | 15:51 |
| sean-k-mooney | oh | 15:51 |
| sean-k-mooney | /usr/lib/python3/dist-packages/oslo_utils/imageutils.py | 15:52 |
| sean-k-mooney | that look wrong | 15:52 |
| sean-k-mooney | it is using setup tools 80 Requirement already satisfied: setuptools in /opt/stack/data/venv/lib/python3.12/site-packages (from pbr>=6.1.0->oslo.utils==8.3.0.dev7) (80.0.0) | 15:53 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: DNM:Run nova-next with n-sch in threading mode https://review.opendev.org/c/openstack/nova/+/948450 | 16:00 |
| bauzas | #startmeeting nova | 16:01 |
| opendevmeet | Meeting started Tue Apr 29 16:01:04 2025 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
| opendevmeet | The meeting name has been set to 'nova' | 16:01 |
| gibi | o/ | 16:01 |
| bauzas | guys, we have a meeting now | 16:01 |
| sean-k-mooney | o/ | 16:01 |
| elodilles | o/ | 16:01 |
| bauzas | do you want to discuss ? | 16:01 |
| gmaan | o/ | 16:01 |
| bauzas | I can chair it for the next 20/30 mins | 16:01 |
| fwiesel | o | 16:01 |
| gibi | I have one gate topic | 16:01 |
| bauzas | and then I'll need to go offline | 16:01 |
| bauzas | okay, let's start | 16:01 |
| bauzas | #topic Bugs (stuck/critical) | 16:02 |
| bauzas | #info No Critical bug | 16:02 |
| bauzas | #topic Gate status | 16:02 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:02 |
| bauzas | Nova gate is blocked until we land https://review.opendev.org/c/openstack/nova/+/948392 | 16:02 |
| gibi | yepp that is my point^^ | 16:02 |
| bauzas | just +wd it | 16:03 |
| gibi | py3.9 are dropped from global req causing version conflict | 16:03 |
| bauzas | #link https://etherpad.opendev.org/p/nova-ci-failures-minimal | 16:03 |
| bauzas | cool | 16:03 |
| gibi | bauzas: thankls | 16:03 |
| bauzas | I saw the failures indeed | 16:03 |
| bauzas | gibi; no thanks to you for the fix | 16:03 |
| bauzas | anything else to say about it ? | 16:04 |
| gibi | nope | 16:04 |
| bauzas | all good | 16:05 |
| bauzas | moving on quickly then | 16:05 |
| bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status | 16:05 |
| bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:05 |
| bauzas | #info Please try to provide meaningful comment when you recheck | 16:05 |
| bauzas | periodics are green, moving on | 16:06 |
| bauzas | #topic tempest-with-latest-microversion job status | 16:06 |
| bauzas | #link https://zuul.opendev.org/t/openstack/builds?job_name=tempest-with-latest-microversion&skip=0 | 16:06 |
| gmaan | I will make it periodic as logs are not visible in experimental run | 16:07 |
| bauzas | moving on while I'm awaiting the result | 16:07 |
| gmaan | and will see how many tests need adjustment | 16:07 |
| bauzas | gmaan: ack thanks | 16:07 |
| bauzas | #topic Release Planning | 16:07 |
| opendevreview | Merged openstack/placement master: Add pyproject.toml to support pip 23.1 https://review.opendev.org/c/openstack/placement/+/932399 | 16:07 |
| bauzas | #link https://releases.openstack.org/flamingo/schedule.html | 16:07 |
| bauzas | #info Nova deadlines are set in the above schedule | 16:07 |
| bauzas | well, technically, those deadlines aren't set AFAICS | 16:08 |
| elodilles | actually they are not: https://review.opendev.org/c/openstack/releases/+/948077 | 16:08 |
| bauzas | elodilles yeah | 16:08 |
| elodilles | o:) | 16:08 |
| bauzas | moving on, that'll be Uggla's responsibility to check those deadlines every week :p | 16:08 |
| elodilles | the above needs some review (at least from nova release liaisons) | 16:08 |
| bauzas | #topic Review priorities | 16:09 |
| bauzas | #link https://etherpad.opendev.org/p/nova-2025.2-status | 16:09 |
| bauzas | elodilles +1d | 16:09 |
| gibi | elodilles: +1d | 16:09 |
| bauzas | apparently Uggla did a work on creating a tracking etherpad, noice | 16:09 |
| bauzas | #topic Stable Branches | 16:10 |
| elodilles | thanks all o/ | 16:10 |
| bauzas | elodilles: shoot your topic | 16:10 |
| elodilles | ah, ok | 16:10 |
| elodilles | #info not aware of any stable gate blocking thing | 16:10 |
| elodilles | i haven't seen any setuptools related issue yet :) | 16:10 |
| elodilles | but there were not much activity in the past days | 16:11 |
| elodilles | #info stable/2023.2 (bobcat) EOL patch for nova: https://review.opendev.org/c/openstack/releases/+/948196 | 16:11 |
| elodilles | another patch that needs some nova release liaison review ^^^ | 16:11 |
| elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | 16:11 |
| elodilles | that's all from me | 16:11 |
| elodilles | bauzas: back to you | 16:12 |
| bauzas | cool | 16:12 |
| bauzas | #topic vmwareapi 3rd-party CI efforts Highlights | 16:12 |
| bauzas | fwiesel: around ? | 16:12 |
| fwiesel | Yes, finally. | 16:12 |
| fwiesel | #info fwiesel reports back to duty | 16:12 |
| fwiesel | So to say | 16:12 |
| fwiesel | #info fwiesel will involve more people from SAP for redundancy | 16:13 |
| fwiesel | When I had my sick leave, I just dropped a short message on slack, hoping someone will take over, but that didn't work. | 16:13 |
| fwiesel | I'll try to make that more formal. | 16:13 |
| fwiesel | That's from my side. | 16:13 |
| bauzas | cool | 16:13 |
| bauzas | #topic Gibi's news about eventlet removal. | 16:14 |
| bauzas | gibi: shoot | 16:14 |
| gibi | ack | 16:14 |
| gibi | we saw the first nova-scheduler run with native threading | 16:14 |
| gibi | actually multiple green CI jobs | 16:14 |
| gibi | e.g. https://zuul.opendev.org/t/openstack/build/636bdbb6ad494ef0b49415034842a640/log/controller/logs/screen-n-sch.txt nova-multi-cell | 16:14 |
| gibi | nothing needs immediate review attention. The ball is on my side to clean up the series that is open | 16:15 |
| gibi | that is it | 16:15 |
| gibi | bauzas: back to you | 16:16 |
| bauzas | noted | 16:16 |
| bauzas | I'll skip the bug scrub as I don't have time to lead this | 16:16 |
| bauzas | and given Uggla is on PTO | 16:17 |
| bauzas | #topic Open discussion | 16:17 |
| bauzas | anything anyone ? | 16:17 |
| fwiesel | Yes, me then. | 16:17 |
| fwiesel | I was curious about what you think about enabling resize between hypervisors? I mean, a full discussion obviously means a blueprint. | 16:18 |
| bauzas | I'll need to disappear shortly | 16:18 |
| bauzas | do people have opinions about ^ ? | 16:18 |
| gibi | fwiesel: yeah probably a spec | 16:18 |
| bauzas | not sure what you mean by an "hypervisor resize" | 16:19 |
| fwiesel | Question there is is it worthwile to open one | 16:19 |
| bauzas | resizing all the instances on that host ? | 16:19 |
| fwiesel | Ah, the idea is we have a flavor for one hypervisor (i.e vmware), and the user wants to migrate to another one, say libvirt. | 16:19 |
| bauzas | or some hardware change for an hypervisor, like a new memory card ? | 16:20 |
| fwiesel | The user can then call a resize from one flavor to another | 16:20 |
| bauzas | ah | 16:20 |
| bauzas | we don't bubble up that to the users because $cloud | 16:20 |
| fwiesel | I can write a spec, but I wanted to know if anyone would even be willing to look at it :) | 16:21 |
| bauzas | but, operators can do multiple scheduling usages for that | 16:21 |
| bauzas | eg. aggregates, traits or anything else decorating | 16:21 |
| bauzas | not sure I want to disclose which hypervisor runs by a flavor | 16:21 |
| fwiesel | Let's turn it the other way around. Is it desireable that a resize should work between two different nova drivers? | 16:23 |
| bauzas | I need to go offline now | 16:23 |
| bauzas | again, anyone want to discuss this ? | 16:23 |
| bauzas | or we could discuss this next week | 16:23 |
| gibi | fwiesel: at least it worth dicussing it. I'm not against it at a first glance | 16:23 |
| fwiesel | I can bring it up next week again. I want to avoid writing a spec, if there is a strong opposition to it. | 16:24 |
| bauzas | thanks, let's then discuss this again nextweek | 16:25 |
| bauzas | thanks all | 16:25 |
| bauzas | #endmeeting | 16:25 |
| opendevmeet | Meeting ended Tue Apr 29 16:25:21 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:25 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2025/nova.2025-04-29-16.01.html | 16:25 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2025/nova.2025-04-29-16.01.txt | 16:25 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2025/nova.2025-04-29-16.01.log.html | 16:25 |
| fwiesel | thanks everyone | 16:25 |
| elodilles | thanks o/ | 16:25 |
| gibi | fwiesel: yeah I understand that you don't want a full blown spec. Maybe start a spec and only fill the up until the Use Case section | 16:26 |
| gibi | so define the what and keep the how empty | 16:26 |
| gibi | what and why | 16:26 |
| * gibi thinks that this was the speediest nova meeting ever :) | 16:27 | |
| gmaan | bauzas: latest-microversion job in periodic- https://review.opendev.org/c/openstack/tempest/+/948458 | 16:27 |
| opendevreview | Dan Smith proposed openstack/nova master: WIP Example NVMe cleaning script for one-time-use https://review.opendev.org/c/openstack/nova/+/948446 | 16:39 |
| melwitt | gibi: cool thanks for fixing it! | 16:45 |
| sean-k-mooney | if by resize between diffent hyperviors that actully means between differnt virt driver then that woudl need a full spec | 16:56 |
| sean-k-mooney | because that bring up the qustion what do you do with things like the disk format | 16:57 |
| sean-k-mooney | even if we do it via shelve liek we do for cross cell i expect there to be dragons | 16:57 |
| dansmith | on the one hand, it shouldn't be a problem at all, | 17:02 |
| dansmith | except for the disk format, the metadata that might make it not a candidate for the other hypervisor, etc | 17:02 |
| dansmith | so I expect there are dragons, but not fundamental ones | 17:02 |
| sean-k-mooney | well even simple things like libvirt expect to be able to scp the disks | 17:03 |
| dansmith | either way, I'm not sure how anyone could argue that a spec isn't warranted :) | 17:03 |
| sean-k-mooney | im sure vmware/hypver/ironic woudl have optinons about that | 17:03 |
| dansmith | sean-k-mooney: I meant for the shelve-based approach, which I think would be the best for something | 17:03 |
| dansmith | like cross-cell resize does | 17:03 |
| sean-k-mooney | oh right | 17:03 |
| sean-k-mooney | ya shelve is likely mortly ok | 17:04 |
| sean-k-mooney | althoguh windows may be unhappy if the mosther board that is virutalise and all the periferals | 17:04 |
| sean-k-mooney | change form whatevver vmware provides to q35 | 17:04 |
| dansmith | sure, that's all under the metadata umbrella I meant above | 17:04 |
| sean-k-mooney | ah | 17:05 |
| sean-k-mooney | i tought you ment the instnace metadata | 17:05 |
| sean-k-mooney | but ya liek it might work | 17:05 |
| dansmith | no, image meta | 17:05 |
| sean-k-mooney | but im not sure if thaty is what they ment about hypervior above | 17:05 |
| dansmith | like requesting something libvirt- or vmware-specific that just makes the scheduler not even willing to send it to another host, etc | 17:05 |
| sean-k-mooney | oh no it is | 17:05 |
| dansmith | right | 17:05 |
| sean-k-mooney | anyway i guess they may have jost wanted to avoid a spec initall to get some high level input | 17:06 |
| sean-k-mooney | i.e. is this somethign we woudl ever want to supprot | 17:06 |
| sean-k-mooney | i dont think its unreasonable but alos not a high on my wishlist | 17:06 |
| dansmith | hard to say if we could/would without knowing the dragons, which is what a spec is for :D | 17:06 |
| dansmith | agreed | 17:07 |
| sean-k-mooney | if you had a vmware backed openstack | 17:07 |
| dansmith | I wonder if the ask is coming from an escape-the-20x-license-increase thing | 17:07 |
| sean-k-mooney | and wanted to move to libvirt i guess it woudl be nice to jsut rezie the vms to do that | 17:07 |
| dansmith | there are also community and commercial migration functions, so.. | 17:07 |
| sean-k-mooney | ya that what im wondering too because fwiesel has been trying to maintain the vmware dirver bcause sap use it internally | 17:08 |
| sean-k-mooney | so this may be realted to enabling them to move to libvirt goign forward | 17:08 |
| sean-k-mooney | so migratekit or os-migrate might help there | 17:08 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: Add mtty/mdpy support for testing fake mdevs https://review.opendev.org/c/openstack/nova/+/898100 | 17:10 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Add mtty support to nova-next https://review.opendev.org/c/openstack/nova/+/922140 | 17:10 |
| *** benj_2 is now known as benj_ | 17:39 | |
| opendevreview | Merged openstack/nova master: Remove python 3.9 support https://review.opendev.org/c/openstack/nova/+/948392 | 18:28 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove tags from README https://review.opendev.org/c/openstack/placement/+/941792 | 19:43 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove tags from README https://review.opendev.org/c/openstack/placement/+/941792 | 19:43 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove tags from README https://review.opendev.org/c/openstack/placement/+/941792 | 19:43 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove tags from README https://review.opendev.org/c/openstack/placement/+/941792 | 19:43 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove tags from README https://review.opendev.org/c/openstack/placement/+/941792 | 19:47 |
| opendevreview | sean mooney proposed openstack/nova master: Add admin docs to cover limitations of OTU devices https://review.opendev.org/c/openstack/nova/+/948492 | 22:13 |
| sean-k-mooney | dansmith: i have tired to capture some fo the design constratis and limiation around OTU device and what a cleaning script, process shoudl do in terms of pre/post constratis | 22:15 |
| sean-k-mooney | dansmith: ^ im not sure if you had anything similar planned as a follow up but hopefully this will be useful for folks. | 22:16 |
| sean-k-mooney | i have tired to spellcheck it but i dint pass it though grammerly. hopefully it pretty readable | 22:17 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove tags from README https://review.opendev.org/c/openstack/placement/+/941792 | 22:19 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove tags from README https://review.opendev.org/c/openstack/placement/+/941792 | 22:26 |
| opendevreview | Ivan Anfimov proposed openstack/placement master: Remove tags from README https://review.opendev.org/c/openstack/placement/+/941792 | 22:26 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!