Friday, 2023-12-08

opendevreviewPranali Deore proposed openstack/nova master: Support glance's new location API  https://review.opendev.org/c/openstack/nova/+/89103607:02
opendevreviewPranali Deore proposed openstack/nova master: DNM: Test glance new location api  https://review.opendev.org/c/openstack/nova/+/89120707:02
opendevreviewMerged openstack/nova master: Use SDK for remaining network operations  https://review.opendev.org/c/openstack/nova/+/86779407:38
gibihm, nobody can say that our gate is blocking us merging things, we merged 9 patches since I left yesterday08:23
opendevreviewAlex Welsh proposed openstack/nova master: Make map_cell0 command update existing mappings  https://review.opendev.org/c/openstack/nova/+/90314009:48
opendevreviewAlex Welsh proposed openstack/nova master: Make map_cell0 command update existing mappings  https://review.opendev.org/c/openstack/nova/+/90314010:29
opendevreviewPranali Deore proposed openstack/nova master: Support glance's new location API  https://review.opendev.org/c/openstack/nova/+/89103610:30
opendevreviewPranali Deore proposed openstack/nova master: DNM: Test glance new location api  https://review.opendev.org/c/openstack/nova/+/89120710:30
opendevreviewAlex Welsh proposed openstack/nova master: Make map_cell0 command update existing mappings  https://review.opendev.org/c/openstack/nova/+/90314010:32
opendevreviewAmit Uniyal proposed openstack/nova master: enforce remote console shutdown  https://review.opendev.org/c/openstack/nova/+/90182410:44
gibibauzas: additional power management bugfix is incoming https://bugs.launchpad.net/nova/+bug/204596611:09
bauzasohch11:11
bauzasI thought scaling_governor was supported since 201511:11
bauzashttps://lwn.net/Articles/682391/11:12
bauzasmy bad, 201611:12
bauzaswhen I meant 'supported', I meant 'enabled by default in the kernel'11:12
opendevreviewBalazs Gibizer proposed openstack/nova master: [pwmgmt]ignore missin governor when  cpu_state used  https://review.opendev.org/c/openstack/nova/+/90316911:12
opendevreviewBalazs Gibizer proposed openstack/nova master: [pwmgmt]ignore missin governor when cpu_state used  https://review.opendev.org/c/openstack/nova/+/90316911:13
gibibauzas: if you install centos 9 stream, by default it has no governor11:13
* bauzas facepalms11:13
gibibauzas: I can give you access to a beaker node downstream :)11:14
gibianyhow the fix is fairly simple11:14
bauzasnah nah I trust you11:14
gibiand I will make sure it will be backported to antelope11:14
bauzasfwiw, my fedora (38 IIRC) ships scaling_governor 11:15
bauzas[sbauza@sbauza nova-specs]$ ll /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor 11:15
bauzas-rw-r--r--. 1 root root 4096  8 déc.  12:14 /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor11:15
bauzaswtf11:15
bauzashumpgf11:16
bauzas[stack@lenovo-sr655-01 nova]$ ll /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor 11:16
bauzas-rw-r--r--. 1 root root 4096 Dec  5 13:30 /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor11:16
bauzas[stack@lenovo-sr655-01 nova]$ cat /etc/redhat-release 11:16
bauzasRed Hat Enterprise Linux release 9.2 (Plow)11:16
bauzasgibi ^11:16
bauzasI don't want Stream f... up with cpufreq but they broke ity11:17
bauzasdon't know* what*11:17
gibi[root@osp-dev-01 ~]# cat /etc/redhat-release 11:18
gibiCentOS Stream release 911:18
gibi[root@osp-dev-01 ~]# ll /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor11:18
gibils: cannot access '/sys/devices/system/cpu/cpu1/cpufreq/scaling_governor': No such file or directory11:18
gibi[root@osp-dev-01 ~]# ll /sys/devices/system/cpu/cpu1/cpufreq/11:18
gibils: cannot access '/sys/devices/system/cpu/cpu1/cpufreq/': No such file or directory11:18
bauzasthat's crazy11:19
bauzasI +2d your change anyway, but I'm very surprised11:20
gibibut I checked onlineing / offlineing works11:20
bauzascpufreq is one very common kernel module11:20
opendevreviewAlex Welsh proposed openstack/nova master: Make map_cell0 command update existing mappings  https://review.opendev.org/c/openstack/nova/+/90314011:28
bauzasgibi: out of curiosity, could you please issue on your c9s box : 11:28
bauzasfind /lib/modules/$(uname -r)/kernel/ | grep cpufreq11:29
gibi[root@osp-dev-01 ~]# find /lib/modules/$(uname -r)/kernel/ | grep cpufreq11:46
gibi/lib/modules/5.14.0-386.el9.x86_64/kernel/drivers/cpufreq11:46
gibi/lib/modules/5.14.0-386.el9.x86_64/kernel/drivers/cpufreq/acpi-cpufreq.ko.xz11:47
gibi/lib/modules/5.14.0-386.el9.x86_64/kernel/drivers/cpufreq/amd-pstate-ut.ko.xz11:47
gibi/lib/modules/5.14.0-386.el9.x86_64/kernel/drivers/cpufreq/amd_freq_sensitivity.ko.xz11:47
gibi/lib/modules/5.14.0-386.el9.x86_64/kernel/drivers/cpufreq/p4-clockmod.ko.xz11:47
gibi/lib/modules/5.14.0-386.el9.x86_64/kernel/drivers/cpufreq/powernow-k8.ko.xz11:47
gibi/lib/modules/5.14.0-386.el9.x86_64/kernel/drivers/cpufreq/speedstep-lib.ko.xz11:47
bauzasgibi: thanks so cpufreq is shipped11:52
bauzasgibi: lsmod | grep cpufreq ?11:52
gibithat is empty11:52
bauzasI'll ask cfontain if he knows why cpufreq isn't there in c9s11:53
gibibut it is also empty in my desktop where I have cpufreq11:53
bauzasmaybe because of tuned11:53
admin1hi all .. how to get a vm out of " Confirm or Revert Resize/Migrate " stage where both confirm or revert fails to work .. these vms are backed by ceph and came to this state where their hypervisor died and were in shutdown state 11:54
bauzasgibi: oh that's a boot kernel config11:55
bauzasgibi: cat /boot/config-$(uname -r)  | grep CONFIG_CPU_FREQ11:56
gibihttps://paste.opendev.org/show/bknmxAOZRtxwb0pNBaSH/11:57
bauzasI'm then out of ideas why the sysfs directory isn't popping up12:01
bauzasanyway, the patch is good to me, it's good to be conservative here12:01
bauzasI'll talk to Christophe in case (which is more than possible) he knows the reason :)12:02
bauzasoh damn, he's on PTO :D12:03
gibibauzas: OK. thanks for looking into it12:03
bauzasnp, thanks for testing it :)12:04
bauzasI could have been more cautious12:04
bauzasand not assume every single box (from RPi to large mainframes) includes cpufreq12:04
gibiit was a surprise for me too12:05
gibifirst I thought it is due to using a VM as compute. But then I checked centos on a real hw and I see the same12:05
bauzasmaybe the journal said the reason12:06
bauzasmaybe it was deactivate12:06
bauzasdeactivated*12:06
bauzasanyway12:06
gibinothing in journalctl | grep cpufreq12:06
gibijournalctl | grep cpufreq12:06
gibisorry12:06
gibibtw, offlining cpus when a compute is a VM works (but only offlines the cpu from the VM perspective obviously)12:07
bauzasmakes sense12:07
gibiso at some point we can have some end to end test for this in openstack-k8s-operators CI upstrea12:07
gibicurrently I only have https://github.com/openstack-k8s-operators/nova-operator/pull/597#issuecomment-1846870113 as we don't have dedicated CPUs in CI yet12:08
bauzashttps://docs.kernel.org/core-api/cpu_hotplug.html12:11
bauzasthat's what the offline does12:11
bauzasI meant https://docs.kernel.org/core-api/cpu_hotplug.html#the-offline-case12:12
gibiyeah I saw in dmesg that IRQ was migrated away 12:15
opendevreviewSylvain Bauza proposed openstack/nova-specs master: Proposes mdev live-migration support in libvirt  https://review.opendev.org/c/openstack/nova-specs/+/90063612:31
opendevreviewSylvain Bauza proposed openstack/nova-specs master: Proposes mdev live-migration support in libvirt  https://review.opendev.org/c/openstack/nova-specs/+/90063612:38
sean-k-mooney[m]gibi: bauzas it might not be there if hardware pstates is enabled i guess or if its didsbaled in the bios12:46
sean-k-mooney[m]gibi: people have a habit of turning off speedstepp ot do a test and not renabling it when they are done12:47
sean-k-mooney[m]c9s does have cpufreq avaible but perhapas not in the beaker image12:49
sean-k-mooney[m]gibi: bauzas  so im off today and about to head out but if ye feel like reivewing some simple patches it would be nice if my codespell seriese could get some eyes https://review.opendev.org/c/openstack/nova/+/89709213:07
gibisean-k-mooney[m]: I approved up until https://review.opendev.org/c/openstack/nova/+/897096 in that melwitt has a question13:11
sean-k-mooney[m]gibi: replied with a link to the docs https://github.com/codespell-project/codespell#ignoring-words13:15
sean-k-mooney[m]tldr is the ignore list must be lowercase13:15
gibiack, thanks13:17
sean-k-mooney[m]https://opendev.org/openstack/nova/src/commit/3fdc97ca5f8e65546840cd453e86cb08cb4869e6/doc/source/admin/cells.rst?display=source#L602 this is what  it was detecting13:20
gibiI'm +2 all over it :)13:23
bauzassean-k-mooney: yeah I was thinking about some BIOS change13:37
bauzaspstates shouldn't be related to the cpu states IIRC13:37
bauzasc-states as well13:37
zigoHi. We're currently testing our upgrades, and I'm being very confused by the "nova-manage db online_data_migrations". First question: under what release will we need the online_data_migrations (we're doing victoria->wallaby->xena->yoga->zed for this run...)? Second question: I'm always being displayed "Total Needed: 50" but nothing in "Completed", am I doing something wrong?!?14:34
zigo# nova-manage db online_data_migrations  --max-count 100014:36
zigo995 rows matched query fill_virtual_interface_list, 0 migrated <--- something wrong ?14:36
bauzaszigo: do you already know nova-status upgrade check ? 14:47
bauzashttps://docs.openstack.org/nova/latest/cli/nova-status.html#upgrade14:47
bauzaszigo: online db migrations are sometimes needed indeed, before upgrading14:49
zigoI do know nova-status upgrade check yes.14:49
bauzasso it will doublecheck whether you need to call online data migrations14:49
zigoSo, if it returns zero, I'm good, right?14:50
opendevreviewArnaud Morin proposed openstack/nova master: Return proper number of pci devices for virtio  https://review.opendev.org/c/openstack/nova/+/90317614:50
zigoIn such case, victoria to zed don't need any online_data_migrations...14:50
zigoThough this still feels weird...14:52
zigobauzas: https://paste.opendev.org/show/bmkrJ3MF5bYFOZvbnUa0/14:53
zigoI can restart it many times, it always says "Total Needed 5" and "Completed 0", which doesn't feel right ...14:53
zigoHow come it's not completing any?14:53
bauzaszigo: I unfortunately need to get my daughter from the school in a sec, so could we discuss that on monday ?14:55
zigoNo worries, sure ! :)14:55
zigoThanks.14:55
bauzashttps://docs.openstack.org/nova/latest/cli/nova-manage.html#db-online-data-migrations14:58
bauzasit tells what you need to update14:58
bauzasand for fill_interface_virtual_list, this is https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L186C17-L186C1714:59
bauzaszigo: that's what it's calling https://github.com/openstack/nova/blob/master/nova/objects/virtual_interface.py#L15615:00
admin1 i have a vm with status active power_state nostate on a hypervisor that failed ..  vm is backed by ceph .. how do i actually migrate this to other host ? 16:07
admin1due to the power state, normal migration fails 16:07
admin1and when done evauate host, a few other vms are in confirm resize/migration phase, where they dont work with confirm or resize and just stay like that16:08
opendevreviewStephen Finucane proposed openstack/nova master: setup: Remove pbr's wsgi_scripts  https://review.opendev.org/c/openstack/nova/+/90268819:39
opendevreviewmelanie witt proposed openstack/nova master: Re-enable "test_tagged_attachment" test.  https://review.opendev.org/c/openstack/nova/+/87669921:12
opendevreviewMerged openstack/nova master: [pwmgmt]ignore missin governor when cpu_state used  https://review.opendev.org/c/openstack/nova/+/90316922:03

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!