*** macz_ has quit IRC | 00:22 | |
*** tosky has quit IRC | 00:36 | |
brinzhang_ | gmann: hi are you around? | 00:41 |
---|---|---|
brinzhang_ | gmann: what's your mean of this https://review.opendev.org/c/openstack/nova/+/729563/26/nova/api/openstack/compute/shelve.py#59 | 00:42 |
brinzhang_ | gmann: do we need to merge your fix into the shelve/unshelve patch? | 00:42 |
*** LinPeiWen94 has quit IRC | 00:45 | |
*** songwenping__ has joined #openstack-nova | 00:49 | |
*** songwenping_ has quit IRC | 00:53 | |
*** sapd1 has quit IRC | 00:55 | |
*** chengsheng has quit IRC | 01:19 | |
openstackgerrit | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List/Show usage APIs https://review.opendev.org/c/openstack/nova/+/768509 | 01:34 |
*** hoonetorg has joined #openstack-nova | 01:42 | |
*** sapd1 has joined #openstack-nova | 01:47 | |
*** songwenping_ has joined #openstack-nova | 01:51 | |
*** songwenping__ has quit IRC | 01:55 | |
*** xinranwang has joined #openstack-nova | 02:00 | |
*** dklyle has quit IRC | 02:01 | |
*** david-lyle has joined #openstack-nova | 02:01 | |
openstackgerrit | Brin Zhang proposed openstack/nova master: Replace tenants* with projects* of policies https://review.opendev.org/c/openstack/nova/+/765315 | 02:14 |
*** zenkuro has quit IRC | 02:18 | |
*** david-lyle has quit IRC | 02:18 | |
*** mlavalle has quit IRC | 02:34 | |
openstackgerrit | Brin Zhang proposed openstack/nova master: Replace os-simple-tenant-usage with os-simple-project-usage https://review.opendev.org/c/openstack/nova/+/768852 | 02:47 |
*** LinPeiWen28 has joined #openstack-nova | 03:01 | |
*** rcernin has quit IRC | 03:04 | |
*** rcernin has joined #openstack-nova | 03:25 | |
*** rcernin has quit IRC | 03:27 | |
*** brinzhang0 has joined #openstack-nova | 03:27 | |
*** brinzhang0 has quit IRC | 03:29 | |
*** brinzhang0 has joined #openstack-nova | 03:29 | |
*** brinzhang_ has quit IRC | 03:30 | |
*** brinzhang0 has quit IRC | 03:30 | |
*** brinzhang0 has joined #openstack-nova | 03:31 | |
*** rcernin has joined #openstack-nova | 03:31 | |
*** rcernin has quit IRC | 03:36 | |
*** rcernin has joined #openstack-nova | 03:37 | |
*** rcernin has quit IRC | 03:38 | |
*** rcernin has joined #openstack-nova | 03:38 | |
*** rcernin has quit IRC | 03:39 | |
*** rcernin has joined #openstack-nova | 03:39 | |
*** psachin has joined #openstack-nova | 03:57 | |
*** whoami-rajat__ has joined #openstack-nova | 04:22 | |
*** artom has quit IRC | 04:24 | |
openstackgerrit | Brin Zhang proposed openstack/nova master: Replace tenant_id with project_id in os-quota-sets path https://review.opendev.org/c/openstack/nova/+/768851 | 04:32 |
*** amotoki has quit IRC | 04:44 | |
*** amotoki has joined #openstack-nova | 04:44 | |
*** ociuhandu has joined #openstack-nova | 05:14 | |
*** sapd1 has quit IRC | 05:19 | |
*** ociuhandu has quit IRC | 05:19 | |
*** xinranwang has quit IRC | 05:19 | |
*** dave-mccowan has quit IRC | 05:43 | |
*** CeeMac has quit IRC | 05:53 | |
*** ratailor has joined #openstack-nova | 05:55 | |
*** nightmare_unreal has joined #openstack-nova | 06:02 | |
openstackgerrit | Wenping Song proposed openstack/nova-specs master: Libvirt supports composing cyborg owned vGPU accelerator into domain XML https://review.opendev.org/c/openstack/nova-specs/+/750116 | 06:05 |
openstackgerrit | Brin Zhang proposed openstack/nova master: Replace tenant_id with project_id in Limits API https://review.opendev.org/c/openstack/nova/+/768862 | 06:08 |
openstackgerrit | Brin Zhang proposed openstack/nova master: Replace tenant* with project* in codes https://review.opendev.org/c/openstack/nova/+/769329 | 06:08 |
*** sapd1 has joined #openstack-nova | 06:17 | |
*** brinzhang_ has joined #openstack-nova | 06:25 | |
*** brinzhang0 has quit IRC | 06:28 | |
*** gyee has quit IRC | 06:31 | |
*** xek has joined #openstack-nova | 06:42 | |
*** sapd1 has quit IRC | 06:54 | |
*** rcernin has quit IRC | 07:40 | |
*** rpittau|afk is now known as rpittau | 07:47 | |
*** belmoreira has joined #openstack-nova | 07:49 | |
*** ralonsoh has joined #openstack-nova | 07:51 | |
*** ralonsoh_ has joined #openstack-nova | 07:56 | |
*** rcernin has joined #openstack-nova | 07:56 | |
*** sapd1 has joined #openstack-nova | 07:58 | |
*** ralonsoh has quit IRC | 07:59 | |
*** rcernin has quit IRC | 08:01 | |
*** slaweq has joined #openstack-nova | 08:03 | |
*** zoharm has joined #openstack-nova | 08:13 | |
*** andrewbonney has joined #openstack-nova | 08:17 | |
*** spatel has joined #openstack-nova | 08:17 | |
*** tesseract has joined #openstack-nova | 08:18 | |
*** spatel has quit IRC | 08:22 | |
*** rcernin has joined #openstack-nova | 08:26 | |
*** ralonsoh has joined #openstack-nova | 08:28 | |
*** ralonsoh_ has quit IRC | 08:28 | |
*** zzzeek has quit IRC | 08:28 | |
*** zzzeek has joined #openstack-nova | 08:31 | |
openstackgerrit | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from Rebuild Server API https://review.opendev.org/c/openstack/nova/+/766380 | 08:39 |
openstackgerrit | Brin Zhang proposed openstack/nova master: Replaces tenant_id with project_id from List SG API https://review.opendev.org/c/openstack/nova/+/766726 | 08:39 |
*** brinzhang0 has joined #openstack-nova | 08:45 | |
*** brinzhang_ has quit IRC | 08:48 | |
*** tosky has joined #openstack-nova | 08:49 | |
*** slaweq has quit IRC | 08:55 | |
*** slaweq has joined #openstack-nova | 09:00 | |
gibi | good morning | 09:06 |
*** ociuhandu has joined #openstack-nova | 09:16 | |
*** zenkuro has joined #openstack-nova | 09:22 | |
*** ircuser-1 has quit IRC | 09:25 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova stable/ussuri: Warn when starting services with older than N-1 computes https://review.opendev.org/c/openstack/nova/+/770764 | 09:25 |
openstackgerrit | Balazs Gibizer proposed openstack/nova stable/ussuri: Add upgrade check about old computes https://review.opendev.org/c/openstack/nova/+/770765 | 09:25 |
*** ociuhandu has quit IRC | 09:26 | |
lyarwood | sean-k-mooney: https://review.opendev.org/q/topic:spec/libvirt-default-machine-type - when you're around and have time can we have a chat about the direction of this series before I finish things up and write docs etc? | 09:44 |
*** derekh has joined #openstack-nova | 09:50 | |
openstackgerrit | Balazs Gibizer proposed openstack/nova stable/ussuri: Reproduce bug 1896463 in func env https://review.opendev.org/c/openstack/nova/+/770768 | 09:56 |
openstack | bug 1896463 in OpenStack Compute (nova) rocky "evacuation failed: Port update failed : Unable to correlate PCI slot " [Low,In progress] https://launchpad.net/bugs/1896463 | 09:56 |
openstackgerrit | Balazs Gibizer proposed openstack/nova stable/ussuri: Set instance host and drop migration under lock https://review.opendev.org/c/openstack/nova/+/770769 | 09:56 |
*** ociuhandu has joined #openstack-nova | 09:57 | |
*** zzzeek has quit IRC | 10:02 | |
*** zzzeek has joined #openstack-nova | 10:02 | |
*** Mamduhala has joined #openstack-nova | 10:04 | |
*** ociuhandu has quit IRC | 10:05 | |
*** rcernin has quit IRC | 10:14 | |
*** Mamduhala has quit IRC | 10:15 | |
*** ociuhandu has joined #openstack-nova | 10:19 | |
*** rcernin has joined #openstack-nova | 10:23 | |
*** ociuhandu has quit IRC | 10:23 | |
*** lpetrut has joined #openstack-nova | 10:33 | |
*** rcernin has quit IRC | 10:40 | |
*** ociuhandu has joined #openstack-nova | 10:44 | |
*** dtantsur|afk is now known as dtantsur | 10:56 | |
*** brinzhang has joined #openstack-nova | 10:57 | |
*** brinzhang has quit IRC | 10:58 | |
*** brinzhang has joined #openstack-nova | 10:58 | |
*** brinzhang0 has quit IRC | 10:59 | |
*** songwenping__ has joined #openstack-nova | 11:07 | |
*** songwenping_ has quit IRC | 11:10 | |
*** sapd1 has quit IRC | 11:12 | |
*** brinzhang has quit IRC | 11:25 | |
*** spatel has joined #openstack-nova | 11:25 | |
*** brinzhang has joined #openstack-nova | 11:25 | |
*** brinzhang_ has joined #openstack-nova | 11:26 | |
*** songwenping_ has joined #openstack-nova | 11:28 | |
*** brinzhang_ has quit IRC | 11:28 | |
*** brinzhang_ has joined #openstack-nova | 11:29 | |
*** spatel has quit IRC | 11:29 | |
*** brinzhang has quit IRC | 11:30 | |
*** songwenping__ has quit IRC | 11:30 | |
sean-k-mooney | lyarwood: https://review.opendev.org/c/openstack/nova/+/767533/4/nova/virt/libvirt/driver.py#5775 | 11:31 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: utils: Remove periods from instance hostnames https://review.opendev.org/c/openstack/nova/+/764482 | 11:33 |
sean-k-mooney | lyarwood: over all that pathch looks good to me be the precidence relationship is wrong in my view | 11:34 |
sean-k-mooney | lyarwood: im going to go grab coffee and ill be back in a minute but after that im free to talk whenever suits | 11:34 |
lyarwood | sean-k-mooney: cool thanks, image, sysmeta and then config. I'll respin this afternoon. | 11:41 |
*** jangutter has quit IRC | 11:45 | |
lyarwood | stephenfin: https://review.opendev.org/c/openstack/openstacksdk/+/767526 sorry about the spam on here btw, I'm using gibi's tampermonkey script to display zuul results and got confused when I saw failures from a previous run. | 11:47 |
*** rcernin has joined #openstack-nova | 11:55 | |
stephenfin | lyarwood: heh, nw :) | 11:57 |
*** eharney has quit IRC | 11:57 | |
*** rcernin has quit IRC | 11:58 | |
*** e0ne has joined #openstack-nova | 12:00 | |
*** brinzhang0 has joined #openstack-nova | 12:03 | |
*** brinzhang_ has quit IRC | 12:05 | |
sean-k-mooney | stephenfin: https://review.opendev.org/c/openstack/python-openstackclient/+/765366 and the other osc changes faild the gate by the way | 12:08 |
sean-k-mooney | that one seams to have pagination issues for list servers which you did not change | 12:08 |
sean-k-mooney | it had a bunch of json decode errors | 12:08 |
*** eharney has joined #openstack-nova | 12:09 | |
sean-k-mooney | oh you have a recheck going | 12:09 |
sean-k-mooney | looks like the same error on all of them | 12:10 |
*** ociuhandu has quit IRC | 12:11 | |
*** eharney has quit IRC | 12:28 | |
*** eharney has joined #openstack-nova | 12:29 | |
*** ratailor has quit IRC | 12:31 | |
openstackgerrit | sean mooney proposed openstack/nova stable/victoria: only wait for plugtime events in pre-live-migration https://review.opendev.org/c/openstack/nova/+/770744 | 12:35 |
*** ociuhandu has joined #openstack-nova | 12:35 | |
openstackgerrit | sean mooney proposed openstack/nova stable/ussuri: only wait for plugtime events in pre-live-migration https://review.opendev.org/c/openstack/nova/+/770745 | 12:37 |
*** psachin has quit IRC | 12:39 | |
openstackgerrit | sean mooney proposed openstack/nova stable/ussuri: only wait for plugtime events in pre-live-migration https://review.opendev.org/c/openstack/nova/+/770745 | 12:45 |
*** ociuhandu has quit IRC | 12:55 | |
*** e0ne has quit IRC | 13:12 | |
*** spatel has joined #openstack-nova | 13:13 | |
*** brinzhang0 has quit IRC | 13:13 | |
*** brinzhang0 has joined #openstack-nova | 13:14 | |
*** spatel has quit IRC | 13:18 | |
*** macz_ has joined #openstack-nova | 13:19 | |
*** macz_ has quit IRC | 13:24 | |
*** ociuhandu has joined #openstack-nova | 13:25 | |
*** ociuhandu has quit IRC | 13:27 | |
*** ociuhandu has joined #openstack-nova | 13:27 | |
*** artom has joined #openstack-nova | 13:30 | |
*** nweinber has joined #openstack-nova | 13:54 | |
*** spatel has joined #openstack-nova | 14:00 | |
gibi | lyarwood: I'm wondering if I should do the cleanup MIN_LIBVIRT_VERSION >= 4.1.0 before I change the detach logic as that cleanup would simplyfy the code that I have to handle | 14:03 |
gibi | https://github.com/openstack/nova/blob/e6f5e814050a19d6f027037424556b2889514ec3/nova/virt/libvirt/guest.py#L418 | 14:03 |
lyarwood | gibi: hey sorry, so that's all part of https://review.opendev.org/q/topic:%22bump-libvirt-qemu-wallaby%22 if I ever get around to it | 14:16 |
lyarwood | gibi: which actually came up downstream earlier today | 14:16 |
lyarwood | gibi: I can refresh that and try to remove that logic for you later today if you'd like | 14:16 |
lyarwood | gibi: avoid adding another thing to your plate | 14:16 |
stephenfin | sean-k-mooney: Sorry, missed the ping. Yeah, the functional tests for OSC aren't great. They're not fully isolated which means running them in parallel can produce all kinds of weird results, depending on the speed on the CI node | 14:16 |
sean-k-mooney | ah i see | 14:17 |
stephenfin | sean-k-mooney: so rechecking is the only thing to do until I've time to rework them all | 14:17 |
sean-k-mooney | well that or run the serially but that proably will be too long | 14:17 |
stephenfin | yeah, and that just masks the problem | 14:18 |
sean-k-mooney | yep | 14:18 |
openstackgerrit | Merged openstack/nova master: tests: Clean up 'test_hypervisors' https://review.opendev.org/c/openstack/nova/+/769903 | 14:18 |
sean-k-mooney | it need to do what tempest does and create seperate tenant per senario when its doing things like testing list instnaces | 14:18 |
stephenfin | yes, exactly | 14:18 |
stephenfin | but I just haven't had time to write a fixture or whatever to do that | 14:19 |
*** dave-mccowan has joined #openstack-nova | 14:19 | |
lyarwood | just dumping this here while I work on something downstream | 14:20 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION and NEXT_MIN_{LIBVIRT,QEMU}_VERSION https://review.opendev.org/c/openstack/nova/+/754700 | 14:20 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Remove MIN_{LIBVIRT,QEMU}_BLOCKDEV https://review.opendev.org/c/openstack/nova/+/754701 | 14:20 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Remove MIN_LIBVIRT_VIR_ERR_DEVICE_MISSING https://review.opendev.org/c/openstack/nova/+/754702 | 14:20 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Remove MIN_LIBVIRT_VTPM https://review.opendev.org/c/openstack/nova/+/754703 | 14:20 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Remove MIN_LIBVIRT_S390X_CPU_COMPARE https://review.opendev.org/c/openstack/nova/+/754704 | 14:20 |
lyarwood | .. git-review really is slow since the update to gerrit .. | 14:20 |
stephenfin | I find it varies | 14:21 |
stephenfin | It's usually fast after a restart | 14:21 |
stephenfin | *feels fast | 14:21 |
stephenfin | I've no data to back up that assertion :-\ | 14:21 |
stephenfin | We're getting close to API microversion 2.100. It could be another cycle or two, but that's coming | 14:23 |
sean-k-mooney | at some point we might go to nova v4 | 14:30 |
*** takamatsu has joined #openstack-nova | 14:35 | |
*** CeeMac has joined #openstack-nova | 14:39 | |
dansmith | lyarwood: has seemed painfully slow to me as well | 14:41 |
dansmith | lyarwood: has seemed painfully slow to me as well | 14:41 |
dansmith | longer for multi-patch uploads | 14:41 |
*** ociuhandu has quit IRC | 14:45 | |
gmann | brinzhang0: no, your patch is all ok as it is and fixed the 500 ->403, mine is to improve it as general when other API use this exception further with unit tests | 14:47 |
gmann | brinzhang0: to avoid conflict I made mine depends on top of your otherwise they both can go independent also | 14:48 |
sean-k-mooney | dansmith: i havent really noticed much of a change but i havent done many multi patch updates recently | 14:54 |
gibi | lyarwood: thanks for https://review.opendev.org/c/openstack/nova/+/754702 I will review that and rebase the libvirt event patch top of that | 14:59 |
*** jangutter has joined #openstack-nova | 15:00 | |
lyarwood | gibi: I'll clean that series up later once I'm done with something downstream | 15:00 |
lyarwood | and np :) | 15:00 |
gibi | lyarwood: ack | 15:01 |
artom | stephenfin, when you have time, mind taking another look at https://review.opendev.org/c/openstack/nova-specs/+/765551 ? It feels awefully short to me, but I think I've covered everything...? | 15:29 |
gibi | lyarwood: as far as I see https://review.opendev.org/c/openstack/nova/+/754702 does not need to depend on the libvirt bump as our current minimum libvirt (5.0.0) already bigger than what is needed (4.1.0) for this patch | 15:29 |
*** lpetrut has quit IRC | 15:33 | |
*** hoonetorg has quit IRC | 15:35 | |
*** macz_ has joined #openstack-nova | 15:36 | |
*** hoonetorg has joined #openstack-nova | 15:40 | |
*** mtreinish has joined #openstack-nova | 15:41 | |
*** andrewbogott has joined #openstack-nova | 15:42 | |
*** hoonetorg has quit IRC | 15:43 | |
andrewbogott | melwitt: on the offhand chance that you are around… I'm in the midst of a db migration from rocky to stein and the migration is stuck in a loop. the stuck phase seems to be from your patch https://review.opendev.org/c/openstack/nova/+/633351 (which I can see you refactored quite a lot in T so I'm wondering if this is a known issue) | 15:44 |
openstackgerrit | Balazs Gibizer proposed openstack/nova master: DNM try to replace retry with libvirt event in detach https://review.opendev.org/c/openstack/nova/+/770246 | 15:44 |
andrewbogott | obviously you're under no obligation to provide me with tech support :) | 15:44 |
sean-k-mooney | andrewbogott: do you have an error | 15:47 |
andrewbogott | sean-k-mooney: stdout says this, forever: "50 rows matched query populate_user_id, 18 migrated" | 15:47 |
andrewbogott | error log says... | 15:47 |
andrewbogott | https://www.irccloud.com/pastebin/pNsHlrU0/ | 15:48 |
andrewbogott | ^ also forever | 15:48 |
andrewbogott | hm, which doesn't seem to be from the same migration phase as stdout is reporting | 15:48 |
*** arturo has joined #openstack-nova | 15:48 | |
sean-k-mooney | ya that for keypairs | 15:48 |
andrewbogott | seems so, although I don't know what gives with stdout logging a different phase | 15:49 |
sean-k-mooney | so it kind of sounds like it was not in a consitent stae when you start the online migrations | 15:49 |
andrewbogott | It does | 15:50 |
gibi | nova meeting starts in 10 minutes in #openstack-meeting-3 | 15:50 |
andrewbogott | Although I've no idea how it got here, this is a cloud that's been running since Mitaka | 15:51 |
andrewbogott | Is the 'keypair' referred to there the user-defined keypairs for ssh access? | 15:52 |
andrewbogott | Because our setup doesn't use those at all, I'd expect it to be unset in all cases. | 15:52 |
andrewbogott | (we have a PAM setup for ssh) | 15:52 |
andrewbogott | sean-k-mooney: any chance that we're just a use case that wasn't anticipated by that code? | 15:53 |
*** dklyle has joined #openstack-nova | 15:53 | |
lyarwood | gibi: ah sorry, it's likely left over from the previous bump cleanup and I rebased it into this series by mistake | 15:59 |
lyarwood | gibi: feel free to pull that out and rebase it under your changes | 15:59 |
gibi | lyarwood: ack | 16:00 |
gibi | lyarwood: I think I will do it but not today | 16:00 |
lyarwood | gibi: np | 16:00 |
sean-k-mooney | andrewbogott: its the user keypair yes | 16:06 |
sean-k-mooney | not an unique constratint or forien key at the db level | 16:06 |
andrewbogott | and yet the migration fails if it's unset :/ | 16:06 |
andrewbogott | I don't see keypairs in my db though, I see mostly NULLS and some json elsewhere | 16:07 |
andrewbogott | https://www.irccloud.com/pastebin/CIh6rCTi/ | 16:07 |
sean-k-mooney | yep so it looke like you missed a previous migration to populate tehm | 16:07 |
andrewbogott | maybe. My test cloud has the same weird state. Lots of nulls, a few json objects. | 16:09 |
andrewbogott | anyway I can insert the empty json object into my null records | 16:10 |
andrewbogott | ok, I've confirmed that the keypair issue is not the thing that's breaking my migration. It fails gracefully if there are no keypairs, as it should. | 16:13 |
*** efried has quit IRC | 16:20 | |
*** efried has joined #openstack-nova | 16:21 | |
*** gyee has joined #openstack-nova | 16:22 | |
*** ganso has quit IRC | 16:24 | |
*** dosaboy has quit IRC | 16:24 | |
*** dosaboy has joined #openstack-nova | 16:25 | |
*** ganso has joined #openstack-nova | 16:26 | |
*** mlavalle has joined #openstack-nova | 16:29 | |
lyarwood | melwitt: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/711625 - so this works if I enable c-bak and c-vol on the subnode, fun. | 16:29 |
sean-k-mooney | bauzas: so lookign that the cyborg vgpu spec the last version i reviewd was version 9 | 16:43 |
sean-k-mooney | bauzas: what was your outstanding issue | 16:43 |
bauzas | sean-k-mooney: the 'other deployer impact' section, starting PS12 AFAICR | 16:43 |
sean-k-mooney | it looks like many of my comments were adress but i need to review it porperly | 16:44 |
sean-k-mooney | ah the advataghes stuff | 16:44 |
bauzas | and the fact that they weren't using the same methods for modifying the XML, but eventually the last revision fixes it | 16:44 |
sean-k-mooney | ya | 16:44 |
sean-k-mooney | that is one of the the thigns i rememebred | 16:45 |
bauzas | anyway, /me needs to stop working early | 16:45 |
sean-k-mooney | i porably would just drop the comparison form the spec | 16:45 |
*** hoonetorg has joined #openstack-nova | 16:45 | |
sean-k-mooney | in the other deployer impact section | 16:45 |
sean-k-mooney | and jsut leave the note about not configuring the same device in both services | 16:45 |
bauzas | sean-k-mooney: that's my point indeed | 16:45 |
bauzas | yeah | 16:45 |
sean-k-mooney | ok ill leave the same comment after i do a full review | 16:46 |
bauzas | my other concern would be to maybe use the same module for creating both inventories | 16:46 |
bauzas | but we could do it as a follow-up | 16:46 |
bauzas | like, creating a common package | 16:46 |
bauzas | for both the cyborg agent and the nova-libvirt code | 16:47 |
bauzas | this way, gibi's concerns about maintenance would be fixed | 16:47 |
bauzas | if both services would eventually share the same code | 16:47 |
bauzas | anyway, /me needs to run | 16:47 |
sean-k-mooney | as in a shared lib | 16:48 |
sean-k-mooney | we could but im not sure we want to create a new repo at this point in the cycle | 16:48 |
sean-k-mooney | given the pacakging work that would entail | 16:49 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: Add nova-live-migration-ceph job https://review.opendev.org/c/openstack/nova/+/768466 | 16:49 |
sean-k-mooney | bauzas: i tought gibis main concern in relation to code duplciation was the rest of the code for tracking and assigning mdevs | 16:50 |
sean-k-mooney | although i may have misread that | 16:50 |
sean-k-mooney | andrewbogott: by the way did you do the normal db migration before running the online ones? | 16:52 |
sean-k-mooney | andrewbogott: e.g. nova-manage db sync | 16:52 |
andrewbogott | I did | 16:52 |
andrewbogott | sean-k-mooney: I've now determined that the keypair errors were a red herring. It might be that the whole thing was a red herring and stdout was just overly verbose but I need to circle back. | 16:53 |
andrewbogott | the keypair migration throws a log warning about missing keypairs but moves on as it should | 16:53 |
sean-k-mooney | andrewbogott: so it might be related to this | 16:53 |
sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/633351/31/nova/objects/instance_mapping.py#250 | 16:53 |
sean-k-mooney | we skipi instance that dont have instance cell mappings | 16:54 |
sean-k-mooney | so before you run that you need to ensure all instance have the cell mappings set | 16:54 |
andrewbogott | yeah, I think that's likely the issue. I need to get out of my deployment window and then I'll circle back and debug :) | 16:54 |
andrewbogott | hm, I thought that the cell mapping thing was handled by an upgrade script many versions ago | 16:55 |
andrewbogott | but since I only have one cell it should be easy to fix by hand in the db | 16:55 |
sean-k-mooney | it could be deleted but unachived/purged isntances | 16:56 |
sean-k-mooney | which is why it suggest nova-manage db archive_deleted_rows --purge | 16:56 |
andrewbogott | yeah, I ran that but it didn't do anything (didn't report anything on stdout and also didn't affect behavior of the migration) | 16:57 |
*** zzzeek has quit IRC | 16:58 | |
*** zzzeek has joined #openstack-nova | 17:00 | |
sean-k-mooney | ya so i would guess you jsut need to add the mappings and hopefully it will correct the isseu you have | 17:01 |
*** belmoreira has quit IRC | 17:04 | |
*** ganso has quit IRC | 17:07 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_VIR_ERR_DEVICE_MISSING https://review.opendev.org/c/openstack/nova/+/754702 | 17:19 |
lyarwood | gibi: ^ fixed, sorry about that. | 17:19 |
*** tesseract has quit IRC | 17:19 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_VIR_ERR_DEVICE_MISSING https://review.opendev.org/c/openstack/nova/+/754702 | 17:24 |
lyarwood | does anyone recall where we list the supported distros for a release? | 17:36 |
openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add spec for configurable-instance-hostnames https://review.opendev.org/c/openstack/nova-specs/+/770842 | 17:37 |
stephenfin | lyarwood: that was in the wiki | 17:37 |
stephenfin | actually, no - I'm thinking of the libvirt version matrix | 17:37 |
lyarwood | https://github.com/openstack/governance/blob/2bdd9cff00fb40b2f95b66cad47ae1cfd14a2f1b/reference/project-testing-interface.rst#linux-distributions is all I can find | 17:38 |
stephenfin | artom: Looking at your spec now, but just FYI I've a super low priority proposal for a '--hostname' parameter here https://review.opendev.org/c/openstack/nova-specs/+/770842 Had a look earlier and it's pretty easy | 17:38 |
lyarwood | and yeah our distro matrix in the wiki just lists versions | 17:38 |
* lyarwood wants to move that in tree btw | 17:38 | |
stephenfin | lyarwood: fungi would probably know | 17:38 |
stephenfin | lyarwood++ good call | 17:38 |
fungi | it's in governance, getting it now | 17:38 |
fungi | https://governance.openstack.org/tc/reference/runtimes/wallaby.html | 17:39 |
lyarwood | oh sorry I see the runtime link at the bottom now, helps if I view the hosted page instead of the source | 17:39 |
lyarwood | fungi: many thanks | 17:39 |
fungi | yw | 17:40 |
*** raildo has quit IRC | 17:42 | |
sean-k-mooney | fungi: that does not cover libvirt verions | 17:42 |
sean-k-mooney | or qemu | 17:42 |
*** rpittau is now known as rpittau|afk | 17:42 | |
sean-k-mooney | we manage that seperatly | 17:43 |
lyarwood | I didn't need that, just the distros | 17:43 |
* lyarwood just wanted to check the version of SLES | 17:43 | |
lyarwood | or openSUSE sorry | 17:43 |
sean-k-mooney | ah then ya latest lts of ubuntu centos and suse | 17:43 |
sean-k-mooney | fungi: has there been any moemente on updating centos 8 sot centos 8 stream | 17:44 |
sean-k-mooney | fungi: i assume we will change that in X | 17:44 |
fungi | sean-k-mooney: i'm not aware of any tc level decision around that yet. discussions were that we only needed to solve it for the cycle in which 8 becomes eol | 17:45 |
sean-k-mooney | ok so i guess technialy Y then | 17:45 |
sean-k-mooney | EOL is decemebr | 17:46 |
fungi | though i'm not entirely sure what it means for stable branches which supported 8 and how we choose to test them | 17:46 |
sean-k-mooney | ya | 17:46 |
fungi | since opendev doesn't usually continue to provide images for eol distro versions for very long | 17:46 |
sean-k-mooney | one option might be to use rocky linux or one of the repelacement for centos | 17:46 |
sean-k-mooney | i guess we will see how things play out | 17:47 |
sean-k-mooney | centos 8 stream should sablise once centos 9 stream is a thing but no idea when that will be a thing or how long centos 8 stream will be supported when its released | 17:48 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION and NEXT_MIN_{LIBVIRT,QEMU}_VERSION https://review.opendev.org/c/openstack/nova/+/754703 | 17:58 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_{LIBVIRT,QEMU}_BLOCKDEV https://review.opendev.org/c/openstack/nova/+/754701 | 17:58 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: libvirt: Remove MIN_LIBVIRT_S390X_CPU_COMPARE https://review.opendev.org/c/openstack/nova/+/754704 | 17:58 |
*** derekh has quit IRC | 18:04 | |
openstackgerrit | sean mooney proposed openstack/nova stable/train: only wait for plugtime events in pre-live-migration https://review.opendev.org/c/openstack/nova/+/770844 | 18:05 |
*** dtantsur is now known as dtantsur|afk | 18:10 | |
stephenfin | lyarwood: Seeing as you're looking at DB stuff at the moment, think you could take a look at these, my two oldest outstanding patches? :) https://review.opendev.org/c/openstack/nova/+/530905/ https://review.opendev.org/c/openstack/nova/+/530487/ | 18:11 |
openstackgerrit | sean mooney proposed openstack/nova master: use psycopg2 binary instead of source package https://review.opendev.org/c/openstack/nova/+/770473 | 18:12 |
*** zzzeek has quit IRC | 18:16 | |
sean-k-mooney | stephenfin: why do we need uuid in the pci devices table? | 18:16 |
*** zzzeek has joined #openstack-nova | 18:16 | |
*** andrewbonney has quit IRC | 18:17 | |
sean-k-mooney | oh its for tracking them in placment | 18:18 |
sean-k-mooney | so its not used currently | 18:18 |
sean-k-mooney | +1 on both but its also strictly speaking not needed currently unless i missed where its beign used | 18:21 |
sean-k-mooney | in anycase time for dinner | 18:21 |
*** dosaboy has quit IRC | 18:21 | |
*** raildo has joined #openstack-nova | 18:26 | |
*** slaweq has quit IRC | 18:38 | |
*** ralonsoh has quit IRC | 18:41 | |
*** nightmare_unreal has quit IRC | 19:04 | |
*** ganso has joined #openstack-nova | 19:11 | |
*** dosaboy has joined #openstack-nova | 19:11 | |
*** whoami-rajat__ has quit IRC | 19:20 | |
artom | melwitt, so, for the record, I am looking at https://review.opendev.org/c/openstack/nova/+/769614 :) | 19:35 |
artom | But it's in an area of hardware.py that I'm not really familiar with, so there's a learning curve | 19:35 |
* artom has to taxi kids around for a bit | 19:36 | |
*** raildo has quit IRC | 19:37 | |
*** raildo_ has joined #openstack-nova | 19:37 | |
dansmith | lyarwood: since you've brought it up a couple times, I think this is going to fix the multistore test hang/fail: https://review.opendev.org/c/openstack/tempest/+/770850 | 19:38 |
dansmith | also gibi ^ | 19:38 |
*** qqmber has joined #openstack-nova | 19:55 | |
qqmber | hi there | 19:56 |
*** qqmber has quit IRC | 20:03 | |
*** lpetrut has joined #openstack-nova | 20:10 | |
*** zoharm has quit IRC | 20:12 | |
*** slaweq has joined #openstack-nova | 20:19 | |
*** slaweq has quit IRC | 20:27 | |
*** dosaboy has quit IRC | 21:00 | |
*** dosaboy has joined #openstack-nova | 21:02 | |
*** lpetrut has quit IRC | 21:07 | |
*** raildo_ has quit IRC | 21:15 | |
*** hamalq has joined #openstack-nova | 21:22 | |
*** jrosser has quit IRC | 21:29 | |
*** fnordahl has quit IRC | 21:30 | |
*** jrosser has joined #openstack-nova | 21:31 | |
*** fnordahl has joined #openstack-nova | 21:31 | |
*** nweinber has quit IRC | 21:42 | |
*** xek has quit IRC | 21:44 | |
*** rcernin has joined #openstack-nova | 22:05 | |
*** eharney has quit IRC | 22:12 | |
*** spatel has quit IRC | 22:27 | |
openstackgerrit | Ghanshyam proposed openstack/nova master: Reuse code from oslo lib for JSON policy migration https://review.opendev.org/c/openstack/nova/+/763487 | 22:41 |
*** markguz_ has quit IRC | 22:43 | |
openstackgerrit | Ghanshyam proposed openstack/placement master: [goal] Deprecate the JSON formatted policy file https://review.opendev.org/c/openstack/placement/+/764394 | 22:46 |
dansmith | melwitt: replied | 22:54 |
melwitt | dansmith: danke | 22:55 |
melwitt | I was coincidentally looking at some different gate failures involving presumable stress on node resources and wanted to see if there are similarities | 22:55 |
dansmith | well, this was a pretty standard OOM once I realized that, | 22:56 |
dansmith | but it's unusual that the OOM is caused by the test runner, so I hadn't seen an OOM manifest itself as a killing of one worker before | 22:57 |
melwitt | I see now, in syslog of the example you linked. literal oom-killer messages. what I've been looking at today is different | 22:58 |
dansmith | this one blows up so fast that it doesn't even register in dstat, | 23:00 |
dansmith | which is what took me a bit to realize it was an oom | 23:00 |
dansmith | it goes from like 45M to 1.5g in a couple seconds locally | 23:00 |
melwitt | wow | 23:00 |
melwitt | how were you able to tell it's the test runner? something in the logs? | 23:01 |
dansmith | well, the oom was a python process, and one worker (which is python) just never reports its results, | 23:01 |
dansmith | then reproing locally (with plenty of memory) I see one test runner blip to 1.5g, before exiting normally, which is the same size as the victim from the killer | 23:02 |
dansmith | I don't oom because I have lots of memory locally, but I could see that balloon, and if it happens at the wrong time in the gate, it'll trigger and die | 23:02 |
melwitt | I see, thanks | 23:03 |
melwitt | nice find. I've been looking at a lot of gate failures lately that seem to be outside of openstack and trying to find hints about what could be causing them | 23:05 |
*** nweinber has joined #openstack-nova | 23:08 | |
*** nweinber has quit IRC | 23:11 | |
openstackgerrit | melanie witt proposed openstack/nova stable/stein: Change default num_retries for glance to 3 https://review.opendev.org/c/openstack/nova/+/764963 | 23:21 |
openstackgerrit | melanie witt proposed openstack/nova stable/stein: Use fair locks in resource tracker https://review.opendev.org/c/openstack/nova/+/770657 | 23:24 |
sean-k-mooney | melwitt: actully regarding ^ | 23:49 |
sean-k-mooney | i realised that while the upper-constatis allows it im not sure lower constartis on stien does | 23:50 |
sean-k-mooney | https://github.com/openstack/nova/blob/stable/stein/lower-constraints.txt#L79 oslo.concurrency==3.26.0 | 23:50 |
sean-k-mooney | it need 3.29.0 | 23:51 |
sean-k-mooney | are we allowed to bump it like that in a backport? | 23:51 |
sean-k-mooney | for what its worth i think its a good thing to backport just dont know what stable policy allows | 23:54 |
melwitt | oh yeah, we aren't. so I guess I was right originally, that it couldn't be backported | 23:54 |
melwitt | sigh | 23:54 |
sean-k-mooney | well with the patch on gerrit if peopel want to use it it should be eaiser for them | 23:55 |
sean-k-mooney | to do the backport you would have to add the fair lock form oslo to noava | 23:56 |
sean-k-mooney | and just use that instead | 23:56 |
melwitt | yeah, I doubt it's worth going that far | 23:57 |
sean-k-mooney | ya | 23:57 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!