| opendevreview | Eric Xie proposed openstack/nova master: Add logs when cannot fit numa https://review.opendev.org/c/openstack/nova/+/798187 | 01:12 |
|---|---|---|
| *** abhishekk is now known as akekane|home | 04:52 | |
| *** akekane|home is now known as abhishekk | 04:52 | |
| *** bhagyashris_ is now known as bhagyashris|ruck | 07:40 | |
| *** rpittau|afk is now known as rpittau | 07:45 | |
| opendevreview | Yongli He proposed openstack/nova master: Smartnic support - cyborg drive https://review.opendev.org/c/openstack/nova/+/771362 | 08:42 |
| opendevreview | Yongli He proposed openstack/nova master: smartnic support - new vnic type https://review.opendev.org/c/openstack/nova/+/771363 | 08:42 |
| opendevreview | Yongli He proposed openstack/nova master: smartnic support - create arqs https://review.opendev.org/c/openstack/nova/+/758944 | 08:42 |
| opendevreview | Yongli He proposed openstack/nova master: smartnic support - build instance with smartnic arqs https://review.opendev.org/c/openstack/nova/+/798249 | 08:42 |
| opendevreview | Yongli He proposed openstack/nova master: smartnic support - cleanup arqs https://review.opendev.org/c/openstack/nova/+/798054 | 08:43 |
| opendevreview | Yongli He proposed openstack/nova master: smartnic support - reject server move and suspend https://review.opendev.org/c/openstack/nova/+/779913 | 08:43 |
| opendevreview | Yongli He proposed openstack/nova master: smartnic support - functional tests https://review.opendev.org/c/openstack/nova/+/780147 | 08:43 |
| lyarwood | stephenfin / gibi ; https://review.opendev.org/c/openstack/nova/+/720769 - would you mind hitting this today, https://launchpad.net/bugs/1860913 spells out the usecase pretty well. | 09:04 |
| gibi | lyarwood: looking | 09:05 |
| lyarwood | many thanks | 09:06 |
| stephenfin | sure | 09:08 |
| *** iurygregory is now known as iury|holiday | 09:18 | |
| *** ralonsoh_ is now known as ralonsoh | 09:20 | |
| gibi | lyarwood, stephenfin: what was the solution for the failure of test_archive_task_logs functional test ? | 09:29 |
| stephenfin | um... | 09:29 |
| stephenfin | gibi: can you give me a bit more context? It's Monday morning :-P | 09:30 |
| gibi | I saw multiple patches failing with https://ef75749a93c952a7bcbb-88cdb958b5e15841b787b658e9835738.ssl.cf5.rackcdn.com/720769/16/check/nova-tox-functional-py39/f1f266b/testr_results.html | 09:30 |
| gibi | and I cloudly recall somebody pinged melwitt about it last week | 09:30 |
| gibi | but I cannot reproduce the issue locally | 09:30 |
| stephenfin | oh, I have no idea. Didn't know there was an issue there | 09:31 |
| alex_xu | gibi: I feel smartnic patches are ready for your review https://review.opendev.org/q/topic:%22bp%252Fsriov-smartnic-support%22+(status:open%20OR%20status:merged) | 09:31 |
| gibi | alex_xu: thanks. I will try to get to them, but based on how many times I made promise review it without actually getting there I don't want to promise any more. I will try | 09:32 |
| alex_xu | gibi: thanks! | 09:32 |
| gibi | stephenfin: no worries | 09:32 |
| alex_xu | I think the major logic is on the third and fourth patch | 09:32 |
| lyarwood | gibi: I think that was me pinging about it | 09:41 |
| lyarwood | gibi: iirc it suddenly stopped failing | 09:41 |
| * lyarwood checks | 09:41 | |
| gibi | strange | 09:41 |
| lyarwood | https://zuul.opendev.org/t/openstack/builds?job_name=nova-tox-functional-py38&project=openstack%2Fnova&branch=master&pipeline=check | 09:41 |
| lyarwood | https://bugs.launchpad.net/nova/+bug/1934519 | 09:41 |
| lyarwood | yeah really odd | 09:41 |
| lyarwood | I can only assume something was reverted in a dep that fixed it | 09:42 |
| * lyarwood moves the bug to incomplete | 09:42 | |
| *** bhagyashris_ is now known as bhagyashris|ruck | 10:08 | |
| lyarwood | test_volume_backed_live_migration is still failing | 10:14 |
| lyarwood | argh | 10:14 |
| lyarwood | actually that might be an issue with the change | 10:15 |
| lyarwood | becuase https://zuul.opendev.org/t/openstack/builds?job_name=nova-live-migration&project=openstack%2Fnova&branch=master&pipeline=check looks pretty good | 10:15 |
| lyarwood | http://paste.openstack.org/show/807166/ - made some progress with this, it isn't related to the patch AFAICT | 10:48 |
| * lyarwood heads to #virt | 10:48 | |
| gibi | lyarwood: good finding | 11:01 |
| * gibi needs a libvirt debugging / log reading course :) | 11:01 | |
| kashyap | gibi: Heh, I once did a bit of it on list: | 11:02 |
| kashyap | gibi: You need to follow the request/response "libvirt-$ID": http://lists.openstack.org/pipermail/openstack-dev/2016-October/105158.html | 11:03 |
| kashyap | Each request and response will have the same libvirt-ID: libvirt-30 (or whatever number) | 11:04 |
| kashyap | lyarwood: I saw the brief chat on #virt; it's the dreaded "Device or resource busy" problem | 11:04 |
| kashyap | It is my list of top two "most notorious libvirt errors" | 11:04 |
| lyarwood | kinda | 11:05 |
| lyarwood | that's causing libvirtd to lockup | 11:05 |
| lyarwood | the migration is in a different thread | 11:05 |
| lyarwood | and the dest times out before the src | 11:05 |
| lyarwood | but the src then continues | 11:05 |
| lyarwood | that's a separate behavioural issue in libvirtd tbh | 11:05 |
| kashyap | Me nods | 11:06 |
| kashyap | There's also the SIGTERM / SIGKILL dance. Often times only killing the instance and restarting seems to be the "solution" | 11:06 |
| kashyap | (Aside: if you're wondering what is the other error in my top-2 list, it's the "cannot acquire state change lock") | 11:07 |
| gibi | kashyap: thank I will read that | 11:11 |
| opendevreview | Merged openstack/nova-specs master: [template]suggest work item ordering https://review.opendev.org/c/openstack/nova-specs/+/793197 | 12:05 |
| sean-k-mooney | bauzas: im +2 on https://review.opendev.org/c/openstack/nova-specs/+/792796 by the way but held off +w to see if you want to adress teh nits in a follow up patch or if you wanted to quickly respin | 12:06 |
| bauzas | sean-k-mooney: ack, will quickly look (was on PTO on Friday) | 12:17 |
| *** tbachman is now known as Guest1548 | 12:34 | |
| alexe9191 | Goodday everyone :) | 14:46 |
| alexe9191 | I was wondering if anyone has information on this blueprint? https://blueprints.launchpad.net/nova/+spec/detach-boot-volume | 14:46 |
| alexe9191 | I see that it is waiting for approval since train and I was wondering if there are any plans to release it anytime soon ? | 14:46 |
| sean-k-mooney | i think the people proposing it did not complete it | 14:46 |
| alexe9191 | too bad | 14:47 |
| sean-k-mooney | https://review.opendev.org/q/topic:%22bp%252Fdetach-boot-volume%22+(status:open%20OR%20status:merged) | 14:48 |
| gibi | yepp it seems it is simply stalled out https://review.opendev.org/c/openstack/nova/+/623981/ | 14:48 |
| sean-k-mooney | https://review.opendev.org/q/topic:%2522bp/detach-boot-volume%2522+ | 14:48 |
| sean-k-mooney | so the most recent code is abandoned | 14:48 |
| sean-k-mooney | alexe9191: it could be picked back up | 14:49 |
| sean-k-mooney | i dont think it stalled out because of design reasons | 14:49 |
| sean-k-mooney | matt's -1 was because we did not need to bump the object verion and the is_multi_attach is not implemted correctly | 14:51 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/623981/24/nova/objects/block_device.py | 14:51 |
| alexe9191 | Abandoned meaning no one is working on it, or it won't be implemented for other reasons? | 14:52 |
| sean-k-mooney | alexe9191: no one is working on it | 14:53 |
| sean-k-mooney | it can be repoposed and implemented if people still want it and have time to work on it | 14:53 |
| opendevreview | Merged openstack/nova-specs master: Re-propose "CPU selection with hypervisor consideration" https://review.opendev.org/c/openstack/nova-specs/+/799096 | 16:11 |
| opendevreview | Lee Yarwood proposed openstack/nova master: WIP/DNM - block_device: Use initialize APIs to refresh when reported as idempotent https://review.opendev.org/c/openstack/nova/+/720769 | 16:18 |
| *** rpittau is now known as rpittau|afk | 16:20 | |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Use module-level imports for sqlalchemy (for real) https://review.opendev.org/c/openstack/nova/+/796519 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Move db.sqalchemy.migration to db.migration https://review.opendev.org/c/openstack/nova/+/799518 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Move main DB migrations https://review.opendev.org/c/openstack/nova/+/799519 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Move 'sqlalchemy.types' up a directory https://review.opendev.org/c/openstack/nova/+/799520 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Remove constant aliases from 'nova.db.api' https://review.opendev.org/c/openstack/nova/+/799521 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Drop support for experimental concurrency https://review.opendev.org/c/openstack/nova/+/799522 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Register database config options ourselves https://review.opendev.org/c/openstack/nova/+/799523 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Unify 'nova.db.api', 'nova.db.sqlalchemy.api' https://review.opendev.org/c/openstack/nova/+/799524 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Move remaining 'nova.db.sqlalchemy' modules https://review.opendev.org/c/openstack/nova/+/799525 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Post reshuffle cleanup https://review.opendev.org/c/openstack/nova/+/799526 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Add initial alembic migration for main DB https://review.opendev.org/c/openstack/nova/+/799527 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Add initial alembic migration for API DB https://review.opendev.org/c/openstack/nova/+/799528 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: db: Trivial style changes https://review.opendev.org/c/openstack/nova/+/799529 | 16:26 |
| opendevreview | Stephen Finucane proposed openstack/nova master: WIP: db: Integrate alembic https://review.opendev.org/c/openstack/nova/+/799530 | 16:26 |
| opendevreview | Ghanshyam proposed openstack/placement master: Fix oslo policy DeprecatedRule warnings https://review.opendev.org/c/openstack/placement/+/799418 | 16:28 |
| stephenfin | dansmith: If you have a bit of spare time, I'd appreciate some eyes on the last one of those patches ^ I can't quite figure out why the 'CellDatabases' fixtures is failing but it is :-\ | 16:29 |
| sean-k-mooney | gibi: how do you feel about https://review.opendev.org/c/openstack/nova-specs/+/791047/2/specs/xena/approved/pci-device-tracking-in-placement.rst#152 doing the pci inventory reservation via moving the instance claim to the conductor | 17:12 |
| sean-k-mooney | stephenfin: ^ also of interest to you | 17:12 |
| sean-k-mooney | we have talked about moving the instance claim to the conductor for years and it will simplfy pci in placment as well as resolve some other numa issues | 17:13 |
| opendevreview | Elod Illes proposed openstack/nova stable/pike: Use subqueryload() instead of joinedload() for (system_)metadata https://review.opendev.org/c/openstack/nova/+/799533 | 17:24 |
| noonedeadpunk | hey there! We just relesed W and switched back to tracking master branch and found our upgrade jobs failing on Nova, after upgrade from stable/wallaby to master | 17:36 |
| noonedeadpunk | with smth like http://paste.openstack.org/show/807170/ | 17:37 |
| noonedeadpunk | and looking at https://opendev.org/openstack/nova/src/branch/master/nova/objects/service.py#L207-L211 it makes me wonder wtf is going on here :) | 17:38 |
| lyarwood | noonedeadpunk: https://github.com/openstack/nova/blob/66fb0ecb5a867c054ab266aadcc06a940967abd4/nova/objects/service.py#L33-L34 looks like it is 53 on stable/victoria at least | 18:03 |
| lyarwood | that would make sense given your error | 18:03 |
| noonedeadpunk | ah, ok, yes | 18:03 |
| noonedeadpunk | now it does :) | 18:04 |
| noonedeadpunk | lyarwood: so, eventually now jumping through releases is not going to work, right? So if I'd love to do V->X upgrade, it's not gonna work (like T->V did nicely) | 18:04 |
| lyarwood | I don't get why the alias is 52 in master however | 18:04 |
| lyarwood | noonedeadpunk: we have only ever supported N to N+1 | 18:05 |
| noonedeadpunk | but toher then that never failed explicitly :D | 18:05 |
| lyarwood | noonedeadpunk: and even with things like FFU downstream we take everything down at N and bring everything back up at N+3 | 18:05 |
| lyarwood | noonedeadpunk: yeah that was more luck than anything, there was no test coverage of anything other than N to N+1 | 18:06 |
| noonedeadpunk | I think I found some mistake and in CI we were upgrading from V to master, which made this part fail | 18:06 |
| noonedeadpunk | yeah, we also test only N to N+1 | 18:06 |
| * lyarwood heads off for dinner \o | 18:07 | |
| noonedeadpunk | btw, maybe you have some idea, would computes in N+1 work with N API? Ie if you upgrade computes first? | 18:07 |
| lyarwood | Yes that should work | 18:22 |
| lyarwood | you just need to pin your computes to the older rpc versions | 18:23 |
| lyarwood | but it isn't the way we test upgrades | 18:23 |
| sean-k-mooney | noonedeadpunk: actully it wont work | 18:24 |
| sean-k-mooney | you have to upgrade the contoler first | 18:24 |
| sean-k-mooney | so you always have to upgrade the api, conductor and schduler together | 18:25 |
| sean-k-mooney | and they must be upgraded to N+1 before you upgade any comnputes to n+1 | 18:25 |
| sean-k-mooney | while you could technially pin the rpc version and you might be able to get them to work it would be unsuppoted upstream and downstream | 18:26 |
| sean-k-mooney | the compute always assume that the contoler are at least teh same version | 18:26 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!