*** k_mouza has joined #openstack-nova | 00:01 | |
*** avolkov has quit IRC | 00:03 | |
*** k_mouza has quit IRC | 00:06 | |
*** k_mouza has joined #openstack-nova | 00:16 | |
*** brinzhang0 has joined #openstack-nova | 00:19 | |
*** k_mouza has quit IRC | 00:21 | |
*** brinzhang_ has quit IRC | 00:22 | |
*** songwenping_ has joined #openstack-nova | 00:29 | |
*** songwenping__ has quit IRC | 00:32 | |
*** eharney has joined #openstack-nova | 00:36 | |
openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: Functions to merge provider configs to provider tree https://review.opendev.org/676522 | 00:49 |
---|---|---|
openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: Enable loading and merging of provider configs https://review.opendev.org/693460 | 00:49 |
*** spatel has joined #openstack-nova | 00:57 | |
*** songwenping__ has joined #openstack-nova | 01:13 | |
*** songwenping_ has quit IRC | 01:17 | |
*** songwenping_ has joined #openstack-nova | 01:22 | |
*** songwenping__ has quit IRC | 01:25 | |
*** spatel has quit IRC | 01:33 | |
*** songwenping__ has joined #openstack-nova | 01:38 | |
*** songwenping_ has quit IRC | 01:41 | |
*** songwenping_ has joined #openstack-nova | 01:53 | |
*** songwenping__ has quit IRC | 01:56 | |
*** sapd1_x has joined #openstack-nova | 02:02 | |
*** bauzas has quit IRC | 02:14 | |
*** tetsuro has quit IRC | 02:19 | |
*** bauzas has joined #openstack-nova | 02:20 | |
*** mkrai has joined #openstack-nova | 02:26 | |
*** rcernin has quit IRC | 02:27 | |
*** bauzas has quit IRC | 02:27 | |
*** gyee has quit IRC | 02:29 | |
*** bauzas has joined #openstack-nova | 02:29 | |
*** rcernin has joined #openstack-nova | 02:30 | |
*** bauzas has quit IRC | 02:42 | |
*** bauzas has joined #openstack-nova | 02:43 | |
*** songwenping__ has joined #openstack-nova | 02:45 | |
*** yaawang has quit IRC | 02:47 | |
*** yaawang has joined #openstack-nova | 02:47 | |
*** songwenping_ has quit IRC | 02:48 | |
tony_su | stephenfin: gibi: the last 3 patches for provider-config-file are ready for a new round of review with comments answered. Thanks. | 02:49 |
tony_su | I checked zuul failure and found no relation to us. | 02:50 |
*** bauzas has quit IRC | 02:50 | |
*** bauzas has joined #openstack-nova | 02:53 | |
*** bauzas has quit IRC | 02:58 | |
*** bauzas has joined #openstack-nova | 02:58 | |
*** songwenping_ has joined #openstack-nova | 03:28 | |
openstackgerrit | renminmin proposed openstack/nova master: SRIOV: Fix instance with macvtap port live migration failed https://review.opendev.org/738432 | 03:28 |
*** songwenping__ has quit IRC | 03:30 | |
*** hongbin has joined #openstack-nova | 03:35 | |
*** psachin has joined #openstack-nova | 03:37 | |
*** vishalmanchanda has joined #openstack-nova | 03:42 | |
*** songwenping__ has joined #openstack-nova | 03:47 | |
*** songwenping_ has quit IRC | 03:50 | |
*** dave-mccowan has quit IRC | 04:02 | |
*** sapd1_x has quit IRC | 04:05 | |
*** songwenping_ has joined #openstack-nova | 04:08 | |
*** hamalq has joined #openstack-nova | 04:09 | |
*** hamalq has quit IRC | 04:09 | |
*** hongbin has quit IRC | 04:10 | |
*** hamalq has joined #openstack-nova | 04:10 | |
*** songwenping__ has quit IRC | 04:11 | |
*** hamalq has quit IRC | 04:17 | |
*** k_mouza has joined #openstack-nova | 04:17 | |
*** ratailor has joined #openstack-nova | 04:21 | |
*** k_mouza has quit IRC | 04:21 | |
*** ratailor has quit IRC | 04:26 | |
*** songwenping__ has joined #openstack-nova | 04:28 | |
*** songwenping_ has quit IRC | 04:31 | |
*** evrardjp has quit IRC | 04:33 | |
*** evrardjp has joined #openstack-nova | 04:33 | |
*** songwenping_ has joined #openstack-nova | 05:03 | |
*** songwenping__ has quit IRC | 05:06 | |
*** xiaolin has quit IRC | 05:10 | |
*** songwenping__ has joined #openstack-nova | 05:18 | |
*** songwenping_ has quit IRC | 05:21 | |
*** songwenping_ has joined #openstack-nova | 05:26 | |
*** songwenping__ has quit IRC | 05:30 | |
*** udesale has joined #openstack-nova | 05:36 | |
*** Yumeng has joined #openstack-nova | 05:41 | |
*** songwenping__ has joined #openstack-nova | 05:46 | |
*** songwenping_ has quit IRC | 05:49 | |
*** songwenping__ has quit IRC | 05:50 | |
*** songwenping__ has joined #openstack-nova | 05:50 | |
*** ircuser-1 has joined #openstack-nova | 05:59 | |
*** belmoreira has joined #openstack-nova | 06:15 | |
*** xek has joined #openstack-nova | 06:19 | |
devfaz_ | hi, just run into an issue with no longer existing server groups. I had to evacuate some instances, but they failed, because the server group no longer exists. Just googled a bit and found https://bugs.launchpad.net/nova/+bug/1298509. I understand, that it should be possible to delete a server group with instances, but there should be also a way to update / remove the entry from the instances - | 06:26 |
openstack | Launchpad bug 1298509 in OpenStack Compute (nova) ""nova server-group-delete" allows deleting server group with members" [Wishlist,Opinion] | 06:26 |
devfaz_ | else the makes the instances unbootable. | 06:26 |
*** eharney has quit IRC | 06:35 | |
*** devfaz_ is now known as devfaz | 06:39 | |
gibi | devfaz: do you have a stack trace from the log with the failure? I've quickly tried with resize and that worked with deleted server group | 06:50 |
devfaz | gibi: one moment | 06:51 |
*** slaweq has joined #openstack-nova | 06:52 | |
*** eharney has joined #openstack-nova | 06:53 | |
devfaz | gibi: no stack-trace, just Filtering removed all hosts for the request with instance ID 'adddf2c9-0252-4463-a97c-f1ec209d9f49'. Filter results: ['AvailabilityZoneFilter: (start: 2, end: 2)', 'ComputeFilter: (start: 2, end: 2)', 'ComputeCapabilitiesFilter: (start: 2, end: 2)', 'ImagePropertiesFilter: (start: 2, end: 2)', 'ServerGroupAntiAffinityFilter: (start: 2, end: 2)', | 06:55 |
devfaz | 'ServerGroupAffinityFilter: (start: 2, end: 0)'] | 06:55 |
gibi | devfaz: I think it worth a bug report. I think nova needs to ignore delete server groups during scheduling | 06:56 |
devfaz | and in the "server show", i get as fault-value. Instance group 4b9a2cc5-cc44-470e-9ad0-68dcc9f8c60e could not be found. | 06:56 |
devfaz | gibi: ok, i will create a new one | 06:57 |
devfaz | thx a lot! | 06:57 |
*** links has joined #openstack-nova | 06:57 | |
*** mvorwerk has joined #openstack-nova | 07:04 | |
devfaz | https://bugs.launchpad.net/nova/+bug/1890244 | 07:05 |
openstack | Launchpad bug 1890244 in OpenStack Compute (nova) "nova scheduler should ignore removed groups" [Undecided,New] | 07:05 |
devfaz | gibi: fyi | 07:05 |
*** mvorwerk_ has joined #openstack-nova | 07:09 | |
*** mvorwerk has quit IRC | 07:09 | |
*** mkrai has quit IRC | 07:10 | |
*** mkrai_ has joined #openstack-nova | 07:10 | |
openstackgerrit | Bence Romsics proposed openstack/nova master: doc: Add IPv6 metadata address https://review.opendev.org/744492 | 07:10 |
*** dtantsur|afk is now known as dtantsur | 07:16 | |
*** tesseract has joined #openstack-nova | 07:17 | |
*** xek has quit IRC | 07:22 | |
*** rcernin has quit IRC | 07:22 | |
*** sapd1_x has joined #openstack-nova | 07:23 | |
*** mkrai_ has quit IRC | 07:30 | |
*** tosky has joined #openstack-nova | 07:48 | |
*** ralonsoh has joined #openstack-nova | 07:50 | |
*** ralonsoh has quit IRC | 07:50 | |
*** ralonsoh has joined #openstack-nova | 07:52 | |
*** martinkennelly has joined #openstack-nova | 07:54 | |
*** mkrai has joined #openstack-nova | 07:58 | |
*** xek has joined #openstack-nova | 08:06 | |
*** mkrai has quit IRC | 08:09 | |
openstackgerrit | Xinran WANG proposed openstack/os-resource-classes master: Add new resource class for QAT card. https://review.opendev.org/726314 | 08:09 |
*** mkrai has joined #openstack-nova | 08:10 | |
*** xek has quit IRC | 08:18 | |
*** jhesketh has quit IRC | 08:24 | |
*** xek has joined #openstack-nova | 08:37 | |
*** priteau has joined #openstack-nova | 08:43 | |
*** derekh has joined #openstack-nova | 08:47 | |
*** k_mouza has joined #openstack-nova | 08:53 | |
*** tetsuro has joined #openstack-nova | 08:55 | |
*** rcernin has joined #openstack-nova | 08:59 | |
*** nightmare_unreal has joined #openstack-nova | 09:06 | |
*** avolkov has joined #openstack-nova | 09:08 | |
*** xek has quit IRC | 09:08 | |
*** jangutter has quit IRC | 09:11 | |
*** jangutter has joined #openstack-nova | 09:11 | |
*** lpetrut has joined #openstack-nova | 09:12 | |
*** rcernin has quit IRC | 09:16 | |
*** ociuhandu_ has joined #openstack-nova | 09:21 | |
*** ociuhandu has quit IRC | 09:21 | |
*** yaawang has quit IRC | 09:23 | |
*** yaawang has joined #openstack-nova | 09:26 | |
lyarwood | gibi: https://review.opendev.org/#/c/744506/ - morning, when you have time this should fix the lower-constraints job on master at least for now | 09:38 |
* lyarwood does note that some of the stable branches are now failing | 09:38 | |
lyarwood | https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-lower-constraints&project=openstack%2Fnova for context | 09:39 |
*** ralonsoh has quit IRC | 09:40 | |
*** ralonsoh has joined #openstack-nova | 09:41 | |
*** jangutter has quit IRC | 09:46 | |
*** jangutter has joined #openstack-nova | 09:46 | |
*** k_mouza has quit IRC | 09:47 | |
*** k_mouza has joined #openstack-nova | 09:59 | |
elod | lyarwood: it seems every job have some *constraint related issue now (but some are not visible as does not cause failure), see discussion from infra channel yesterday: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2020-08-03.log.html#t2020-08-03T16:22:05 | 10:03 |
elod | lyarwood: the interesting thing is that I cannot reproduce that locally :/ | 10:03 |
*** yaawang has quit IRC | 10:04 | |
*** yaawang has joined #openstack-nova | 10:05 | |
*** xek has joined #openstack-nova | 10:11 | |
*** udesale_ has joined #openstack-nova | 10:12 | |
*** tkajinam has quit IRC | 10:12 | |
lyarwood | elod: ack thanks | 10:13 |
lyarwood | elod: I was eventually able to reproduce the master issue yesterday on a 18.04 vm but yeah it wasn't easy at all | 10:13 |
*** udesale has quit IRC | 10:14 | |
elod | elod: oh, sounds good! \o/ | 10:15 |
elod | lyarwood: ^^^ :] | 10:15 |
elod | lyarwood: also there's a new release of pip, there's an interesting change: "Correctly find already-installed distributions with dot (.) in the name and uninstall them when needed. " ( https://pip.pypa.io/en/stable/news/#id1 ) | 10:16 |
elod | especially as the failure came when already installed packages were reinstalled (without constraints) | 10:16 |
*** belmoreira has quit IRC | 10:17 | |
*** k_mouza has quit IRC | 10:21 | |
*** jhesketh has joined #openstack-nova | 10:21 | |
*** sapd1_x has quit IRC | 10:24 | |
lyarwood | elod: ah that is interesting | 10:26 |
*** k_mouza has joined #openstack-nova | 10:27 | |
*** ociuhandu_ has quit IRC | 10:30 | |
*** ociuhandu has joined #openstack-nova | 10:31 | |
*** songwenping_ has joined #openstack-nova | 10:33 | |
elod | lyarwood: well, it seems something changed, but I still see the issue in zuul jobs :S | 10:35 |
*** songwenping__ has quit IRC | 10:36 | |
*** sapd1_x has joined #openstack-nova | 10:36 | |
*** ociuhandu has quit IRC | 10:36 | |
elod | lyarwood: do you have somewhere the steps, how the failure can be reproduced locally? | 10:36 |
lyarwood | elod: I didn't write them down sorry, I just built an env using virt-builder and installed tox directly from pypy instead of packages | 10:40 |
lyarwood | elod: iirc I had to bump MarkupSafe to workaround another issue | 10:40 |
*** tetsuro has quit IRC | 10:44 | |
elod | lyarwood: ok, no problem. actually I did the same, installed latest pip and tox from pypi. worth to mention that once it worked for me as in zuul, but then I couldn't reproduce it again with the steps I thought it should be reproducable :S | 10:49 |
*** xek has quit IRC | 10:50 | |
*** rcernin has joined #openstack-nova | 10:56 | |
*** dklyle has quit IRC | 10:56 | |
*** songwenping__ has joined #openstack-nova | 11:03 | |
gibi | devfaz: thanks | 11:03 |
sean-k-mooney | gibi: stephenfin alex_xu could ye take a lok at https://review.opendev.org/#/c/739131/ im trying to move on to numa in placment work so if i can close that out it would make me happy | 11:05 |
gibi | lyarwood: thanks. I've approved https://review.opendev.org/#/c/744506/ | 11:06 |
*** songwenping_ has quit IRC | 11:06 | |
sean-k-mooney | that said gibi i just booted my sriov servers so ill be deploying your sriov detach patches while i rebase some os-vif patches. ill let you know what i find in my testing | 11:07 |
gibi | sean-k-mooney: thanks a lot for those tests | 11:08 |
gibi | sean-k-mooney: I'm looking at https://review.opendev.org/#/c/739131/ now | 11:08 |
*** mkrai has quit IRC | 11:11 | |
*** mkrai_ has joined #openstack-nova | 11:11 | |
*** jangutter has quit IRC | 11:11 | |
*** jangutter has joined #openstack-nova | 11:12 | |
*** eharney has quit IRC | 11:13 | |
*** Luzi has joined #openstack-nova | 11:20 | |
*** tosky_ has joined #openstack-nova | 11:20 | |
*** tosky has quit IRC | 11:20 | |
*** tosky_ is now known as tosky | 11:20 | |
*** luyao has quit IRC | 11:22 | |
*** ociuhandu has joined #openstack-nova | 11:23 | |
*** ociuhandu has quit IRC | 11:24 | |
*** ociuhandu has joined #openstack-nova | 11:25 | |
*** udesale has joined #openstack-nova | 11:27 | |
*** udesale_ has quit IRC | 11:27 | |
*** ociuhandu has quit IRC | 11:30 | |
*** udesale_ has joined #openstack-nova | 11:31 | |
*** udesale has quit IRC | 11:32 | |
*** eharney has joined #openstack-nova | 11:32 | |
openstackgerrit | Harshavardhan Metla proposed openstack/nova stable/rocky: [stable-only] Moved the quoted section https://review.opendev.org/744681 | 11:32 |
*** sapd1_x has quit IRC | 11:36 | |
*** udesale_ has quit IRC | 11:38 | |
*** ociuhandu has joined #openstack-nova | 11:40 | |
*** udesale has joined #openstack-nova | 11:41 | |
*** mkrai_ has quit IRC | 11:45 | |
*** k_mouza has quit IRC | 11:47 | |
gibi | sean-k-mooney: I have left comments in https://review.opendev.org/#/c/744506/ | 11:50 |
*** belmoreira has joined #openstack-nova | 11:51 | |
*** rcernin has quit IRC | 11:53 | |
sean-k-mooney | gibi: did you? that is the lower constrits fix :) | 11:53 |
sean-k-mooney | you mean https://review.opendev.org/#/c/739131 | 11:54 |
*** brinzhang_ has joined #openstack-nova | 11:57 | |
*** raildo has joined #openstack-nova | 11:58 | |
*** brinzhang0 has quit IRC | 12:00 | |
gibi | I've mixed up the links | 12:02 |
gibi | I did left some review on lookup nic patch https://review.opendev.org/#/c/739131/ | 12:02 |
sean-k-mooney | yep reading them now | 12:03 |
*** takamatsu has joined #openstack-nova | 12:04 | |
*** artom has joined #openstack-nova | 12:10 | |
*** k_mouza has joined #openstack-nova | 12:12 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: Migrate to zuulv3 and break up the nova-live-migration job https://review.opendev.org/711604 | 12:13 |
*** tosky has quit IRC | 12:14 | |
*** tosky_ has joined #openstack-nova | 12:14 | |
*** tosky_ is now known as tosky | 12:15 | |
*** dave-mccowan has joined #openstack-nova | 12:15 | |
*** belmoreira has quit IRC | 12:23 | |
*** ralonsoh_ has joined #openstack-nova | 12:26 | |
*** ralonsoh__ has joined #openstack-nova | 12:28 | |
*** ralonsoh has quit IRC | 12:30 | |
sean-k-mooney | gibi: thanks i have responded to the comments ill quickly adress the comparison change since i just forgot to fully revert that and ill re layout the types traits | 12:30 |
*** ralonsoh_ has quit IRC | 12:31 | |
sean-k-mooney | gibi: over all i did not see any functional changes requested so i think the patch is still fucntionally correct as is and the change are jsut style related for the most part right? | 12:31 |
*** xek has joined #openstack-nova | 12:33 | |
*** arxcruz is now known as arxcruz|off | 12:43 | |
*** ralonsoh__ is now known as ralonsoh | 12:44 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: Migrate to zuulv3 and break up the nova-live-migration job https://review.opendev.org/711604 | 12:48 |
*** yaawang has quit IRC | 12:50 | |
*** yaawang has joined #openstack-nova | 12:51 | |
*** nweinber has joined #openstack-nova | 12:52 | |
openstackgerrit | Merged openstack/nova stable/stein: objects: Update keypairs when saving an instance https://review.opendev.org/743264 | 12:55 |
*** eharney has quit IRC | 12:55 | |
*** eharney has joined #openstack-nova | 12:56 | |
gibi | sean-k-mooney: I'm on a call I will get back to you in an hour | 12:56 |
sean-k-mooney | gibi: no rush | 12:56 |
sean-k-mooney | i should have your comments adress by then :) | 12:56 |
openstackgerrit | sean mooney proposed openstack/nova stable/train: Silence amqp heartbeat warning https://review.opendev.org/728057 | 12:58 |
lyarwood | does anyone know if there's a way of reproducing zuul jobs locally? | 13:04 |
lyarwood | ./me is pretty sure there isn't | 13:04 |
*** mgoddard has quit IRC | 13:07 | |
lyarwood | sean-k-mooney: ^ any ideas? | 13:08 |
artom | lyarwood, IIRC I asked in #infra a while ago - "no" was the answer | 13:09 |
lyarwood | artom: yup thanks, I'll keep spamming small changes then | 13:11 |
sean-k-mooney | lyarwood: not really you can technically just run the playbooks locally | 13:11 |
*** mgoddard has joined #openstack-nova | 13:11 | |
lyarwood | sean-k-mooney: yeah it's more the job definitions that I need to test at this point | 13:11 |
sean-k-mooney | lyarwood: but the issue is creating the inventory file and runing them is non trivial | 13:12 |
lyarwood | sean-k-mooney: just feels wrong spamming zuul | 13:12 |
artom | lyarwood, why small changes? | 13:12 |
sean-k-mooney | lyarwood: for what its worth its not that hard to deploy a zuul instance using docker-compose | 13:12 |
sean-k-mooney | but mimicing the upstream enve is still a little more inovled | 13:12 |
sean-k-mooney | lyarwood: i assume you are doint what i do and disablig all the other jobs when your deveolping one | 13:13 |
lyarwood | sean-k-mooney: yup | 13:13 |
sean-k-mooney | lyarwood: also if you want to make it go faster you can cheat and use the experimental pipeline | 13:13 |
sean-k-mooney | in which case you also need to disbale the experimental josb other then your own | 13:14 |
sean-k-mooney | lyarwood: zuul will also do basic syntax checking too so if youy have a typo in the yaml file it will somethime cacht that before runnign your job | 13:14 |
lyarwood | sean-k-mooney: yup cool | 13:17 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: Migrate to zuulv3 and break up the nova-live-migration job https://review.opendev.org/711604 | 13:17 |
lyarwood | artom: well incremental changes then as I'm slowly working out how to covert the jobs | 13:17 |
lyarwood | something I'd normally do locally but have to do in the open thanks to zuul now :| | 13:18 |
artom | lyarwood, tbh, I don't think anyone will look, unless they understand it's a WIP and want to check in/help | 13:18 |
artom | Aka no shame | 13:18 |
sean-k-mooney | artom: i think lyarwood just did not want to waste gate resouces | 13:19 |
*** songwenping__ has quit IRC | 13:19 | |
artom | sean-k-mooney, ah, yeah. Well, what are you gonna do | 13:20 |
*** songwenping__ has joined #openstack-nova | 13:20 | |
sean-k-mooney | disable all jobs you are not updatinging and ask stephenfin to run unit test locally :P | 13:20 |
*** songwenping__ has quit IRC | 13:21 | |
sean-k-mooney | you can replicate the gate as i said but its a lot of setup if you dont do this regularly | 13:21 |
*** songwenping__ has joined #openstack-nova | 13:21 | |
* stephenfin notes machines are cheaper than stephenfin :) | 13:22 | |
sean-k-mooney | that is true | 13:22 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: doc: Update references to image properties https://review.opendev.org/744198 | 13:22 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Drop support for UML https://review.opendev.org/743230 | 13:22 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Drop support for Xen https://review.opendev.org/743231 | 13:22 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Remove 'hypervisor_version' from 'libvirt_info' https://review.opendev.org/744199 | 13:22 |
sean-k-mooney | stephenfin: https://www.sciencealert.com/images/How-Much-Are-We-Worth-V2-large.jpg | 13:23 |
sean-k-mooney | thats even before you take into account your value as a software engineer or a good person | 13:24 |
sean-k-mooney | machines are cheap in comparison :P | 13:24 |
*** lbragstad_ has joined #openstack-nova | 13:24 | |
kashyap | sean-k-mooney: Heh; I never realized the 0.2mg of gold even | 13:24 |
*** lbragstad has quit IRC | 13:26 | |
*** sapd1_x has joined #openstack-nova | 13:26 | |
artom | "It would take 40,000 people to get enough gold for one uk sovereign coin" | 13:29 |
artom | Challenge accepted! | 13:29 |
*** hoonetorg has quit IRC | 13:36 | |
*** hoonetorg has joined #openstack-nova | 13:39 | |
*** lbragstad_ has quit IRC | 13:41 | |
*** mvorwerk has joined #openstack-nova | 13:43 | |
*** mvorwerk_ has quit IRC | 13:45 | |
*** lbragstad has joined #openstack-nova | 13:46 | |
*** mvorwerk has quit IRC | 13:47 | |
*** mvorwerk has joined #openstack-nova | 13:48 | |
stephenfin | artom the terrible | 13:49 |
artom | Great Uncle Ivan would be proud | 13:49 |
*** jangutter has quit IRC | 13:54 | |
*** jangutter has joined #openstack-nova | 13:55 | |
*** mlavalle has joined #openstack-nova | 14:00 | |
*** Yumeng has quit IRC | 14:04 | |
*** tbachman has quit IRC | 14:09 | |
*** tbachman has joined #openstack-nova | 14:10 | |
*** dave-mccowan has quit IRC | 14:25 | |
*** Luzi has quit IRC | 14:27 | |
ralonsoh | hello folks | 14:29 |
ralonsoh | we are hitting in the Neutron CI | 14:29 |
ralonsoh | a problem with grenade jobs | 14:30 |
ralonsoh | https://026716e2072b41f3360b-f5b932bfaec6245f8b3ddc87df48d7a4.ssl.cf2.rackcdn.com/640258/23/check/neutron-grenade-multinode/bcc8e1e/controller/logs/grenade.sh_log.txt | 14:30 |
ralonsoh | just an example | 14:30 |
*** dougsz has joined #openstack-nova | 14:30 | |
*** dklyle has joined #openstack-nova | 14:31 | |
ralonsoh | when retrieving the RP for VCPU | 14:31 |
ralonsoh | openstack resource provider list -f value | head -n1 | cut -d ' ' -f 1 | 14:31 |
ralonsoh | that returns nothing | 14:31 |
*** psachin has quit IRC | 14:33 | |
*** udesale_ has joined #openstack-nova | 14:35 | |
ralonsoh | stephenfin, hi ^^ | 14:37 |
gibi | ralonsoh: placement log stars at 13:43 and the failed query starts at 2020-08-04 13:42:39.140 | 14:38 |
*** udesale has quit IRC | 14:38 | |
gibi | sorry, never mind | 14:39 |
gibi | the placement log ends at https://www.sciencealert.com/images/How-Much-Are-We-Worth-V2-large.jpg | 14:39 |
gibi | bah | 14:39 |
gibi | so the placement log ends 13:43 so it should have some clue in it | 14:39 |
*** donnyd has joined #openstack-nova | 14:40 | |
ralonsoh | gibi, ok I took a look at the placement logs | 14:40 |
ralonsoh | https://026716e2072b41f3360b-f5b932bfaec6245f8b3ddc87df48d7a4.ssl.cf2.rackcdn.com/640258/23/check/neutron-grenade-multinode/bcc8e1e/controller/logs/screen-placement-api.txt | 14:40 |
ralonsoh | I don't know why is failing | 14:40 |
ralonsoh | keystoneauth1.exceptions.catalog.EndpointNotFound: internal endpoint for identity service not found | 14:40 |
gibi | the config looks similar before and after the placement restart based on the log | 14:44 |
*** brinzhang_ has quit IRC | 14:47 | |
*** mvorwerk has quit IRC | 14:49 | |
gibi | keystone log seems normal around that time | 14:52 |
ralonsoh | yes, I don't see any error there | 14:53 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: Migrate to zuulv3 and break up the nova-live-migration job https://review.opendev.org/711604 | 14:55 |
gibi | nova compute also fails as gets back the same HTTP 500 from placement than the grenade script | 14:56 |
gibi | ralonsoh: neutron server also gets eystoneauth1.exceptions.catalog.EndpointNotFound: internal endpoint for identity service not found from keystone | 14:59 |
gibi | around the same time Aug 04 13:42:19. | 14:59 |
ralonsoh | gibi, I'm trying to find why placement stops replying | 15:00 |
gibi | so this is not a placement issue but something is probably wrong with keystone | 15:00 |
ralonsoh | ok | 15:00 |
*** artom has quit IRC | 15:03 | |
*** artom has joined #openstack-nova | 15:04 | |
*** derekh has quit IRC | 15:05 | |
*** derekh has joined #openstack-nova | 15:05 | |
ralonsoh | gibi, just one question | 15:08 |
gibi | sure | 15:08 |
ralonsoh | who sets this RP? | 15:08 |
ralonsoh | because I see n-conductor starting just at this moment | 15:08 |
ralonsoh | https://026716e2072b41f3360b-f5b932bfaec6245f8b3ddc87df48d7a4.ssl.cf2.rackcdn.com/640258/23/check/neutron-grenade-multinode/bcc8e1e/controller/logs/screen-n-cond.txt | 15:08 |
ralonsoh | 13:41:14 ~ 15 | 15:09 |
gibi | the RP is created by n-cpi | 15:10 |
gibi | n-cpu | 15:10 |
gibi | which fails to create it | 15:10 |
gibi | as placement returns http 500 | 15:10 |
gibi | but even if the RP would exists the grenade script cannot read it as it also gets http 500 from placement | 15:11 |
ralonsoh | exactly | 15:11 |
gibi | based on the exception placement cannot find the internal endpoints in keystone | 15:12 |
gibi | also around the same time neutron also tries and fails to find the internal endpoints in keystone (independendly from placement) | 15:13 |
ralonsoh | I think, according to the timestamps that this problem could be due to https://review.opendev.org/#/c/744507/ | 15:14 |
ralonsoh | this is just a guess, I'll try that in a patch | 15:14 |
gibi | culd be a change in the middleware yes | 15:15 |
*** lpetrut has quit IRC | 15:15 | |
*** sdmitriev has quit IRC | 15:16 | |
*** mvorwerk has joined #openstack-nova | 15:19 | |
*** openstackgerrit has quit IRC | 15:20 | |
*** mvorwerk has quit IRC | 15:25 | |
*** mvorwerk has joined #openstack-nova | 15:27 | |
*** mvorwerk has quit IRC | 15:32 | |
*** mvorwerk has joined #openstack-nova | 15:32 | |
*** k_mouza has quit IRC | 15:40 | |
*** lpetrut has joined #openstack-nova | 15:43 | |
* gibi gives up for today | 15:52 | |
*** k_mouza has joined #openstack-nova | 15:54 | |
prometheanfire | do we still need the nova client to evacuate (do the docs need updating) https://docs.openstack.org/nova/ussuri/admin/evacuate.html | 15:55 |
sean-k-mooney | prometheanfire: yes | 15:55 |
sean-k-mooney | i started a patch a year ago but never got aroudn to finishing it | 15:56 |
sean-k-mooney | https://review.opendev.org/#/c/643578/ | 15:56 |
sean-k-mooney | i have local change to change it back to evacuate but i never finished renaming it back | 15:56 |
prometheanfire | ah, kk thanks :D | 15:57 |
sean-k-mooney | if you feel like updating it feel free. i guess i can try and finish it again | 15:59 |
sean-k-mooney | more urgent things keep coming up | 15:59 |
sean-k-mooney | but it should be done sooner rather then later | 16:00 |
prometheanfire | eh, was more a curiousity, trying to wean off project clients and all | 16:02 |
sean-k-mooney | ya that is why i created the patch in the first place | 16:02 |
sean-k-mooney | i wrote it after we had the denver fourm session on closing the gaps | 16:02 |
sean-k-mooney | then it didnt really get reviewed for a few months and i kind of lost interest | 16:03 |
prometheanfire | yarp | 16:04 |
*** derekh has quit IRC | 16:15 | |
*** lpetrut has quit IRC | 16:21 | |
*** mlavalle has quit IRC | 16:22 | |
*** mlavalle has joined #openstack-nova | 16:23 | |
*** ociuhandu has quit IRC | 16:27 | |
*** dougsz has quit IRC | 16:38 | |
*** hamalq has joined #openstack-nova | 16:38 | |
*** gregwork has joined #openstack-nova | 16:39 | |
*** hamalq_ has joined #openstack-nova | 16:41 | |
*** hamalq has quit IRC | 16:44 | |
*** gyee has joined #openstack-nova | 16:49 | |
*** openstackgerrit has joined #openstack-nova | 16:49 | |
openstackgerrit | Elancheran S proposed openstack/nova master: Removes the delta file once image is extracted https://review.opendev.org/734421 | 16:49 |
*** priteau has quit IRC | 16:51 | |
*** priteau has joined #openstack-nova | 16:52 | |
*** xek has quit IRC | 16:55 | |
*** mvorwerk has quit IRC | 16:57 | |
*** mvorwerk has joined #openstack-nova | 16:57 | |
*** dtantsur is now known as dtantsur|afk | 17:00 | |
*** ralonsoh has quit IRC | 17:00 | |
*** ralonsoh has joined #openstack-nova | 17:01 | |
*** mvorwerk has quit IRC | 17:05 | |
*** mvorwerk has joined #openstack-nova | 17:05 | |
*** ralonsoh has quit IRC | 17:06 | |
dansmith | lyarwood: in case you didn't notice, that failure you're seeing appears to be a race on the glance side that I opened a bug for and have a fix up for, | 17:07 |
*** ralonsoh has joined #openstack-nova | 17:07 | |
dansmith | but it's a pretty fundamentally-broken thing on the glance side so the fix is non-trivial and will likely take a while to get agreement on and land | 17:07 |
dansmith | lyarwood: if it's happening a lot, then I guess we could move that to non-voting and add back the base ceph job for the time being, but... it's nice to keep the pressure up for a fix too :/ | 17:08 |
lyarwood | dansmith: yeah no issues, that's only the second hit tha tI had seen against the job | 17:08 |
dansmith | ack | 17:09 |
lyarwood | dansmith: so I'm happy to leave it voting for now | 17:09 |
dansmith | it's actually cool, | 17:09 |
dansmith | because I filed that bug as "I only saw this when something else was broken, but clearly there's a real race here" and the fact that we're hitting it in the happy path is sweet sweet vindication :P | 17:09 |
lyarwood | ^_^ | 17:10 |
lyarwood | have you closed my bug out as a dup already? | 17:10 |
lyarwood | if not I can do it now | 17:10 |
dansmith | no, is that what you want? | 17:11 |
*** nightmare_unreal has quit IRC | 17:11 | |
dansmith | you've got links to logs which is good, but I guess maybe the dup linkage will keep it handy enough | 17:11 |
lyarwood | true but I can just copy that across | 17:11 |
lyarwood | feel free to close it as a dup | 17:12 |
dansmith | ack | 17:12 |
*** mvorwerk has quit IRC | 17:13 | |
*** udesale_ has quit IRC | 17:16 | |
*** mvorwerk has joined #openstack-nova | 17:23 | |
*** tesseract has quit IRC | 17:23 | |
*** mvorwerk has quit IRC | 17:27 | |
*** priteau has quit IRC | 17:33 | |
*** k_mouza has quit IRC | 17:36 | |
*** sapd1_x has quit IRC | 17:45 | |
*** lpetrut has joined #openstack-nova | 17:49 | |
*** mvorwerk has joined #openstack-nova | 17:49 | |
*** lpetrut has quit IRC | 17:50 | |
*** mvorwerk has quit IRC | 17:53 | |
*** links has quit IRC | 18:05 | |
*** mvorwerk has joined #openstack-nova | 18:14 | |
*** mvorwerk has quit IRC | 18:21 | |
*** songwenping_ has joined #openstack-nova | 18:38 | |
*** dougsz has joined #openstack-nova | 18:40 | |
*** songwenping__ has quit IRC | 18:41 | |
sean-k-mooney | gibi: finally got around to testing the WIP attach/detach patch. i commented no the review but http://paste.openstack.org/show/796572/ are the results it looks like thre are still issues with macvtap and direct phsyical. | 18:45 |
*** k_mouza has joined #openstack-nova | 18:48 | |
*** songwenping_ has quit IRC | 18:52 | |
*** songwenping_ has joined #openstack-nova | 18:52 | |
*** markvoelker has joined #openstack-nova | 18:56 | |
*** dougsz has quit IRC | 19:05 | |
*** k_mouza has quit IRC | 19:07 | |
*** vishalmanchanda has quit IRC | 19:12 | |
*** dougsz has joined #openstack-nova | 19:20 | |
*** dougsz has quit IRC | 19:24 | |
*** mtreinish has quit IRC | 19:27 | |
*** ralonsoh has quit IRC | 19:47 | |
*** mvorwerk has joined #openstack-nova | 19:52 | |
openstackgerrit | Ghanshyam Mann proposed openstack/nova master: migrate testing to ubuntu focal https://review.opendev.org/738126 | 19:54 |
*** nweinber has quit IRC | 20:09 | |
*** mvorwerk has quit IRC | 20:41 | |
*** avolkov has quit IRC | 20:48 | |
*** maciejjozefczyk has quit IRC | 20:56 | |
*** grandchild has joined #openstack-nova | 21:05 | |
*** raildo has quit IRC | 21:08 | |
*** grandchild has quit IRC | 21:19 | |
*** markvoelker has quit IRC | 21:30 | |
*** martinkennelly has quit IRC | 21:44 | |
*** slaweq has quit IRC | 21:58 | |
*** ociuhandu has joined #openstack-nova | 22:15 | |
*** ociuhandu has quit IRC | 22:20 | |
openstackgerrit | sean mooney proposed openstack/nova master: Lookup nic feature by PCI address https://review.opendev.org/739131 | 22:24 |
openstackgerrit | sean mooney proposed openstack/nova master: Lookup nic feature by PCI address https://review.opendev.org/739131 | 22:27 |
sean-k-mooney | stephenfin: gibi ^ adressed your comments then did a rebase | 22:30 |
*** tosky has quit IRC | 22:50 | |
*** tbachman has quit IRC | 22:55 | |
*** dpawlik2 has quit IRC | 22:55 | |
*** bnemec-pto has quit IRC | 22:55 | |
*** kevinbenton has quit IRC | 22:55 | |
*** tobberydberg_ has quit IRC | 22:55 | |
*** sorrison has quit IRC | 22:55 | |
*** dansmith has quit IRC | 22:55 | |
*** frickler has quit IRC | 22:55 | |
*** Corwin has quit IRC | 22:55 | |
*** tkajinam has joined #openstack-nova | 22:55 | |
*** tbachman has joined #openstack-nova | 22:57 | |
*** dpawlik2 has joined #openstack-nova | 22:57 | |
*** bnemec-pto has joined #openstack-nova | 22:57 | |
*** kevinbenton has joined #openstack-nova | 22:57 | |
*** tobberydberg_ has joined #openstack-nova | 22:57 | |
*** sorrison has joined #openstack-nova | 22:57 | |
*** dansmith has joined #openstack-nova | 22:57 | |
*** frickler has joined #openstack-nova | 22:57 | |
*** Corwin has joined #openstack-nova | 22:57 | |
*** mlavalle has quit IRC | 22:59 | |
openstackgerrit | sean mooney proposed openstack/nova master: remove deprecated schduler filters https://review.opendev.org/744800 | 23:11 |
sean-k-mooney | gmann: that os-vif failure is real | 23:35 |
sean-k-mooney | gmann: ip route change on of there funciton to return an empty list instead of raise an excption | 23:35 |
sean-k-mooney | gmann: im going to fix it | 23:35 |
gmann | sean-k-mooney: ohk, I thought it is something constraint i made incompatible for bionic and focal. | 23:36 |
sean-k-mooney | am well im testing on focal | 23:37 |
sean-k-mooney | so it might be related to focal | 23:37 |
sean-k-mooney | in any case ill fix it | 23:38 |
gmann | but the current failure is on bionic as current PS is without depends-on | 23:38 |
sean-k-mooney | oh ok then its both | 23:38 |
gmann | ok | 23:39 |
sean-k-mooney | iproute basically used to raise ipexc.NetlinkError | 23:39 |
sean-k-mooney | if you looked up an interface that did not exist | 23:39 |
sean-k-mooney | not it returns an empty list | 23:39 |
sean-k-mooney | so i guess we just dont cap that in upper-constiratins or a new release has come out | 23:40 |
sean-k-mooney | let me check | 23:40 |
sean-k-mooney | ya july 27th | 23:40 |
sean-k-mooney | https://pypi.org/project/pyroute2/#history | 23:40 |
sean-k-mooney | 0.5.13 | 23:41 |
gmann | yeah that is what installed on failing jobs | 23:42 |
sean-k-mooney | ya 0.5.12 proably will work. | 23:42 |
sean-k-mooney | let me stash my change and ill down grade it locally | 23:42 |
sean-k-mooney | ya 0.5.12 works | 23:44 |
gmann | https://review.opendev.org/#/c/743277/ | 23:45 |
gmann | sean-k-mooney: it is better to run os-vif unit test on requirement gate | 23:45 |
sean-k-mooney | we could they are really fast | 23:46 |
sean-k-mooney | Sum of execute time for each test: 0.3669 sec. | 23:46 |
sean-k-mooney | less then a second | 23:46 |
sean-k-mooney | well + installing packages :) | 23:47 |
sean-k-mooney | anyway give me 15 mins and ill file a bug and submit the fix to make it work fine with both | 23:48 |
gmann | sean-k-mooney: thanks i am going for walk so no hurry. | 23:51 |
gmann | sean-k-mooney: adding job https://review.opendev.org/#/c/744803/1 | 23:51 |
*** hamalq_ has quit IRC | 23:52 | |
sean-k-mooney | cool that will fail until we merge the os-vif fix but ill comment on that with the depens-on link once its pushed | 23:54 |
gmann | yeah | 23:55 |
sean-k-mooney | did you bump MarkupSafe in one fo the lower constratins jobs | 23:57 |
sean-k-mooney | i think i need to do that too | 23:57 |
gmann | yeah - for focal what all needed are in this - https://review.opendev.org/#/c/738130/5/lower-constraints.txt | 23:57 |
gmann | they passed with 0.5.12 | 23:58 |
sean-k-mooney | yep just found it | 23:58 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!