| *** lemko2 has joined #openstack-nova | 00:02 | |
| *** tetsuro has joined #openstack-nova | 00:03 | |
| *** lemko has quit IRC | 00:04 | |
| *** lemko2 is now known as lemko | 00:04 | |
| *** artom has quit IRC | 00:05 | |
| *** tetsuro has quit IRC | 00:07 | |
| *** tetsuro has joined #openstack-nova | 00:10 | |
| *** LinPeiWen has joined #openstack-nova | 00:32 | |
| *** rm_work has joined #openstack-nova | 00:33 | |
| *** rm_work has quit IRC | 00:34 | |
| *** freenzyfriday has joined #openstack-nova | 00:35 | |
| *** freenzyfriday has quit IRC | 00:39 | |
| *** tetsuro has quit IRC | 00:40 | |
| *** sapd1 has joined #openstack-nova | 00:48 | |
| *** sapd1 has quit IRC | 01:00 | |
| *** gyee has quit IRC | 01:08 | |
| *** yingjisun has joined #openstack-nova | 01:08 | |
| *** spatel has joined #openstack-nova | 01:09 | |
| *** spatel has quit IRC | 01:10 | |
| *** sapd1 has joined #openstack-nova | 01:17 | |
| *** sapd1 has quit IRC | 01:21 | |
| *** cgoncalves has quit IRC | 01:23 | |
| *** cgoncalves has joined #openstack-nova | 01:23 | |
| *** yankcrime has quit IRC | 01:23 | |
| *** Roamer` has quit IRC | 01:24 | |
| *** rm_work has joined #openstack-nova | 01:39 | |
| *** rm_work has quit IRC | 01:39 | |
| *** sapd1 has joined #openstack-nova | 01:49 | |
| *** sapd1 has quit IRC | 02:02 | |
| *** zzzeek has quit IRC | 02:07 | |
| *** zzzeek has joined #openstack-nova | 02:10 | |
| *** Liang__ has joined #openstack-nova | 02:11 | |
| *** arne_wiebalck has quit IRC | 02:14 | |
| *** jmlowe has quit IRC | 02:14 | |
| *** tkajinam has quit IRC | 02:14 | |
| *** ttx has quit IRC | 02:14 | |
| *** lyarwood has quit IRC | 02:14 | |
| *** StevenK has quit IRC | 02:14 | |
| *** tkajinam_ has joined #openstack-nova | 02:15 | |
| *** StevenK has joined #openstack-nova | 02:15 | |
| *** arne_wiebalck has joined #openstack-nova | 02:15 | |
| *** jmlowe has joined #openstack-nova | 02:18 | |
| *** lyarwood has joined #openstack-nova | 02:20 | |
| *** viks____ has joined #openstack-nova | 02:28 | |
| *** brinzhang_ has joined #openstack-nova | 02:30 | |
| *** brinzhang0 has quit IRC | 02:34 | |
| *** spatel has joined #openstack-nova | 02:34 | |
| *** rm_work has joined #openstack-nova | 02:34 | |
| *** rm_work has quit IRC | 02:34 | |
| *** brinzhang_ has quit IRC | 02:35 | |
| *** brinzhang has joined #openstack-nova | 02:36 | |
| *** spatel has quit IRC | 02:48 | |
| *** rcernin has quit IRC | 02:55 | |
| *** rcernin_ has joined #openstack-nova | 02:56 | |
| openstackgerrit | wu.shiming proposed openstack/os-vif master: Fix hacking min version to 3.0.1 https://review.opendev.org/755168 | 03:05 |
|---|---|---|
| *** yonglihe has quit IRC | 03:10 | |
| *** sapd1 has joined #openstack-nova | 03:15 | |
| *** sapd1 has quit IRC | 03:19 | |
| *** psachin has joined #openstack-nova | 03:31 | |
| *** psachin has quit IRC | 03:32 | |
| *** sapd1 has joined #openstack-nova | 03:33 | |
| *** psachin has joined #openstack-nova | 03:33 | |
| *** sapd1 has quit IRC | 03:38 | |
| *** rm_work has joined #openstack-nova | 03:40 | |
| *** sapd1 has joined #openstack-nova | 03:44 | |
| *** sapd1 has quit IRC | 03:48 | |
| *** sapd1 has joined #openstack-nova | 03:49 | |
| *** Liang__ has quit IRC | 04:01 | |
| *** freenzyfriday has joined #openstack-nova | 04:13 | |
| *** yingjisun has quit IRC | 04:13 | |
| *** freenzyfriday has quit IRC | 04:17 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-nova | 04:33 | |
| *** hongbin has quit IRC | 05:07 | |
| *** freenzyfriday has joined #openstack-nova | 05:07 | |
| *** brinzhang_ has joined #openstack-nova | 05:07 | |
| *** brinzhang_ has quit IRC | 05:07 | |
| *** brinzhang_ has joined #openstack-nova | 05:08 | |
| *** ratailor has joined #openstack-nova | 05:08 | |
| *** brinzhang has quit IRC | 05:10 | |
| *** freenzyfriday has quit IRC | 05:16 | |
| *** yingjisun has joined #openstack-nova | 05:20 | |
| *** sapd1 has quit IRC | 05:27 | |
| *** brinzhang has joined #openstack-nova | 05:59 | |
| *** brinzhang_ has quit IRC | 06:00 | |
| *** links has joined #openstack-nova | 06:01 | |
| *** sapd1 has joined #openstack-nova | 06:11 | |
| *** brinzhang_ has joined #openstack-nova | 06:14 | |
| *** brinzhang has quit IRC | 06:14 | |
| *** lpetrut has joined #openstack-nova | 06:24 | |
| *** freenzyfriday has joined #openstack-nova | 06:30 | |
| *** maciejjozefczyk_ has joined #openstack-nova | 06:32 | |
| *** sapd1 has quit IRC | 06:35 | |
| *** dklyle has quit IRC | 06:38 | |
| *** ralonsoh has joined #openstack-nova | 06:38 | |
| *** brinzhang_ has quit IRC | 06:50 | |
| *** bbowen has quit IRC | 07:04 | |
| *** bbowen has joined #openstack-nova | 07:05 | |
| *** rcernin_ has quit IRC | 07:06 | |
| *** slaweq has joined #openstack-nova | 07:09 | |
| *** brinzhang has joined #openstack-nova | 07:11 | |
| *** tesseract has joined #openstack-nova | 07:16 | |
| *** ttx has joined #openstack-nova | 07:17 | |
| *** rcernin_ has joined #openstack-nova | 07:17 | |
| *** rcernin_ has quit IRC | 07:20 | |
| *** rcernin has joined #openstack-nova | 07:20 | |
| *** tosky has joined #openstack-nova | 07:30 | |
| *** maciejjozefczyk_ has quit IRC | 07:34 | |
| *** maciejjozefczyk_ has joined #openstack-nova | 07:34 | |
| *** ociuhandu has joined #openstack-nova | 07:50 | |
| *** ociuhandu has quit IRC | 07:54 | |
| *** links has quit IRC | 07:59 | |
| *** links has joined #openstack-nova | 08:07 | |
| *** brinzhang_ has joined #openstack-nova | 08:07 | |
| *** ociuhandu has joined #openstack-nova | 08:09 | |
| *** brinzhang has quit IRC | 08:11 | |
| ralonsoh | hi folks, do you know what is this message? https://a5e1b7092bd931de9d7c-99461a827f8c9e81159099d5f417814c.ssl.cf1.rackcdn.com/738163/21/check/neutron-ovn-tempest-full-multinode-ovs-master/462d24b/compute1/logs/screen-n-cpu.txt | 08:11 |
| ralonsoh | --> http://paste.openstack.org/show/798558/ | 08:12 |
| ralonsoh | sean-k-mooney, stephenfin ^^ | 08:12 |
| ralonsoh | if you know that | 08:12 |
| lyarwood | bauzas: https://review.opendev.org/#/c/752486/ & https://review.opendev.org/#/c/749233/ & https://review.opendev.org/#/c/729178/ & https://review.opendev.org/#/c/744552/ - would you mind taking a look at these stable backports today if you have time? | 08:12 |
| bauzas | lyarwood: rushing to finish my talk recording this morning, hopefully would be able to review later today | 08:24 |
| bauzas | 7 mins left for recording | 08:24 |
| bauzas | and I can't count the number of rushes i already had | 08:24 |
| lyarwood | bauzas: ack np good luck! | 08:26 |
| *** k_mouza has joined #openstack-nova | 08:28 | |
| frickler | ralonsoh: that looks like https://bugs.launchpad.net/nova/+bug/1863021 but I assumed that that bug had long been fixed everywhere | 08:37 |
| openstack | Launchpad bug 1863021 in OpenStack Object Storage (swift) "[SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError" [Undecided,In progress] - Assigned to Chris MacNaughton (chris.macnaughton) | 08:37 |
| frickler | maybe not in ovs? | 08:39 |
| stephenfin | ralonsoh: frickler: Yeah, that should have been fixed with github.com/openstack/nova/commit/2078ef850c9 | 08:39 |
| stephenfin | although that's using a subprocess so maybe the fix is needed elsewhere | 08:39 |
| ralonsoh | frickler, stephenfin thanks a lot | 08:41 |
| *** derekh has joined #openstack-nova | 08:45 | |
| *** rcernin has quit IRC | 08:48 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova-specs master: Add modernize-os-hypervisors-api spec https://review.opendev.org/755109 | 09:18 |
| lyarwood | stephenfin: haha is that poop emoji supposed to be in there | 09:20 |
| stephenfin | yes | 09:20 |
| lyarwood | hahaha | 09:20 |
| *** LinPeiWen has quit IRC | 09:31 | |
| *** tesseract has quit IRC | 09:32 | |
| *** derekh has quit IRC | 09:32 | |
| *** derekh has joined #openstack-nova | 09:33 | |
| *** tesseract has joined #openstack-nova | 09:33 | |
| *** tesseract has quit IRC | 09:33 | |
| *** tesseract has joined #openstack-nova | 09:34 | |
| *** ab-a has quit IRC | 09:35 | |
| tobias-urdin | lyarwood: can i borrow your brain for 2 minutes? :) trying to troubleshoot an issue, am i assuming correctly that for a volume-backed instance being resized, and scheduled to another node than it resides on, _is_storage_shared_with() would return false since (source host != dest host) and remotefs.create_dir() would be executed over a SSH connection even if it's volume-backed, so the code | 09:37 |
| tobias-urdin | path is not only for instances with ephemeral disks | 09:37 |
| tobias-urdin | https://github.com/openstack/nova/blob/stable/train/nova/virt/libvirt/driver.py#L9860 | 09:37 |
| bauzas | holy shit, 32 mins recorded for a 25 mins session /o\ | 09:38 |
| lyarwood | tobias-urdin: hey sure | 09:38 |
| bauzas | I need to reshot all my slides... | 09:38 |
| tobias-urdin | i.e a resize with shared storage would still issue a SSH connection to the destination compute (from the source compute) to create the instance basepath in /var/lib/nova/instances/<uuid> | 09:38 |
| bauzas | Christopher Nolan, be aware I'm coming | 09:38 |
| lyarwood | tobias-urdin: yeah even with volume backed instances we still have to copy some instance files over to the dest | 09:39 |
| tobias-urdin | in theory, if we were to implement a RPC call to create those instance files on dest we could, if all storage is shared, drop the requirement of SSH between compute nodes source -> dest? | 09:40 |
| lyarwood | tobias-urdin: yes we could move this up into the compute layer, this is just legacy cruft in the driver that no one has wanted to change | 09:40 |
| tobias-urdin | lyarwood: roger that, thanks for confirming for a confused brain :) | 09:41 |
| lyarwood | tobias-urdin: it would take a spec however as it will likely impact a few virt drivers doing the same kind of thing in migrate_disk_and_power_off | 09:42 |
| lyarwood | tobias-urdin: or did you mean call the compute rpc_api from the libvirt driver instead of ssh? | 09:42 |
| lyarwood | tobias-urdin: I don't think we've allowed that in the past | 09:43 |
| lyarwood | tobias-urdin: but we could look into it again in W | 09:43 |
| tobias-urdin | i see, either way just wanted to confirm since we were hitting issues with resizing between nodes, getting rid of ssh connection would be preferrable somehow | 09:44 |
| frickler | bauzas: just play back at 32 fps instead of 25? ;) | 09:45 |
| tobias-urdin | i assume that would mean abstracting away the action so that the manager does a rpc call to the destination host, which as you said would mean a change for all drivers to implement some new method to prepare the dest node (or not implement it at all if not needed) | 09:46 |
| tobias-urdin | lyarwood: appreciate it! not gonna steal more of your time :) | 09:46 |
| lyarwood | yeah something like that, it would be easier to call from the virt driver but like I said there's no prior art for doing that and I think that's by design | 09:46 |
| lyarwood | tobias-urdin: anytime! | 09:47 |
| lyarwood | gmann / stephenfin: https://review.opendev.org/#/c/754684/ - would you mind hitting this btw? | 09:47 |
| lyarwood | thanks | 09:52 |
| openstackgerrit | Wenping Song proposed openstack/nova master: Add instance project_id for cyborg arq https://review.opendev.org/738428 | 09:55 |
| *** songwenping__ has joined #openstack-nova | 09:58 | |
| *** xek has joined #openstack-nova | 10:04 | |
| *** sapd1 has joined #openstack-nova | 10:17 | |
| *** salmankhan has joined #openstack-nova | 10:28 | |
| *** salmankhan1 has joined #openstack-nova | 10:31 | |
| *** salmankhan has quit IRC | 10:32 | |
| *** salmankhan1 is now known as salmankhan | 10:32 | |
| sean-k-mooney | tobias-urdin: you dont have to usee ssh for resize | 10:33 |
| sean-k-mooney | tobias-urdin: you can use rsync instead | 10:33 |
| sean-k-mooney | we can also use an rpc instead fo the current test we use for detecting shared storage but that would require a spec/blueprint | 10:34 |
| sean-k-mooney | tobias-urdin: lyarwood alternitivly we could finally stop supporting implict shared storgage | 10:35 |
| lyarwood | sean-k-mooney: doubt that would fly with operators | 10:35 |
| sean-k-mooney | e.g. if you put the nova state dir on shared storage you must set a config flag instead of detecting it | 10:35 |
| lyarwood | sean-k-mooney: oh, well yeah that would be nicer | 10:36 |
| sean-k-mooney | yep an ooo or whatever could set that | 10:36 |
| lyarwood | also TIL about [libvirt]/remote_filesystem_transport | 10:36 |
| lyarwood | I didn't think rsync would support that | 10:36 |
| sean-k-mooney | rsync can be used to copy the disk | 10:36 |
| sean-k-mooney | i am not sure about the detection of the shared filesystem | 10:37 |
| sean-k-mooney | but ya https://github.com/openstack/nova/blob/cff7382fb3de4eef7aeddc34b6ae3409cbe50ea0/nova/virt/libvirt/volume/remotefs.py#L201 | 10:37 |
| lyarwood | yeah looks like it can also do that | 10:38 |
| sean-k-mooney | oh cause it can create the file on the dest and see if it shows up on the source | 10:38 |
| lyarwood | yeah | 10:38 |
| lyarwood | haha we create the dir and rsync it across, of course. | 10:38 |
| sean-k-mooney | tobias-urdin: what was you issue by the way i did not read all of scrollback | 10:40 |
| lyarwood | I don't think there was an issue, he just wanted to drop the use of ssh | 10:41 |
| sean-k-mooney | ah ok | 10:41 |
| *** artom has joined #openstack-nova | 10:41 | |
| sean-k-mooney | mnaser ask about the same about 2 weeks ago. e.g. could you drop ssh | 10:41 |
| sean-k-mooney | i suggested an rpc at that time too although the config value occured to me after | 10:42 |
| tobias-urdin | sean-k-mooney: my deployment assumed that no ssh access was required and since our resize operations never changed the destination host in the past i never hit the ssh issue | 10:44 |
| tobias-urdin | and I was assuming ssh access was only needed when supporting ephemeral disks | 10:44 |
| tobias-urdin | (we run soley on cinder volumes and images_type=rbd backend) | 10:44 |
| sean-k-mooney | i see | 10:46 |
| sean-k-mooney | images_type=rbd should not need this path | 10:46 |
| sean-k-mooney | we know that is shared | 10:46 |
| sean-k-mooney | so for that case we do not need to check | 10:46 |
| tobias-urdin | which is what i thought as well, apparantely not since resize fails trying to ssh to the destination host | 10:47 |
| *** haleyb has quit IRC | 10:47 | |
| sean-k-mooney | we dont support resizing across images_type | 10:47 |
| tobias-urdin | i've been trying to follow the code path | 10:47 |
| sean-k-mooney | tobias-urdin: no i mean we can file a bug for that specific case | 10:47 |
| sean-k-mooney | and we can fix that it should not be needed for rbd | 10:47 |
| sean-k-mooney | we can just add an if around the call and check if the images type is knonw to be shared | 10:48 |
| lyarwood | sean-k-mooney: just reading back, no we still need to check even with images_type=rbd | 10:50 |
| tobias-urdin | that would be great, how can we determine if it's shared? | 10:50 |
| sean-k-mooney | lyarwood: why would we | 10:50 |
| lyarwood | sean-k-mooney: the instance directory and instance storage being shared are two different things | 10:50 |
| sean-k-mooney | lyarwood: we have never supported resizing between host with different backends | 10:50 |
| sean-k-mooney | lyarwood: oh ya your right | 10:50 |
| sean-k-mooney | lyarwood: actully didnt we skip this and revert it for rbd before | 10:51 |
| lyarwood | yeah | 10:51 |
| lyarwood | it's checking again now | 10:51 |
| lyarwood | as the skip broke when the instance directory wasn't shared | 10:51 |
| sean-k-mooney | ya | 10:51 |
| sean-k-mooney | is there anything in the instance directory we care about by the way | 10:52 |
| sean-k-mooney | config drive maybe? | 10:52 |
| lyarwood | yeah that the console log | 10:52 |
| *** slaweq_ has joined #openstack-nova | 10:52 | |
| sean-k-mooney | the console log not really | 10:52 |
| lyarwood | and maybe other things, I forget | 10:52 |
| sean-k-mooney | it will be cleared when the vm is started again | 10:52 |
| lyarwood | right but this path is also used during LM | 10:52 |
| sean-k-mooney | ah ok | 10:53 |
| *** dansmith has quit IRC | 10:53 | |
| sean-k-mooney | ya so config option of rpc are the only things we can do then | 10:53 |
| sean-k-mooney | well in the rbd case there is one other thing we could do but its a bit of a hack | 10:54 |
| sean-k-mooney | mount the instance dir in an rbd volumn | 10:54 |
| sean-k-mooney | or on cephfs but nova doing that is a bit of a hack | 10:55 |
| sean-k-mooney | its really something the installer should do | 10:55 |
| *** slaweq has quit IRC | 10:55 | |
| *** dansmith has joined #openstack-nova | 10:55 | |
| *** tobias-urdin is now known as tobias-urdin|afk | 10:57 | |
| *** StevenK has quit IRC | 10:58 | |
| *** StevenK has joined #openstack-nova | 10:58 | |
| lyarwood | sean-k-mooney: yeah this is all just debt from the early days of Nova | 10:59 |
| lyarwood | sean-k-mooney: we could rework this now or just live with it | 10:59 |
| noonedeadpunk | hey everyone! question: is it possible that during resize siolated aggregates are just ignored? | 11:07 |
| noonedeadpunk | as I have a host which is in the isolated agrregate, and vm's are not created on it. but, during vm resize it catched it... | 11:08 |
| *** yingjisun has quit IRC | 11:08 | |
| noonedeadpunk | it's on train though | 11:08 |
| ralonsoh | frickler, sorry again. What do you mean with "not in ovs"? | 11:09 |
| ralonsoh | related to http://paste.openstack.org/show/798558/ | 11:09 |
| ralonsoh | maybe I'm wrong but this command is executed from nova-compute | 11:10 |
| ralonsoh | I checked oslo_concurrency but this is just a library and fix should be in nova too | 11:10 |
| sean-k-mooney | noonedeadpunk: i dont think it should change the behavior | 11:11 |
| sean-k-mooney | noonedeadpunk: which form of isolated aggreates are you using | 11:11 |
| sean-k-mooney | noonedeadpunk: tenant isolation or the placement/traits based isolated aggreates | 11:11 |
| *** dtantsur|afk is now known as dtantsur | 11:11 | |
| noonedeadpunk | placement traits | 11:12 |
| noonedeadpunk | but once you said that... | 11:12 |
| sean-k-mooney | we shoudl be using the traits request form the new flavor | 11:12 |
| noonedeadpunk | how can I make tenant isolation?:) | 11:12 |
| noonedeadpunk | so right now aggregate looks like this in nova http://paste.openstack.org/show/798568/ | 11:13 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/scheduler/filters/aggregate_multitenancy_isolation.py | 11:13 |
| noonedeadpunk | and no flavors or images have this trait set.. | 11:13 |
| sean-k-mooney | ralonsoh: that traceback i think is an eventlet issue | 11:13 |
| ralonsoh | sean-k-mooney, yes, related to https://bugs.launchpad.net/nova/+bug/1863021 | 11:14 |
| openstack | Launchpad bug 1863021 in OpenStack Object Storage (swift) "[SRU] eventlet monkey patch results in assert len(_active) == 1 AssertionError" [Undecided,In progress] - Assigned to Chris MacNaughton (chris.macnaughton) | 11:14 |
| ralonsoh | but that should be already fixed in Nova | 11:14 |
| noonedeadpunk | sean-k-mooney: I guess tenant aggregation will allow other tenants to create instances in this aggregate? | 11:14 |
| sean-k-mooney | ralonsoh: ya so the approch they are takign to fix it is a hack | 11:14 |
| noonedeadpunk | *tenant isolation | 11:14 |
| sean-k-mooney | ralonsoh: the real fix is to fix eventlets | 11:14 |
| ralonsoh | sean-k-mooney, I know, but works for now | 11:15 |
| ralonsoh | sean-k-mooney, what I don't understand is why this is failing when calling processutils.execute | 11:15 |
| sean-k-mooney | ralonsoh: because olso need to be fixed | 11:15 |
| ralonsoh | sean-k-mooney, but this is just a library | 11:15 |
| sean-k-mooney | ralonsoh: im pretty sure i sent an email about this a few months ago | 11:16 |
| sean-k-mooney | ralonsoh: yes | 11:16 |
| ralonsoh | sean-k-mooney, sorry, I don't get it | 11:16 |
| sean-k-mooney | ralonsoh: this need to be fixed in the libs too | 11:16 |
| *** sapd1 has quit IRC | 11:16 | |
| sean-k-mooney | we use processutils.execute comes form oslo | 11:16 |
| ralonsoh | sean-k-mooney, but, and I could be wrong, when we call .execute, the fix is called | 11:16 |
| ralonsoh | we are calling this from a monkey patched process with the "fix" | 11:17 |
| sean-k-mooney | not if its executed via privsep | 11:17 |
| ralonsoh | oh my... | 11:17 |
| ralonsoh | ok, got it!!! | 11:17 |
| sean-k-mooney | noonedeadpunk: you enabled the placment prefilter for this in the nova.conf too right | 11:17 |
| sean-k-mooney | noonedeadpunk: https://docs.openstack.org/nova/latest/configuration/config.html#scheduler.enable_isolated_aggregate_filtering | 11:19 |
| noonedeadpunk | sean-k-mooney: I think I just set `enable_isolated_aggregate_filtering` | 11:19 |
| noonedeadpunk | yeah | 11:19 |
| noonedeadpunk | and actually VMs are not created on host. the only one which is wrong placed for the last month is this one | 11:19 |
| sean-k-mooney | noonedeadpunk: by the way you can also do tenant isolation with placement too | 11:20 |
| noonedeadpunk | which was spawned there after resize | 11:20 |
| sean-k-mooney | noonedeadpunk: did you way your using train | 11:20 |
| noonedeadpunk | sorry didn't get the question:( | 11:21 |
| sean-k-mooney | what release are you running train? | 11:21 |
| sean-k-mooney | i start typeing "by the way are you using train" the swaped to "did you say your using train" in my head and type a mix of the two hehe | 11:22 |
| sean-k-mooney | there is nothing here that is resize specific | 11:23 |
| sean-k-mooney | https://github.com/openstack/nova/blob/stable/train/nova/scheduler/request_filter.py#L31-L88 | 11:23 |
| noonedeadpunk | it's a... 20.2.1.dev1, which to be exact 118ee682571a4bd41c8009dbe2e47fdd1f85a630 | 11:23 |
| sean-k-mooney | noonedeadpunk: can you check for that "'Request filter %r took %.1f seconds'," debug message if you have debug logs | 11:24 |
| sean-k-mooney | the simpelst way to check would be to boot a vm and set the schduler into debug mode then resize it | 11:24 |
| sean-k-mooney | and confirm the isolate_aggregates prefilter is run | 11:25 |
| sean-k-mooney | but they shoudl all be run https://github.com/openstack/nova/blob/stable/train/nova/scheduler/request_filter.py#L221-L237 | 11:25 |
| noonedeadpunk | sean-k-mooney: I thiink I have only info level, not debug one... | 11:34 |
| sean-k-mooney | thats ok. with a quick look i dont see any reason that it woudl be skipped | 11:34 |
| *** haleyb has joined #openstack-nova | 11:34 | |
| noonedeadpunk | I see... | 11:34 |
| sean-k-mooney | if it is being skipped for some reason then a functional test should be able to repoduces it | 11:35 |
| sean-k-mooney | this logic should be driver independent | 11:36 |
| noonedeadpunk | have the same ALL_REQUEST_FILTERS here | 11:38 |
| noonedeadpunk | sean-k-mooney: ok, thanks for your help, will try to reproduce it and catch logs | 11:39 |
| sean-k-mooney | if you do feel free to file a bug with the steps. we shoudl be able to trun it into a functional regression test and then fix it but im not aware of something that would cause it to break currently | 11:40 |
| sean-k-mooney | that does not mean it cant just not seeing it quickly | 11:40 |
| *** tesseract has quit IRC | 11:40 | |
| *** tesseract has joined #openstack-nova | 11:42 | |
| *** tobias-urdin|afk is now known as tobias-urdin | 11:45 | |
| tobias-urdin | sean-k-mooney: sorry was afk for a while, should it be treated as a bug? | 11:46 |
| sean-k-mooney | tobias-urdin: no we previusly had a bug for this for rbd then meged a skip and reverted it | 11:47 |
| sean-k-mooney | tobias-urdin: the use of rbd does not put the instance state dir on shared storage | 11:47 |
| sean-k-mooney | tobias-urdin: this could be a blueprint/spec | 11:48 |
| sean-k-mooney | but its a new feature not a bug | 11:48 |
| tobias-urdin | sean-k-mooney: ok, thanks :) | 11:48 |
| sean-k-mooney | tobias-urdin: its one i would not mind seeing added by the way. i think it would be useful | 11:48 |
| tobias-urdin | sean-k-mooney: i agree | 11:49 |
| *** rcernin has joined #openstack-nova | 11:50 | |
| noonedeadpunk | sean-k-mooney: looking at code, I have one question - should I need another aggregate which would include the rest of the hosts? | 11:55 |
| *** raildo has joined #openstack-nova | 11:55 | |
| sean-k-mooney | noonedeadpunk: off the top of my head i dont think so | 11:56 |
| noonedeadpunk | ok, just to double check, as I don't have one | 11:56 |
| noonedeadpunk | I mean I have the only aggregate with required trait | 11:56 |
| sean-k-mooney | all the rest will be in the implcit nova aggreate | 11:56 |
| noonedeadpunk | was just thinking how this works. https://github.com/openstack/nova/blob/stable/train/nova/scheduler/request_filter.py#L74-L76 Like we get list of aggregates without the trait? | 11:59 |
| sean-k-mooney | yes so we first get all the required traits from the instance | 12:02 |
| sean-k-mooney | then we all the aggreates that have require traits that are not in the set we require for the instnace | 12:03 |
| sean-k-mooney | we then add those as forbiden aggreates to the placment query | 12:03 |
| noonedeadpunk | ah, ok, yeah | 12:03 |
| sean-k-mooney | so since you have no required traits in the flaovr or image for that instance | 12:03 |
| sean-k-mooney | it will get back all aggreates with a require trait | 12:03 |
| sean-k-mooney | in this case the one aggreate you created | 12:04 |
| sean-k-mooney | and then add those to the forbidden list | 12:04 |
| noonedeadpunk | ok, thanks so much for explanation | 12:04 |
| sean-k-mooney | that becmoes a "mem_of=!<aggreate uuid>" paramter to placemnt | 12:04 |
| sean-k-mooney | ! being not | 12:05 |
| openstack | sean-k-mooney: Error: "being" is not a valid command. | 12:05 |
| noonedeadpunk | lol | 12:05 |
| sean-k-mooney | hehe | 12:05 |
| sean-k-mooney | sorry that hsoul be member_of | 12:05 |
| *** efried has quit IRC | 12:06 | |
| sean-k-mooney | anyway you get the point we prefix the uuid with ! whenever we want to exclude it | 12:06 |
| *** efried has joined #openstack-nova | 12:07 | |
| noonedeadpunk | yes, sure, thanks again) | 12:07 |
| *** spatel has joined #openstack-nova | 12:11 | |
| *** spatel has quit IRC | 12:15 | |
| *** rcernin has quit IRC | 12:16 | |
| *** ratailor has quit IRC | 12:37 | |
| *** lbragstad has quit IRC | 12:42 | |
| *** spatel has joined #openstack-nova | 12:54 | |
| *** spatel has quit IRC | 12:56 | |
| *** yingjisun has joined #openstack-nova | 12:58 | |
| *** yingjisun has quit IRC | 12:58 | |
| *** links has quit IRC | 13:00 | |
| *** derekh has quit IRC | 13:01 | |
| *** derekh has joined #openstack-nova | 13:03 | |
| *** nweinber has joined #openstack-nova | 13:03 | |
| *** tkajinam_ has quit IRC | 13:12 | |
| *** lbragstad has joined #openstack-nova | 13:16 | |
| *** priteau has joined #openstack-nova | 13:24 | |
| dansmith | lyarwood: I may have missed something about the live migration ceph job.. this is a grenade job, but this is the one I care most about: https://review.opendev.org/#/c/742056/6/.zuul.yaml@90 | 13:33 |
| *** slaweq_ has quit IRC | 13:33 | |
| dansmith | which looks like it has already been removed and we're not running, right? | 13:33 |
| *** yankcrime has joined #openstack-nova | 13:33 | |
| lyarwood | dansmith: it's still there just hidden in the original nova-live-migration job | 13:36 |
| lyarwood | dansmith: https://github.com/openstack/nova/blob/2beb18444fccac2ffe73292423ff801a9f327f49/.zuul.yaml#L59-L70 | 13:36 |
| lyarwood | dansmith: https://github.com/openstack/nova/blob/2beb18444fccac2ffe73292423ff801a9f327f49/playbooks/legacy/nova-live-migration/run.yaml#L51-L54 | 13:37 |
| lyarwood | dansmith: https://github.com/openstack/nova/blob/2beb18444fccac2ffe73292423ff801a9f327f49/gate/live_migration/hooks/run_tests.sh#L55-L65 | 13:37 |
| dansmith | lyarwood: confusing, but okay.. so *that* is the thing I think is important not to lose, but grenade I guess I'm less worried about | 13:37 |
| lyarwood | kk ack, I've -W'd my switch over change until I've covered that in a ceph specific job | 13:38 |
| dansmith | I'm still worried about losing pressure to close the loop and these gaps sticking around for a while, but if the basic multinode live migration switcheroo bit is still there, then I don't want to block smaller process hacks like the grenade bit if that makes it easier | 13:38 |
| lyarwood | np this shouldn't take long to work out and backport to stable/victoria so we can close the zuulv3 stuff out there tbh | 13:39 |
| lyarwood | I've worked out how to copy the config and keys around already | 13:39 |
| lyarwood | in here https://review.opendev.org/#/c/755270/ | 13:40 |
| lyarwood | but I think the role needs to land in devstack itself | 13:40 |
| dansmith | ah, wait wait, | 13:40 |
| dansmith | gmann's patch *does* remove that chunk: https://review.opendev.org/#/c/752557/8/.zuul.yaml@a68 | 13:41 |
| dansmith | you were saying that job isn't already gone, | 13:41 |
| dansmith | but his patch *does* reduce that coverage... | 13:41 |
| lyarwood | that's my change | 13:41 |
| lyarwood | and is -W'd | 13:41 |
| lyarwood | until we get the coverage sorted | 13:42 |
| dansmith | okay, I guess I'm not sure which that is, | 13:42 |
| lyarwood | I'll add a comment | 13:42 |
| dansmith | ah, this guy: https://review.opendev.org/#/c/752557/ | 13:43 |
| lyarwood | that's the same change as above | 13:44 |
| dansmith | lol | 13:44 |
| lyarwood | https://review.opendev.org/#/c/742056/ is the grenade change | 13:45 |
| dansmith | I know, I just didn't realize that was yours, thought it was gmann's because he had the -W until a minute ago | 13:45 |
| lyarwood | ah np | 13:45 |
| dansmith | yeah, I got it now | 13:45 |
| *** slaweq has joined #openstack-nova | 13:46 | |
| dansmith | btw, I'm super happy to see you guys also getting spanked by zuul on syntax stuff | 13:46 |
| dansmith | fixing up glance's jobs, zuul was getting sick of me throwing up patches that didn't even parse :) | 13:46 |
| lyarwood | haha yeah, all the damn time | 13:46 |
| lyarwood | wish there was a way of running some of this stuff locally | 13:47 |
| dansmith | there needs to be a separately-installable zuulint tool or something we can throw into a tox env | 13:47 |
| dansmith | yeah | 13:47 |
| *** maciejjozefczyk_ has quit IRC | 13:54 | |
| *** kaisers has joined #openstack-nova | 13:55 | |
| *** k_mouza has quit IRC | 13:59 | |
| *** haleyb has quit IRC | 14:01 | |
| sean-k-mooney | lyarwood: there is a docker compose file for running zull quickly | 14:02 |
| sean-k-mooney | but ya gettign the gate jobs to work is non trivial | 14:02 |
| sean-k-mooney | the devstack/tempest based jobs pull in a lot of repos so you need to donload a lot of repos to get them to work unless you take a few steps that mean your nolonger really mirrioring what happens in the gate | 14:03 |
| sean-k-mooney | dansmith: but ya a zuul-lint tool would be nice | 14:04 |
| *** haleyb has joined #openstack-nova | 14:05 | |
| *** maciejjozefczyk_ has joined #openstack-nova | 14:10 | |
| *** jdillaman has joined #openstack-nova | 14:15 | |
| *** bhagyashri|rover is now known as bhagyashri | 14:20 | |
| *** xek has quit IRC | 14:32 | |
| *** k_mouza has joined #openstack-nova | 14:34 | |
| *** maciejjozefczyk_ has quit IRC | 14:36 | |
| *** k_mouza has quit IRC | 14:39 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: Replace nova-live-migration with zuulv3 jobs https://review.opendev.org/752557 | 14:47 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: Merge nova-evacuate into nova-multinode-live-migration https://review.opendev.org/755309 | 14:47 |
| *** mlavalle has joined #openstack-nova | 14:48 | |
| *** freenzyfriday has quit IRC | 14:49 | |
| *** gyee has joined #openstack-nova | 14:59 | |
| *** lpetrut has quit IRC | 15:04 | |
| *** xek has joined #openstack-nova | 15:06 | |
| *** eharney has quit IRC | 15:06 | |
| *** dklyle has joined #openstack-nova | 15:11 | |
| *** lpetrut has joined #openstack-nova | 15:14 | |
| *** david-lyle has joined #openstack-nova | 15:16 | |
| *** david-lyle has quit IRC | 15:17 | |
| *** k_mouza has joined #openstack-nova | 15:17 | |
| *** lpetrut has quit IRC | 15:27 | |
| *** belmoreira has joined #openstack-nova | 15:33 | |
| *** openstackgerrit has quit IRC | 15:46 | |
| *** k_mouza_ has joined #openstack-nova | 16:03 | |
| *** k_mouza has quit IRC | 16:06 | |
| *** tosky has quit IRC | 16:19 | |
| *** salmankhan1 has joined #openstack-nova | 16:19 | |
| *** salmankhan has quit IRC | 16:22 | |
| *** salmankhan1 is now known as salmankhan | 16:22 | |
| *** xek has quit IRC | 16:24 | |
| *** psachin has quit IRC | 16:34 | |
| *** tesseract has quit IRC | 16:51 | |
| *** ociuhandu has quit IRC | 16:57 | |
| *** derekh has quit IRC | 17:00 | |
| *** dtantsur is now known as dtantsur|afk | 17:23 | |
| *** bbowen has quit IRC | 17:33 | |
| *** k_mouza_ has quit IRC | 17:40 | |
| *** brinzhang0 has joined #openstack-nova | 17:41 | |
| *** jobewan has joined #openstack-nova | 17:43 | |
| *** brinzhang_ has quit IRC | 17:44 | |
| *** salmankhan has quit IRC | 17:45 | |
| *** openstackgerrit has joined #openstack-nova | 17:46 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/victoria: [stable-only] nova-live-migration: Disable libvirtd during negative tests https://review.opendev.org/754688 | 17:46 |
| openstackgerrit | melanie witt proposed openstack/nova master: Add note about setting CONF.max_disk_devices_to_attach = 0 https://review.opendev.org/755360 | 17:49 |
| *** hamalq has joined #openstack-nova | 17:52 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add note about setting CONF.max_disk_devices_to_attach = 0 https://review.opendev.org/755360 | 17:56 |
| *** eharney has joined #openstack-nova | 17:58 | |
| *** ralonsoh has quit IRC | 18:11 | |
| *** hamalq_ has joined #openstack-nova | 18:29 | |
| *** hamalq has quit IRC | 18:31 | |
| *** bbowen has joined #openstack-nova | 18:47 | |
| *** priteau has quit IRC | 18:56 | |
| *** ociuhandu has joined #openstack-nova | 19:02 | |
| *** belmoreira has quit IRC | 19:05 | |
| *** ociuhandu has quit IRC | 19:06 | |
| *** tosky has joined #openstack-nova | 19:32 | |
| *** xek has joined #openstack-nova | 19:51 | |
| *** xek has quit IRC | 20:09 | |
| *** xek has joined #openstack-nova | 20:09 | |
| ganso | Hi Nova folks! I know most people are very busy with bugs for the RC1, but just in case you have 1 minute to spare, here is a backport to stable/ussuri that is waiting for +2s, if you could please take a look at. Thanks in advance! https://review.opendev.org/#/c/752245/ | 20:11 |
| *** xek has quit IRC | 20:14 | |
| *** xek has joined #openstack-nova | 20:15 | |
| *** nweinber has quit IRC | 20:22 | |
| sean-k-mooney | lyarwood: elod ^ you might be able to help ganso when ye are around | 20:31 |
| *** xek has quit IRC | 20:42 | |
| *** xek has joined #openstack-nova | 20:52 | |
| *** maciejjozefczyk_ has joined #openstack-nova | 20:55 | |
| *** raildo has quit IRC | 20:56 | |
| *** xek has quit IRC | 20:57 | |
| *** maciejjozefczyk_ has quit IRC | 21:10 | |
| *** maciejjozefczyk_ has joined #openstack-nova | 21:10 | |
| *** _erlon_ has joined #openstack-nova | 21:13 | |
| *** maciejjozefczyk_ has quit IRC | 21:15 | |
| *** slaweq has quit IRC | 21:39 | |
| *** slaweq has joined #openstack-nova | 21:42 | |
| *** mlavalle has quit IRC | 21:44 | |
| ganso | sean-k-mooney: thanks! | 21:58 |
| *** mlavalle has joined #openstack-nova | 22:08 | |
| *** rcernin has joined #openstack-nova | 22:12 | |
| *** slaweq has quit IRC | 22:17 | |
| *** slaweq has joined #openstack-nova | 22:18 | |
| *** slaweq has quit IRC | 22:24 | |
| *** k_mouza has joined #openstack-nova | 22:27 | |
| *** tosky has quit IRC | 22:54 | |
| *** tkajinam has joined #openstack-nova | 23:00 | |
| *** k_mouza has quit IRC | 23:00 | |
| *** mlavalle has quit IRC | 23:14 | |
| *** k_mouza has joined #openstack-nova | 23:37 | |
| *** k_mouza has quit IRC | 23:45 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!