15:03:43 <noonedeadpunk> #startmeeting openstack_ansible_meeting 15:03:43 <opendevmeet> Meeting started Tue Nov 30 15:03:43 2021 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:03:43 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:03:43 <opendevmeet> The meeting name has been set to 'openstack_ansible_meeting' 15:04:00 <noonedeadpunk> #topic bug triage 15:04:34 <jrosser> o/ hello 15:06:01 <damiandabrowski[m]> hey! 15:07:26 <noonedeadpunk> there's a new bug reported lately by jamesdenton 15:07:48 <noonedeadpunk> I was able to reproduce failure https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/819661 15:09:20 <jrosser> do we have some wierd situation in the AIO where the host is both nova_compute and ironic_compute 15:09:43 <noonedeadpunk> I think it's aio it's not the issue 15:10:03 <noonedeadpunk> but it is in the real deployments, when nova and ironic computes are different 15:10:20 <noonedeadpunk> or you meant it's the reason why patch fails? 15:10:42 <jrosser> yes 15:10:43 <noonedeadpunk> oh, it actually can be the case 15:10:49 <jrosser> like we disable running the PKI role 15:10:56 <jrosser> then libvirt blows up with no cert 15:11:06 <noonedeadpunk> I was a bit confused with https://zuul.opendev.org/t/openstack/build/93c8f19940f9489eb3189f1a9fd28753/log/job-output.txt#14820 15:11:19 <noonedeadpunk> as we kind of run pki role with nova... 15:12:17 <jrosser> confused how? 15:12:21 <noonedeadpunk> and eventually if you look into inventory it will have ironic empty 15:13:31 <noonedeadpunk> um, that pki role included and nova certs are distributed except of the CA 15:14:05 <noonedeadpunk> Also I reproduced in AIO and in invenotry-manage I see https://paste.opendev.org/show/811354/ 15:14:15 <noonedeadpunk> so no ironic defined 15:14:17 <jrosser> that is all novncproxy stuff 15:14:36 <noonedeadpunk> ah damn 15:14:40 <jrosser> and 14827 is the intermediate+CA chain 15:14:58 <noonedeadpunk> indeed 15:15:11 <jrosser> this stuff all behaves kind of strange 15:15:13 <jrosser> some wants the CA 15:15:18 <jrosser> some wants the CA+intermediate 15:15:38 <jrosser> some wants the chain for the cert, some wants the chain for the CA 15:15:40 <jrosser> totally messy 15:16:08 <noonedeadpunk> hm... so what condition there should be to satisfy us.... 15:16:52 <noonedeadpunk> also o hosts for ironic there https://zuul.opendev.org/t/openstack/build/93c8f19940f9489eb3189f1a9fd28753/log/logs/etc/host/openstack_deploy/openstack_inventory.json.txt 15:17:22 <noonedeadpunk> probably it should be "'ironic_compute' not in group_names"? 15:18:06 <jrosser> i don't really follow this 15:18:31 <jrosser> ironic_compute is an agent? or the actual ironic bare metal nodes? 15:19:27 <jrosser> but regardless i agree that "X not in Y" is prefereable to "not X in Y" 15:19:52 <jamesdenton> ironic_compute, i assume, is a node running nova-compute service but not libvirt 15:20:16 <jrosser> that sounds like a control plane thing to me 15:20:29 <jrosser> but i never did anything with ironic 15:21:30 <jamesdenton> the problem came from the task attempting to copy certs to nova_compute group (ok) and change ownership to libvirt group (not ok for ironic nodes that are members of nova_compute) 15:23:12 <jamesdenton> and by ironic nodes, i mean the controller nodes running ironic services and an instance of nova-compute for ironic (not the baremetal nodes themselves) 15:23:26 <jrosser> ahha ok 15:24:59 <jamesdenton> the compute_driver = ironic.IronicDriver for those nodes 15:25:21 <jamesdenton> vs compute_driver = libvirt.LibvirtDriver 15:25:39 <jamesdenton> so, i would not expect a legit compute node to be a member of ironic_compute 15:26:18 <jamesdenton> but a controller node running ironic could be in both ironic_compute and nova_compute 15:29:06 <noonedeadpunk> jrosser: yep, that worked out in my sandbox) 15:29:25 <jrosser> we're missing a depends-on for https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/819767 on https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/819661 ? 15:29:28 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Exclude ironic_compute hosts from PKI tasks https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/819661 15:30:00 <noonedeadpunk> jrosser: well, for some reason it passes both ways... 15:30:34 <jrosser> :) i am sooo confused right now 15:30:39 <jrosser> 819661 is failing to start libvirt 15:31:40 <noonedeadpunk> and that's why https://paste.opendev.org/show/811355/ 15:32:43 <noonedeadpunk> so things are working even without 819767 by a co-incidence I believe 15:33:25 <noonedeadpunk> and for 819661 is was really `not` in the wrong place that makes difference 15:33:39 <jamesdenton> oh, my bad 15:33:47 <jamesdenton> thanks for fixing that 15:33:55 <jrosser> feels like thats a thing the linter might spot 15:34:08 <noonedeadpunk> it probably somehow valid? 15:34:25 <noonedeadpunk> but jsut doing wrong thing for the usecase? I'm not sure... 15:34:28 <jrosser> very easy to do that and also mess up with "foo and bar | ternary(....)" which totally doesnt do what you expect either 15:34:33 <jrosser> though it's valid syntax 15:34:36 <noonedeadpunk> As eventually play is not failing because of that 15:34:46 <noonedeadpunk> yes, exactly 15:35:15 <noonedeadpunk> but I agree that it would be great to write up a rule for ansible-lint to cover that 15:35:33 <noonedeadpunk> #topic office hours 15:39:31 <jrosser> we need to make a release :) 15:42:33 <noonedeadpunk> yes 15:42:35 <noonedeadpunk> exactly 15:43:25 <noonedeadpunk> I think https://review.opendev.org/q/topic:"systemd_run_dir" is pretty critical thing to land though? 15:43:50 <noonedeadpunk> It's also great to land patches for stable branches to make bugfix releases we haven't done in a while 15:44:57 <jrosser> looks like a number of those are approved now 15:47:41 <noonedeadpunk> oh, indeed 15:47:58 <noonedeadpunk> So, should I create a branching patch now? 15:48:24 <noonedeadpunk> and we will backport things? 15:48:40 <noonedeadpunk> or see where systemd stuff will lead us? 15:48:54 <jrosser> looks like there are lots of patches in flight 15:48:57 <noonedeadpunk> eventually zun and manila seems broken 15:49:14 <jrosser> maybe if we can push to merge those in the next 24hours, +/- rabbitmq repo 15:49:14 <noonedeadpunk> well, except https://review.opendev.org/q/topic:%22config_template_collection%22+(status:open%20OR%20status:merged) not that much 15:49:40 <noonedeadpunk> and what's with rabbitmq? 15:49:54 <jrosser> you said the apt repo was broken again 15:50:02 <noonedeadpunk> ah, that 15:50:08 <noonedeadpunk> I thought smth with role) 15:50:57 <jrosser> i think we also need a patch to the tests repo for the config-template collection 15:51:17 <jrosser> still a few functional jobs hanging around 15:51:28 <andrewbonney> zun should be generally better once these patches get through, but bullseye keeps having kernel panics. I can replicate but can't diagnose 15:51:28 <noonedeadpunk> true, I totally missed that 15:51:47 <noonedeadpunk> will land it 15:52:08 <noonedeadpunk> or at least push 15:54:45 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-tests master: Add config_template to collections https://review.opendev.org/c/openstack/openstack-ansible-tests/+/819897 15:55:46 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_glance master: Use config_template as a collection https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/819854 15:56:10 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone master: Use config_template as a collection https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/819860 16:05:00 <noonedeadpunk> #endmeeting