15:03:43 <noonedeadpunk> #startmeeting openstack_ansible_meeting
15:03:43 <opendevmeet> Meeting started Tue Nov 30 15:03:43 2021 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:03:43 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:03:43 <opendevmeet> The meeting name has been set to 'openstack_ansible_meeting'
15:04:00 <noonedeadpunk> #topic bug triage
15:04:34 <jrosser> o/ hello
15:06:01 <damiandabrowski[m]> hey!
15:07:26 <noonedeadpunk> there's a new bug reported lately by jamesdenton
15:07:48 <noonedeadpunk> I was able to reproduce failure https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/819661
15:09:20 <jrosser> do we have some wierd situation in the AIO where the host is both nova_compute and ironic_compute
15:09:43 <noonedeadpunk> I think it's aio it's not the issue
15:10:03 <noonedeadpunk> but it is in the real deployments, when nova and ironic computes are different
15:10:20 <noonedeadpunk> or you meant it's the reason why patch fails?
15:10:42 <jrosser> yes
15:10:43 <noonedeadpunk> oh, it actually can be the case
15:10:49 <jrosser> like we disable running the PKI role
15:10:56 <jrosser> then libvirt blows up with no cert
15:11:06 <noonedeadpunk> I was a bit confused with https://zuul.opendev.org/t/openstack/build/93c8f19940f9489eb3189f1a9fd28753/log/job-output.txt#14820
15:11:19 <noonedeadpunk> as we kind of run pki role with nova...
15:12:17 <jrosser> confused how?
15:12:21 <noonedeadpunk> and eventually if you look into inventory it will have ironic empty
15:13:31 <noonedeadpunk> um, that pki role included and nova certs are distributed except of the CA
15:14:05 <noonedeadpunk> Also I reproduced in AIO and in invenotry-manage I see https://paste.opendev.org/show/811354/
15:14:15 <noonedeadpunk> so no ironic defined
15:14:17 <jrosser> that is all novncproxy stuff
15:14:36 <noonedeadpunk> ah damn
15:14:40 <jrosser> and 14827 is the intermediate+CA chain
15:14:58 <noonedeadpunk> indeed
15:15:11 <jrosser> this stuff all behaves kind of strange
15:15:13 <jrosser> some wants the CA
15:15:18 <jrosser> some wants the CA+intermediate
15:15:38 <jrosser> some wants the chain for the cert, some wants the chain for the CA
15:15:40 <jrosser> totally messy
15:16:08 <noonedeadpunk> hm... so what condition there should be to satisfy us....
15:16:52 <noonedeadpunk> also o hosts for ironic there https://zuul.opendev.org/t/openstack/build/93c8f19940f9489eb3189f1a9fd28753/log/logs/etc/host/openstack_deploy/openstack_inventory.json.txt
15:17:22 <noonedeadpunk> probably it should be "'ironic_compute' not in group_names"?
15:18:06 <jrosser> i don't really follow this
15:18:31 <jrosser> ironic_compute is an agent? or the actual ironic bare metal nodes?
15:19:27 <jrosser> but regardless i agree that "X not in Y" is prefereable to "not X in Y"
15:19:52 <jamesdenton> ironic_compute, i assume, is a node running nova-compute service but not libvirt
15:20:16 <jrosser> that sounds like a control plane thing to me
15:20:29 <jrosser> but i never did anything with ironic
15:21:30 <jamesdenton> the problem came from the task attempting to copy certs to nova_compute group (ok) and change ownership to libvirt group (not ok for ironic nodes that are members of nova_compute)
15:23:12 <jamesdenton> and by ironic nodes, i mean the controller nodes running ironic services and an instance of nova-compute for ironic (not the baremetal nodes themselves)
15:23:26 <jrosser> ahha ok
15:24:59 <jamesdenton> the compute_driver = ironic.IronicDriver for those nodes
15:25:21 <jamesdenton> vs compute_driver = libvirt.LibvirtDriver
15:25:39 <jamesdenton> so, i would not expect a legit compute node to be a member of ironic_compute
15:26:18 <jamesdenton> but a controller node running ironic could be in both ironic_compute and nova_compute
15:29:06 <noonedeadpunk> jrosser: yep, that worked out in my sandbox)
15:29:25 <jrosser> we're missing a depends-on for https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/819767 on https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/819661 ?
15:29:28 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Exclude ironic_compute hosts from PKI tasks  https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/819661
15:30:00 <noonedeadpunk> jrosser: well, for some reason it passes both ways...
15:30:34 <jrosser> :) i am sooo confused right now
15:30:39 <jrosser> 819661 is failing to start libvirt
15:31:40 <noonedeadpunk> and that's why https://paste.opendev.org/show/811355/
15:32:43 <noonedeadpunk> so things are working even without 819767 by a co-incidence I believe
15:33:25 <noonedeadpunk> and for 819661 is was really `not` in the wrong place that makes difference
15:33:39 <jamesdenton> oh, my bad
15:33:47 <jamesdenton> thanks for fixing that
15:33:55 <jrosser> feels like thats a thing the linter might spot
15:34:08 <noonedeadpunk> it probably somehow valid?
15:34:25 <noonedeadpunk> but jsut doing wrong thing for the usecase? I'm not sure...
15:34:28 <jrosser> very easy to do that and also mess up with "foo and bar | ternary(....)" which totally doesnt do what you expect either
15:34:33 <jrosser> though it's valid syntax
15:34:36 <noonedeadpunk> As eventually play is not failing because of that
15:34:46 <noonedeadpunk> yes, exactly
15:35:15 <noonedeadpunk> but I agree that it would be great to write up a rule for ansible-lint to cover that
15:35:33 <noonedeadpunk> #topic office hours
15:39:31 <jrosser> we need to make a release :)
15:42:33 <noonedeadpunk> yes
15:42:35 <noonedeadpunk> exactly
15:43:25 <noonedeadpunk> I think https://review.opendev.org/q/topic:"systemd_run_dir" is pretty critical thing to land though?
15:43:50 <noonedeadpunk> It's also great to land patches for stable branches to make bugfix releases we haven't done in a while
15:44:57 <jrosser> looks like a number of those are approved now
15:47:41 <noonedeadpunk> oh, indeed
15:47:58 <noonedeadpunk> So, should I create a branching patch now?
15:48:24 <noonedeadpunk> and we will backport things?
15:48:40 <noonedeadpunk> or see where systemd stuff will lead us?
15:48:54 <jrosser> looks like there are lots of patches in flight
15:48:57 <noonedeadpunk> eventually zun and manila seems broken
15:49:14 <jrosser> maybe if we can push to merge those in the next 24hours, +/- rabbitmq repo
15:49:14 <noonedeadpunk> well, except https://review.opendev.org/q/topic:%22config_template_collection%22+(status:open%20OR%20status:merged) not that much
15:49:40 <noonedeadpunk> and what's with rabbitmq?
15:49:54 <jrosser> you said the apt repo was broken again
15:50:02 <noonedeadpunk> ah, that
15:50:08 <noonedeadpunk> I thought smth with role)
15:50:57 <jrosser> i think we also need a patch to the tests repo for the config-template collection
15:51:17 <jrosser> still a few functional jobs hanging around
15:51:28 <andrewbonney> zun should be generally better once these patches get through, but bullseye keeps having kernel panics. I can replicate but can't diagnose
15:51:28 <noonedeadpunk> true, I totally missed that
15:51:47 <noonedeadpunk> will land it
15:52:08 <noonedeadpunk> or at least push
15:54:45 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-tests master: Add config_template to collections  https://review.opendev.org/c/openstack/openstack-ansible-tests/+/819897
15:55:46 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_glance master: Use config_template as a collection  https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/819854
15:56:10 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone master: Use config_template as a collection  https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/819860
16:05:00 <noonedeadpunk> #endmeeting