15:00:17 <noonedeadpunk> #startmeeting openstack_ansible_meeting 15:00:17 <opendevmeet> Meeting started Tue May 14 15:00:17 2024 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:17 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:17 <opendevmeet> The meeting name has been set to 'openstack_ansible_meeting' 15:00:21 <noonedeadpunk> #topic office hours 15:00:41 <noonedeadpunk> well, it should have been rollcall... but whatever :D 15:01:34 <damiandabrowski> hi! 15:02:14 <noonedeadpunk> huge chunk of https://review.opendev.org/q/topic:%22osa/messaging_improvements%22 has been merged 15:02:26 <noonedeadpunk> thanks jrosser_ and andrewbonney for taking time on reviewing that! 15:02:45 <jrosser_> o/ hello 15:02:50 <jrosser_> no worries 15:02:54 <noonedeadpunk> some roles are broken, so I was planning to look deeper during the week about reasons 15:02:58 <jrosser_> there are some awkward bits left i think 15:03:23 <jrosser_> but tbh this is not terrible, becasue these patches for messaging give us a good health check before we release 15:03:34 <noonedeadpunk> yeah 15:03:46 <noonedeadpunk> I was expecting more worse situation kinda 15:04:27 <noonedeadpunk> I actually tried to add tempest tests for trove, but it expects for datastores to be created outside of tempest runtime 15:04:41 <noonedeadpunk> which is non-trivial due to missing ansible modules for that 15:06:03 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Add information about hook playbooks to the 'extending-osa' docs https://review.opendev.org/c/openstack/openstack-ansible/+/919555 15:06:35 <noonedeadpunk> we also have quite some failures for upgrade jobs, which are mainly intermittent, but highly annoying 15:07:16 <jrosser_> seems to be some amount of tls / rocky / upgrade failures 15:07:30 <jrosser_> i did look in some logs and it was pretty hard to find anything specific 15:07:57 <noonedeadpunk> yeah, also haven't found anything too obvious 15:08:10 <noonedeadpunk> rather then potentially some nodes just being slow or smth... 15:08:13 <NeilHanlon> o/ hiya 15:08:46 <NeilHanlon> i am going to check on repos for rocky today; make sure there aren't bad entries still being served. i.e, mirrors which are not serving 9.4 15:09:15 <jrosser_> tbh i have not seen repo specific errors for ~ a week at least 15:09:30 <noonedeadpunk> actually ,another part we've recently realized being annoying - is that volume types are created regardless if operator want to manage them through osa 15:09:33 <jrosser_> just somehow our rocky CI feels less stable than other things 15:09:42 <noonedeadpunk> or well, naming of default one is really opinionated 15:09:54 <NeilHanlon> jrosser_: ack; well, i'll still do that, but will check into CI generally and see if I can find anything common 15:10:19 <noonedeadpunk> to be completely frank - I was spotting issues with Ubuntu as well 15:10:46 <jrosser_> yep, i also saw what looked like sqla troubles too 15:12:59 <noonedeadpunk> out of capi topic - one patch left: https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/916647 15:13:28 <noonedeadpunk> and actually - magnum feels tough on passing ci lately :( 15:13:38 <jrosser_> yes, that is not super critical, but it's needed to make a bunch of jobs to test all the k8s versions 15:13:41 <jrosser_> yes it does 15:13:44 <noonedeadpunk> ie - there're some capi intermittent issues 15:13:53 <jrosser_> and i think there is brokenness in vexxhost nodepool too 15:13:55 <noonedeadpunk> on top of upgrade failures 15:14:00 <jrosser_> ^ mnaser 15:14:16 <jrosser_> yeah i agree that the capi job has failed for unspecific reasons 15:14:45 <jrosser_> though surprisingly we already collect a ton of logs from the control plane which should make it possible to see whats happening 15:15:21 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum master: Move insecure param to keystone_auth section https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/905110 15:16:32 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Implement support for octavia-ovn-provider driver https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/868462 15:16:53 <noonedeadpunk> octavia-ovn-provider - I still failed to work closer on testing this path :( 15:17:14 <jrosser_> hmm yes another NODE_FAILURE - thats because we need the 32G flavor for capi tests 15:17:23 <jrosser_> and something is broken with those 15:18:08 <noonedeadpunk> or well, there's probably something validly broken with ovn driver in octavia in our aio 15:19:00 <noonedeadpunk> or well... a different set of tempest tests should run for ovn: https://96de7bed307fb2a6a065-7f155e7c59383dfa4a196ee8803910a9.ssl.cf1.rackcdn.com/868462/16/check/openstack-ansible-deploy-aio_lxc_ovnprovider-ubuntu-jammy/de203dd/logs/openstack/aio1-utility-container-3ffa4338/utility/stestr_results.html 15:19:13 <noonedeadpunk> as it's getting just `Got NotImplemented error` 15:19:22 <noonedeadpunk> which is fair.... 15:19:43 <noonedeadpunk> will try to talk to octavia folks to see how we can test that 15:20:01 <noonedeadpunk> and if they do have any tempest for that at all.. 15:20:03 <jrosser_> maybe thats just not valid to let octavia tempest go against the ovn provider 15:20:24 <noonedeadpunk> yeah, but then how to test it... 15:20:34 <jrosser_> there might be some clue in how neutron set up to test that 15:20:57 <jrosser_> (i assume this happens.....) 15:21:19 <jrosser_> perhaps some already existing skip list we can use for example 15:26:30 <mgariepy> isn't i only that the flavor in octavia is not needed for ovn ? 15:26:31 <mgariepy> https://docs.openstack.org/ovn-octavia-provider/latest/admin/driver.html 15:26:47 <mgariepy> * knows nothing about octavia tho. haha 15:27:13 <noonedeadpunk> well, ovn also is only l4 balancing... and only source_ip_port algo iirc 15:28:05 <mgariepy> Details: b'{"faultcode": "Server", "faultstring": "Provider \'ovn\' does not support a requested action: This provider does not support validating flavors.", "debuginfo": null}' 15:28:21 <noonedeadpunk> yeah, which is fair... 15:28:52 <noonedeadpunk> ok, will check on that a bit later 15:29:14 * noonedeadpunk looking through https://etherpad.opendev.org/p/osa-dalmatian-ptg 15:29:36 <noonedeadpunk> ovn-bgp-agent merged... 15:29:48 <noonedeadpunk> inactive projects are removed... 15:29:54 <noonedeadpunk> EOM branches... 15:30:00 <noonedeadpunk> EOM branches are pita... 15:30:20 <noonedeadpunk> We outdue to create Zed EOM 15:30:27 <noonedeadpunk> *overdue 15:30:43 <jrosser_> we also need to choose if we work in the unmaintained branches at all 15:30:49 <noonedeadpunk> yeah 15:31:19 <jrosser_> victoria is nearly OK, and i was planning to work back toward the maintained ones 15:31:24 <noonedeadpunk> I guess this depends... but for work on them, they should be revived first and I failed to follow on that 15:31:26 <johnsom> jrosser_ The Octavia tempest plugin will run against the OVN provider, however most of the tests will "skip" as the OVN provider doesn't support many of the features of Octavia. There are gate jobs that run with it. 15:31:28 <jrosser_> but just not managed to make progress 15:31:40 <opendevreview> Merged openstack/openstack-ansible-os_aodh master: Add variable to globally control notifications enablement https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/917948 15:32:08 <noonedeadpunk> johnsom: well, we kinda try to run just octavia_tempest_plugin.tests.scenario.v2.test_load_balancer 15:32:20 <jrosser_> and this happens https://96de7bed307fb2a6a065-7f155e7c59383dfa4a196ee8803910a9.ssl.cf1.rackcdn.com/868462/16/check/openstack-ansible-deploy-aio_lxc_ovnprovider-ubuntu-jammy/de203dd/logs/openstack/aio1-utility-container-3ffa4338/utility/stestr_results.html 15:32:22 <noonedeadpunk> which fails... but I guess I need to check what you're running in gates 15:33:10 <noonedeadpunk> also, looking at test list... I wonder if we kinda want to always run `tempest.scenario.test_server_basic_ops.TestServerBasicOps`? 15:33:28 <johnsom> I know what it is, one second 15:33:37 <jrosser_> i have to go to another meeting - but andrewbonney was interested in feedback on https://review.opendev.org/q/topic:%22osa/rmq-migrate%22 15:33:44 <noonedeadpunk> oh, yes 15:33:53 <jrosser_> particularly fixing up the tags, and whats best approach 15:34:31 <johnsom> jrosser_ https://github.com/openstack/octavia-tempest-plugin/blob/master/zuul.d/jobs.yaml#L1100 15:34:37 <jrosser_> i did discuss it with him and we could not see the point in tags like 'nova' - and the behaviour of them is just very wierd right now 15:34:57 <jrosser_> johnsom: ahha! 15:35:18 <opendevreview> Merged openstack/openstack-ansible-os_trove master: Add variable to globally control notifications enablement https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/917226 15:35:19 <johnsom> We are strict with the main test jobs 15:35:28 <noonedeadpunk> ok, that explains it 15:35:57 <noonedeadpunk> I wonder if anything will run though 15:36:32 <johnsom> Mostly just a few TCP and UDP tests work with OVN 15:36:33 <opendevreview> Merged openstack/openstack-ansible-os_trove master: Implement variables to address oslo.messaging improvements https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/917997 15:38:34 <noonedeadpunk> so we have just that right now: https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/templates/user_variables_octavia.yml.j2#L14 15:38:59 <opendevreview> Merged openstack/openstack-ansible-os_magnum master: reno: Update master for unmaintained/zed https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/919171 15:39:00 <noonedeadpunk> as we pretty much want just bare minimal thing to see if it's basically operational 15:39:13 <noonedeadpunk> we don't want to take your job by re-running all tests available :D 15:39:27 <mgariepy> i guess that we need to set: not_implemented_is_error: False 15:39:44 <johnsom> Yep, understandable. Yeah, I think all you need is to set that tempest variable 15:40:40 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not fail on NotImplemented tests for OVN Octavia https://review.opendev.org/c/openstack/openstack-ansible/+/919599 15:40:46 <noonedeadpunk> let's see:) 15:40:46 <johnsom> Here is a list of scenario tests run with OVN: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_833/917076/1/check/neutron-ovn-provider-v2-scenario/83362d1/testr_results.html 15:40:48 <noonedeadpunk> thanks! 15:41:24 <noonedeadpunk> ok, so octavia_tempest_plugin.tests.scenario.v2.test_load_balancer.LoadBalancerScenarioTest is part of that, amazing 15:41:47 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Implement support for octavia-ovn-provider driver https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/868462 15:48:10 <opendevreview> Merged openstack/openstack-ansible-os_aodh master: Implement variables to address oslo.messaging improvements https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/917949 15:53:34 <noonedeadpunk> so, looking at ptg doc, we're pretty much done? 15:53:50 <noonedeadpunk> except renaming of groups.... 15:54:04 <noonedeadpunk> but I think, we will do that for the next release.... 15:54:39 <noonedeadpunk> and potentially - if we wanna replace nginx with apache for SKyline? https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/919529 15:55:26 <NeilHanlon> I'm pretty okay with it. would rather have haproxy + one other thing, not three 16:11:54 <noonedeadpunk> #endmeeting