15:00:28 <slaweq> #startmeeting neutron_ci 15:00:28 <opendevmeet> Meeting started Tue Feb 21 15:00:28 2023 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:28 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:28 <opendevmeet> The meeting name has been set to 'neutron_ci' 15:00:30 <slaweq> bcafarel, lajoskatona, mlavalle, mtomaska, ralonsoh, ykarel, jlibosva is starting now 15:00:31 <slaweq> This will be video meeting this time: https://meetpad.opendev.org/neutron-ci-meetings 15:02:11 <lajoskatona> o/ 15:02:53 <slaweq> #link https://grafana.opendev.org/d/f913631585/neutron-failure-rate?orgId=1 15:02:58 <slaweq> #topic Actions from previous meetings 15:03:03 <slaweq> lajoskatona to continue checking dvr functional tests issues 15:03:50 <lajoskatona> https://review.opendev.org/c/openstack/neutron/+/873111 15:04:05 <lajoskatona> https://zuul.opendev.org/t/openstack/status#873111 15:05:57 <slaweq> https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_594/874167/3/check/neutron-fullstack-with-uwsgi/5943274/testr_results.html 15:07:04 <slaweq> #action lajoskatona to continue checking dvr functional tests issues 15:07:14 <slaweq> ralonsoh to try to store journal log in UT job's results to debug "no such table" issues 15:08:42 <slaweq> slaweq to report bug about failed to bind LRP in functional tests 15:08:48 <slaweq> https://bugs.launchpad.net/neutron/+bug/2007353 15:09:07 <slaweq> lajoskatona to talk with zhouhenglc about fwaas jobs issues 15:10:02 <slaweq> slaweq to report bug with failed ping in grenade jobs 15:10:08 <slaweq> Bug: https://bugs.launchpad.net/neutron/+bug/2007357 15:10:59 <lajoskatona> https://review.opendev.org/c/openstack/grenade/+/874417 15:11:45 <ozzzo_work> I'm using the unified python API (cloud.network.security_groups) to pull security groups, but in large clusters it times out: The server didn't respond in time.: 504 Gateway Time-out abraden@osjump2.shared-bo.brn1:~/cloud-support/cloud-scripts [ent-ansible-2.9.20:qde3:admin]$ 15:12:00 <ozzzo_work> How can I extend the timeout? 15:12:06 <slaweq> ykarel to fix grenade random fails pcp installation 15:12:14 <ozzzo_work> I can pull the list via CLI no problem;; apparently CLI uses a longer timeout 15:12:17 <slaweq> #topic Stadium projects 15:13:59 <slaweq> #topic Grafana 15:15:23 <slaweq> #topic Rechecks 15:15:32 <slaweq> +---------+----------+... (full message at <https://matrix.org/_matrix/media/v3/download/matrix.org/zGdkCHJPJcPaKOFRsHSnkNhR>) 15:16:00 <slaweq> https://review.opendev.org/c/openstack/neutron/+/873247 15:18:10 <slaweq> #topic Unit tests 15:18:24 <slaweq> https://bugs.launchpad.net/neutron/+bug/2007254 15:18:51 <slaweq> #topic fullstack/functional 15:19:09 <slaweq> https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b8d/periodic/opendev.org/openstack/neutron/master/neutron-functional-with-oslo-master/b8d8c53/testr_results.html 15:20:02 <ozzzo_work> Should I try a different channel? Where's the best place to ask about the API? 15:20:29 <slaweq> https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_978/periodic/opendev.org/openstack/neutron/master/neutron-functional/9789c89/testr_results.html 15:20:36 <ralonsoh> ozzzo_work, we are in a meeting now 15:21:21 <ozzzo_work> oic ok 15:21:39 <slaweq> https://64e9807acd80d4cab1ab-851182d93d98b3728f798963c90c7371.ssl.cf5.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-functional-with-uwsgi-fips/1914083/testr_results.html 15:22:11 <slaweq> #action slaweq to check https://64e9807acd80d4cab1ab-851182d93d98b3728f798963c90c7371.ssl.cf5.rackcdn.com/periodic/opendev.org/openstack/neutron/master/neutron-functional-with-uwsgi-fips/1914083/testr_results.html 15:22:40 <slaweq> https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_138/874342/2/check/neutron-ovn-tempest-ipv6-only-ovs-release/138097d/testr_results.html 15:23:02 <slaweq> #topic Tempest/Scenario 15:23:12 <slaweq> https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_138/874342/2/check/neutron-ovn-tempest-ipv6-only-ovs-release/138097d/testr_results.html 15:23:47 <slaweq> https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_992/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-slow/992d108/testr_results.html 15:23:54 <slaweq> https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_64e/periodic/opendev.org/openstack/neutron/master/neutron-ovn-tempest-slow/64eb479/testr_results.html 15:25:18 <slaweq> #action slaweq to report bug about macspoofing_port test in ovn-tempest-slow job 15:25:24 <slaweq> #topic Periodic 15:25:39 <slaweq> neutron-functional-with-sqlalchemy-master 15:27:52 <slaweq> #action ralonsoh to check neutron-functional-with-sqlalchemy-master failures 15:28:09 <slaweq> #topic On Demand 15:28:16 <slaweq> Regarding to the discussion in https://review.opendev.org/c/openstack/neutron/+/869741 I proposed today https://review.opendev.org/c/openstack/neutron/+/874536 - please tell me what do You think about it 15:30:04 <ralonsoh> one quick topic: #link https://bugs.launchpad.net/neutron/+bug/2007986 15:30:12 <ralonsoh> https://review.opendev.org/c/openstack/tempest/+/873055 15:33:11 <slaweq> https://github.com/openstack/neutron-tempest-plugin/blob/master/zuul.d/wallaby_jobs.yaml 15:35:12 <slaweq> https://github.com/openstack/neutron-tempest-plugin/blob/1.8.0/zuul.d/rocky_jobs.yaml#L120 15:36:04 <ykarel> https://review.opendev.org/c/openstack/devstack/+/871782 15:36:29 <ykarel> https://zuul.openstack.org/builds?job_name=tempest-full-py3&branch=stable%2Fwallaby&skip=0 15:36:45 <opendevreview> Lajos Katona proposed openstack/python-neutronclient master: OSC: Remove BGP calls to neutronclient https://review.opendev.org/c/openstack/python-neutronclient/+/868321 15:41:53 <opendevreview> Fernando Royo proposed openstack/ovn-octavia-provider master: Member provisioning_status comes back to NO_MONITOR after HM delete https://review.opendev.org/c/openstack/ovn-octavia-provider/+/874609 16:01:50 <opendevreview> Fernando Royo proposed openstack/ovn-octavia-provider stable/zed: Reduce coverage threshold on stable branches https://review.opendev.org/c/openstack/ovn-octavia-provider/+/874426 16:11:02 <opendevreview> Rodolfo Alonso proposed openstack/neutron master: Format correctly (dialect=mac_unix_expanded) the MAC addresses https://review.opendev.org/c/openstack/neutron/+/874654 16:14:59 <opendevreview> Fernando Royo proposed openstack/ovn-octavia-provider master: Reset member provisioning status to NO_MONITOR when a HM is deleted https://review.opendev.org/c/openstack/ovn-octavia-provider/+/874609 16:24:50 <opendevreview> Rodolfo Alonso proposed openstack/neutron master: Check port.tag is not DEAD_VLAN_TAG in ``DHCPAgentOVSTestFramework`` https://review.opendev.org/c/openstack/neutron/+/874658 17:16:01 <opendevreview> Maurice Escher proposed openstack/neutron master: ml2 plugin: use const from neutron-lib https://review.opendev.org/c/openstack/neutron/+/874631 17:16:25 <opendevreview> Rodolfo Alonso proposed openstack/neutron master: DNM WIP remove FT OVN workaround for sqlite https://review.opendev.org/c/openstack/neutron/+/874669 17:40:22 <ozzzo_work> Is the meeting over now? Does anyone have any ideas on my API question? 17:40:53 <ozzzo_work> I'm using the unified python API (cloud.network.security_groups) to pull security groups, but in large clusters it times out: The server didn't respond in time.: 504 Gateway Time-out 17:46:49 <ralonsoh> ozzzo_work, what is the unified python API? 17:46:59 <ralonsoh> do you mean you are using the SDK bindings 17:47:12 <ralonsoh> how are you calling this method? 17:54:10 <ozzzo_work> yes sdk 17:55:33 <ralonsoh> and what client is using it? 17:56:29 <ozzzo_work> https://paste.openstack.org/show/bVPEmD2n75LhMfqBaf6e/ 17:58:50 <ozzzo_work> I think I need to increase the timeout value 18:02:40 <ozzzo_work> Looking here: https://docs.openstack.org/openstacksdk/latest/user/proxies/network.html 18:03:05 <ozzzo_work> But it doesn't mention the timeout value. 18:08:16 <ralonsoh> ozzzo_work, in the "register_argparse_arguments", when creating the connect object 18:08:24 <ralonsoh> there is a mention to --timeout 18:08:58 <ralonsoh> what I don't know is how to build this "options" object (that seems to be an ArgumentParser 18:10:54 <ozzzo_work> where do you see --timeout? 18:13:12 <ralonsoh> https://github.com/openstack/openstacksdk/blob/master/openstack/config/loader.py#L755 18:13:17 <ralonsoh> do this 18:13:31 <ralonsoh> https://paste.opendev.org/show/b3E6bv7jQEaRQzSwTePW/ 18:13:44 <ralonsoh> and when calling this script, pass an input argument 18:13:54 <ralonsoh> ./script.py --timeout=1000 18:14:18 <ralonsoh> if you check "options" in https://github.com/openstack/openstacksdk/blob/master/openstack/config/loader.py#L770 18:14:31 <ralonsoh> you'll see that the input argument has been read 18:17:17 <ozzzo_work> ok I'll try that, ty! 18:24:00 <gmann> ralonsoh: RE on https://review.opendev.org/c/openstack/tempest/+/764226/2/zuul.d/base.yaml#22 18:24:40 <gmann> ralonsoh: basically it count the minimum compute node and set in tempest conf so that we can decide on mulitnode test to be skipped or run 18:25:12 <gmann> example https://github.com/openstack/tempest/blob/1569290be06e61d63061ae35a997aff0ebad68f1/tempest/api/compute/admin/test_live_migration.py#L47 18:28:46 <gmann> ralonsoh: basically it count the number of compute defined in jobs, for example in base job https://github.com/openstack/devstack/blob/e5c8e2951f8eed2d618bcb7c1d99adddeca4fffe/.zuul.yaml#L128 18:31:35 <gmann> ralonsoh: slaweq lajoskatona frickler on new RBAC fixes backport to zed, as background yes mnaser was testing the new defaults on zed and to make new RBAC work we need to backport those fixes. I think we said ok to backport at the time fixing those on master but did not find ref. 18:33:56 <gmann> basically we had new RBAC released in zed and they were disable by default but anyone can enable them and use it. for that I think we should fix it. basically migration plan will be: 18:33:56 <gmann> - operator upgrading from Yoga to Zed get the new RBAC option to enable but as it is disabled by default in zed they can try and fix the things during this cycle. 18:33:56 <gmann> - in operator upgrading from zed to 2023.1, get those new RBAC as default so no surprise to them. 18:34:38 <gmann> otherwise operator will get chance to use/try new RBAC only in 2023.1 and it is enabled by default there so it does not give them a cycle time for something new coming as default 18:35:04 <gmann> I like to slaweq idea of backporting only new RBAC rule and do not remove the old policy rule in zed backport 18:37:49 <gmann> ralonsoh: slaweq: lajoskatona: if we are not making neutron new RBAC working in zed then I will say we should not enable it by default in 2023.1 instead enable by default in 2023.2. BUT to ship nova+neutron together with new RBAC and make it usable at operator side, we should backport and make neutron new rbac work on zed 18:41:51 <gmann> ralonsoh: slaweq: lajoskatona: I just realized we should keep old rule on master also because old rules are still supported (disabled by default) unless we remove them https://review.opendev.org/c/openstack/neutron/+/865032/1/neutron/conf/policies/network.py#b204 18:50:50 <lajoskatona> gmann: thanks for background 18:51:23 <lajoskatona> gmann, ralonsoh, slaweq: should we discuss this perhaps on Friday during the drivers meeting? 21:20:21 <opendevreview> Slawek Kaplonski proposed openstack/neutron master: Change neutron-ovs-tempest-dvr-ha-multinode-full job's config https://review.opendev.org/c/openstack/neutron/+/874536 21:35:58 <opendevreview> Slawek Kaplonski proposed openstack/neutron master: [S-RBAC] Add release note about full support for new policies https://review.opendev.org/c/openstack/neutron/+/874706 21:42:16 <opendevreview> Merged openstack/neutron stable/victoria: Improve scheduling L3/DHCP agents, missing lower binding indexes https://review.opendev.org/c/openstack/neutron/+/873628 21:44:49 <opendevreview> Slawek Kaplonski proposed openstack/neutron-tempest-plugin master: [Secure RBAC] Add scope enforcement enabled job for Zed branch https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/874709 02:10:19 <opendevreview> Merged openstack/neutron-tempest-plugin master: [Secure RBAC] Add scope enforcement enabled job for master branch https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/867518 07:30:13 <ralonsoh> lajoskatona, gmann I don't think this friday is a good day, we all have PTO in Red Hat 07:30:22 <ralonsoh> but let me check that first 07:30:59 <ralonsoh> in any case, I've talked to slaweq and he is in favor of backporting all these patches 07:31:09 <ralonsoh> including the router:external one 07:32:15 <ralonsoh> if the plan is to support sRBAC in Zed to make the transition to A with full support, I'm in favor of this 07:59:22 <lajoskatona> ralonsoh: ack, anyway, let's discuss this to have a common understanding, I think next week is good also 08:00:45 <lajoskatona> ralonsoh: I am fine with it, my concern is only that yesterday we agreed the other way and perhaps there are more background which we forgot yesterday and considering those the team will agree to do the backport 08:13:31 <ralonsoh> lajoskatona, sure, let's discuss first during the next team meeting 08:19:06 <slaweq> Hi, yesterday I proposed to have this "new-policies" job also for stable/zed 08:19:28 <slaweq> I will update the router:external patch today to not remove old rule but maybe just change the new one somehow 08:20:07 <slaweq> but still I think that if there is no rule at all, it will work as "RULE_ANY" so default behaviour of this get network:router:external shouldn't change IMO 08:20:59 <ralonsoh> I think we can keep the patch as is now 08:21:32 <ralonsoh> about the RBAC backport, we'll have a new discussing next tuesday 08:21:53 <slaweq> ++ 08:24:05 <opendevreview> Slawek Kaplonski proposed openstack/neutron-tempest-plugin master: [Secure RBAC] Add scope enforcement enabled job for Zed branch https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/874709 08:24:40 <opendevreview> Slawek Kaplonski proposed openstack/neutron stable/zed: Remove policy rule for get_network:router:external https://review.opendev.org/c/openstack/neutron/+/874398 08:25:30 <opendevreview> Slawek Kaplonski proposed openstack/neutron master: Set DVR qr-xyz interfaces DOWN on backup node https://review.opendev.org/c/openstack/neutron/+/869741 08:26:21 <opendevreview> Slawek Kaplonski proposed openstack/neutron master: Set DVR qr-xyz interfaces DOWN on backup node https://review.opendev.org/c/openstack/neutron/+/869741 08:26:31 <opendevreview> Slawek Kaplonski proposed openstack/neutron master: Set DVR qr-xyz interfaces DOWN on backup node https://review.opendev.org/c/openstack/neutron/+/869741 08:26:51 <slaweq> ralonsoh lajoskatona patch https://review.opendev.org/c/openstack/neutron/+/874536 seems to be fine now and I just proposed https://review.opendev.org/c/openstack/neutron/+/869741 on top of this mine patch to check if this dvr job will now be fine really 08:27:59 <ralonsoh> cool, let me check your patch 08:28:05 <slaweq> thx 08:38:37 <lajoskatona> +1 08:48:19 <slaweq> #endmeeting