| opendevreview | yatin proposed x/whitebox-neutron-tempest-plugin master: Remove duplicated test_ping_global_ip_from_vm_with_fip https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/949436 | 03:55 |
|---|---|---|
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: WIP == [eventlet-removal] Change the state change WSGI server https://review.opendev.org/c/openstack/neutron/+/938404 | 05:37 |
| opendevreview | yatin proposed x/whitebox-neutron-tempest-plugin master: Remove duplicated test_ping_global_ip_from_vm_with_fip https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/949436 | 06:30 |
| ralonsoh | ykarel, hello! | 07:03 |
| ralonsoh | I'm not sure how to continue with the Jammy image | 07:03 |
| ralonsoh | I've manually tested in my env | 07:03 |
| ralonsoh | creating the VMs and also executing the test | 07:03 |
| ralonsoh | {0} neutron_tempest_plugin.scenario.test_multicast.MulticastTestIPv4.test_multicast_between_vms_on_same_network [152.368879s] ... ok | 07:03 |
| ykarel | ralonsoh, ok then let's try it in CI node only, pushing test patch to hold it for some hours | 07:25 |
| opendevreview | yatin proposed openstack/neutron master: [DNM] Check ubuntu jammy guest vm multicast issue https://review.opendev.org/c/openstack/neutron/+/949445 | 07:34 |
| ykarel | let's see in ^, once reproduced will ping you | 07:36 |
| ralonsoh | ykarel, thanks! | 07:37 |
| ykarel | ralonsoh, node is ready 158.69.73.168 | 08:18 |
| ralonsoh | ykarel, accessing now | 08:19 |
| ykarel | ralonsoh, created a tmux there | 08:20 |
| ralonsoh | ykarel, hmmm let me check this public key | 08:21 |
| ykarel | ralonsoh, added now all from https://github.com/ralonsoh.keys | 08:22 |
| ralonsoh | yeah... | 08:23 |
| ralonsoh | I don't know why I have 3 | 08:23 |
| ralonsoh | the only one valid is the last one... | 08:23 |
| ykarel | ack must be there from old systems | 08:23 |
| ralonsoh | ykarel, ok, let me push again this patch | 08:25 |
| ykarel | ralonsoh, not able to access this system? | 08:25 |
| ralonsoh | no, sorry | 08:26 |
| ykarel | 1 sec | 08:26 |
| opendevreview | yatin proposed openstack/neutron-tempest-plugin master: [WIP] Use infra mirrors for image prepare https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/949484 | 10:28 |
| opendevreview | yatin proposed openstack/neutron master: [DNM] Check ubuntu jammy guest vm multicast issue https://review.opendev.org/c/openstack/neutron/+/949445 | 10:40 |
| stephenfin | ralonsoh: lajoskatona: Do either of you know what workaround sean-k-mooney is referring to here? https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/2065685 | 11:06 |
| stephenfin | Context is that I'm trying to bump the Gophercloud and k8s cloud-provider-openstack jobs to Noble and am seeing a failure in the ovn_agent devstack plugin https://github.com/gophercloud/gophercloud/actions/runs/14969772718/job/42047885458?pr=3374 | 11:07 |
| ralonsoh | stephenfin, no that I'm aware, sorry | 11:08 |
| ralonsoh | let me check this code | 11:08 |
| lajoskatona | stephenfin: no idea of such an issue, to tell the truth last week I worked on noble and I had no issue with it | 11:30 |
| stephenfin | lajoskatona: was this locally or in CI? If locally, can you check if apparmor is enabled? | 11:31 |
| lajoskatona | locally | 11:31 |
| lajoskatona | stephenfin: yes, apparmor is enabled (I just booted the vm I used) | 11:33 |
| lajoskatona | ralonsoh: fyi I started to play with threading backend for oslo.service (https://review.opendev.org/c/openstack/oslo.service/+/945720 ) and dhcp agent locally | 11:34 |
| stephenfin | and you were using OVN? | 11:34 |
| lajoskatona | stephenfin: yes this was an OVN environment | 11:34 |
| stephenfin | lastly: I assume if you try 'sudo aa-complain /etc/apparmor.d/usr.sbin.libvirtd' it fails? | 11:35 |
| lajoskatona | with devstack | 11:35 |
| lajoskatona | no it passed actually: $ sudo aa-complain /etc/apparmor.d/usr.sbin.libvirtd | 11:35 |
| lajoskatona | Setting /etc/apparmor.d/usr.sbin.libvirtd to complain mode. | 11:35 |
| lajoskatona | $ echo $? | 11:35 |
| lajoskatona | 0 | 11:35 |
| stephenfin | oh, weird. I see if fail on an oldish noble VM I have | 11:36 |
| * stephenfin tries updating | 11:38 | |
| stephenfin | no dice. very odd :/ | 12:03 |
| opendevreview | Merged x/whitebox-neutron-tempest-plugin master: Remove duplicated test_ping_global_ip_from_vm_with_fip https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/949436 | 12:22 |
| sean-k-mooney | lajoskatona: as noted in the bug this is cause by the passt and pasta package which are install if you install podman | 12:42 |
| sean-k-mooney | the apparmor rules for thos are broken in 24.04 and it will break disabling apparmor on libvirt as a result | 12:42 |
| lajoskatona | sean-k-mooney: ack, thanks, so not affecting devstack based deployments | 12:42 |
| sean-k-mooney | its is | 12:43 |
| sean-k-mooney | but it depends on what plugins you use | 12:43 |
| sean-k-mooney | if you enable podman or maybe ceph it could | 12:43 |
| sean-k-mooney | normaly we dont pull in the broken package in the job | 12:43 |
| sean-k-mooney | so it normally does not affect us. its fixed in later ubuntu but they have not backported the fix to noble | 12:44 |
| lajoskatona | sean-k-mooney: thanks, so it is an accident that I live without it :-) | 12:45 |
| sean-k-mooney | i debated updatign devstack to auto fix this but maybe haleyb|out coudl help with geting the fix backported | 12:45 |
| sean-k-mooney | lajoskatona: yep. i didnt hit this until i started workign on wachter and needed promethous which is installed as a contianer | 12:45 |
| sean-k-mooney | the manual fix is documented in comment 7 https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/2065685/comments/7 | 12:46 |
| sean-k-mooney | you have ot update 3 rules and hten that vm will continue to work until you reinstall the OS | 12:46 |
| sean-k-mooney | so its one of those things you fix onece and hten forgetabout it until you setup another vm | 12:47 |
| sean-k-mooney | lajoskatona: the code that triggers teh faildure is this https://github.com/openstack/devstack/blob/ddcd052b1e4ed707f431de56cfa81ac24328bbaa/lib/neutron_plugins/ovn_agent#L303-L313 which realy shoudl nto be in the ovn_agnet file at all | 12:56 |
| sean-k-mooney | a better way to fix it might be to 1 move that function to teh correctly place and two make it fix the know broken rule before putting libvirt in complein mode | 12:57 |
| *** iurygregory_ is now known as iurygregory | 13:08 | |
| lajoskatona | sean-k-mooney: as I see from the git log it was copy-pasted from networking-ovn or neutron devstack lib, so I suppose can be a discussion to find better place for them | 13:31 |
| sean-k-mooney | well the concern is the way we deploy libvirt shouoll not change between usign ml2/ovs and ml2/ovn | 13:31 |
| sean-k-mooney | i think that was added because ovn/ovs was being built form souce at one point | 13:32 |
| sean-k-mooney | im not sure its required anymroe but if it is it whould be done constitently regardless fo the neutorn backend | 13:32 |
| opendevreview | Pierre Riteau proposed openstack/ovn-octavia-provider master: Respect passed arguments for Neutron client connection https://review.opendev.org/c/openstack/ovn-octavia-provider/+/925893 | 13:36 |
| opendevreview | Lajos Katona proposed openstack/networking-bagpipe master: DNM: test master https://review.opendev.org/c/openstack/networking-bagpipe/+/949506 | 13:54 |
| ykarel | #startmeeting neutron_ci | 14:02 |
| opendevmeet | Meeting started Mon May 12 14:02:04 2025 UTC and is due to finish in 60 minutes. The chair is ykarel. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:02 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:02 |
| opendevmeet | The meeting name has been set to 'neutron_ci' | 14:02 |
| ykarel | Ping list: bcafarel, lajoskatona, slawek, mlavalle, mtomaska, ralonsoh, ykarel, jlibosva, elvira | 14:02 |
| ykarel | This will be video meeting this time: https://meetpad.opendev.org/neutron-ci-meetings | 14:02 |
| ralonsoh | hello | 14:02 |
| slaweq | o/ | 14:02 |
| lajoskatona | o/ | 14:03 |
| ykarel | slaweq, it's video today | 14:05 |
| slaweq | ahh, sorry | 14:06 |
| slaweq | joining | 14:06 |
| ykarel | #topic Actions from previous meetings | 14:06 |
| ykarel | lajoskatona to check bagpipe tempest failure | 14:06 |
| opendevreview | Merged x/whitebox-neutron-tempest-plugin master: Add WSGI check for devstack https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/948953 | 14:07 |
| ykarel | ykarel to check and report bug for functional timeouts | 14:08 |
| ykarel | #action ykarel to check https://bugs.launchpad.net/neutron/+bug/2110004 | 14:11 |
| ykarel | #topic Stable branches | 14:11 |
| ykarel | all good mostly | 14:11 |
| ykarel | #topic Stadium projects | 14:11 |
| ykarel | all green | 14:11 |
| ykarel | #topic Rechecks | 14:11 |
| ykarel | 1/13 bare rechecks | 14:12 |
| ykarel | #topic fullstack/functional | 14:12 |
| ykarel | random timeout in functional https://bugs.launchpad.net/neutron/+bug/2110004 | 14:12 |
| ykarel | #topic Tempest/Scenario | 14:12 |
| ykarel | focal image customize failures https://bugs.launchpad.net/neutron/+bug/2110191 | 14:13 |
| ykarel | mirror issue looks resolved since 8th May, but we still move forward with switch to jammy image as focal End of Standard Support is May 2025 | 14:13 |
| ykarel | #topic Periodic | 14:14 |
| ykarel | 9-stream fips jobs still broken | 14:14 |
| ykarel | waiting on few more devstack patches to merge https://review.opendev.org/q/topic:%22bug/2109591%22+status:open | 14:14 |
| ykarel | focal jobs failing with sudo -H -E python3.8 /opt/stack/devstack/files/get-pip.py, basically in wallaby, xena | 14:15 |
| ykarel | https://9dfd50a9331e5f6b74ea-fdc6c3614653bf0b139fa050a159caa6.ssl.cf1.rackcdn.com/openstack/044415052eff464e8e02cdcf1139ef26/job-output.txt | 14:15 |
| ykarel | there are some fixes for this already https://review.opendev.org/q/topic:%22get-pip-py38%22 | 14:15 |
| ykarel | devstack-tobiko-neutron broken on unmaintained/zed and 2023.1 | 14:15 |
| ykarel | - https://zuul.openstack.org/builds?job_name=devstack-tobiko-neutron&project=openstack%2Fneutron&branch=unmaintained%2Fzed&branch=unmaintained%2F2023.1&skip=0 | 14:15 |
| ykarel | #action ralonsoh to send patch to drop devstack-tobiko-neutron from zed and 2023.1 | 14:17 |
| ykarel | #topic Grafana | 14:18 |
| ykarel | https://grafana.opendev.org/d/f913631585/neutron-failure-rate | 14:18 |
| ykarel | #topic On Demand | 14:20 |
| ykarel | #endmeeting | 14:20 |
| opendevmeet | Meeting ended Mon May 12 14:20:49 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:20 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/neutron_ci/2025/neutron_ci.2025-05-12-14.02.html | 14:20 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/neutron_ci/2025/neutron_ci.2025-05-12-14.02.txt | 14:20 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/neutron_ci/2025/neutron_ci.2025-05-12-14.02.log.html | 14:20 |
| stephenfin | sean-k-mooney: lajoskatona: I'm hoping it shouldn't be needed anymore. Let's see what CI says https://review.opendev.org/c/openstack/devstack/+/949510 | 14:21 |
| stephenfin | From https://bugs.launchpad.net/networking-ovn/+bug/1466631, it seems it'll be pretty obvious if it is needed since the VM won't be able to be made active | 14:21 |
| sean-k-mooney | stephenfin: i think "but im not sure" that it shouldnt but ya we will see | 14:21 |
| stephenfin | YOLO | 14:22 |
| ykarel | stephenfin, likely in CI that part is not tested? | 14:22 |
| ykarel | because in some logs i noticed aa-status command do not exist, so that part is skipped? | 14:22 |
| sean-k-mooney | well we eont use libvirt to add the port anymore | 14:22 |
| sean-k-mooney | os-vif does it so its very likely that it wont be | 14:22 |
| stephenfin | then...we won't care? | 14:22 |
| ykarel | but that command can exists in some envs like it's sean in your failure logs :) | 14:23 |
| sean-k-mooney | well my point is i thikn just deleting it shoudl be good but it might fail if you compile ovs form souce but we will see | 14:23 |
| sean-k-mooney | it might be worth addign a DNM to neutron that depend on the devstaCK CHANGE | 14:23 |
| ykarel | yes in some jobs we do compile, not sure if in any devstack job we do | 14:23 |
| sean-k-mooney | to run there jobs | 14:23 |
| stephenfin | sean-k-mooney: can do | 14:23 |
| ykarel | ++ | 14:23 |
| opendevreview | Stephen Finucane proposed openstack/neutron master: DNM: Test removal of libvirt OVN workaround https://review.opendev.org/c/openstack/neutron/+/949511 | 14:30 |
| stephenfin | to be seen if zuul handles empty commits or not... | 14:30 |
| opendevreview | Sami Yessou proposed openstack/ovn-octavia-provider master: Passing coverage and pep tests https://review.opendev.org/c/openstack/ovn-octavia-provider/+/949516 | 15:15 |
| opendevreview | Sami Yessou proposed openstack/ovn-octavia-provider master: Patching Octavia LB pool removal issue and adding updated unit tests https://review.opendev.org/c/openstack/ovn-octavia-provider/+/949227 | 15:20 |
| opendevreview | Pierre Riteau proposed openstack/ovn-octavia-provider master: Respect passed arguments for Neutron client connection https://review.opendev.org/c/openstack/ovn-octavia-provider/+/925893 | 15:27 |
| priteau | Hello Neutron developers. We discovered a bug in the OVN provider after upgrading to Caracal (I think it affects Bobcat too), I found an existing patch and updated it. Could you please review? https://review.opendev.org/c/openstack/ovn-octavia-provider/+/925893 | 15:29 |
| opendevreview | Sami Yessou proposed openstack/ovn-octavia-provider master: Patching Octavia LB pool removal issue and adding updated unit tests https://review.opendev.org/c/openstack/ovn-octavia-provider/+/949520 | 15:37 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron-tempest-plugin master: WIP -- Bump advance image to Ubuntu Noble 24.04 https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/949522 | 15:58 |
| stephenfin | Looks like CI is happy with removing that workaround. ralonsoh lajoskatona sean-k-mooney Could I get a +1 on https://review.opendev.org/c/openstack/devstack/+/949510 when you've a chance? | 17:10 |
| stephenfin | The DNM patch is here https://review.opendev.org/c/openstack/neutron/+/949511 I'll abandon it tomorrow assuming it's served its purpose | 17:10 |
| sean-k-mooney | done with a comment | 17:15 |
| opendevreview | Merged openstack/neutron-tempest-plugin master: Remove Bobcat 2023.2 jobs https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/948850 | 17:26 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!