| ralonsoh | hi folks, are you aware of any issue with multicast traffic in Ubuntu Noble? | 09:47 |
|---|---|---|
| ralonsoh | we switched from Focal to Jammy in neutron-tempest-plugin | 09:48 |
| ralonsoh | and the multicast test is failing | 09:48 |
| ralonsoh | at least in the CI | 09:48 |
| ralonsoh | I'm doing the same locally and this test is passing | 09:49 |
| frickler | ralonsoh: that doesn't ring a bell for me. do you use the same tunneled setup between nodes locally which we have in CI? we could also hold a job for you if you want to test this in the CI manually | 10:17 |
| ralonsoh | frickler, we have been playing with a node for 2 hours | 10:21 |
| ralonsoh | we can pass this test locally but not in the CI | 10:21 |
| ralonsoh | and this is happening with Jammy VMs, not Focal | 10:22 |
| ykarel | frickler, actually it's seen in single node devstack setup, so CI tunnels shouldn't be involved here | 10:41 |
| ykarel | frickler, can you do autohold request for neutron-tempest-plugin-ovn change:- https://review.opendev.org/c/openstack/neutron/+/949445 | 10:41 |
| stephenfin | clarkb: Yeah, I figured out why https://review.opendev.org/c/openstack/pbr/+/949049 wasn't work (comment left inline) but have yet to investigate a fix though | 11:13 |
| stephenfin | As for the fine finders entry point: I tested that for reno (which currently has a distutils 'build_reno' Command that we attempt to execute in pbr if found). It didn't work because we don't copy across '.git' to the tmpdir when building wheels, meaning reno can't construct history. I'll have to try another approach | 11:15 |
| frickler | ykarel: ralonsoh: so the issue only depends on the image running in the nested VM? that sounds very weird | 12:21 |
| ykarel | frickler, yes | 12:21 |
| fungi | stephenfin: as a temporary workaround, disabling build isolation would likely work | 13:07 |
| fungi | but yes, it would be good to find a better option. maybe we should be generating release notes in the dist stage instead of bdist? that's basically why pbr is able to get the details it needs for versioning and changelog/authors file generation | 13:08 |
| fungi | i don't personally object to including release notes in our sdist tarballs | 13:08 |
| *** iurygregory_ is now known as iurygregory | 13:08 | |
| ralonsoh | frickler, hi, we found that the jammy VM does not send the igmp register message | 14:38 |
| ralonsoh | thus OVN does not add it to the IGMP group | 14:38 |
| ralonsoh | I see that, by default, the multicast is enabled in the interface ens3 | 14:38 |
| clarkb | stephenfin: setuptools-scm relies on file finders and version setting entrypoints. Similar to what fungi is suggesting maybe we can use the version setting entrypoint to set the version and everything else that needs to look at git (I'm assuming we'll have access to git there since setuptools-scm must) | 14:44 |
| frickler | ralonsoh: ok, but that's for the advanced CI image that neutron is building, not for the nodepool image that infra is building, right? | 15:02 |
| ralonsoh | right | 15:02 |
| ralonsoh | frickler, is there any config option, start up value or any module I need to add? | 15:03 |
| frickler | I have no idea who would send the igmp register, the kernel itself? might be some sysctl then? otherwise possibly take a closer look at systemd(-networkd)? | 15:15 |
| ralonsoh | frickler, the IGMP is being sent by a python script | 15:17 |
| ralonsoh | tailor made for the test | 15:17 |
| ralonsoh | I have an easier one, from the internet | 15:18 |
| ralonsoh | https://paste.opendev.org/show/bugPHlMxnsKrS6gwjVCZ/ | 15:19 |
| ralonsoh | with that script, the focal image sends the igmp group register | 15:20 |
| ralonsoh | but this is not happening with the jammy one | 15:20 |
| ralonsoh | I'll try with the stock one, directly downloaded from the ubuntu website, without any modification | 15:20 |
| fungi | is the suspicion that the packet is being filtered before it reaches the listener? what do the iptables rules look like at the point where that is called? what user is sending it? could the problem be changes in kernel capabilities allowed for non-root users? | 15:21 |
| ralonsoh | fungi, I'm dumping the TAP interface | 15:22 |
| fungi | okay, so you've got a packet capture | 15:22 |
| ralonsoh | I'm manually using root | 15:22 |
| ralonsoh | fungi, no no | 15:22 |
| ralonsoh | not in Jammy, only in Focal | 15:22 |
| ralonsoh | the VM is not sending anything (at least not in the only interface) | 15:23 |
| ralonsoh | ok, with the image directly downloaded and nothing else, the igmp request is sent... | 15:27 |
| ralonsoh | we are doing something wrong with this image | 15:27 |
| ralonsoh | I'll check the devstack script we use to customize it | 15:27 |
| ralonsoh | sorry for the noise! | 15:27 |
| fungi | ralonsoh: my first guess would be difference in the included/loaded kernel modules | 15:28 |
| ralonsoh | yeah, most probably, investigating right now | 15:32 |
| ralonsoh | fungi, no, I was wrong. I was using an incorrect image | 15:52 |
| ralonsoh | even with a fresh image, the igmp is blocked | 15:52 |
| ralonsoh | I've used https://cloud-images.ubuntu.com/minimal/releases/jammy/release/ubuntu-22.04-minimal-cloudimg-amd64.img | 15:53 |
| ralonsoh | where can I open a bug for this? | 15:53 |
| fungi | odds of a regression in jammy being fixed at this point are pretty low, can you reproduce the problem in noble? | 15:54 |
| ralonsoh | fungi, when I used the noble image, the CI exploded | 15:55 |
| ralonsoh | https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/949140 | 15:55 |
| ralonsoh | PS1 results | 15:55 |
| ralonsoh | hmmm I stopped the execution before storing the results | 15:56 |
| ralonsoh | I'll push a new PS | 15:56 |
| fungi | https://bugs.launchpad.net/cloud-images seems to be the place to report bugs about ubuntu cloud images, fwiw | 15:57 |
| ralonsoh | thanks! | 15:58 |
| * fungi is a meat-based web search engine | 15:58 | |
| clarkb | ralonsoh: did you check sysctl? | 16:20 |
| clarkb | frickler suggested this earlier. YOu can probably just dump systcl -a from focal and compare to jammy and grep for multi and see what shows up | 16:20 |
| opendevreview | James E. Blair proposed openstack/project-config master: Add zuul-launcher dashboards https://review.opendev.org/c/openstack/project-config/+/949241 | 19:03 |
| opendevreview | Merged openstack/project-config master: Add zuul-launcher dashboards https://review.opendev.org/c/openstack/project-config/+/949241 | 19:57 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!