| opendevreview | Merged openstack/neutron-lib master: Use py312 for all neutron-lib jobs https://review.opendev.org/c/openstack/neutron-lib/+/931680 | 00:07 |
|---|---|---|
| opendevreview | yatin proposed openstack/neutron stable/2024.2: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931863 | 04:01 |
| opendevreview | yatin proposed openstack/neutron stable/2024.1: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931864 | 04:02 |
| opendevreview | yatin proposed openstack/neutron stable/2023.2: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931865 | 04:03 |
| opendevreview | yatin proposed openstack/neutron stable/2023.1: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931866 | 04:03 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron stable/2024.1: [OVN] Check metadata HA proxy configuration before restart https://review.opendev.org/c/openstack/neutron/+/931791 | 06:02 |
| opendevreview | yatin proposed openstack/neutron unmaintained/zed: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931871 | 06:17 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: DNM - Test "neutron-ovn-tempest-ipv6-only-ovs*" with eventlet https://review.opendev.org/c/openstack/neutron/+/931842 | 06:35 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: [Fullstack] Add active waiting for one HA agent to be active for router https://review.opendev.org/c/openstack/neutron/+/931766 | 07:17 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: [Fullstack] Add active waiting for one HA agent to be active for router https://review.opendev.org/c/openstack/neutron/+/931766 | 07:19 |
| slaweq | haleyb ralonsoh ykarel: based on our discussion during last CI meeting, I added topic about Postgresql to the TC etherpad: https://etherpad.opendev.org/p/oct2024-ptg-os-tc | 07:29 |
| slaweq | please add your notes there if you have anything more there | 07:29 |
| ykarel | thx slaweq | 07:30 |
| ralonsoh | thanks | 07:32 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron stable/2024.2: Use the declarative attribute ``standard_attr_id`` https://review.opendev.org/c/openstack/neutron/+/931877 | 07:40 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron stable/2024.1: Use the declarative attribute ``standard_attr_id`` https://review.opendev.org/c/openstack/neutron/+/931878 | 07:40 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron stable/2023.2: Use the declarative attribute ``standard_attr_id`` https://review.opendev.org/c/openstack/neutron/+/931880 | 07:42 |
| ykarel | ralonsoh, slaweq got the root cause for too many open files functional failures | 08:04 |
| ykarel | it's infra :) | 08:04 |
| ykarel | a new cloud with different node config https://review.opendev.org/c/openstack/project-config/+/928321 | 08:06 |
| ykarel | and since then only we hitting the issue | 08:06 |
| ykarel | opensearch also confirms all failures on provider raxflex-SJC3 | 08:07 |
| ykarel | so 4096 ulimit is good enough for us | 08:07 |
| ralonsoh | ykarel, but are the CI VMs configured in a different way? | 08:09 |
| ralonsoh | I mean, the ulimit is a OS config parameter | 08:09 |
| ralonsoh | why this node is affecting us? | 08:10 |
| ykarel | these new nodes are 4 cpu vs usual 8 cpus | 08:10 |
| ralonsoh | really??? pffffff | 08:10 |
| ykarel | tempest run longer and with more runs per worker fd utilization increases and when stestr worker finishes then the fds are released | 08:11 |
| ralonsoh | yeah, we are doing something wrong in the FTs, leaving some unnecessary FDs open | 08:11 |
| ralonsoh | but yes, that could be a plausible cause | 08:12 |
| ralonsoh | thanks a lot! | 08:12 |
| ykarel | with latest debug logs we have open fd lists now | 08:12 |
| ralonsoh | yes, most of them related to the ovn and ovs databases | 08:13 |
| ralonsoh | well, not most of them but too many | 08:13 |
| ralonsoh | ykarel, check that: https://review.opendev.org/c/openstack/neutron/+/931842 | 08:14 |
| ralonsoh | no errors... pfffff | 08:14 |
| ralonsoh | 2 jobs with eventlet, 1 with wsgi | 08:15 |
| slaweq | great catch ykarel | 08:50 |
| noonedeadpunk | hey ! I was just reading through the meeting logs and around for discussion: https://meetings.opendev.org/irclogs/%23openstack-neutron/%23openstack-neutron.2024-10-08.log.html#t2024-10-08T14:26:33 | 09:02 |
| noonedeadpunk | I think that snat already works in this case, with the https://review.opendev.org/c/openstack/neutron/+/909194 | 09:03 |
| noonedeadpunk | at very least, just avoid binding NAT rule to port does work for us (ie smth like https://review.opendev.org/c/openstack/neutron/+/931495) | 09:04 |
| noonedeadpunk | as the current behaviour I find just wrong - there's no reason to point NAT to the port, which is not residing on chassis - it just can not work | 09:05 |
| noonedeadpunk | even if access from top-layer network is not smth that should be supported, I'm pretty sure that routers and FIPs that just created on geneve should be supproted | 09:06 |
| noonedeadpunk | As logic around - if FIP will work or not is built around that it should be either VLAN or FLAT network | 09:06 |
| noonedeadpunk | but we can discuss that on the drivers meeting indeed | 09:07 |
| ralonsoh | noonedeadpunk, I think your patch makes sense since the LRP is not bound but the router | 09:09 |
| ralonsoh | in that case maybe https://bugs.launchpad.net/neutron/+bug/2035281 won't work but it will the FIP | 09:09 |
| ralonsoh | we need more testing on this | 09:10 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/neutron master: [OVN] Do not supply gateway_port if it's not binded to chassis https://review.opendev.org/c/openstack/neutron/+/931495 | 09:11 |
| noonedeadpunk | Yeah, I'm gonna work on testing for sure, just trying to confirm we totally get an intended behaviour and this covers the usecase described in https://bugs.launchpad.net/neutron/+bug/2083527 | 09:13 |
| ykarel | ralonsoh, ack thx, let's see if hit its with recheck | 09:14 |
| noonedeadpunk | So I have an impression, that https://bugs.launchpad.net/neutron/+bug/2035281 is not applicable for usecase, where LR is binded to chassis instead of LRP | 09:14 |
| noonedeadpunk | as you won't get flows in this case anyway? | 09:15 |
| ralonsoh | your patch at least if fixing an issue that I think the code has: the FIP doesn't work for routers with tunnelled GW networks | 09:15 |
| noonedeadpunk | or well, from what I saw, NAT is being just ignored | 09:15 |
| noonedeadpunk | ++ | 09:15 |
| ralonsoh | I'm not saying the LP bug you created, with multiple GW networks nested should work | 09:15 |
| ralonsoh | but at least your patch is solving another issue | 09:15 |
| ralonsoh | I'm writting this topic now in the drivers meeting for this friday | 09:16 |
| noonedeadpunk | To mention that, I'm not 100% getting the usecase which is intended to be done, as I don't think the intention is to access the A-net from real provider network... | 09:17 |
| noonedeadpunk | But so far the team working on implementation reported that this patch seems to solve the issue they had. So it could be I mis-phrased what the intention after all is in the bug report. | 09:18 |
| ykarel | slaweq, can you check these CI backports https://review.opendev.org/q/Iff86599678ebdd5189d5b56d11f3373c9b138562 | 09:18 |
| noonedeadpunk | But "how to reproduce" section is still valid and what they do check atm | 09:18 |
| opendevreview | Eduardo Olivares proposed openstack/ovn-bgp-agent master: Refactor ovn_bgp_agent.privileged.ovs_vsctl https://review.opendev.org/c/openstack/ovn-bgp-agent/+/931743 | 09:32 |
| opendevreview | Gabriel Adrian Samfira proposed openstack/neutron master: Bind internal router port for VLAN network https://review.opendev.org/c/openstack/neutron/+/931892 | 09:42 |
| gsamfira | ralonsoh: Hey! I uploaded the patch I mentioned yesterday. I also added some context on the bug. | 09:43 |
| gsamfira | not sure if it's correct, but it seems to work in our lab | 09:45 |
| gsamfira | for context, we have an ironic deployment with the genericswitch mechanism driver, 3 nodes set up with enable-chassis-as-gw and a bunch of VLAN networks | 09:46 |
| ralonsoh | gsamfira, first, the scheduling of the router GW ports does not use "ha_chassis_group" but "gateway_chassis" | 09:49 |
| ralonsoh | this is why I proposed a new OVN L3 scheduler in https://review.opendev.org/c/openstack/neutron/+/872033 | 09:49 |
| ralonsoh | second, you are applying this change to all router LRPs | 09:49 |
| ralonsoh | we bind only the GW LRP | 09:50 |
| gsamfira | yep. That's the problem. If we don't bind the internal LRP of the VLAN network that the external ports are attached to, if the ha_chassis_group of the VLAN network differs from the gateway_chassis of the external port, nothing works. | 09:51 |
| gsamfira | if we have a vrouter without an external port on the vrouter, routing between a VLAN network and a geneve network, doesn;t wok | 09:52 |
| ralonsoh | gsamfira, so, because external ports use the OVN "internal" scheduler used by HA_Chassis_Group, we should use the same for the router GW LRPs | 09:52 |
| ralonsoh | so far, the OVN L3 scheduler uses "manual" scheduling, done by Neutron creating several gateway_chassis and adding them to the LRP | 09:53 |
| ralonsoh | in other to sync the external ports and the LRPs, both should use the same scheduling method, that is HA_Chassis_Group | 09:53 |
| ralonsoh | but not in this way but implementing a new OVN L3 scheduler | 09:54 |
| ralonsoh | please check my patch | 09:54 |
| ralonsoh | and feel free to rebase and continue with it (I don't have time right now) | 09:54 |
| gsamfira | will do. I need to see about the difference between the existing schedulers and "internal" schedulers. Not yet familiar enpough with the codebase | 09:55 |
| gsamfira | the scheduler in your patch seems to apply only to external schedulers | 09:55 |
| gsamfira | the scheduler in your patch seems to apply only to external gw ports | 09:56 |
| ralonsoh | right, this is what you need | 09:56 |
| gsamfira | damn. Forgot irc doesn't have edit message :D | 09:56 |
| gsamfira | cool. Thanks for the pointers ralonsoh ! | 09:56 |
| gsamfira | I need to push through a deployment for now, but as soon as I finish, I will look into making the needed changes. | 09:57 |
| gsamfira | ralonsoh: just to confirm. We can't only bind the external GW port. We also need to bind the internal port of VLAN networks. Otherwise, if we connect 2 or more VLAN networks to the same vrouter, each with their own ha_chassis_group potentially on different sets of chassis, we will basically need to make sure that both VLAN networks have the exact same ha_chassis_group with the exact same priorities. Otherwise we end up with a small world probl | 10:12 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Use the constant UNLIMITED_QUOTA=-1 in quota engine https://review.opendev.org/c/openstack/neutron/+/931143 | 10:12 |
| gsamfira | all ports on the same router | 10:13 |
| gsamfira | *same chassis | 10:13 |
| ralonsoh | gsamfira, why do you need that? If you have ports in internal networks (tunnelled or not), the traffic from this VM ports with be sent to the GW LRP chassis, if the traffic needs to go the external network | 10:15 |
| ralonsoh | but there is no need to bind the internal LRP | 10:15 |
| gsamfira | the traffic works from vm --> BM | 10:15 |
| gsamfira | but not the other way around | 10:15 |
| gsamfira | without binding the internal port to the same chassis as the external port in that same VLAN | 10:16 |
| gsamfira | it gets no arp reply | 10:16 |
| gsamfira | from the gw | 10:16 |
| gsamfira | in tcp dump we just see arp who has 10.31.1.1 | 10:16 |
| gsamfira | and no reply | 10:16 |
| ralonsoh | why do you use internal VLAN networks? | 10:16 |
| gsamfira | we'd like to be able to connect ironic managed servers to other tenant networks | 10:17 |
| ralonsoh | ok, I don't know if we test this configuration | 10:18 |
| ralonsoh | but that's another topic (maybe another bug) | 10:18 |
| gsamfira | k8s cluster running on VMs, a few large databases running on bare metal, they should be able to communicate and benefit from security groups and all that | 10:19 |
| gsamfira | I can open another bug and perhaps kick off a discussion. But I'm new to neutron; not sure what the best way to do that would be. | 10:20 |
| gsamfira | this feels like a change that should be discussed first :) | 10:21 |
| ralonsoh | open a LP bug: https://bugs.launchpad.net/neutron/ | 10:21 |
| gsamfira | and I don't want to waste people's time | 10:21 |
| gsamfira | sounds good | 10:21 |
| ralonsoh | at least to report the use of VLAN networks as internal ports in a router | 10:21 |
| gsamfira | https://bugs.launchpad.net/neutron/+bug/1995078/comments/34 <-- seems like the same scenario I described. They created a router, set a GW and attached a VLAN network | 10:33 |
| gsamfira | but I will open a new bug and add more context | 10:33 |
| ralonsoh | much better to have this is a separate context | 10:34 |
| gsamfira | yup | 10:34 |
| ralonsoh | this is a problem that has entity by itself | 10:34 |
| gsamfira | I appreciate the guidance ralonsoh . I know you folks are swamped :) | 10:34 |
| gsamfira | is there a neutron + ironic channel or meeting that takes place? | 10:35 |
| ralonsoh | there is a team meeting on tuesday | 10:41 |
| ralonsoh | https://meetings.opendev.org/#Neutron_Team_Meeting | 10:42 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: add pyproject.toml to support pip 23.1 https://review.opendev.org/c/openstack/neutron/+/899956 | 10:58 |
| opendevreview | Takashi Kajinami proposed openstack/neutron master: Replace deprecated is_advsvc https://review.opendev.org/c/openstack/neutron/+/931574 | 11:29 |
| opendevreview | Jakub Libosvar proposed openstack/ovn-bgp-agent master: Refactor ovn_bgp_agent.privileged.ovs_vsctl https://review.opendev.org/c/openstack/ovn-bgp-agent/+/931743 | 11:52 |
| gsamfira | awesome! Thanks! | 11:56 |
| opendevreview | Eduardo Olivares proposed openstack/ovn-bgp-agent master: Refactor ovn_bgp_agent.privileged.ovs_vsctl https://review.opendev.org/c/openstack/ovn-bgp-agent/+/931743 | 12:45 |
| opendevreview | Brian Haley proposed openstack/neutron stable/2024.1: [OVN] Check metadata HA proxy configuration before restart https://review.opendev.org/c/openstack/neutron/+/931791 | 14:43 |
| opendevreview | Merged openstack/neutron master: [Fullstack] Add active waiting for one HA agent to be active for router https://review.opendev.org/c/openstack/neutron/+/931766 | 16:19 |
| opendevreview | Merged openstack/ovn-bgp-agent master: Refactor ovn_bgp_agent.privileged.ovs_vsctl https://review.opendev.org/c/openstack/ovn-bgp-agent/+/931743 | 16:20 |
| opendevreview | Merged openstack/neutron stable/2024.2: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931863 | 16:53 |
| opendevreview | Merged openstack/neutron stable/2024.1: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931864 | 16:53 |
| opendevreview | Merged openstack/neutron stable/2023.1: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931866 | 17:27 |
| opendevreview | Merged openstack/neutron unmaintained/zed: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931871 | 17:48 |
| opendevreview | Merged openstack/neutron master: Use the constant UNLIMITED_QUOTA=-1 in quota engine https://review.opendev.org/c/openstack/neutron/+/931143 | 19:13 |
| opendevreview | Merged openstack/neutron stable/2024.2: Ensure the OVSDB and ovn-northd process stop https://review.opendev.org/c/openstack/neutron/+/931014 | 19:13 |
| opendevreview | Merged openstack/neutron stable/2024.1: [OVN] Check metadata HA proxy configuration before restart https://review.opendev.org/c/openstack/neutron/+/931791 | 19:13 |
| opendevreview | Merged openstack/neutron stable/2023.2: [CI] Functional: Increase Ulimit to 4096 https://review.opendev.org/c/openstack/neutron/+/931865 | 20:52 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: mypy: neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/* https://review.opendev.org/c/openstack/neutron/+/929866 | 21:38 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: mypy: neutron/db/l3* https://review.opendev.org/c/openstack/neutron/+/929867 | 21:38 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: mypy: use dataclass for AgentConsumer https://review.opendev.org/c/openstack/neutron/+/929868 | 21:38 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: mypy: neutron/extensions/* https://review.opendev.org/c/openstack/neutron/+/929869 | 21:38 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: mypy: neutron/agent/* https://review.opendev.org/c/openstack/neutron/+/929870 | 21:38 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: mypy: neutron/objects/* https://review.opendev.org/c/openstack/neutron/+/929871 | 21:38 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: Remove dhcp.PORTS attribute https://review.opendev.org/c/openstack/neutron/+/931993 | 21:47 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!