| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: Introduce ovs-lts jobs that run against the latest ovn LTS branch https://review.opendev.org/c/openstack/neutron/+/927221 | 00:33 |
|---|---|---|
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM test nested snat with ovn https://review.opendev.org/c/openstack/neutron/+/927222 | 00:51 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: Remove remnants of Open Daylight https://review.opendev.org/c/openstack/neutron/+/927223 | 01:28 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: Remove remnants of NSX https://review.opendev.org/c/openstack/neutron/+/927224 | 01:31 |
| mlavalle | haleyb: I implemented the improvement you recommended for https://review.opendev.org/c/openstack/neutron/+/917800. It is ready for the next review. Thanks in advance! | 01:43 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron-tempest-plugin master: DNM: enable nested_snat scenario for ovn https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/927225 | 01:52 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM test nested snat with ovn https://review.opendev.org/c/openstack/neutron/+/927222 | 01:53 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Make RPC event cast synchronous with the event https://review.opendev.org/c/openstack/neutron/+/926922 | 06:28 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Monkey patch the system libraries before calling them https://review.opendev.org/c/openstack/neutron/+/925376 | 06:29 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [WSGI] Move all OVN jobs to use WSGI API module https://review.opendev.org/c/openstack/neutron/+/924317 | 06:29 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: [OVN][FT] Ensure the non OVN agent is correctly created https://review.opendev.org/c/openstack/neutron/+/927236 | 07:36 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Add new "tagging" API method: create (POST) https://review.opendev.org/c/openstack/neutron/+/924724 | 08:05 |
| opendevreview | Merged openstack/neutron master: Bump ``oslo.serialization`` to version 5.5.0 https://review.opendev.org/c/openstack/neutron/+/927098 | 09:47 |
| opendevreview | Merged openstack/ovn-octavia-provider stable/2024.1: Error log for missing certs with NB and SB DBs https://review.opendev.org/c/openstack/ovn-octavia-provider/+/927114 | 10:24 |
| opendevreview | Merged openstack/ovn-octavia-provider stable/2023.2: Error log for missing certs with NB and SB DBs https://review.opendev.org/c/openstack/ovn-octavia-provider/+/927203 | 10:24 |
| opendevreview | Merged openstack/ovn-octavia-provider stable/2023.1: Error log for missing certs with NB and SB DBs https://review.opendev.org/c/openstack/ovn-octavia-provider/+/927174 | 10:38 |
| opendevreview | Lajos Katona proposed openstack/neutron-tempest-plugin master: DNM test os-vif change in neutron gate https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/923389 | 12:21 |
| sean-k-mooney | hi folks im unsure how to proceed with https://review.opendev.org/c/openstack/os-vif/+/923036/comment/65489f07_383b2fe2/ | 13:01 |
| sean-k-mooney | it seam liek since we implemnted supprot for vlan trunking for ml2/ovs some changes have been made to neutron that changes the behavior which breaks os-vif | 13:02 |
| sean-k-mooney | also a config option explicitly_egress_direct=true | 13:02 |
| sean-k-mooney | was added that replaces using the normal action which is part of the core design of how the trunk prots implementation works | 13:03 |
| sean-k-mooney | this is entrily invisable to nova/os-vif which is concerning | 13:04 |
| sean-k-mooney | the core part of the fix is correct | 13:04 |
| sean-k-mooney | https://review.opendev.org/c/openstack/os-vif/+/923036/2/vif_plug_ovs/ovs.py#213 | 13:04 |
| sean-k-mooney | setting kwargs['vlan_mode'] = 'trunk' | 13:05 |
| sean-k-mooney | kwargs['trunks'] = constants.DEAD_VLAN | 13:05 |
| sean-k-mooney | is the bit i kind of take issue with as that to me is not backportable | 13:05 |
| ralonsoh | sean-k-mooney, explicitly_egress_direct is false by default, it is used to avoid the normal action flooding when using the OVS FW | 13:05 |
| ralonsoh | the release note has a comment about using it with other services | 13:06 |
| sean-k-mooney | yep i read | 13:06 |
| sean-k-mooney | but either neutorn has to reject vlan trunking when its set to true | 13:06 |
| sean-k-mooney | or you need to reimplemnt the trunking behavior | 13:06 |
| ralonsoh | I don't think this config option was even tested with trunks | 13:06 |
| sean-k-mooney | and that was not part fo the orginal commit that added it as far as i saw when skiming it | 13:06 |
| ralonsoh | actually this config option is for optimizing the OVS FW, trunks cannot be used with FW | 13:07 |
| sean-k-mooney | yep so the issue i have with https://review.opendev.org/c/openstack/os-vif/+/923036/2/vif_plug_ovs/ovs.py#215 | 13:07 |
| sean-k-mooney | is that we have never supproted anything other then acesss port when we were using libvirt to do the pluggin | 13:07 |
| sean-k-mooney | we now dont and use os-vif explcively | 13:07 |
| sean-k-mooney | but the current config was not doable when libvirt did the port plugging | 13:08 |
| ralonsoh | why https://review.opendev.org/c/openstack/os-vif/+/923036/4/vif_plug_ovs/ovs.py is not possible? | 13:08 |
| sean-k-mooney | so to me this is more of a feature then a bug fix if you actuly want to have os-vif create ovs ports as trunks not acces ports | 13:08 |
| ralonsoh | sorry, I don't understand the relationship | 13:08 |
| ralonsoh | no no, this code is to mark the port as dead, a similar code is used in Neutron | 13:09 |
| ralonsoh | by marking the vlan_mode=trunk, this port only accepts vlan traffic, and the we use DEAD_VLAN | 13:09 |
| sean-k-mooney | i know but untill wallaby ish this port was not created by os-vif when not using hybrid plug | 13:09 |
| sean-k-mooney | ralonsoh: right which is not how this feature is ment ot work | 13:10 |
| sean-k-mooney | ralonsoh: form my point of view this is a backward incompatible change in behaivor and change in the contract between os-vif and neutron | 13:11 |
| ralonsoh | sean-k-mooney, but I don't understand, os-vif always creates the port (we are not talking here of trunk ports) | 13:11 |
| ralonsoh | sean-k-mooney, https://review.opendev.org/c/openstack/neutron/+/827315/12/neutron/agent/common/ovs_lib.py | 13:11 |
| sean-k-mooney | so the ports on the br-int can be trunk ports if neutron wants | 13:12 |
| sean-k-mooney | those are created by neutron | 13:12 |
| sean-k-mooney | but the tap device on the trunk bridge | 13:12 |
| sean-k-mooney | is ment to allow all vlans | 13:12 |
| sean-k-mooney | the bug here is that we incorrectly applien teh dead vlan to the tap when the tap was on a trunk bridge | 13:13 |
| sean-k-mooney | we should not be marking the tap as a trunkin my view that what i take issue with | 13:13 |
| sean-k-mooney | the vlan mapping/filtering is done by taging each end of the patch port that conencts the trunk bridge and br-int | 13:14 |
| sean-k-mooney | and we rely on the normal actoin to do the vlan translation | 13:14 |
| ralonsoh | sean-k-mooney, but this code will apply only if "not self._is_trunk_bridge(bridge)" | 13:14 |
| ralonsoh | we are not doing that on ports on the trunk bridge | 13:15 |
| sean-k-mooney | it will but until now nova/os-vif/libvirt has never created trunk prots on br-int | 13:15 |
| ralonsoh | sean-k-mooney, this is NOT for trunk ports | 13:15 |
| sean-k-mooney | i know | 13:15 |
| sean-k-mooney | but we are changing how prots are created on br-int form being a accessport to now always being a trunk port | 13:15 |
| sean-k-mooney | which would mean that neurton will now have to fix that | 13:16 |
| ralonsoh | we are doing it but later, once the TAP port has been created | 13:16 |
| sean-k-mooney | so we cant safely backport this unless we know that the neutron code has already been backported | 13:16 |
| ralonsoh | the Neutron code that undo this config (in other words, the code that unsets the vlan_mode and sets the correct tag) is https://review.opendev.org/c/openstack/neutron/+/819567 | 13:18 |
| ralonsoh | has been there since Zed | 13:18 |
| sean-k-mooney | the packet flooding issue by the way i think is because at somepoint neturon change the drop rule for the dead vlan | 13:19 |
| sean-k-mooney | it change to the one that also supprot the qos bits instad of the older and simpler dl_vlan mathc or whatever the old one was | 13:20 |
| sean-k-mooney | usin the tsi one or what ever its using currently didnt work properly | 13:20 |
| ralonsoh | sean-k-mooney, sorry, I don't know what code are you talking about | 13:21 |
| sean-k-mooney | the orgainl code that was added to ensure that the dead valn used dl_vlan=4095 to match in the flow rule | 13:21 |
| sean-k-mooney | at some point it was updated to vlan_tci=0x0fff/0x1fff | 13:21 |
| sean-k-mooney | whic did not work | 13:21 |
| sean-k-mooney | that is what caused the packet flooding i belvie | 13:22 |
| ralonsoh | we only use this match for flat traffic: 0x0000/0x1fff | 13:24 |
| ralonsoh | FLAT_VLAN_TCI = '0x0000/0x1fff' | 13:24 |
| sean-k-mooney | when we did the dead vlan stuff orginally we add priority=65535,dl_vlan=4095,action=drop | 13:25 |
| sean-k-mooney | im 99% sure of that but i could be wrong | 13:25 |
| sean-k-mooney | this was part of https://bugs.launchpad.net/neutron/+bug/1734320 | 13:26 |
| sean-k-mooney | https://review.opendev.org/c/openstack/neutron/+/616609/2/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/br_int.py#46 | 13:27 |
| lajoskatona | sean-k-mooney, ralonsoh: I planned to bring this topic to today's Neutron meeting's on-demand agenda, but worst case we can add this also to the drivers meeting agenda to give time for people to think about the issue and the history of it | 13:28 |
| sean-k-mooney | https://review.opendev.org/c/openstack/neutron/+/616609/2/neutron/tests/unit/plugins/ml2/drivers/openvswitch/agent/openflow/native/test_br_int.py | 13:28 |
| sean-k-mooney | so when i fixed the cve i expleictly use priority=65535,dl_vlan=4095,action=drop | 13:28 |
| ralonsoh | sean-k-mooney, that was updated in https://review.opendev.org/c/openstack/neutron/+/820897 | 13:30 |
| ralonsoh | the rationale is in the commit message | 13:30 |
| sean-k-mooney | so i dont think that commit was correct | 13:30 |
| sean-k-mooney | also its not the same as i orginally impelmented | 13:30 |
| sean-k-mooney | the comit says the flow looks liek | 13:31 |
| sean-k-mooney | priority=65535,vlan_tci=0x0fff/0x1fff actions=drop | 13:31 |
| sean-k-mooney | which is not what i used | 13:31 |
| sean-k-mooney | i sued dl_vlan=4095 intentially becaue it ignore the qos bits and only matches on the vlan id | 13:31 |
| sean-k-mooney | if you use the tsi version you have to also match on the tos bits on the valan frame | 13:32 |
| sean-k-mooney | so at some poitn the python bidnign stop generating "priority=65535,dl_vlan=4095,action=drop" and started generating "priority=65535,vlan_tci=0x0fff/0x1fff actions=drop" which are not semanticly the same | 13:33 |
| ralonsoh | sean-k-mooney, I'll check what "install_drop" is doing in this case | 13:34 |
| sean-k-mooney | i assume neutron does not stil have the ofctl driver? | 13:37 |
| ralonsoh | no, long time ago | 13:38 |
| sean-k-mooney | so i think this was a silent secuiryt bug intoduced in os-ken | 13:38 |
| sean-k-mooney | or neuton causing dl_vlan to be tanslated to vlan_tci incorretly | 13:39 |
| sean-k-mooney | ralonsoh: the commit message for that fix in neutorn say tey were settin the top bit to emulaate the old behavior | 13:39 |
| ralonsoh | I'll open a LP bug to investigate it | 13:39 |
| sean-k-mooney | "priority=65535,vlan_tci=0x0fff/0x1fff actions=drop" to "priority=65535,vlan_tci=0x1fff/0x1fff actions=drop" | 13:39 |
| sean-k-mooney | but if that was actully working then it would not need use to mark the prots as trunk prots when adding them to br-int | 13:40 |
| sean-k-mooney | os-ken has translation code to upgrade flow to new version of openflow and i dont think that is correctly handeling dl_vlan | 13:44 |
| sean-k-mooney | https://github.com/openstack/os-ken/blob/fc77cfdbc9df7af9cb45fc4474551cf6c226a0eb/os_ken/lib/ofctl_v1_5.py#L138-L180 | 13:44 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM try with Ales' fix for FIP-to-FIP vs zero-snat https://review.opendev.org/c/openstack/neutron/+/927272 | 13:45 |
| sean-k-mooney | https://github.com/openstack/os-ken/blob/unmaintained/zed/os_ken/lib/ofctl_utils.py#L174-L179 | 13:46 |
| sean-k-mooney | its proably becasue of this note about hex notation | 13:46 |
| rubasov | even if the vlan_tci match is corrected to the dl_vlan match, the dead_vlan drop rule cannot see the vlan of an access port, please read the email on the ovs-dev mailing list, I referred to from the review comments | 13:47 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM try with Ales' fix for FIP-to-FIP vs zero-snat https://review.opendev.org/c/openstack/neutron/+/927272 | 13:47 |
| sean-k-mooney | rubasov: im pretty sure we verifed that the drop rule worked as part of fixing the cve by looking at the ovs-dpctl and ovs-ofctl output | 13:52 |
| sean-k-mooney | its been 5 years since we did this so i cant recall all thet details but im pretty sure we check this actully worked as part fo the cve verificaion | 13:53 |
| sean-k-mooney | the ovs behvior may have chnaged but if the drop rule did not work the the cve was never patched | 13:54 |
| rubasov | I assumed that the dead vlan drop rule must have worked (and was tested by somebody) when it was introduced, but today it does not. I also do not understand how this could have changed. I was flabbergasted until people from ovs-dev explained to me what's going on. | 13:56 |
| sean-k-mooney | so the problem is that os-vif and nova cant assume that neutorn has supprot for managing ovs ports with vlan mode = trunk | 13:59 |
| sean-k-mooney | nova/os-vif to date has expclivily supproted only access ports | 13:59 |
| sean-k-mooney | now ralonsoh say this supprot was backported to zed | 13:59 |
| sean-k-mooney | so we may be able to say any stable branch supprots this | 14:00 |
| ralonsoh | what support? the patch I mentioned? it was implemented in zed | 14:00 |
| ralonsoh | but I think it was also backported | 14:00 |
| sean-k-mooney | but that a pretty fundemental change in the prot bluging bevhiaor | 14:00 |
| haleyb | #startmeeting networking | 14:00 |
| opendevmeet | Meeting started Tue Aug 27 14:00:52 2024 UTC and is due to finish in 60 minutes. The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'networking' | 14:00 |
| haleyb | Ping list: bcafarel, elvira, frickler, mlavalle, mtomaska, obondarev, slaweq, tobias-urdin, ykarel, lajoskatona, jlibosva, averdagu, amotoki, haleyb, ralonsoh | 14:00 |
| mlavalle | \o | 14:00 |
| rubasov | o/ | 14:01 |
| ralonsoh | hello | 14:01 |
| obondarev | o/ | 14:01 |
| sean-k-mooney | o/ | 14:01 |
| ihrachys | o/ | 14:01 |
| elvira | o/ | 14:01 |
| ykarel | o/ | 14:01 |
| haleyb | #topic announcements | 14:01 |
| haleyb | I had forgotten to mention i was going to be out yesterday, i'll catch-up with the pings today | 14:02 |
| frickler | \o | 14:02 |
| haleyb | I will also be out Thursday but will at least change my nick :) | 14:02 |
| cbuggy | o/ | 14:02 |
| slaweq | o/ | 14:02 |
| haleyb | We are now in Dalmatian release week (R - 5) | 14:03 |
| haleyb | Work on libraries should be wrapping up! | 14:03 |
| haleyb | Non-client library freeze was last week | 14:03 |
| haleyb | Client library freeze: August 29th | 14:03 |
| haleyb | Dalmatian-3 milestone: August 29th | 14:03 |
| haleyb | Only bugfixes should be allowed in the master branch beyond this point. Any feature work past that deadline has to be raised as a Feature Freeze Exception (FFE) and approved by the team PTL | 14:04 |
| haleyb | i.e. after D-3 | 14:04 |
| ihrachys | what's the timeline for release branch spin-off? | 14:04 |
| haleyb | when they will tag it? that should be thursday | 14:05 |
| lajoskatona | o/ | 14:06 |
| ihrachys | sorry. was the note about master branch applying to client / lib only? | 14:06 |
| ihrachys | context: I have a feature. I don't have a requirement to land it for dalmatian. but I would like to be able to land it earlier. so if master is locked, I'd like to understand when it gets unlocked. | 14:07 |
| frickler | branch creation patches will be proposed next week by the release team iiuc | 14:08 |
| haleyb | ihrachys: what is the feature? we can discuss exceptions | 14:08 |
| ralonsoh | the stable branches are proposed by the release team | 14:09 |
| ihrachys | the nested snat thingy for ovn. | 14:09 |
| haleyb | frickler: so next thursday? everything seems to be on a thursday | 14:09 |
| ihrachys | frickler: ah ok so a matter of a week or two | 14:09 |
| ralonsoh | and should be this thursday | 14:09 |
| haleyb | ihrachys: i'd call that a bug fix not a feature | 14:09 |
| slaweq | frickler aren't branches created during the RC-1 week? | 14:10 |
| slaweq | and according to https://releases.openstack.org/dalmatian/schedule.html is in 2 weeks | 14:11 |
| slaweq | not next week | 14:11 |
| frickler | slaweq: this is the proposal task https://etherpad.opendev.org/p/dalmatian-relmgt-tracking#L352 | 14:11 |
| ralonsoh | right | 14:11 |
| haleyb | i think i still have vacation on the brain, yes that schedule is the best place for info | 14:11 |
| frickler | they then have some time to get approved or amended if needed | 14:11 |
| haleyb | RC-1 target week is R-3, which is in two weeks, so basically features should be done (or exceptions filed) by this week, continue fixing bugs for 2 weeks | 14:12 |
| slaweq | ahh, ok, so next week it will be for libraries | 14:13 |
| slaweq | and branch for e.g. neutron will be created in R-3 week which is in 2 weeks | 14:13 |
| haleyb | libraries should be tagged this week, R-5 | 14:14 |
| haleyb | sorry if i confused anyone, but the schedule slaweq linked is the source of truth | 14:15 |
| haleyb | one thing i will mention is if you have any cycle highlights i will start making a short list, please ping me if you have something that should make that list | 14:16 |
| haleyb | Reminder: If you have a topic for the drivers meeting on Friday, please add it to the wiki @ https://wiki.openstack.org/wiki/Meetings/NeutronDrivers | 14:17 |
| slaweq | please remember that cycle highlights should be "marketing oriented list of features", not very technical one | 14:17 |
| haleyb | right, that might be a short list, and as i remember we only include 2-3 at the most anyways | 14:19 |
| slaweq | yeap | 14:20 |
| haleyb | i did not have any other announcements, does anyone else? | 14:20 |
| haleyb | i think nominations for PTL/TC elections end tomorrow, please see the ML email which has more info | 14:21 |
| haleyb | looks like there are only 3 nominations for 4 TC openings at the moment | 14:22 |
| slaweq | yes, tomorrow at 23:45 utc | 14:22 |
| slaweq | There is 4 nominations for TC already | 14:22 |
| slaweq | but anyone is welcome to nominate themself :) | 14:23 |
| haleyb | oh, ok, your email was at 3am my time so a lot has happened :) | 14:23 |
| mlavalle | we live in the past this side of the pond | 14:23 |
| slaweq | yes, after I sent it, there was 4th nomination proposed so that has changed already | 14:23 |
| frickler | yes, this morning we were at 31% PTL candidacies, this is looking a bit better now | 14:23 |
| slaweq | :) | 14:23 |
| lajoskatona | mlavalle: :D | 14:24 |
| haleyb | I did have one more announcement regarding priorities for the rest of the cycle... | 14:25 |
| haleyb | Please use the priorities dashboard for patches in the "ready to merge" state. This could be older changes as well as new ones | 14:25 |
| haleyb | so set RP+1 - that way we can check that dashboard and get it reviewed | 14:26 |
| haleyb | that was all i had, can move to bugs | 14:27 |
| haleyb | #topic bugs | 14:27 |
| haleyb | rubasov was the bug deputy last week, his report is at | 14:27 |
| haleyb | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/3HPRIASSKJPRRMFU6XBACVNE5EXDJR6M/ | 14:27 |
| mlavalle | rememeber to set RP+1 again if your submit a revision to your patch | 14:27 |
| haleyb | mlavalle: oh, it gets cleared? | 14:28 |
| rubasov | ovn-octavia maintainers please have a look at the untriaged bug | 14:28 |
| mlavalle | yes, I made that mistake last night | 14:28 |
| rubasov | I'll follow up with Anton on the incomplete one, when he responds | 14:28 |
| rubasov | otherwise all new bugs are assigned | 14:28 |
| haleyb | i did not see this one assigned, unless anton will also take that over? | 14:29 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/2077533 | 14:29 |
| haleyb | regarding DVR router processing | 14:29 |
| rubasov | that's the incomplete - at least that's how I understood it | 14:30 |
| haleyb | ack, i'll just follow it | 14:30 |
| rubasov | but please correct me there if you think differently | 14:31 |
| haleyb | there was one related to fwaas and IPv6 cidrs, which almost seemed like a bug | 14:31 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/2077596 | 14:31 |
| haleyb | rubasov: no, that's fine | 14:31 |
| rubasov | we had a previous similar one handled as an rfe, because there was an api behavior change | 14:32 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/1869129 | 14:33 |
| rubasov | I believe that's why the reporter opened it as an rfe, however he did not write about yet his proposed change (I assume it would similarly involve an api behavior change) | 14:33 |
| haleyb | i hadn't noticed we tagged that as an rfe due to the nature of the change, but makes sense | 14:33 |
| haleyb | i will re-read and see what Liu's response it | 14:35 |
| slaweq | I think we had discussion about that long time ago and we added "normalized_cidr" field to the SG rule | 14:37 |
| slaweq | to make people aware what is really set by the backend | 14:37 |
| haleyb | slaweq: right, thanks for the reminder | 14:37 |
| ihrachys | curious - did we talk to ovn then to see if they should just support it? | 14:38 |
| slaweq | it was related to the RFE you linked already https://bugs.launchpad.net/neutron/+bug/1869129 | 14:38 |
| haleyb | ihrachys: the review was https://review.opendev.org/c/openstack/neutron/+/736386 | 14:41 |
| ihrachys | looks like horizon UX concerns were pushed down to neutron. I guess the ship has sailed. | 14:43 |
| sean-k-mooney | kind of | 14:43 |
| sean-k-mooney | so conceptually the neturon api should be concistent acrros backends | 14:44 |
| sean-k-mooney | i.e. if openstack security group rule create --protocol icmp --remote-ip 10.10.10.175/32 cidr | 14:44 |
| sean-k-mooney | is valide for one backend it should be valid for all of them that supprot security groups | 14:45 |
| haleyb | it is now after that change i believe | 14:45 |
| sean-k-mooney | ah ok | 14:45 |
| sean-k-mooney | we have a similar probelm in nova related to regex matchs in list ectra | 14:45 |
| sean-k-mooney | the regex has to be in the db's native format... | 14:46 |
| haleyb | job security :) | 14:46 |
| sean-k-mooney | it shoudl be normalise vai a standar pcre step good to know cidrs are now normalised | 14:46 |
| haleyb | ralonsoh: i did have a question for you - did your RPC change make the gate better? | 14:48 |
| ralonsoh | yes, please check the DNM patch | 14:48 |
| ralonsoh | https://review.opendev.org/c/openstack/neutron/+/926788 | 14:48 |
| haleyb | ack, will look after meeting | 14:48 |
| haleyb | This week mlavalle is the bug deputy, next week will be ihrachys | 14:49 |
| haleyb | is that good for both of you | 14:49 |
| noonedeadpunk | hey folks! I'm trying to debug ovn + octavia provider misbehaviour here and got slightly cornered. Maybe you can point me to how it is supposed to work | 14:49 |
| mlavalle | yes | 14:49 |
| mlavalle | fine with me | 14:49 |
| haleyb | just trying to move along since we have 10 minutes left | 14:49 |
| ihrachys | haleyb: ok | 14:49 |
| haleyb | Current bug count this week: 717, down 5 from last week | 14:50 |
| haleyb | noonedeadpunk: someone can sync with you after the meeting perhaps | 14:50 |
| haleyb | #topic community-goals | 14:50 |
| lajoskatona | I had no time to work on the SDK realted topic last week, so no news from me | 14:52 |
| haleyb | ralonsoh: seems like once the RPC change is merged the monkeypatch one will follow. then just the template change? | 14:52 |
| haleyb | #link https://review.opendev.org/c/openstack/neutron/+/924317/9 | 14:52 |
| haleyb | for evenlet that is | 14:52 |
| ralonsoh | yes, that's correct. Right now there is an issue with the ovn tempest ipv6 job | 14:52 |
| ralonsoh | but the other jobs (ovn and ovs) seems to run fine with wsgi | 14:53 |
| noonedeadpunk | so the thing is that I'm able to reach octavia backends via VIPs (an unattached port) but somehow FIP is not reachable | 14:53 |
| ralonsoh | the next step: to disable eventlet in oslo.service and make the agents run fine... | 14:53 |
| ralonsoh | that's all from me | 14:53 |
| haleyb | ralonsoh: great, thanks | 14:53 |
| noonedeadpunk | though I do see in OVN NAT rules for the port. But what I'm confused about (and not sure if that should matter), is that port is not binded to any gateway node | 14:54 |
| lajoskatona | ralonsoh: but that is (make oslo.service eventletless) is for the coming cycles am I right< | 14:54 |
| mlavalle | ralonsoh haleyb: should https://review.opendev.org/c/openstack/neutron/+/926922 be in the priority list? | 14:54 |
| lajoskatona | ? | 14:54 |
| ralonsoh | lajoskatona, yes, we are still reviewing the spec for this change | 14:54 |
| ralonsoh | mlavalle, could be, yes, it could be interesting to have in before we end Dalmatian | 14:55 |
| haleyb | mlavalle: yes, i was going to review right after meeting so maybe moot if we approve, but will add if not | 14:55 |
| mlavalle | yeah, the monkey patch follow up was in the priority list. so it makes sense | 14:55 |
| haleyb | so i did have one RBAC question in the community goals section, seems the default override change is still failing the designate job | 14:57 |
| haleyb | #link https://review.opendev.org/c/openstack/neutron/+/926085 | 14:57 |
| slaweq | yes, I have it in my todo list to check that locally | 14:58 |
| slaweq | but I still didn't had time to get to this yet | 14:58 |
| haleyb | slaweq: is that a bug on the neutron or designate side you think? | 14:59 |
| johnsom | It passes in the Designate gates | 14:59 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron-tempest-plugin master: DNM Use fip-to-fip-vs-snat Ales patch for plugin jobs https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/927283 | 14:59 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM try with Ales' fix for FIP-to-FIP vs zero-snat https://review.opendev.org/c/openstack/neutron/+/927272 | 14:59 |
| johnsom | It seems to fail when neutron is using the oslo policy 4.4 in the neutron API. | 15:00 |
| johnsom | Though I don't have a "why" either | 15:00 |
| slaweq | most likely on neutron, as we are using probably wrong context to make request | 15:00 |
| slaweq | but as I said, I will deploy it locally and try to reproduce and investigate, probably next week | 15:00 |
| haleyb | ah, ok, i'll watch the change and thanks everyone that's looked into it | 15:01 |
| haleyb | #topic on-demand | 15:01 |
| opendevreview | Merged openstack/neutron master: Get ips from system dns resolver without scope https://review.opendev.org/c/openstack/neutron/+/926079 | 15:01 |
| haleyb | we are over time, did anyone have another topic? | 15:01 |
| mlavalle | can I get eyes on https://review.opendev.org/c/openstack/neutron/+/917800. I implemented the improvement recommended by haleyb. And I want it to be included in Dalmatian | 15:01 |
| haleyb | mlavalle: yes, i will take a look, thanks for the update! | 15:02 |
| mlavalle | thank you, that was a nice optimization | 15:02 |
| haleyb | thanks for attending and the good discussions, as i said i'm out thursday if you're looking for me then, back on friday | 15:03 |
| haleyb | #endmeeting | 15:03 |
| opendevmeet | Meeting ended Tue Aug 27 15:03:25 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:03 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/networking/2024/networking.2024-08-27-14.00.html | 15:03 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/networking/2024/networking.2024-08-27-14.00.txt | 15:03 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/networking/2024/networking.2024-08-27-14.00.log.html | 15:03 |
| ralonsoh | bye | 15:03 |
| mlavalle | \o | 15:03 |
| otherwiseguy | haleyb: My testing of https://patchwork.ozlabs.org/project/ovn/patch/20240827085252.458355-1-amusil@redhat.com/ with setting logical_ip=0.0.0.0/0 for nested nat shows it working for host2fip, fip2fip, and nested scenarios. You might want to test too just to make sure that it works for your specific case. | 15:07 |
| haleyb | ralonsoh: i had a nitty comment in https://review.opendev.org/c/openstack/neutron/+/926922 - i will approve if you just want to fix it in a follow-on | 15:10 |
| ralonsoh | haleyb, right, let me push a new PS | 15:10 |
| opendevreview | Rodolfo Alonso proposed openstack/neutron master: Make RPC event cast synchronous with the event https://review.opendev.org/c/openstack/neutron/+/926922 | 15:11 |
| mlavalle | ralonsoh haleyb and I approved to monkey patch one, which depended on ^^^^ | 15:12 |
| ralonsoh | thanks | 15:12 |
| haleyb | otherwiseguy: great! i will have to pull in that change locally and test. And I saw your comment that 22.04 might work (i think) but maybe i mis-understood that? it's almost like it worked by accident? | 15:15 |
| otherwiseguy | haleyb: 23.09 might work, but 22.03 didn't. | 15:16 |
| otherwiseguy | (I mistyped before) | 15:17 |
| haleyb | otherwiseguy: ah, i had it backwards, so i'd need to look for a needle in that haystack, then try and apply ales' change | 15:17 |
| otherwiseguy | haleyb: I think the plan is to backport the fix though. | 15:18 |
| ihrachys | haleyb: on a related note, https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/618557 I think we should enable this before landing anything for ovn nested since it should catch one of the regressions that were present in v1 of the Ales' patch | 15:18 |
| otherwiseguy | not sure exactly how far they'll go. | 15:18 |
| ihrachys | (also it seems like overall a good idea to actually cover fip-to-fip communication...) | 15:18 |
| * haleyb will get his popcorn and watch that one | 15:18 | |
| ihrachys | otherwiseguy: I don't think Ales knows what is missing in 22.03 or that he planned to look into it himself. I may be wrong. haleyb may have to bribe someone! | 15:19 |
| ihrachys | on a more witty note, haleyb I wonder what's your gut reaction to https://review.opendev.org/c/openstack/neutron/+/927216 ? | 15:20 |
| haleyb | ihrachys: i can ship beer anywhere, or if they are in the Boston area will deliver :) | 15:20 |
| otherwiseguy | ihrachys: haleyb: yeah, and if it was an LTS, I think those get support for 2 years which would be over by now? | 15:20 |
| haleyb | if we get it back to a certain point, i will have to bug our ovn engineering team to look at it | 15:21 |
| ihrachys | current lts is 24.03 and "LTS releases will receive bug fixes until the point that another LTS is released. At that point, the old LTS will receive an additional year of critical and security fixes. Critical fixes are those that are required to ensure basic operation (e.g. memory leak fixes, crash fixes). Security fixes are those that address concerns about exploitable flaws in OVN and that | 15:21 |
| ihrachys | have a corresponding CVE report." | 15:21 |
| ihrachys | is this "critical"? that's where bribes would make a difference. | 15:22 |
| haleyb | otherwiseguy: we support LTS for like 10-12 years :-o | 15:22 |
| haleyb | i mean downstream of course | 15:22 |
| ihrachys | haleyb: obviously we as in people paid for products support it for as long as they do pay; they as in ovn upstream community will drop 22.03 some time in 2025 for good | 15:22 |
| ihrachys | on a relevant note, we should also run our tests against their lts: https://review.opendev.org/c/openstack/neutron/+/927221 this adds experimental checks but we could promote (some of) them to periodic or whatever | 15:24 |
| haleyb | ihrachys: yeah, and that's when we start supporting things like 24.xx on older releases when possible since it gets ugly dealing with backporting | 15:24 |
| ihrachys | at one point the gate switches to a "later" ubuntu that ships a newer ovn, correct? | 15:25 |
| opendevreview | Elvira García Ruiz proposed openstack/neutron stable/2024.1: Get ips from system dns resolver without scope https://review.opendev.org/c/openstack/neutron/+/927292 | 15:27 |
| haleyb | ihrachys: regarding linux bridge, i will put that in the topics for Epoxy meetings. We had a recent-ish bug and there is noone that can really dig into such things | 15:27 |
| ihrachys | also dibbler | 15:27 |
| ihrachys | I'd say the experimental section is currently misused | 15:27 |
| ihrachys | or has a wrong name | 15:27 |
| ihrachys | it's not experimental, it's "on_death_bed" | 15:28 |
| opendevreview | Elvira García Ruiz proposed openstack/neutron stable/2023.2: Get ips from system dns resolver without scope https://review.opendev.org/c/openstack/neutron/+/927293 | 15:28 |
| haleyb | ihrachys: yes, there will be a time when the gate moves forward, no comment from me on that :( | 15:28 |
| ihrachys | if I were to decide, i'd just pull the plug and let the code go with peace :p | 15:28 |
| haleyb | ihrachys: 'check on_death_bed' | 15:29 |
| opendevreview | Elvira García Ruiz proposed openstack/neutron stable/2023.1: Get ips from system dns resolver without scope https://review.opendev.org/c/openstack/neutron/+/927294 | 15:29 |
| ihrachys | haleyb: epoxy is slurp; probably a good time to clean up rotten code after it's cut off then you think? or before that? | 15:31 |
| haleyb | ihrachys: yes, good time for cleanup | 15:32 |
| ihrachys | before or after? do we want to ship a slurp release with lb / dibbler / (probably there's more)? | 15:33 |
| haleyb | ihrachys: i don't think we can deprecate in dalmatian, would need to consult the docs | 15:35 |
| ihrachys | experimental in my mind is even worse than deprecated - you already have to do some legwork to allow it in the first place. | 15:36 |
| ihrachys | can't miss the fact | 15:36 |
| ihrachys | so what would be the plan you think? mark deprecated post-dalmatian (if not already), then kill for good post-epoxy? | 15:37 |
| ihrachys | I'm talking about lb and dibbler so far but we can discuss the feature list more systematically after dalmatian. | 15:38 |
| ihrachys | e.g. I don't know if we test / people use metering agent or macvtap agent. or what's the plan for iptables fw for ml2/ovs. etc. etc. | 15:38 |
| ihrachys | we can consult user surveys (btw 2024 one closes tomorrow I think), but e.g. 2023 one showed ~30% on LB. should this fact stop us from killing the unattended code? | 15:39 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: DNM try with Ales' fix for FIP-to-FIP vs zero-snat https://review.opendev.org/c/openstack/neutron/+/927272 | 15:43 |
| opendevreview | Merged openstack/neutron stable/2023.1: [stable-only][2023.1] Remove CI job "tox-py311" https://review.opendev.org/c/openstack/neutron/+/927157 | 15:58 |
| sean-k-mooney | macvtap is a very simple ml2 plugin but we dont test it in ci | 16:00 |
| sean-k-mooney | it is nice if you want really really simple networking | 16:00 |
| sean-k-mooney | but it has much less functionality | 16:00 |
| sean-k-mooney | i would expect ml2/ovs with iptabels still proably has a large userbase | 16:01 |
| sean-k-mooney | but it would be interesting to see if that pans out in the survay | 16:02 |
| haleyb | ihrachys: sorry, too many meetings here. With LB moving to experimental, and then having almost no bugs filed, is that 30% number accurate? It's either super stable or people just click the box (along with others) since they know the name | 16:17 |
| haleyb | i've never seen a deployment in production here, mostly OVN with some ML2/OVS/DVR because our installer is opinionated | 16:19 |
| opendevreview | Chris Buggy proposed openstack/neutron stable/2023.1: Add rate-limiting to metadata agents https://review.opendev.org/c/openstack/neutron/+/927301 | 16:38 |
| opendevreview | Merged openstack/neutron-vpnaas master: VPNaaS support for OVN: documentation https://review.opendev.org/c/openstack/neutron-vpnaas/+/895651 | 17:34 |
| ihrachys | haleyb: I'm looking at the charts here: https://www.openstack.org/analytics | 17:54 |
| ihrachys | "Which OpenStack Network (Neutron) drivers are you using?" | 17:54 |
| ihrachys | it says 14% Linuxbridge + 16% ML2 - Linuxbridge | 17:54 |
| ihrachys | ovs is 46+41 | 17:54 |
| ihrachys | we don't support linuxbridge for redhat openstack either | 17:55 |
| ihrachys | we won't even support ml2/ovs for the upcoming release. and we deprecated ml2/ovs workloads in the previous (current) release. | 17:56 |
| ihrachys | so from my perspective, unless community steps in, these pieces will naturally die. the last time I checked, redhatters were quite a slice of contributions to neutron. this situation will naturally transform in less attention to ml2/ovs over time, at least for new releases. | 17:57 |
| ihrachys | one explanation as to why people still pick linuxbridge in surveys is that they run really old deployments. whether this should define our actions for main branch is debatable. | 17:58 |
| slaweq | last time when I started talking about deprecation of LB backend there was many users who said there are using it still and wants to use it | 17:59 |
| slaweq | but nobody really wanted to maintain it | 17:59 |
| slaweq | that's why we moved it to be "experimental" so that we don't really guarantee anything and don't need to run any ci jobs with it | 18:00 |
| ihrachys | i hardly see how redhat can justify pulling the driver forward when we don't support it since ages. same applies to canonical. that we do still do seems like a charity, and that's not a great foundation for sustainability. | 18:01 |
| ihrachys | we still run ci jobs with it. unit tests, functional, fullstack. | 18:02 |
| slaweq | I know, it was kind of compromise between maintainers and users who still uses it - we keep it in tree but with no guarantee that it will work, and if any test/job will stop working, we can simply remove it to unblock gate and don't care about it | 18:05 |
| slaweq | I agree that for us would be better to just drop this code from repo but apparently I wasn't "brave" enough when I was PTL :) | 18:06 |
| ihrachys | haleyb: ARE YOU BRAVE ENOUGH | 18:06 |
| ihrachys | I've just checked the election program of our BDFL here https://review.opendev.org/c/openstack/election/+/927043/1/candidates/2025.1/Neutron/haleyb.dev@gmail.com and it seems like a good vote for stability program. which is fine. I wonder though if considering the SLURP coming etc. we should be more aggressively conservative as to what the project should be. | 18:08 |
| haleyb | ihrachys: regarding the analytics, my guess is people chose both the Linuxbridge and ML2 Linuxbridge variants since (even to me) it's confusing to have both, and didn't want to be missed. If that's accurate it means 15% run it | 18:40 |
| haleyb | It's most likely self-managed clouds since we (collectively) don't see customers running it | 18:41 |
| slaweq | there is more confusing answers there IMO and I was trying to change it at some point but it wasn't easy really | 18:41 |
| slaweq | so finally I gave up on that | 18:42 |
| haleyb | Am I brave enough? We could send another email to the list saying it's unmaintained for X years now, and as that progresses our risk increases for having a CVE appear, and noone will fix it (imo) as noone has stepped-up yet | 18:43 |
| haleyb | Is that the next logical step? Marking it deprecated? If that list is accurate that's 15% of users that won't be happy, even if that is the state it's in already | 18:44 |
| haleyb | slaweq: are we just ending a cycle of feedback there? i.e. we'll have new numbers by end of cycle timeframe? | 18:45 |
| haleyb | i literally just got the email, closing tomorrow on user survey | 18:47 |
| slaweq | user survey is closing tomorrow, and I think that we should have anonymized data around october (it was like that last year as far as I checked it) | 18:49 |
| haleyb | ack. using the filter there and choosing Antelope, 58% of deployments were OVN, 21% LB, OVS was somewhere in the middle (OVS vs ML2/OVS) | 18:52 |
| haleyb | if the next one has a | 18:53 |
| haleyb | Caracal option, will those numbers be even higher for OVN? | 18:53 |
| slaweq | unfortunatelly noonedeadpunk just pointed out in the tc channel that this year's survey has only releases up to Antelope to choose :/ | 18:55 |
| slaweq | so there will be no Caracal info until next year's survey for sure | 18:55 |
| haleyb | :( at least most things will be in the ML2 bucket, meaning ML2/OVN, ML2/OVS, ML2/LB, i hope | 18:59 |
| jrosser | the structure of the survey is unhelpful | 19:00 |
| jrosser | I totally missed that chowing ml2 makes another set of options appear right at the bottom of the page | 19:00 |
| jrosser | *choosing | 19:00 |
| ihrachys | haleyb: interesting that lb is on par with ovs. | 19:05 |
| haleyb | ihrachys: by in the middle it might be 40%, i didn't know how to could OVS+ML2/OVS, hopefully the new survey will help with that | 19:09 |
| haleyb | i'm going through the survey again and it's really long | 19:11 |
| haleyb | and you can choose >1 network backend. which is why i was more interested in filtering by latest release - since it gets at the "is anyone using this on master" question | 19:12 |
| opendevreview | Ghanshyam proposed openstack/neutron master: DNM debugging designate job https://review.opendev.org/c/openstack/neutron/+/926945 | 19:39 |
| mlavalle | haleyb: this is the maintenace task https://review.opendev.org/c/openstack/neutron/+/918151 associated to https://review.opendev.org/c/openstack/neutron/+/917800 | 20:05 |
| mlavalle | already merged | 20:05 |
| mlavalle | thanks for the review! | 20:05 |
| opendevreview | Miro Tomaska proposed openstack/neutron stable/2024.1: Fix support of IPv6 only networks in OVN metadata agent https://review.opendev.org/c/openstack/neutron/+/927321 | 20:06 |
| opendevreview | Miro Tomaska proposed openstack/neutron stable/2023.2: Fix support of IPv6 only networks in OVN metadata agent https://review.opendev.org/c/openstack/neutron/+/927322 | 20:07 |
| opendevreview | Miro Tomaska proposed openstack/neutron stable/2023.2: [ovn] Add support for IPv6 metadata https://review.opendev.org/c/openstack/neutron/+/927324 | 20:41 |
| opendevreview | Miro Tomaska proposed openstack/neutron stable/2023.2: [ovn] Add support for IPv6 metadata https://review.opendev.org/c/openstack/neutron/+/927324 | 20:49 |
| opendevreview | Ihar Hrachyshka proposed openstack/neutron master: Introduce ovs-lts jobs that run against the latest ovn LTS branch https://review.opendev.org/c/openstack/neutron/+/927221 | 23:56 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!