| opendevreview | liuyulong proposed openstack/neutron master: Meter flows and ovsdb action for ovs bridge https://review.opendev.org/c/openstack/neutron/+/816800 | 03:35 |
|---|---|---|
| opendevreview | liuyulong proposed openstack/neutron master: Add QoS pps limit rule to support list https://review.opendev.org/c/openstack/neutron/+/816801 | 03:35 |
| opendevreview | liuyulong proposed openstack/neutron master: Support pps limitation for openvswitch agent https://review.opendev.org/c/openstack/neutron/+/816802 | 03:35 |
| opendevreview | Manu B proposed openstack/neutron-dynamic-routing master: BGPaaS server side implementation https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/802445 | 05:29 |
| opendevreview | Manu B proposed openstack/neutron-dynamic-routing master: BGP speaker route handling https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/809173 | 05:35 |
| opendevreview | Manu B proposed openstack/neutron-dynamic-routing master: Updating status, name field and handling reboot operation https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/811300 | 05:43 |
| opendevreview | Manu B proposed openstack/neutron-dynamic-routing master: Updating status, name field and handling reboot operation https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/811300 | 05:49 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: [OVN] Execute OVN migration transactions independently https://review.opendev.org/c/openstack/neutron/+/814181 | 08:25 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Remove not used parameter from migrate_neutron_database_to_ovn function https://review.opendev.org/c/openstack/neutron/+/817162 | 08:25 |
| ralonsoh | slaweq, ^^ I don't understand | 08:27 |
| ralonsoh | I did in https://review.opendev.org/c/openstack/neutron/+/814181/6 | 08:27 |
| opendevreview | liuyulong proposed openstack/neutron master: Meter flows and ovsdb action for ovs bridge https://review.opendev.org/c/openstack/neutron/+/816800 | 09:09 |
| opendevreview | liuyulong proposed openstack/neutron master: Add QoS pps limit rule to support list https://review.opendev.org/c/openstack/neutron/+/816801 | 09:09 |
| opendevreview | liuyulong proposed openstack/neutron master: Support pps limitation for openvswitch agent https://review.opendev.org/c/openstack/neutron/+/816802 | 09:09 |
| *** gthiemon1e is now known as gthiemonge | 09:14 | |
| opendevreview | Lajos Katona proposed openstack/neutron master: Change QosRuleType object version id https://review.opendev.org/c/openstack/neutron/+/817178 | 10:24 |
| opendevreview | Lajos Katona proposed openstack/networking-odl master: CI: Make tempest and rally jobs non-OVN based https://review.opendev.org/c/openstack/networking-odl/+/817186 | 10:48 |
| kevko | hi, is there any issue in neutron wallaby ? when trying to spawn several instances at once via heat ..getting this in nova -> Timeout waiting for [('network-vif-plugged', ...... | 12:58 |
| kevko | ml2/ovs | 12:59 |
| opendevreview | Lajos Katona proposed openstack/networking-odl master: CI: Make tempest and rally jobs non-OVN based https://review.opendev.org/c/openstack/networking-odl/+/817186 | 13:02 |
| opendevreview | Lajos Katona proposed openstack/networking-bgpvpn stable/victoria: [stable/victoria] Dropping lower-constraints testing https://review.opendev.org/c/openstack/networking-bgpvpn/+/816500 | 13:12 |
| opendevreview | Lajos Katona proposed openstack/networking-bgpvpn stable/victoria: [stable/victoria] Dropping lower-constraints testing https://review.opendev.org/c/openstack/networking-bgpvpn/+/816500 | 13:17 |
| lajoskatona | #startmeeting networking | 14:00 |
| opendevmeet | Meeting started Tue Nov 9 14:00:29 2021 UTC and is due to finish in 60 minutes. The chair is lajoskatona. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'networking' | 14:00 |
| lajoskatona | Hi | 14:00 |
| obondarev | hi | 14:00 |
| mlavalle | o/ | 14:00 |
| isabek | Hi | 14:00 |
| bcafarel | hi | 14:00 |
| rubasov | o/ | 14:00 |
| lajoskatona | Let's start | 14:02 |
| lajoskatona | #topic Announcements | 14:02 |
| lajoskatona | Yoga cycle calendar #link https://releases.openstack.org/yoga/schedule.html | 14:02 |
| slaweq | hi | 14:02 |
| lajoskatona | This week is the final week for Ussuri, so if you have anything to merge and release for ussuri please push that | 14:03 |
| amotoki | hi | 14:03 |
| lajoskatona | we have one patch series hanging: https://review.opendev.org/q/I409bc674b65e4f495ebd42d03e97a09d51482339 | 14:03 |
| lajoskatona | Hi slaweq & amotoki :-) | 14:04 |
| bcafarel | yes apart from it it looks good: https://review.opendev.org/q/project:openstack/neutron+branch:stable/ussuri+status:open | 14:04 |
| *** jlibosva is now known as Guest5426 | 14:04 | |
| lajoskatona | bcafarel: thanks | 14:04 |
| lajoskatona | We have to propose last release patch for ussuri ( Iahve to check if isn't there an automatic one already) | 14:05 |
| lajoskatona | and actually that's all announcements from me, do you have anything in mind or questions for announcements? | 14:06 |
| lajoskatona | ohh, no I have few more lines.... | 14:06 |
| lajoskatona | CI meeting time slot: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025578.html please vote, if you a fan of the CI meeting ;) | 14:06 |
| slaweq | ++ | 14:07 |
| lajoskatona | And I sent out a vote for the team meeting as well: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025702.html | 14:07 |
| lajoskatona | and for the drivers meeting: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025705.html | 14:07 |
| lajoskatona | I thought that next week we can discuss the results for the last 2 | 14:07 |
| lajoskatona | slaweq: whan we will have final result for CI meeting slot? | 14:08 |
| slaweq | I though to summarize it this week | 14:08 |
| slaweq | so let's say that deadline is end of this week | 14:08 |
| lajoskatona | slaweq: ok, thanks, | 14:08 |
| lajoskatona | ok, so if nothing more for announcements..... | 14:09 |
| lajoskatona | # topic Community Goals | 14:09 |
| mlavalle | so no meeting times change this week, right? | 14:10 |
| lajoskatona | on this meeting we have't discussed community goals recently, but RBAC was a hot and long topic on the PTG.... | 14:10 |
| lajoskatona | mlavalle: next weeks meeting will be the same time | 14:10 |
| mlavalle | ack | 14:11 |
| lajoskatona | so for communit goals there was a mail from last meeting of TC: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025693.html | 14:11 |
| lajoskatona | and they linked the etherpad for RBAC discussion an etherpad with notes: #link https://etherpad.opendev.org/p/policy-popup-yoga-ptg | 14:11 |
| lajoskatona | to tell the truth I haven't read it in details, so it is more a headsup that there should be finally some outcome of how to handle system scopes and related things..... | 14:12 |
| lajoskatona | Do you have some comments, question for the RBAC and community goal for Yoga cycle? | 14:13 |
| amotoki | perhaps we can focus on improving system-scope related things in neutron side. | 14:14 |
| mlavalle | so hold for now, right? | 14:14 |
| lajoskatona | amotoki: sure, but to tell the truth I lost what is the direction.... | 14:14 |
| amotoki | and we can reflect openstack-wide discussion results to our implementations if needed | 14:14 |
| amotoki | lajoskatona: the last week discussion was mainly focused on how to handle project-specific resources with system-scoped token, but it is still unclear to me.... | 14:15 |
| lajoskatona | amotoki: +1, that makes things easier for sure, and we have painpoints where nova-neutron interaction is in question for example | 14:15 |
| slaweq | amotoki: I wanted to do that but I'm still not sure how new rules should looks like | 14:16 |
| lajoskatona | amotoki: you saved my day, I thought I am the last one who understands half of this whole topic :-) | 14:16 |
| amotoki | slaweq: me neither :( | 14:16 |
| slaweq | IIUC we should have system_scope only for APIs like for agents or logs maybe | 14:17 |
| slaweq | only things which don't really have project_id | 14:17 |
| slaweq | but there should be also some new persona IIUC | 14:17 |
| slaweq | so I would like to change all at once and that's why I didn't moving on with this yet | 14:17 |
| lajoskatona | slaweq, amotoki: thanks for keeping an eye on this topic | 14:18 |
| amotoki | slaweq: my undrestanding is same (though I am not sure I caught the discusion fully) but I agree we should wait till the direction is clear. | 14:19 |
| slaweq | ++ | 14:19 |
| lajoskatona | ok, so as mlavalle said: hold for now, till we have clear direction | 14:20 |
| lajoskatona | ok if nothing more for RBAC and community goals, let's move on | 14:20 |
| lajoskatona | #topic Bugs | 14:21 |
| lajoskatona | bcfarel was bug deputy last week: http://lists.openstack.org/pipermail/openstack-discuss/2021-November/025703.html | 14:21 |
| lajoskatona | From the mail it seems that every bug is handled, bcafarel: do you have anything to add? | 14:22 |
| bcafarel | no, all bugs have good progress or assignee in that week :) | 14:23 |
| lajoskatona | bcafarel: thanks for the report :-) | 14:23 |
| lajoskatona | This week slaweq is the deputy, and next week hongbin will be. | 14:23 |
| slaweq | I'm on it already :) | 14:24 |
| lajoskatona | slaweq: thanks | 14:24 |
| lajoskatona | I will write a mail to hongbin to be sure | 14:24 |
| lajoskatona | ok next topic | 14:24 |
| lajoskatona | #topic L3 subteam | 14:24 |
| lajoskatona | #link https://wiki.openstack.org/wiki/Network/Meetings#L3_subteam | 14:24 |
| lajoskatona | liuyulong collected a lot of blueprints to the wiki, but I can't see him | 14:25 |
| lajoskatona | some of them are already finished, like ECMP support.... | 14:27 |
| lajoskatona | as liuyulong is not here, I think we can move forward | 14:29 |
| lajoskatona | #topic ryu and os-ken | 14:29 |
| lajoskatona | #link https://etherpad.opendev.org/p/make_os-ken_and_ryu_sync | 14:29 |
| slaweq | there is no ralonsoh today so I guess no update | 14:29 |
| lajoskatona | no new patches in ryu this week, so we can close this | 14:29 |
| slaweq | but good news is that we merged all backports from ryu to os-ken last week | 14:29 |
| lajoskatona | yes exactly and we have an os-ken release with them | 14:30 |
| slaweq | yes | 14:30 |
| lajoskatona | #topic On Demand Agenda | 14:30 |
| lajoskatona | I added a topic, but I am more the post man for it: | 14:31 |
| lajoskatona | What to move to neutron-lib? | 14:31 |
| lajoskatona | There is an interesting debate over few patches what code parts/ constants etc to move to n-lib: | 14:31 |
| lajoskatona | https://review.opendev.org/c/openstack/neutron/+/797121 | 14:31 |
| lajoskatona | https://review.opendev.org/c/openstack/neutron/+/797120 | 14:32 |
| lajoskatona | https://review.opendev.org/c/openstack/neutron-lib/+/807224 | 14:32 |
| lajoskatona | I hope I collected all of the relevant patches :-) | 14:32 |
| slaweq | yes, thx | 14:32 |
| slaweq | actually https://review.opendev.org/c/openstack/neutron/+/797120 and https://review.opendev.org/c/openstack/neutron-lib/+/807224 are important in that discussion | 14:32 |
| slaweq | basically I wanted to move all ovs constants to the neutron-lib | 14:33 |
| lajoskatona | liuyulong is on this discussion so sad that we can't have him | 14:33 |
| slaweq | as e.g. some of them were (are) used by networking-bagpipe IIRC | 14:33 |
| slaweq | and all of them are rehomed already | 14:33 |
| slaweq | now liuyulong wants to keep *ALL_TABLES lists in the neutron | 14:33 |
| slaweq | as it may "speed" development process in some cases | 14:34 |
| slaweq | but obondarev had IMO very good point that if we will keep those lists separately, in the future we will forget to add new tables to it | 14:34 |
| slaweq | so it's better to move all to neutron-lib and use it from neutron-lib | 14:34 |
| slaweq | and now I think we need some more votes to judge which approach would be better :) | 14:35 |
| obondarev | slaweq: do you know which constants are used by networking-bagpipe? Maybe we should move only those constants that are known to be used by 2+ projects? | 14:36 |
| lajoskatona | I think this is that comment and discussion under it: https://review.opendev.org/c/openstack/neutron/+/797120/comment/41ae240e_35fcd1b2/ | 14:36 |
| slaweq | yes | 14:37 |
| obondarev | such "common" constants will be saved in lib and used by other projects. Specific constants may stay in specific repos until needed by someone else | 14:38 |
| obondarev | I think that was the workflow for neutron-lib originally | 14:39 |
| lajoskatona | obondarev: +1 | 14:39 |
| amotoki | yeah, that is the principle. in case of OF tables, we need to avoid conflicts among table numbers. I think that's the point. | 14:39 |
| slaweq | yes | 14:40 |
| obondarev | good point | 14:41 |
| mlavalle | +1 | 14:41 |
| slaweq | if You'll have some time, please comment on that patch | 14:42 |
| slaweq | so I can move on with it in one or another way :) | 14:42 |
| slaweq | thx a lot | 14:43 |
| lajoskatona | so let's revisit these changes and follow the "old" rule: move only things that used by multiple projects, otherwise avoid conflicts in places like OF tables. | 14:43 |
| lajoskatona | Can that be the summary? | 14:43 |
| slaweq | yes | 14:43 |
| lajoskatona | ok, than I think we can close this dicussion here, and continue under the reviews, I will point it to ralonsoh and liuyulong to have all of us on the same page | 14:44 |
| slaweq | ++ thx | 14:45 |
| lajoskatona | If no more topics for the On Demand agenda..... | 14:45 |
| dmitriis | looking for a second core review on https://review.opendev.org/c/openstack/neutron-specs/+/788821/ if anybody has cycles for it. | 14:45 |
| dmitriis | Liu looked at it but only gave a +1 so I guess it has to be somebody else | 14:47 |
| lajoskatona | yes, please if you have time to review this spec, check it, as it is again a cross-project effort with Nova and with OVN (out of our control) and nova waits for the neutron-spec review | 14:48 |
| mlavalle | I'll look at it later today | 14:48 |
| dmitriis | tyvm | 14:48 |
| lajoskatona | mlavalle: thanks | 14:48 |
| lajoskatona | if nothing more..... | 14:48 |
| lajoskatona | #endmeeting | 14:49 |
| opendevmeet | Meeting ended Tue Nov 9 14:49:26 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:49 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/networking/2021/networking.2021-11-09-14.00.html | 14:49 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/networking/2021/networking.2021-11-09-14.00.txt | 14:49 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/networking/2021/networking.2021-11-09-14.00.log.html | 14:49 |
| slaweq | o/ | 14:49 |
| lajoskatona | thanks for attending, Bye | 14:49 |
| amotoki | o/ | 14:49 |
| dmitriis | o/ | 14:49 |
| mlavalle | o/ | 14:49 |
| rubasov | o/ | 14:50 |
| rubasov | zigo: I see you asked about v6-only metadata | 14:51 |
| rubasov | we do not have a v6-only tempest test for it, because these tests use huge images and are very costly | 14:52 |
| rubasov | but I just checked manually in my environment and it was working | 14:53 |
| rubasov | we intended to cover the v6-only case and if that's not working that's definitely a bug | 14:53 |
| rubasov | regarding the namespaces: depending on whether your network is an isolated network or one with a router please check both the qrouter and the qdhcp namespaces | 14:54 |
| rubasov | and for a tests sake it may be worth to create a net with both v4 and v6 subnest and check if metadata works over ipv4 in the same environment | 14:55 |
| jrosser_ | some cloud images arent configured to try the v6 metadata service at all | 14:55 |
| rubasov | jrosser_: yes, probably most of them are not using it by default, but the service itself should be available | 14:56 |
| jrosser_ | yeah, we found the metadata service worked but stock images were tricky to use | 14:58 |
| rubasov | I remember that some older OSes (for example ubuntu 16.04) had even problems with general IPv6 link local addresses (though I don't remember the details) | 14:59 |
| slaweq | #startmeeting neutron_ci | 15:00 |
| opendevmeet | Meeting started Tue Nov 9 15:00:11 2021 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'neutron_ci' | 15:00 |
| slaweq | Grafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:00 |
| opendevreview | Merged openstack/neutron master: Bump OVN version for functional job to 21.06 https://review.opendev.org/c/openstack/neutron/+/816614 | 15:00 |
| slaweq | bcafarel: lajoskatona obondarev CI meeting is starting :) | 15:00 |
| obondarev | hi | 15:00 |
| slaweq | hi | 15:00 |
| bcafarel | o/ | 15:00 |
| lajoskatona | Hi | 15:02 |
| slaweq | ok, I think we can start | 15:02 |
| slaweq | this week is just on IRC | 15:02 |
| slaweq | #topic Actions from previous meetings | 15:02 |
| slaweq | slaweq to work on https://bugs.launchpad.net/neutron/+bug/1948832 | 15:03 |
| slaweq | I was checking it today | 15:03 |
| slaweq | and TBH I don't think it's the same bug as mentioned by ralonsoh in the LP | 15:03 |
| slaweq | I wrote my findings in the comment there | 15:04 |
| slaweq | I will try to investigate it little bit more this week | 15:04 |
| slaweq | #action slaweq to check more deeply https://bugs.launchpad.net/neutron/+bug/1948832 | 15:05 |
| slaweq | next one | 15:05 |
| slaweq | ralonsoh to check metadata issue https://1bdefef51603346d84af-53302f911195502b1bb2d87ad2b01ca2.ssl.cf5.rackcdn.com/814807/4/check/neutron-tempest-plugin-scenario-openvswitch/3de2195/testr_results.html | 15:05 |
| slaweq | I guess we need to move that one for next week as ralonsoh is not here today | 15:06 |
| slaweq | #action ralonsoh to check metadata issue https://1bdefef51603346d84af-53302f911195502b1bb2d87ad2b01ca2.ssl.cf5.rackcdn.com/814807/4/check/neutron-tempest-plugin-scenario-openvswitch/3de2195/testr_results.html | 15:07 |
| slaweq | next one | 15:07 |
| slaweq | slaweq to check https://5a5cde44dedb81c8bd48-91d0b9dca863bf6ffc8b1718d062319a.ssl.cf5.rackcdn.com/805391/13/check/neutron-tempest-plugin-scenario-ovn/84283cb/testr_results.html | 15:07 |
| slaweq | It seems for me that the issue is similar to https://launchpad.net/bugs/1892861 | 15:07 |
| slaweq | The problem is that "login: " message in console log appears pretty quickly but cloud-init is still doing some things on the node, so maybe SSH isn't working yet and we hit the same bug. | 15:07 |
| lajoskatona | this happens with cirros? | 15:09 |
| slaweq | no, it's with Ubuntu | 15:09 |
| lajoskatona | ok | 15:09 |
| slaweq | maybe we should find better way to check if guest OS is really booted before doing SSH | 15:09 |
| bcafarel | advanced OS wanting to show that login prompt too fast :) | 15:09 |
| slaweq | it seems like that for me | 15:10 |
| bcafarel | the problem is that sounds OS-dependant? though maybe systemd check for advanced + login: for cirros would be good enough | 15:10 |
| slaweq | because I didn't saw anything else wrong there really | 15:10 |
| slaweq | anyway, I will keep an eye on that. If it will happen more often I will report bug and try to figure out something :) | 15:12 |
| slaweq | for now let's move on to the last one | 15:12 |
| slaweq | lajoskatona to check https://581819ea67919485b97e-6002fae613cad806f99007086c39ea60.ssl.cf2.rackcdn.com/813977/5/gate/neutron-tempest-plugin-scenario-linuxbridge/8fdcf6f/testr_results.html | 15:12 |
| lajoskatona | yeah I checked this one, and the startnge is that the FIP goes UP but too late (~few secs) | 15:12 |
| slaweq | to UP or to DOWN? | 15:13 |
| lajoskatona | slaweq: I mean down, thanks.... | 15:13 |
| slaweq | error message says "attached port status failed to transition to DOWN " | 15:13 |
| slaweq | ahh, ok | 15:13 |
| slaweq | how long we are waiting for that transition? | 15:14 |
| lajoskatona | 120sec | 15:14 |
| obondarev | should be more than enough.. | 15:15 |
| slaweq | yeah | 15:15 |
| lajoskatona | tempest stoppes waiting at: 2021-10-21 08:09:14.583994 | 15:15 |
| lajoskatona | and q-svc reports it's down: Oct 21 08:09:17.015995 ubuntu-focal-iweb-mtl01-0027033075 neutron-server[81347]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ca3c8a33-07ab-41b7-b946-2411e666af30 None None] New status for floating IP aa555045-e872-47f5-a5f4-b4b59017b474: DOWN {{(pid=81347) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} | 15:15 |
| lajoskatona | from https://581819ea67919485b97e-6002fae613cad806f99007086c39ea60.ssl.cf2.rackcdn.com/813977/5/gate/neutron-tempest-plugin-scenario-linuxbridge/8fdcf6f/job-output.txt and https://581819ea67919485b97e-6002fae613cad806f99007086c39ea60.ssl.cf2.rackcdn.com/813977/5/gate/neutron-tempest-plugin-scenario-linuxbridge/8fdcf6f/controller/logs/screen-q-svc.txt | 15:16 |
| slaweq | was L3 agent very busy during that time? | 15:16 |
| lajoskatona | yeah it was refreshing continuusly the iptables rules | 15:16 |
| lajoskatona | my question: do we need ha router enabled for this for example? | 15:17 |
| slaweq | maybe there is some issue in the L3 agent then? | 15:17 |
| lajoskatona | this is a singlenode job as I see | 15:18 |
| slaweq | lajoskatona: we enabled HA for those routers to have at least some HA test coverage | 15:18 |
| slaweq | as we don't have any other jobs with HA routers | 15:18 |
| slaweq | and even if that job is singlenode, when router is HA, all HA codepath is tested | 15:18 |
| slaweq | like keepalived and other stuff | 15:19 |
| slaweq | it's just that router is always transitioned to be primary on that single node :) | 15:19 |
| lajoskatona | slaweq: ok | 15:19 |
| lajoskatona | I can check l3-agent if I can see something why it took so long to make the fip down | 15:20 |
| slaweq | ++ | 15:21 |
| slaweq | #action lajoskatona to check why make FIP down took more than 120 seconds in the L3 agent | 15:21 |
| slaweq | thx lajoskatona | 15:21 |
| slaweq | ok, that are all actions from last week | 15:21 |
| slaweq | let's move on | 15:21 |
| slaweq | #topic Stadium projects | 15:21 |
| lajoskatona | nothing new | 15:22 |
| lajoskatona | This week I realized that odl tempest jobs try to run with ovn | 15:22 |
| lajoskatona | so I fight against that, but the jobs will still fail, but at least we have services up and tempest/rally started | 15:23 |
| lajoskatona | that's it for stadiums | 15:24 |
| slaweq | do You have patch for that? | 15:24 |
| slaweq | I can review it if You want | 15:24 |
| lajoskatona | thanks | 15:24 |
| lajoskatona | it's not fully ready yet by zuul as I see: https://review.opendev.org/c/openstack/networking-odl/+/817186 | 15:24 |
| slaweq | ok | 15:26 |
| lajoskatona | ohh, no tempest is ok at least. It's failing bu with the usual way which is due to ODL is slow behind it.... | 15:26 |
| lajoskatona | ok, so I have to tune rally job | 15:27 |
| lajoskatona | I will send it on IRC when that is ok | 15:27 |
| slaweq | I added it to my review list for tomorrow morning :) | 15:27 |
| lajoskatona | slaweq: thanks | 15:27 |
| slaweq | ok, next topic then | 15:28 |
| slaweq | #topic Stable branches | 15:28 |
| bcafarel | overall good this week, as mentioned ussuri has only https://review.opendev.org/c/openstack/neutron/+/816661 left (before EM transition) | 15:28 |
| bcafarel | though functional test failing in it looks related | 15:29 |
| bcafarel | and I found just before meeting train has neutron-tempest-plugin-designate-scenario-train failing :( https://zuul.opendev.org/t/openstack/build/a6a1142368b742248be710f902f541f5 | 15:29 |
| slaweq | yes, indeed | 15:29 |
| slaweq | I will check it tomorrow | 15:29 |
| bcafarel | I will fill a bug for that designate train one after meetings | 15:29 |
| bcafarel | but at least full support branches are good :) | 15:30 |
| slaweq | ok, thx bcafarel | 15:30 |
| slaweq | I think we can move on then | 15:30 |
| slaweq | #topic Grafana | 15:30 |
| slaweq | http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 15:31 |
| slaweq | rally was broken last week | 15:31 |
| slaweq | but it's I guess due to that missing service endpoint in keystone, right? | 15:32 |
| slaweq | and it's already fixed | 15:32 |
| lajoskatona | yes | 15:32 |
| lajoskatona | actually it's funny, in devstack the endpoint creation was deleted and now we create it from the job | 15:32 |
| lajoskatona | and seems like neutron runs mostly as voting job rally | 15:33 |
| slaweq | so only rally job needs it? | 15:33 |
| lajoskatona | yes as I remember | 15:33 |
| slaweq | hmm, ok | 15:34 |
| slaweq | thx for fixing that issue quickly | 15:34 |
| lajoskatona | and the proper fix should be in rally, but I checked and not clear for first view where to fix it..... | 15:34 |
| lajoskatona | it was ralonsoh actually, I just run another "alternative" path, but that was not enough.... | 15:34 |
| slaweq | :) | 15:34 |
| slaweq | from other things I see that neutron-tempest-plugin-scenario-linuxbridge job is failing pretty often (more than other similar jobs) | 15:35 |
| slaweq | I'm not sure why it is like that really | 15:35 |
| slaweq | probably some timeouts which I saw pretty often in the scenario jobs last week | 15:35 |
| slaweq | like e.g. https://zuul.opendev.org/t/openstack/build/7a2bc299305249c5911e7107bb4d4a37 | 15:36 |
| slaweq | I think we need to investigate why so often our tests are so slow recently | 15:38 |
| slaweq | I doubt I will have time for that this week but maybe there is anyone who could check that | 15:39 |
| obondarev | also neutron-tempest-plugin-scenario-ovn started failing I think | 15:40 |
| obondarev | example: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_330/807116/23/check/neutron-tempest-plugin-scenario-ovn/330ce5c/testr_results.html | 15:40 |
| slaweq | obondarev: You mean with timeouts? | 15:40 |
| obondarev | nope | 15:40 |
| obondarev | a couple of tests failing with ssh I believe | 15:40 |
| slaweq | hmm, in those 2 tests for example we are missing console log from instance | 15:41 |
| slaweq | so it's hard to say | 15:41 |
| slaweq | but it may be the same issue like in https://5a5cde44dedb81c8bd48-91d0b9dca863bf6ffc8b1718d062319a.ssl.cf5.rackcdn.com/805391/13/check/neutron-tempest-plugin-scenario-ovn/84283cb/testr_results.html | 15:41 |
| slaweq | which I spoke about at the begining of the meeting | 15:42 |
| slaweq | let's observe it and if someone will have some time, maybe investigate it a bit more :) | 15:44 |
| obondarev | I'll keep an eye on it and report a bug if it continue to fail with similar symptoms | 15:44 |
| slaweq | obondarev: sounds good, thx | 15:44 |
| slaweq | ok, I think we can move on quickly | 15:44 |
| slaweq | we already spoke about most urgent issues in our jobs, at least those which I had prepared for today | 15:45 |
| slaweq | #topic tempest/scenario | 15:45 |
| slaweq | here I have one more thing to mention | 15:45 |
| slaweq | this time, good news I think | 15:45 |
| slaweq | our CI job found bug :) | 15:45 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1950273 | 15:45 |
| slaweq | so sometimes those jobs seems to be useful ;) | 15:46 |
| obondarev | nice! :) | 15:46 |
| slaweq | it seems to be some race condition or something else but even in such case we shouldn't return error 500 to the user | 15:46 |
| lajoskatona | +1 | 15:47 |
| slaweq | so at least we should properly handle that error | 15:47 |
| slaweq | that's why I marked it as "Higt" | 15:47 |
| slaweq | *High | 15:47 |
| slaweq | ok, and last topic from me for today | 15:47 |
| slaweq | #topic Periodic | 15:47 |
| slaweq | here all except UT with neutron-lib master looks good | 15:48 |
| slaweq | I opened bug https://bugs.launchpad.net/neutron/+bug/1950275 today | 15:48 |
| lajoskatona | For that I pushed a patch: https://review.opendev.org/c/openstack/neutron/+/817178 | 15:48 |
| slaweq | thx lajoskatona | 15:49 |
| slaweq | but my question is: why some change in neutron-lib caused that failure? | 15:49 |
| slaweq | as OVO object is defined in the Neutron repo, isn't it? | 15:49 |
| lajoskatona | but that breaks unit test with n-lib 2.16.0, and what I can't consume: | 15:49 |
| obondarev | I believe order of constants on which OVO depends - changed | 15:49 |
| lajoskatona | seems like this patch brings the failure: https://review.opendev.org/c/openstack/neutron-lib/+/816447 | 15:49 |
| lajoskatona | yeah, possible, but that can change the hash for ovo? | 15:50 |
| obondarev | I think we faced similar issues in the past | 15:50 |
| lajoskatona | so If I understand it we have to release n-lib with constants change and bump n-lib for neutron? | 15:51 |
| obondarev | I think so too | 15:51 |
| lajoskatona | ok | 15:51 |
| slaweq | but the problem may be that in the release patches it runs some UT job for neutron, no? | 15:51 |
| slaweq | and this job will fail then | 15:51 |
| slaweq | or am I missing something? | 15:52 |
| lajoskatona | that's possible | 15:52 |
| lajoskatona | I push release patch and that will tell us if this way can work | 15:53 |
| slaweq | ++ | 15:53 |
| slaweq | maybe I'm wrong here | 15:54 |
| slaweq | in the releases it don't runs UT probably | 15:54 |
| slaweq | I hope so at least :) | 15:54 |
| slaweq | ok, so that's all what I had for today | 15:54 |
| slaweq | anything else You want to discuss regarding CI? | 15:54 |
| obondarev | lajoskatona: I would appreciate if you check https://review.opendev.org/c/openstack/neutron-lib/+/816468 before lib release :) | 15:54 |
| lajoskatona | obondarev: I will | 15:55 |
| obondarev | lajoskatona:thanks a lot! | 15:55 |
| slaweq | ok, so I think we can finish our meeting for today | 15:56 |
| slaweq | have a great week and see You all next week on the video call again :) | 15:56 |
| slaweq | o/ | 15:56 |
| slaweq | #endmeeting | 15:56 |
| opendevmeet | Meeting ended Tue Nov 9 15:56:58 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:56 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-11-09-15.00.html | 15:56 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-11-09-15.00.txt | 15:56 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-11-09-15.00.log.html | 15:56 |
| lajoskatona | o/ | 15:57 |
| obondarev | o/ | 15:57 |
| opendevreview | Merged openstack/neutron master: Revert "[Fullstack] Mark TestHAL3Agent fip_qos test as unstable" https://review.opendev.org/c/openstack/neutron/+/815235 | 16:06 |
| opendevreview | Merged openstack/neutron stable/xena: [DVR] Fix update of the MTU in the DVR HA routers https://review.opendev.org/c/openstack/neutron/+/816658 | 16:06 |
| opendevreview | Slawek Kaplonski proposed openstack/neutron master: Add functional and fullstack jobs with FIPS enabled https://review.opendev.org/c/openstack/neutron/+/814009 | 16:10 |
| opendevreview | Merged openstack/neutron master: Remove todo's in Y release https://review.opendev.org/c/openstack/neutron/+/815853 | 16:11 |
| kevko | slaweq: I've read today meeting, and maybe I have different issue ..but my openstack wallaby is somehow slow when starting instances .. | 17:16 |
| kevko | when I run heat stack ..sometimes it is in minute ..sometimes it is running and running and completed after several minutes .. | 17:17 |
| kevko | it looks like sometimes event about vif plugged from neutron is sent and received by nova...and in other cases I am waiting for event for a long time .. | 17:18 |
| kevko | could you please advice me ? | 17:18 |
| slaweq | kevko: You should check when port is transitioned to be up. And what is preventing that transition. Usually it's L2 or DHCP agent | 17:20 |
| slaweq | Sorry but now I'm off for the day so i will not be able to response to your questions | 17:21 |
| opendevreview | Lajos Katona proposed openstack/networking-odl master: CI: Make tempest and rally jobs non-OVN based https://review.opendev.org/c/openstack/networking-odl/+/817186 | 17:21 |
| kevko | slaweq: ok, thanks, just quick question -> 2021-11-09 16:40:42.749 8 ERROR neutron.agent.dhcp.agent [-] Unexpected number of DHCP interfaces for metadata proxy, expected 1, got 2 | 17:22 |
| kevko | this is normal ? | 17:22 |
| slaweq | Nope. I don't think it's normal | 17:23 |
| kevko | hmm | 17:23 |
| slaweq | But i don't know why it's like that | 17:23 |
| slaweq | I would need to look into code where that error comes from | 17:24 |
| opendevreview | Lajos Katona proposed openstack/networking-odl master: CI: Make tempest and rally jobs non-OVN based https://review.opendev.org/c/openstack/networking-odl/+/817186 | 19:15 |
| opendevreview | Seena Fallah proposed openstack/ovsdbapp master: ovn-sb: remove chassis from Chassis_Private on ChassisDelCommand https://review.opendev.org/c/openstack/ovsdbapp/+/817300 | 19:51 |
| opendevreview | Seena Fallah proposed openstack/ovsdbapp master: ovn-sb: remove chassis from Chassis_Private on ChassisDelCommand https://review.opendev.org/c/openstack/ovsdbapp/+/817300 | 21:35 |
| opendevreview | Merged openstack/neutron-specs master: Off-path SmartNIC Port Binding with OVN https://review.opendev.org/c/openstack/neutron-specs/+/788821 | 21:57 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!