Tuesday, 2021-08-24

*** yoctozepto8 is now known as yoctozepto00:33
opendevreviewMerged openstack/neutron-lib master: Do not use deprecated attributes of RequestContext  https://review.opendev.org/c/openstack/neutron-lib/+/80548800:53
*** yoctozepto6 is now known as yoctozepto01:21
*** yoctozepto3 is now known as yoctozepto02:12
*** yoctozepto8 is now known as yoctozepto03:43
*** yoctozepto8 is now known as yoctozepto03:57
*** yoctozepto0 is now known as yoctozepto04:12
*** yoctozepto1 is now known as yoctozepto04:40
*** yoctozepto7 is now known as yoctozepto05:43
opendevreviewOleg Bondarev proposed openstack/neutron-lib master: Add Local IP API def  https://review.opendev.org/c/openstack/neutron-lib/+/80305106:15
*** yoctozepto0 is now known as yoctozepto06:24
*** redrobot0 is now known as redrobot06:29
*** yoctozepto9 is now known as yoctozepto07:00
opendevreviewShawn.Lu proposed openstack/neutron stable/wallaby: Do not call delete segement when segment plugin disabled  https://review.opendev.org/c/openstack/neutron/+/80578807:25
*** rpittau|afk is now known as rpittau07:27
opendevreviewPrzemyslaw Szczerbik proposed openstack/neutron master: Fix gate for neutron-lib v2.14  https://review.opendev.org/c/openstack/neutron/+/80562707:31
*** yoctozepto1 is now known as yoctozepto07:52
opendevreviewJavier Peña proposed openstack/neutron stable/wallaby: Replace assertItemsEqual with assertCountEqual  https://review.opendev.org/c/openstack/neutron/+/80579108:06
opendevreviewJavier Peña proposed openstack/neutron stable/wallaby: Follow up for replacing assertItemsEqual  https://review.opendev.org/c/openstack/neutron/+/80579208:06
opendevreviewRodolfo Alonso proposed openstack/neutron master: [OVS][FW] Initialize ConjIdMap._max_id depending on the current OFs  https://review.opendev.org/c/openstack/neutron/+/80423608:13
opendevreviewPrzemyslaw Szczerbik proposed openstack/neutron master: Fix gate for neutron-lib v2.14  https://review.opendev.org/c/openstack/neutron/+/80562708:39
slaweqjlibosva: hi, did You maybe had any chance to look again at https://bugs.launchpad.net/neutron/+bug/1938766 ?08:53
* jlibosva looks08:53
slaweqamotoki added some comment to that bug recently08:53
slaweqand this is most common issue in functional tests job now :/08:53
lajoskatonalast week it was easy to reproduce locally, but I had no idea other than random tries08:56
jlibosvaslaweq: would it be a solution to fallback to older OVS?08:58
opendevreviewPrzemyslaw Szczerbik proposed openstack/neutron master: Bump os-resource-classes lib to 1.1.0  https://review.opendev.org/c/openstack/neutron/+/80106609:01
slaweqjlibosva: we can try I think but that's short term workaround, right?09:03
jlibosvaslaweq: yes, I think the fix should go to OVS09:03
slaweqjlibosva: do You know if that issue is reported for OVS somewhere?09:04
slaweqso we can add note about it in job's definition09:04
jlibosvaslaweq: yes, it's reported in RH BZ and the team is working on it09:04
slaweqjlibosva great, so can You propose patch or do You want me to do it?09:04
jlibosvaslaweq: I can try to reproduce locally and figure out what version we need. The patch I found was quite old and doesn't correlate with the time we started to see the issue09:05
slaweqjlibosva: ok, thx a lot09:05
opendevreviewEduardo Olivares proposed openstack/neutron-tempest-plugin master: Checking IP version from extra-dhcp-options  https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/80475509:07
opendevreviewBernard Cafarelli proposed openstack/neutron stable/wallaby: Replace assertItemsEqual with assertCountEqual  https://review.opendev.org/c/openstack/neutron/+/80579110:01
opendevreviewRodolfo Alonso proposed openstack/neutron-specs master: Create intermediate OVS bridge to improve live-migration in OVN  https://review.opendev.org/c/openstack/neutron-specs/+/79919810:07
opendevreviewMark Goddard proposed openstack/neutron stable/wallaby: Add wait for the post-fork event to nb/sb objects  https://review.opendev.org/c/openstack/neutron/+/80576810:13
opendevreviewMark Goddard proposed openstack/neutron stable/wallaby: Add wait for the post-fork event to nb/sb objects  https://review.opendev.org/c/openstack/neutron/+/80576810:14
opendevreviewMark Goddard proposed openstack/neutron stable/victoria: Add wait for the post-fork event to nb/sb objects  https://review.opendev.org/c/openstack/neutron/+/80576910:18
JohnnyWHi is there a way to speed up routers to propagate on neutron, after restart of neutron-l3-agent? I see that first routers propagate quickly, but after 50-60 routers it is really slowing down(and I have around 250 routers to migrate). I couldn't find anything in logs, so I'm not sure what is a reason here. Any advice what can be checked or? Thanks10:23
JohnnyWin advance!10:23
kklimondahmm, with ussuri ml2/ovn it seems wsgi workers are responsible for processing updates sent by ovsdb servers?11:04
kklimondaI'm seeing a problem where the amount of changes in southbound db (chassis_private) seems to be blocking wsgi workers and affect api11:04
ralonsohkklimonda, wsgi server (neutron API) creates an IDL connection to OVN SB and NB. So yes, the neutron API process all OVN updates (as there are not Openstack compute agents)11:05
ralonsohwhat is happening?11:06
kklimondaI'm probably not running enough api_workers due to it being a virtualized environment that ran out of cpu, but I'm curious if there is a scaling issue here - does hash ring implementation split ovn updates between api workers?11:06
kklimondaI have 250 chassis in this deployment11:07
ralonsohyes, all requests are attended by a single API worker11:07
ralonsohin parallel 11:07
ralonsohwhat operation is blocking the neutron server?11:07
ralonsohthere should not be so many changes in chassis_private11:08
kklimondawhy, every change to northbound db turns into 250 updates to chassis_private11:10
ralonsohhow11:11
kklimondagive me a sec11:11
kklimondasorry, just got a quick call to end11:11
kklimondawhen neutron makes change to northbound db it updates nb_cfg field11:12
kklimondaovn-controller then updates chassis_private setting nb_cfg there11:14
kklimondaand this turns into an update that has to be processed by neutron - that's probably because I'm running with https://review.opendev.org/c/openstack/neutron/+/752795/ backpored11:14
ralonsohnb_cfg is updated when the chassis is updated. If a change in NB implies a change in a chassis, that will be needee11:19
ralonsohwhat I don't understand is that the NB changes implies 250 updates in nb_cfg field11:19
ralonsohif you have an example, that could be useful11:19
opendevreviewPrzemyslaw Szczerbik proposed openstack/neutron master: Fix gate for neutron-lib v2.14  https://review.opendev.org/c/openstack/neutron/+/80562711:25
ralonsoh^^^ folks, we need this n-lib patch11:28
obondarevralonsoh: can you please re-post the link again? Just connected11:30
ralonsohobondarev, https://review.opendev.org/c/openstack/neutron/+/80562711:30
obondarevthanks11:31
kklimondaralonsoh: ok, so this is how I understand this:11:32
kklimondaralonsoh: when neutron makes a change to ovn it increments nb_cfg field in NB_Global11:33
ralonsohsame as in sb.SB_Global.sb_cfg11:35
ralonsohbut this is not the chassis_private.nb_cfg11:36
kklimondaralonsoh: this change is processed via ovn-northd into SB_Global.nb_cfg11:37
ralonsohyes, but this is not chassis_private.nb_cfg11:38
ralonsohI don't think we have an event for SB_Global11:38
ralonsohlet me check11:38
kklimondaovn-controller then updates it's entry in Chassis_Private setting nb_cfg columnt11:38
ralonsohyes, the corresponding one11:39
kklimondathis is with https://review.opendev.org/c/openstack/neutron/+/752795/ applied that changes how agent liveness is handled11:39
ralonsohhmmm in https://review.opendev.org/c/openstack/neutron/+/752795/29/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py we capture any SB_Global event11:40
ralonsohthen we filter, but that implies we first attend to the change in SB_Global11:40
ralonsohyeah, that could be too much11:40
ralonsohare you going to open a LP bug?11:41
kklimondasure11:41
ralonsohthanks a lot11:41
ralonsohkklimonda, please, refer to patch https://review.opendev.org/c/openstack/neutron/+/752795/29/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovsdb_monitor.py#24911:41
ralonsohand this file11:41
opendevreviewBernard Cafarelli proposed openstack/neutron stable/wallaby: Follow up for replacing assertItemsEqual  https://review.opendev.org/c/openstack/neutron/+/80579212:02
opendevreviewLajos Katona proposed openstack/neutron-lib master: Fix hostroute validation with BFD  https://review.opendev.org/c/openstack/neutron-lib/+/80583412:03
kklimondaralonsoh: https://bugs.launchpad.net/neutron/+bug/194095012:09
opendevreviewMerged openstack/neutron-lib master: Add network_ip_availability pagging and sorting support  https://review.opendev.org/c/openstack/neutron-lib/+/80305012:39
opendevreviewOleg Bondarev proposed openstack/neutron master: [WIP] Add Local IP Extension and DB  https://review.opendev.org/c/openstack/neutron/+/80452313:00
opendevreviewRodolfo Alonso proposed openstack/neutron master: Do not fail when releasing a quota reservation  https://review.opendev.org/c/openstack/neutron/+/80503113:03
opendevreviewRodolfo Alonso proposed openstack/neutron master: Replace "tenant_id" with "project_id" in Quota engine  https://review.opendev.org/c/openstack/neutron/+/80584913:25
slaweq#startmeeting networking14:00
opendevmeetMeeting started Tue Aug 24 14:00:04 2021 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
opendevmeetThe meeting name has been set to 'networking'14:00
mlavalleo/14:00
ralonsohhi14:00
slaweqhi14:00
gansohi14:00
lajoskatonaHi14:00
bcafarelo/14:00
obondarevhi14:01
amotokio/14:01
slaweqlet's wait 2 more minutes for others14:01
slaweqand we will start14:01
slaweqok, let's go14:02
slaweq#topic Announcements14:02
slaweqfirst of all Xena cycle calendar https://releases.openstack.org/xena/schedule.html14:02
slaweqlast week we released final releases for non-client libraries14:02
slaweqbut I think I made some mistake when I was checking calendar14:03
thomasb06hi14:03
manubkhi14:03
slaweqand deadline for non-client libs is this week really :)14:03
slaweqI really don't know how I made that mistake14:04
lajoskatonathere was  a mail about it: http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024291.html14:04
lajoskatonaso it was an openstack global miscommunication :-)14:04
slaweqlajoskatona thx14:04
slaweqI missed that email as on Friday I was off14:04
slaweqso it's not that bad with me at least :)14:04
mlavallelol14:05
slaweqso, if You have any last minute think which You would like to include in Xena e.g. in neutron-lib, please ping me - we can make another release if needed14:05
slaweqotherwise we are good with those libs already :)14:05
slaweqit's better to made mistake that way than in the other one :D14:06
lajoskatona+114:06
slaweqnext week we will have Xena-3 milestone and final release for client libraries14:07
slaweqso we will need to cut python-neutronclient but I don't think there is anything urgent waiting for review there14:07
slaweqif there is something important for You, please ping me about it on irc or by email14:07
slaweqok, moving on to the next announcement14:08
slaweqTC & PTL Nominations ends today: https://governance.openstack.org/election/14:08
slaweqwe have great candidate already - thx lajoskatona for stepping in :)14:08
ralonsoh+114:09
mlavalle+114:09
obondarev+114:09
slaweqso it's probably that next week we will have new PTL already :)14:09
lajoskatonahope that can't destroy years work in half a year....14:09
ralonsohI'll help you to do it (to destroy, I mean)14:09
slaweqlajoskatona I'm sure You will do great :)14:09
slaweqralonsoh: LOL14:10
bcafarelI'm not worried either :)14:10
obondarevlajoskatona: didn't see that in your goals! :D14:10
amotokilet's break our bad things and improve them :)14:10
lajoskatona+114:11
slaweqnext one14:11
slaweqOctober PTG14:12
slaweqetherpad https://etherpad.opendev.org/p/neutron-yoga-ptg14:12
slaweqPlease add Your topics there14:12
slaweqOperator pain points etherpad https://etherpad.opendev.org/p/pain-point-elimination14:12
slaweqplease add Yours if You have any14:12
slaweqand that's are all announcements/reminders which I had for You14:13
slaweqanything else You want to add in that section?14:13
slaweqif not, let's move on to the next topic14:15
slaweq#topic Blueprints14:15
slaweqfor Xena-3 we still have those BPs https://bugs.launchpad.net/neutron/+milestone/xena-314:15
slaweqI don't think we will be able to complete any of them in this cycle really14:15
slaweqfor BGP related stuff I updated status to "good progress" today as all of them have at least api-ref merged already14:16
slaweqbut there's no neutron implementation for any of them now14:16
slaweqnext week I will probably move them to neutron-next14:16
slaweqas I don't think we need to schedule any of them for the Xena RC milestone14:16
slaweqare You ok with that?14:17
lajoskatonafor me ok14:18
ralonsoh+114:18
mlavalle+114:19
slaweqthx14:19
slaweqso, I think we can move on14:20
slaweqto the next topic14:20
slaweq#topic Bugs14:20
slaweqI was bug deputy last week. Report http://lists.openstack.org/pipermail/openstack-discuss/2021-August/024308.html14:20
slaweqthere is couple of bugs which I wanted to raise today14:20
slaweqhttps://bugs.launchpad.net/neutron/+bug/1940071 - that is vpnaas issue14:20
slaweqseems like some memory leak in vpnaas14:21
slaweqmaybe someone who is more familiar with vpnaas could take a look14:21
ralonsohI wish this could help: https://review.opendev.org/c/openstack/neutron/+/80303414:22
slaweqralonsoh yes, hopefully it will :)14:22
slaweqnext one14:23
slaweqhttps://bugs.launchpad.net/neutron/+bug/1940425 - that is gate failure, not very often but I think it's worth to take a look14:24
ralonsohcould be related to a bug we have in RH14:25
ralonsohthis is related to the RPC timeout14:25
ralonsohI'll check the logs14:25
slaweqralonsoh thx14:25
ralonsoh(in a nutshell, the parent port activation waits until all subports are bound)14:25
slaweqnext we have 2 low-hanging-fruits (IMO):14:26
slaweqhttps://bugs.launchpad.net/neutron/+bug/194007414:26
slaweqhttps://bugs.launchpad.net/neutron/+bug/194007314:26
slaweqfor https://bugs.launchpad.net/neutron/+bug/1940074 there is actually patch already14:26
slaweqso only https://bugs.launchpad.net/neutron/+bug/1940073 is still free to take :)14:27
slaweqand the last one: https://bugs.launchpad.net/neutron/+bug/1940086 - that is api-ref thing14:27
slaweqmaybe someone wants to propose update there14:28
slaweqall other bugs from last week are already assigned to someone14:28
slaweqany other bugs You want to discuss today?14:29
slaweqok, I guess that this means "no"14:30
slaweqthis week bug deputy is hongbin14:31
slaweqand I already asked him last week about it14:31
slaweqhe confirmed me that he will do it14:31
slaweqnext week will be haleyb14:32
slaweqI will ask him if he will be able to do it14:32
slaweqand that's basically all what I have for today14:32
slaweq#topic On Demand Agenda14:32
slaweqdo You have any other topics to discuss today?14:33
gansoI do14:33
gansosorry I updated the agenda after the meeting had started14:33
slaweqganso go on then :)14:33
gansothanks14:33
opendevreviewRodolfo Alonso proposed openstack/neutron master: Replace "tenant_id" with "project_id" in Quota engine  https://review.opendev.org/c/openstack/neutron/+/80584914:33
gansoI'm interested in implementing pagination across some pages in horizon, main network list14:34
gansobut also routers, SGs, FIPs14:34
gansoone thing that drew my attention about network list is that the horizon network list is different than the CLI list14:34
gansobecause of this: https://github.com/openstack/horizon/blob/1800750804502adf9ff31366daa987aeb9acba31/openstack_dashboard/api/neutron.py#L107514:34
gansobasically CLI list lists everything, even across tenants, if the user has those privileges14:35
gansothe horizon one doesn't, it filters by tenants, and by doing that it causes 2 things14:35
ganso1) the initial query filters out shared and external (especially when those are not created by the tenant listing the networks)14:35
ganso2) It makes it very complicated ( impossible perhaps) to paginate this consistently, because what it does today it performs other separate queries to integrate the shared and external networks in the list14:36
gansosince we don't know how many external / shared ones there are beforehand, we would need to do very hacky things to paginate14:37
gansoso, I would like to ask: why do we need horizon to be different than the CLI? can't we make them behave similarly and get rid of this tenant filter?14:37
amotokiit is due to the nature of horizon "project" panel. The project panel focuses on operations owned by a target tenant.14:38
amotoki"admin" users can list all networks (or resources) from all projects. It is confuisng as the "project" panel.14:38
amotokithis is the reason I added such hacky thing you explained.14:38
slaweqpersonally from Neutron pov I don't see any reason why it shouldn't be the same14:38
slaweqit's more question to the Horizon team IMO14:39
slaweqso amotoki is the best one to answer :)14:39
gansoamotoki: looking at the other side of this, the neutron CLI is the only one that does this, when compared to nova and cinder. The Horizon way of doing is actually more consistent with how nova and cinder do14:39
gansonova and cinder's CLI does not list resources of other tenants, despite the user being admin, unless you specify "--all-projects"14:40
amotoki"CLI" can be reworeed with "API" as neturon CLI does not filter anything.14:40
amotokicompared to the nova/cinder API, the neutron API behavior differently.14:40
gansoyes14:40
amotokiin case of nova/clnder, even though an API consumer has "admin"-ness, nova/cinder API returns resources owned by a target project.14:41
amotokithis is the different point from neutron.14:41
gansoI see the neutron API behavior as the official and desired one by its community, so I wouldn't say we would  have to change how the API behaves. Horizon on the other hand, could do the same as the API instead of doing something different14:41
opendevreviewBernard Cafarelli proposed openstack/neutron-tempest-plugin master: Check if advanced image flavor already exists  https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/80586414:42
amotokiwhen nova/cinder/neutron API were designed there was no consensus on how API should work when called with "admin"-ness. this leads to different behaviors between nova/cinder and neutron APIs.14:42
gansoso, how do we move forward from here? I understand Horizon's premise of being different, but do we all agree with this? is this better than not having pagination? My main motivation is because customers are having timeouts listing networks while loading the network list page14:43
amotokihorizon implementation is just to handle these differences. Basically horizon "project" panel behavior assumes nova/cinder API behaviors and the logic for neutron APi is to handle it.14:43
amotokiganso: I don't have a clear answer right now. 14:44
amotokiat least changing the existing API behaviors in nova/cinder/neutron is really confusing.14:45
obondarevcan we add a parameter to API, that horizon can use to get similar to nova/cinder bahavior?14:46
slaweqjust an idea: maybe we could add some flag in neutron API, something like "include_shared" so user could do call like "network list --tenant-id <tenant> --include-shared True" to get all shared and own networks in one request14:46
amotokiit is one possible idea but on the other hand it is not a good thing to change the API behavior beased on configurations.14:47
obondarevnot config14:47
amotokiah, I misunderstood.14:47
slaweqit's API parameter14:47
slaweqnot config14:47
amotokiit is about "API parameter"14:47
slaweqso horizon could use it14:48
gansoalternatively add a "--all-projects" that default to True (defaulting to True is weird, but trying to maintain the same behavior)14:48
gansothen horizon would say "--all-projects" false14:48
slaweqthat's also some possibility14:49
slaweqfor sure we will need RFE for that14:49
slaweqand we will need to discuss it carefully in drivers meeting :)14:50
slaweqganso would that be ok to propose such RFE for Neutron?14:50
slaweqamotoki and would it be ok for You to work with ganso on that RFE? You are the best guy in our team to help with that :)14:51
amotokislaweq: yes14:51
gansoLet me ask something on the other extreme first, as I assume we would all like to avoid making API changes for this. If I am able to implement something hacky that accomplishes pagination despite being inefficient, would that be acceptable?14:51
slaweqin Neutron or Horizon You mean?14:52
gansoHorizon14:52
gansoonly touching Horizon14:52
slaweqthat's question to Horizon team I guess :)14:52
amotokihorizon is also studying the support of system-scope and the situation may change as API reuqests with a regular user no longer have admin-ness (after switching to system-scope token)14:53
gansooh ok. And the question of if that would be backportable (given that pagination is perhaps a new feature, but addressing the lack of pagination as being a bug), is also up to Horizon team?14:53
slaweqganso all changes in Horizon are up to Horizon team14:54
gansook14:54
slaweqfrom us here only amotoki is part of Horizon team also14:54
amotokiganso: yes. it is up to horizon team but generally speaking it depends on how big the change is. the stable policy is not break a released deliverable.14:54
gansoI will spend some more time on that and see what I can come up with, now knowing that the alternative route is through adding parameters to Neutron API14:54
slaweqganso that's some possiblitity which we may explore more for sure14:55
amotokiganso: thanks for raising this. I am glad to help14:55
gansook, thanks! that's all I had, I will keep in touch with amotoki and slaweq. =)14:56
slaweqthx ganso14:56
slaweqany other last minute topics?14:56
amotokifor long term, it may be a good thing to achieve consistency between nova/cinder/neutron APi behaviors.14:57
amotokiit would not happen in short term, but it is worth considering it as long term14:57
amotokino more from me :p14:57
slaweqamotoki thx a lot14:57
slaweqso I think we are good to go today14:58
slaweqthx for attending the meeting and see You next week (hopefully with our new great PTL already :))14:58
slaweqo/14:58
slaweq#endmeeting14:58
opendevmeetMeeting ended Tue Aug 24 14:58:39 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:58
opendevmeetMinutes:        https://meetings.opendev.org/meetings/networking/2021/networking.2021-08-24-14.00.html14:58
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/networking/2021/networking.2021-08-24-14.00.txt14:58
opendevmeetLog:            https://meetings.opendev.org/meetings/networking/2021/networking.2021-08-24-14.00.log.html14:58
mlavalleo/14:58
amotokio/14:58
ralonsohbye14:58
thomasb06 bye14:58
lajoskatonabye14:59
slaweq#startmeeting neutron_ci15:00
opendevmeetMeeting started Tue Aug 24 15:00:06 2021 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'neutron_ci'15:00
slaweqhi (again) :)15:00
obondarevhi15:00
ralonsohhi15:00
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:00
bcafarelthat was a short break :)15:00
thomasb06hi15:00
lajoskatonaHi15:00
slaweqlet's do it quick as ralonsoh needs to leave early today :)15:01
slaweq#topic Actions from previous meetings15:01
slaweqobondarev to update grafana after promoting dvr-ha job to be voting15:01
obondarevhttps://review.opendev.org/c/openstack/project-config/+/80559415:01
obondarevnot sure if it's all needed however15:01
obondarevI mean that I might missed something15:02
slaweqthx obondarev15:02
slaweqI think it's ok15:02
bcafarelfor graph itself probably not, but it is nice to have proper labels :)15:02
slaweqnext one15:03
slaweqralonsoh to send patches to use smaller flavors and concurency 1 for advanced scenario tests15:03
ralonsohand bcafarel 15:03
ralonsohhttps://review.opendev.org/q/79c679683dd93c1df412f81533af6bf72e2395d015:03
ralonsohfirst one is merged15:03
ralonsohplease check bcafarel's one15:03
ralonsohI think with this patch we can close the bug (for now)15:03
ralonsohthe concurrency patch won't be needed15:04
slaweqlooks good, thx bcafarel and ralonsoh15:04
slaweqso, next one15:04
slaweqralonsoh to check UT failing with neutron-lib master15:04
ralonsohdone, one sec15:05
ralonsohhttps://review.opendev.org/c/openstack/neutron-lib/+/80539515:05
ralonsohand merged15:05
ralonsohnonono15:05
lajoskatonasalweq: You reported another one today: https://bugs.launchpad.net/neutron/+bug/194090515:05
ralonsohhttps://review.opendev.org/c/openstack/neutron-lib/+/80489415:05
ralonsoh^^ this patch15:05
lajoskatonaI checked and based on opestack health page it looks ok, though I am not sure if openstack health is healthy....15:06
ralonsohthere is patch for this bug15:06
slaweqthx ralonsoh for fixing that one bug15:06
slaweqlajoskatona yes, but I though it was failing even today and yesterday with this new issue15:07
slaweqlet me check it15:07
slaweqhttps://zuul.openstack.org/build/7760210f28514e6fb3e0258cbfcbd4b815:07
slaweqsee here15:07
slaweqit's from today morning15:07
ralonsohhttps://review.opendev.org/c/openstack/neutron/+/80562715:07
ralonsohthis patch15:07
ralonsohthat fixes the problem with the qos rule types constant15:08
lajoskatonaok15:08
slaweqralonsoh and what about test_versions issue, like https://4982338ebecb97a12f63-bcc0089d8984128efe3d45fe2388916e.ssl.cf5.rackcdn.com/periodic/opendev.org/openstack/neutron/master/openstack-tox-py36-with-neutron-lib-master/a100726/testr_results.html ?15:08
slaweqit happened e.g. yesterday15:08
lajoskatonaso openstack health is behind and out of sunc15:08
lajoskatonasync15:08
slaweqdo You know what could cause this?15:08
ralonsohslaweq, I think this problem is solved with https://review.opendev.org/c/openstack/neutron/+/80562715:09
slaweqok15:09
slaweqthx for taking care of it15:09
slaweqI will link that patch to that bug15:09
slaweqok, that are all actions from last week15:10
slaweqlet's move on15:10
slaweq#topic Stadium projects15:10
slaweqlajoskatona anything to discuss here?15:10
lajoskatonathere's a few patches for tap-as-a-service15:10
lajoskatonai.e.: https://review.opendev.org/c/openstack/tap-as-a-service/+/80342215:11
lajoskatonahttps://review.opendev.org/c/openstack/tap-as-a-service/+/80470715:11
lajoskatonabut otherwise quiet (except the vpnaas bug for memory leak)15:12
slaweqok, I will review them tonight or tomorrow morning15:12
bcafarelhalf stadium and half stable, vpnaas gate fixes victoria https://review.opendev.org/c/openstack/neutron-vpnaas/+/803455 and ussuri https://review.opendev.org/c/openstack/neutron-vpnaas/+/74602415:12
bcafarellajoskatona: now that you are full stable core you can probably update your +1 there now :)15:12
lajoskatonabcafarel: yeah, I can, just seen that once I checked them :-)15:13
slaweqthx15:13
slaweqand ci of other stadium projects is ok, right?15:14
slaweqno new issues there I hope15:14
lajoskatonano15:14
slaweqgreat, thx lajoskatona15:14
slaweqlet's move on15:14
slaweq#topic Stable branches15:14
slaweqbcafarel any new issues/updates?15:15
bcafareloverall it looks good, from what I checked since coming back from PTO15:15
bcafarelonly the already mentioned https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/80586415:15
slaweqthx15:16
bcafarelso overall you are leaving CI in nice shape for next PTL :)15:16
slaweqso let's move on15:16
slaweq#topic Grafana15:16
slaweqI was looking at dashboard today and I noticed that it's not up to date15:16
slaweqso I proposed https://review.opendev.org/c/openstack/project-config/+/805809 with many changes15:17
slaweqI will rebase it on top of obondarev's patch15:17
slaweqbut please review it when You will have some time15:17
amotokiI noticed that functional job in neutorn-lib is covered last week. Is it better to add it?15:18
amotoki* in grafana15:18
amotokiwoops ** is not covered15:19
amotokiI mean https://grafana.opendev.org/d/C5Wot6PMk/neutron-lib-failure-rate15:19
slaweq* amotoki TBH I didn't check that dashboard for long time :/15:20
slaweqbut yes, we should add it there15:20
amotokihehe15:20
slaweqand probably also neutron-tempest-plugin-api as this also runs in neutron-lib gate/check15:20
slaweqI will do it15:21
slaweqthx amotoki15:21
slaweq#action slaweq to update neutron-lib grafana dashboard15:21
slaweqfrom other things I just noticed now that neutron-ovn-tempest-slow is going very high with failure rates15:22
slaweqdid You maybe already checked it?15:22
ralonsohone problem was related to the new image15:22
bcafarelI need to check some logs to confirm it, but probably same issue as seen in wallaby15:22
ralonsohexactly15:23
slaweqFlavor with name ntp_image_384M already exists.15:23
slaweqyes15:23
ralonsoh(my bad...)15:23
slaweqok, so we need to merge that bcafarel's fix fast15:23
slaweqand we should be good with that :)15:23
slaweqthx once again15:23
slaweqthat's all from me regarding grafana15:24
slaweqand regarding other issues in functional/fullstack or tempest jobs, I didn't found anything new what we should discuss today15:24
slaweqin functional job we are still hitting the ovn related issue https://bugs.launchpad.net/neutron/+bug/193876615:25
slaweqfrom what jlibosva told me - that is most likely issue in openvswitch15:25
slaweqso he is going to check what earlier version of ovs we could use to unblock our gate15:26
slaweqand we will probably go with such workaround15:26
slaweqthx amotoki for help with that one too15:26
slaweqdo You have any other issues You would like to discuss today?15:27
ralonsoh(I need to leave now, see you tomorrow)15:27
slaweqif not, I think we can all finish earlier and ralonsoh will be able to leave without losing anything :)15:27
slaweqsee You ralonsoh15:27
ralonsohheheh thanks15:27
bcafarel:)15:28
slaweqok, I guess that this silence means "we want to go home now!!" :)15:29
slaweqso thx for attending the meeting today and see You all online15:30
obondarev\o/15:30
slaweq#endmeeting15:30
opendevmeetMeeting ended Tue Aug 24 15:30:15 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:30
opendevmeetMinutes:        https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-08-24-15.00.html15:30
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-08-24-15.00.txt15:30
opendevmeetLog:            https://meetings.opendev.org/meetings/neutron_ci/2021/neutron_ci.2021-08-24-15.00.log.html15:30
lajoskatonao/15:30
amotokio/15:30
bcafarelo/15:30
thomasb06o/15:31
jlibosvaslaweq: btw about that issue - it's really related to the short living DB connections because of the pg_drop group15:32
jlibosvaslaweq: I'm thinking about removing the short living connection because it causes issues at scale too15:35
slaweqjlibosva do we need those short living connections and pg_drop group creation in functional tests? maybe we could mock it there to workaround that issue?15:36
jlibosvaslaweq: I'd like to remove the short living connections as a whole. It was a bad idea of mine :)15:37
slaweqjlibosva ok15:38
*** rpittau is now known as rpittau|afk16:00
opendevreviewMerged openstack/neutron-vpnaas stable/ussuri: [stable/ussuri] Fix gates  https://review.opendev.org/c/openstack/neutron-vpnaas/+/74602416:01
opendevreviewMerged openstack/neutron-vpnaas stable/victoria: Fix inconsistency in requirements  https://review.opendev.org/c/openstack/neutron-vpnaas/+/80345516:03
opendevreviewMerged openstack/neutron stable/stein: Fix notify listener syntax for SEGMENT_HOST_MAPPING  https://review.opendev.org/c/openstack/neutron/+/80373820:38
*** tbachman is now known as Guest530120:57
*** tbachman is now known as Guest530221:01

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!