Tuesday, 2020-12-08

*** SWDevAngel has quit IRC00:04
*** macz_ has quit IRC00:46
*** tosky has quit IRC00:54
*** hemanth_n has joined #openstack-meeting-302:36
*** hemanth_n has quit IRC02:49
*** psachin has joined #openstack-meeting-303:16
*** hemanth_n has joined #openstack-meeting-303:29
*** hemanth_n has quit IRC03:33
*** hemanth_n has joined #openstack-meeting-303:48
*** benj_ has quit IRC06:04
*** benj_ has joined #openstack-meeting-306:05
*** ricolin has quit IRC06:24
*** ricolin has joined #openstack-meeting-306:30
*** ricolin has quit IRC06:36
*** ricolin has joined #openstack-meeting-306:53
*** ricolin has quit IRC07:23
*** ricolin has joined #openstack-meeting-307:39
*** lpetrut has joined #openstack-meeting-307:55
*** slaweq has joined #openstack-meeting-308:08
*** baojg has quit IRC08:44
*** baojg has joined #openstack-meeting-309:05
*** baojg has quit IRC09:08
*** baojg has joined #openstack-meeting-309:09
*** fnordahl has joined #openstack-meeting-309:19
*** e0ne has joined #openstack-meeting-309:39
*** baojg has quit IRC09:57
*** baojg has joined #openstack-meeting-309:58
*** lkoranda has joined #openstack-meeting-310:16
*** e0ne has quit IRC11:22
*** lkoranda has quit IRC11:24
*** e0ne has joined #openstack-meeting-311:51
*** raildo has joined #openstack-meeting-311:58
*** baojg has quit IRC12:07
*** baojg has joined #openstack-meeting-312:07
*** Luzi has joined #openstack-meeting-313:00
*** tosky has joined #openstack-meeting-313:44
*** obondarev has joined #openstack-meeting-313:50
*** hemanth_n has quit IRC13:50
*** lajoskatona has joined #openstack-meeting-314:02
*** elvira has joined #openstack-meeting-314:02
mlavalleno Neutron meeting today?14:02
*** slaweq has quit IRC14:03
rubasovslaweq in the ML said there will be one14:03
rubasovmaybe something urgent came up14:04
*** slaweq has joined #openstack-meeting-314:04
mlavalleyeap14:04
slaweq#startmeeting networking14:04
openstackMeeting started Tue Dec  8 14:04:48 2020 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.14:04
slaweqhi14:04
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:04
slaweqsorry for being late14:04
*** openstack changes topic to " (Meeting topic: networking)"14:04
openstackThe meeting name has been set to 'networking'14:04
mlavalleo/14:04
slaweqmy laptop hanged and I had to reboot it :/14:04
rubasovhi14:05
haleybo/14:05
obondarevhi14:05
bcafarelIRC is too heavy for your laptop slaweq ? :)14:05
bcafarel(and o/ too)14:05
slaweqbcafarel: not irc14:05
slaweq#topic Announcements14:05
*** openstack changes topic to "Announcements (Meeting topic: networking)"14:05
njohnstono/14:05
slaweq6umvvpv5V88326114:06
slaweqWe are now in the Wallaby-2 time period14:06
slaweqmilestone will be in the week of Jan 18th14:06
lajoskatonao/14:06
slaweqso we have few more weeks but there is holiday season in the middle of it14:07
slaweqso not a lot of time14:07
slaweqI would like to ask You all to spent some time reviewing specs related to the approved rfes:14:07
slaweq     https://review.opendev.org/c/openstack/neutron-specs/+/72953214:07
slaweq    https://review.opendev.org/c/openstack/neutron-specs/+/65845114:07
slaweq    https://review.opendev.org/c/openstack/neutron-specs/+/72862814:07
slaweq    https://review.opendev.org/c/openstack/neutron-specs/+/73954914:07
slaweqand one additional announcement from me14:08
slaweqOVS Conf starts today: https://www.openvswitch.org/support/ovscon2020/14:08
slaweqmaybe someone will be interested and wants to look at some of the talks there :)14:08
slaweqand that's all for announcements/reminders for today from me14:09
slaweqanything else You want to say now?14:10
slaweqif not, lest move on14:12
slaweq#topic Blueprints14:12
*** openstack changes topic to "Blueprints (Meeting topic: networking)"14:12
slaweqWallaby-2 https://bugs.launchpad.net/neutron/+milestone/wallaby-214:12
slaweqI have an update about https://blueprints.launchpad.net/neutron/+spec/secure-bac-roles14:13
slaweqtogether with lbragstad we did progress on it14:14
slaweqpatches are proposed https://review.opendev.org/q/topic:%2522secure-rbac%2522+(status:open+OR+status:merged)+project:openstack/neutron,14:14
slaweqplease review them if You will have time14:14
slaweqin most cases it's pretty "mechanical" change of the policy rules14:14
bcafarelwow at that list of patches14:14
slaweqbut we need to be careful to not make new rules to be e.g. more widely open than the old ones14:14
amotokithanks for the great progress.14:16
slaweqand that's all for updates about BPs from me14:16
slaweqanything else do You have maybe?14:16
slaweqok, I guess this means "no"14:19
slaweqlets move on to the next topic14:19
slaweq#topic Community Goals14:19
*** openstack changes topic to "Community Goals (Meeting topic: networking)"14:19
slaweqamotoki: any updates about migration policy.json to yaml?14:19
amotokislaweq: gmann updated the migration patch and my unclear points were resolved.14:20
amotokiit generally looks good but needs to pass the gate.14:20
slaweqamotoki: neutron-lib patch is approved already14:21
gmannamotoki: slaweq one more update on that, i moving the config registration part in oslo side - https://review.opendev.org/c/openstack/oslo.upgradecheck/+/765631/514:21
amotokigmann: is it better to wait the oslo side change?14:22
gmannwith that we do not need to register on each service side, i mean avoid code duplicity.  once this oslo one is merged today, i will make new release for oslo.upgradecheck and then use that in neutron side14:22
gmannamotoki: yeah. I will update neutron patch with that14:23
amotokisounds nice14:23
slaweqgood14:23
slaweqwe will probably need to release neutron-lib with https://review.opendev.org/c/openstack/neutron-lib/+/764416 before we will be able to move on with neutron patch, right?14:24
amotokimy understanding is that it is not a requirement. Both neutron patch and neutron-lib patch modify the default policy_file value. I think we will start to look up policy.yaml once we cut a new release of neutron-lib.14:26
amotokibut it looks good to cut a new release.14:26
slaweqok14:27
slaweqthx for clarification14:27
gmann+1 neutron patch will be more additional things like warning + upgrade checks etc14:27
slaweqok14:28
slaweqso I think this one is clear now14:28
slaweqlets move on14:28
slaweq#topic Bugs14:28
*** openstack changes topic to "Bugs (Meeting topic: networking)"14:28
slaweqbcafarel was our bug deputy last week14:29
slaweqreport is available at http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019244.html14:29
slaweqbcafarel: any bugs You would like to highlight?14:29
bcafarelmost bugs are already under way, just https://bugs.launchpad.net/neutron/+bug/1906490 (OVN tempest failure) is unassigned14:30
openstackLaunchpad bug 1906490 in neutron "SSH failures in the neutron-ovn-tempest-ovs-release-ipv6-only job" [Critical,Confirmed]14:30
bcafareland https://bugs.launchpad.net/neutron/+bug/1906406 also is not assigned, related to dnsmasq initial leases file creation when using segments14:30
openstackLaunchpad bug 1906406 in neutron "[segments] dnsmasq can't delete lease for instance due to mismatch between client ip and local addr" [Medium,Confirmed]14:30
bcafarelfor the others it is mostly check relevant proposed patchs14:31
slaweqthx14:32
slaweqI also wanted to mention https://bugs.launchpad.net/neutron/+bug/1906406 :)14:32
openstackLaunchpad bug 1906406 in neutron "[segments] dnsmasq can't delete lease for instance due to mismatch between client ip and local addr" [Medium,Confirmed]14:32
slaweqand ask folks familiar with segments and routed networks to take a look at it14:33
slaweqany other bugs You want to discuss today?14:34
slaweqok, I guess that there are no other bugs to discuss today14:37
slaweqthis week I am bug deputy14:38
slaweqahh, one more thing14:39
slaweqregarding bug https://bugs.launchpad.net/neutron/+bug/190353114:39
openstackLaunchpad bug 1903531 in neutron "Update of neutron-server breaks compatibility to previous neutron-agent version" [Critical,Confirmed] - Assigned to Slawek Kaplonski (slaweq)14:39
slaweqI was talking with folks from the stable main team and I also sent email to ML about it14:39
slaweqand we decided that we should go with liu's patch https://review.opendev.org/c/openstack/neutron/+/764108 and backport it to the Ussuri as that was branch where this change was originally made14:40
slaweqI think that this will be the best way to go14:40
slaweqand we should then have backward compatybility also14:40
slaweqany thoughts about it?14:40
bcafarelthat sounds good14:41
bcafarelwith new RPC version and no need to break again for existing ussuri/victoria users14:42
slaweqok14:44
slaweqthat's all what I had for today14:44
slaweqif there is nothing else, I will give You 15 minutes back today14:45
slaweqthx for attending14:45
slaweqand see You online14:45
slaweqo/14:45
slaweq#endmeeting14:45
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:45
openstackMeeting ended Tue Dec  8 14:45:19 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:45
openstackMinutes:        http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-12-08-14.04.html14:45
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-12-08-14.04.txt14:45
openstackLog:            http://eavesdrop.openstack.org/meetings/networking/2020/networking.2020-12-08-14.04.log.html14:45
rubasovo/14:45
amotokio/14:45
lajoskatonao/14:46
bcafarelo/14:46
*** elvira has left #openstack-meeting-314:47
*** elvira has joined #openstack-meeting-314:49
mlavalleo/14:51
slaweq#startmeeting neutron_ci15:00
openstackMeeting started Tue Dec  8 15:00:56 2020 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
slaweqwelcome again :)15:01
*** openstack changes topic to " (Meeting topic: neutron_ci)"15:01
openstackThe meeting name has been set to 'neutron_ci'15:01
bcafarelhi again15:01
slaweqlets wait few more minutes for other folks15:02
lajoskatonaHi15:03
bcafarelit gives me more time to watch progress/failure on https://review.opendev.org/c/openstack/neutron/+/76600015:03
slaweqok, lets start and do that quickly15:04
slaweqGrafana dashboard: http://grafana.openstack.org/dashboard/db/neutron-failure-rate15:04
slaweq#topic Actions from previous meetings15:04
*** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)"15:04
slaweq    bcafarel to check and update doc https://docs.openstack.org/neutron/latest/contributor/policies/release-checklist.html15:05
* bcafarel looks for link15:05
bcafarelhttps://review.opendev.org/c/openstack/neutron/+/765959 I added also note on neutron-tempest-plugin template and *-master jobs15:05
slaweqthx15:06
slaweqI will check that later15:07
slaweqnext one15:08
slaweqralonsoh to report and check issue with TestSimpleMonitorInterface in functional tests15:08
slaweqralonsoh is not here today15:08
slaweqbut as this hits us pretty often recently, I spent some time yesterday to check that15:09
slaweq    LP https://bugs.launchpad.net/neutron/+bug/190706815:09
openstackLaunchpad bug 1907068 in neutron "Functional test neutron.tests.functional.agent.linux.test_ovsdb_monitor.TestSimpleInterfaceMonitor.test_get_events" [Critical,In progress] - Assigned to Slawek Kaplonski (slaweq)15:09
slaweq    Patch https://review.opendev.org/c/openstack/neutron/+/76579215:09
slaweqI hope that this will help with that issue15:09
slaweqso please review it :)15:09
bcafareluniversal solution (aka add some sleep() ) detected :)15:10
slaweqbcafarel: yes :/15:10
slaweqbut I don't see better way to solve it really15:10
slaweqnext one15:11
slaweq    slaweq to check if test_dhcp_port_status_active will be still failing after https://review.opendev.org/c/openstack/neutron/+/755313 will be merged15:11
slaweqit didn't help for sure15:11
slaweqso bug reported https://bugs.launchpad.net/neutron/+bug/190665415:11
openstackLaunchpad bug 1906654 in neutron "neutron_tempest_plugin.api.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_dhcp_port_status_active is failing often" [Critical,Confirmed]15:11
slaweqand Skip proposed https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/76532715:12
slaweqit's already merged so should give us a breath on CI15:12
slaweqbut we need to investigate why this happens really15:12
slaweqnext one15:12
slaweqslaweq to report LP about SSH failures in the neutron-ovn-tempest-ovs-release-ipv6-only15:12
slaweq    Bug https://bugs.launchpad.net/neutron/+bug/190649015:13
openstackLaunchpad bug 1906490 in neutron "SSH failures in the neutron-ovn-tempest-ovs-release-ipv6-only job" [Critical,Confirmed]15:13
slaweq    Patch to skip failing test https://review.opendev.org/c/openstack/neutron/+/76507015:13
*** obondarev has quit IRC15:13
slaweqas in most or all cases it's this one test which is failing, I proposed to skip it for now15:13
slaweqso please review that patch too :)15:13
slaweqand that's all regarding last week15:15
slaweqlets move on15:16
slaweqanything regarding stadium projects or stable branches?15:16
bcafarelnot much from my side, https://review.opendev.org/c/openstack/requirements/+/764021 still not merged for ussuri15:17
bcafareland I did not check stable branches yet much, though nobody complained they were broken so it should be OK15:17
slaweq:)15:18
lajoskatonafor stadium: the patch to make unsatble some bgpvpn tests was just merged15:18
slaweqlajoskatona: yes, I saw15:18
slaweqand it makes job to be voting again, right?15:18
lajoskatonaso the job is voting with again15:19
lajoskatonayes exactly15:19
slaweq++15:19
slaweqW15:21
slaweqsorry15:21
slaweqso I think we can move on15:21
slaweq#topic Grafana15:21
*** openstack changes topic to "Grafana (Meeting topic: neutron_ci)"15:21
slaweqin overall check queue seems to be in very bad state due to that issue with pip dependencies15:22
slaweqalso functional and fullstack jobs are failing 100% of times15:23
bcafarelsigh15:25
bcafarelfor pip deps I am playing whack-a-mole "fix one dep, have new error" in https://review.opendev.org/c/openstack/neutron/+/76600015:26
slaweqfor fullstack it seems that error is similar https://zuul.opendev.org/t/openstack/build/d9c4e4ac0b794b7f8c667249a9eb3a3515:26
slaweqbut not exactly the same15:26
*** jlibosva has joined #openstack-meeting-315:26
slaweqit's the same for functional and fullstack15:26
lajoskatonabcafarel: LOL15:26
bcafarelyes that sounds like similar root cause (new resolver in pip or whatever)15:26
slaweqlajoskatona: bcafarel: as You already playing with it, can You also check fullstack and functional jobs?15:27
bcafarelat least to pass CI we will need all of them fixed indeed15:28
lajoskatonaslaweq: yeah, I think I ask around if some common wisdom come from infra team or similar place15:28
slaweqlajoskatona: thx15:28
slaweqplease use this LP which I reported to track all those issues, I don't think we need another one for those fullstack/functional jobs too15:29
bcafarel+115:29
slaweqthx15:30
*** Luzi has quit IRC15:31
slaweqother than that I don't have too many other things to discuss today15:31
slaweqin most cases I think that we are hitting known issues like failure with TestSimpleMonitor in functional job (should be fixed with sleep()), fullstack lack of resources which should be fixed by lajoskatona's patch15:32
slaweqand neutron-ovn-tempest-ovs-release-ipv6-only issues with ssh15:32
slaweqI have just one more topic to discuss15:34
slaweqfew days ago I sent email http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019240.html15:34
slaweqabout kernel panics in guest vms15:34
slaweqaccording to the log there we should try to use "noapic" option during boot of the vm15:35
slaweqdo You know if there is any way we can do that from our jobs? (I'm not expert there and I don't really know that)15:36
slaweqor we would need to change cirros image to achieve that?15:36
bcafarelhmm15:37
bcafarelI think this can be done in qemu itself (passing this kind of option), I guess nova folks will know better15:37
slaweqaccording to sean-k-mooney reply nova can provide such option so IIUC currently it isn't possible really15:39
sean-k-mooneycurrently we cant disable it no15:39
sean-k-mooneywe woudl have to add a new flag to the glance image properties15:39
sean-k-mooneythis look like its a cirrios kernel bug15:40
sean-k-mooneyi dont think the 5.3 kernel has been patched15:40
sean-k-mooneycirros is i belive more or less unmaintianed so we will eventurally have to move to somethign like alpine to file the same role as a longterm solution15:41
slaweqsean-k-mooney: are You talking about https://alpinelinux.org/ ?15:41
slaweqor something else?15:41
sean-k-mooneyyes that15:41
slaweqbut I saw there that those images are much bigger than cirros15:42
sean-k-mooneyits the smallest actvily developed os that woudl be a resonable replacement15:42
sean-k-mooneynot tha much15:42
sean-k-mooneythey are still ~ 40mb15:42
sean-k-mooneyless then 100mb certenly15:42
sean-k-mooneywe have a min vm size of 1GB15:43
slaweqhttp://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/alpine-virt-3.12.1-x86_64.iso15:43
slaweqthis one, right?15:43
bcafareland with larger community around too, when you use cirros people often reply "oh you are from openstack"15:43
sean-k-mooneyslaweq: ya one sec15:43
sean-k-mooneyhttps://review.opendev.org/q/topic:%22alpine%22+(status:open%20OR%20status:merged)15:44
sean-k-mooneyi started adding support alpine but i hit an initramfs issue using the minimal filesystem image15:44
sean-k-mooneywhich si for contienr and chroots15:44
*** macz_ has joined #openstack-meeting-315:44
sean-k-mooneyi had planned to eventually swap to that iso as a base15:45
sean-k-mooneybut have not had time to work on it15:45
*** tosky has quit IRC15:45
sean-k-mooneyshort termif ths is blocking the gate15:45
slaweqok, I will try to check this iso maybe locally15:45
sean-k-mooneyi can add a flag to nova quickly for this15:45
sean-k-mooneyand then ping the core team to review15:46
sean-k-mooneye.g. to disabel the ioapic15:46
slaweqit's not blocking the gate but I see some jobs failed due to kernel panics in guest vm at least few times a week15:46
slaweqfor sure we have more urgent and critical issues but fixing somehow this one would also help, I think not only Neutron but also other projects :)15:46
sean-k-mooneyill bring it up on thet nova channel shortly and get buy in if there is no objectsion ill quickly write a patch as a workaround15:47
slaweqsean-k-mooney: would be great, thx a lot15:47
sean-k-mooneyya it showe up in the nova jobs too15:47
*** macz_ has quit IRC15:48
slaweqok, that was the last thing which I had for this week15:48
*** macz_ has joined #openstack-meeting-315:48
slaweqif You don't have anything else, I will give You few minutes back15:49
slaweqthx for attending the meeting15:50
bcafarelsounds good15:50
slaweqhave a great week15:50
bcafarelo/15:50
lajoskatonaBye15:50
slaweqo/15:50
slaweq#endmeeting15:50
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:50
openstackMeeting ended Tue Dec  8 15:50:44 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:50
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-12-08-15.00.html15:50
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-12-08-15.00.txt15:50
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-12-08-15.00.log.html15:50
*** psachin has quit IRC16:38
*** jlibosva has quit IRC16:42
*** lpetrut has quit IRC16:48
*** e0ne has quit IRC17:12
*** lajoskatona has left #openstack-meeting-317:12
*** e0ne has joined #openstack-meeting-317:13
*** elvira has quit IRC17:13
*** e0ne has quit IRC17:13
*** tosky has joined #openstack-meeting-317:26
*** frickler has joined #openstack-meeting-319:02
*** ianychoi__ has quit IRC19:44
*** artom has quit IRC21:39
*** artom has joined #openstack-meeting-321:40

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!