14:00:22 <slaweq> #startmeeting networking
14:00:23 <openstack> Meeting started Tue Dec  1 14:00:22 2020 UTC and is due to finish in 60 minutes.  The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:27 <openstack> The meeting name has been set to 'networking'
14:00:31 <slaweq> hi!
14:00:31 <mlavalle> o/
14:00:47 <amotoki> o/
14:00:49 <ralonsoh> hi
14:00:53 <bcafarel> hi
14:01:09 <obondarev> hi
14:01:09 <rubasov> hi
14:01:25 <lajoskatona> i
14:01:27 <lajoskatona> Hi
14:01:38 <slaweq> #topic Announcements
14:01:58 <slaweq> After migration to new Gerrit automation scripts which integrates Launchpad and Gerrit seems that are not working
14:02:15 <slaweq> So please update Your LPs when You push patch related to some bug report
14:02:26 <slaweq> like set LP to be in progress
14:02:39 <slaweq> and paste link to the proposed fix in comment
14:02:52 <slaweq> and also assign LP to yourself in such case
14:03:34 <slaweq> next one
14:03:45 <slaweq> according to the https://releases.openstack.org/wallaby/schedule.html
14:03:54 <slaweq> this week we have Wallaby-1 milestone
14:04:13 <slaweq> releases of some libs are already proposed or even done for things like neutron-lib
14:04:20 <slaweq> so we should be good there
14:04:54 <slaweq> but also, I would like ask You all once again to spent some time on reviewing opened specs: https://review.opendev.org/q/project:openstack/neutron-specs+status:open
14:05:10 <slaweq> so hopefully authors of them will be able to start working on implementation soon
14:05:32 <slaweq> and that's all announcements/reminders from me for today
14:05:41 <slaweq> do You have anything else You want to share?
14:07:03 <slaweq> ok, I guess that this means "no"
14:07:07 <slaweq> so lets move on
14:07:13 <slaweq> #topic Blueprints
14:07:26 <slaweq> I moved things from W-1 to W-2 now
14:07:30 <slaweq> https://bugs.launchpad.net/neutron/+milestone/wallaby-2
14:07:44 <slaweq> and I updated today BP https://blueprints.launchpad.net/neutron/+spec/enginefacade-switch which is almost done on neutron side
14:08:16 <slaweq> last week we merged ralonsoh's patch which we though that finishes that but there were still some leftovers in the neutron code
14:08:27 <slaweq> so I pushed couple of new patches today https://review.opendev.org/q/project:openstack/neutron-specs+status:open
14:08:27 <ralonsoh> (sorry for that)
14:08:32 <bcafarel> still, worth some \o/
14:08:32 <slaweq> ralonsoh: np at all :)
14:08:49 <slaweq> You did worst part actually
14:09:18 <slaweq> sorry, wrong link
14:09:20 <ralonsoh> btw, correct link
14:09:20 <slaweq> https://review.opendev.org/#/q/status:open+topic:bp/enginefacade-switch
14:09:20 <ralonsoh> https://review.opendev.org/q/topic:%22bp%252Fenginefacade-switch%22+(status:open%20OR%20status:merged)
14:09:25 <slaweq> thx ralonsoh :)
14:10:20 <slaweq> we still need to make same transition for the stadium projects before we will mark this BP as completed
14:10:55 <mlavalle> Great work!
14:11:07 <njohnston> Great job ralonsoh, slaweq, and everyone!  The dragon is slain (at least in neutron)!
14:11:30 <slaweq> :)
14:12:59 <slaweq> regarding other BPs, I created new one https://blueprints.launchpad.net/neutron/+spec/secure-bac-roles
14:13:09 <slaweq> to track migration to secure rbac roles in Neutron
14:13:54 <slaweq> I know that mlavalle and amotoki was volunteering to work on that but recently it became high prio for Red Hat so we have more pressure to work on that and I already started sending some patches
14:14:07 <mlavalle> ok
14:14:09 <slaweq> https://review.opendev.org/q/topic:%2522secure-rbac%2522+(status:open+OR+status:merged)+project:openstack/neutron
14:14:19 <slaweq> any help with that is of course welcome :)
14:14:23 <mlavalle> ok
14:14:29 <amotoki> slaweq: sorry for the delay and thanks for starting the work. I will help it.
14:14:54 <slaweq> no need to sorry, we all have own priorities and limited time
14:15:08 <slaweq> and thx for help and valuable reviews there :)
14:15:12 <amotoki> btw, do you continue to use the current blueprint with typo.
14:15:44 <amotoki> ?
14:15:58 <slaweq> amotoki: is there way to change it? or the only way is to create new BP?
14:16:00 <slaweq> do You know?
14:16:28 <amotoki> slaweq: I will check it. otherwise we can create a new one and mark it as superceded
14:16:39 <slaweq> ok, thx
14:16:58 <amotoki> renamed it to https://blueprints.launchpad.net/neutron/+spec/secure-rbac-roles now
14:17:12 <amotoki> we can change it from Change details at the right top corner.
14:17:14 <slaweq> thx a lot
14:17:27 <slaweq> good to know
14:18:53 <slaweq> that's all updates from me regarding BPs
14:19:06 <slaweq> do You have anything else
14:19:08 <slaweq> ?
14:19:09 <mlavalle> I have an update / request regarding address groups
14:19:56 <mlavalle> We are facing some challenges updating the firewall after an address group has been updated. hangyang left a comment yesterday in https://review.opendev.org/c/openstack/neutron/+/757650
14:20:06 <mlavalle> could ralonsoh take a look?
14:20:10 <ralonsoh> sure
14:20:17 <mlavalle> Tnaks!
14:20:24 <mlavalle> that's all
14:21:11 <slaweq> ahh, conjunctions
14:21:15 <slaweq> :/
14:21:17 <slaweq> good luck :P
14:21:45 <mlavalle> LOL
14:21:59 <lajoskatona> Another conjunction: https://earthsky.org/astronomy-essentials/great-jupiter-saturn-conjunction-dec-21-2020
14:22:20 <lajoskatona> sorry for hyjacking the meeting....
14:22:24 <slaweq> :)
14:23:16 <bcafarel> I very much prefer the later one
14:23:45 <njohnston> lol, for sure
14:24:09 <slaweq> ok, if there are no other updates about BPs, lets move on
14:24:17 <slaweq> #topic Community Goals
14:24:39 <slaweq> regarding "Migrate RBAC Policy Format from JSON to YAML" I saw that gmann already pushed some patches
14:24:44 <slaweq> https://review.opendev.org/c/openstack/neutron/+/764401
14:24:46 <slaweq> https://review.opendev.org/c/openstack/neutron-lib/+/764416
14:24:52 <slaweq> and there is also mail http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019079.html
14:25:01 <slaweq> amotoki: You are probably aware of all of that
14:25:17 <slaweq> but just in case I wanted to paste it all here :)
14:25:19 <amotoki> yeah. I tried it and saw some unclear behavior.
14:26:31 <amotoki> I will follow it up.
14:27:12 <slaweq> thx amotoki
14:27:40 <slaweq> amotoki: if You will need any help, please ping me
14:27:47 <amotoki> slaweq: sure. thanks
14:28:17 <slaweq> so that one should be under control for now
14:28:26 <slaweq> ralonsoh: any updates about migration to privsep?
14:28:47 <ralonsoh> yes, but I'm facing some problems with one patch (maybe pyroute2 related)
14:28:57 <ralonsoh> and I have the "grenade" patch
14:28:58 <ralonsoh> (one sec)
14:29:15 <ralonsoh> https://review.opendev.org/c/openstack/neutron/+/764015
14:29:27 <ralonsoh> that will replace rootwrap, generically, with a privsep context
14:29:35 <ralonsoh> I know this is gross but effective
14:30:04 <slaweq> yeah, so kind of "one patch to migrate them all", right?
14:30:08 <ralonsoh> yes
14:30:15 <ralonsoh> but, of course, many errors still there
14:30:38 <ralonsoh> so I'll be focused first in the short ones
14:31:43 <slaweq> ok
14:31:56 <slaweq> thx for working on that
14:33:03 <slaweq> I think we can move on
14:33:06 <slaweq> next topic
14:33:11 <slaweq> #topic Bugs
14:33:19 <slaweq> obondarev was bug deputy last week
14:33:29 <slaweq> report is at http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019119.html
14:33:39 <slaweq> obondarev: any bugs You would like to bring to the team now?
14:33:48 <obondarev> not really
14:34:00 <obondarev> pretty quiet week
14:34:11 <ralonsoh> (cool)
14:34:28 <bcafarel> that was until some PTL started filling bugs today :)
14:34:35 <slaweq> yeah, I saw Your report
14:34:47 <slaweq> there is only one unassigned bug there
14:34:52 <slaweq> https://bugs.launchpad.net/neutron/+bug/1905551
14:34:54 <openstack> Launchpad bug 1905551 in neutron "functional: test_gateway_chassis_rebalance fails" [Medium,Confirmed]
14:35:19 <slaweq> related to ovn functional tests
14:35:32 <slaweq> so would be great if someone could take a look at it
14:37:29 <slaweq> ok, any other bugs You want to discuss today?
14:38:59 <slaweq> if not, I have one
14:39:10 <slaweq> I added it to the "On Demand Agenda" but we can discuss now
14:39:16 <slaweq> https://bugs.launchpad.net/neutron/+bug/1903531
14:39:18 <openstack> Launchpad bug 1903531 in neutron "Update of neutron-server breaks compatibility to previous neutron-agent version" [Critical,Confirmed] - Assigned to Slawek Kaplonski (slaweq)
14:39:45 <slaweq> after some discussion on irc and ML last week I decided that will be better to remove it from all branches
14:40:04 <slaweq> and then propose it make again to master but with properly provided backward compatibility
14:40:15 <slaweq> patches are proposed:
14:40:17 <slaweq> master - https://review.opendev.org/c/openstack/neutron/+/764189
14:40:19 <slaweq> victoria - https://review.opendev.org/c/openstack/neutron/+/764190
14:40:21 <slaweq> ussuri - https://review.opendev.org/c/openstack/neutron/+/764191
14:40:53 <slaweq> liuyulong proposed https://review.opendev.org/c/openstack/neutron/+/764108 to avoid reverting original patch from the master branch
14:42:26 <slaweq> so I wanted to discuss it once again here - should we not revert original patch from master and just go with liuyulong's additional patch? Or what is the best approach in Your opinion?
14:42:47 <slaweq> because we basically broke upgrades/updates with that patch and we have to fix it somehow
14:43:24 <ralonsoh> this is not a bad idea: to convert the message from the server and cap the version given to the agents
14:43:25 <bcafarel> well for master it does not change much, by wallaby release we should have updated API target and relevant code
14:43:36 <bcafarel> wether we revert fully and reapply or just bump
14:44:10 <bcafarel> but for stable... train to ussuri we will need old API to fix upgrade right?
14:44:34 <slaweq> for stable we need IMO to revert that patch
14:44:34 <ralonsoh> liu's patch should handle both
14:44:42 <ralonsoh> (if server is updated first)
14:44:52 <bcafarel> which means revert (breaking minor updates but can probably not be helpd, hence possible release note here) and then only fix in wallaby
14:44:53 <slaweq> as otherwise we will always have some point where we will have incompatibility
14:45:37 <slaweq> so do You think it will be good to go with revert for stable/victoria and ussuri and with liuyulong's patch for master, right?
14:45:47 <amotoki> I agree that we need to revert that patch in all stable branches.
14:46:13 <ralonsoh> yes for master, because Liu's patch can handle when server is bumped and agents not
14:46:16 <amotoki> +1 for slaweq's proposal
14:46:19 <ralonsoh> +1
14:46:34 <slaweq> ok, thx
14:47:09 <slaweq> so lets make it that way to fix it
14:47:13 <slaweq> thx for Your opinions
14:47:24 <liuyulong> Why not backport the fix to the patch original merged release? V? or U?
14:47:43 <slaweq> liuyulong: I don't think we should/can really backport things which bumps rpc version
14:47:52 <ralonsoh> exactly
14:48:07 <ralonsoh> (and we already had this problem with this bug)
14:48:16 <slaweq> this original patch should have bumped rpc version and we should never backport it to stable branches tbh
14:48:22 <liuyulong> It should be there, we missed that, so I'm OK if we can break the law once.
14:48:49 <slaweq> liuyulong: I disagree with that - we already broke that law once with that patch and made a lot of mess
14:48:55 <slaweq> we shouldn't do that again
14:49:20 <liuyulong> I'm not saying the release before V or U release.
14:49:29 <slaweq> and that's also what I got from the releases team when I discussed with them
14:49:47 <amotoki> I am not sure we cannot bump RPC version in stalbe branhces but we must keep backward compatibility between server and agent.
14:50:49 <slaweq> amotoki: liuyulong: ok, I will ask once again stable main cores about opinion on that
14:51:00 <liuyulong> The patch was in stable/ussuri natively, just checked that.
14:51:02 <slaweq> if they will tell me that we can go with that, then ok
14:51:13 <slaweq> liuyulong: yes, I know
14:51:18 <amotoki> in this case, agent is not upgraded and requests an older version, so the server should talk the older version at least.
14:51:58 <bcafarel> if you follow a correct order in upgrade yes with versioned it works OK, though if you for some reason update agents first it will complain
14:52:35 <ralonsoh> bcafarel, well, the upgrade procedure says that you should update the servers first
14:52:35 <slaweq> bcafarel: but we do support update when older agents can work with newer server
14:52:39 <slaweq> not vice versa
14:52:53 <slaweq> and we should only really care about such scenario
14:53:05 <lajoskatona> https://docs.openstack.org/project-team-guide/stable-branches.html#review-guidelines
14:53:38 <lajoskatona> here it is mentions only Nova’s internal AMQP API, but I suppose that can be true for other projects as well
14:53:39 <bcafarel> yes, so it may be acceptable (as in "security fix with proper RPC version bump")
14:54:27 <amotoki> I think we are discussing two aspects: stable backports and upgrade scenarios.
14:54:57 <amotoki> upgrade scnearios should be considered even if we are in development cycle.
14:54:59 <slaweq> amotoki: for me now the question is if we can do backport of that patch which bumps rpc version
14:55:25 <slaweq> at least for me :)
14:56:06 <amotoki> slaweq: yes you're right
14:56:13 <liuyulong> Actually the RPC was only called when use iptables_hybrid and enable ebtables. The openflow firewall driver will not call this function. Openflow firewall uses local cache directly. Maybe that's why we missed that RPC bump.
14:56:41 <slaweq> liuyulong: yes, because of that and because of lack of grenade testing
14:56:57 <amotoki> I wonder whether during stable upgrade process both agent and server can be upgrade first... If we allow to upgrade either first, we cannot upgrade RPC version at all.
14:56:59 <slaweq> we are testing grenade only with openvswitch firewall driver which is default in devstack
14:57:39 <slaweq> amotoki: in the docs we guarantee that You can update server first and newer server will be compatible to work with older agents
14:57:47 <slaweq> that's what we broke with that patch actually
14:57:56 <bcafarel> yep
14:57:57 <liuyulong> amotoki, yes, if the client side has higher version, the RPC will get failed directly.
14:58:36 <bcafarel> seeing as time is running out, at least first steps seem OK for everyone right? fixing version in master, reverting ussuri/victoria
14:58:40 <amotoki> thanks. I missed that point.
14:58:54 <bcafarel> and then discuss if this can be backported again with new rpc
14:59:02 <ralonsoh> bcafarel, I think the proposal is different now
14:59:04 <slaweq> we are almost on top of the hour now, I will ask stable-maint cores about backporting liuyulong's patch to V/U also
14:59:14 <slaweq> if that will be ok for them, I will abandon my reverts
14:59:29 <slaweq> if not, then lets revert it from those branches and go with liuyulong's patch in master
14:59:37 <slaweq> sounds good?
14:59:48 <bcafarel> ack
14:59:55 <liuyulong> OK
14:59:58 <amotoki> sounds good
15:00:04 <slaweq> ok, thx for attending the meeting
15:00:10 <slaweq> have a great week :)
15:00:13 <slaweq> #endmeeting