14:00:22 <slaweq> #startmeeting networking 14:00:23 <openstack> Meeting started Tue Dec 1 14:00:22 2020 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:27 <openstack> The meeting name has been set to 'networking' 14:00:31 <slaweq> hi! 14:00:31 <mlavalle> o/ 14:00:47 <amotoki> o/ 14:00:49 <ralonsoh> hi 14:00:53 <bcafarel> hi 14:01:09 <obondarev> hi 14:01:09 <rubasov> hi 14:01:25 <lajoskatona> i 14:01:27 <lajoskatona> Hi 14:01:38 <slaweq> #topic Announcements 14:01:58 <slaweq> After migration to new Gerrit automation scripts which integrates Launchpad and Gerrit seems that are not working 14:02:15 <slaweq> So please update Your LPs when You push patch related to some bug report 14:02:26 <slaweq> like set LP to be in progress 14:02:39 <slaweq> and paste link to the proposed fix in comment 14:02:52 <slaweq> and also assign LP to yourself in such case 14:03:34 <slaweq> next one 14:03:45 <slaweq> according to the https://releases.openstack.org/wallaby/schedule.html 14:03:54 <slaweq> this week we have Wallaby-1 milestone 14:04:13 <slaweq> releases of some libs are already proposed or even done for things like neutron-lib 14:04:20 <slaweq> so we should be good there 14:04:54 <slaweq> but also, I would like ask You all once again to spent some time on reviewing opened specs: https://review.opendev.org/q/project:openstack/neutron-specs+status:open 14:05:10 <slaweq> so hopefully authors of them will be able to start working on implementation soon 14:05:32 <slaweq> and that's all announcements/reminders from me for today 14:05:41 <slaweq> do You have anything else You want to share? 14:07:03 <slaweq> ok, I guess that this means "no" 14:07:07 <slaweq> so lets move on 14:07:13 <slaweq> #topic Blueprints 14:07:26 <slaweq> I moved things from W-1 to W-2 now 14:07:30 <slaweq> https://bugs.launchpad.net/neutron/+milestone/wallaby-2 14:07:44 <slaweq> and I updated today BP https://blueprints.launchpad.net/neutron/+spec/enginefacade-switch which is almost done on neutron side 14:08:16 <slaweq> last week we merged ralonsoh's patch which we though that finishes that but there were still some leftovers in the neutron code 14:08:27 <slaweq> so I pushed couple of new patches today https://review.opendev.org/q/project:openstack/neutron-specs+status:open 14:08:27 <ralonsoh> (sorry for that) 14:08:32 <bcafarel> still, worth some \o/ 14:08:32 <slaweq> ralonsoh: np at all :) 14:08:49 <slaweq> You did worst part actually 14:09:18 <slaweq> sorry, wrong link 14:09:20 <ralonsoh> btw, correct link 14:09:20 <slaweq> https://review.opendev.org/#/q/status:open+topic:bp/enginefacade-switch 14:09:20 <ralonsoh> https://review.opendev.org/q/topic:%22bp%252Fenginefacade-switch%22+(status:open%20OR%20status:merged) 14:09:25 <slaweq> thx ralonsoh :) 14:10:20 <slaweq> we still need to make same transition for the stadium projects before we will mark this BP as completed 14:10:55 <mlavalle> Great work! 14:11:07 <njohnston> Great job ralonsoh, slaweq, and everyone! The dragon is slain (at least in neutron)! 14:11:30 <slaweq> :) 14:12:59 <slaweq> regarding other BPs, I created new one https://blueprints.launchpad.net/neutron/+spec/secure-bac-roles 14:13:09 <slaweq> to track migration to secure rbac roles in Neutron 14:13:54 <slaweq> I know that mlavalle and amotoki was volunteering to work on that but recently it became high prio for Red Hat so we have more pressure to work on that and I already started sending some patches 14:14:07 <mlavalle> ok 14:14:09 <slaweq> https://review.opendev.org/q/topic:%2522secure-rbac%2522+(status:open+OR+status:merged)+project:openstack/neutron 14:14:19 <slaweq> any help with that is of course welcome :) 14:14:23 <mlavalle> ok 14:14:29 <amotoki> slaweq: sorry for the delay and thanks for starting the work. I will help it. 14:14:54 <slaweq> no need to sorry, we all have own priorities and limited time 14:15:08 <slaweq> and thx for help and valuable reviews there :) 14:15:12 <amotoki> btw, do you continue to use the current blueprint with typo. 14:15:44 <amotoki> ? 14:15:58 <slaweq> amotoki: is there way to change it? or the only way is to create new BP? 14:16:00 <slaweq> do You know? 14:16:28 <amotoki> slaweq: I will check it. otherwise we can create a new one and mark it as superceded 14:16:39 <slaweq> ok, thx 14:16:58 <amotoki> renamed it to https://blueprints.launchpad.net/neutron/+spec/secure-rbac-roles now 14:17:12 <amotoki> we can change it from Change details at the right top corner. 14:17:14 <slaweq> thx a lot 14:17:27 <slaweq> good to know 14:18:53 <slaweq> that's all updates from me regarding BPs 14:19:06 <slaweq> do You have anything else 14:19:08 <slaweq> ? 14:19:09 <mlavalle> I have an update / request regarding address groups 14:19:56 <mlavalle> We are facing some challenges updating the firewall after an address group has been updated. hangyang left a comment yesterday in https://review.opendev.org/c/openstack/neutron/+/757650 14:20:06 <mlavalle> could ralonsoh take a look? 14:20:10 <ralonsoh> sure 14:20:17 <mlavalle> Tnaks! 14:20:24 <mlavalle> that's all 14:21:11 <slaweq> ahh, conjunctions 14:21:15 <slaweq> :/ 14:21:17 <slaweq> good luck :P 14:21:45 <mlavalle> LOL 14:21:59 <lajoskatona> Another conjunction: https://earthsky.org/astronomy-essentials/great-jupiter-saturn-conjunction-dec-21-2020 14:22:20 <lajoskatona> sorry for hyjacking the meeting.... 14:22:24 <slaweq> :) 14:23:16 <bcafarel> I very much prefer the later one 14:23:45 <njohnston> lol, for sure 14:24:09 <slaweq> ok, if there are no other updates about BPs, lets move on 14:24:17 <slaweq> #topic Community Goals 14:24:39 <slaweq> regarding "Migrate RBAC Policy Format from JSON to YAML" I saw that gmann already pushed some patches 14:24:44 <slaweq> https://review.opendev.org/c/openstack/neutron/+/764401 14:24:46 <slaweq> https://review.opendev.org/c/openstack/neutron-lib/+/764416 14:24:52 <slaweq> and there is also mail http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019079.html 14:25:01 <slaweq> amotoki: You are probably aware of all of that 14:25:17 <slaweq> but just in case I wanted to paste it all here :) 14:25:19 <amotoki> yeah. I tried it and saw some unclear behavior. 14:26:31 <amotoki> I will follow it up. 14:27:12 <slaweq> thx amotoki 14:27:40 <slaweq> amotoki: if You will need any help, please ping me 14:27:47 <amotoki> slaweq: sure. thanks 14:28:17 <slaweq> so that one should be under control for now 14:28:26 <slaweq> ralonsoh: any updates about migration to privsep? 14:28:47 <ralonsoh> yes, but I'm facing some problems with one patch (maybe pyroute2 related) 14:28:57 <ralonsoh> and I have the "grenade" patch 14:28:58 <ralonsoh> (one sec) 14:29:15 <ralonsoh> https://review.opendev.org/c/openstack/neutron/+/764015 14:29:27 <ralonsoh> that will replace rootwrap, generically, with a privsep context 14:29:35 <ralonsoh> I know this is gross but effective 14:30:04 <slaweq> yeah, so kind of "one patch to migrate them all", right? 14:30:08 <ralonsoh> yes 14:30:15 <ralonsoh> but, of course, many errors still there 14:30:38 <ralonsoh> so I'll be focused first in the short ones 14:31:43 <slaweq> ok 14:31:56 <slaweq> thx for working on that 14:33:03 <slaweq> I think we can move on 14:33:06 <slaweq> next topic 14:33:11 <slaweq> #topic Bugs 14:33:19 <slaweq> obondarev was bug deputy last week 14:33:29 <slaweq> report is at http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019119.html 14:33:39 <slaweq> obondarev: any bugs You would like to bring to the team now? 14:33:48 <obondarev> not really 14:34:00 <obondarev> pretty quiet week 14:34:11 <ralonsoh> (cool) 14:34:28 <bcafarel> that was until some PTL started filling bugs today :) 14:34:35 <slaweq> yeah, I saw Your report 14:34:47 <slaweq> there is only one unassigned bug there 14:34:52 <slaweq> https://bugs.launchpad.net/neutron/+bug/1905551 14:34:54 <openstack> Launchpad bug 1905551 in neutron "functional: test_gateway_chassis_rebalance fails" [Medium,Confirmed] 14:35:19 <slaweq> related to ovn functional tests 14:35:32 <slaweq> so would be great if someone could take a look at it 14:37:29 <slaweq> ok, any other bugs You want to discuss today? 14:38:59 <slaweq> if not, I have one 14:39:10 <slaweq> I added it to the "On Demand Agenda" but we can discuss now 14:39:16 <slaweq> https://bugs.launchpad.net/neutron/+bug/1903531 14:39:18 <openstack> Launchpad bug 1903531 in neutron "Update of neutron-server breaks compatibility to previous neutron-agent version" [Critical,Confirmed] - Assigned to Slawek Kaplonski (slaweq) 14:39:45 <slaweq> after some discussion on irc and ML last week I decided that will be better to remove it from all branches 14:40:04 <slaweq> and then propose it make again to master but with properly provided backward compatibility 14:40:15 <slaweq> patches are proposed: 14:40:17 <slaweq> master - https://review.opendev.org/c/openstack/neutron/+/764189 14:40:19 <slaweq> victoria - https://review.opendev.org/c/openstack/neutron/+/764190 14:40:21 <slaweq> ussuri - https://review.opendev.org/c/openstack/neutron/+/764191 14:40:53 <slaweq> liuyulong proposed https://review.opendev.org/c/openstack/neutron/+/764108 to avoid reverting original patch from the master branch 14:42:26 <slaweq> so I wanted to discuss it once again here - should we not revert original patch from master and just go with liuyulong's additional patch? Or what is the best approach in Your opinion? 14:42:47 <slaweq> because we basically broke upgrades/updates with that patch and we have to fix it somehow 14:43:24 <ralonsoh> this is not a bad idea: to convert the message from the server and cap the version given to the agents 14:43:25 <bcafarel> well for master it does not change much, by wallaby release we should have updated API target and relevant code 14:43:36 <bcafarel> wether we revert fully and reapply or just bump 14:44:10 <bcafarel> but for stable... train to ussuri we will need old API to fix upgrade right? 14:44:34 <slaweq> for stable we need IMO to revert that patch 14:44:34 <ralonsoh> liu's patch should handle both 14:44:42 <ralonsoh> (if server is updated first) 14:44:52 <bcafarel> which means revert (breaking minor updates but can probably not be helpd, hence possible release note here) and then only fix in wallaby 14:44:53 <slaweq> as otherwise we will always have some point where we will have incompatibility 14:45:37 <slaweq> so do You think it will be good to go with revert for stable/victoria and ussuri and with liuyulong's patch for master, right? 14:45:47 <amotoki> I agree that we need to revert that patch in all stable branches. 14:46:13 <ralonsoh> yes for master, because Liu's patch can handle when server is bumped and agents not 14:46:16 <amotoki> +1 for slaweq's proposal 14:46:19 <ralonsoh> +1 14:46:34 <slaweq> ok, thx 14:47:09 <slaweq> so lets make it that way to fix it 14:47:13 <slaweq> thx for Your opinions 14:47:24 <liuyulong> Why not backport the fix to the patch original merged release? V? or U? 14:47:43 <slaweq> liuyulong: I don't think we should/can really backport things which bumps rpc version 14:47:52 <ralonsoh> exactly 14:48:07 <ralonsoh> (and we already had this problem with this bug) 14:48:16 <slaweq> this original patch should have bumped rpc version and we should never backport it to stable branches tbh 14:48:22 <liuyulong> It should be there, we missed that, so I'm OK if we can break the law once. 14:48:49 <slaweq> liuyulong: I disagree with that - we already broke that law once with that patch and made a lot of mess 14:48:55 <slaweq> we shouldn't do that again 14:49:20 <liuyulong> I'm not saying the release before V or U release. 14:49:29 <slaweq> and that's also what I got from the releases team when I discussed with them 14:49:47 <amotoki> I am not sure we cannot bump RPC version in stalbe branhces but we must keep backward compatibility between server and agent. 14:50:49 <slaweq> amotoki: liuyulong: ok, I will ask once again stable main cores about opinion on that 14:51:00 <liuyulong> The patch was in stable/ussuri natively, just checked that. 14:51:02 <slaweq> if they will tell me that we can go with that, then ok 14:51:13 <slaweq> liuyulong: yes, I know 14:51:18 <amotoki> in this case, agent is not upgraded and requests an older version, so the server should talk the older version at least. 14:51:58 <bcafarel> if you follow a correct order in upgrade yes with versioned it works OK, though if you for some reason update agents first it will complain 14:52:35 <ralonsoh> bcafarel, well, the upgrade procedure says that you should update the servers first 14:52:35 <slaweq> bcafarel: but we do support update when older agents can work with newer server 14:52:39 <slaweq> not vice versa 14:52:53 <slaweq> and we should only really care about such scenario 14:53:05 <lajoskatona> https://docs.openstack.org/project-team-guide/stable-branches.html#review-guidelines 14:53:38 <lajoskatona> here it is mentions only Nova’s internal AMQP API, but I suppose that can be true for other projects as well 14:53:39 <bcafarel> yes, so it may be acceptable (as in "security fix with proper RPC version bump") 14:54:27 <amotoki> I think we are discussing two aspects: stable backports and upgrade scenarios. 14:54:57 <amotoki> upgrade scnearios should be considered even if we are in development cycle. 14:54:59 <slaweq> amotoki: for me now the question is if we can do backport of that patch which bumps rpc version 14:55:25 <slaweq> at least for me :) 14:56:06 <amotoki> slaweq: yes you're right 14:56:13 <liuyulong> Actually the RPC was only called when use iptables_hybrid and enable ebtables. The openflow firewall driver will not call this function. Openflow firewall uses local cache directly. Maybe that's why we missed that RPC bump. 14:56:41 <slaweq> liuyulong: yes, because of that and because of lack of grenade testing 14:56:57 <amotoki> I wonder whether during stable upgrade process both agent and server can be upgrade first... If we allow to upgrade either first, we cannot upgrade RPC version at all. 14:56:59 <slaweq> we are testing grenade only with openvswitch firewall driver which is default in devstack 14:57:39 <slaweq> amotoki: in the docs we guarantee that You can update server first and newer server will be compatible to work with older agents 14:57:47 <slaweq> that's what we broke with that patch actually 14:57:56 <bcafarel> yep 14:57:57 <liuyulong> amotoki, yes, if the client side has higher version, the RPC will get failed directly. 14:58:36 <bcafarel> seeing as time is running out, at least first steps seem OK for everyone right? fixing version in master, reverting ussuri/victoria 14:58:40 <amotoki> thanks. I missed that point. 14:58:54 <bcafarel> and then discuss if this can be backported again with new rpc 14:59:02 <ralonsoh> bcafarel, I think the proposal is different now 14:59:04 <slaweq> we are almost on top of the hour now, I will ask stable-maint cores about backporting liuyulong's patch to V/U also 14:59:14 <slaweq> if that will be ok for them, I will abandon my reverts 14:59:29 <slaweq> if not, then lets revert it from those branches and go with liuyulong's patch in master 14:59:37 <slaweq> sounds good? 14:59:48 <bcafarel> ack 14:59:55 <liuyulong> OK 14:59:58 <amotoki> sounds good 15:00:04 <slaweq> ok, thx for attending the meeting 15:00:10 <slaweq> have a great week :) 15:00:13 <slaweq> #endmeeting