14:00:07 #startmeeting neutron_drivers 14:00:08 Meeting started Fri Jul 12 14:00:07 2019 UTC and is due to finish in 60 minutes. The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:09 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:11 The meeting name has been set to 'neutron_drivers' 14:00:19 o/ 14:01:54 hi 14:02:36 Yamamoto underwent surgery recently, so he won't attend today's meeting 14:03:03 hi 14:03:10 Hi 14:03:22 Hey 14:03:32 hi 14:03:42 mlavalle: Goodness I hope it was nothing serious 14:04:36 he wrote an email, so he seems to be ok 14:05:10 for those interested to know, my wife's kidney transplant was a total suceess yesterday 14:05:19 \o/ 14:05:22 great news :) 14:05:28 and with that, let's get going 14:05:28 :) 14:05:48 #topic RFEs 14:06:24 today we are going to discuss 2 submissions from adrianc 14:06:58 They are closely related. Let's start with https://bugs.launchpad.net/neutron/+bug/1834174 14:06:59 Launchpad bug 1834174 in neutron "[RFE] Add support for IPoIB interface driver" [Wishlist,In progress] - Assigned to Adrian Chiris (adrian.chiris) 14:08:04 and this is the related PoC code: https://review.opendev.org/668510 14:08:27 I wrote a comment in this rfe today 14:08:45 I would like others opinion about it :) 14:08:56 especially about point related to testing it in u/s 14:09:46 So in this case L3 is IP, and L2 is InfiniBand instead of Ethernet, and would require not just Infiniband cards in the machine stesting it but also a TOR that supports InfiniBand protocol, yes? 14:10:20 this RFE only mentions DHCP and L3 agents. how about nova side? Is IB used for compute nodes too? 14:10:46 slaweq: in regards to testing i was thinking about unit-tests and possibly adding a test to run on Mellanox CI 14:11:06 amotoki: yes , with SR-IOV 14:11:09 njohnston: Infinibad fabric would be used instead of usual ToR ethernet switches. 14:11:27 adrianc: so, do you need nova change too? 14:12:14 its basically passthrough of SR-IOV to the VM so its already supported in nova 14:12:23 the L2 wiring is done via vendor specific driver 14:12:53 https://github.com/openstack/networking-mlnx 14:14:05 at high level in contrast to the L2 wiring (pkey configurations - which is the equivalent of vlan) 14:14:36 the creation of IPoIB interfaces is somthing that is upstream in linux 14:15:29 So why does this code need to be in Neutron as opposed to networking-mlnx? Help me understand. 14:15:42 same question hits me 14:16:48 I have an example from midonet. networking-midonet has its own interface driver https://github.com/openstack/networking-midonet/blob/master/setup.cfg#L73-L74 14:17:06 (I am not familiar with it though) 14:17:09 well, it can be, but in contrast to the L2 wiring, there is only one way of creating an IPoIB interface. 14:17:58 i agree that in its own the interface driver would be incomplete without an L2 entity (L2 agent) to configure it. 14:20:34 so IPoIB is a standard: https://tools.ietf.org/html/rfc4392 14:21:27 and it seems you are suggesting that implementing it might be useful for other infiniband vendors. Am I understanding correctly? 14:21:45 i have no strong objection in putting it as part of networking-mlnx however if another infiniband vendor comes along, he will probably need to use the same interface driver 14:22:02 slow typer :) 14:22:17 mlavalle that is correct 14:22:44 My only issue is that upstream gates will never have the hardware to test this effectively 14:23:01 Looking at https://review.opendev.org/668510, this has two points: the one is a new interface driver for IPoIB and the second is a change in the interface driver interface (i.e. to pass segmentation_id) 14:23:02 do we have any evidence that there is interest from other IB vendors at this time? 14:23:26 Which is something that can be understood and passed over if there is a compelling reason why this part of the code has to be in the neutron repo, and can't be in networking-mlnx where the proper kinds of CI can be specified 14:23:45 amotoki: yes current kernel API is to provide the pkey (segmentation_id in our case) when creating the interface 14:23:57 njohnston++ 14:24:00 adrianc: what is 'pkey'? 14:24:07 primary key? 14:24:14 infiniband's equivalent to vlan 14:24:18 partition key 14:24:20 adrianc: so in case when this driver is used, on host where L3/DHCP agent is running there is no any L2 agent? 14:24:32 which could manage e.g. segmentation_id 14:24:44 like it is in case of "standard" openvswitch driver for example 14:25:26 IPoIB interface is similar to macvtap , where you create multiple interfaces on the same physical netdev 14:26:00 slaweq: there is an L2 agent, someone needs to do the L2 configurations 14:26:19 adrianc: so why we need to pass segmentation_id to this driver than? 14:27:03 the current kernel API requires you to set it during interface creation 14:27:17 (iproute command) 14:27:44 ahh, I didn't know that for example :) 14:27:51 njohnson: mellanox CI today runs on neutron commits 14:28:10 yeah, the interface of interface driver like plug() is called by neutron agent. Who passes segmentation_id to plug_new? 14:28:16 *njohnston sorry :) 14:29:05 amotoki: currently this driver is used only by neutron DHCP/L3 agent to plug interfaces, everything else is done by L2 agent later in same way like for ports which belongs to vm 14:29:11 in this case it will be a bit changed 14:29:12 you can fetch segmentation_id inside the IPoIB driver as MultiInterface does if needed. I don't see any reason we need to change the interface so far. 14:30:09 yes it can be done in the IPoIB driver in a similart fasion as MultiInterface 14:30:46 adrianc: did you see my question about evidence of other Infiniband vendors having intirest in this? 14:31:22 i did, i am not aware of other vendors 14:31:53 maybe others do ? 14:32:14 adrianc: Just FYI the last time I see the Mellanox CI run on any change I am a reviewer on is July 4th, there may be an issue with it. 14:32:26 The way I see it, if there were other vendors with interest in this, it would make sense to place this new driver in a shared place, like the neutron repo 14:32:48 mlavalle: to be strict IMO shared place is neutron-lib :) 14:32:58 njohnston, noted thx. will take care of it 14:33:07 slaweq: fair enough 14:33:51 but at this point in time, I think that the best place to implement this is networking-mellanox where you have all the needed infra 14:34:40 we can even start developing a spec and invite the broader community to contribute and see if in that way we drive broader interest 14:34:45 yes, you'd want a mellanox CI to be able to -2 a change in the gate queue if it would break IPoIB 14:35:16 and the best place for that is networking-mlnx 14:36:05 and if there is broader interest, then we can tackle the problem of placing the code in a shared / neutral place with the proper CI infra 14:37:12 im fine with it as its unclear if there are other interested parties 14:37:43 if at any moment other parties come along, we can reconsider 14:37:54 does it make sense to everybody? 14:38:04 currently im working with CSIRO on it as they have an infiniband cluster 14:38:15 *infiniband openstack cluster 14:38:19 +1 14:39:05 +1 14:39:55 +1 14:40:14 adrianc: this also pretty much applies to https://bugs.launchpad.net/neutron/+bug/1834176, right? 14:40:15 Launchpad bug 1834176 in neutron "[RFE] Add support for per-physnet interface driver" [Wishlist,In progress] - Assigned to Adrian Chiris (adrian.chiris) 14:41:11 the main use case was enabling L3 and DHCP support for a mix of ethernet and inifiniband 14:41:22 yeap 14:41:40 however to enable this the interface driver will require some knowledge of the network 14:41:52 e.g in the form of RPC 14:42:02 to query the physnet 14:42:12 and segmentation ID for IPoIB 14:42:41 today interface drivers dont have this ability 14:43:35 but at this point, why do we need this in Neutron? 14:43:43 i.e a proposed change to neutron can be to allow passing an RPC client 14:44:28 so the out of tree interface driver can query the needed informatio 14:44:29 adrianc: let me clarify. "the ability" you mean is to support mix use cases of ethernet and IB, or to pass more net info to an interface driver ? 14:44:30 we can accept some changes required by such driver in neutron even when driver lives in other repo IMO 14:44:51 yes we can 14:45:09 amotoki: MultiInterfaceDriver can be out of tree for now but it will still need that RPC 14:45:11 I am just trying to understand if we need to do it know 14:45:29 now in Neutron^^^^ 14:45:56 i didnt understand the last part mlavalle 14:46:43 as of now, DHCP/L3 agent takes an interface driver as its configuration, so it can take only one interface driver. 14:46:46 adrianc: in regards to the second RFE and in light of what we discussed about the first one (IPoIB), do we need the second one? 14:46:47 in my understandng, adrianc tries to switch interface drivers per physnet information 14:47:29 s/switch/select/ 14:47:48 yes, I understand the same 14:48:21 in light of what was discussed on IPoIB, we dont need MultiInterface driver in tree, But to enable them to work they will require some information to be passed from neutron 14:49:03 that is in the form of an RPC client to be able to query some network infromation 14:49:12 will that be acceptable by the neutron community 14:49:13 can we repurpose / adjust the second RFE? 14:49:36 yes 14:49:37 and yes, in principle it would be acceptable 14:49:57 if you adapt the second RFE, we can discuss it next week 14:50:10 ok, i will adjust the second one 14:50:10 if you are willing to sacrifice a second weekend in a row 14:50:16 :) no worries 14:50:34 for those of you who don't know, adrianc lives in Israel and it is weekend for him 14:50:58 sunday is also a work day for me :P 14:51:08 I know that 14:51:45 thx for taking the time to review the RFEs 14:52:01 adrianc: on the contrary, tyanks for your submissions 14:52:40 Thanks everybody for attending 14:52:53 thx and have a great weekend :) 14:53:02 have a great weekend or rest of your weekend 14:53:03 mlavalle: and all the best for Your wife :) 14:53:05 have a nice weekend ,all1 14:53:09 have a good weekend thanks 14:53:11 o/ 14:53:11 slaweq: Thanks! 14:53:22 #endmeeting