14:00:07 <mlavalle> #startmeeting neutron_drivers
14:00:08 <openstack> Meeting started Fri Jul 12 14:00:07 2019 UTC and is due to finish in 60 minutes.  The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:11 <openstack> The meeting name has been set to 'neutron_drivers'
14:00:19 <njohnston> o/
14:01:54 <amotoki> hi
14:02:36 <mlavalle> Yamamoto underwent surgery recently, so he won't attend today's meeting
14:03:03 <haleyb> hi
14:03:10 <adrianc> Hi
14:03:22 <davidsha> Hey
14:03:32 <slaweq> hi
14:03:42 <njohnston> mlavalle: Goodness I hope it was nothing serious
14:04:36 <mlavalle> he wrote an email, so he seems to be ok
14:05:10 <mlavalle> for those interested to know, my wife's kidney transplant was a total suceess yesterday
14:05:19 <njohnston> \o/
14:05:22 <slaweq> great news :)
14:05:28 <mlavalle> and with that, let's get going
14:05:28 <amotoki> :)
14:05:48 <mlavalle> #topic RFEs
14:06:24 <mlavalle> today we are going to discuss 2 submissions from adrianc
14:06:58 <mlavalle> They are closely related. Let's start with https://bugs.launchpad.net/neutron/+bug/1834174
14:06:59 <openstack> Launchpad bug 1834174 in neutron "[RFE] Add support for IPoIB interface driver" [Wishlist,In progress] - Assigned to Adrian Chiris (adrian.chiris)
14:08:04 <mlavalle> and this is the related PoC code: https://review.opendev.org/668510
14:08:27 <slaweq> I wrote a comment in this rfe today
14:08:45 <slaweq> I would like others opinion about it :)
14:08:56 <slaweq> especially about point related to testing it in u/s
14:09:46 <njohnston> So in this case L3 is IP, and L2 is InfiniBand instead of Ethernet, and would require not just Infiniband cards in the machine stesting it but also a TOR that supports InfiniBand protocol, yes?
14:10:20 <amotoki> this RFE only mentions DHCP and L3 agents. how about nova side? Is IB used for compute nodes too?
14:10:46 <adrianc> slaweq: in regards to testing i was thinking about unit-tests and possibly adding a test to run on Mellanox CI
14:11:06 <adrianc> amotoki: yes , with SR-IOV
14:11:09 <amotoki> njohnston: Infinibad fabric would be used instead of usual ToR ethernet switches.
14:11:27 <amotoki> adrianc: so, do you need nova change too?
14:12:14 <adrianc> its basically passthrough of SR-IOV to the VM so its already supported in nova
14:12:23 <adrianc> the L2 wiring is done via vendor specific driver
14:12:53 <adrianc> https://github.com/openstack/networking-mlnx
14:14:05 <adrianc> at high level in contrast to the L2 wiring (pkey configurations - which is the equivalent of vlan)
14:14:36 <adrianc> the creation of IPoIB interfaces is somthing that is upstream in linux
14:15:29 <njohnston> So why does this code need to be in Neutron as opposed to networking-mlnx?  Help me understand.
14:15:42 <amotoki> same question hits me
14:16:48 <amotoki> I have an example from midonet. networking-midonet has its own interface driver https://github.com/openstack/networking-midonet/blob/master/setup.cfg#L73-L74
14:17:06 <amotoki> (I am not familiar with it though)
14:17:09 <adrianc> well, it can be, but in contrast to the L2 wiring, there is only one way of creating an IPoIB interface.
14:17:58 <adrianc> i agree that in its own the interface driver would be incomplete without an L2 entity (L2 agent) to configure it.
14:20:34 <mlavalle> so IPoIB is a standard: https://tools.ietf.org/html/rfc4392
14:21:27 <mlavalle> and it seems you are suggesting that implementing it might be useful for other infiniband vendors. Am I understanding correctly?
14:21:45 <adrianc> i have no strong objection in putting it as part of networking-mlnx however if another infiniband vendor comes along, he will probably need to use the same interface driver
14:22:02 <adrianc> slow typer :)
14:22:17 <adrianc> mlavalle that is correct
14:22:44 <njohnston> My only issue is that upstream gates will never have the hardware to test this effectively
14:23:01 <amotoki> Looking at https://review.opendev.org/668510, this has two points: the one is a new interface driver for IPoIB and the second is a change in the interface driver interface (i.e. to pass segmentation_id)
14:23:02 <mlavalle> do we have any evidence that there is interest from other IB vendors at this time?
14:23:26 <njohnston> Which is something that can be understood and passed over if there is a compelling reason why this part of the code has to be in the neutron repo, and can't be in networking-mlnx where the proper kinds of CI can be specified
14:23:45 <adrianc> amotoki: yes current kernel API is to provide the pkey (segmentation_id in our case) when creating the interface
14:23:57 <slaweq> njohnston++
14:24:00 <amotoki> adrianc: what is 'pkey'?
14:24:07 <amotoki> primary key?
14:24:14 <adrianc> infiniband's equivalent to vlan
14:24:18 <adrianc> partition key
14:24:20 <slaweq> adrianc: so in case when this driver is used, on host where L3/DHCP agent is running there is no any L2 agent?
14:24:32 <slaweq> which could manage e.g. segmentation_id
14:24:44 <slaweq> like it is in case of "standard" openvswitch driver for example
14:25:26 <adrianc> IPoIB interface is similar to macvtap , where you create multiple interfaces on the same physical netdev
14:26:00 <adrianc> slaweq: there is an L2 agent, someone needs to do the L2 configurations
14:26:19 <slaweq> adrianc: so why we need to pass segmentation_id to this driver than?
14:27:03 <adrianc> the current kernel API requires you to set it during interface creation
14:27:17 <adrianc> (iproute command)
14:27:44 <slaweq> ahh, I didn't know that for example :)
14:27:51 <adrianc> njohnson: mellanox CI today runs on neutron commits
14:28:10 <amotoki> yeah, the interface of interface driver like plug() is called by neutron agent. Who passes segmentation_id to plug_new?
14:28:16 <adrianc> *njohnston sorry :)
14:29:05 <slaweq> amotoki: currently this driver is used only by neutron DHCP/L3 agent to plug interfaces, everything else is done by L2 agent later in same way like for ports which belongs to vm
14:29:11 <slaweq> in this case it will be a bit changed
14:29:12 <amotoki> you can fetch segmentation_id inside the IPoIB driver as MultiInterface does if needed. I don't see any reason we need to change the interface so far.
14:30:09 <adrianc> yes it can be done in the IPoIB driver in a similart fasion as MultiInterface
14:30:46 <mlavalle> adrianc: did you see my question about evidence of other Infiniband vendors having intirest in this?
14:31:22 <adrianc> i did, i am not aware of other vendors
14:31:53 <adrianc> maybe others do ?
14:32:14 <njohnston> adrianc: Just FYI the last time I see the Mellanox CI run on any change I am a reviewer on is July 4th, there may be an issue with it.
14:32:26 <mlavalle> The way I see it, if there were other vendors with interest in this, it would make sense to place this new driver in a shared place, like the neutron repo
14:32:48 <slaweq> mlavalle: to be strict IMO shared place is neutron-lib :)
14:32:58 <adrianc> njohnston, noted thx. will take care of it
14:33:07 <mlavalle> slaweq: fair enough
14:33:51 <mlavalle> but at this point in time, I think that the best place to implement this is networking-mellanox where you have all the needed infra
14:34:40 <mlavalle> we can even start developing a spec and invite the broader community to contribute and see if in that way we drive broader interest
14:34:45 <njohnston> yes, you'd want a mellanox CI to be able to -2 a change in the gate queue if it would break IPoIB
14:35:16 <njohnston> and the best place for that is networking-mlnx
14:36:05 <mlavalle> and if there is broader interest, then we can tackle the problem of placing the code in a shared / neutral place with the proper CI infra
14:37:12 <adrianc> im fine with it as its unclear if there are other interested parties
14:37:43 <mlavalle> if at any moment other parties come along, we can reconsider
14:37:54 <mlavalle> does it make sense to everybody?
14:38:04 <adrianc> currently im working with CSIRO on it as they have an infiniband cluster
14:38:15 <adrianc> *infiniband openstack cluster
14:38:19 <njohnston> +1
14:39:05 <slaweq> +1
14:39:55 <amotoki> +1
14:40:14 <mlavalle> adrianc: this also pretty much applies to https://bugs.launchpad.net/neutron/+bug/1834176, right?
14:40:15 <openstack> Launchpad bug 1834176 in neutron "[RFE] Add support for per-physnet interface driver" [Wishlist,In progress] - Assigned to Adrian Chiris (adrian.chiris)
14:41:11 <adrianc> the main use case was enabling L3 and DHCP support for a mix of ethernet and inifiniband
14:41:22 <mlavalle> yeap
14:41:40 <adrianc> however to enable this the interface driver will require some knowledge of the network
14:41:52 <adrianc> e.g in the form of RPC
14:42:02 <adrianc> to query the physnet
14:42:12 <adrianc> and segmentation ID for IPoIB
14:42:41 <adrianc> today interface drivers dont have this ability
14:43:35 <mlavalle> but at this point, why do we need this in Neutron?
14:43:43 <adrianc> i.e a proposed change to neutron can be to allow passing an RPC client
14:44:28 <adrianc> so the out of tree interface driver can query the needed informatio
14:44:29 <amotoki> adrianc: let me clarify. "the ability" you mean is to support mix use cases of ethernet and IB, or to pass  more net info to an interface driver ?
14:44:30 <slaweq> we can accept some changes required by such driver in neutron even when driver lives in other repo IMO
14:44:51 <mlavalle> yes we can
14:45:09 <adrianc> amotoki: MultiInterfaceDriver can be out of tree for now but it will still need that RPC
14:45:11 <mlavalle> I am just trying to understand if we need to do it know
14:45:29 <mlavalle> now in Neutron^^^^
14:45:56 <adrianc> i didnt understand the last part mlavalle
14:46:43 <amotoki> as of now, DHCP/L3 agent takes an interface driver as its configuration, so it can take only one interface driver.
14:46:46 <mlavalle> adrianc: in regards to the second RFE and in light of what we discussed about the first one (IPoIB), do we need the second one?
14:46:47 <amotoki> in my understandng, adrianc tries to switch interface drivers per physnet information
14:47:29 <amotoki> s/switch/select/
14:47:48 <mlavalle> yes, I understand the same
14:48:21 <adrianc> in light of what was discussed on IPoIB, we dont need MultiInterface driver in tree, But to enable them to work they will require some information to be passed from neutron
14:49:03 <adrianc> that is in the form of an RPC client to be able to query some network infromation
14:49:12 <adrianc> will that be acceptable by the neutron community
14:49:13 <mlavalle> can we repurpose / adjust the second RFE?
14:49:36 <adrianc> yes
14:49:37 <mlavalle> and yes, in principle it would be acceptable
14:49:57 <mlavalle> if you adapt the second RFE, we can discuss it next week
14:50:10 <adrianc> ok, i will adjust the second one
14:50:10 <mlavalle> if you are willing to sacrifice a second weekend in a row
14:50:16 <adrianc> :) no worries
14:50:34 <mlavalle> for those of you who don't know, adrianc lives in Israel and it is weekend for him
14:50:58 <adrianc> sunday is also a work day for me :P
14:51:08 <mlavalle> I know that
14:51:45 <adrianc> thx for taking the time to review the RFEs
14:52:01 <mlavalle> adrianc: on the contrary, tyanks for your submissions
14:52:40 <mlavalle> Thanks everybody for attending
14:52:53 <slaweq> thx and have a great weekend :)
14:53:02 <mlavalle> have a great weekend or rest of your weekend
14:53:03 <slaweq> mlavalle: and all the best for Your wife :)
14:53:05 <amotoki> have a nice weekend ,all1
14:53:09 <adrianc> have a good weekend thanks
14:53:11 <slaweq> o/
14:53:11 <mlavalle> slaweq: Thanks!
14:53:22 <mlavalle> #endmeeting