13:00:07 <baoli> #startmeeting PCI Passthrough
13:00:08 <openstack> Meeting started Tue Mar 11 13:00:07 2014 UTC and is due to finish in 60 minutes.  The chair is baoli. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:11 <openstack> The meeting name has been set to 'pci_passthrough'
13:00:21 <baoli> Hello
13:00:28 <irenab> Hi
13:00:28 <rkukura> hi
13:00:32 <heyongli> hi
13:00:41 <baoli> Great, everyone is here
13:01:04 <baoli> Let's start with code review
13:01:15 <irenab> ok
13:01:29 <irenab> nova or neutron first?
13:01:46 <baoli> Irenab, can we start with yours?
13:02:42 <irenab> sure
13:02:54 <baoli> My main question is whether or not we'll support non-identical vNICs (therefore multiple sriov MDs) in the same cloud
13:02:54 <irenab> Iink https://review.openstack.org/#/c/74464/
13:03:24 <irenab> I think that teoretically it can be possible, but for real cloud not sure...
13:04:26 <baoli> If that's not a concern, then I think that it's ok to check the vendor info after the agent query
13:04:28 <irenab> baoli: any reason you need the assumtion for single SRIOV MD?
13:05:04 <baoli> irenab, my comments to your change is that the vendor infor should be checked before the agent query
13:05:45 <irenab> baoli: I probably miss your point here, why the order is critical?
13:06:22 <rkukura> I think it does make sense for the MD to ignore (not bind) ports using unrecognized devices
13:06:23 <baoli> irenab, the agent query requires DB access.
13:06:50 <sadasu> baoli, irenab: from the info that I have so far, it is possible to have a cloud with sr-iov devices from diff vendors
13:07:13 <sadasu> so we need to check for vendor info before L2 agent query
13:07:33 <irenab> baoli: so the order is just for optimization, right?
13:08:19 <irenab> rkukura: any chance you can take a look at MechnismDriver at review I posted to see if looks OK or you suggest reordering?
13:08:22 <baoli> irenab, yea, it won't affect functionality
13:09:15 <rkukura> irenab: I'll give it a closer review, but at a quick glance, I think I agree with baoli and sadasu that checking the vendor 1st makes sense
13:09:31 <irenab> baoli: good comment anyway. I need to see how to take an advantage of basic agent MD logic
13:09:42 <baoli> irenab, cool
13:10:05 <baoli> Let's move on
13:10:13 <baoli> #topic port admin_state
13:10:20 <irenab> sadsu: regarding PCI vendor info check, shall we add some utils?
13:10:42 <irenab> baoli: just few more minutes on previous topic, please
13:10:48 <baoli> Sorry, sure
13:11:11 <sadasu> irenab: yes...makes sense
13:11:21 <baoli> irenab, rkukura, is it ok to have a separate base class for SRIOV?
13:11:39 <irenab> I am not sure that creating SRIOVBaseMD for Agent based makes sense
13:11:56 <rkukura> baoli: sure, but keep in mind that some use agents and some don't, so mixin might make sense
13:11:59 <sadasu> takes supported vendor info as params and checks against the port that is trying to bind
13:12:27 <baoli> or is it ok to add vendor_info the current agent base class?
13:12:50 <irenab> sadasu: ok
13:13:03 <irenab> baoli: can you please elaborate a bit
13:13:15 <sadasu> rkukura: +1 for Mixin, although I don't know of any other MD (other than mine) that does not use an agent
13:13:21 <rkukura> given we are not rushing to get this in icehouse, my general advice would be to not initially worry too much about sharing common code between SR-IOV drivers. That can be done later as a refactoring step once we see what's really common.
13:14:01 <baoli> irenab, in the current AgentBase class, we can add vendor_info
13:14:14 <baoli> rkukura, I agree.
13:14:51 <irenab> rkukura: still we joined effort here, seems we do not want to spend cycles at both sides, both for code and unittest for common parts
13:15:19 <sadasu> rkukura: agreed.
13:15:22 <rkukura> agreed that sharing code where it is obvious will reduce effort
13:15:28 <irenab> at least we can try to share
13:15:57 <irenab> seems we ca skip to next topic
13:16:09 <rkukura> maybe start with a simple mixing for common SR-IOV-specific code
13:16:15 <rkukura> s/mixing/mixin/
13:16:35 <irenab> rkukura: you had some objections for Mixin before
13:16:51 <irenab> not relevant here?
13:16:54 <sadasu> rkukura, irenab: this method of adding proposed common code to existing patches so others can take a look, is working for me
13:17:19 <sadasu> then when we converge, we can move that to a diff BP
13:17:26 <rkukura> no objection to using a mixin from me
13:17:46 <rkukura> refactoring can often be handled as a bug
13:17:56 <irenab> sadasu: I am ok with it too
13:17:57 <baoli> rkukura, I took an attempt to come up with a common SRIOV base class, take a look at this thread when you have time: RE: [openstack-dev] [nova][neutron] PCI pass-through SRIOV binding of ports
13:18:00 <sadasu> rkukura: ok
13:18:35 <irenab> I think its possible to push number of patches for same bp, so common part can be pushed separatly if needed
13:19:28 <rkukura> baoli: hadn't seen that, looking
13:19:52 <baoli> Ok then, can we move on now?
13:20:00 <irenab> sure
13:20:43 <baoli> so neutron port API supports change of port state.
13:20:54 <baoli> I checked the ovs implementation for it
13:21:23 <irenab> ovs pushed DEAD vlan for port disable
13:21:31 <baoli> it basically installed a dead vlan and a flow to drop packets to/from the port
13:21:52 <baoli> and I think the vm is not aware of the status change of the port
13:22:03 <rkukura> baoli: right
13:22:20 <irenab> I thought to use 'ip link set ' command to enforce it on VF
13:22:28 <rkukura> although there is a patch in review to push some port status changes from neutron to nova
13:22:57 <beagles> rkukura, +1 yeah that is one of the future purposes of that series of patches
13:22:59 <baoli> rkukura, that seesms to make more sense
13:23:33 <irenab> what nova action will be once get the notification?
13:23:46 <baoli> rkukura, can you share links for those patches/bugs, etc?
13:23:47 <beagles> https://review.openstack.org/#/q/status:merged+project:openstack/nova+branch:master+topic:bp/admin-event-callback-api,n,z
13:23:59 <baoli> beagles: thanks
13:24:06 <rkukura> Right now, my understanding of nova's use of this is simply to block VM boot until the ports are up.
13:24:25 <baoli> rkukura, can you change the status after the VM is up?
13:24:30 <beagles> in short the mechanisms are in place, but it is not employed everywhere it might be
13:24:53 <irenab> admin state can be changed after VM is already up
13:25:00 <rkukura> The neutron API allows the admin state to be changed any time, and I don't think nova is involved in those changes after boot
13:25:23 <baoli> rkukura, that's what I'm trying to discuss
13:25:28 <rkukura> In fact, it should be possible to create a port with admin state down, and allow nova to boot with it, and bring it up later via the API
13:25:50 <irenab> rkukura: agree
13:26:00 <baoli> The question is should the VM be aware of port status change?
13:26:15 <irenab> admin state should be enforced on port and not VIF
13:26:26 <baoli> another question is why would admin decides to change the port status?
13:27:04 <irenab> baoli: VM awareness probably depends on Vif type
13:27:14 <irenab> For SR-IOV it should be aware
13:27:46 <baoli> irenab, my question to you is that with direct passthrough, how do you admin down/up a port?
13:28:07 <rkukura> Can that be done using the PF?
13:28:07 <baoli> after it's attached to a vm
13:28:15 <irenab> probably via 'ip link set' command
13:28:35 <baoli> irenab, after it's attached, you can't do that any more
13:28:59 <baoli> it's not a ip link on the host any more, isn't it?
13:29:24 <irenab> I think you cando it via pf net device and vf index.
13:29:39 <sadasu> baoli: lets clarify that it works this way for the cisco pci passthrough ports
13:29:40 <irenab> I need to recheck, but I think it works for mellanox nics
13:30:08 <baoli> sadasu, we need to check as weel
13:30:14 <baoli> s/weel/well
13:30:30 <sadasu> baoli: ok
13:30:44 <irenab> baoli: sadasu: if it works for your case, I think you may need L2 agent to handle it
13:31:17 <sadasu> irenab: no, even L2 agent will not solve the problem
13:31:31 <sadasu> because the host looses control of the port to the VM
13:32:11 <sadasu> if 'ip link set' works then L2 agent would work
13:32:17 <rkukura> I think the model for admin state is that it just stops/starts flow of packets. The VM should not see it as a hot-plug or anything like that.
13:32:33 <sadasu> but with some initial testing 'ip link set' isn't working
13:32:53 <irenab> baoli: speaking of hot-plug, does your patch support it?
13:33:05 <sadasu> lets stay on topic
13:33:12 <baoli> irenab, not yet
13:33:38 <irenab> ok
13:33:46 <baoli> irenab, the patch can get you going with basic testing.
13:34:00 <sadasu> rkukura: so the VM will not know that the port is admined down, and may still keep sending traffic?
13:34:16 <rkukura> I think that's the model, but not positive
13:34:42 <irenab> any action items to record?
13:35:38 <baoli> #action, check ip link to see if it can change a vf state after it's attached to a vm
13:35:57 <sadasu> is it Ok to reject an admin down of the port after VM is attached?
13:36:28 <sadasu> before moving on, I had this additional q on this topic?
13:37:10 <baoli> sadasu, I'm still thinking about the question why would a admin like to change the port state after it's attached to a vm which is running.
13:37:15 <irenab> sadasu: you may add another cisco MD that talks to switches/controller to enforce admin state on physical switch
13:37:56 <irenab> baoli: it may be for the case tenant is not paying :-)
13:38:29 <sadasu> irenab: in that case we could also unbind
13:38:35 <baoli> irenab, that would be an interesting way to get the money back
13:38:56 <rkukura> how about some sort of fencing for fault isolation?
13:39:38 <baoli> rkukura, that sounds like a possibility.
13:39:44 <irenab> rkukura: looks like real case
13:39:59 <irenab> rkukura: so unbind is an option for this?
13:40:00 <rkukura> no idea if anyone is using it that way, just a thought
13:40:45 <irenab> I wanted to discuss plans for Juno summit
13:40:56 <rkukura> irenab: Changing the host_id or something to force unbinding would prevent the port from being plugged, but doesn'
13:41:12 <rkukura> doesn't necessarily do anything after the port is plugged
13:42:19 <irenab> rkukura: thanks
13:42:24 <rkukura> I don't think we should spend too much time on this right now, but I'm not sure the semantics of admin state and status are all that clear/consistent right now.
13:42:51 <baoli> rkukura, ok. let's move on
13:42:55 <rkukura> This might be something to try to get agreement on at the summit.
13:43:04 <irenab> works for me
13:43:08 <baoli> agreed
13:43:11 <sadasu> rkukura: point taken...will look into this some more
13:43:53 <irenab> the design sessions proposal is open: link http://summit.openstack.org/
13:44:48 <sadasu> we had agreed upon a joint nova/neutron session
13:45:10 <irenab> I think we need to arrange the plan and proposal
13:45:39 <irenab> seems also that there is more content for nova and maybe need one joined and one anly for nova
13:45:56 <irenab> ^only
13:46:04 <sadasu> agree
13:46:42 <irenab> we need to see how to get nova cores to sponsor this
13:47:02 <heyongli> irenab, this is the key problem.
13:47:38 <baoli> first thing first, we can add the proposed sessions first, right?
13:48:11 <irenab> baoli: I think so
13:48:16 <heyongli> i will proposed a nova common sriov support topic
13:48:21 <rkukura> lets propose beagles for nova core!
13:48:29 <irenab> rkukura: +1
13:48:33 <beagles> rkukura, don't embarass me :)
13:48:52 <beagles> it will quickly come to everyones attention how I'm so distributed that I'm not fit to be core anywhere ;)
13:49:13 <beagles> I do need to talk to some cores about some related subjects
13:49:26 <rkukura> It could be argued that nova really needs a core focusing on the networking
13:49:29 <beagles> while I can't promise anything maybe that could garner some interest
13:49:57 <beagles> rkukura, no question... for the moment it'll be proxy I guess
13:50:02 <beagles> by proxy that is
13:50:08 <beagles> (how appropriate ;))
13:51:01 <beagles> in all seriousness, I was in a nova related conversation and SR-IOV/PCI Pass through came up. I'm following up on some details
13:51:13 <irenab> for joint nova-neutron session on SRIOV what do we want to discuss?
13:51:16 <beagles> maybe if we get the summit proposal in before I have the conversation I can reference that
13:51:49 <baoli> irenab, certainly the interface between the two
13:52:34 <irenab> baoli: seems it is defined already,thanks to rkukura work on profile and vif_details
13:52:51 <irenab> do I miss something?
13:53:16 <beagles> are the use-cases the group have in mind sufficiently fleshed out to use in that or a related discussion. I'm aware (painfully one might say) that these discussions have occurred but would it be fair to say that they are "set" at the moment?
13:53:27 <rkukura> maybe a nova session on scheduling taking network connectivity (including special things like SR-IOV, QoS) into account?
13:53:29 <baoli> irenab, agreed. And also how neutron make uses of the nova pci generic support
13:53:41 <beagles> by "set" I mean established or at least tentatively agreed upon
13:54:07 <sadasu> beagles: I think that is the case w.r.t use cases
13:54:47 <irenab> rkukura: agree
13:55:12 <irenab> beagles: proably need to refresh and put together
13:55:15 <baoli> beagles, is there a IRC for the meeting you attends?
13:55:21 <sadasu> rkukura: we can keep that as a separate session...neutron aware nova scheduler
13:55:40 <beagles> sadasu, okay cool. That's one of the things want to make sure is generally "known" among at-least-a-few cores so we are all on the same page
13:56:04 <irenab> we need also the nova api related discussion, for requesting vnic_type
13:56:41 <sadasu> at this point  our "base" session should cover use cases, API changes and flavor discussion
13:56:42 <beagles> irenab, not at the moment, but this is something we might want to look at... ie. acting as a subgroup that is attached to both neutron and nova
13:57:12 <beagles> I should've done something like that for parity, but wasn't savvy and afaik, nobody has done before
13:57:42 <sadasu> that was the major point of contention for icehouse
13:58:12 <irenab> beagles: I think we need to define and get agreement how to request specific vnic_type, working via neutron port is not convenient for end users
13:58:27 <beagles> irenab, agreed
13:59:03 <irenab> shall we set some etherpad for topics to discuss and send it to ml?
13:59:07 <beagles> afaict, this is a specifc point of contention... how this kind of thing can be used in a fashion that is compatible with 'cloudness'
13:59:42 <baoli> ok, times is up.
13:59:57 <baoli> we should continue with eitherpad, and in the next meeting.
14:00:23 <irenab> baoli:  do you want to open one?
14:00:24 <baoli> in the mean time, we should add the session in the design summit.
14:00:29 <baoli> irenab, sure
14:00:44 <irenab> great, thanks
14:00:48 <baoli> thanks, everyone
14:00:56 <baoli> #endmeeting