| *** Ashana has quit IRC | 00:02 | |
| *** svenkat has joined #openstack-powervm | 00:13 | |
| *** thorst has joined #openstack-powervm | 00:19 | |
| *** jwcroppe_ has quit IRC | 00:23 | |
| thorst | efried: you still around? | 00:31 |
|---|---|---|
| *** jwcroppe has joined #openstack-powervm | 00:33 | |
| *** thorst has quit IRC | 00:40 | |
| *** thorst has joined #openstack-powervm | 00:41 | |
| *** thorst has quit IRC | 00:49 | |
| *** thorst has joined #openstack-powervm | 01:34 | |
| *** arnoldje has joined #openstack-powervm | 01:58 | |
| *** thorst has quit IRC | 03:00 | |
| *** thorst has joined #openstack-powervm | 03:00 | |
| *** thorst has quit IRC | 03:09 | |
| *** openstackgerrit has quit IRC | 03:11 | |
| *** openstackgerrit has joined #openstack-powervm | 03:11 | |
| *** apearson_ has quit IRC | 03:22 | |
| *** svenkat has quit IRC | 03:50 | |
| *** thorst has joined #openstack-powervm | 04:07 | |
| *** thorst has quit IRC | 04:14 | |
| *** tlian has joined #openstack-powervm | 04:48 | |
| *** thorst has joined #openstack-powervm | 05:12 | |
| *** thorst has quit IRC | 05:19 | |
| *** arnoldje has quit IRC | 05:20 | |
| *** erlarese has quit IRC | 05:29 | |
| *** erlarese has joined #openstack-powervm | 05:31 | |
| *** tlian has quit IRC | 06:00 | |
| *** thorst has joined #openstack-powervm | 06:16 | |
| *** thorst has quit IRC | 06:24 | |
| *** jwcroppe has quit IRC | 06:38 | |
| *** jwcroppe has joined #openstack-powervm | 06:47 | |
| *** openstackgerrit has quit IRC | 06:48 | |
| *** openstackgerrit has joined #openstack-powervm | 06:48 | |
| *** Cartoon has joined #openstack-powervm | 06:59 | |
| *** thorst has joined #openstack-powervm | 07:22 | |
| *** k0da has joined #openstack-powervm | 07:24 | |
| *** thorst has quit IRC | 07:29 | |
| *** Cartoon_ has joined #openstack-powervm | 07:32 | |
| *** Cartoon has quit IRC | 07:35 | |
| *** thorst has joined #openstack-powervm | 08:26 | |
| *** thorst has quit IRC | 08:34 | |
| *** Cartoon_ has quit IRC | 09:25 | |
| *** thorst has joined #openstack-powervm | 09:32 | |
| *** thorst has quit IRC | 09:39 | |
| *** thorst has joined #openstack-powervm | 10:35 | |
| *** thorst has quit IRC | 10:44 | |
| *** jwcroppe has quit IRC | 10:51 | |
| *** jwcroppe has joined #openstack-powervm | 10:51 | |
| *** jwcroppe has quit IRC | 11:14 | |
| *** jwcroppe has joined #openstack-powervm | 11:23 | |
| efried | esberglu, yt? | 11:41 |
| *** thorst has joined #openstack-powervm | 11:41 | |
| efried | thorst, did you need something? | 11:42 |
| *** thorst_ has joined #openstack-powervm | 11:43 | |
| efried | thorst_, did you need something? | 11:44 |
| *** thorst has quit IRC | 11:47 | |
| thorst_ | efried: I think I was pinging last night for the latest pypowervm update I pushed out | 11:47 |
| thorst_ | not urgent | 11:47 |
| *** thorst_ is now known as thorst | 11:49 | |
| efried | thorst, +1. Did you want adreznec to give it a final nod, or you want my +2 now? | 11:50 |
| thorst | efried: I'm actually going to just patch it on and see what happens. | 11:50 |
| efried | k. | 11:50 |
| thorst | I clearly also need a nova-powervm change to take advantage of it | 11:50 |
| thorst | I'm going to change all migration flows to pass both overrides | 11:50 |
| efried | Did you see https://review.openstack.org/#/c/329205/1 ? First attempt to fix up the hostname regex problem. | 11:51 |
| thorst | no, not yet | 11:51 |
| thorst | let me take a peak | 11:51 |
| thorst | I like your commit message tho | 11:51 |
| efried | It's not quiiite working. I can't figure out where else we set/retrieve the hostname of the system. | 11:51 |
| thorst | efried: yeah, I was never 100% sure that was right? | 11:51 |
| thorst | remember there is the CONF.host setting | 11:52 |
| efried | that could be it | 11:52 |
| efried | wherezat? | 11:52 |
| thorst | efried: let me find it | 11:52 |
| thorst | hmmm...is it just me or did our lab network go out? | 11:54 |
| thorst | no, worse. Our openstack host went out | 11:55 |
| thorst | esberglu: this isn't going to be fun | 11:55 |
| thorst | :-) | 11:55 |
| thorst | efried: CI is out until we get that sorted... | 11:57 |
| efried | Fine. But wouldn't mind getting closer to that fix. | 11:58 |
| thorst | efried: yep...I'm searching through the code yet... | 11:59 |
| thorst | efried: I can't find it anywhere... | 12:05 |
| efried | I couldn't either. | 12:06 |
| thorst | I want to hop on a ready node...look there | 12:06 |
| thorst | but we don't really have one of those | 12:06 |
| *** svenkat has joined #openstack-powervm | 12:07 | |
| thorst | efried: I at least know why we're having trouble with the ssh command to the VM. We're still booting the 100 MB 'zeros' image instead of a real bootable image | 12:09 |
| efried | That'd do it. | 12:09 |
| efried | For my part, I can't figure out wtf this test is doing. | 12:09 |
| efried | Indirection through dynamic impls, across the REST API, through a database. | 12:09 |
| efried | No idea where the actual code is. | 12:09 |
| efried | bbiab | 12:10 |
| thorst | efried: same. I can tell its not in the local.conf.aio (only variable there is HOSTNAME which is just the 'hostname') | 12:10 |
| *** seroyer has joined #openstack-powervm | 12:11 | |
| thorst | seroyer: Had a question for you | 12:11 |
| thorst | those overrides for 'OVS' | 12:11 |
| thorst | can I turn that on, but still use 'SEA'? | 12:12 |
| seroyer | thorst, I think so. | 12:12 |
| thorst | I'm looking to just have nova-powervm blanket always turn that override on, because the migration call doesn't really have the network info | 12:12 |
| openstackgerrit | Sridhar Venkat proposed openstack/nova-powervm: Blueprint for nova-powervm SR-IOV VIFs https://review.openstack.org/322203 | 12:21 |
| *** smatzek has joined #openstack-powervm | 12:27 | |
| *** Ashana has joined #openstack-powervm | 12:28 | |
| *** Ashana has quit IRC | 12:30 | |
| *** Ashana has joined #openstack-powervm | 12:30 | |
| *** tlian has joined #openstack-powervm | 12:31 | |
| *** seroyer has quit IRC | 13:01 | |
| *** mdrabe has joined #openstack-powervm | 13:10 | |
| *** kylek3h has joined #openstack-powervm | 13:19 | |
| efried | thorst, svenkat: I believe I've sussed what needs to be done in order to make pci_passthrough_whitelist work for us. (svenkat, get your "add to blueprint" scratchpad ready) | 13:21 |
| svenkat | efried: sure. | 13:21 |
| efried | First, initialization. We can avoid all the Linux special file checks in one of two ways. | 13:21 |
| efried | For starters, we can't use "devname" at all. There's an unavoidable special file check if we specify that guy. | 13:22 |
| efried | So we either have to use "address" (with caveats - forthcoming) or our own new field. | 13:22 |
| efried | If we use "address", we can avoid special file checks by ensuring that there's at least one wildcard in the value. | 13:23 |
| thorst | efried: following so far | 13:25 |
| efried | That is: domain:bus:slot.func <= at least one of these must be '*' | 13:26 |
| efried | MAX_FUNC = 0x7 | 13:26 |
| efried | MAX_DOMAIN = 0xFFFF | 13:26 |
| efried | MAX_BUS = 0xFF | 13:26 |
| efried | MAX_SLOT = 0x1F | 13:26 |
| efried | So for example, if we use *:bus:slot.func, we've got 0x7 + 0xFF + 0x1F bits to work with. | 13:26 |
| efried | If we cheat a little more and use domain:bus:slot.*, then we have a bunch more bits to work with. | 13:26 |
| efried | Would have to check the DRC index spec with seroyer to see how many bits we actually need. | 13:26 |
| svenkat | efried: ok… | 13:27 |
| thorst | FUNC, Domain, Bus, slot... | 13:27 |
| efried | thorst wha? | 13:27 |
| thorst | I don't know what domain and bus really mean there...but we probably don't care? | 13:28 |
| efried | The format of the address, according to the commentary in pci.py, is | 13:28 |
| efried | ["address": "[[[[<domain>]:]<bus>]:][<slot>][.[<function>]]" | 13:28 |
| efried | thorst, agree, since we won't be using a real PCI address anyway. | 13:28 |
| efried | It would have been nice to make the "bus" and "slot" encoded into the DRC index match up with the "bus" and "slot" fields of the PCI address; but if we need more bits, we can break that. | 13:29 |
| efried | The other option is to avoid both "devname" and "address" (the code will default the address to *:*:*.*), and introduce our own brand new key. | 13:30 |
| efried | If we want to do this *without* changing nova code, it will simply require us to parse the conf value afresh in our vif driver, because the existing Whitelist/PciDeviceSpec code won't know about our key. | 13:31 |
| efried | But said existing code will accept an entry thus presented. | 13:31 |
| efried | Let's come back to that decision ("address" or new key) in a sec. | 13:32 |
| thorst | k | 13:32 |
| *** seroyer has joined #openstack-powervm | 13:32 | |
| efried | The way we avoid having this stuff processed at all after the fact is simply to keep our driver's impl of get_available_resource (including our existing "build_host_resource_from_ms" method) from including a "pci_passthrough_devices" key. So no-op there. | 13:34 |
| efried | And I *think* that's it. | 13:37 |
| thorst | phewww | 13:38 |
| efried | seroyer: how many bits do I actually need out of the DRC index to make sure I can reference the phys port uniquely on the system? | 13:38 |
| seroyer | DRC index is 32 bits. You need all 32 bits. | 13:38 |
| efried | damn | 13:39 |
| seroyer | But I need to verify we have a DRC index per “physical port” if that’s really what you need. | 13:39 |
| efried | Well, the physical port definitely has a DRC index. | 13:39 |
| seroyer | I know we have one per adapter and per logical port. | 13:39 |
| efried | Are you questioning whether that index is different for each pport? | 13:39 |
| *** smatzek has quit IRC | 13:40 | |
| seroyer | Correct. I don’t see one per physical port using lshwres today. | 13:40 |
| efried | Well, doesn't matter, cause we don't have 32 bits to work with. | 13:41 |
| efried | But I think I have another solution if we still want to go the "address" route. | 13:41 |
| *** edmondsw has joined #openstack-powervm | 13:41 | |
| efried | Let me check something... | 13:41 |
| seroyer | adapter ID + port ID, but I don’t know how big those fields are. | 13:41 |
| efried | seroyer, that's what I was just about to ask. | 13:42 |
| seroyer | I would guess port ID should be smallish (like 8 bits). But not sure about adapter ID. | 13:42 |
| efried | According to the schema, the SRIOV Adapter ID is 'int', and the Physical Port ID is byte. | 13:42 |
| seroyer | How many bits do you have to work with? | 13:43 |
| efried | Max of 25 | 13:43 |
| seroyer | I’ll have to check with some folks. | 13:43 |
| efried | thanks | 13:43 |
| *** seroyer has quit IRC | 13:43 | |
| efried | seroyer: Correction, 29 bits. | 13:45 |
| efried | svenkat, thorst, I had been thinking we wanted to avoid using the SRIOV Adapter ID to build the "address", because that ID doesn't exist until you configure the card in SRIOV mode. | 13:49 |
| efried | But we need it to be configured thus anyway because otherwise the pports don't even show up. | 13:49 |
| efried | Which brings up another topic: what does the user need to do (particularly outside the auspices of nova) to get set up? | 13:50 |
| *** apearson_ has joined #openstack-powervm | 13:52 | |
| efried | Answer: | 13:55 |
| efried | 1) Set SRIOV mode, e.g. | 13:55 |
| efried | pvmctl sriov list --where mode=Dedicated -d physloc | while read physloc; do pvmctl sriov update -i physloc=$physloc -s mode=Sriov; done | 13:55 |
| efried | 2) Glean addresses (probably in relation to physlocs, which is how the customer will be thinking of which cards/ports are physically located/cabled where), e.g. | 13:55 |
| efried | pvmctl sriov list -d physloc address | 13:55 |
| efried | 3) Set up pci_passthrough_whitelist in nova.conf: | 13:55 |
| efried | pci_passthrough_whitelist = [ { "address": <address_from_above>, "physical_network": <network_name> }, ... ] | 13:55 |
| svenkat | efried: ok... | 13:56 |
| efried | That's assuming we do end up going with "address". If not, it'll just be "physloc" in steps 2 & 3. | 13:56 |
| efried | which I feel is a bit of an easier user experience - but has the disadvantage of deviating from the existing pci_passthrough_whitelist format. | 13:57 |
| svenkat | efried ok… in BP i mentioned we will use devname, i will rework words | 13:57 |
| efried | svenkat, yeah, devname won't work at all. (Just discovered that a few minutes ago, though.) | 13:58 |
| *** lmtaylor1 has joined #openstack-powervm | 13:59 | |
| efried | svenkat, may want to wait until we've landed on which route we're going to take. | 13:59 |
| svenkat | efried: yes.. sure. | 14:00 |
| *** burgerk has joined #openstack-powervm | 14:02 | |
| *** smatzek has joined #openstack-powervm | 14:03 | |
| *** arnoldje has joined #openstack-powervm | 14:04 | |
| *** seroyer has joined #openstack-powervm | 14:09 | |
| *** kriskend_ has joined #openstack-powervm | 14:14 | |
| *** kriskend has joined #openstack-powervm | 14:14 | |
| *** jwcroppe_ has joined #openstack-powervm | 14:39 | |
| *** jwcroppe has quit IRC | 14:41 | |
| *** mdrabe has quit IRC | 14:42 | |
| *** jwcroppe_ has quit IRC | 14:45 | |
| *** jwcroppe_ has joined #openstack-powervm | 14:47 | |
| *** jwcroppe has joined #openstack-powervm | 14:50 | |
| *** jwcroppe_ has quit IRC | 14:52 | |
| *** mdrabe has joined #openstack-powervm | 14:58 | |
| *** jwcroppe has quit IRC | 15:00 | |
| *** jwcroppe has joined #openstack-powervm | 15:01 | |
| *** jwcroppe_ has joined #openstack-powervm | 15:01 | |
| seroyer | efried, adapter ID is actually 16 bits, port number is 8 bits. The two added together are 24 bits. | 15:03 |
| efried | seroyer, beautiful! | 15:03 |
| *** jwcroppe has quit IRC | 15:05 | |
| efried | svenkat, thorst ^^ | 15:09 |
| efried | Domain is 16 bits, bus is 8; so the simplest would be xx:xx:*.* | 15:10 |
| *** jwcroppe has joined #openstack-powervm | 15:11 | |
| efried | Technically, adapter should be "slot" and port should be "function". But we would have to do some creative bit manipulation and put the extra bits into "domain". | 15:12 |
| svenkat | efrid: good. | 15:12 |
| efried | Which would be xx:*:xx.xx | 15:12 |
| efried | Not sure how y'all feel about that. | 15:12 |
| *** jwcroppe_ has quit IRC | 15:13 | |
| svenkat | efried: can you give an example for xx:*:xx.xx | 15:14 |
| efried | I believe *usually* the domain would be 00 in that case, because the port IDs count off monotonically from 1 on the card (I believe) and the adapter IDs should do nearly the same. | 15:14 |
| efried | svenkat, yeah, hold on a tick. | 15:15 |
| efried | d = { 'slot': sriov_adap.sriov_adap_id & 0x1F, 'func': pport.port_id & 0x7, 'domain': <leftover bits from the other two - let me get back to you on this> } | 15:19 |
| efried | address = "%(domain)x:*:%(slot)x.%(func)x" % d | 15:19 |
| efried | domain = (adapid & 0xFFE0) | (pport_id >> 3) | 15:21 |
| efried | I think that might work. | 15:21 |
| efried | Something like that, anyway. | 15:21 |
| efried | (svenkat ^^) | 15:22 |
| svenkat | efried: thanks | 15:22 |
| efried | Whereas if we went the other route, it would be | 15:23 |
| efried | address = "%(domain)x:%(bus)x:*.*" % { 'domain': adapid, 'bus': pport_id } | 15:23 |
| efried | They're both ugly for different reasons. thorst, opinion? | 15:23 |
| svenkat | i will replace discussion on devname with address with these details in BP. | 15:23 |
| efried | svenkat, not yet, wait until we've decided which route we're actually going to go. | 15:23 |
| efried | Inventing a new key is still on the table too. | 15:23 |
| svenkat | after agreement from all. i am not making changes rightaway... | 15:23 |
| svenkat | ok. | 15:24 |
| thorst | efried: trying to catch up...guilty of doing three things at once | 15:24 |
| thorst | I prefer the first but just because it captures the function in it | 15:25 |
| efried | thorst, what do you mean, "captures the function"? | 15:25 |
| efried | You mean, "the function field of the address actually has (usually all of, but at least most of) the function in it"? | 15:26 |
| thorst | the VF | 15:27 |
| openstackgerrit | Kris Kendall proposed openstack/nova-powervm: Initial LB VIF Type https://review.openstack.org/302447 | 15:28 |
| efried | Both mechanisms have the VF in them. Realistically, they'll almost always both have the same digits in them. | 15:30 |
| thorst | yeah, I honestly don't know that I understand it enough to have a full vote in the matter | 15:30 |
| efried | That is, in the vast majority of cases, it would be "0:*:x.y" vs. "x:y:*.*" | 15:30 |
| *** kriskend has quit IRC | 15:31 | |
| efried | It's just sometimes the former might look like "z:*:a.b", where one or both of 'a' and 'b' aren't actually the adapter/port ID. | 15:31 |
| efried | Which is a definite con. | 15:32 |
| efried | But it makes the adapter ID be in the "slot" field and the port ID be in the "funtion" field. | 15:32 |
| efried | thorst, the VF shows up nowhere, ever. | 15:32 |
| thorst | efried: Was just looking at this one: address = "%(domain)x:*:%(slot)x.%(func)x" % d | 15:33 |
| efried | thorst, right. As I say, *usually* domain will be 0, slot will be adapter ID, and func will be port ID. | 15:33 |
| efried | But sometimes, domain will be (unrecognizable garble of bits), and one or both of slot/func will be (just the lower bits of adapter ID / port ID) | 15:34 |
| efried | (From what I can tell, in the KVM context, 'func' corresponds to "physical function", not VF. So using 'func' for pport ID is a good parallel.) | 15:35 |
| thorst | ahh | 15:35 |
| thorst | I missed that | 15:35 |
| efried | (Just don't teel kriskend_) | 15:35 |
| efried | tell* | 15:35 |
| efried | Really, the question is, do we care more about the numbers matching up most closely with the "proper" chunks of the address; or about simplicity of code and guaranteed ability to correlate adapter & port ID at a glance? | 15:37 |
| efried | (thorst, also remember we still have the other option on the table: using our own custom key e.g. "physloc") | 15:39 |
| kriskend_ | efried I am actually ok with it in this case :-) | 15:40 |
| efried | kriskend_, it's closer than using domain for the adapter ID and bus for the port ID, anyway. | 15:41 |
| efried | thorst, this is still your call as far as I'm concerned. Want me to bring in another opinion? adreznec? | 15:54 |
| *** jwcroppe_ has joined #openstack-powervm | 15:54 | |
| thorst | efried: Can we discuss in scrum later? | 15:54 |
| efried | aaight. | 15:54 |
| thorst | I'm not heads down enough to make a valid call at the moment | 15:54 |
| efried | aaight. | 15:54 |
| adreznec | efried: Reading backscroll, but I think a live discussion would be good | 15:54 |
| efried | I will compose a concise summary and email it. | 15:55 |
| adreznec | I've been in and out as the discussion has jumped between here and slack | 15:55 |
| *** tjakobs has joined #openstack-powervm | 15:55 | |
| *** jwcroppe has quit IRC | 15:55 | |
| *** k0da has quit IRC | 15:57 | |
| thorst | efried: Just live tested https://review.openstack.org/#/c/312240/14 | 16:00 |
| thorst | failed...can you take a look at my comment? | 16:00 |
| *** thorst is now known as thorst_afk | 16:01 | |
| efried | thorst_afk, done. Should be an easy fix... for someone who understands what's called for, which I don't. | 16:08 |
| *** kriskend_ has quit IRC | 16:49 | |
| *** tblakeslee has joined #openstack-powervm | 17:01 | |
| *** tblakeslee_ has joined #openstack-powervm | 17:02 | |
| *** tblakeslee has quit IRC | 17:06 | |
| *** tblakeslee_ is now known as tblakeslee | 17:06 | |
| *** thorst_afk is now known as thorst | 17:07 | |
| *** tblakeslee has quit IRC | 17:09 | |
| openstackgerrit | Drew Thorstensen proposed openstack/nova-powervm: Support override migration flags https://review.openstack.org/329592 | 17:30 |
| *** jwcroppe has joined #openstack-powervm | 17:37 | |
| *** jwcroppe_ has quit IRC | 17:39 | |
| *** jwcroppe_ has joined #openstack-powervm | 17:41 | |
| *** jwcroppe has quit IRC | 17:42 | |
| openstackgerrit | Drew Thorstensen proposed openstack/nova-powervm: Initial LB VIF Type https://review.openstack.org/302447 | 17:48 |
| thorst | efried: can you see if you're OK with my responses above? | 17:48 |
| thorst | I know seroyer really wants that one in today | 17:48 |
| efried | thorst, ack | 17:48 |
| *** jwcroppe has joined #openstack-powervm | 17:53 | |
| efried | thorst, done. Who's the +2? | 17:54 |
| thorst | adreznec: Can you be the +2 on 302447? | 17:54 |
| thorst | and maybe also 329592? | 17:54 |
| *** jwcroppe_ has quit IRC | 17:55 | |
| *** kriskend_ has joined #openstack-powervm | 17:59 | |
| efried | thorst: "the glance image metadata can override which VIF type you use" -- it can? | 18:17 |
| efried | So we could have any number of vif drivers in play. | 18:18 |
| thorst | efried: 'kinda' | 18:22 |
| thorst | there is a difference between the edge connection technology (vNIC versus say VF) and the bridging type (ex. SEA to OVS) | 18:22 |
| efried | Yet we're using PvmVifDrivers for all of 'em. | 18:23 |
| thorst | :-D | 18:23 |
| thorst | true...true. That's because we don't have the same differences that say libvirt does | 18:23 |
| thorst | where you can pass in the card 'type' | 18:23 |
| thorst | I see VF and vNIC as effectively different 'card types' | 18:24 |
| thorst | but, I also recognize I'm stretching the truth a bit | 18:24 |
| efried | Do you (or svenkat) have some idea of how one specifies these in the configuration? I'm getting confused by the wording in the blueprint, need some context. | 18:24 |
| thorst | the card type? | 18:24 |
| efried | Uh | 18:25 |
| efried | The... | 18:25 |
| efried | ...whatever tells us which PvmVifDriver to load up. | 18:25 |
| efried | Which, as far as I can tell, is all that matters. | 18:25 |
| thorst | heh...well right now its just the vif['type'] | 18:26 |
| thorst | which is pvm_sea, ovs or bridge | 18:26 |
| thorst | which is provided solely form the neutron agent | 18:26 |
| svenkat | thorst: default is in nova conf and overwrite is in glance metadata - from your review comments. | 18:26 |
| thorst | well, in libvirt the glance metadata specifies the card type...not necessarily the vif driver. But I was proposing that vnic versus VF direct would be different vif drivers... | 18:27 |
| efried | They definitely need to be different vif drivers, because their plug/unplug methods operate totally differently. | 18:28 |
| efried | What's vif['type']? Where is that specified? | 18:28 |
| thorst | comes back from neutron | 18:28 |
| efried | And neutron gets it from....? | 18:28 |
| thorst | its a maze...I can try walking you through it after I get coffee | 18:28 |
| svenkat | can vif type can be specified when a port is pre-created, and it drives which vif driver to load to plug/unplug. | 18:28 |
| efried | At some point the user has to tell us, right? | 18:28 |
| thorst | neutron gets it from the mechanism driver | 18:28 |
| thorst | svenkat: No, the closest you get to specifying it is the 'direct' thing you get to pass in | 18:29 |
| thorst | efried: Not really...but kinda | 18:29 |
| svenkat | thorst that is the vnic_type in port you are referring to? (direct) | 18:29 |
| thorst | svenkat: yep | 18:29 |
| svenkat | vs normal for sea currently (we do not set this today, it comes in as default) | 18:30 |
| *** k0da has joined #openstack-powervm | 18:30 | |
| efried | ml2_conf.ini: mechanism_drivers | 18:30 |
| efried | ? | 18:30 |
| thorst | pretty much | 18:30 |
| svenkat | so how about we use direct as vnic _type for vf direct vif and macvtap as vnic_type for vnic. this can drive which vif driver to load during plug | 18:30 |
| thorst | based on the vnic_type, I think one of three agents can handle it. Bare metal, 'standard' (ovs, linux bridge, sea) or SR-IOV | 18:31 |
| thorst | which is why you can't mix/match ovs with SEA | 18:31 |
| efried | k, so getcher coffee. I need to get a better understanding of this. | 18:31 |
| thorst | efried: see step 3: https://wiki.openstack.org/wiki/Nova-neutron-sriov#3:_create_a_neutron_port | 18:52 |
| thorst | their term vnic != powervm vnic | 18:52 |
| thorst | but there are various vnic types. And the standard (not sure what that is) basically says 'let the standard agent decide' | 18:52 |
| thorst | which is one of "OVS, Linux Bridge, or SEA" | 18:52 |
| thorst | BEHIND the scenes, that agent generates a "VIF" type. So the SEA agent's VIF type is 'pvm_sea', the OVS agents VIF type is 'openvswitch' and the Linux Bridge agents vif type is 'bridge' | 18:53 |
| efried | thorst, we don't actually need to do this neutron port-create step for any of our stuff (old or new), right? | 18:55 |
| efried | That said, does it work if we do? Then instead of choosing a network for my instance, I could choose an existing port instead? | 18:56 |
| *** kriskend_ has quit IRC | 18:58 | |
| thorst | I think we do...otherwise it'll default to not being a direct VIF | 18:59 |
| thorst | nova will create a neutron port on your behalf...sure. But I don't think it'll create a SR-IOV neutron port on your behalf | 18:59 |
| svenkat | when you deploy a vm use "networks":[{"port":"df547f3c-d75b-427d-afdf-1159b98ca1a5”}],in the body to use aprecreataed port, use "networks":[{"uuid":"ddc24826-504d-422b-b758-a84bcbe77992","fixed_ip":"10.0.0.192"}], to use ip directly | 19:00 |
| svenkat | if used second option, a ‘normal’port will be created under the covers. which is used for sea only (currently) | 19:01 |
| svenkat | i mean vnic_type normal. | 19:01 |
| svenkat | the uuid in the second case is network id | 19:01 |
| thorst | svenkat: just to confirm my understanding. To use SR-IOV, you MUST pre-create the neutron port? | 19:03 |
| *** seroyer has quit IRC | 19:04 | |
| efried | thorst, in kvm, yes. In power, no! | 19:04 |
| efried | Power win! | 19:04 |
| svenkat | thorst: that is one option. we need to find a way to do specify ip and not port id or sr-iov. i cannot think of any automatic way as we need to support sea as well. | 19:05 |
| svenkat | how does it work for linux bridge for example. is that available along with sea in a given environment? | 19:06 |
| svenkat | and also - whats wrong with precreating a port and use it to deploy/ | 19:07 |
| thorst | svenkat: why do we NEED to do that | 19:07 |
| thorst | I see no problem with pre-creating the neutron port | 19:07 |
| svenkat | in that case i do not have an issue. we can always pre-create a port for sriov… i thought you were leaning towards specifying an ip is an expection for sriov | 19:08 |
| svenkat | so we precreate a port for sriov (use direct as vnic type - say). then how do we differentiate between vf direct vs vf-vnic | 19:09 |
| efried | thorst, what do you mean by pre-creating the neutron port? Pre as in "during plug"? Or pre as in "before you start up the driver, issue a neutron port-create command"? | 19:09 |
| thorst | before you spawn, it is very common to pre-create a neutron port. Then pass that in on the spawn | 19:10 |
| svenkat | pre as in - you create it using openstack cli or curl or rest… for powervc, UI can do precreate port and then kick off deploy - in two steps sequentially | 19:10 |
| efried | That's going to be a problem | 19:12 |
| efried | In REST/core, a VF belongs to a partition, period. You can't just create one and then assign it. | 19:12 |
| svenkat | when a neutorn port is created it is not bound to anything. it can be bound later on… | 19:13 |
| efried | So what I'm saying is there's no way to do that with SRIOV as implemented in PowerVM, as far as I know. | 19:13 |
| efried | And I'm not sure whether you could create it (e.g. attached to the NL partition) and then move it. | 19:13 |
| svenkat | whena neutron port is created, it is not attached to any nova instance | 19:14 |
| svenkat | when a port is createad it looks like : | 19:15 |
| svenkat | +---------------------+---------------------------------------------------------+ | 19:15 |
| svenkat | | Field | Value | | 19:15 |
| svenkat | +---------------------+---------------------------------------------------------+ | 19:15 |
| svenkat | | admin_state_up | True | | 19:15 |
| svenkat | | binding:host_id | | | 19:15 |
| svenkat | | binding:profile | {} | | 19:15 |
| svenkat | | binding:vif_details | {} | | 19:15 |
| svenkat | | binding:vif_type | unbound | | 19:15 |
| svenkat | | binding:vnic_type | normal | | 19:15 |
| svenkat | | created_at | 2016-05-10T17:27:54 | | 19:15 |
| svenkat | | description | | | 19:15 |
| svenkat | | device_id | | | 19:15 |
| svenkat | | device_owner | | | 19:15 |
| svenkat | | extra_dhcp_opts | | | 19:15 |
| svenkat | | fixed_ips | {"subnet_id": "0cf7a2f5-49f7-43a7-a456-0fe4adafc1ab", "ip_address": "192.168.0.2"} | | 19:15 |
| svenkat | | id | df547f3c-d75b-427d-afdf-1159b98ca1a5 | | 19:15 |
| svenkat | | mac_address | fa:16:3e:dc:5e:db | | 19:15 |
| svenkat | | name | | | 19:15 |
| svenkat | | network_id | 8622fe68-afce-435f-be65-cdaa66ef44ea | | 19:15 |
| svenkat | | status | DOWN | | 19:15 |
| svenkat | | tenant_id | f14fb833dd764ad7808390e52c83728f | | 19:15 |
| svenkat | | updated_at | 2016-05-10T17:27:54 | | 19:15 |
| svenkat | +---------------------+---------------------------------------------------------+ | 19:15 |
| svenkat | a port like this can be used while deploying a vm - currently for sea | 19:15 |
| svenkat | when it is bound it looks like : | 19:16 |
| thorst | the vif_type is what will get set on binding... | 19:16 |
| svenkat | +---------------------+---------------------------------------------------------+ | 19:16 |
| svenkat | | Field | Value | | 19:16 |
| svenkat | +---------------------+---------------------------------------------------------+ | 19:16 |
| svenkat | | admin_state_up | True | | 19:16 |
| svenkat | | binding:host_id | 828642A_21C1B6V | | 19:16 |
| svenkat | | binding:profile | {} | | 19:16 |
| svenkat | | binding:vif_details | {"port_filter": false, "ovs_hybrid_plug": false} | | 19:16 |
| svenkat | | binding:vif_type | pvc_sea | | 19:16 |
| svenkat | | binding:vnic_type | normal | | 19:16 |
| svenkat | | created_at | 2016-05-10T18:39:18 | | 19:16 |
| svenkat | | description | | | 19:16 |
| svenkat | | device_id | 68fd8a57-ad84-4a09-b114-e4d234f98d26 | | 19:16 |
| svenkat | | device_owner | compute:None | | 19:16 |
| svenkat | | extra_dhcp_opts | | | 19:16 |
| svenkat | | fixed_ips | {"subnet_id": "34459206-1ab9-4388-84ee-3f28d924cad6", "ip_address": "10.0.0.192"} | | 19:16 |
| svenkat | | id | 393cc099-b9ef-41ad-a8f9-79d60308715b | | 19:16 |
| svenkat | | mac_address | fa:71:4c:bd:2a:20 | | 19:16 |
| svenkat | | name | | | 19:16 |
| svenkat | | network_id | ddc24826-504d-422b-b758-a84bcbe77992 | | 19:16 |
| svenkat | | status | ACTIVE | | 19:16 |
| svenkat | | tenant_id | f14fb833dd764ad7808390e52c83728f | | 19:16 |
| svenkat | | updated_at | 2016-05-10T18:41:32 | | 19:16 |
| svenkat | +---------------------+---------------------------------------------------------+ | 19:16 |
| svenkat | notice mac is also updated | 19:16 |
| adreznec | svenkat: You should really use pastebin for that kind of stuff... | 19:17 |
| efried | A port can only belong to a single VM? | 19:17 |
| svenkat | ok… | 19:18 |
| svenkat | yes. a single vm | 19:18 |
| svenkat | device id above is nova instance id | 19:19 |
| efried | So I reiterate: There's no way to create an unassociated VF or vNIC. Is that a show-stopper? | 19:20 |
| svenkat | when you create a port it is not for aVF or vNIC. it is just an ‘empty’ port. it can be updated later on | 19:20 |
| svenkat | as part of deploy process (spawn or build_and_run_instance., etc | 19:21 |
| svenkat | ) | 19:21 |
| svenkat | i am talkng based on whats happening today for SEA | 19:21 |
| efried | I see. So creating the neutron port itself doesn't actually even hit our code. | 19:22 |
| efried | Until you plug it in. | 19:22 |
| svenkat | thats right… | 19:22 |
| efried | Phew. | 19:22 |
| *** jwcroppe has quit IRC | 19:22 | |
| adreznec | Yep, until you plug the VIF all that code is handled in Neutron | 19:22 |
| efried | Okay. Today the drop-down for VNIC Type in port creation has values Normal, Direct, and MacVTap. Where do those values come from? Are we talking about adding one, e.g. PowerVNIC, that will appear on that list? | 19:26 |
| *** seroyer has joined #openstack-powervm | 19:26 | |
| svenkat | direct and macvtap are for sriov.. these are vnic_types in neuton port | 19:27 |
| svenkat | pvm_sea is vif_type and proposal is to add pvm_vf and pvm_vnic for sriov as vif_types | 19:27 |
| svenkat | so to create a sriov port in devstack , i think you will end up picking direct. (for vf-direct) | 19:28 |
| svenkat | i am thinking you shold pick macvtap for vf-vnic | 19:28 |
| svenkat | or use direct for that as well, and somehow differentiate between vf-direct vs vf-vnic under the covers). that was my earlier question in this channel | 19:29 |
| *** tblakeslee has joined #openstack-powervm | 19:29 | |
| efried | I don't agree with overriding macvtap for power vnic. | 19:30 |
| svenkat | ok.. | 19:30 |
| efried | direct makes sense | 19:31 |
| efried | for direct | 19:31 |
| svenkat | isitnt macvtap conceptually same as vnic ? an intermediary on host between sriov and vm. | 19:31 |
| efried | Perhaps conceptually, but macvtap refers to an extremely specific technology. | 19:32 |
| svenkat | ok… | 19:32 |
| efried | Which our technology doesn't resemble at all. | 19:32 |
| svenkat | ok… | 19:32 |
| efried | At least, that's my understanding. I could be wrong. | 19:32 |
| svenkat | efried: when you said ‘direct makes sense, for direct’ you mean for vf-direct only or for vf-vnic also? | 19:33 |
| svenkat | just to be clear | 19:33 |
| efried | I'm certain that 'direct' makes sense for "direct VF to VM". | 19:33 |
| efried | I'm not certain (up or down) whether 'direct' makes sense for Power vNIC. | 19:33 |
| efried | It's not really direct, is it? | 19:34 |
| svenkat | ok… well, once it is setup, vm talks to vf directly isitnt? (LRDMA) | 19:34 |
| efried | Dangerous to use that as a criterion. Theoretically any of our technologies could do that. | 19:35 |
| efried | That's semantics, though, which I might be convinced to overlook. Right now I'm more interested in the practical aspects. | 19:36 |
| svenkat | i agree.. | 19:36 |
| svenkat | sounds like we need soemthing similar to ‘macvtap’ for vnic | 19:36 |
| efried | Yes, that's what I was assuming - we would add an entry to that list: Normal, Direct, MacVTap, PowerVNIC | 19:37 |
| svenkat | makes sense. | 19:38 |
| svenkat | i guess that will be a mechanism driver work | 19:38 |
| efried | This would mean that the list element in spawn's network_info would be a dict which would have a 'vif_type' key whose value would be set to whatever our new value is? | 19:39 |
| thorst | efried svenkat: one other option is we use direct for both, and a nova.conf setting indicates vnic or VF direct to VM | 19:40 |
| svenkat | could be. i have to run through spawn in pdb mode and look at networkrequest object | 19:40 |
| thorst | and you're all one way | 19:40 |
| svenkat | and glance metadata will have overwrite during runtime if other option is needed? | 19:41 |
| thorst | yeah | 19:43 |
| efried | thorst, seems reasonable, as generally I imagine the customer would want to be all vNIC. If there's some reason to use direct, it would likely be a technical one that *forces* them to do so, whereupon it would have to be across the board. | 19:43 |
| svenkat | Powervc will use vf-vnic only. | 19:44 |
| efried | Presumably direct vf performs better. | 19:44 |
| svenkat | yes. but with no failover and migration support. trade-off. | 19:45 |
| efried | Remind me why we need to support direct in the community? | 19:45 |
| efried | thorst ^^ | 19:46 |
| *** burgerk has quit IRC | 19:46 | |
| svenkat | efried: i think it is beause it is closer to what we have in communtity today. | 19:48 |
| svenkat | i will be back. | 19:48 |
| efried | If that's the only reason, I'd like to put it back on the table and challenge it. | 19:48 |
| efried | Our vNIC solution looks *enough* like KVM's direct attach, but it leapfrogs a lot of the limitations. It's simply a better technology. And it's not like there's some special extra requirement; if we can do direct, we've got everything we need to do vNIC. | 19:50 |
| efried | The inputs (phys ports on phys nets) are the same. The outputs (vif appears on VM) are the same. | 19:51 |
| *** burgerk has joined #openstack-powervm | 19:58 | |
| *** ManojK has joined #openstack-powervm | 20:27 | |
| openstackgerrit | Taylor Jakobson proposed openstack/nova-powervm: Create trunk on target of LPM https://review.openstack.org/312240 | 20:41 |
| *** ManojK has quit IRC | 20:41 | |
| *** smatzek has quit IRC | 20:49 | |
| *** svenkat has quit IRC | 20:49 | |
| openstackgerrit | Taylor Jakobson proposed openstack/nova-powervm: Create trunk on target of LPM https://review.openstack.org/312240 | 20:52 |
| *** edmondsw has quit IRC | 20:52 | |
| *** thorst has quit IRC | 20:54 | |
| *** thorst has joined #openstack-powervm | 20:55 | |
| *** ManojK has joined #openstack-powervm | 20:58 | |
| *** thorst has quit IRC | 20:59 | |
| *** smatzek has joined #openstack-powervm | 21:09 | |
| openstackgerrit | Taylor Jakobson proposed openstack/nova-powervm: Create trunk on target of LPM https://review.openstack.org/312240 | 21:12 |
| *** smatzek has quit IRC | 21:22 | |
| *** burgerk has quit IRC | 21:26 | |
| *** jwcroppe has joined #openstack-powervm | 21:38 | |
| *** lmtaylor1 has quit IRC | 21:53 | |
| *** kylek3h has quit IRC | 21:56 | |
| *** tjakobs has quit IRC | 22:08 | |
| *** ManojK has quit IRC | 22:09 | |
| *** mdrabe has quit IRC | 22:11 | |
| *** arnoldje has quit IRC | 22:16 | |
| *** jwcroppe has quit IRC | 22:20 | |
| *** openstackstatus has quit IRC | 22:25 | |
| *** openstack has joined #openstack-powervm | 22:27 | |
| *** catintheroof has quit IRC | 22:30 | |
| *** k0da has quit IRC | 23:08 | |
| *** Ashana has quit IRC | 23:20 | |
| *** seroyer has quit IRC | 23:32 | |
| *** apearson_ has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!