Monday, 2016-08-22

*** thorst has joined #openstack-powervm00:13
*** thorst has quit IRC00:17
*** thorst has joined #openstack-powervm00:21
*** thorst_ has joined #openstack-powervm00:23
*** thorst has quit IRC00:26
*** thorst_ has quit IRC01:37
*** thorst has joined #openstack-powervm02:11
*** thorst has quit IRC02:11
*** thorst has joined #openstack-powervm02:12
*** thorst has quit IRC02:20
*** cody_ has joined #openstack-powervm02:27
*** cody_ is now known as wangqwsh02:27
*** wangqwsh has quit IRC03:09
*** thorst has joined #openstack-powervm03:18
*** thorst has quit IRC03:26
*** seroyer has joined #openstack-powervm03:56
*** thorst has joined #openstack-powervm04:25
*** thorst has quit IRC04:30
*** seroyer has quit IRC04:41
*** kotra03 has joined #openstack-powervm04:51
*** kotra03 has quit IRC04:51
*** kotra03 has joined #openstack-powervm04:56
*** thorst has joined #openstack-powervm05:27
*** thorst has quit IRC05:36
*** Cartoon_ has quit IRC06:00
*** thorst has joined #openstack-powervm06:35
*** thorst has quit IRC06:41
*** Cartoon_ has joined #openstack-powervm07:09
*** kotra03 has quit IRC07:10
*** kotra03 has joined #openstack-powervm07:12
*** thorst has joined #openstack-powervm07:39
*** thorst has quit IRC07:46
*** k0da has joined #openstack-powervm08:16
*** thorst has joined #openstack-powervm08:44
*** thorst has quit IRC08:51
*** thorst has joined #openstack-powervm09:48
*** thorst has quit IRC09:55
*** Cartoon_ has quit IRC10:40
*** thorst has joined #openstack-powervm10:54
*** mdrabe has joined #openstack-powervm10:59
*** thorst has quit IRC11:01
*** openstackgerrit has quit IRC11:03
*** openstackgerrit has joined #openstack-powervm11:04
*** apearson_ has quit IRC11:31
*** openstackstatus has quit IRC11:36
*** openstackstatus has joined #openstack-powervm11:38
*** ChanServ sets mode: +v openstackstatus11:38
*** thorst has joined #openstack-powervm11:49
*** kairo has joined #openstack-powervm12:16
*** edmondsw has joined #openstack-powervm12:30
*** tblakes has joined #openstack-powervm12:42
*** tblakes has quit IRC12:45
*** edmondsw has quit IRC13:09
*** tblakes has joined #openstack-powervm13:20
*** kylek3h has joined #openstack-powervm13:23
*** adreznec has joined #openstack-powervm13:23
*** adreznec has quit IRC13:25
*** adreznec has joined #openstack-powervm13:26
*** kotra03_ has joined #openstack-powervm13:30
*** kotra03 has quit IRC13:31
*** kotra03_ is now known as kotra0313:31
*** esberglu has joined #openstack-powervm13:33
*** seroyer has joined #openstack-powervm13:37
*** tjakobs has joined #openstack-powervm13:55
*** kairo has quit IRC13:56
thorstesberglu: how's the CI looking?  I saw Hsien's stuff merged...do we need to get it deployed across the cloud?14:07
*** edmondsw has joined #openstack-powervm14:07
*** esberglu has left #openstack-powervm14:08
*** esberglu has joined #openstack-powervm14:08
esbergluthorst: Zuul died over the weekend so I restarted it this morning. Reverting the change that broke stacking merged over the weeked though, so we should be seeing some good runs today14:10
*** burgerk has joined #openstack-powervm14:10
adreznecthorst: efried Is this ready to merge in? https://review.openstack.org/#/c/322210/14:22
adreznecMostly looking for a networking-powervm change to drop in to trigger a docs run14:23
adreznecand it feels like that should already be merged14:23
efriedLet me make sure svenkat is okay with merging it.14:24
efriedadreznec, is this for the readthedocs thing?14:24
adreznecYeah14:24
adreznecThat merged in14:24
adreznecSo the next patch we push should publish docs14:25
efriedDoes that pick up docs from networking-powervm/doc/* as well?14:25
adreznecShould14:25
efriedokay.14:25
efriedI'll have one ready there soon too.14:25
adreznecAssuming our docs config is set up properly14:25
efriedDidn't you set it up?  How could it not be?14:26
adreznecefried: It will pick up anything in /doc/source14:27
efriedAs well as anything in specs/ ?14:27
adreznecefried: No, that isn't getting built into our docs. Do we want it to? We're kind of in a weird middle area14:29
adreznecWhere we have specs, but we don't have a dedicated specs repo14:29
efriedWell, the change set you mentioned above is in specs, not in doc/source.14:29
efriedUnless you're saying that any merge in the whole project will trigger a build of the docs.14:30
adreznecYeah, shouldn't matter though. The docs publishing job will publish new docs on every commit14:30
efriedGotcha.14:30
adreznecYou raise a good point though14:30
efriedsvenkat is okay with merging that blueprint.14:30
adreznecToday the only way to view our specs is in the github/git.openstack.org repos14:30
adreznecThere's no place we generate/publish the spec RST14:30
adrezneclike specs.openstack.org for example14:30
efriedSo should we move the specs, or is there a way to make them publish from where they are?14:31
adreznecProbably not a big deal, but it would be nice long term14:31
adreznecefried: We could probably update the build_sphinx part of the setup.cfg to include that somehow14:32
adreznecWould require investigation14:32
efriedadreznec, thorst - on another topic, can y'all help me get attention for https://bugs.launchpad.net/neutron/+bug/1615128 / https://review.openstack.org/#/c/358125/ ?14:32
openstackLaunchpad bug 1615128 in neutron "Custom binding:profile values not coming through" [Undecided,In progress] - Assigned to Eric Fried (efried)14:32
*** seroyer has quit IRC14:33
adreznecefried: Maybe, let me look it over. It can be hard to catch people's attention in Neutron sometimes14:34
openstackgerritTaylor Jakobson proposed openstack/nova-powervm: Add iscsi support  https://review.openstack.org/34692014:36
efriedadreznec: I found the people who had touched the surrounding code last, and called them out specifically in #openstack-neutron, but none of them responded.14:39
efriedThat was after I had put out a general call for help, both there and in #openstack-nova14:39
efriedWondering if I should hit a mailing list?14:40
openstackgerritEric Fried proposed openstack/networking-powervm: Mechanism driver & agent for powervm SR-IOV  https://review.openstack.org/34342314:42
adreznecefried: Hmm, that's really unpromising then...14:44
efriedMaybe I'm on a list somewhere.14:44
thorstefried: was this working, but is not now?14:44
*** efried1 has joined #openstack-powervm14:47
efried1Sorry, my PC crashed again.  Did I miss anything since 9:44:49?14:47
adreznec[09:44:49]  <thorst>efried: was this working, but is not now?14:48
adreznecWas the last thing14:48
efried1thorst: I have no idea if it was ever working.  I just know that when I went to test it out in my code, it didn't.14:48
thorstbut svenkat said it was...  That's the gap I'm trying to collapse...14:48
efried1When you create the port, it shows up in the output.  When you go to show the port (before you use it), it shows up in the output.14:48
efried1But when you go to *use* the port, the binding:profile values don't come through - and then when you go to show the port again, they're gone.14:48
*** efried has quit IRC14:49
thorsthmm.14:49
efried1And it's pretty obvious by following the code I called out in the above bug why that's happening.14:49
efried1hmm indeed.14:49
efried1And there may be a reason for it; and there may be some other way I'm supposed to get around it - but until I can get a SME to look at it...14:50
*** Cartoon_ has joined #openstack-powervm14:53
*** seroyer has joined #openstack-powervm14:54
*** seroyer has quit IRC14:55
*** mdrabe_ has joined #openstack-powervm15:01
*** mdrabe has quit IRC15:03
thorstefried1: yeah...I was wondering if it was that way because you can't have a binding profile until you're bound15:04
thorstbut that's not really...true.  I'm just guessing that may be the case?  Who knows when a binding profile should exist...15:04
*** seroyer has joined #openstack-powervm15:08
efried1thorst: Yeah, the binding:profile shows up when you do a neutron port-show when it's unbound.  And then when it's bound, that disappears.  That can't be right.15:08
efried1...unless it is.15:09
thorstexactly...unless it is15:10
thorstand the VLAN comes through...why?15:10
efried1Because that's not part of the binding:profile15:11
thorstefried1: but could we put this stuff we want in the binding:profile into some meta field?15:12
thorstor, is the issue that if we do that, we don't actually store it in the db15:12
thorst(I think I caught myself up...)15:12
efried1thorst, yeah, we had this discussion a couple of weeks ago, and figured out that binding:profile was the perfect place for it.15:13
thorstyep yep15:14
thorsttakes me a while to catch up15:14
efried1So we need someone from neutron to either confirm that the bug is a bug and agree to a fix (mine or whatever); or tell us we're using binding:profile wrong and (gods willing) suggest a better way for us to do what we need.15:14
thorstefried1: take a peak at 35417915:17
thorstthey've been trying for a while to get that through and while it works...I'm only a +1 on it.  I don't want to block it though because I know they need it...so you can tell me if I'm being silly (probably am)15:18
thorstand efried1...according to this:  http://docs.openstack.org/developer/networking-midonet/specs/kilo/port_binding.html15:19
thorstit seems like you're using it properly...15:19
thorstI think the issue is that you're perhaps changing the host_id?  and therefore the binding:profile gets zapped?15:19
openstackgerritMerged openstack/networking-powervm: Checkin blueprint for networking-powervm for SR-IOV VIFs support  https://review.openstack.org/32221015:22
efried1thorst: it's not the host ID that makes the profile get zapped.  Even if I was changing it, which I'm not afaik15:24
thorstyeah...you're just doing a show15:24
thorstso I think you've got a valid bug here...15:24
thorstI'd assume midonet is dead too...we should look at that plugin15:24
*** Cartoon_ has quit IRC15:25
efried1wha?15:26
thorstefried1: I'm going to push 343423 through15:26
thorstefried1: plugins besides us are trying to use binding:profile this way15:26
thorstso ... are they broken?15:26
efried1no idea.  How to find out?15:26
thorsthttps://github.com/openstack/?utf8=%E2%9C%93&query=networking-15:27
thorstonly a gruesome search ensues15:27
efried1holy...15:28
thorsthttps://github.com/openstack/networking-midonet/blob/53d51142573027a07ba3e0af2fa28bbdd84bef69/midonet/neutron/db/port_binding_db.py15:28
thorstthat's midonet, the plugin that introduced this.15:28
thorstlooks like they have to do stuff we haven't imagined?15:29
thorstnot excited if we have to do db things.15:30
efried1we shouldn't have to.  Did you look at the fix I proposed?15:31
thorstpssssh15:31
thorstno15:31
thorstseems reasonable...but does fail port binding tests15:32
thorstmay in fact be worth a mailing list thread.15:32
*** k0da has quit IRC15:37
*** kriskend has joined #openstack-powervm15:38
*** dwayne has joined #openstack-powervm15:45
efried1aw, thorst, I have no idea on https://review.openstack.org/#/c/354179/15:54
thorstwell, the rollback is fine15:55
thorstits just do we embed the rollback name in the task15:55
thorstor have the invoker pass it in15:55
efried1I'm actually liking your suggestion - save off the previous name in the execute.15:56
efried1That's the most readable and provably correct way to do it.15:56
efried1And it's only less clean because we've hidden the LPAR GET under the vm.update deal.15:57
efried1...which we have an easy remedy for, via the entry param.15:58
thorstwell16:01
thorstso you can't save it off16:01
thorstwe got into a big hub-bub16:02
thorstthe resize flow runs over many distinct manager calls down into the driver16:02
efried1why can't you save it off?16:02
thorstso by the time it hits that, its actually already named the wrong thing16:02
thorstwe went round and round on that16:02
efried1Then the un-rename shouldn't be done here.16:02
thorstwell, or a 'on failure, rename to'...16:02
efried1The un-rename should be done in the revert of whatever Task did the rename.  Or, as you say, have a totally separate "on failure" thing - but that defeats the purpose of reverts.16:03
thorstright...16:04
efried1It's tough enough to read this stuff where we've insisted on four layers of indirection (driver method => Task => nova-powervm lib => pypowervm lib) to do what could be accomplished in one.  It's one thing to have to understand how a specific Task works in order to understand it in the context of a whole flow - asking the reverse is just too much.16:07
efried1thorst tblakes mdrabe_16:07
thorstyeah.  I like bundling things in tasks to build pipelines...but I don't like them embedding too much logic about previous steps in them...16:08
thorstI'll propose up a change here in a bit.16:08
efried1I put up a review response.16:09
efried1thorst, it looks like they're replacing the binding:profile on purpose, but it's because they think the following is true:16:29
efried1       # This will happen if a new host or profile was specified that         # differs from the current one.16:29
efried1So maybe we have to figure out how it is that we're "specifying" a "new" profile.  I know we're not doing it on purpose.16:30
efried1mm, maybe it's happening through the claim.  Cause that's where we're getting the physical_network name and the other (wildcarded) values from the pci_passthrough_device we generate in the driver periodic task.16:31
*** mdrabe_ is now known as mdrabe16:32
efried1If we were gonna fix that, we would need to intercept the Claim somehow.16:32
mdrabeefried1: We don't have to save the instance name off at tall16:33
mdrabeall*16:33
mdrabeWe only change the instance name out on the hypervisor, the instance object's name stays the same the whole time16:33
mdrabeSo I guess, we should add a comment to clarify16:33
efried1At least.16:33
efried1And we should add logic to vm.rename that only does the update if it's needed.  (Compare old name to new name)16:34
mdrabeSaving off the old name is tricky16:34
mdrabeBecause the name change happens in the Rename task, not the Resize task16:35
efried1So we should be reverting it in the Rename task.16:35
efried1And not passing it in at all in the update in the Resize task.16:35
mdrabeI *think* they're called in different taskflows16:35
mdrabeLemme double check16:35
efried1Or if there's some other code path where a rename is done in Resize, then the same rename-on-revert logic should be added to both.16:36
efried1See, I'm seeing this as the kind of code change I should be able to review without having to understand all the flows involved.16:36
mdrabeheh wouldn't that be nice16:37
efried1That's kinda the whole point of a Task - a self-contained, self-reverting unit of work that stands alone.16:37
mdraberight but we gotta make sure what we *want* to do will actually get done16:37
efried1reusable, too.16:37
efried1So: Any Task that does a rename needs to revert the rename iff performed.16:37
mdrabeefried1: Here's Resize16:37
mdrabehttps://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/driver.py#L1310-L131116:38
mdrabeHere's Rename16:38
mdrabehttps://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/driver.py#L122216:38
mdrabe2 different flows16:38
mdrabeIf Resize fails, Rename doesn't get reverted16:38
mdrabeRegardless, we don't need to do any saving off16:38
thorstmdrabe: I'll propose up a second option16:39
mdrabeThe instance object's name property reflects the hypervisor's name as it should be16:39
mdrabethorst, efried1: tblakes and I stepped through this code, debugged it16:39
efried1Should it or should it not be the case that both Resize and Rename revert the name change?16:40
efried1...if and only if one occurred16:40
thorstyeah, I dont' disagree with anything you've done so far.  And I agree it'll work.  What I don't like is that if someone else comes in and uses it...its kinda odd.  It works for everything we have today, but kinda breaks the task contract.16:40
efried1+116:40
thorstso its totally a "this isn't part of the task really"16:41
thorstsince we've gone back and forth so much....give me 20 min...see if my proposal is ok enough.16:41
mdrabefloor's yours16:42
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-powervm: Added revert for Resize  https://review.openstack.org/35417916:54
thorstmdrabe: ^^ something kinda like that16:54
thorstis what I had in mind...over complex, yeah a bit...but holds the contract a bit better...16:54
thorstefried1: you should look too.16:56
mdrabetblakes: When you were debuggin this resize/rename stuff, did you notice ever that the VM's name would be resize_resize_<name> on the hypervisor?17:11
mdrabethorst: Do we even know why we have to rename the LPAR's name?17:12
tblakesmdrabe: Yes, if the resize failed multiple times you could end up with multiple resize_ prepended.17:13
thorstmdrabe: kylek3h set that up.  I think its because that is in line with the libvirt driver17:16
efried1thorst, mdrabe: I thought it had something to do with not being allowed to have two VMs with the same name.  And one of the flows involved two "copies" of the VM existing at the same time.17:19
thorstahh, yeah17:20
thorstresize on same host17:20
mdrabeIs that a libvirt thing only? Or PowerVM makes this new partition when "resizing" as well?17:20
thorstmdrabe: I thought it was a compute manager thing17:21
thorstbut again, we'd need to bounce off kylek3h17:21
mdrabenevermind "We use the resize name so we don't destroy it on a revert when it's on the same host"17:21
thorstso does the revert you have proposed cause an issue?17:22
mdrabeYes, but looking into the fact the vm.rename appears to be called twice for a resize right now...17:23
mdrabethorst: Your review made me notice that Resize also renames the VM17:24
thorstmdrabe: right...17:25
thorstthis isn't the rename task we're talking about here.17:25
mdrabeOooh I see why it's not 'resize_resize_'17:26
mdrabebecause it's always keying off the instance.name17:26
mdrabeSo renaming the LPAR in the Resize task is redundant...17:27
efried1thorst, still think we're making this way too complicated.17:27
thorstefried1: I wouldn't doubt that...17:28
efried1You're not losing anything (processing-wise) by saving off the original name during execute.  (If you even need to - if mdrabe's argument is correct, and instance.name is always the right thing to un-rename to, then you *really* don't lose anything.)  And quite simply, if Rename renames, it should un-rename on revert; and if Resize renames, it should un-rename on revert.  How can it need to be more complicated than that?17:29
efried1Show me where there's even an extra GET.17:29
efried1There isn't.17:30
mdrabeefried1: Yea I agree with that except I think we can just use instance.name in the revert17:31
efried1mdrabe: The more I think about it, although that may be provably correct in the existing flows, it would still be more readable/clean/provably-correct to save off the old name during execute.17:32
efried1and make a minor improvement to vm.rename that only updates if the name is different.17:32
efried1see, then we even *improve* the performance ;-)17:33
mdrabeMaybe, Although I think the redundant name change just happens as part of the overall 'resize' PUT17:36
mdrabeor POST17:36
mdrabebut it's still redundant17:36
thorstso remember...that the resize is a multi step call from the manager to the driver17:37
thorstthat's what tjakobs pointed out to me17:37
efried1On resize we use vm.update, wherein we don't have an easy way to tell whether anything changed in the wrapper (because of LPARBuilder).  We could add the condition into the bit where we set the name, but that doesn't save us anything really.17:37
thorstbut the time the task is even initialized, some previous call from the manager has already renamed it17:37
thorstand your driver has no context of the 'original' name, because its outside of its process scope17:37
thorstthat original name may have been on a different host.17:37
efried1But in your revert, you were using vm.rename - the *only* thing that guy changes is the name and then POST.17:38
mdrabeExcept we always know what the original name should be: instance.name17:39
mdrabeWhat really needs to be made clear is that we're only changing the name out on the hypervisor17:40
efried1okay, that's an argument that could convince me that we should use instance.name instead of saving/restoring - as long as there's a very clear comment explaining why that's the case.17:40
efried1Now, is it *always* the case that we should be un-renaming if Resize fails?17:41
mdrabeIf we're saving off and restoring instance.name just for the sake of clarity, it also gives the impression that we're appeasing the taskflow, instead of the openstack dom17:41
mdrabeefried1: We can always use instance.name all the time (I say that with 90% confidence)17:41
efried1mdrabe, I'm asking if there's ever a case where we *don't* want to remove the resize_ prefix when the flow fails.17:42
mdrabeoh right good question17:42
efried1e.g. if doing so would (attempt to) result in two LPARs of the same name on the same system.17:42
thorsthow about for now, we just embed a really solid comment17:43
mdrabeefried1: That comment I mentioned earlier now concerns me17:43
thorstand cross that bridge when we get to it?17:43
efried1cross which bridge?  You mean potentially inject a different bug and wait to solve it until it's noticed??17:44
thorstefried1: well it is passing tempest  ;-)17:44
efried1Was this code change prompted by a tempest failure?17:45
thorstno, not sure what caused this actually17:45
tblakesNot directly. Manas changed an error message to use vm.name in a printout and noticed that it had resize_ prepended to it17:46
mdrabethorst, efried1: Welp I think all this discussion may have been for nothing...17:46
mdrabeI think we want to keep resize_ prepended in the event of a resize failure17:46
efried1Right, so if it's passing tempest, that means nothing, because clearly tempest doesn't have a test case that's hitting these edges.17:46
efried1mdrabe, do tell17:46
thorstmdrabe: maybe a failed resize should clean up the VM?17:47
mdrabeBecause, this is based off a comment, "resize_" being prepended is how openstack knows not to delete the VM17:47
thorstif its a copy of the original?17:47
mdrabeThe rabbit hole's all over the place with this, not entirely sure what the purpose of "resize_" is just yet17:49
mdrabethorst, efried1: Looks like driver.destroy gets called in the event of a resize failure: https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/driver.py#L708-L71617:51
thorstmdrabe: yeah...I had seen that...17:51
thorstthere was also some finish-resize-revert flow too17:52
mdrabeSounds like the "resize_" VM is the correct VM....17:52
mdrabeOh but this is the API17:52
mdrabenot something that triggers automatically if resize fails17:52
*** kotra03 has quit IRC18:33
*** apearson_ has joined #openstack-powervm19:10
*** burgerk has quit IRC19:30
*** k0da has joined #openstack-powervm19:38
*** burgerk has joined #openstack-powervm19:58
*** burgerk has quit IRC20:01
efried1thorst: this new_vif thing - should new_vif=False mean SRIOV plug becomes a no-op?20:05
thorstyes20:05
efried1thorst, or perhaps this is how we effect an "update"?20:06
efried1(in the future)20:06
thorstit depends on where the update comes from20:07
efried1thorst, fyi, finally got some traction: http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-neutron.2016-08-22.log.html#t2016-08-22T18:03:5120:08
efried1Also may have found a better place to fix -  about to test.20:08
*** apearson_ has quit IRC20:32
openstackgerritTaylor Jakobson proposed openstack/nova-powervm: Add iscsi support  https://review.openstack.org/34692020:45
*** apearson_ has joined #openstack-powervm21:00
*** tblakes has quit IRC21:02
*** kriskend has quit IRC21:19
*** thorst has quit IRC21:20
*** kriskend has joined #openstack-powervm21:21
*** apearson_ has quit IRC21:36
*** apearson_ has joined #openstack-powervm21:41
*** k0da has quit IRC21:48
*** tjakobs has quit IRC21:53
*** esberglu has quit IRC21:56
*** kylek3h has quit IRC22:07
*** edmondsw has quit IRC22:10
openstackgerritEric Fried proposed openstack/nova-powervm: VIF driver implementation for SR-IOV  https://review.openstack.org/34341922:10
*** kriskend has quit IRC22:12
*** apearson_ has quit IRC22:35
*** seroyer has quit IRC22:44
*** miltonm has quit IRC23:39
*** apearson_ has joined #openstack-powervm23:43
*** thorst has joined #openstack-powervm23:44
*** thorst_ has joined #openstack-powervm23:55
*** miltonm has joined #openstack-powervm23:56
*** thorst has quit IRC23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!