17:01:55 <hartsocks> #startmeeting vmwareapi
17:01:56 <openstack> Meeting started Wed Mar 26 17:01:55 2014 UTC and is due to finish in 60 minutes.  The chair is hartsocks. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:01:59 <openstack> The meeting name has been set to 'vmwareapi'
17:02:31 <hartsocks> Greetings! As new thing… I'd like folks to introduce themselves and state what they are actively working on right now…
17:02:57 <hartsocks> I'm Shawn Hartsock and I'm currently coordinating https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/vmware-spawn-refactor,n,z with several other developers.
17:03:07 <mdbooth> mdbooth: Red Hat, currently flitting about
17:03:09 <browne> hi.  Eric from VMware.  Working on bugs
17:03:19 <jay-lau-513> harksocks: Jay Lau, I want to discuss something related to vmware live migration on esx server level even with one nova compute, I'm from IBM
17:03:33 <tjones1> tjones working on backports and bug coordination
17:03:46 <sabari> Hey All
17:03:53 <vincent_hou> Vincent  Hou from IBM.
17:03:53 <rgerganov> I am Rado Gerganov, I am currently working on bugs and refactoring
17:04:25 <vincent_hou> Working on resource pool support.
17:06:06 <hartsocks> garyk: ping?
17:06:49 <garyk> hartsocks: i am on a call - i'll join when i am done
17:07:04 <jay-lau-513> do we have plan talk about https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory ? This can help bug https://bugs.launchpad.net/nova/+bug/1192192 which is related to live migration
17:07:07 <uvirtbot> Launchpad bug 1192192 in nova "Nova initiated Live Migration regression for vmware VCDriver" [Medium,Confirmed]
17:07:33 <hartsocks> jay-lau-513: I think we can start to address some of this this week.
17:07:45 <hartsocks> jay-lau-513: at least in discussion...
17:07:52 <sabari> jay-lau-513: I missed getting back to you last week, sorry about that.
17:07:53 <jay-lau-513> harksocks: great
17:07:54 <vuil> Vui from VMware, working on bugs, refactoring, vSAN/ova goodness
17:08:18 <jay-lau-513> sabari: no problem. I append some of my ideas on the bug 1192192
17:08:22 <uvirtbot> Launchpad bug 1192192 in nova "Nova initiated Live Migration regression for vmware VCDriver" [Medium,Confirmed] https://launchpad.net/bugs/1192192
17:08:34 <hartsocks> #topic agenda
17:08:35 <sabari> jay-lau-513: I am taking a look.
17:08:42 <hartsocks> #link https://wiki.openstack.org/wiki/Meetings/VMwareAPI#Agenda
17:08:45 <jay-lau-513> sabari: ok
17:09:14 <hartsocks> Typically we discuss in-flight blueprints and then bugs and then open discussion.
17:09:21 <sabari> Yes, live-migration has the same issue as cold-migration as the node information is not passed to the virt layer
17:10:00 <sabari> I proposed a patch for cold-migration, I agree that we should do something similar for live-mig.
17:10:01 <hartsocks> Because icehouse is effectively closed for blueprints, it's time to start discussing Juno plans.
17:10:24 <sabari> oh okay sorry about that. I will speak up when it's time for bugs
17:10:25 * hartsocks waits for channel to clear
17:10:33 <sabari> *cleared*
17:10:45 <hartsocks> :-)
17:11:09 <jay-lau-513> https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory , any plan for this one?
17:11:17 <hartsocks> A few months ago mdbooth posted this http://lists.openstack.org/pipermail/openstack-dev/2014-February/028077.html it's an analysis of patches in flight and our driver sits at 24% in flight.
17:11:33 <hartsocks> jay-lau-513: I will get to that.
17:11:58 <jay-lau-513> ok
17:12:01 <hartsocks> I think garyk is one of the only developers from this group that has successfully merged blueprints. The rest of us have not had success.
17:12:40 <tjones1> :-(
17:12:49 <hartsocks> I think we can all learn from garyk's example.
17:12:55 <hartsocks> I know a saying…
17:13:18 <hartsocks> "a fool learns from experience, a wise man learns from history" and I have plenty experience with both.
17:13:36 <hartsocks> I would like us as a group to learn from garyk's example. In particular, I would like to call out...
17:13:43 <garyk> n boer maak n plan - afrikaans - for a farmer will always make a plan
17:13:43 <hartsocks> #link https://review.openstack.org/#/c/56416/
17:14:02 <hartsocks> jay-lau-513 … this is part of getting to things.
17:14:27 <hartsocks> That patch was in flight from November 14th through March 12th.
17:14:50 <hartsocks> I'm to understand garyk worked day and night on that patch. And I'm sure he worked at great personal sacrifice.
17:15:09 <hartsocks> There are 73 revisions on that patch.
17:15:43 <hartsocks> There appear to be as many as 3 or 4 revisions *per day*
17:16:31 <garyk> hartsocks: i am not sure that i understand the point here.
17:16:37 <hartsocks> I'm hoping garyk is available to share some of his experiences on this work… but primarily…
17:16:42 <garyk> originally the BP was blocked because it was not generic -
17:16:43 <hartsocks> I hope that neither garyk nor any of the rest of us have to work that hard again.
17:16:51 <hartsocks> garyk: go on.
17:16:53 <garyk> rebascing here was around 30 iterations
17:17:15 <garyk> basically the fact that we got review cycles late in the game was a cause of problem.
17:17:22 <garyk> and the fact that i am a bad programmer :)
17:17:57 <garyk> sorry for interrupting. please continue
17:18:06 <hartsocks> garyk: I would never say that.
17:18:14 <tjones1> garyk - hardly!
17:18:23 <hartsocks> garyk: no, I actually was summarizing while you were on a call.
17:18:34 <garyk> in hebrew they call me - nezek (damage)
17:18:49 <hartsocks> garyk: I wanted to highlight your effort. And give you a chance to voice/complain about how that went so we could all learn from it.
17:19:16 <garyk> thank you
17:19:45 <hartsocks> garyk: no matter what other nazek I can't argue that by any measure you are our only successful developer for icehouse.
17:20:00 <hartsocks> garyk: :-)
17:20:17 <rgerganov> garyk, how would you compare merging a blueprint and running 50k?
17:20:32 <garyk> the blueprint is harder
17:20:38 <rgerganov> just kidding :) you are a hero
17:20:42 <hartsocks> garyk: so I noticed you went through as many as 3 or 4 revisions per day at the end… why do you think that happened?
17:21:21 <garyk> there were either pep8 failures/comments that were addressed from reviewrer
17:22:16 <hartsocks> garyk: I've finally figured out my own pep8 breakage and maybe that's one small thing we can do for the OpenStack community at large…
17:22:32 <hartsocks> garyk: document how to properly avoid posting things with silly pep8 errors and things like that.
17:22:52 <tjones1> i always forget to run it and then get embarassed
17:23:07 <mdbooth> tjones1: +1
17:23:17 <hartsocks> Maybe if we had a pre-review tool that helped us check everything?
17:23:23 <browne> it might be useful if there was a check that prevent a git review without running local test first
17:23:30 <hartsocks> :-)
17:24:03 <tjones1> i wish tox were faster
17:24:09 <hartsocks> That's probably something that could help everyone and avoid making multiple posts in a row. I've been really bad about that myself.
17:24:25 <garyk> big reviews are also very problematic. a reviewer may see something in a second iteration that she/he did not see in the first
17:24:54 <mdbooth> hartsocks: In the grand scheme, though, pep8 failures are only an annoyance
17:25:13 <garyk> small is better, but no idea how to do a small on for som efatures that are more than 10 lines of code (i have seen patches for big features done in piece meal), they were very hard to follow and at time added and deleted code 3 pateches later...
17:25:27 <hartsocks> mdbooth: agreed, but they do cause spam on -nova and maybe people ignore our work if they think we "cry wolf" too much?
17:26:15 <hartsocks> garyk: I've seen some work done on the main nova branch that adds a hunk of code… but it's not used… just kind of placed there.
17:26:28 <hartsocks> garyk: then later a number of patches start applying the new thing.
17:26:48 <mdbooth> As long as the commit message explains why, I like that sort of commit
17:27:46 <mdbooth> I don't think there's a firm rule here, but the principal that smaller is usually better is a good one imho
17:27:50 <garyk> commit message here was very specific : aging support :)
17:27:59 <garyk> one liner :)
17:28:43 <mdbooth> hartsocks: I have a general comment on reviews if now's a good time
17:28:45 <rgerganov> submitting code that is not used is wrong to me; how do you define an interface/utility/whatever with no clients?
17:28:50 <hartsocks> well, I'll leave everyone with doing their own dissection of the history of that patch and we can discuss conclusions on the mailing list.
17:29:13 <hartsocks> mdbooth: sure… I'm about to launch into another topic for the group to consider.
17:29:21 <mdbooth> Comments!
17:29:34 <mdbooth> Lots and lots of comments. Big design comments.
17:29:46 <mdbooth> If a bit of code looks weird, why does it look weird?
17:30:02 <hartsocks> *lol* I tend to write small books in my patches but I've never managed to merge a feature yet.
17:30:04 <mdbooth> There's no need to comment code flow
17:30:16 <mdbooth> Not extensively, anyay
17:30:33 <mdbooth> But it's vital to comment why something is done in a particular way rather than some other way
17:30:48 <mdbooth> Assumptions that the code makes
17:31:02 <hartsocks> +1
17:31:07 <vuil> good points
17:31:23 <mdbooth> Comments are better than blueprints for documenting design long term
17:31:24 <rgerganov> I would argue that using good names is far more important than comments
17:31:37 <mdbooth> rgerganov: That's complementary
17:31:49 <rgerganov> calling something "uuid" which is not uuid is a huge mistake no matter how many comments you have
17:32:14 <hartsocks> rgerganov: I saw a very good comment you made on a review that caused the developer to change from "get" to "search" in the name of a method. I thought that's a very good thing.
17:32:51 <hartsocks> rgerganov: and I think we do have a problem around uuid abuse right now.
17:33:08 <mdbooth> That's a classic example of a need for a big comment block somewhere
17:33:24 <tjones1> rgerganov: yes you have a point (and i know what you are referring to).  Still i agree with mdbooth that some -1 would have not happened if there were good comments
17:33:28 <mdbooth> AFAIK, there's nowhere that we describe the association between an instance and a VM
17:33:35 <mdbooth> And it's not a simple association
17:33:40 <mdbooth> You have to infer it from the code
17:33:51 <mdbooth> And some of the code is only there for legacy reasons
17:34:01 <hartsocks> I actually have a blueprint proposed for this kind of concern.
17:34:21 <hartsocks> But, I'll refrain from plugging it just yet.
17:34:55 <hartsocks> A few weeks ago dansmith asked me to write this up:
17:34:59 <hartsocks> #link https://blueprints.launchpad.net/nova/+spec/vmware-spawn-refactor
17:35:11 <hartsocks> and there's been much discussion around it on the mailing list.
17:35:25 <ssurana> since we are discussing about my patch I would like to clear few things here if it's okay
17:35:58 <tjones1> ssurana: good idea - time's a wasting
17:35:58 <mdbooth> hartsocks: point 3 is a separate bp imho
17:36:05 <hartsocks> ssurana: I'll let you defend yourself if you feel attacked because I think that's only fair. But please keep it targeted.
17:36:22 <ssurana> sure
17:36:45 <ssurana> objection#1 misuse of the "nvp.vm-uuid"
17:37:18 <ssurana> The reason this needs to stay like this is, we want to make sure that we can get the reference to this vm in the context of the openstack instance later as well. e.g. when the user chooses the back out of the resize then we want to find this vm again and then re-do the linkage.
17:37:18 <ssurana> If we put the nvp.vm-uuid as null, then we loose the original context of what this VM was used for originally.
17:37:18 <ssurana> Having said the above one could argue that we are renaming that VM as well with the same uuid+suffix, and that it should be sufficient for our purpose. I wanted to mention here that even if that does solve the problem temporarily, but since "name" being a field that is quickly editable by any user who is using a client. Thus it makes it less reliable, which was the original reason why we moved away from relying on the name for linkage at the first
17:37:40 <hartsocks> ssurana: okay. Let's table that to the bug discussion now.
17:37:49 <ssurana> sure
17:38:07 <hartsocks> so, Juno will open for commit in a few weeks.
17:38:23 <hartsocks> maybe sooner.
17:38:41 <tjones1> rc1 on friday
17:38:45 <tjones1> so sooner
17:39:13 <hartsocks> If you follow the ML on this topic it seems the chief complaint isn't necessarily the spawn method itself … it's our tests.
17:39:38 <tjones1> um and the fact that it's complicated and hard to follow
17:40:05 <hartsocks> The reviews in the past used to be "yeah, there's tests" but it seems the attitude has changed…
17:40:12 <vuil> really? the spawn method code is not worth a complain?
17:40:21 <browne> as someone new to the code, i agree its hard to follow the tests
17:40:46 <browne> and some tests look like noops
17:41:07 <hartsocks> vuil: it's one of the easiest places to put a finger on and say "fix that" but really…
17:41:35 <hartsocks> vuil: the complaint I hear under "spawn is ugly" is … "how do I know it's tested" and that's to tjones1's point.
17:41:49 <hartsocks> both things are related.
17:42:10 <hartsocks> I've been on this project since Havana opened … and in that time…
17:42:23 <hartsocks> H1 had 0 merged blueprints for this driver
17:42:32 <hartsocks> H2 had 0 merged blueprints for this driver
17:42:40 <hartsocks> with a number of near misses.
17:42:42 <garyk> i am most probably going against the grain here,but the spawn is actually tested very well. both with the fakes and the mocking. the coverage is very good
17:43:12 <hartsocks> H3 had a few blueprints that merged and a number of misses. Most interestingly the ephemeral disk support missed.
17:43:36 <tjones1> it missed 2 releases
17:43:36 <sabari> garyk: concur with gary, just that it's not explicit, once the refactoring is done, tests will be evident.
17:43:41 <hartsocks> H3 to H RC1 looked almost identical to I3 to RC1
17:43:53 <hartsocks> I1 had 0 merged blueprints.
17:44:24 <hartsocks> I2 had 0 merged blueprints with a number of near misses… most interestingly ephemeral disk support missed (to me interesting anyway)
17:44:38 <hartsocks> I3 had a small number of open blueprints merge.
17:44:40 <vuil> gary,sabari: agree. But it is not to say the tests does miss things.
17:44:56 <arnaud> hartsocks: what's your point?
17:45:13 <hartsocks> If we continue to do business as we have. I don't see things changing.
17:45:29 <vuil> In fact through refactoring I discovered a few more things we can add to fake to make it represent reality better :-)
17:45:54 <hartsocks> vuil, arnaud, garyk, sabari: the test coverage numbers do not lie. We have very good test coverage in many places. The complaint is core-reviewers can't follow it.
17:46:40 <hartsocks> At the moment we have the core-reviewers attention and they are interested in seeing us refactor things both in the code… and in test…
17:47:10 <hartsocks> and if we had a J-1 that was 0 merged features… this would be no different than everything that has happened since grizzly-1
17:47:50 <hartsocks> We are going on two years now as a community with this same pattern of extreme effort and only marginal success.
17:48:20 <tjones1> hartsocks: ok - suggestions??
17:48:37 <hartsocks> I suggest we make priority for Juno-1 be refactoring work.
17:48:41 <hartsocks> No new features.
17:48:59 <hartsocks> (at least features behind refactoring work)
17:49:30 <tjones1> what about the 6 features we did for icehouse that did not make it?
17:49:31 <hartsocks> That means a number of parallel refactoring efforts… and some coordination… I can't propose all that here and expect us to resolve it in 15 minutes.
17:49:43 <hartsocks> tjones1: a good point for discussion.
17:50:00 <tjones1> glance work. vsan, ova, hot plug, etc
17:50:07 <hartsocks> tjones1: will the core-reviewers allow those 6 to merge first before we refactor or will they block it?
17:50:49 <tjones1> hartsocks: i am not disagreeing but i am mentioning we left significant work on the table….  i don't know the answer to that
17:50:59 <tjones1> we will have to ask
17:51:05 <hartsocks> I am proposing something radical.
17:51:17 <hartsocks> I won't and can't say we *must* do this or else.
17:51:39 <hartsocks> I rarely use this forum as a platform. But I felt this needed to be aired.
17:51:49 <tjones1> you want to do refactor and then the icehouse features after? just trying to clarify
17:52:13 <tjones1> and where does oslo integration come in - we clearly need like 2 or 3 hours for this meeting :-)
17:52:15 <hartsocks> That's the possibility that makes the most sense to me. Assuming we have core-reviewer support.
17:52:50 <hartsocks> I just wanted to air this while I have everyone's attention...
17:53:01 <hartsocks> #link https://etherpad.openstack.org/p/vmware-subteam-icehouse
17:53:14 <hartsocks> That's our icehouse etherpad where we all coordinated efforts.
17:53:22 <hartsocks> Here's our Juno etherpad...
17:53:31 <hartsocks> https://etherpad.openstack.org/p/vmware-subteam-juno
17:54:07 <hartsocks> … please use it for coordination. None of us has anymore authority than the others.
17:54:21 <hartsocks> And I genuinely feel this code base is sick.
17:54:38 <hartsocks> As in "ill" or "has a fever" not as in "awesome dude"
17:55:05 <mdbooth> Curse those young people and their language redefinition
17:55:33 <hartsocks> Because if I don't draw that conclusion … based on the valiant efforts of garyk, russellb, and other reviewers then I have to draw different conclusions that aren't good for collaboration.
17:55:39 <garyk> hartsocks: i look at it a little different
17:55:39 <hartsocks> mdbooth: :-)
17:56:03 <garyk> i think it is an opportunity to improve things. that is what we have
17:56:28 <hartsocks> garyk: that doesn't feel like anything opposing what I've said.
17:56:42 <tjones1> i agree - we have been living with some legacy stuff that we all hated - now we have the opportunity to fix it
17:57:02 <garyk> correct, but lets look ahead. we have a nice proposal. lets start to comment on that.
17:57:28 <mdbooth> The specific proposal was that juno-1 be a 0 feature release
17:57:33 <mdbooth> Does anybody object to that?
17:57:48 <garyk> there were the three patches by vui providing an framework. lets look at those and see if it is the right way to go
17:58:16 <garyk> mdbooth: i do
17:58:33 <vuil> Addressed by: https://review.openstack.org/82958
17:58:33 <vuil> vmwareapi: WIP: (nonworking draft) refactor spawn logic proposal
17:58:51 <garyk> my two cents are lets look at the spawn issues, try and get them done asap. hopefully have a little window for features
17:58:59 <vuil> would be the last of the three that the other two was setting up for. I would like some comments on the general approach
17:59:15 <hartsocks> any objection for putting the specific proposal Juno-1 as a 0 feature release to a vote in 7 days time?
17:59:36 <hartsocks> would people agree to abide by the results of the vote?
17:59:39 <vuil> which I think can proceed in parallel with the other refactoring work items listed in the BP
17:59:45 <mdbooth> hartsocks: Do you want to kick off a mailing list discussion?
17:59:56 <mdbooth> Given that we have 1 minute, might be better taken there
17:59:59 <hartsocks> mdbooth: that should happen as well.
18:00:00 <tjones1> so very clearly we are saying.  do the refactor, THEN add the icehouse features that did not back it back on.  we need to clarify the olso integtation
18:00:02 <russellb> it's probably easier to say "here are the specific refactorings" we're doing first
18:00:16 <russellb> and hopefully that's within juno-1
18:00:23 <russellb> if so, maybe some features make it
18:00:32 <russellb> if not, have to finish it, because it will be disruptive to feature work anyway
18:00:33 <russellb> right?
18:00:34 <garyk> russellb: agree 100%
18:00:36 <russellb> k
18:00:46 <hartsocks> russellb: I agree with that.
18:00:59 <russellb> other big juno ask is please try to coordinate your patch stream so that there's less conflict
18:01:10 <russellb> i think that was one of the things that has slowed down reviews/merges
18:01:15 <vuil> the oslo work is a refactoring of sorts. I removes dependencies on a bunch of code that is now way better tested elsewhere.
18:01:31 <hartsocks> vuil: I agree with that.
18:01:55 <tjones1> ok i'd really like to get the refactor done asap so we can get the icehouse misses into j-1
18:02:02 <tjones1> can that be a goal?
18:02:03 <hartsocks> we're out of time. I will kick off a thread on the ML to further discussion.
18:02:10 <sabari> jay-lau-513: we didn't have an opportunity to discuss the live-migration today, would be happy to have a discussion on the mailing list/irc.
18:02:36 <hartsocks> tjones1: that can be an appropriate compromise.
18:02:51 <hartsocks> I'm over on #openstack-nova for further IRC discussion.
18:02:57 <hartsocks> #endmeeting