17:03:21 <hartsocks> #startmeeting vmwareapi
17:03:21 <openstack> Meeting started Wed Sep 11 17:03:21 2013 UTC and is due to finish in 60 minutes.  The chair is hartsocks. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:03:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:03:25 <openstack> The meeting name has been set to 'vmwareapi'
17:03:35 <hartsocks> Greetings! Who's about?
17:03:46 <tjones> hi
17:03:48 <garyk> hi
17:03:50 <vuil> yo
17:05:03 <hartsocks> We're past feature freeze!
17:05:21 <hartsocks> That hopefully means there's not much in the way of blueprints to talk about.
17:05:36 <hartsocks> Does anyone have a blueprint we need to discuss?
17:06:17 <garyk> hartsocks: i would like to mention a few things about the multi cluster
17:06:25 <hartsocks> go for it.
17:06:51 <garyk> it is a nice addition. i think that we need to do some extensive testing here.
17:07:06 <vuil> agreed
17:07:29 <garyk> there look like there are a few places where the _vmops parameter is not set correctly. i am checking now.
17:07:45 <garyk> can you guys please look at the vnc_console. this may be one of them
17:08:00 <garyk> give me a sec and i'll paste the line
17:08:22 <hartsocks> There's a general state keeping problem in the driver… something we may need to address in the future.
17:08:32 <hartsocks> IE: don't keep state
17:08:47 <garyk> https://github.com/openstack/nova/blob/master/nova/virt/vmwareapi/driver.py#L468
17:09:15 <garyk> i think it is missing _vmops = self._get_vmops_for_compute_node(instance['node'])
17:09:35 <garyk> but as i said we need to do additional testing
17:10:00 <hartsocks> hmm: https://github.com/openstack/nova/blob/master/nova/virt/vmwareapi/driver.py#L422
17:10:18 <garyk> but if the instance is on another cluster
17:10:31 <hartsocks> I see… this is interesting… adding the multi-cluster breaks assumptions.
17:10:44 <hartsocks> The assumption being _vmops is set at init.
17:10:51 <hartsocks> This is definitely a problem.
17:11:08 <garyk> i'll test and fix tomorrow. thanks for the input
17:11:11 <vuil> quite a few more self._vmops…. too
17:11:24 <hartsocks> woah… hold your horses there...
17:11:30 <garyk> witht he work on the migrations i have seen the issues there and am dealing with them
17:11:52 <hartsocks> Lines 422 through 425 imply the driver's object's state is set at init time.
17:12:39 <hartsocks> I think this means you're going to have to switch something around.
17:12:41 <garyk> that is for the first cluster but not for the additional ones (which are pat of the _vmops dictionary)
17:13:02 <vuil> which means some ops only work for the first.
17:13:07 <vuil> cluster
17:13:11 <garyk> correct
17:13:49 <garyk> i think that this is good that it is on the radar. harlowja
17:13:58 <harlowja> ???
17:13:59 <harlowja> oh
17:14:05 <hartsocks> I think we need to remove self._vmops and a few of these others.
17:14:40 <garyk> harlowja: that was a auto completion. sorry
17:15:00 <harlowja> np
17:15:09 <garyk> hartsocks: i think that at this stage of the game we should do a case by case fix
17:15:56 <garyk> i am just happy that this is on everyone radar now. i guess we can continue to discuss after the meeting
17:15:59 <hartsocks> garyk: it will be easier to get a bug fix through case-by-case but the fact you have a self._vmops at all becomes dangerous.
17:16:10 <garyk> agreed
17:16:31 <hartsocks> Should we consider this a first refactor for IceHouse?
17:16:43 <hartsocks> (spot fix the bugs naturally)
17:17:20 <hartsocks> Something like bp/vmware_refactor_cluster_sensitive_objects
17:17:56 <hartsocks> Then we can delete the self._vmops and other sensitive objects that might lead to a bug sneaking through.
17:18:40 <hartsocks> The comment at 419 troubles me too… BTW.
17:18:46 <garyk> lets hope our testing finds all of the issues so we will not need to do something like that
17:18:52 <hartsocks> https://github.com/openstack/nova/blob/master/nova/virt/vmwareapi/driver.py#L419
17:19:26 <hartsocks> garyk: okay, but that design feels fundamentally wrong.
17:19:31 <garyk> agreed
17:20:35 <hartsocks> I really don't want to leave that code in place since it will let bugs slip through.
17:20:43 <hartsocks> If designed properly...
17:20:44 <hartsocks> https://github.com/openstack/nova/blob/master/nova/virt/vmwareapi/driver.py#L468
17:20:55 <hartsocks> should have thrown a NoneType exception
17:21:09 <hartsocks> That actually troubles me more than anything else.
17:21:25 <hartsocks> #action triage bug https://github.com/openstack/nova/blob/master/nova/virt/vmwareapi/driver.py#L468
17:21:36 <hartsocks> garyk: you will report that bug? yes?
17:22:14 <hartsocks> wow… I need to stop looking there...
17:22:19 <garyk> hartsocks: sec
17:22:24 <hartsocks> https://github.com/openstack/nova/blob/master/nova/virt/vmwareapi/driver.py#L458
17:22:44 <vuil> that's the migration stuff that gary was talking about
17:22:50 <garyk> hartsocks: i'd like to add the fix to https://review.openstack.org/#/c/43616/
17:23:16 <garyk> i'll post and update soon
17:23:19 <hartsocks> Okay. I did not understand what that was talking about.
17:23:38 <garyk> vuil: yes, that is correct. i am busy working on that now
17:24:30 <hartsocks> #undo
17:24:31 <openstack> Removing item from minutes: <ircmeeting.items.Link object at 0x322a550>
17:24:48 <hartsocks> So, let me get this straight...
17:25:10 <hartsocks> You are fixing all these other issues that I see on that page in this bug...
17:25:18 <hartsocks> #link https://bugs.launchpad.net/nova/+bug/1216510
17:25:19 <uvirtbot> Launchpad bug 1216510 in nova "VMware: exception when accessing invalid nodename" [Medium,In progress]
17:26:51 <garyk> hartsocks: that is where i think that we should have the console fix
17:27:07 <hartsocks> console fix?
17:27:10 <garyk> hartsocks: in addition to this i am looking into the resize issue
17:27:42 <garyk> have i managed to confuse everyone including myself.
17:27:43 <garyk> sorry
17:27:53 <hartsocks> so… what are you doing in bug 1216510?
17:27:54 <uvirtbot> Launchpad bug 1216510 in nova "VMware: exception when accessing invalid nodename" [Medium,In progress] https://launchpad.net/bugs/1216510
17:28:19 <garyk> in that bug fix i have done 2 things:
17:28:26 <hartsocks> I understand we have separate issues: resize, console, etc. Are you saying these are all one bug?
17:28:43 <garyk> 1. made fixes to the data structure so that it does not need a redirection to get the _vmops
17:28:52 <garyk> 2. validated that the resource exists prior to accessing
17:29:00 <garyk> the console will be treated here too
17:29:24 <garyk> make sense?
17:29:47 <hartsocks> I understand multiple patches under one bug… that I understand.
17:30:07 <garyk> i have pushed the chnage. i guess that we can discuss it on gerrit unless you want to continue here
17:30:52 <hartsocks> Well… I'll make it a higher priority than medium!
17:31:07 <garyk> agreed. thanks!
17:31:30 <hartsocks> I'm twiddling some bits on that bug… hold on...
17:31:54 <hartsocks> okay… that's done...
17:32:00 <hartsocks> #link https://bugs.launchpad.net/openstack-vmwareapi-team/+bug/1216510
17:32:03 <uvirtbot> Launchpad bug 1216510 in openstack-vmwareapi-team "VMware: exception when accessing invalid nodename" [Critical,In progress]
17:32:24 <hartsocks> So, what I've done is link the bug back to a VMwareAPI-Team project.
17:32:30 <garyk> sorry for taking up everyone times
17:32:50 <hartsocks> Better to talk it out than fly around in the dark!
17:33:07 <vuil> no these are impt to get right.
17:33:09 <garyk> i have retinitis pigmentosa so i am always in the dark :)
17:33:51 <hartsocks> So that project is something Tracy, Dan, and I built because … notice this bug got bumped down to "Medium" … the Nova team guys didn't think our driver bugs were prioritized right.
17:33:58 <tjones> while we are talking about bugs - im almost ready to push the bug on booting multiple instances with a concurrent image upload - but i am going to make it WIP as i'd like a few pairs of eyes on it since i am mucking with concurrency issues
17:34:11 <garyk> tjones: great
17:34:19 <hartsocks> tjones: yay!
17:35:35 <hartsocks> I was going to ask about bugs next anyway… :-)
17:36:03 <hartsocks> #link https://bugs.launchpad.net/nova/+bug/1190515
17:36:05 <uvirtbot> Launchpad bug 1190515 in openstack-vmwareapi-team "Incorrect host stats reported by VMWare VCDriver" [High,In progress]
17:36:28 <hartsocks> That one is the only bug from my FF list that hasn't merged.
17:36:41 <hartsocks> #link https://review.openstack.org/#/c/33100/
17:36:58 <hartsocks> So with that at the top of my list and RC1 coming up on Sept. 26th ...
17:37:23 <hartsocks> what else should I track?
17:37:39 <hartsocks> (I think we got a good idea on 2 or three right now)
17:39:05 * hartsocks did I kill the conversation?
17:39:24 <garyk> i have a few.
17:39:29 <vuil> gary sent out a list two days ago, and Sabari and I tagged on a few
17:39:59 <garyk> https://review.openstack.org/#/c/41977/ - i think that we need a bug for this one
17:40:14 <garyk> https://review.openstack.org/#/c/43268/ - vnc password
17:40:38 <garyk> https://review.openstack.org/#/c/43616/ - we spoke about this at the beginning of the meeting
17:41:26 <garyk> issues with volumes - https://review.openstack.org/#/c/45864/ and https://review.openstack.org/#/c/46027/
17:41:42 <garyk> guess we should maybe go back to the mail list or have a wiki with all of the issues
17:42:42 <hartsocks> busy guy.
17:43:02 <hartsocks> … and you're running the scheduler meetings now too!
17:43:54 <vuil> two on my list:
17:43:56 <vuil> https://review.openstack.org/#/c/40298/: snapshot failure
17:44:20 <vuil> https://review.openstack.org/#/c/43994/: spawn failure with sparse disk
17:45:11 <tjones> can we just add these to the VMwareAPI-Team project so we have 1 place to track them??
17:45:25 <tjones> id rather do that than a wiki that we would forget to update
17:45:29 <hartsocks> tjones: that sounds like a good idea.
17:45:47 <hartsocks> We also need to have a working priority system we can manage ourselves.
17:45:56 <tjones> which we have on that list
17:46:28 <hartsocks> So, to me that sounds good.
17:46:36 <tjones> AND that list ends up being on this report - which tracks where they land http://partnerweb.vmware.com/programs/vmdkimage/customer_bugs.html
17:47:23 <hartsocks> hurrah!
17:47:50 <hartsocks> So what we track there on partnerweb...
17:47:57 <hartsocks> that's...
17:48:03 <hartsocks> Critical = don't deploy without
17:48:10 <hartsocks> High = Strongly recommended
17:48:21 <hartsocks> and nothing else right now...
17:48:23 <hartsocks> right
17:48:24 <hartsocks> ?
17:48:34 <tjones> oh yeah.  lets continue this discussion after the meeting.  i don't want to add "yet another project"
17:48:48 <hartsocks> okay.
17:49:14 <hartsocks> But, in general…
17:49:31 <hartsocks> I think we just need to identify things for our driver *we*
17:49:35 <hartsocks> that is all of us...
17:49:44 <hartsocks> would classify as "critical" or "high"
17:49:53 <hartsocks> but the nova team would bump to "Medium"
17:50:08 <hartsocks> So far, I'm pretty sure that's everything that has been linked to so far.
17:50:20 <hartsocks> (in this meeting)
17:51:08 <hartsocks> I'll go back over the minutes and fix each related bug report to a driver-specific priority then.
17:51:33 <hartsocks> anything else on bugs?
17:52:06 <hartsocks> or does anyone have anything else we need to talk about as a group?
17:52:06 <tjones> can i get some reviews on https://review.openstack.org/#/c/33504/  please?
17:52:55 <vuil> will do
17:53:06 <hartsocks> oh no, jenkins tripped.
17:53:12 <hartsocks> I'll look at it later.
17:53:52 <garyk> hartsocks: that grumpy old man again. i have run a 'recheck'
17:54:10 <garyk> arosen: thanks for the review
17:54:20 <hartsocks> BTW: I've been looking at some of these Jenkins failures… lots of that testing code is multi-process, asynchronous, and uses timers!
17:55:36 <hartsocks> sadly, we're just going to have to kick Jenkins in the shins from time to time.
17:57:12 <hartsocks> So it sounds like we have some really high priority issues to fix-up.
17:57:25 <garyk> sorry guys. i need to go and put the kids to bed.
17:57:38 <hartsocks> Just about to close the meeting anyhow.
17:57:42 <hartsocks> Thanks.
17:58:53 <hartsocks> We're over in #openstack-vmware if you need us. Let's try and synchronize efforts more over there.
17:59:01 <hartsocks> #endmeeting