16:00:01 <smcginnis> #startmeeting Cinder
16:00:02 <openstack> Meeting started Wed Nov  9 16:00:01 2016 UTC and is due to finish in 60 minutes.  The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:05 <smcginnis> ping dulek duncant eharney geguileo winston-d e0ne jungleboyj jgriffith thingee smcginnis hemna xyang1 tbarron scottda erlon rhedlind jbernard _alastor_ bluex patrickeast dongwenjuan JaniceLee cFouts Thelo vivekd adrianofr mtanino yuriy_n17 karlamrhein diablo_rojo jay.xu jgregor baumann rajinir wilson-l reduxio wanghao thrawn01 chris_morrell stevemar watanabe.isao,tommylike.hu mdovgal
16:00:06 <openstack> The meeting name has been set to 'cinder'
16:00:12 <eharney> hi
16:00:19 <smcginnis> Hi everyone.
16:00:20 <xyang1> hi
16:00:23 <rajinir> o/
16:00:24 <mdovgal> hi all
16:00:24 <hemna> hi
16:00:26 <DuncanT> Hi
16:00:32 <diablo_rojo> Hello
16:00:35 <patrickeast> o/
16:00:35 <tsg> Hi
16:00:48 <scottda> hi
16:01:00 <smcginnis> #topic Announcements
16:01:01 <dulek> o/
16:01:29 <smcginnis> Public service announcement - nova-net was deprecated last release. If you run a CI and you are still using it, time to switch.
16:01:36 <smcginnis> It will be going away this release.
16:01:54 <smcginnis> And based on past experience, you may need a little time to figure out how to set things up right. ;)
16:02:04 <hemna> neutron works now?
16:02:06 <hemna> :P
16:02:14 <smcginnis> :)
16:02:15 <flip214> hi
16:02:31 <geguileo> Hi!
16:02:42 <smcginnis> #link https://etherpad.openstack.org/p/cinder-spec-review-tracking Review focus
16:02:50 <dulek> jgriffith must be delighted with Neutron. :)
16:02:51 <smcginnis> Just typical reminder on review tracking.
16:03:16 <smcginnis> Feel free to add priority stuff there, and if you are a reviewer looking for what to look at, that's a good place to go.
16:03:32 <dulek> Hm, should we clean it up a little as Ocata started?
16:03:32 <smcginnis> #topic Cinder HA A/A
16:03:41 <smcginnis> dulek: Yeah, we should.
16:03:49 <smcginnis> geguileo: You're up. :)
16:03:57 <diablo_rojo> dulek, I was about to ask the same thing.
16:05:04 <smcginnis> #info Concern about drivers supporting HA A/A without testing.
16:05:39 <smcginnis> #link https://review.openstack.org/#/c/393460/ PoC of code to disable until vendor acknowledges tested ok.
16:05:41 <dulek> Is there anything controversial about Gorka's proposal?
16:06:06 <geguileo> dulek: I discussed this with smcginnis but we thought we should hear everybody's opinion
16:06:08 <smcginnis> Not really from me.
16:06:12 <Swanson> Hi
16:06:15 <smcginnis> geguileo: +1
16:06:17 <erlon-airlong> hey
16:06:40 <smcginnis> So anybody with questions/thoughts/differing opinions?
16:06:42 <dulek> Okay, to me it's pretty straightforward, but I don't really own a driver…
16:07:04 <smcginnis> geguileo: Huh, that was easy. :)
16:07:17 <erlon-airlong> I like the idea just the point I added in the driver about having the report in get_stats
16:07:46 <hemna> shouldn't the driver just report the capability in get volume stats?
16:07:51 <geguileo> erlon-airlong: I've been away for a couple of days and haven't read the comment
16:07:53 <smcginnis> #info Driver maintainers will need to explicitly enable support for HA A/A for their drivers.
16:08:04 <dulek> hemna: I don't think you
16:08:18 <geguileo> The idea is to prevent users from running A/A without the vendor saying it's ok
16:08:22 <dulek> hemna: you'll be able to stop starting cinder-volume in that case.
16:08:28 <erlon-airlong> hemna: geguileo: all capabilities of drivers are reported in get_stats
16:08:37 <eharney> doing this as a class variable rather than in volume stats seems to be the right thing to me
16:08:43 <erlon-airlong> geguileo: I think  should keep the same in this case
16:08:44 <geguileo> erlon-airlong: Yeah, but the service won't start
16:09:02 <eharney> it's not the same kind of info, and this is simpler
16:09:02 <geguileo> erlon-airlong: That's the whole point
16:09:10 <smcginnis> eharney: +1
16:09:18 <e0ne> eharney:  +1
16:09:23 <geguileo> We don't need to report anything we just don't start because the vendor says the driver is not tested for A/A
16:09:28 <smcginnis> I do agree this is a little different than our other "capabilities".
16:09:30 <erlon-airlong> geguileo: hmm, got it, yep, I was wondering the reason why o put it there
16:09:33 <hemna> so we have capabilities and then another capability elsewhere
16:09:39 <hemna> I dunno if I'm into it
16:09:47 <geguileo> This is not capabilities
16:09:51 <smcginnis> hemna: Meta-capabilities. ;)
16:09:52 <geguileo> This is to prevent the start of the service
16:09:55 <hemna> it is a capability
16:10:13 <hemna> which can be done other ways as well
16:10:18 <erlon-airlong> hemna: it it a capability but does not affect on how volumes are scheduled
16:10:27 <hemna> logically this is a capability of the backend and or driver
16:10:46 <e0ne> hemna: it's a driver capability, not a backend's
16:10:49 <geguileo> hemna: We can report it, that's not an issue, but it won't affect anything
16:10:51 <dulek> e0ne: +1
16:11:01 <erlon-airlong> this could be a line, if affect the scheduler in any way, then it should go to get_stats
16:11:01 <smcginnis> I think less of a capability, more of a valid configuration setting.
16:11:04 <geguileo> Scheduling will be the same
16:11:07 <dulek> Not even tell anything useful for the user.
16:11:15 <hemna> I disagree
16:11:18 <hemna> -1 from me
16:11:20 <geguileo> smcginnis: +1
16:11:20 <dulek> It's like info important for deployer only.
16:11:30 <hemna> I'll stop arguing, I just don't like it.
16:11:39 <geguileo> hemna: You don't like that we don't report it
16:11:51 <geguileo> hemna: Or that we are preventing the service from starting with an incorrect configuration?
16:12:03 <smcginnis> Well, anybody with any opinions, please comment on the PoC patch: https://review.openstack.org/#/c/393460/
16:12:09 <hemna> I'd prefer the driver to report it's capability the normal way
16:12:19 <hemna> and the service starting or stopping is a separate matter
16:12:24 <geguileo> hemna: And to start with a bad configuration?
16:12:33 <DuncanT> hemna: It's too late by then....
16:12:33 <hemna> I like the idea of preventing the service from starting if the driver/backend can't do AA
16:12:34 <geguileo> One that could potentially lead to problems?
16:12:41 <dulek> geguileo: It's not bad. It's simply dangerous configuration.
16:12:53 <dulek> geguileo: Worst case - some operations cause data loss.
16:12:56 <geguileo> dulek: In my mind danger = bad
16:12:58 <geguileo> XD
16:13:01 <dulek> :)
16:13:08 <DuncanT> This is more like a high level check_for_setup_errors
16:13:13 <smcginnis> Data loss == really bad
16:13:25 <hemna> also, another thing is that this would prevent the scheduler from ever sending requests to the cluster
16:13:29 <hemna> as it wouldn't be supported
16:13:31 <geguileo> hemna: So you would be OK if the drivers reported this capability?
16:13:42 <hemna> anyway, I'm in the minority here.
16:13:53 <DuncanT> This is more like a high level check_for_setup_errors
16:13:53 <hemna> geguileo, yah, drivers should report it
16:14:06 <geguileo> hemna: OK, I don't think that would be a problem
16:14:20 <geguileo> But changing the scheduler to take it into cosideration may be overboard
16:14:21 <dulek> Is scheduler able to tell if driver is configured as A/A? Hm, it probably is if it's cinder.conf is same as c-vols.
16:14:40 <geguileo> dulek: If it's reported the scheduler will know
16:14:47 <geguileo> dulek: But since the service won't be starting...
16:14:58 <geguileo> dulek: It won't be able to schedule anything there anyway
16:15:37 <DuncanT> It will know the difference between an H/A backend (correcetly configured) and a none-H/A backend though, which might be something people want to schedule on
16:15:40 <dulek> Yeah, I see. Moreover requests failing in scheduler because of wrong configuration would be quite confusing for admin.
16:16:03 <dulek> At least in comparison with c-vol simply failing on the start.
16:16:22 <DuncanT> dulek: ++
16:16:37 <geguileo> dulek: I agree
16:17:03 <smcginnis> OK, again, please comment in the review if you see any issues.
16:17:09 <smcginnis> geguileo: Anything else we should cover here?
16:17:17 <geguileo> smcginnis: No, that was all
16:17:22 <smcginnis> geguileo: Thanks!
16:17:26 <smcginnis> #topic NVMe-over-Fabrics (NVMe-oF) target driver support for Cinder
16:17:33 <dulek> Hi!
16:17:36 <smcginnis> dulek, tsg: Hey
16:17:58 <smcginnis> #link https://www.youtube.com/watch?v=fTPe0sOHU0w PoC demo video
16:18:09 <dulek> So this is mostly a heads-up that we're looking over integrating NVM-over-RDMA-Fabric protocol with Cinder.
16:18:39 <hemna> dulek, is a brick connector needed ?
16:18:46 <dulek> hemna: Definitely.
16:18:58 <dulek> I wonder if any existing drivers in Cinder looked at/experimented with this protocol?
16:19:12 <flip214> The DRBD driver is using RDMA.
16:19:17 <flip214> or CAN use RDMA.
16:19:22 <e0ne> hemna: yes, we've got PoC both for cinder and os-brick
16:19:23 <smcginnis> dulek: Lot's of looking at, no experimenting yet.
16:19:42 <hemna> ok cool
16:19:56 <dulek> flip214: This is quite interesting.
16:20:14 <smcginnis> dulek: Was there a patch submitted? Seems like I saw something, but maybe I'm just thinking of the bp?
16:20:15 <hemna> dulek, do we have a CI for it or can we get one?
16:20:39 <e0ne> smcginnis: we didn't submit any patches yet
16:20:40 <dulek> hemna: We would need one, but that would require a driver being able to support the protocol I think.
16:20:47 <hemna> yup
16:20:53 <smcginnis> e0ne: OK, thanks. Must have just been thinking of the blueprint then.
16:20:55 <xyang> e0ne: do you have a nova patch too
16:20:56 <e0ne> #link https://blueprints.launchpad.net/cinder/+spec/nvme-volume-driver
16:21:04 <e0ne> xyang: I have
16:21:18 <dulek> And it's a little problematic with LVM at this moment. I wonder if we could try to start with BDD?
16:21:20 <e0ne> #link https://github.com/e0ne/cinder/tree/nvme
16:21:28 <e0ne> #link https://github.com/e0ne/nova/tree/nvme
16:21:32 <e0ne> #link https://github.com/e0ne/os-brick/tree/nvme
16:21:37 <smcginnis> e0ne: Oh, nice!
16:21:44 <e0ne> all code are in PoC state now
16:21:57 <e0ne> please, don't look on the codestyle now:(
16:22:04 <e0ne> I'll cleanup it early next week
16:23:13 <smcginnis> dulek: Cool, so just a head's up for now? Looking for other involvement?
16:24:04 <dulek> If someone is interested in consuming such TargetDriver and os.brick connector in his driver, that could help us innovate on that.
16:24:25 <e0ne> dulek: :)
16:24:39 <dulek> I'll look into DRBD as flip214 mentioned.
16:25:06 <flip214> dulek: that won't help.
16:25:10 <e0ne> to be clear, it's not only about RDMA support
16:25:10 <flip214> the DRBD kernel module uses RDMA
16:25:20 <smcginnis> flip214: Is that NVMe, or just using RDMA?
16:25:27 <smcginnis> Ah, just RDMA.
16:25:31 <flip214> smcginnis: we can use NVMe as lower-level storage too, of course.
16:25:38 <smcginnis> flip214: OK
16:25:40 <e0ne> it's more about NVMeoF and RDMA will be one of the supported protocols
16:25:40 <flip214> but the RDMA transport is just configured from userspace
16:25:47 <flip214> the communication is done in kernel only
16:26:00 <flip214> so there's nothing for brick or cinder, apart from "done the right way it works"
16:26:12 <smcginnis> OK, let's move on then if that's OK dulek.
16:26:24 <smcginnis> And anyone interested, contact dulek (and e0ne) to collaborate.
16:26:28 <dulek> Sure, thanks, if someone is interested in collaboration - ping me or e0ne.
16:26:34 <smcginnis> dulek: Thank you
16:26:37 <dulek> Thanks!
16:26:42 <smcginnis> #topic In-tree Cinder tests
16:26:48 <smcginnis> erlon-airlong: Take er away...
16:26:50 <erlon-airlong> hey
16:26:50 * dulek needs to run now, sorry.
16:26:58 <smcginnis> dulek: Fine, be like that.
16:26:59 <smcginnis> :)
16:27:15 <erlon-airlong> so, Iv been trying to merge some tests in tempest for a while
16:27:42 <erlon-airlong> but, there sooo few cores active there that its taking a life
16:27:51 <smcginnis> erlon-airlong: We've seen that many times now.
16:28:14 <e0ne> erlon-airlong: +1
16:28:23 <erlon-airlong> yep, so, we should start to add tests directly into cinder tree, and add hooks in the gate to run/search them
16:28:57 <erlon-airlong> like manila does
16:28:59 <eharney> i think we came to this same conclusion in another discussion recently
16:29:07 <scottda> erlon-airlong: Yup. And we discussed in Barcelona that this would be a good first step, even if later we wanted them to run in the tempest directory
16:29:08 <smcginnis> So the only question I had for the team on that was whether everyone thought the gate should run our in-tree tests.
16:29:09 <e0ne> erlon-airlong: I think everybody agree with you
16:29:19 <geguileo> eharney: Yep, during the summit
16:29:31 <smcginnis> Way back when we added it I (just personally) was thinking this would be a good way for third party CI to add additional stuff.
16:29:43 <smcginnis> But I think we really do want all tests to run in gate.
16:29:55 <e0ne> smcginnis: +1 for all tests in gate
16:30:01 <erlon-airlong> e0ne: great ! just ill put some patches to make that happen
16:30:02 <eharney> the gate should definitely run the in-tree tests for configs that are possible
16:30:06 <patrickeast> might as well, assuming we are careful to block off features with config options
16:30:06 <smcginnis> OK, good.
16:30:27 <erlon-airlong> smcginnis: they should run in the gate the same way that tempest tests do
16:30:30 <smcginnis> So the next step is probably to work with infra to change the tempest runs to run the all_plugins target.
16:30:44 <smcginnis> That will find and run the local tests along with the normal tempest ones.
16:30:57 <erlon-airlong> ok
16:31:16 <smcginnis> As far as how that is done, I'm no help there. ;)
16:31:46 <smcginnis> OK, if no objections, should we move on?
16:31:59 <erlon-airlong> smcginnis: me to, dont have a clear idea, Ill find you how to do that
16:32:19 <smcginnis> #action erlon-airlong to investigate changing gate jobs to pick up in-tree tempest tests
16:32:23 <smcginnis> erlon-airlong: Thanks!
16:32:32 <smcginnis> #topic How to deal with cinderclient help with microversions
16:32:33 <erlon-airlong> smcginnis: welcome
16:32:38 <smcginnis> scottda: .
16:32:44 <scottda> So, Cao filed a bug because cinderclient help would show parameters and features in, say version 3.0, that weren't present until later versions...
16:32:49 <scottda> https://bugs.launchpad.net/python-cinderclient/+bug/1600567
16:32:49 <openstack> Launchpad bug 1600567 in python-cinderclient ""cinder help" does not support microversion" [Medium,In progress] - Assigned to Nate Potter (ntpttr)
16:32:52 <scottda> And we merged a fix:
16:32:52 <patrickeast> erlon-airlong: its not too bad, there should just be a conf option in devstack-gate for it
16:33:01 <scottda> https://git.openstack.org/cgit/openstack/python-cinderclient/commit/?id=b76f5944130e29ee1bf3095c966a393c489c05e6
16:33:24 <smcginnis> Personal opinion - I think that was probably wrong.
16:33:28 <scottda> The question then arose: How does the user know about features or parameters that show up in later versions?
16:33:29 <erlon-airlong> patrickeast: great, also, manila is doing that, so, Ill just borrow from them
16:33:48 <smcginnis> I'd rather see all options shown, with something denoting the version needed to use the ones that aren't in the base v3.
16:33:56 <scottda> Should the help show all features/parameters, and then name the version when they are first supported?
16:34:34 <smcginnis> scottda: That would be my preference.
16:35:01 <scottda> smcginnis: Cool. I can see that point and am not sure I have a strong preference. What do others think?
16:35:31 <patrickeast> what are other micro-versiony projects doing about this? surely this ins't a unique cinder problem
16:35:46 <eharney> i also agree with that preference, otherwise it's too hard for anyone to find all of the options/parameters
16:35:48 <smcginnis> patrickeast: Good point. Do we know what nova and others do?
16:35:56 <scottda> patrickeast: Good question. Manila people? (xyang bswartz ...
16:36:55 <scottda> I think nova is busily deprecating the Nova CLI. So also of interest what OSC folks think. But they don't have microversion support yet...
16:38:06 <smcginnis> Maybe we can set a good precendent with this then.
16:38:11 <DuncanT> We alrady have things in the cinder cli that may or may not be supported
16:38:21 <smcginnis> I do think it's more user friendly to be able to discover what is possible.
16:38:47 <smcginnis> DuncanT: And with this, at least we can tell them what version they can expect that to be supported from.
16:38:50 <scottda> smcginnis: Well, putting everything in the help doesn't let you discover what is possible.
16:39:08 <DuncanT> smcginnis: ++
16:39:08 <smcginnis> scottda: It let's them know what they can check based on version.
16:40:07 <erlon-airlong> smcginnis: agreed, I have strugled trying to find help for hidden options
16:40:24 <smcginnis> No one else cares? US folks stayed up too late watching election coverage? :)
16:40:26 <erlon-airlong> it totally blows the meaning of the word help
16:40:30 <smcginnis> erlon-airlong: +1
16:41:10 <bswartz> scottda: sorry was afk
16:41:11 <geguileo> If you display everything that's available then you also have another problem
16:41:14 <DuncanT> It would be nice if we could make the help appropriate for the cloud it is talking to....
16:41:19 <geguileo> In later versions we may remove arguments
16:41:27 * bswartz reads scrollback
16:41:39 <geguileo> And the help starts getting crazy to read
16:41:43 <DuncanT> geguileo: We keep supporting the old microversion though
16:41:46 <smcginnis> bswartz: High level - wondering how help is displayed for microversion stuff in Manila.
16:41:53 <geguileo> Trying to figure out what you have to pass for each version
16:42:13 <smcginnis> geguileo: Hopefully we don't have too much of that anyway.
16:42:13 <geguileo> DuncanT: Yeah, but you have to read through all the options to see which ones go together
16:42:26 <erlon-airlong> geguileo: wouldnt be possible to add sections for each version if the command has different behaviours?
16:42:28 <geguileo> smcginnis: Hopefully  :-)
16:42:52 <bswartz> smcginnis: CLI works with latest version, and help reflects latest version IIRC
16:42:56 <geguileo> erlon-airlong: Sure, if somebody has time to work on that it would be grea  XD
16:43:08 <scottda> DuncanT: I'd like to add auto-negotiate like Manila has. The client and server would use the hightest mutually supported version....
16:43:14 <bswartz> the CLI should try to work as less as possible with old versions, but there are limits
16:43:18 <erlon-airlong> geguileo: lol, yep, that wouldn't be easy to do
16:43:40 <scottda> erlon-airlong: That's what this controversial patch does...
16:43:40 <erlon-airlong> geguileo: not hard actually but a lot of details
16:43:53 <scottda> It has the ability to version the help.
16:43:53 <smcginnis> #info Currently help for new feature/parameters only shows up when that microversion is requested
16:44:18 <bswartz> CLI users should _NEVER_ have to request a microversion
16:44:26 <smcginnis> bswartz: +1
16:44:28 <bswartz> that's a huge fail of UI design
16:44:44 <bswartz> you should just get the latest
16:45:02 <scottda> bswartz: Yup. I can add that to the client. Most of the plumbing is there and I've a patch up to fix a bug...
16:45:18 <bswartz> if the server is somehow older than the client, then the client needs to ensure compatibility as much as it can
16:45:28 <erlon-airlong> scottda: not exacly, it shows the help for a set of versions, my idea would to always (unless specified) print the help for all versions, diveded in sections when needed
16:45:53 <erlon-airlong> bswartz: +1
16:45:54 <scottda> erlon-airlong: ahh..OK. That is different
16:45:58 <DuncanT> bswartz: ++
16:46:24 <bswartz> it's acceptable to limit backwards compatibility to a certain oldest version though -- and tell users with servers older than that to use an older client
16:46:46 <bswartz> ideally that window of backwards compatibility goes back at least 2 years
16:46:47 <erlon-airlong> IMO, this would be something to be fixed in OSC
16:47:17 <erlon-airlong> and as they have plans to catchup cinderclient, that's something they coud start working on
16:47:49 <scottda> erlon-airlong: Yes, OSC has started to look at microversion support. But aren't there yet.
16:48:35 <scottda> But this question is simpler: Should the help show all features, even those the server doesn't support? OR should it show only those available on the server (or in our current situation, the version requested on the client side)?
16:49:11 <scottda> Soon, cinderclient auto-negotiation will prevent the user from input of the requested version. But the help question is the same...
16:49:39 <erlon-airlong> if the client supports all features, then its help should reflect that
16:49:52 <bswartz> scottda: yes the help should show what the client supports, not what the server supports
16:50:11 <bswartz> otherwise you can't print out help without a connection to a server, which is kind of dumb
16:50:19 <xyang> scottda: maybe bswartz has already answered this: manila help does not require a microversion
16:51:01 <scottda> bswartz: Well, today you can manually input the cinderclient version you want, and get help appropriate to that version.
16:51:01 <bswartz> remember that in the majority of cases, the client and server will be at the same version
16:51:25 <bswartz> the reason we allow for any variance at all is for clients/apps that needs to talk to multiple clouds
16:51:48 <bswartz> in those cases we assume the user or developer is a little smarter than average
16:52:08 <bswartz> and they don't need "help" text to figure out what they're doing
16:52:15 <scottda> So, is there anyone that thinks we should keep the help as it is, giving only features supported by the requested version? or the hightest version on the server?
16:52:34 <scottda> OR does everyone agree that cinderclient help should show all features supported by the client?
16:52:49 <bswartz> #vote all features
16:53:03 <smcginnis> My preference is option 2.
16:53:26 <scottda> #vote all features
16:53:33 <scottda> Sounds good to me ^^
16:53:47 <erlon-airlong> #vote all features
16:54:00 <xyang> I prefer all features too
16:54:27 <smcginnis> scottda: Seems pretty unanimous so far.
16:54:43 <scottda> Seems like no dissent. I'll file a bug to track this. I cannot guarantee when I'll get to it, so anyone else feel free to pick it up.
16:54:57 <smcginnis> scottda: Thanks!
16:55:24 <smcginnis> #action scottda to file bug to change help to show all options
16:55:38 <smcginnis> #topic Open Discussion
16:55:45 <smcginnis> Anything else today?
16:55:55 <bswartz> PTG is 3 days (wed-fri) right?
16:56:11 <smcginnis> bswartz: Good topic - for Cinder, yes.
16:56:20 <bswartz> yeah I meant cinder
16:56:32 <bswartz> manila will be wed-thurs so 100% overlap :-(
16:56:37 <smcginnis> We were asked if our team would like Wed-Thurs or Wed-Fr. I have requested for Cinder we get three days.
16:57:02 <smcginnis> bswartz: Maybe we could coordinate a couple joint sessions if needed.
16:57:24 <smcginnis> They will be updating the PTG info to note which teams will have 2 or 3 days.
16:57:27 <smcginnis> That should help.
16:57:30 <jungleboyj> Ahhh!  Was the Cinder meeting at 10 am!?!
16:57:37 <bswartz> I'm not sure if joint sessions would help -- mostly I'll just miss you guys
16:57:41 <smcginnis> If folks haven't seen it yet, registration is officially open.
16:57:45 <smcginnis> jungleboyj: LOL
16:57:51 <jungleboyj> smcginnis: Damn it!
16:57:53 <bswartz> jungleboyj: 1600 UTC
16:58:00 <bswartz> it never moves
16:58:01 <smcginnis> bswartz: We'll miss you too. ;)
16:58:03 <diablo_rojo> jungleboyj, Time change ;)
16:58:13 <jungleboyj> diablo_rojo: Why didn't you remind me!?!
16:58:20 <jungleboyj> :-)
16:58:29 <smcginnis> At least there's Friday.
16:58:44 <bswartz> yeah I'll plan to hang out with yall on friday
16:58:54 <jungleboyj> smcginnis: What is Friday?
16:59:16 <smcginnis> jungleboyj: Oh sure, come in late and expect me to start over from the beginning!
16:59:19 <smcginnis> :P
16:59:27 <smcginnis> jungleboyj: Just talking about the PTG.
16:59:28 * jungleboyj goes to read the logs.
16:59:49 <smcginnis> jungleboyj: Cinder will have three days. Manila will have two, so those working in both can at least join up with us Friday.
17:00:17 <jungleboyj> smcginnis: Ah, good deal.
17:00:39 <smcginnis> Anyone have anything else?
17:00:53 <smcginnis> Oh, times up.
17:00:54 <bswartz> let's build a wall between cinder and nova and make nova pay for it
17:00:55 <smcginnis> Thanks everyone.
17:01:00 <bswartz> oh wait wrong meeting
17:01:00 <smcginnis> bswartz: Hah!
17:01:03 <jungleboyj> bswartz: Nice!
17:01:07 <smcginnis> #endmeeting