16:04:33 <DuncanT1> #startmeeting cinder
16:04:34 <openstack> Meeting started Wed Feb 13 16:04:33 2013 UTC.  The chair is DuncanT1. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:04:35 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:04:38 <openstack> The meeting name has been set to 'cinder'
16:04:53 * DuncanT1 hopes there are some folks here or he's going to end up looking very silly ;-)
16:05:04 <avishay> I'm here
16:05:04 <winston-d> hi DuncanT1
16:05:13 <rushiagr1> DuncanT1: o/
16:05:13 <kmartin> hello
16:05:21 <DuncanT1> 'lo all
16:05:39 <xyang_> Hi
16:05:43 <DuncanT1> Shall we start with status updates? Who's got things they've been working on?
16:05:48 <bswartz> hi
16:05:53 <kmartin> FC update
16:06:06 <avishay> FC update please
16:06:13 <DuncanT1> #topic FC update
16:06:27 <kmartin> Will submit a patch today on the cinder side as well I hope final patch on the nova side
16:06:41 <kmartin> getting good reviews on the nova side
16:07:27 <kmartin> idea is to submit two review on the cinder side, one for the base FC class
16:07:43 <kmartin> and one for the 3PAR FC driver that dependson the base
16:08:17 <xyang_> kmartin: Is the base FC class mostly empty?
16:08:18 <bswartz> kmartin: sounds like good news!
16:08:24 <DuncanT1> Sounds good, I'm sure people waiting on it will be keen to review.
16:08:25 <kmartin> This of course all depens on the nova changes getting in, but we feel like they will
16:08:32 <kmartin> yes
16:08:32 <avishay> More classes :(
16:08:59 <kmartin> It was suggested in the draft that we should put the base case in now
16:09:50 <kmartin> so all driver would not need to change in the future, some common functions that can go in there will be copy vol to image and image to vol
16:10:19 <avishay> I have something to say about that, but don't want to hijack this topic :)
16:10:21 <kmartin> if we left it out then all drivers would need to change when we added it
16:10:58 <xyang_> kmartin: agree base class should be in first as it doesn't depends on nova changes
16:10:59 <kmartin> yeah, the goal is to beat the rush on review that will happen next week, i
16:11:29 <DuncanT1> avishay: Might as well make your point now as later....
16:11:42 <avishay> My point: https://blueprints.launchpad.net/cinder/+spec/reorganize-connection-specific-cinder-driver-code
16:11:45 <kmartin> one change that came up in the nova reviews is we are changing the type from fibrechan to fibre_channel
16:12:25 <avishay> The whole driver class hierarchy seems off as it stands
16:12:35 <xyang_> kmartin: so cinder driver needs to change type too
16:12:43 <kmartin> avishay: xyang_ and jgrifftith both suggested we seperate
16:13:07 <kmartin> xyang_: yes, it will be fibre_channel instead of fibrechan
16:13:12 <DuncanT1> avishay: While I don't necessarily disagree with the blueprint, there is no way that is going to go into Grizzly
16:13:14 <xyang_> I think it's cleaner to have a FC base class
16:13:25 <DuncanT1> avishay: It would break everybody's driver, in tree and out
16:13:27 <avishay> DuncanT1: that's fine - not necessarily for Grizzly
16:13:57 <avishay> DuncanT1: for Grizzly my driver works fine with FC and iSCSI in one class...just something to think about onward
16:14:19 <rushiagr1> avishay: I agree with your view, but its topic for H release
16:14:24 <DuncanT1> Maybe discuss the class (& file) layout at the summit? There are some divergences of opinion on the subject
16:14:24 <kmartin> that's all I had
16:14:42 <kmartin> DuncanT1: +1
16:14:44 <avishay> rushiagr1: DuncanT1: where do we put summit topics?
16:14:52 <avishay> that's why i made the blueprint
16:15:23 <avishay> there are lots of things that people say "we'll talk about it at the summit" and i don't know if anyone's keeping track :)
16:15:27 <bswartz> avishay: design summit talks open up about a month before the summit
16:15:31 <DuncanT1> avishay: Start a wiki page & dump the link in the cinder channel & mailing list I guess? JGriffith can use it as a starting point then
16:15:46 <avishay> DuncanT1: OK good idea
16:15:47 <kmartin> avishay: good topic for the summit
16:15:53 <DuncanT1> No harm and probably lots of benefit in starting the list early
16:16:04 <avishay> Will do
16:16:05 <DuncanT1> So any more questions on FC?
16:16:08 <rushiagr1> DuncanT1: +1
16:16:31 <avishay> I tested the current FC code with the Storwize/SVC driver - seems to work
16:16:43 <DuncanT1> Good to hear
16:16:43 <xyang_> When is the deadline for submit summit talks?  Is it this week?
16:16:53 <winston-d> all, design summit session submission is open at: http://summit.openstack.org
16:17:03 <rushiagr1> xyang_: I guess its 15th this month
16:17:23 <bswartz> wow they opend early this time
16:17:25 <winston-d> nope, that was for conference
16:17:36 <kmartin> xyang_: the 15th is for conference sessions
16:17:39 <bswartz> yeah confernce talks and design summit talks are different
16:17:39 <winston-d> i mean 15th deadline
16:17:56 <DuncanT1> I don't know when the deadline is, I'm afraid
16:18:08 <rushiagr1> okay
16:18:08 <kmartin> design session I don't believe have open up yet
16:18:32 <winston-d> kmartin: check out summit.openstack.org
16:19:08 <DuncanT1> Moving along...
16:19:10 * kmartin tail between his legs :)
16:19:10 <winston-d> don't you received mail from Thierry(ttx)?
16:19:24 <DuncanT1> #topic minimum driver requirements
16:19:57 <DuncanT1> We discussed this and I think kmartin put it up on the wiki: http://wiki.openstack.org/Cinder
16:20:32 <DuncanT1> I don't think there is much to discuss on the subject, but I wanted people to be aware that there is a current 'official' list for new drivers
16:20:45 <kmartin> DuncanT1: yep
16:20:53 <Yada> Thanks kmartin, guess many of us did not know what Volume Stats need to be ;-)
16:21:16 <Yada> DuncanT1 : where is this list ?
16:21:36 <DuncanT1> The wiki is the list
16:21:56 <kmartin> I can then to the wiki if you think it would help
16:22:05 <kmartin> I can add them^
16:22:09 <Yada> Cause we are working to add our, feature freeze is still the 19th isn't it ?
16:22:24 <DuncanT1> I'd certainly list to see a volume_stats list please :-)
16:22:29 <Yada> martin : will help IMHO
16:22:44 <avishay> DuncanT1: +1
16:22:57 <kmartin> I'll add them today
16:23:03 <DuncanT1> You're happy to take that action on kmartin? Fantastic
16:23:08 <Yada> but questions on this list of features : does it mean all existing and new driver MUST implement all of them ?
16:23:29 <DuncanT1> Yada: It means any new driver is unlikely to pass review without them
16:23:44 <bswartz> Yada: yes
16:23:50 <Yada> fine with me (new driver is coming from us)
16:23:56 <kmartin> DuncanT1: no problem
16:24:00 <avishay> I guess another summit topic is when to toss old drivers in the garbage that don't get updated?
16:24:08 <bswartz> although for new features added in a future release, I think existing drivers get 1 release to catch up with feature parity
16:24:18 <Yada> avishay:  +1
16:24:32 <DuncanT1> Yada: The details of supporting existing drivers needs discussion IMO, but something like one release to catch up or we drop them seems reasonable
16:24:56 <avishay> DuncanT1: +1
16:24:59 <DuncanT1> #action kmartin to document get_driver_stats for wike
16:25:08 <DuncanT1> #action summit session on supporting old drivers
16:25:50 <Yada> DuncantT1 : so feature freeze will have no impact for old drivers (regarding volume stats implementation... ) : correct ?
16:26:16 <DuncanT1> Yada: No, though if we can get them fixed before feature freeze that would be better
16:26:24 <Yada> can imagine
16:26:46 <DuncanT1> Ok, next update? Anybody want to say something on multi-backend perhaps?
16:26:56 <avishay> I'm taking the liberty of removing 'Goals for Folsom release' from the wiki :)
16:27:25 <DuncanT1> avishay: +1
16:27:51 <winston-d> hub_cap is not here
16:27:54 <rushiagr1> hub_cap not present
16:27:58 <DuncanT1> :-(
16:28:12 <DuncanT1> Scheduler status update?
16:28:37 <DuncanT1> Anybody got anything to say on that?
16:29:20 <winston-d> yes
16:29:35 <DuncanT1> The stage is all yours
16:29:56 <kmartin> Yada: Current list of stats if you are still unsure stats = {'driver_version': '1.0',
16:29:57 <kmartin> 'free_capacity_gb': 'unknown',
16:29:57 <kmartin> 'reserved_percentage': 0,
16:29:57 <kmartin> 'storage_protocol': 'iSCSI',
16:29:57 <kmartin> 'total_capacity_gb': 'unknown',
16:29:58 <kmartin> 'vendor_name': 'Hewlett-Packard',
16:30:00 <kmartin> 'volume_backend_name': 'HP3PARISCSIDriver'}
16:30:33 <kmartin> sorry thought the was a private msg
16:30:55 <winston-d> so I was asking around about how to sort capacity if some back-end reports 'unknown' capacity and we kinda agree to add some other weigher if caapcity weigher cannot apply
16:31:39 <winston-d> there are two basic ideas: one is to sort back-ends with allocated bytes, the other is allocated volumes.
16:32:07 <winston-d> these two can be easily queried from DB, so it should apply to all back-end.
16:32:57 <winston-d> the allocated-byte weigher and allocated-volume weigher should be used when capacity weigher cannot apply (in the case when any back-end may report 'unknown' capacity)
16:33:20 <avishay> seems reasonable
16:33:37 <winston-d> feel free to raise question to me if you have any question about filter scheduler.
16:33:43 <rushiagr1> winston-d: isnt it kinda naive to judge on the number of volumes? so a backend with 3 volumes will have weight thrice as much as with 1 volume? (or 1/3rd)
16:34:10 <xyang_> winston-d: are you using both allocated bytes and allocated volumes, or just one of them?
16:34:34 <DuncanT1> rushiagr1: That depends on the backend and what you are trying to optimise placement for.
16:34:37 <winston-d> xyang_: i think just one of them
16:35:47 <winston-d> rushiagr1: yeah, the most straightfoward is to weight be free capacity, unfortunately, not all back-ends are able to report a number
16:37:05 <winston-d> more advantage weighers (such as DuncanT1's idea: weight back-end based on volume activities) may not apply to all back-ends, especailly those simple back-ends, such as lvm
16:37:16 <rushiagr1> hmmm... I was thinking if we can fall back to how simple scheduler works.. because using number of volumes is like skewing on a slightly unreasonable attribute
16:37:55 <Yada> winston-d : +1 need to be flexible to not lock some drivers cause they can not report some figures on this point
16:38:37 <avishay> I think the default scheduler should not require anything from drivers at this point
16:38:49 <DuncanT1> rushiagr1: from simple.py def schedule_create_volume(self, context, request_spec, filter_properties):   """Picks a host that is up and has the fewest volumes."""
16:38:50 <avishay> Allocated bytes seems reasonable
16:39:07 <winston-d> i need to clarify what i am doing is to provide some (at least one) weigher that can be applied to all back-ends, so that we can set that as default weigher in Cinder while we are also providing a few other weighers.
16:39:33 <winston-d> in production, adimn will have to decide which weigher to use, not just follow the default
16:39:51 <rushiagr1> DuncanT1: oh, I thought it said "picks a host that is up and is able to fulfil the request". Thanks for pointing out
16:40:10 <winston-d> rushiagr1: that is 'chance' scheduler
16:40:12 <DuncanT1> rushiagr1: Chance scheduler picks a host that is up at random
16:40:31 <rushiagr1> oh, okay. My bad
16:40:51 <winston-d> avishay: well, filter scheduler requires some volume status to make decisions.
16:41:13 <bswartz> a scheduler that goes by volume count would effectively result in a round-robin policy, which is at least deterministic, if not smart
16:41:26 <avishay> winston-d: volume status?
16:41:47 <winston-d> avishay: volumes stats
16:43:16 <winston-d> anyway, i own you all a document for filter scheduler, sorry about that. i'm working on it, should be ready very soon.
16:43:33 <avishay> winston-d: that would be great - thanks!
16:43:38 <xyang_> winston-d: based on allocated bytes and allocated volumes, can you calculate how much free space is left?
16:43:56 <rushiagr1> winston-d: would be helpful a lot
16:43:57 <xyang_> winston-d: doc will be very helpful.
16:44:11 <winston-d> xyang_: only if back-end can report total capacity
16:44:12 <DuncanT1> xyang_: Not if the backend doesn't know how big it is (and there are some that don't, due to compression, de-dupe, etc)
16:44:15 <avishay> xyang_: except for infinity and unknown
16:44:40 <xyang_> ok
16:45:01 <winston-d> yup, if it was that simple, back-end driver should be able to do it when report free capacity
16:46:23 <avishay> In my driver I report what the controller says in terms of capacity/free-space, but there is thin provisioning, possibly compression...is that right?
16:46:24 <winston-d> oh, by the way, if capacity weigher is used when 'unknown' free capacity is reported, it will be treated the same as 'infinite'.
16:46:31 <winston-d> does that sound reasonable?
16:47:11 <DuncanT1> I can't think of anything else to do...
16:47:18 <winston-d> avishay: i think it should be ok.
16:47:27 <avishay> winston-d: ok, thanks
16:47:43 <avishay> +1 for treating inf and unknown the same
16:47:48 <kmartin> sounds fine with me
16:47:50 <xyang_> winston-d: doesn't seem to be a better way
16:48:17 <winston-d> and please review the re-try patch here: https://review.openstack.org/#/c/20514/
16:48:40 <rushiagr1> I think there is a difference between 'infinity' and 'unknown'
16:49:01 <avishay> rushiagr1: which is larger, infinity or unknown? :P
16:49:34 <winston-d> yeah, if there's a commonly-accepted answer, i can take it.
16:49:35 <avishay> rushiagr1: the question is how to sort - they will be the same
16:49:52 <rushiagr1> there are storage boxes which really can add capacity at a later point of time, bswartz am I correct?
16:49:57 <DuncanT1> rushiagr1: They are different, but specifically when sorting in order of capacity for the capacity weighter, is there any benefit to treating them differently
16:50:16 <xyang_> If it is unknown, you have to assume there's space available.  It will fail if out of space.
16:50:18 <bswartz> rushiagr1: we talked about this last week and I think we have to treat them the same
16:50:36 <bswartz> I can't figure out any practical difference between them, as far as the scheduler is concerend
16:50:46 <rushiagr1> hmm... I agree with that
16:51:01 <Yada> in this case why having two ?
16:51:03 <kmartin> just to be clear it's infinite not infinity
16:51:19 <avishay> at least with winston-d's retry patch if the driver reports incorrectly the volume can be allocated elsewhere
16:51:32 <bswartz> Yada: while the scheduler may not care, something else might want to distinguish between the 2 cases
16:51:39 <kmartin> at least that was the term we agreed on before
16:51:42 * rushiagr1 remembers the saying 'dont go into specifics so much that you forget the bigger picture'
16:51:57 <winston-d> bswartz: +1
16:52:06 <DuncanT1> bswartz: +1
16:52:19 <xyang_> bswartz: +1
16:52:20 <kmartin> bswartz: +1
16:52:24 <DuncanT1> Right, starting to run out of time, so...
16:52:28 <DuncanT1> #topic reviews
16:52:41 <avishay> I submitted a patch for the Storwize/SVC driver - it's ready for review (please take a look!), but keeping it tagged as WIP until the Nova FC patch goes in
16:52:57 <DuncanT1> There are lots of open reviews, and the number is growing not shrinking
16:53:20 <Yada> bswartz:  agree but as long as the something else will not generate issues aswell (impact select back-ends .../...)
16:53:30 <kmartin> avishay: I will try to look at it today
16:53:35 <avishay> kmartin: thanks!
16:54:19 <kmartin> seems a few have a number of+1's but still not getting approved
16:54:21 <avishay> DuncanT1: there are a few patches that have lots of +1s but aren't merged, and some patches that are kind of stale IMO
16:54:27 <DuncanT1> I'm not sure how to get more attention on specific reviews other than by asking in the cinder channel.
16:55:37 <DuncanT1> We are running towards being tight for time for feature freeze now, so 1) Please please test & review what you can... The more +1 a patch gets the more confidence core reviews have in the patch 2) Please shout up if you feel a patch is being ignored
16:56:06 <DuncanT1> avishay: If you think a patch is stale, can you put a note either in a PM or in the cider channel please?
16:56:18 <avishay> DuncanT1: yep
16:57:14 <jgallard> hello all, I'm taking a look at the multi-backend patch
16:58:04 <DuncanT1> jgallard: How are you getting on?
16:58:54 <jgallard> I'm testing the last patch (v3)
16:59:15 <jgallard> and it's seems to be OK now (I had an issue with greenthread)
16:59:30 <jgallard> I will add a comment on the gerrit review
16:59:45 <DuncanT1> jgallard: Please do chime in on the review with you results, positive or negative
17:00:00 <DuncanT1> #topic Any other business
17:00:48 <rushiagr1> The work for NAS as a service is into unit test phase
17:01:10 <DuncanT1> Anybody who has opinions on how AZs work, I'd appreciate feedback on https://review.openstack.org/#/c/21672/
17:01:38 <DuncanT1> rushiagr1: Nice. Think you're on track for merge?
17:02:39 <rushiagr1> DuncanT1: yes, but seems like not many people have reviewed it
17:03:11 <rushiagr1> DuncanT1: it would be problematic if someone comes up asking for a major overhaul in the last week
17:04:11 <DuncanT1> rushiagr1: Always a risk, sadly. I suggest asking around for reviewers... I will try to have another look too
17:04:18 <rushiagr1> so I am just assuming people are more or less fine with it
17:04:32 <DuncanT1> Ok, I think somebody else will want this room now... Thanks everybody
17:04:38 <rushiagr1> okay. Your review would be helpful
17:04:48 <DuncanT1> See you over in #openstack-cinder
17:04:54 <DuncanT1> #endmeeting