15:00:10 <bswartz> #startmeeting manila
15:00:11 <openstack> Meeting started Thu Feb 19 15:00:10 2015 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:15 <openstack> The meeting name has been set to 'manila'
15:00:15 <ganso_> hi
15:00:20 <xyang2> hi
15:00:22 <bswartz> hello
15:00:22 <marcusvrn> hi
15:00:28 <vponomaryov> hello
15:00:55 <bswartz> welcome back all!
15:00:59 <tbarron> hi
15:01:07 <bswartz> I hope you enjoyed the midcycle meetup
15:01:30 <bswartz> I'm feeling better now and my voice has recovered
15:02:06 <bswartz> hmm the agenda isn't really accurate
15:02:28 <bswartz> but let's start with dev status
15:02:30 <bswartz> #topic dev status
15:02:45 <vponomaryov> dev status:
15:02:49 <vponomaryov> 1) Support of Nova network within Generic driver
15:02:54 <vponomaryov> BP: #link https://blueprints.launchpad.net/manila/+spec/nova-network-support-in-generic-driver
15:02:57 <vponomaryov> gerrit: #link https://review.openstack.org/155989
15:03:00 <vponomaryov> status: ready for review
15:03:10 <vponomaryov> 2) Volume types were renamed to share types.
15:03:13 <vponomaryov> BP: #link https://blueprints.launchpad.net/manila/+spec/rename-volume-type-to-share-type
15:03:17 <vponomaryov> status: merged
15:03:20 <vponomaryov> 3) Manila now has "default" share types.
15:03:25 <vponomaryov> BP: #link https://blueprints.launchpad.net/manila/+spec/default-volume-type
15:03:28 <vponomaryov> status: merged
15:03:31 <vponomaryov> 4) Keystone session support in manilaclient
15:03:36 <vponomaryov> BP: #link https://blueprints.launchpad.net/python-manilaclient/+spec/add-keystone-session-support
15:03:39 <vponomaryov> gerrit: #link https://review.openstack.org/153599
15:03:42 <vponomaryov> status: ready for review
15:03:46 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings
15:03:53 <bswartz> (I fixed it up)
15:04:13 <vponomaryov> that's the main
15:04:25 <bswartz> thanks vponomaryov
15:04:57 <bswartz> vponomaryov: what else big is coming before the feature freeze?
15:05:05 <geguileo> thanks
15:05:14 <vponomaryov> bswartz: big?
15:05:28 <bswartz> well, anything big you want to see go in
15:05:34 <bswartz> I have my own list of stuff
15:05:49 <bswartz> I've got my eye on this: https://launchpad.net/manila/+milestone/kilo-3
15:05:57 <vponomaryov> I would like to see nova network plugin for all other drivers
15:06:09 <bswartz> yeah
15:06:18 <bswartz> you and I should talk about that one after the meeting
15:06:26 <vponomaryov> so, community can say do they need it or not
15:06:36 <bswartz> I don't think we'll have a problem
15:06:51 <bswartz> other than nova plugin, anything you think is essential?
15:07:15 <vponomaryov> toher essential are fit k-3
15:07:32 <vponomaryov> like updates of share types
15:07:51 <bswartz> personally I want I see the multiple-export locations get into Kilo
15:08:12 <bswartz> that was a big topic last week and I think it will benefit several drivers
15:08:20 <vponomaryov> agree
15:08:25 <bswartz> the question is whether it's something we can get done in the time we have
15:08:26 <vponomaryov> it is in our task list
15:08:28 <u_glide> Guys, What about private share_types?
15:08:58 <bswartz> u_glide: I'm not sure that will make kilo
15:09:04 <bswartz> do you have the BP link?
15:09:18 <u_glide> not yet, I will create it
15:09:59 <bswartz> oh that's already being worked on I see
15:10:08 <bswartz> yeah we just need the BP targetted at k-3
15:10:37 <bswartz> #topic k-3 deadlines
15:10:46 <bswartz> I just want to remind everyone, we're 2 weeks into k-3
15:10:55 <vponomaryov> why two?
15:10:56 <vponomaryov> 4!
15:11:09 <vponomaryov> march 19 is deadline, is not it?
15:11:10 <xyang2> 3/19
15:11:21 <bswartz> I meant it started 2 weeks ago
15:11:27 <bswartz> so we have 2 weeks left to get code up
15:11:28 <ganso_> lol
15:11:36 <bswartz> and 2 weeks after that to review/merge all features
15:11:46 <ganso_> you just scared everyone
15:11:49 <bswartz> not a lot of time
15:11:51 <nileshb> for bug fixes also?
15:11:53 <bswartz> #link https://wiki.openstack.org/wiki/Kilo_Release_Schedule
15:12:01 <vponomaryov> nileshb: kilo-4 for it
15:12:04 <bswartz> nileshb: features
15:12:15 <nileshb> yeah ok .. thnx
15:12:45 <bswartz> I'm just very nervous about the list of BPs
15:12:55 <bswartz> this time around it won't be like the last 2 milestones
15:12:56 <xyang2> bswartz: do we have until 3/12 or 3/19 to get features merged
15:13:04 <bswartz> where we just let things slips
15:13:06 <xyang2> bswartz: 3/12 is feature freeze
15:13:37 <bswartz> xyang2: we can merge features up to 3/19
15:13:37 <xyang2> maybe that's for oslo
15:13:44 <xyang2> ok
15:13:45 <bswartz> but we'd only wait to the last minute if it was essential
15:14:01 <bswartz> if a driver isn't ready I'll retarget it
15:14:10 <nileshb> any deadline for proposing new BP's? or is it already past?
15:14:13 <bswartz> realistically we can't have multiple open things that last week
15:14:46 <bswartz> nileshb: I care about the code being on gerrit in a mergeable state
15:15:15 <bswartz> if it's a feature for your driver, you're welcome to add a BP, but if the code isn't ready, I'll target it for liberty
15:15:39 <bswartz> if it's a core feature, the BP should be up right now
15:16:25 <xyang2> bswartz: is there a hard requirement on percentage of test coverage?
15:16:37 <bswartz> xyang2: no
15:16:39 <nileshb> we want to support NFS Ganesha 2.2 with GPFS .. BP not yet created .. will create right away
15:16:47 <vponomaryov> xyang2: just do 100% without asking =)
15:16:56 <bswartz> xyang2: are you asking as a reviewer or a submitter?
15:16:59 <xyang2> vponomaryov: of course:)
15:17:21 <xyang2> bswartz: we got a review comment on that, just wondering how much is enough
15:17:29 <xyang2> 100% is ideal of course
15:17:45 <bswartz> I think we should try to push for test coverage as reviewers
15:17:57 <bswartz> but judgement is needed
15:18:05 <bswartz> we don't have a hard requirement
15:18:13 <bswartz> 100% is the goal
15:18:27 <bswartz> sometimes there's code that just makes no sense to unit test though
15:19:10 <bswartz> so I probably need to do a pass over the BPs on launchpad
15:19:21 <vponomaryov> bswartz: our DB functions are not covered with unit tests =)
15:19:39 <vponomaryov> it works on honest word =)
15:19:43 <bswartz> I'm nervous about the ones that say Not Started
15:20:37 <bswartz> we'll keep on reviewing that list as the deadline gets closer
15:20:51 <bswartz> today just make sure that nothing's missing
15:21:12 <bswartz> I might need to add 1 or 2
15:21:24 <bswartz> so there's another topic
15:21:35 <bswartz> #topic api extensions
15:21:47 <bswartz> this is something I wanted to discuss with cknight around, and he's not here
15:21:58 <bswartz> so I think we can revisit it next week
15:22:03 <bswartz> but I wanted to mention what it is
15:22:22 <bswartz> we inherited a bunch of code from cinder, as you all know
15:22:38 <bswartz> and one of the problems with the code is that lots of things are implemented as API extensions
15:22:52 <bswartz> stuff that's really core functionality
15:23:10 <bswartz> now the cinder team is struggling with how to manage these extensions that are really core features of cinder
15:23:18 <bswartz> and I'd like to avoid that fate
15:23:51 <xyang2> I think we need to define what is core functionality. if it is core functionality, does that mean driver is required to implement it?
15:23:57 <bswartz> so next week I want to come up with a plan for where APIs should go and how we plan to version stuff going forward
15:24:19 <bswartz> xyang2: I think that's what cinder tried to do and failed pretty badly
15:24:49 <bswartz> cinder has like 3 different ways of expressing whether a feature is a core feature or an extension
15:24:54 <bswartz> and they don't always agree
15:25:27 <bswartz> we need 1 mechanism, with a clear plan for how to move optional features to required and how to change things in future versions
15:25:53 <bswartz> so we'll talk about this next week when we have cknight because he has some ideas
15:26:08 <xyang2> ok, agree we need to have a plan
15:26:32 <u_glide> Can we merge manage/unmanage without it?
15:26:45 <bswartz> u_glide: without what?
15:27:07 <u_glide> Without plan :)
15:27:16 <bswartz> u_glide: yes I think we have to
15:27:34 <bswartz> it's what we've been doing, so it won't make anything worse
15:27:58 <bswartz> but we should make things better before kilo is done so we don't lock in anything that we don't want to keep going forward
15:28:03 <xyang2> bswartz: u_glide has moved it to extension
15:28:18 <bswartz> that's okay I think for new features
15:28:31 <bswartz> but stuff shouldn't stay as an extension forever -- that's the problem
15:28:50 <xyang2> bswartz: so you are saying new api start as extension, and then we move it to core when it is mature?
15:29:19 <u_glide> sounds good
15:29:25 <bswartz> xyang2: in theory yes -- but cinder said the same thing and it never happened, so I'd like to figure out the in practice part too
15:29:53 <xyang2> bswartz: ok. I remember that came up in cinder meetup
15:30:04 <bswartz> yes that's what's driving my concern
15:30:14 <xyang2> bswartz: so the move will be internal
15:30:23 <xyang2> bswartz: interface won't change
15:30:42 <bswartz> yeah that's a requirement
15:30:50 <bswartz> we don't break backwards compatibility
15:31:28 <u_glide> but probably we can create aliases and deprecate old api
15:31:29 <bswartz> but we need a clear statement of HOW we will move to v2 when we decide to do that, and what that will mean for extensions
15:31:55 <xyang2> u_glide: Is that needed?  I think rest api and cli won't change
15:32:18 <xyang2> u_glide: do you have to change rest api when you move it from core to extension?
15:32:32 <bswartz> xyang2: I don't think so
15:32:32 <xyang2> It's just how API is discovered is different
15:32:34 <u_glide> sometimes it's required to have consistent API
15:32:46 <u_glide> xyang2: Yes, I have
15:32:57 <bswartz> contrib APIs look identical to core APIs from the perspective of the REST client
15:33:05 <xyang2> bswartz:+1
15:33:15 <bswartz> it's a question of how they're documented, supported, and versioned
15:33:25 <xyang2> u_glide: implementation is different, but rest api doesn't change
15:33:43 <vponomaryov> xyang2: it is not in all cases, but can be
15:34:12 <vponomaryov> xyang2: presense/absense of something can make core functionality have or not some optional keys
15:35:01 <bswartz> so I had one other topic that's not on the agenda
15:35:21 <bswartz> #topic multiple export locations
15:35:31 <bswartz> I think it's clear that everyone likes this idea
15:35:39 <bswartz> there are questions about the details though
15:36:44 <bswartz> the main one is, how can a driver change the list of locations after a share is created
15:37:06 <bswartz> imagine I have a glusterfs driver, with a 4 node glusterfs cluster
15:37:18 <bswartz> when I create a share, the driver could provide 4 export locations
15:37:44 <bswartz> if I then expand the cluster to 6 nodes, the share *should* be reachable from 6 locations now
15:37:59 <bswartz> but how does that fact get discovered and how does the database get updated?
15:38:25 <vponomaryov> on periodic basis using new driver interface
15:38:28 <bswartz> my proposal is that we return a model update from ensure_share()
15:38:58 <bswartz> so that at backend startup time the driver can make updates for ALL shares
15:39:11 <bswartz> vponomaryov: my concern with a periodic task is that it won't scale well
15:39:46 <ganso_> isn't a backend restart being required every time to update the number of export locations too much?
15:39:58 <bswartz> because the periodic task would happen (number of backends) x (number of shares) x (task frequency) times
15:40:28 <bswartz> ganso_: that's what I'm trying to figure out
15:40:38 <vponomaryov> we can add API with possibility to force it
15:40:51 <vponomaryov> admin knows when new nodes are added
15:40:51 <bswartz> in theory changes to export locations should only be caused by network topology changes on the storage controller itself
15:41:11 <bswartz> that's something that happens incredibly rarely and the administrator would almost certainly know about it
15:42:05 <bswartz> an admin API would work, but what if the admin doesn't know about it or forgets to call it?
15:42:22 <ganso_> I think the usage of the ensure_share function is great... but I think we may need to understand what could trigger the exports being updated, such as a share-server update, or a backend update... we do not have such scenarios right now
15:42:25 <bswartz> it's something the driver should be able determine on its own
15:43:20 <ganso_> vponomaryov: +1
15:43:21 <bswartz> I just had and idea
15:43:51 <bswartz> vponomaryov: what if there was a period task that the manager called the driver asking it if the network topology changed
15:44:11 <bswartz> and only if the driver says yes, then the manager goes and updates all shares on that backend?
15:44:23 <bswartz> s/period/periodic/
15:44:31 <vponomaryov> bswartz: manager and driver are running in same process
15:44:50 <vponomaryov> so each driver has its own manger
15:44:51 <bswartz> yes, but the driver can't access database and enumerate shares
15:44:58 <vponomaryov> can
15:45:07 <vponomaryov> every driver has attr "db"
15:45:10 <bswartz> and the driver shouldn't make upcalls to the manager
15:45:30 <bswartz> well they shouldn't be touching the DB
15:45:39 <bswartz> if we can make that impossible I'd be happier
15:45:40 <vponomaryov> it is another question
15:45:42 <vponomaryov> they can
15:46:09 <xyang2> ya, driver can access the db without manager
15:46:19 <bswartz> xyang2: do any drivers do that today?
15:46:31 <tbarron> but driver *should* not be doing that, right?
15:46:37 <xyang2> bswartz: I don't remember in manila
15:46:47 <bswartz> maybe we should do an audit
15:46:55 <bswartz> this is another concern we share with cinder
15:47:04 <bswartz> drivers accessing the database is a bad thing
15:47:20 <bswartz> the manager should manage all interaction with the DB
15:47:28 <xyang2> tbarron: I think it depends.  there are may be special cases where drivers needs to check.  it should not be encouraged
15:47:45 <bswartz> xyang2: I think we can eliminate those cases
15:48:11 <vponomaryov> such cases are: update of DB in case of failure with data that can be used for resource release
15:48:14 <tbarron> xyang2: I would like to find those cases and figure a clean way to avoid them w/o removing driver functionality
15:48:38 <bswartz> vponomaryov: yes that's a good case
15:48:51 <tbarron> vponomaryov: yes, cause we don't get to the model update
15:48:51 <bswartz> but we should be able to engineer around that
15:49:26 <bswartz> either by returning a model update before throwing the exception, or maybe even attaching a model update to the exception
15:49:38 <bswartz> we should investigate the options
15:49:39 <vponomaryov> it is already so
15:49:41 <xyang2> I think driver should not write to db, but read is fine
15:49:50 <ganso_> xyang2: +1
15:49:56 <bswartz> xyang2: what's a use case for that?
15:50:10 <vponomaryov> manager already expects share server share data to push to DB within exception
15:50:30 <ganso_> my driver checks for other existing share servers
15:50:40 <xyang2> bswartz: I don't have a specific case, but sometimes it may needs to find additional info that is not provided by manager
15:51:03 <bswartz> xyang2: my worry is that if that happens it maybe point to a design flaw
15:51:35 <bswartz> the driver is supposed to be stateless, and it should get all the info it needs from the manager
15:51:50 <bswartz> in cases like that the fix might be to make the manager pass in more parameters
15:52:15 <xyang2> bswartz: at one point I had to make a performance enhancement for my cinder driver which requires driver access to db, but that problem doesn't exist any more.  manager is handling it now
15:52:34 <tbarron> ganso_: if the manager gave you that info, would that work for you?
15:52:35 <vponomaryov> bswartz: I am not sure it is predictable enough to provide all data that can be useful for share drivers
15:52:51 <ganso_> tbarron: yes
15:52:53 <bswartz> yeah we've had similar situations -- but as you suggested it was really just a workaround for a design flaw
15:53:11 <bswartz> once the design flaw is fixed the driver no longer needs to do that
15:53:35 <xyang2> tbarron: so if the manager can do everything, then driver doesn't have to do it
15:53:37 <bswartz> vponomaryov: it should be an easy problem to bound
15:53:52 <bswartz> any given request should involve exactly 1 share, and everything related to that share
15:53:56 <xyang2> but there are cases we may need to fix the driver first
15:54:24 <ganso_> also, by using neutron_api (or network_plugin) to retrieve network information, isn't it the same as reading db?
15:54:48 <bswartz> ganso_: the network plugins exist to provide that abstraction
15:54:49 <xyang2> ganso_: I think so
15:55:17 <bswartz> network plugins do what they need to do so the drivers don't have to know anything
15:55:35 <tbarron> ganso: I think it's a question of interfaces and module state, not just performance (how many DB accesses get triggered)
15:55:42 <ganso_> then maybe we could code a db abstraction, with select functions that drivers may use and do not interfere, like safe ways to read the db, and do not provide ways to update it, etc
15:55:47 <bswartz> some network plugins may access the DB, while others may talk to external components
15:56:07 <bswartz> ganso_: it's worth considering
15:56:16 <bswartz> ganso_: the current model is pretty clean and nice though
15:56:23 <bswartz> as long as we adhere to it
15:56:47 <bswartz> if there are specific problems we just can't solve, then that's a good time to consider changing the DB interaction model
15:57:01 <bswartz> I don't think we're there yet
15:57:11 <bswartz> anyways, we're nearly out of time
15:57:14 <bswartz> #topic open discussion
15:57:21 <bswartz> any last topics before we have to go?
15:58:00 <bswartz> please update your BPs!
15:58:14 <bswartz> add missing ones and make sure existing ones reflect the true state!
15:58:26 <bswartz> I'm going to do the same right now
15:58:58 <bswartz> #endmeeting