15:00:44 <bswartz> #startmeeting manila
15:00:45 <openstack> Meeting started Thu Feb 26 15:00:44 2015 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:46 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:49 <openstack> The meeting name has been set to 'manila'
15:00:51 <bswartz> hello all
15:00:56 <ganso_> hello
15:00:56 <xyang2> hi
15:00:56 <vponomaryov> hello
15:00:58 <geguileo> Hi bswartz
15:01:03 <geguileo> Hi all
15:01:04 <markstur_> hi
15:01:06 <csaba> hi
15:01:11 <bswartz> so I'm buried in deep snow and without power
15:01:19 <bswartz> using backup power and backup internet
15:01:27 <bswartz> if I disappear, you know why....
15:01:40 <bswartz> but hopefully things will hold out long enough for this meeting
15:01:48 <xyang2> bswartz: in nc?
15:01:53 * bswartz crosses fingers
15:01:59 <xyang2> bswartz: you are not in boston, right
15:02:13 <bswartz> xyang2: yes, raleigh/durham got a large snowstorm last night
15:02:25 <bswartz> (large for us anyways)
15:02:48 <u_glide> hello all
15:02:57 <xyang2> bswartz: I see.  we had too much snow already
15:03:08 <bswartz> okay so, I didn't update the agenda
15:03:16 <bswartz> but we have 1 item from last week that needs discussing
15:03:22 <bswartz> cknight: are you here?
15:04:00 <bswartz> hmm
15:04:01 <ganso_> I remember last week we did not come to a conclusion on 2 topics: removing db access from share driver, and multiple export locations
15:04:14 <bswartz> ok
15:04:25 <bswartz> we will have to pend the discussion around api extensions another week it seems
15:04:39 <bswartz> unless someone has done some research in the last week
15:04:58 <bswartz> I still haven't heard any proposals that sound like an improvement on the status quo
15:05:14 <bswartz> ganso_: we can cover those 2 topics
15:05:51 <bswartz> first let's update dev status
15:05:56 <bswartz> because that covers one topic
15:05:58 <bswartz> #topic dev status
15:06:08 <vponomaryov> Dev status:
15:06:24 <vponomaryov> 1) Manila now can be installed using upstream devstack without extra files
15:06:29 <vponomaryov> #link https://github.com/openstack/manila/tree/master/devstack
15:06:41 <vponomaryov> 2) Manilaclient now uses keystoneclient
15:06:47 <vponomaryov> BP: #link https://blueprints.launchpad.net/python-manilaclient/+spec/add-keystone-session-support
15:06:51 <vponomaryov> gerrit: #link https://review.openstack.org/153599
15:06:59 <vponomaryov> status: merged
15:07:05 <cknight> Hi
15:07:12 <vponomaryov> 3) Separate gigabytes quota for snapshots
15:07:16 <bswartz> welcome cknight!
15:07:16 <vponomaryov> BP: #link https://blueprints.launchpad.net/manila/+spec/add-snapshot-gb-quota
15:07:32 <vponomaryov> gerrit:
15:07:32 <vponomaryov> server: #link https://review.openstack.org/158708
15:07:32 <vponomaryov> client: #link https://review.openstack.org/158795
15:07:32 <vponomaryov> status: ready for review
15:07:39 <vponomaryov> 4) Private share types
15:07:43 <vponomaryov> BP: #link https://blueprints.launchpad.net/manila/+spec/private-share-types
15:07:46 <vponomaryov> gerrit: #link https://review.openstack.org/157794
15:07:46 <vponomaryov> status: ready for review
15:07:56 <vponomaryov> that's the main
15:08:10 <bswartz> vponomaryov:: no update on multiple export locations?
15:08:29 <vponomaryov> no
15:08:34 <bswartz> okay
15:08:39 <bswartz> we will cover that below
15:08:50 <bswartz> since we have cknight I'd like to cover the api extension topic
15:09:07 <bswartz> #topic API extensions and versioning
15:09:54 <bswartz> so as I mentioned last week, a large fraction of our rest API is actually implemented using extensions (they're in the contrib dir)
15:10:05 <bswartz> this is something we inherited from cinder
15:10:34 <bswartz> the cinder team is facing problems with the extensions APIs because they haven't figured out a way to version them
15:11:01 <bswartz> I would like to solve that problem before we have an urgent need to change an API in a way that requires versioning
15:11:22 <bswartz> so the open questions are:
15:11:45 <bswartz> 1) how can we change the existing contrib APIs to core APIs without breaking existing clients
15:12:16 <bswartz> 2) what is the process for deciding which APIs should be contrib APIs vs core APIs
15:12:29 <bswartz> 3) and how can we handle versioning of contrib APis
15:12:42 <bswartz> cknight: does that about cover it?
15:12:50 <cknight> bswartz: I think so
15:13:08 <bswartz> cknight: any ideas on how to answer those?
15:13:24 <cknight> For #1, the URLs wouldn't change if we move extension APIs to v1
15:13:35 <cknight> But the question concerns the API to list extensions
15:13:42 <cknight> and if anyone is using that
15:13:46 <bswartz> I would like to at least have a plan before Kilo closes down, so the API we ship with kilo can be as stable as possible going into liberty
15:13:48 <vponomaryov> cknight: urls will be changed
15:13:57 <cknight> vponomaryov:  how?
15:14:09 <vponomaryov> extensions has template os-foo-bar
15:14:18 <vponomaryov> core APis foo-bar
15:14:34 <vponomaryov> s/has/have/
15:14:55 <xyang2> vponomaryov: can you use the manage api as an example
15:15:21 <cknight> vponomaryov: well, that's a problem then.  My understanding is that extensions are a mechanism for users to add their own APIs, or possibly for projects to ship experimental APIs.
15:15:37 <vponomaryov> https://review.openstack.org/#/c/147495/26/manila/api/contrib/share_manage.py - line 108
15:15:53 <vponomaryov> as extension it is "os-share-manage"
15:16:01 <vponomaryov> as core API - should be "share-manage"
15:16:02 <bswartz> vponomaryov: what about types_manage.py?
15:16:23 <vponomaryov> what about them?
15:16:28 <bswartz> share types are contrib APIs
15:16:38 <bswartz> but the REST URL is /types
15:17:37 <bswartz> I think that cknight was pointing out that we don't *have* to change the API URL when we move it from contrib to core -- the name is just by convention
15:17:56 <bswartz> part of the problem is that we don't even enforce the convention very well
15:18:22 <cknight> bswartz: that's true.  we could clean up the convention later in v2, but we have to move away from using extensions like we are now.
15:19:08 <vponomaryov> bswartz: types case mean it did not satisfy contrib url naming =)
15:19:09 <cknight> I'd prefer to move everything into v1, but also add a way for APIs to be marked supported (i.e. not changing) or experimental (subject to change)
15:19:28 <bswartz> vponomaryov: correct -- but it should never have been a contrib API to begin with, as it's a core feature
15:19:43 <cknight> Then the extension mechanism is free for deployers to use.
15:19:59 <cknight> Arguably, every API we have now is core, not an optional extension.
15:20:08 <bswartz> I agree with cknight -- we need to clean up the existing APIs and start enforcing whatever convention we agree upon in the future
15:20:59 <bswartz> as for the badly-chosen REST URLs, we can introduce the correct ones, and keep the old ones for backwards compatibility
15:21:02 <cknight> If we need optional APIs, they could be marked so in the API status, but optional APIs have a way of becoming widely used and therefore standard over time.
15:21:16 <bswartz> then when it's time to move to v2, we can drop support for all the old ones
15:21:23 <cknight> bswartz:  +1
15:21:29 <bswartz> but getting everything moved from contrib into v1 is essential before moving to v2
15:21:53 <cknight> The API that lists extensions could be enhanced to list all APIs, including their version and status.
15:22:08 <cknight> So users could always know what APIs are available.
15:22:21 <bswartz> cknight: regarding that last part, I want to know more about how other projects are handling that
15:22:25 <xyang2> If you use API V1 to list extensions, it will only list extensions under v1
15:22:39 <bswartz> there are interactions with keystone and the keystone service catalog that many clients rely on
15:22:43 <cknight> xyang2: yes, that's certainly one way to do it.
15:23:31 <cknight> Does anyone else have suggestions on how to solve this?
15:23:35 <xyang2> cknight: contrib apis are under versioned folder where they are discovered
15:23:41 <bswartz> okay based on the discussion, this is sounding like a large piece of work
15:24:19 <bswartz> we have to make some decisions about what naming convention we want to follow, and what we plan to enforce regarding additions of new APIs in the future
15:24:23 <cknight> bswartz: yes, but it can be done piecemeal once we have a direction
15:24:34 <bswartz> and we need to do some experiments before we can make those decisions
15:24:46 <bswartz> and then we need to start the cleanup effort
15:25:19 <bswartz> none of those things should be undertaken during K-4, and we're practically out of time for K-3
15:26:04 <cknight> bswartz: +1, but experiments can start any time.  I'm glad to help get this written down and test the ideas.
15:26:07 <xyang2> bswartz: A core API doesn't mean it is required for every driver to implement it though, right?  Otherwise, every API is required
15:26:11 <bswartz> I'll take some notes on the topic and bring it up for discussion again
15:26:22 <cknight> bswartz: But I'm keen to hear if anyone else has ideas.
15:26:35 <bswartz> xyang2: the question of what features drivers must support is a separate question
15:26:49 <bswartz> it's related, but only loosely
15:27:33 <bswartz> I think an API should have 3 possible states:
15:27:46 <bswartz> 1) experimental API in contrib, supported by some drivers
15:28:11 <bswartz> 2) core API, supported by some drivers
15:28:19 <bswartz> 3) core API, supported by all drivers
15:29:02 <cknight> bswartz: That's fine, but we'll need a convention about contrib APIs, like they only live there for one release before we decide what to do with it.
15:29:48 <bswartz> hmmm, I'm not sure about that
15:30:07 <bswartz> I can imagine some new features might go straight to the core API
15:30:19 <bswartz> and some features might remain experimental for several releases
15:30:33 <u_glide> bswartz:+1
15:30:49 <cknight> bswartz: OK, I can see that.  We just want to be sure we don't get stuck with unversioned APIs like Cinder.
15:31:03 <bswartz> yes exactly
15:31:43 <bswartz> anyways, expect to hear more about this
15:31:56 <bswartz> we need to make decisions, but I still don't have enough information
15:32:07 <bswartz> #topic public service announcement
15:32:12 <xyang2> what is the immediate plan for the manage/unmanage?  move back to core?
15:32:51 <bswartz> this is a reminder that FPF (Feature proposal freeze) is one week away
15:33:24 <bswartz> all new features for Kilo need to be in gerrit and passing jenkins (and substantially complete) by March 5
15:33:42 <bswartz> that gives the team 2 weeks to review, and for developers to respond to feedback
15:34:11 <u_glide> xyang2: I think, we have decided to leave it in contrib on last meeting
15:34:13 <bswartz> the merge deadline is still March 19
15:34:45 <xyang2> u_glide: ok
15:35:01 <bswartz> xyang2: I'm okay with it remaining as contrib until we sort out our intentions
15:35:28 <bswartz> I'm more interested in the public-facing REST URL than the place it lives actually
15:35:29 <xyang2> bswartz: sure. I don't want to ask u_glide to move it back and forth
15:35:40 <bswartz> because that's the part we promise not to break
15:35:57 <xyang2> bswartz: +1
15:36:33 <bswartz> okay
15:36:50 <bswartz> ganso's topics
15:37:00 <bswartz> #topic DB access in the share drivers
15:37:10 <bswartz> driver shouldn't be accessing the DB
15:37:21 <vponomaryov> RO access is OK
15:37:26 <bswartz> If I had noticed it happening, I would have -2 it
15:37:26 <vponomaryov> don't you think so?
15:37:36 <bswartz> I don't think so
15:37:50 <xyang2> bswartz: I think that is too restrict
15:37:50 <bswartz> if drivers need info from the database, then the manager should be providing it
15:38:10 <bswartz> I don't have a problem with changes to the ShareDriver interface to allow passing in more information
15:38:35 <bswartz> In fact I'm in favor of adding a lot more information, as I mentioned last week with the driver-specific per-share-metadata
15:38:51 <bswartz> but those accesses need to go through a layer -- not direct to DB
15:39:08 <bswartz> otherwise we have big problems when we do DB schema changes
15:39:22 <ganso_> that would be the "DB plugin", similar to network plugin
15:39:32 <ganso_> bswartz: +1, DB changes would break drivers
15:39:52 <bswartz> I'd like to address the problems on a case-by-case basis
15:40:17 <bswartz> if drivers have a good reason for pulling data out of the database, then the share manager should simply provide it so they don't have to get it themselves
15:40:17 <xyang2> bswartz: +1.  we should look at each case
15:40:59 <bswartz> and driver shouldn't be writing to the database -- all updates should happen through model updates handled by the manager
15:41:36 <bswartz> is there a specific case that we can discuss today?
15:41:42 <cknight> As an example, there are use cases where a Manila driver doesn't go directly to the storage controller, but instead contacts a remote orchestration layer.  So all required DB info would have to be passed along.
15:41:54 <cknight> We have customers doing exactly that.
15:42:20 <bswartz> cknight: I think you mean info *from* the DB, not info about the DB
15:42:30 <cknight> bswartz:  Correct.  Which drivers are accessing the DB?
15:42:31 <bswartz> we don't want remote things talking to the Manila DB
15:42:47 <bswartz> ganso_: You brought this one up, are you aware of an issue?
15:42:57 <xyang2> bswartz: heat may have to do that
15:43:09 <bswartz> I might need to write myself an AI to go hunting for violations
15:43:09 <ganso_> I don't know exactly if I coded it the wrong way, but I have a driver (not submitted yet) that handles share servers, and since I manage a lot of network code, I need to query for existing share server from the DB
15:43:23 <bswartz> ah
15:43:35 <bswartz> that's something that the manager should be doing for you
15:43:48 <bswartz> is it going to be submitted in kilo?
15:43:55 <bswartz> your time is running out...
15:44:00 <ganso_> no, not anymore
15:44:20 <ganso_> but I am discussing this thinking already as a future improvement
15:44:34 <bswartz> ok
15:44:39 <bswartz> other topic was
15:44:45 <bswartz> #topic multiple export locations
15:44:53 <bswartz> u_glide is working on this
15:45:10 <bswartz> u_glide: is there a change on gerrit yet that people can look at?
15:45:18 <cknight> ganso_: Please submit a BP on the manager improvements you need to get share server info.  We can consider that in Liberty.
15:45:31 <u_glide> bswartz: not yet
15:45:53 <bswartz> u_glide: do you have an estimate?
15:46:52 <u_glide> bswartz: I need 2-3 days
15:46:56 <bswartz> okay
15:47:00 <ganso_> cknight: I will, thanks
15:47:41 <bswartz> so the feature will hopefully go into kilo, but drivers might not implement it until liberty, given how late we are in the release
15:48:06 <cknight> bswartz: Would you consider an FFE for that?  It's a simple driver change.
15:48:57 <bswartz> the design for this feature consists of a change to the DB schema, change to the REST API (backwards compatible, of course), and a change to the model update drivers provide to the manager when creating shares
15:49:27 <cknight> bswartz: It seems odd to put in a feature with no driver support, since updating at least one driver is very useful for proving out the feature design.
15:49:30 <bswartz> the last and most tricky part is that, driver may change the list of export locations, but only by restarting the backend (currently)
15:50:09 <bswartz> cknight: I agree, but this always happens with new features that require driver changes
15:50:21 <bswartz> if you make them late, then very few drivers can support them
15:50:32 <xyang2> bswartz: we should have one reference implementation
15:50:46 <bswartz> yes I wouldn't want to merge the feature without at least one driver supporting it
15:50:55 <bswartz> but that's a pretty low bar
15:51:18 <xyang2> that's always the case with new features added late
15:51:21 <bswartz> anything not merged has a risk of slipping to Liberty -- this change included
15:51:48 <bswartz> but due to the high value of the feature I'd like to try to get it in even with limitted driver support
15:52:40 <bswartz> to answer cknight's earlier question, this would be a good case to argue for a FFE, if you can make a few lines of change to the driver to support the new feature
15:52:55 <cknight> bswartz: Thanks, we can probably make that happen, then.
15:53:19 <bswartz> but such an FFE would be limitted to a small change that just supported this feature because it went in so late (assuming it make it in)
15:53:39 <bswartz> don't expect a larger change to be allowed to go in late because it also includes support for this small feature
15:53:55 <bswartz> okay that's about all I have
15:54:02 <xyang2> bswartz: any update on tags?
15:54:07 <bswartz> amazingly my backup power is still going
15:54:12 <bswartz> #topic open discussion
15:54:25 <bswartz> xyang2: I don't know if you made the TC meeting on Tuesday
15:54:34 <bswartz> they discussed one tag only
15:54:40 <bswartz> the release: tag
15:55:23 <bswartz> the release tag is the one we have the fewest problems with because we're perfectly integrated into the openstack release schedule
15:55:36 <bswartz> but the TC hasn't closed on how they want to handle that one
15:55:52 <bswartz> and until they do, I don't expect that they'll define any other tags
15:56:08 <xyang2> bswartz: ok
15:56:13 <vponomaryov> how is it influences us?
15:56:21 <bswartz> I'm more interested in how they view maturity and integration between projects
15:56:37 <bswartz> we continue to be frozen out of other projects from an integration perspective
15:56:42 <xyang2> bswartz: the tag is supposed to tell us that?
15:57:02 <bswartz> so we have to do slightly awkward things like the devstack plugin  vponomaryov added last week
15:57:05 <xyang2> bswartz: if manila gets integrated_release tag for Kilo, that will be good
15:57:28 <bswartz> xyang2: yes I think that one should be easy for us to obtain, but I'm afraid it won't have any value
15:57:51 <bswartz> it will merely be a statement of fact that our release is integrated with openstack, which has been true since icehouse
15:58:33 <bswartz> maturity obviously will require production deployments, and those are coming along well
15:58:55 <bswartz> the key is going to be getting the deployers to join the community and contribute back
15:59:14 <bswartz> okay we're almost out of time
15:59:22 <bswartz> any last minute things?
15:59:25 <bswartz> I have one...
15:59:47 <bswartz> http://lists.openstack.org/pipermail/openstack-dev/2015-February/057854.html
15:59:51 <bswartz> read that ML post if you haven't
15:59:53 <bswartz> thanks all!
16:00:05 <bswartz> #endmeeting