15:00:44 #startmeeting manila 15:00:45 Meeting started Thu Feb 26 15:00:44 2015 UTC and is due to finish in 60 minutes. The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:46 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:49 The meeting name has been set to 'manila' 15:00:51 hello all 15:00:56 hello 15:00:56 hi 15:00:56 hello 15:00:58 Hi bswartz 15:01:03 Hi all 15:01:04 hi 15:01:06 hi 15:01:11 so I'm buried in deep snow and without power 15:01:19 using backup power and backup internet 15:01:27 if I disappear, you know why.... 15:01:40 but hopefully things will hold out long enough for this meeting 15:01:48 bswartz: in nc? 15:01:53 * bswartz crosses fingers 15:01:59 bswartz: you are not in boston, right 15:02:13 xyang2: yes, raleigh/durham got a large snowstorm last night 15:02:25 (large for us anyways) 15:02:48 hello all 15:02:57 bswartz: I see. we had too much snow already 15:03:08 okay so, I didn't update the agenda 15:03:16 but we have 1 item from last week that needs discussing 15:03:22 cknight: are you here? 15:04:00 hmm 15:04:01 I remember last week we did not come to a conclusion on 2 topics: removing db access from share driver, and multiple export locations 15:04:14 ok 15:04:25 we will have to pend the discussion around api extensions another week it seems 15:04:39 unless someone has done some research in the last week 15:04:58 I still haven't heard any proposals that sound like an improvement on the status quo 15:05:14 ganso_: we can cover those 2 topics 15:05:51 first let's update dev status 15:05:56 because that covers one topic 15:05:58 #topic dev status 15:06:08 Dev status: 15:06:24 1) Manila now can be installed using upstream devstack without extra files 15:06:29 #link https://github.com/openstack/manila/tree/master/devstack 15:06:41 2) Manilaclient now uses keystoneclient 15:06:47 BP: #link https://blueprints.launchpad.net/python-manilaclient/+spec/add-keystone-session-support 15:06:51 gerrit: #link https://review.openstack.org/153599 15:06:59 status: merged 15:07:05 Hi 15:07:12 3) Separate gigabytes quota for snapshots 15:07:16 welcome cknight! 15:07:16 BP: #link https://blueprints.launchpad.net/manila/+spec/add-snapshot-gb-quota 15:07:32 gerrit: 15:07:32 server: #link https://review.openstack.org/158708 15:07:32 client: #link https://review.openstack.org/158795 15:07:32 status: ready for review 15:07:39 4) Private share types 15:07:43 BP: #link https://blueprints.launchpad.net/manila/+spec/private-share-types 15:07:46 gerrit: #link https://review.openstack.org/157794 15:07:46 status: ready for review 15:07:56 that's the main 15:08:10 vponomaryov:: no update on multiple export locations? 15:08:29 no 15:08:34 okay 15:08:39 we will cover that below 15:08:50 since we have cknight I'd like to cover the api extension topic 15:09:07 #topic API extensions and versioning 15:09:54 so as I mentioned last week, a large fraction of our rest API is actually implemented using extensions (they're in the contrib dir) 15:10:05 this is something we inherited from cinder 15:10:34 the cinder team is facing problems with the extensions APIs because they haven't figured out a way to version them 15:11:01 I would like to solve that problem before we have an urgent need to change an API in a way that requires versioning 15:11:22 so the open questions are: 15:11:45 1) how can we change the existing contrib APIs to core APIs without breaking existing clients 15:12:16 2) what is the process for deciding which APIs should be contrib APIs vs core APIs 15:12:29 3) and how can we handle versioning of contrib APis 15:12:42 cknight: does that about cover it? 15:12:50 bswartz: I think so 15:13:08 cknight: any ideas on how to answer those? 15:13:24 For #1, the URLs wouldn't change if we move extension APIs to v1 15:13:35 But the question concerns the API to list extensions 15:13:42 and if anyone is using that 15:13:46 I would like to at least have a plan before Kilo closes down, so the API we ship with kilo can be as stable as possible going into liberty 15:13:48 cknight: urls will be changed 15:13:57 vponomaryov: how? 15:14:09 extensions has template os-foo-bar 15:14:18 core APis foo-bar 15:14:34 s/has/have/ 15:14:55 vponomaryov: can you use the manage api as an example 15:15:21 vponomaryov: well, that's a problem then. My understanding is that extensions are a mechanism for users to add their own APIs, or possibly for projects to ship experimental APIs. 15:15:37 https://review.openstack.org/#/c/147495/26/manila/api/contrib/share_manage.py - line 108 15:15:53 as extension it is "os-share-manage" 15:16:01 as core API - should be "share-manage" 15:16:02 vponomaryov: what about types_manage.py? 15:16:23 what about them? 15:16:28 share types are contrib APIs 15:16:38 but the REST URL is /types 15:17:37 I think that cknight was pointing out that we don't *have* to change the API URL when we move it from contrib to core -- the name is just by convention 15:17:56 part of the problem is that we don't even enforce the convention very well 15:18:22 bswartz: that's true. we could clean up the convention later in v2, but we have to move away from using extensions like we are now. 15:19:08 bswartz: types case mean it did not satisfy contrib url naming =) 15:19:09 I'd prefer to move everything into v1, but also add a way for APIs to be marked supported (i.e. not changing) or experimental (subject to change) 15:19:28 vponomaryov: correct -- but it should never have been a contrib API to begin with, as it's a core feature 15:19:43 Then the extension mechanism is free for deployers to use. 15:19:59 Arguably, every API we have now is core, not an optional extension. 15:20:08 I agree with cknight -- we need to clean up the existing APIs and start enforcing whatever convention we agree upon in the future 15:20:59 as for the badly-chosen REST URLs, we can introduce the correct ones, and keep the old ones for backwards compatibility 15:21:02 If we need optional APIs, they could be marked so in the API status, but optional APIs have a way of becoming widely used and therefore standard over time. 15:21:16 then when it's time to move to v2, we can drop support for all the old ones 15:21:23 bswartz: +1 15:21:29 but getting everything moved from contrib into v1 is essential before moving to v2 15:21:53 The API that lists extensions could be enhanced to list all APIs, including their version and status. 15:22:08 So users could always know what APIs are available. 15:22:21 cknight: regarding that last part, I want to know more about how other projects are handling that 15:22:25 If you use API V1 to list extensions, it will only list extensions under v1 15:22:39 there are interactions with keystone and the keystone service catalog that many clients rely on 15:22:43 xyang2: yes, that's certainly one way to do it. 15:23:31 Does anyone else have suggestions on how to solve this? 15:23:35 cknight: contrib apis are under versioned folder where they are discovered 15:23:41 okay based on the discussion, this is sounding like a large piece of work 15:24:19 we have to make some decisions about what naming convention we want to follow, and what we plan to enforce regarding additions of new APIs in the future 15:24:23 bswartz: yes, but it can be done piecemeal once we have a direction 15:24:34 and we need to do some experiments before we can make those decisions 15:24:46 and then we need to start the cleanup effort 15:25:19 none of those things should be undertaken during K-4, and we're practically out of time for K-3 15:26:04 bswartz: +1, but experiments can start any time. I'm glad to help get this written down and test the ideas. 15:26:07 bswartz: A core API doesn't mean it is required for every driver to implement it though, right? Otherwise, every API is required 15:26:11 I'll take some notes on the topic and bring it up for discussion again 15:26:22 bswartz: But I'm keen to hear if anyone else has ideas. 15:26:35 xyang2: the question of what features drivers must support is a separate question 15:26:49 it's related, but only loosely 15:27:33 I think an API should have 3 possible states: 15:27:46 1) experimental API in contrib, supported by some drivers 15:28:11 2) core API, supported by some drivers 15:28:19 3) core API, supported by all drivers 15:29:02 bswartz: That's fine, but we'll need a convention about contrib APIs, like they only live there for one release before we decide what to do with it. 15:29:48 hmmm, I'm not sure about that 15:30:07 I can imagine some new features might go straight to the core API 15:30:19 and some features might remain experimental for several releases 15:30:33 bswartz:+1 15:30:49 bswartz: OK, I can see that. We just want to be sure we don't get stuck with unversioned APIs like Cinder. 15:31:03 yes exactly 15:31:43 anyways, expect to hear more about this 15:31:56 we need to make decisions, but I still don't have enough information 15:32:07 #topic public service announcement 15:32:12 what is the immediate plan for the manage/unmanage? move back to core? 15:32:51 this is a reminder that FPF (Feature proposal freeze) is one week away 15:33:24 all new features for Kilo need to be in gerrit and passing jenkins (and substantially complete) by March 5 15:33:42 that gives the team 2 weeks to review, and for developers to respond to feedback 15:34:11 xyang2: I think, we have decided to leave it in contrib on last meeting 15:34:13 the merge deadline is still March 19 15:34:45 u_glide: ok 15:35:01 xyang2: I'm okay with it remaining as contrib until we sort out our intentions 15:35:28 I'm more interested in the public-facing REST URL than the place it lives actually 15:35:29 bswartz: sure. I don't want to ask u_glide to move it back and forth 15:35:40 because that's the part we promise not to break 15:35:57 bswartz: +1 15:36:33 okay 15:36:50 ganso's topics 15:37:00 #topic DB access in the share drivers 15:37:10 driver shouldn't be accessing the DB 15:37:21 RO access is OK 15:37:26 If I had noticed it happening, I would have -2 it 15:37:26 don't you think so? 15:37:36 I don't think so 15:37:50 bswartz: I think that is too restrict 15:37:50 if drivers need info from the database, then the manager should be providing it 15:38:10 I don't have a problem with changes to the ShareDriver interface to allow passing in more information 15:38:35 In fact I'm in favor of adding a lot more information, as I mentioned last week with the driver-specific per-share-metadata 15:38:51 but those accesses need to go through a layer -- not direct to DB 15:39:08 otherwise we have big problems when we do DB schema changes 15:39:22 that would be the "DB plugin", similar to network plugin 15:39:32 bswartz: +1, DB changes would break drivers 15:39:52 I'd like to address the problems on a case-by-case basis 15:40:17 if drivers have a good reason for pulling data out of the database, then the share manager should simply provide it so they don't have to get it themselves 15:40:17 bswartz: +1. we should look at each case 15:40:59 and driver shouldn't be writing to the database -- all updates should happen through model updates handled by the manager 15:41:36 is there a specific case that we can discuss today? 15:41:42 As an example, there are use cases where a Manila driver doesn't go directly to the storage controller, but instead contacts a remote orchestration layer. So all required DB info would have to be passed along. 15:41:54 We have customers doing exactly that. 15:42:20 cknight: I think you mean info *from* the DB, not info about the DB 15:42:30 bswartz: Correct. Which drivers are accessing the DB? 15:42:31 we don't want remote things talking to the Manila DB 15:42:47 ganso_: You brought this one up, are you aware of an issue? 15:42:57 bswartz: heat may have to do that 15:43:09 I might need to write myself an AI to go hunting for violations 15:43:09 I don't know exactly if I coded it the wrong way, but I have a driver (not submitted yet) that handles share servers, and since I manage a lot of network code, I need to query for existing share server from the DB 15:43:23 ah 15:43:35 that's something that the manager should be doing for you 15:43:48 is it going to be submitted in kilo? 15:43:55 your time is running out... 15:44:00 no, not anymore 15:44:20 but I am discussing this thinking already as a future improvement 15:44:34 ok 15:44:39 other topic was 15:44:45 #topic multiple export locations 15:44:53 u_glide is working on this 15:45:10 u_glide: is there a change on gerrit yet that people can look at? 15:45:18 ganso_: Please submit a BP on the manager improvements you need to get share server info. We can consider that in Liberty. 15:45:31 bswartz: not yet 15:45:53 u_glide: do you have an estimate? 15:46:52 bswartz: I need 2-3 days 15:46:56 okay 15:47:00 cknight: I will, thanks 15:47:41 so the feature will hopefully go into kilo, but drivers might not implement it until liberty, given how late we are in the release 15:48:06 bswartz: Would you consider an FFE for that? It's a simple driver change. 15:48:57 the design for this feature consists of a change to the DB schema, change to the REST API (backwards compatible, of course), and a change to the model update drivers provide to the manager when creating shares 15:49:27 bswartz: It seems odd to put in a feature with no driver support, since updating at least one driver is very useful for proving out the feature design. 15:49:30 the last and most tricky part is that, driver may change the list of export locations, but only by restarting the backend (currently) 15:50:09 cknight: I agree, but this always happens with new features that require driver changes 15:50:21 if you make them late, then very few drivers can support them 15:50:32 bswartz: we should have one reference implementation 15:50:46 yes I wouldn't want to merge the feature without at least one driver supporting it 15:50:55 but that's a pretty low bar 15:51:18 that's always the case with new features added late 15:51:21 anything not merged has a risk of slipping to Liberty -- this change included 15:51:48 but due to the high value of the feature I'd like to try to get it in even with limitted driver support 15:52:40 to answer cknight's earlier question, this would be a good case to argue for a FFE, if you can make a few lines of change to the driver to support the new feature 15:52:55 bswartz: Thanks, we can probably make that happen, then. 15:53:19 but such an FFE would be limitted to a small change that just supported this feature because it went in so late (assuming it make it in) 15:53:39 don't expect a larger change to be allowed to go in late because it also includes support for this small feature 15:53:55 okay that's about all I have 15:54:02 bswartz: any update on tags? 15:54:07 amazingly my backup power is still going 15:54:12 #topic open discussion 15:54:25 xyang2: I don't know if you made the TC meeting on Tuesday 15:54:34 they discussed one tag only 15:54:40 the release: tag 15:55:23 the release tag is the one we have the fewest problems with because we're perfectly integrated into the openstack release schedule 15:55:36 but the TC hasn't closed on how they want to handle that one 15:55:52 and until they do, I don't expect that they'll define any other tags 15:56:08 bswartz: ok 15:56:13 how is it influences us? 15:56:21 I'm more interested in how they view maturity and integration between projects 15:56:37 we continue to be frozen out of other projects from an integration perspective 15:56:42 bswartz: the tag is supposed to tell us that? 15:57:02 so we have to do slightly awkward things like the devstack plugin vponomaryov added last week 15:57:05 bswartz: if manila gets integrated_release tag for Kilo, that will be good 15:57:28 xyang2: yes I think that one should be easy for us to obtain, but I'm afraid it won't have any value 15:57:51 it will merely be a statement of fact that our release is integrated with openstack, which has been true since icehouse 15:58:33 maturity obviously will require production deployments, and those are coming along well 15:58:55 the key is going to be getting the deployers to join the community and contribute back 15:59:14 okay we're almost out of time 15:59:22 any last minute things? 15:59:25 I have one... 15:59:47 http://lists.openstack.org/pipermail/openstack-dev/2015-February/057854.html 15:59:51 read that ML post if you haven't 15:59:53 thanks all! 16:00:05 #endmeeting