15:00:13 #startmeeting manila 15:00:14 Meeting started Thu Apr 16 15:00:13 2015 UTC and is due to finish in 60 minutes. The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:15 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:18 The meeting name has been set to 'manila' 15:00:21 Hi 15:00:25 hey 15:00:26 hi 15:00:29 Hi 15:00:31 hello all 15:00:31 hi 15:00:32 Hello 15:00:38 hi 15:00:44 hi 15:00:57 #agenda https://wiki.openstack.org/wiki/Manila/Meetings 15:01:09 hello! 15:01:14 just one announcement 15:01:15 hi 15:01:18 hi 15:01:27 for anyone who didn't know, we cut RC1 on Monday 15:01:43 everyone should be TESTING it and looking for bugs 15:02:01 we've got 2 more weeks until release so we have time to fix critical issues 15:02:29 so far nobody has tagged any bug with kilo-rc-potential 15:03:02 okay 15:03:07 #topic specs repo 15:03:20 we discussed this last week 15:03:38 the advantages, the disadvantages, the possible guidelines we'd need to adopt 15:03:54 hopefully everyone has formed an opinion about specs repos 15:05:08 I'm going to call a vote, and everyone is welcome to participate -- this is mostly to guage feedback, the vote won't be binding 15:05:56 bswartz: vote via mail-list? 15:05:57 #startvote Should the Manila team start using a specs repo to review new functional specs? Yes, No, Later 15:05:58 Begin voting on: Should the Manila team start using a specs repo to review new functional specs? Valid vote options are Yes, No, Later. 15:05:59 Vote using '#vote OPTION'. Only your last vote counts. 15:06:06 #vote no 15:06:15 #vote no 15:06:15 #vote Later 15:06:25 #vote Yes 15:06:26 #vote yes 15:06:28 #vote Later 15:06:32 #vote yes 15:06:35 Later 15:06:44 #vote Later 15:06:49 toabctl: wrong format 15:06:50 #vote Later 15:06:50 #vote Later 15:06:53 sorry :) 15:07:05 ... but I think we should only use it when appropriate of course 15:07:13 Vote 15:07:16 30 more seconds 15:07:17 Ye 15:07:25 toabctl, I thought you were apologizing for your opinion at first 15:07:36 #vote Later 15:07:48 #endvote 15:07:49 Voted on "Should the Manila team start using a specs repo to review new functional specs?" Results are 15:07:50 Yes (3): xyang2, markstur, chen12 15:07:51 Later (6): toabctl, u_glide, cknight, vponomaryov, csaba1, ganso_ 15:07:52 No (2): bswartz, lpabon 15:08:10 markstur: :) 15:08:20 okay, looks like we're pretty split with the "laters" winning 15:08:49 we should keep an open mind about specs repos and revisit this topic in a few months 15:09:01 bswartz: was it not the plan last time that you drop an email to initiate a discussion on possible schemes? 15:09:02 or weekly 15:09:14 for now I would encourage people to continue to use wikis to write functional specs, and to link the FS from your BPs on launchpad 15:09:41 csaba1: I did say I'd do an ML post, and that didn't happen :-( 15:09:58 csaba1: hence the later -- on my part at least 15:09:59 In most cases, the commit message and code covers enough anyway. It's just the bigger collab designs where this would even be helpful 15:10:49 functional specs are still very important -- I think the main concern is that we don't want to adopt a heavy process (we've seen it start to get out of control in other projects) 15:11:20 and also we don't want to give anyone the impression that the "specs" are the source of truth, because oftentimes the actual implementation diverges from the original specs after testing and code review feedback 15:11:42 so we still need to make sure to document what we actually did in the various user/admin-facing docs 15:12:02 bswartz: nevertheless it helps reviewers of some feature 15:12:40 for major new features we'll expect to see a functional spec linked from the blueprint, and we will review it as a team 15:12:51 some of those reviews will happen in vancouver 15:13:05 some will happen before then or after then too 15:13:23 okay let's move on 15:13:29 #topic Asking for approval for bp: versioned-objects 15:13:43 #link https://blueprints.launchpad.net/manila/+spec/versioned-objects 15:13:53 chen12: would you like to describe this? 15:13:54 It has already been mentioned in last virtual meet up. 15:14:16 and now Bps have been registered crossed openstack projects. 15:14:35 https://blueprints.launchpad.net/openstack/?searchtext=versioned-objects 15:14:51 I thinik manila should definitly do this 15:14:58 chen12: how familiar are you with the progress on this topic in the cinder/heat projects? 15:15:15 here is my plan: 15:15:24 I think it is done in cinder, but I didn't check the details 15:15:30 3 basic steps are needed for it: 1) import base object implement without version Like what have been done in heat: https://blueprints.launchpad.net/heat/+spec/versioned-objects And, I just upload the first object: ShareExportLocation https://review.openstack.org/#/c/173692/ 2) Implement object register method. See an example of registering objects with a decorator here: https://review.openstack.org/#/c/159577/5/nova/ 15:15:36 cinder proposed doing this back in paris, but I don't know if they completed the work 15:15:49 cider did it without oslo_versionedobject 15:16:08 we can follow heat at beginning 15:16:19 so oslo has created something that makes this easier for us? 15:16:28 this is useful when we are going to tag new API versions 15:16:34 bswartz: yes. 15:16:36 for the moment we did not do this 15:16:40 with lots of API changes 15:16:45 vponomaryov: no, not related to API 15:16:46 this sounds like a great idea 15:16:59 vponomaryov: +1 15:17:01 okay well I'm in favor of the high level idea -- anything that makes it easier to perform upgrades without downtime is a good thing 15:17:12 vponomaryov: just would help Rolling Upgrades 15:17:25 I'll have to look closer at the specfics 15:17:45 bswartz: +1. We should understand this well before making a decision. 15:17:46 one concern I have is: will this be enough? or is there other work that's also essential to support rolling upgrades? 15:17:59 vponomaryov: +1 15:18:11 I fixed a couple of cinder bugs due to changing to objects 15:18:15 bswartz: it will require changes for each model 15:18:19 bswartz: migration guide at the minimum 15:18:32 it is a moderate amount of work to start using objects 15:18:52 this sounds like a great discussion for the design summit :-) 15:19:02 bswartz: in simple words - it is new abstraction between DB and users of data. Now models will be combined with methods to operated data for them 15:19:04 as long as we don't adopt a "nova-conductor" style of database interaction, it will still be hard for us to alter our DB schema without taking the whole system down, I believe 15:19:06 at first they may break some stuff, specially drivers 15:19:24 and it's incredible good documented. see http://docs.openstack.org/developer/oslo.versionedobjects/usage.html 15:19:42 bswartz: +1 15:20:03 bswartz: conductor do not affect object I think. nova is the beginer of versionedobject 15:20:54 chen12: what I mean is, versioned objects help deal with different version of services in the RPC layer, but at the DB schema layer, I don't think they help 15:20:55 am I wrong? 15:21:24 bswartz: o~yes. you're right 15:21:31 would versioned objects make it possible to add a new column to the shares table without restarting all my services? 15:22:14 bswartz: yes, it would work 15:22:35 okay then I'm more excited about this 15:22:38 bswartz: after read data from db, object would transfer it to object first 15:22:40 I will take a look at the oslo docs 15:22:55 I'm going to approve the direction of this BP, and we can continue discussing the details 15:23:05 bswartz: great. 15:23:07 the right definition will be essential 15:23:31 bswartz: I will not attend the design summit, but someone would attend it in my behalf. 15:23:40 bswartz: Added to etherpad of potential summit topics 15:23:41 chen12: do you know who that will be? 15:23:43 bswartz: I will do more work on this then 15:24:03 let's get a name next to the proposal so we know who will be presenting in vancouver 15:24:10 bswartz: I suppose it would be ken (ken.chen@intel.com) 15:24:21 thanks 15:24:30 anything else on this before we move on? 15:25:02 #topic Horizon Plugin for Kilo 15:25:29 #link https://github.com/hp-storage/manila-ui 15:25:32 The horizon UI for manila has been extracted into a plugin, available at https://github.com/hp-storage/manila-ui 15:25:47 so thanks to the work of gary-smith_, our horizon integration has been moved from a fork to a standalone "plugin" repo 15:26:07 I plan to submit the change into openstack governance next week 15:26:18 Just have to get all of the unit tests working first :-) 15:26:25 garysmith_: excellent 15:26:36 garysmith_: it iwll be hosted under stackforge? 15:26:41 It will be similar to tuskar-ui, under the horizon umbrella, see http://governance.openstack.org/reference/projects/horizon.html 15:26:56 hopefully openstack/manila-ui 15:27:10 yeah, openstack/manila-ui 15:27:32 garysmith_: do you know how ACLs are handled for this stuff? 15:28:04 The model that tuskar-ui used was that the horizon cores would be cores on manila-ui 15:28:24 So I am presuming the same would be true here 15:28:26 garysmith_: fyi, the top three links on the github README.md do not work 15:28:27 so we'll have to rely on horizon team to +2 our UI changes then 15:29:10 garysmith_: do you think there will be any problem with immediately branching stable/kilo branch when the project is added? 15:29:13 lpabon: thanks. I'll fix it. That was from the cookie-cutter boilerplate for new projects 15:29:26 garysmith_: np :-) 15:29:43 bswartz: no. I already branched proposed/kilo, so stable/kilo can also be branched 15:29:44 so we can continue to maintain a kilo version of the plugin for packagers to use? 15:29:57 yes 15:30:03 okay 15:30:12 this is perfect from my perspective 15:30:30 I suppose per the latest message on the developer ML, I should create the stable/kilo branch now 15:30:43 yes -- no more proposed branches 15:30:49 I am not sure it would be enough to have only Horizon core-members be cores in this project 15:31:05 http://lists.openstack.org/pipermail/openstack-dev/2015-April/061657.html (if you haven't seen it) 15:31:07 garysmith_: would be good to have a bugtracker at launchpad for manila-ui soon 15:31:11 vponomaryov: I can add manila cores as well 15:31:24 garysmith_: do you plan to update https://wiki.openstack.org/wiki/Manila/docs/HOWTO_use_manila_with_horizon ? 15:31:34 vponomaryov: we now have cross-project dependencies, so if there are things that need to merge in lockstep, we can manage that 15:31:42 toabctl: my understanding is that it happens automagically with the governance submission, but I'll check on that 15:31:52 garysmith_: ok 15:31:53 csaba1: yes 15:32:01 cool 15:32:16 sorry folks - I have to leave now. bye 15:32:47 garysmith_: quick question... why not part of stackforge? is it simplicity? 15:32:51 garysmith_: so I'm keen to see the governance change happen asap 15:33:12 lpabon: openstack is better than stackforge, namespace-wise 15:33:24 they're functionally the same though 15:33:37 bswartz: no i mean the repo is currenlty in github 15:33:55 lpabon: it is in github now purely for simplicity 15:33:58 i was wondering if it could be in stackforge so that we can use Gerrit 15:34:06 garysmith_: yeah, ok ,that is what i thought 15:34:25 garysmith_: I get that you might need to fix some unit tests, but I'd like to allow packagers to start working with the plugin the sooner the better, we can fix tests after it's in a new repo too 15:34:34 as soon as we get it under the governance, it will be under the normal review.openstack.org gerrit processes 15:34:52 garysmith_: issue with Github is the CI ( i went through the same process moving a project called swiftonfile from github to stackforge) 15:35:03 garysmith_: awesome 15:35:34 bswartz: my understanding is that the unit tests need to be present and working before acceptance into governance 15:36:12 garysmith_: okay -- in that case please reach out for help if any of us can make that go faster 15:36:41 bswartz: sounds good 15:37:03 #topic Liberty design summit 15:37:11 #link https://etherpad.openstack.org/p/manila-liberty-proposed-sessions 15:37:31 so there's not many updates here 15:37:58 I think we have about 2 weeks until we need to make decisions 15:38:01 ctrl-shift-R if the etherpad update makes it unreadable 15:38:34 I know some people were too busy last week to focus on Liberty, but now we really should start capturing and writing down the proposals 15:39:09 markstur: are you using chrome? 15:39:17 lpabon, yes 15:39:48 I did see a note on the ML that we're getting three (3) fishbowl sessions and four (4) working sessions, and that looks pretty final 15:39:50 markstur: yeah, that happened to me a few days ago, weird thing is that i *fixed* it using ctrl-shift-R to reload the page cache 15:40:06 that's about what I asked for so I'm happy with that 15:40:32 yeah for those who don't know etherpad rolled out a software upgrade recently that breaks on some browsers 15:40:53 hooray for SaaS! 15:40:58 lol 15:42:23 On April 30 we will discuss the design summit proposals and schedule them 15:42:30 bswartz: next topic? 15:42:49 so have your proposals on the etherpad before then 15:42:57 #topic Limited share data API for drivers 15:43:03 #link https://wiki.openstack.org/wiki/Manila/Limited_share_data_API_for_drivers 15:43:11 u_glide: you have the floor! 15:43:25 :) 15:43:34 now I'm working on removal of direct DB API calls from drivers. And this feature is the next step. 15:44:31 hopefully you all remember that we talked about this in the midcycle 15:44:32 we have discussed this internally with vponomaryov 15:44:42 drivers shouldn't be accessing the database directly at all, but a few do 15:45:11 so we need to add functionality that allows drivers to do what they need to without resorting to direct DB access 15:45:49 there is a related concern that's come up recently 15:46:14 right now when you do create_share() you can return one model update when you're done (in your driver) 15:46:15 https://wiki.openstack.org/wiki/Manila/Limited_share_data_API_for_drivers#This_concept_vs_model_updates_in_Share_Manager 15:47:26 however frequently, drivers need to undertake multiple steps to actually create a share, and it would be ideal if they could make multiple model updates along the way, such that if there is a sudden failure at any point, the DB reflects reality well enough that we can delete the share or continue creating the share cleanly 15:48:12 In particular I'm thinking of the case when a driver has half-created a share, and the m-shr process is kill -9'd 15:48:25 u_glide, This is also a new way for drivers to store their own key/value pairs, right? 15:48:40 markstur: yes 15:48:42 in that case, then the m-shr process comes back, it should either finish what it was doing, or at least be able to clean up what it did 15:49:13 bswartz: I think this is another problem 15:49:15 what I'm describing is a bit of an enhancement to what's proposed 15:49:15 bswartz: In that scenario, would the share manager do the cleanup, or the drivers? 15:49:45 cknight: it would have to be the driver 15:50:08 the share manager does a scan of all the shares it owns at startup time though, so it could trigger driver actions for shares in intermediate states 15:50:10 sure driver, only driver knows how to release his resources 15:50:27 bswartz: ok, that makes sense. better than taskflow. 15:50:43 bswartz: Igor idea is about different thing 15:51:01 bswartz: it is about private key-value storage 15:51:01 vponomaryov, u_glide: sorry for sidetracking 15:51:14 yes let's get back to the proposal 15:52:49 u_glide's proposal allows driver to store key/values for each share 15:53:07 driver-makers, please share your opinions :) 15:53:15 bswartz: actually there are cases when not only for share will be some data 15:53:20 this could be really useful if the storage controller doesn't support 32 character share-names, for example 15:53:23 u_glide: does that mean this will make two db access instead of 1? 15:53:35 u_glide: it is better to reduce the # of db access 15:54:11 you could create a shorter share name (or even an admin-friendly share name) and store it in the manila DB through this interface, and then still reliably be able to find it 15:54:18 xyang2: no, it doesn't. Drivers will have access only to private storage interface 15:54:53 u_glide: I see two db update in the new code there 15:54:54 bswartz: shorter share and/or snapshot name, so we need to come to idea of key-value storage that will serve not only for shares 15:55:13 it's important to note that the stored values won't be visible to users, just like the user metadata should not be visible to the drivers 15:55:22 I agree with vponomaryov 15:55:33 vponomaryov: what do you have in mind? snapshots? share-servers? 15:55:38 vponomaryov: +1 15:55:47 bswartz: share servers already has its details 15:55:52 bswartz: snapshots 15:55:58 u_glide: self.db.share_export_locations_update and self.db.private_share_data_update(context, share_id, private_share_data) 15:56:09 so private driver data for snapshots too? 15:56:20 bswartz: yes 15:56:21 +1 15:56:26 xyang2: Did you read text? :) 15:56:30 bswartz: they have ID of 32 length too 15:56:46 bswartz: it is reffering to problem of short IDs for some backends 15:57:31 yeah some backends cannot store the full UUID of the share/snapshot, and truncating the UUID makes the value ambiguous 15:57:56 vponomaryov: So potentially you would need to store details for any model object that comes into the driver? There will be others in the future. 15:57:57 that's just one use case for the feature though -- presumably driver authors will think of other things to store 15:58:23 snapshot details could be stored with the share, but it might turn out better if we have private share data, snapshot data (and of course share server details) 15:58:37 we'll think of lots of things to keep in there :) 15:58:40 bswartz: I'd want to keep this as generic as possible, since we can't enumerate all the use cases now. 15:58:51 cknight: that is the point - to create such approach where we will be able to easily extend some new model with key-value storage 15:59:31 and it is great that we don this instead of each driver coming up with sneaky ways of stashing data 15:59:38 s/don/do 15:59:39 I like that idea, as long as it doesn't get too large and complicated (don't see why it would though) 15:59:53 vponomaryov: so I don't need the feature to distinguish between object types. each has a uuid, which could serve as the key, or part of the key. 15:59:53 markstur: yes! 16:00:08 bswartz, I thought it was your idea. Assumed you liked it. 16:01:16 yes, in most cases drivers will be simpler than now 16:01:49 markstur: just agreeing that sneakiness is a bad thing -- right now you have to choose between that or simply not supporting a feature 16:01:49 without any tricky templates and not necessarily API calls 16:01:57 okay we're past our time 16:02:25 I think we have general agreement on this topic, with a recommendation that it's extended to support things other than shares (possibly even future things not yet imagined) 16:02:31 thanks all 16:02:40 #endmeeting