15:00:13 <bswartz> #startmeeting manila
15:00:14 <openstack> Meeting started Thu Apr 16 15:00:13 2015 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:18 <openstack> The meeting name has been set to 'manila'
15:00:21 <cknight> Hi
15:00:25 <toabctl> hey
15:00:26 <lpabon> hi
15:00:29 <Zhongjun> Hi
15:00:31 <bswartz> hello all
15:00:31 <garysmith_> hi
15:00:32 <vponomaryov> Hello
15:00:38 <xyang2> hi
15:00:44 <markstur> hi
15:00:57 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings
15:01:09 <ganso_> hello!
15:01:14 <bswartz> just one announcement
15:01:15 <csaba1> hi
15:01:18 <rraja> hi
15:01:27 <bswartz> for anyone who didn't know, we cut RC1 on Monday
15:01:43 <bswartz> everyone should be TESTING it and looking for bugs
15:02:01 <bswartz> we've got 2 more weeks until release so we have time to fix critical issues
15:02:29 <bswartz> so far nobody has tagged any bug with kilo-rc-potential
15:03:02 <bswartz> okay
15:03:07 <bswartz> #topic specs repo
15:03:20 <bswartz> we discussed this last week
15:03:38 <bswartz> the advantages, the disadvantages, the possible guidelines we'd need to adopt
15:03:54 <bswartz> hopefully everyone has formed an opinion about specs repos
15:05:08 <bswartz> I'm going to call a vote, and everyone is welcome to participate -- this is mostly to guage feedback, the vote won't be binding
15:05:56 <vponomaryov> bswartz: vote via mail-list?
15:05:57 <bswartz> #startvote Should the Manila team start using a specs repo to review new functional specs? Yes, No, Later
15:05:58 <openstack> Begin voting on: Should the Manila team start using a specs repo to review new functional specs? Valid vote options are Yes, No, Later.
15:05:59 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
15:06:06 <bswartz> #vote no
15:06:15 <lpabon> #vote no
15:06:15 <cknight> #vote Later
15:06:25 <markstur> #vote Yes
15:06:26 <chen12> #vote yes
15:06:28 <u_glide> #vote Later
15:06:32 <xyang2> #vote yes
15:06:35 <toabctl> Later
15:06:44 <vponomaryov> #vote Later
15:06:49 <bswartz> toabctl: wrong format
15:06:50 <csaba1> #vote Later
15:06:50 <toabctl> #vote Later
15:06:53 <toabctl> sorry :)
15:07:05 <markstur> ... but I think we should only use it when appropriate of course
15:07:13 <Zhongjun> Vote
15:07:16 <bswartz> 30 more seconds
15:07:17 <Zhongjun> Ye
15:07:25 <markstur> toabctl, I thought you were apologizing for your opinion at first
15:07:36 <ganso_> #vote Later
15:07:48 <bswartz> #endvote
15:07:49 <openstack> Voted on "Should the Manila team start using a specs repo to review new functional specs?" Results are
15:07:50 <openstack> Yes (3): xyang2, markstur, chen12
15:07:51 <openstack> Later (6): toabctl, u_glide, cknight, vponomaryov, csaba1, ganso_
15:07:52 <openstack> No (2): bswartz, lpabon
15:08:10 <toabctl> markstur: :)
15:08:20 <bswartz> okay, looks like we're pretty split with the "laters" winning
15:08:49 <bswartz> we should keep an open mind about specs repos and revisit this topic in a few months
15:09:01 <csaba1> bswartz: was it not the plan last time that you drop an email to initiate a discussion on possible schemes?
15:09:02 <markstur> or weekly
15:09:14 <bswartz> for now I would encourage people to continue to use wikis to write functional specs, and to link the FS from your BPs on launchpad
15:09:41 <bswartz> csaba1: I did say I'd do an ML post, and that didn't happen :-(
15:09:58 <csaba1> csaba1: hence the later -- on my part at least
15:09:59 <markstur> In most cases, the commit message and code covers enough anyway.  It's just the bigger collab designs where this would even be helpful
15:10:49 <bswartz> functional specs are still very important -- I think the main concern is that we don't want to adopt a heavy process (we've seen it start to get out of control in other projects)
15:11:20 <bswartz> and also we don't want to give anyone the impression that the "specs" are the source of truth, because oftentimes the actual implementation diverges from the original specs after testing and code review feedback
15:11:42 <bswartz> so we still need to make sure to document what we actually did in the various user/admin-facing docs
15:12:02 <vponomaryov> bswartz: nevertheless it helps reviewers of some feature
15:12:40 <bswartz> for major new features we'll expect to see a functional spec linked from the blueprint, and we will review it as a team
15:12:51 <bswartz> some of those reviews will happen in vancouver
15:13:05 <bswartz> some will happen before then or after then too
15:13:23 <bswartz> okay let's move on
15:13:29 <bswartz> #topic Asking for approval for bp: versioned-objects
15:13:43 <bswartz> #link https://blueprints.launchpad.net/manila/+spec/versioned-objects
15:13:53 <bswartz> chen12: would you like to describe this?
15:13:54 <chen12> It has already been mentioned in last virtual meet up.
15:14:16 <chen12> and now Bps have been registered crossed openstack projects.
15:14:35 <chen12> https://blueprints.launchpad.net/openstack/?searchtext=versioned-objects
15:14:51 <chen12> I thinik manila should definitly do this
15:14:58 <bswartz> chen12: how familiar are you with the progress on this topic in the cinder/heat projects?
15:15:15 <chen12> here is my plan:
15:15:24 <markstur> I think it is done in cinder, but I didn't check the details
15:15:30 <chen12> 3 basic steps are needed for it: 1)	import base object implement without version Like what have been done in heat:  https://blueprints.launchpad.net/heat/+spec/versioned-objects And, I just upload the first object: ShareExportLocation https://review.openstack.org/#/c/173692/  2)	Implement object register method.  See an example of registering objects with a decorator here: https://review.openstack.org/#/c/159577/5/nova/
15:15:36 <bswartz> cinder proposed doing this back in paris, but I don't know if they completed the work
15:15:49 <chen12> cider did it without oslo_versionedobject
15:16:08 <chen12> we can follow heat at beginning
15:16:19 <bswartz> so oslo has created something that makes this easier for us?
15:16:28 <vponomaryov> this is useful when we are going to tag new API versions
15:16:34 <chen12> bswartz: yes.
15:16:36 <vponomaryov> for the moment we did not do this
15:16:40 <vponomaryov> with lots of API changes
15:16:45 <chen12> vponomaryov: no, not related to API
15:16:46 <lpabon> this sounds like a great idea
15:16:59 <u_glide> vponomaryov: +1
15:17:01 <bswartz> okay well I'm in favor of the high level idea -- anything that makes it easier to perform upgrades without downtime is a good thing
15:17:12 <chen12> vponomaryov: just would help Rolling Upgrades
15:17:25 <bswartz> I'll have to look closer at the specfics
15:17:45 <cknight> bswartz: +1.  We should understand this well before making a decision.
15:17:46 <bswartz> one concern I have is: will this be enough? or is there other work that's also essential to support rolling upgrades?
15:17:59 <ganso_> vponomaryov: +1
15:18:11 <ganso_> I fixed a couple of cinder bugs due to changing to objects
15:18:15 <vponomaryov> bswartz: it will require changes for each model
15:18:19 <u_glide> bswartz: migration guide at the minimum
15:18:32 <ganso_> it is a moderate amount of work to start using objects
15:18:52 <lpabon> this sounds like a great discussion for the design summit :-)
15:19:02 <vponomaryov> bswartz: in simple words - it is new abstraction between DB and users of data. Now models will be combined with methods to operated data for them
15:19:04 <bswartz> as long as we don't adopt a "nova-conductor" style of database interaction, it will still be hard for us to alter our DB schema without taking the whole system down, I believe
15:19:06 <ganso_> at first they may break some stuff, specially drivers
15:19:24 <toabctl> and it's incredible good documented. see http://docs.openstack.org/developer/oslo.versionedobjects/usage.html
15:19:42 <u_glide> bswartz: +1
15:20:03 <chen12> bswartz: conductor do not affect object I think. nova is the beginer of versionedobject
15:20:54 <bswartz> chen12: what I mean is, versioned objects help deal with different version of services in the RPC layer, but at the DB schema layer, I don't think they help
15:20:55 <bswartz> am I wrong?
15:21:24 <chen12> bswartz: o~yes. you're right
15:21:31 <bswartz> would versioned objects make it possible to add a new column to the shares table without restarting all my services?
15:22:14 <chen12> bswartz: yes, it would work
15:22:35 <bswartz> okay then I'm more excited about this
15:22:38 <chen12> bswartz: after read data from db, object would transfer it to object first
15:22:40 <bswartz> I will take a look at the oslo docs
15:22:55 <bswartz> I'm going to approve the direction of this BP, and we can continue discussing the details
15:23:05 <chen12> bswartz: great.
15:23:07 <bswartz> the right definition will be essential
15:23:31 <chen12> bswartz: I will not attend the design summit, but someone would attend it in my behalf.
15:23:40 <cknight> bswartz: Added to etherpad of potential summit topics
15:23:41 <bswartz> chen12: do you know who that will be?
15:23:43 <chen12> bswartz: I will do more work on this then
15:24:03 <bswartz> let's get a name next to the proposal so we know who will be presenting in vancouver
15:24:10 <chen12> bswartz: I suppose it would be ken （ken.chen@intel.com）
15:24:21 <bswartz> thanks
15:24:30 <bswartz> anything else on this before we move on?
15:25:02 <bswartz> #topic Horizon Plugin for Kilo
15:25:29 <vponomaryov> #link https://github.com/hp-storage/manila-ui
15:25:32 <garysmith_> The horizon UI for manila has been extracted into a plugin, available at https://github.com/hp-storage/manila-ui
15:25:47 <bswartz> so thanks to the work of gary-smith_, our horizon integration has been moved from a fork to a standalone "plugin" repo
15:26:07 <garysmith_> I plan to submit the change into openstack governance next week
15:26:18 <garysmith_> Just have to get all of the unit tests working first :-)
15:26:25 <bswartz> garysmith_: excellent
15:26:36 <vponomaryov> garysmith_: it iwll be hosted under stackforge?
15:26:41 <garysmith_> It will be similar to tuskar-ui, under the horizon umbrella, see http://governance.openstack.org/reference/projects/horizon.html
15:26:56 <bswartz> hopefully openstack/manila-ui
15:27:10 <garysmith_> yeah, openstack/manila-ui
15:27:32 <bswartz> garysmith_: do you know how ACLs are handled for this stuff?
15:28:04 <garysmith_> The model that tuskar-ui used was that the horizon cores would be cores on manila-ui
15:28:24 <garysmith_> So I am presuming the same would be true here
15:28:26 <lpabon> garysmith_: fyi, the top three links on the github README.md do not work
15:28:27 <bswartz> so we'll have to rely on horizon team to +2 our UI changes then
15:29:10 <bswartz> garysmith_: do you think there will be any problem with immediately branching stable/kilo branch when the project is added?
15:29:13 <garysmith_> lpabon: thanks. I'll fix it. That was from the cookie-cutter boilerplate for new projects
15:29:26 <lpabon> garysmith_: np :-)
15:29:43 <garysmith_> bswartz: no. I already branched proposed/kilo, so stable/kilo can also be branched
15:29:44 <bswartz> so we can continue to maintain a kilo version of the plugin for packagers to use?
15:29:57 <garysmith_> yes
15:30:03 <bswartz> okay
15:30:12 <bswartz> this is perfect from my perspective
15:30:30 <garysmith_> I suppose per the latest message on the developer ML, I should create the stable/kilo branch now
15:30:43 <bswartz> yes -- no more proposed branches
15:30:49 <vponomaryov> I am not sure it would be enough to have only Horizon core-members  be cores in this project
15:31:05 <bswartz> http://lists.openstack.org/pipermail/openstack-dev/2015-April/061657.html (if you haven't seen it)
15:31:07 <toabctl> garysmith_: would be good to have a bugtracker at launchpad for manila-ui soon
15:31:11 <garysmith_> vponomaryov: I can add manila cores as well
15:31:24 <csaba1> garysmith_: do you plan to update https://wiki.openstack.org/wiki/Manila/docs/HOWTO_use_manila_with_horizon ?
15:31:34 <bswartz> vponomaryov: we now have cross-project dependencies, so if there are things that need to merge in lockstep, we can manage that
15:31:42 <garysmith_> toabctl: my understanding is that it happens automagically with the governance submission, but I'll check on that
15:31:52 <toabctl> garysmith_: ok
15:31:53 <garysmith_> csaba1: yes
15:32:01 <csaba1> cool
15:32:16 <toabctl> sorry folks - I have to leave now. bye
15:32:47 <lpabon> garysmith_: quick question... why not part of stackforge?  is it simplicity?
15:32:51 <bswartz> garysmith_: so I'm keen to see the governance change happen asap
15:33:12 <bswartz> lpabon: openstack is better than stackforge, namespace-wise
15:33:24 <bswartz> they're functionally the same though
15:33:37 <lpabon> bswartz: no i mean the repo is currenlty in github
15:33:55 <garysmith_> lpabon: it is in github now purely for simplicity
15:33:58 <lpabon> i was wondering if it could be in stackforge so that we can use  Gerrit
15:34:06 <lpabon> garysmith_: yeah, ok ,that is what i thought
15:34:25 <bswartz> garysmith_: I get that you might need to fix some unit tests, but I'd like to allow packagers to start working with the plugin the sooner the better, we can fix tests after it's in a new repo too
15:34:34 <garysmith_> as soon as we get it under the governance, it will be under the normal review.openstack.org gerrit processes
15:34:52 <lpabon> garysmith_: issue with Github is the CI ( i went through the same process moving a project called swiftonfile from github to stackforge)
15:35:03 <lpabon> garysmith_: awesome
15:35:34 <garysmith_> bswartz: my understanding is that the unit tests need to be present and working before acceptance into governance
15:36:12 <bswartz> garysmith_: okay -- in that case please reach out for help if any of us can make that go faster
15:36:41 <garysmith_> bswartz: sounds good
15:37:03 <bswartz> #topic Liberty design summit
15:37:11 <bswartz> #link https://etherpad.openstack.org/p/manila-liberty-proposed-sessions
15:37:31 <bswartz> so there's not many updates here
15:37:58 <bswartz> I think we have about 2 weeks until we need to make decisions
15:38:01 <markstur> ctrl-shift-R if the etherpad update makes it unreadable
15:38:34 <bswartz> I know some people were too busy last week to focus on Liberty, but now we really should start capturing and writing down the proposals
15:39:09 <lpabon> markstur: are you using chrome?
15:39:17 <markstur> lpabon, yes
15:39:48 <bswartz> I did see a note on the ML that we're getting three (3) fishbowl sessions and four (4) working sessions, and that looks pretty final
15:39:50 <lpabon> markstur: yeah, that happened to me a few days ago, weird thing is that i *fixed* it using ctrl-shift-R to reload the page cache
15:40:06 <bswartz> that's about what I asked for so I'm happy with that
15:40:32 <bswartz> yeah for those who don't know etherpad rolled out a software upgrade recently that breaks on some browsers
15:40:53 <bswartz> hooray for SaaS!
15:40:58 <lpabon> lol
15:42:23 <bswartz> On April 30 we will discuss the design summit proposals and schedule them
15:42:30 <u_glide> bswartz: next topic?
15:42:49 <bswartz> so have your proposals on the etherpad before then
15:42:57 <bswartz> #topic Limited share data API for drivers
15:43:03 <bswartz> #link https://wiki.openstack.org/wiki/Manila/Limited_share_data_API_for_drivers
15:43:11 <bswartz> u_glide: you have the floor!
15:43:25 <u_glide> :)
15:43:34 <u_glide> now I'm working on removal of direct DB API calls from drivers. And this feature is the next step.
15:44:31 <bswartz> hopefully you all remember that we talked about this in the midcycle
15:44:32 <u_glide> we have discussed this internally with  vponomaryov
15:44:42 <bswartz> drivers shouldn't be accessing the database directly at all, but a few do
15:45:11 <bswartz> so we need to add functionality that allows drivers to do what they need to without resorting to direct DB access
15:45:49 <bswartz> there is a related concern that's come up recently
15:46:14 <bswartz> right now when you do create_share() you can return one model update when you're done (in your driver)
15:46:15 <u_glide> https://wiki.openstack.org/wiki/Manila/Limited_share_data_API_for_drivers#This_concept_vs_model_updates_in_Share_Manager
15:47:26 <bswartz> however frequently, drivers need to undertake multiple steps to actually create a share, and it would be ideal if they could make multiple model updates along the way, such that if there is a sudden failure at any point, the DB reflects reality well enough that we can delete the share or continue creating the share cleanly
15:48:12 <bswartz> In particular I'm thinking of the case when a driver has half-created a share, and the m-shr process is kill -9'd
15:48:25 <markstur> u_glide, This is also a new way for drivers to store their own key/value pairs, right?
15:48:40 <u_glide> markstur: yes
15:48:42 <bswartz> in that case, then the m-shr process comes back, it should either finish what it was doing, or at least be able to clean up what it did
15:49:13 <u_glide> bswartz: I think this is another problem
15:49:15 <bswartz> what I'm describing is a bit of an enhancement to what's proposed
15:49:15 <cknight> bswartz: In that scenario, would the share manager do the cleanup, or the drivers?
15:49:45 <bswartz> cknight: it would have to be the driver
15:50:08 <bswartz> the share manager does a scan of all the shares it owns at startup time though, so it could trigger driver actions for shares in intermediate states
15:50:10 <vponomaryov> sure driver, only driver knows how to release his resources
15:50:27 <cknight> bswartz: ok, that makes sense.  better than taskflow.
15:50:43 <vponomaryov> bswartz: Igor idea is about different thing
15:51:01 <vponomaryov> bswartz: it is about private key-value storage
15:51:01 <bswartz> vponomaryov, u_glide: sorry for sidetracking
15:51:14 <bswartz> yes let's get back to the proposal
15:52:49 <bswartz> u_glide's proposal allows driver to store key/values for each share
15:53:07 <u_glide> driver-makers, please share your opinions :)
15:53:15 <vponomaryov> bswartz: actually there are cases when not only for share will be some data
15:53:20 <bswartz> this could be really useful if the storage controller doesn't support 32 character share-names, for example
15:53:23 <xyang2> u_glide: does that mean this will make two db access instead of 1?
15:53:35 <xyang2> u_glide: it is better to reduce the # of db access
15:54:11 <bswartz> you could create a shorter share name (or even an admin-friendly share name) and store it in the manila DB through this interface, and then still reliably be able to find it
15:54:18 <u_glide> xyang2: no, it doesn't. Drivers will have access only to private storage interface
15:54:53 <xyang2> u_glide: I see two db update in the new code there
15:54:54 <vponomaryov> bswartz: shorter share and/or snapshot name, so we need to come to idea of key-value storage that will serve not only for shares
15:55:13 <bswartz> it's important to note that the stored values won't be visible to users, just like the user metadata should not be visible to the drivers
15:55:22 <u_glide> I agree with  vponomaryov
15:55:33 <bswartz> vponomaryov: what do you have in mind? snapshots? share-servers?
15:55:38 <cknight> vponomaryov: +1
15:55:47 <vponomaryov> bswartz: share servers already has its details
15:55:52 <vponomaryov> bswartz: snapshots
15:55:58 <xyang2> u_glide: self.db.share_export_locations_update and self.db.private_share_data_update(context, share_id, private_share_data)
15:56:09 <bswartz> so private driver data for snapshots too?
15:56:20 <vponomaryov> bswartz: yes
15:56:21 <bswartz> +1
15:56:26 <u_glide> xyang2: Did you read text? :)
15:56:30 <vponomaryov> bswartz: they have ID of 32 length too
15:56:46 <vponomaryov> bswartz: it is reffering to problem of short IDs for some backends
15:57:31 <bswartz> yeah some backends cannot store the full UUID of the share/snapshot, and truncating the UUID makes the value ambiguous
15:57:56 <cknight> vponomaryov: So potentially you would need to store details for any model object that comes into the driver?  There will be others in the future.
15:57:57 <bswartz> that's just one use case for the feature though -- presumably driver authors will think of other things to store
15:58:23 <markstur> snapshot details could be stored with the share, but it might turn out better if we have private share data, snapshot data (and of course share server details)
15:58:37 <markstur> we'll think of lots of things to keep in there :)
15:58:40 <cknight> bswartz: I'd want to keep this as generic as possible, since we can't enumerate all the use cases now.
15:58:51 <vponomaryov> cknight: that is the point - to create such approach where we will be able to easily extend some new model with key-value storage
15:59:31 <markstur> and it is great that we don this instead of each driver coming up with sneaky ways of stashing data
15:59:38 <markstur> s/don/do
15:59:39 <bswartz> I like that idea, as long as it doesn't get too large and complicated (don't see why it would though)
15:59:53 <cknight> vponomaryov: so I don't need the feature to distinguish between object types.  each has a uuid, which could serve as the key, or part of the key.
15:59:53 <bswartz> markstur: yes!
16:00:08 <markstur> bswartz, I thought it was your idea.  Assumed you liked it.
16:01:16 <u_glide> yes, in most cases drivers will be simpler than now
16:01:49 <bswartz> markstur: just agreeing that sneakiness is a bad thing -- right now you have to choose between that or simply not supporting a feature
16:01:49 <u_glide> without any tricky templates and not necessarily API calls
16:01:57 <bswartz> okay we're past our time
16:02:25 <bswartz> I think we have general agreement on this topic, with a recommendation that it's extended to support things other than shares (possibly even future things not yet imagined)
16:02:31 <bswartz> thanks all
16:02:40 <bswartz> #endmeeting