15:00:12 <bswartz> #startmeeting manila
15:00:13 <openstack> Meeting started Thu Aug  6 15:00:12 2015 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:17 <openstack> The meeting name has been set to 'manila'
15:00:24 <cknight> Hi
15:00:25 <bswartz> hello all
15:00:32 <dustins> o/
15:00:32 <chen12> hi
15:00:33 <ganso_> hello
15:00:39 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings
15:00:39 <vponomaryov> hi
15:00:45 <u_glide> hi
15:01:18 <bswartz> hey guys I'm chairing this meeting from the HP office in ft collins
15:01:33 <bswartz> I'm at cinder midcycle meetup and it hasn't started up yet today
15:02:06 <bswartz> hopefully I won't get pulled into anything else during this meeting
15:02:07 <markstur> Welcome to HP, Ben!
15:02:11 <bswartz> lol
15:02:23 <cknight> markstur:  You can't have him.
15:02:30 <bswartz> #topic Common capabilities, next steps
15:02:36 <markstur> cknight, :)
15:03:01 <bswartz> so we agreed during the midcycle to implement some common capabilities
15:03:18 <bswartz> the first ones will be dedupe, and thin/thick provisioning
15:03:52 <bswartz> #link https://review.openstack.org/#/c/209781
15:04:01 <bswartz> markstur: is this your topic?
15:04:06 <markstur> Yeah
15:04:15 <bswartz> go ahead then
15:04:19 <markstur> I ran out of time and only did a very rough draft in the patch
15:04:35 <markstur> but I think we move forward
15:04:54 <markstur> The main question is do we agree on these common ones
15:05:03 <bswartz> okay so you're looking for reviews of the doc
15:05:09 <toabctl> hi
15:05:15 <markstur> oversubscription already has thin_provisioning_support and thick...
15:05:15 <lpabon> o/
15:05:35 <markstur> In the mid-cycle we saw some cinder proposals
15:05:37 <bswartz> this is an important doc because it's where we will point driver developers to the official list of "manila approved" extra specs
15:05:55 <markstur> Yes.  And it'll definitely need to be improved.
15:06:13 <markstur> We'll also have an Admin version of it for the people that actually use the extra-specs
15:06:31 <bswartz> so aside from the doc review
15:06:33 <markstur> Since I did it last night, I started with just a dev draft
15:06:39 <markstur> yes
15:06:47 <markstur> So with dedup...
15:06:53 <bswartz> does anyone disagree with these 2 features for common extra specs?
15:06:55 <markstur> I think we agree that several vendors will do it
15:07:03 <bswartz> s/dedup/dedupe/
15:07:31 <markstur> No disagrees?
15:07:40 <cknight> bswartz: +1.  'dedup' always bothered me.
15:07:52 <bswartz> yeah I'll put that in the code review too
15:07:55 <markstur> There is kind of a 3rd with "thick" a.k.a. "full"
15:08:01 <bswartz> dedupe is the correct spelling
15:08:13 <bswartz> markstur: what?
15:08:17 <markstur> I can agree with "dedup"
15:08:31 <markstur> xyang added thick_provisioning_support
15:08:40 <bswartz> thick means thick
15:08:52 <markstur> which is just a boolean so you could specifically ask for fully provisioned shares (not thin)
15:09:06 <Zhongjun> i agree with dedup
15:09:14 <markstur> And I'm find with "thick" I think HP prefers "full", but we often say "thick" as opposite of "thin"
15:09:35 <bswartz> oh I see, you just want a different term not a different meaning
15:09:35 <markstur> Oops. I meant to say I can agree with "dedupe" I just can't type it.
15:09:46 <ekarlso> o/
15:10:01 <ekarlso> lol, wrong meeting :D
15:10:19 <bswartz> okay looks like no dissent
15:10:30 <markstur> We also should decide if we want to prefix the common ones with something like "manila_"
15:10:34 <bswartz> #agreed thick/thin and dedupe will be first common capabilities
15:10:43 <lpabon> just to reiterate: bswartz you are asking for the extra specs flag for the scheduler: think and dedupe being two of them?
15:11:03 <bswartz> markstur: yeah the manila_ prefix is worth disucssing
15:11:33 <markstur> generally extra-specs are scoped with vendorprefix and colon
15:11:33 <bswartz> lpabon: we're talking about defining some standards for things that should mean the same thing across all drivers
15:11:41 <markstur> The scheduler doesn't see those.
15:11:47 <vponomaryov> why do we use two extra specs 'thick' and 'thin' when it they are sides of one coin?
15:11:50 <markstur> The convention is to change the : to _ for capabilities
15:11:52 <lpabon> bswartz: great
15:12:23 <bswartz> vponomarayov: the issue is that there are really 3 possibilities
15:12:27 <lpabon> vponomaryov: good point.. it's either thin or not-thin, right?
15:12:34 <markstur> But a prefix is not necessary for un-scoped it is a matter of us deciding.
15:12:39 <bswartz> backends can support "either" and you need a way to communicate that
15:12:42 <vponomaryov> bswartz: which is third?
15:12:58 <bswartz> and admins can request "don't care" and you need a way to specify that
15:13:11 <vponomaryov> bswartz: 'who'and 'when' decides what to choose?
15:13:18 <markstur> 3rd is that the backend will do either and some extra-spec (or default) tells it what to do
15:13:23 <bswartz> we can hash out the details in the doc review
15:13:24 <lpabon> can admins set a default?
15:13:34 <markstur> Like netapp:thin=True
15:13:35 <bswartz> the defaults will be hardcoded
15:13:51 <bswartz> the admin can set what he wants in his share type if he doesn't like the default
15:13:55 <vponomaryov> anyway, it is still two values to provide if set extra spec, and absense of one extra spec- is third value
15:13:59 <markstur> we also discussed that we  could reject both being true if we decide
15:14:23 <bswartz> yeah there's some subtleties
15:14:36 <bswartz> I have a specific recommendation but I haven't written it down yet
15:14:42 <Zhongjun> how about compress？
15:14:42 <bswartz> so I will do that in the doc review
15:14:51 <bswartz> Zhongjun: yes but later
15:15:20 <markstur> Should we decide about prefix standards today?  Or try to?
15:15:30 <bswartz> yes I'd like to talk about prefixing with manila_
15:15:55 <bswartz> it's pointed out that we already have a de-facto common capability: driver_handles_share_servers
15:16:02 <bswartz> which doesn't have the prefix
15:16:03 <vponomaryov> why do we need it for Manila extra specs?
15:16:20 <vponomaryov> it should define "type" of extra spec
15:16:27 <vponomaryov> capability_foo
15:16:36 <bswartz> vponomaryov: the idea is to communicate to admin which ones are defined by the core team vs defined by vendors
15:17:03 <vponomaryov> bswartz: then use prefixes for vendor-specific things
15:17:23 <bswartz> okay so the other choice is: no prefix means it's defined by the core team
15:17:24 <u_glide> vponomaryov: +1
15:17:24 <markstur> For example, zhongjun could add "compression" today without a concensus -- just at review.  But if we made it "common" and agreed upon we'd either use manila_compression or just add it to an official list as-is
15:17:53 <bswartz> that means that any capability/extra spec without a vendor specific prefix needs to be vetted with the community and agreed to
15:18:03 <bswartz> how does that sound?
15:18:10 <chen12> no prefix +1
15:18:16 <vponomaryov> agreement +1
15:18:44 <lpabon> either way is fine, as long as we are all in agreement one way or the other and it is documented, I think it will be working fine
15:19:44 <markstur> I'm ok with no prefix
15:19:57 <Zhongjun> i agree with no prefix
15:20:06 <bswartz> I'm leaning towards no prefix due to existing capability with no prefix
15:20:12 <cknight> bswartz:  +1
15:20:16 <dustins> bswartz: +1
15:20:18 <ganso_> no prefix +1
15:20:19 <bswartz> I'd rather not have that confusion or have to rename them
15:20:20 <markstur> I would suggest that we might consider rejecting un-sanctioned capabilities without vendor prefix then
15:20:28 <bswartz> okay
15:20:39 <bswartz> #agreed manila-defined capabilities have NO prefix
15:20:41 <markstur> Initially we can do that by review
15:20:53 <bswartz> anything else on this topic markstur?
15:21:03 <markstur> The other thing would be
15:21:08 <bswartz> I'll explain my thinking for the 3 options on thick/thin in the code review
15:21:18 <markstur> thin_provisioning_support
15:21:24 <markstur> I think it is fine, but
15:21:40 <bswartz> markstur: I think we can manage with 2 booleans
15:21:40 <markstur> Do we want to use something like _support
15:21:50 <markstur> e.g.  dedupe_support
15:21:51 <bswartz> no I think that makes it more complex
15:22:00 <markstur> agreed for dedup
15:22:03 <markstur> dedupe!
15:22:20 <markstur> but will we change the existing 2 (they are pretty new)
15:22:43 <bswartz> yeah any extra spec that haven't shipped can simply be changed to match the standard
15:22:45 <vponomaryov> markstur: we are at the moment when it is not late to change
15:22:56 <bswartz> vendors with legacy capabilities can support both
15:23:10 <bswartz> if there are any
15:23:14 <markstur> that reminds me
15:23:21 <markstur> provisioned_capacity_gb
15:23:32 <markstur> is not exactly a "capability" but
15:23:46 <bswartz> yeah it's a share stat...
15:23:46 <markstur> also an agreed upon think that goes with thin and oversubscription
15:23:52 <markstur> and also it is merged
15:24:10 <bswartz> the dev doc should explain how to implemen that
15:24:31 <markstur> OK so to summarize:
15:24:39 <markstur> 1. dedupe
15:24:55 <markstur> 2. thin_provisioning (the _support will be dropped)
15:25:25 <markstur> 3. provisioned_capacity_gb (stat goes with 2)
15:25:25 <xyang> Provisioned_capability_gb is similar to total_capacity_gb
15:25:36 <xyang> Any concern with that
15:25:41 <markstur> 4. ? not sure we agreed on thick_provisioning
15:26:01 <bswartz> yes but detail should be hashed out in code review
15:26:30 <bswartz> we need thick/thin/both capabilities and thick/thin/don't care extra specs
15:26:33 <bswartz> I'll write something
15:26:55 <bswartz> let's move on to enxt topic
15:27:23 <bswartz> #topic New repository devstack-plugin-hdfs for HDFS CI
15:27:33 <bswartz> chen12: you're up
15:27:40 <chen12> I'm working on HDFS CI recently
15:27:45 <bswartz> #link https://review.openstack.org/#/c/207050/
15:27:59 <bswartz> #link  https://review.openstack.org/#/c/209728/
15:28:05 <chen12> I'm asking 1. need review for patches in infra
15:28:13 <chen12> 2. do I doing all this correct ?
15:28:46 <bswartz> chen12: did you follow the model that software-based cinder drivers followed for CI?
15:28:47 <vponomaryov> chen12: why do you need separate repo?
15:29:03 <bswartz> I haven't actually looked at the details behind the ceph/gluster cinder CI systems
15:29:06 <vponomaryov> chen12: why update of https://github.com/openstack/manila/tree/master/contrib/ci is not enough?
15:29:13 <chen12> yes, I follwed cinder-drbd
15:29:17 <bswartz> I can check though and make code reviews based on that
15:30:05 <chen12> vponomaryov, I need a place to hold devstack-plugin-hdfs =>  followed https://review.openstack.org/171754
15:30:27 <bswartz> chen12: this seems reasonable to me
15:30:45 <bswartz> you might want to check if gluster/ceph have a better way though
15:30:57 <bswartz> cinder has several software-only drivers with CI
15:31:10 <bswartz> you should look at several and pick the easiest one to follow
15:31:25 <chen12> gluster/sheepdog has devstack-plugin too
15:31:32 <vponomaryov> chen12: installation of deps and configuration are different, and the latter can be stored in Manila repo
15:31:57 <bswartz> vponomaryov: does it really matter?
15:32:22 <chen12> vponomaryov, ? sorry, can you explain more ?  I need to build a running hdfs cluster => devstack-plugin
15:32:22 <bswartz> vponomaryov: if it's in a separate repo then Intel can have exclusive access to merge changes to it and we don't have to review
15:32:37 <bswartz> if it's in manila then we have to review changes
15:33:03 <vponomaryov> chen12: current devstack plugin allows to set up any configuration
15:33:10 <vponomaryov> chen12: of any driver
15:33:21 <vponomaryov> bswartz: I am asking for clarification
15:33:28 <bswartz> vponomaryov: the issue is that the driver needs to install more packages
15:33:46 <bswartz> not only does the HDFS driver need to be setup in manila, but you have to do a bunch of HDFS configuration too (presumably)
15:34:27 <chen12> vponomaryov, o... still confusing... I just did as examples I get. any other examples I can check
15:34:52 <bswartz> chen12: I think you're on the right track
15:35:12 <bswartz> chen12: keep asking for help if you need it and lmk if you're not getting any
15:35:24 <bswartz> I'll be more helpful when I'm not travelling
15:35:45 <bswartz> I'll review those 2 changes too
15:35:49 <chen12> bswartz, o. I think next thing I need to do is, write my devstack-plugin and make all test pass, right ? Then I can ask to add my job into manila ci ?
15:36:12 <chen12> bswartz, sure.
15:36:13 <bswartz> chen12: once you have the job working you can add it even if all tests are not passing
15:36:30 <bswartz> the deadline to have CI working (with failures) is L-3
15:36:43 <bswartz> the deadline to have them all passing is RC1
15:36:55 <chen12> bswartz, understand. I will do it ASAP.
15:37:01 <bswartz> so focus on getting the jobs to run and worry about failures later
15:37:08 <bswartz> chen12: thanks!
15:37:10 <markstur> chen12, great to see progress on an upstream CI for HDFS!
15:37:21 <bswartz> #topic Share Migration Dev Status Update
15:37:37 <bswartz> ganso_: you're up
15:37:50 <ganso_> Share Migration main patch it out of WIP
15:38:06 <ganso_> /s/it/is
15:38:14 <ganso_> #link https://review.openstack.org/179790/ https://review.openstack.org/179791/ https://review.openstack.org/179803/
15:38:26 <ganso_> I also rebased generic driver patch
15:38:35 <ganso_> it can easily be tested now
15:38:46 <bswartz> awesome
15:39:03 <lpabon> ganso_: i will help review
15:39:10 <ganso_> Generic driver is still WIP as I need CIFS, prepare_share (read only) and remaining unit tests implementation
15:39:44 <lpabon> ganso_: quick question, does the code have 'cancel' capability? or will it be added later?
15:39:52 <ganso_> I also did some very small changes to manila client patch as suggested by bswartz
15:40:00 <ganso_> lpabon: it will be added later
15:40:11 <ganso_> lpabon: I'm not sure if added in Liberty
15:40:12 <bswartz> ganso_: on the server side I think we'll want to use microversioning for the API
15:40:27 <lpabon> ganso_: no problem, thanks.
15:40:27 <ganso_> lpabon: depends on remaning changes until deadlines
15:40:32 <bswartz> what order those changes merge in shouldn't matter as long as one is based on the other
15:40:52 <ganso_> ganso_: is the microversioning patch merged?
15:40:54 <cknight> bswartz: extension API or microversioned core API?
15:41:04 <ganso_> bswartz: is the microversioning patch merged?
15:41:06 <cknight> ganso_: No, but it's progressing well.
15:41:16 <cknight> ganso_: Should have a patch up this week.
15:41:23 <bswartz> I'd like to get @experimental or something like that working alongside microversions
15:41:39 <ganso_> cknight: ok, then I can make this change as soon as it is up, this would be a minor change, I am worried about more significant changes
15:41:51 <bswartz> then we could use that to communicate that the migration feature is still evolving
15:42:30 <ganso_> that's all
15:42:39 <bswartz> we know that migration needs more effort to work universally
15:43:08 <bswartz> so tagging it experimental is a great way to allow it to move forward and get it in users hands without locking us into a design
15:44:20 <bswartz> okay
15:44:25 <ganso_> also, it is very important to have u_glide 's share instances patch reviewed and merged
15:44:33 <bswartz> #topic  Tempest Idempotent UUID Integrations
15:44:34 <ganso_> it directly impacts Share Migration
15:44:42 <ganso_> and it is not getting that many reviews
15:44:43 <bswartz> ganso_: yes I'm aware
15:44:53 <bswartz> dustins: you're up
15:44:58 <dustins> bswartz: Thanks!
15:45:03 <dustins> #link http://docs.openstack.org/developer/tempest/HACKING.html#test-identification-with-idempotent-id
15:45:31 <dustins> So, upstream Tempest has added UUIDs to each of the tests through an Idempotent ID decorator
15:45:35 <bswartz> dustins: is this a new tempest feature?
15:45:42 <bswartz> I haven't heard of this
15:45:45 <dustins> bswartz: It is indeed
15:45:52 <dustins> I hadn't either until this morning :)
15:46:03 <bswartz> vponomaryov, mkoderer: have you seen it?
15:46:22 <vponomaryov> dustins: does it allow to "inherit" test suites?
15:46:48 <vponomaryov> bswartz: heard about such plans in paris
15:47:00 <dustins> vponomaryov: I don't think it's so much "inheriting" test suites as it is tracking test functionality through refactoring
15:47:14 <bswartz> so do all tests need this or just some?
15:47:19 <vponomaryov> dustins: no, I asking about capability
15:47:21 <bswartz> uuids in the code seems ugly to me
15:47:26 <dustins> So that the what the test is testing is tracked even as the implementation thereof might change
15:47:45 <dustins> bswartz: There's a tool in upstream tempest that will add UUIDs to tests that don't have it
15:48:04 <vponomaryov> dustins: but can it change it with inheritance?
15:48:21 <dustins> vponomaryov: Good question, I'd have to investigate
15:48:57 * dustins makes a note to explore that today
15:49:02 <bswartz> dustins: what's the downside to not implementing this change?
15:49:24 <vponomaryov> bswartz: change of test names and problems of tracking history
15:49:37 <dustins> You won't be able to get through OpenStack's gate when Manila is promoted, since it checks to see if the tests have UUIDs
15:49:39 <bswartz> vponomaryov: does anyone actually look at that?
15:49:46 <vponomaryov> bswartz: I do not =)
15:50:11 <vponomaryov> dustins: what do you mean?
15:50:15 <bswartz> dustins: promoted?
15:50:37 <bswartz> we control our own check/gate jobs -- we don't require this today
15:50:52 <bswartz> I'm trying to figure out what is lost if we make the choice to ignore this feature
15:50:59 <vponomaryov> dustins: we are not going to merge our tempest tests to tempest repo
15:51:15 <dustins> vponomaryov: that I didn't know
15:51:15 <vponomaryov> dustins: we will keep it in Manila repo
15:51:34 <bswartz> yeah even the other tests in tempest will gradually move out of tempest into projects (like cinder)
15:51:45 <bswartz> tempest will only have cross-project tests
15:51:47 <vponomaryov> dustins: ok, then does it still makes sense then?
15:51:59 <dustins> But those tests that come back into the projects will still have the IDs, correct?
15:52:19 <bswartz> dustins: that's a openstack-qa question
15:52:32 <dustins> vponomaryov: I still think it does, as it allows us to track tests independent of the implementation of them
15:52:35 <bswartz> manila has never had tests in tempest and we have no plans to add any
15:52:39 <dustins> bswartz: I'll be sure to ask them
15:52:50 <markstur> Does this make it easier to find the part of a log where a specific test failed?
15:53:00 <vponomaryov> markstur: no
15:53:14 <vponomaryov> markstur: only keeping history
15:53:20 <bswartz> oh god tempest logging could stand to be improved greatly
15:53:30 <bswartz> I can never read those logs
15:53:39 <dustins> bswartz: It's on my "to-do" list :)
15:53:49 <bswartz> dustins: awesome!
15:54:16 <bswartz> okay it sounds like this is something that we could choose to borrow from tempest but there's no strong motivation to do so
15:54:34 <bswartz> if anyone wants to start doing historical tracking of test coverage then this would be valuable
15:54:48 <bswartz> but the obvious downside is additional clutter in test code
15:55:03 <dustins> bswartz: I could put up a patch adding it...
15:55:10 <dustins> And see what everyone thinks of it
15:55:24 <bswartz> dustins: you'd need to explain the motivation for doing so
15:55:26 <dustins> But yes, it would add an additional line per test for storing the UUID
15:55:52 <bswartz> right now I'm not inclined to merge something like that
15:55:53 <dustins> bswartz: indeed, and "test case history" might not be a strong one
15:56:12 <bswartz> okay just a few minutes left
15:56:18 <dustins> My motivation was to bring Manila's Tempest more in line with Upstream's Tempest
15:56:24 <bswartz> #topic open discussion
15:56:43 <bswartz> I wanted to remind everyone about feature proposal freeze august 20
15:56:58 <ganso_> bswartz: wasn't it August 14th?
15:57:03 <bswartz> that means all new features need to be in gerrit and with +1 from jenkins
15:57:21 <bswartz> errr I don't remember saying that
15:57:29 <bswartz> aug 20 is 2 weeks before L-3
15:57:48 <ganso_> bswartz: ok
15:57:50 <bswartz> that should be enough time to do reviews and get stuff merged
15:58:27 <bswartz> new features pushed to gerrit after aug 20 won't be considered for Liberty without a FFE
15:58:39 <bswartz> any questions?
15:59:10 <bswartz> anything else?
15:59:25 <markstur> FYI.  I'll be out Mon-Thu next week
15:59:33 <bswartz> okay thanks everyone I'll be back on Monday
15:59:42 <bswartz> markstur: thanks for reminder
15:59:51 <bswartz> thanks everyone
15:59:55 <dustins> thanks!
16:00:00 <bswartz> #endmeeting