15:00:05 <bswartz> #startmeeting manila
15:00:06 <openstack> Meeting started Thu Sep 10 15:00:05 2015 UTC and is due to finish in 60 minutes.  The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:10 <bswartz> hello all
15:00:10 <openstack> The meeting name has been set to 'manila'
15:00:10 <cknight> Hi
15:00:22 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings
15:00:24 <markstur_> hi
15:00:26 <ganso_> hello
15:00:34 <tbarron> hi
15:00:42 <xyang1> Hi
15:00:50 <Zhongjun> hi
15:01:09 <toabctl> hi
15:01:11 <u_glide> hi
15:01:12 <bswartz> so this is the first meeting since the debacle last week with the gate
15:01:26 <vponomaryov> Hello
15:01:43 <bswartz> I sent out some ML posts about that over the past week
15:02:05 <Shamail> Hi
15:02:07 <bswartz> the good news is that all of the BPs which were granted technical FFEs did get merged before Tuesday
15:02:32 <bswartz> so we were able to start our bugfix and stabilization period only 5 days late
15:02:51 <bswartz> more about that later in the agenda
15:02:58 <bswartz> first thing is
15:03:03 <bswartz> #topic 3rd-party CI status
15:03:11 <bswartz> #link http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack/manila&user=&timeframe=72&start=&end=&page_size=500
15:03:15 <lpabon> o/
15:03:17 <bswartz> #link http://ci-watch.tintri.com/project?project=manila
15:03:29 <dustins> \o
15:03:49 <bswartz> so these are 2 nice tools that give you a quick view of the health of various CI systems
15:04:03 <bswartz> thanks whoever added those links to the agenda
15:04:36 <cknight> bswartz: welcome
15:04:45 <bswartz> There was a deadline for CI systems last thursday, which did not get discussed because of the gate breakage and urgent need to close out remaining features
15:04:52 <bswartz> cknight: ty!
15:05:02 <casusbelli> hi
15:05:25 <bswartz> I've been tracking several drivers progress on CI and the good news is that most of driver do have reporting CI systems
15:05:51 <bswartz> some of them are reporting very sporadically, and some of them fail nearly always
15:05:59 <bswartz> those things need to be fixed
15:06:19 <bswartz> but the deadline for healthy CI systems is Sept 24
15:07:12 <bswartz> if anyone thinks there will be a problem with meeting that date please ask for help
15:07:44 <bswartz> the first round of driver removal patches will go up today, but we can still remove drivers that have non-passing CI systems on Sept 24
15:08:08 <xyang1> bswartz: do you have a list?
15:08:12 <bswartz> so there are 2 driver that currently have no reporting CI systems
15:08:17 <bswartz> xyang1: yes
15:08:31 <bswartz> IBM GPFS has never reported a result
15:08:53 <xyang1> bswartz: ok
15:09:50 <bswartz> The maintainer for IBM GPFS is Gaurang Tapase (gaurangt)
15:10:20 <bswartz> I've talked to him and he says it's mostly working now and will be fully working by the deadline of Sept 24
15:11:06 <bswartz> I'm going to submit the patch to remove the GPFS driver and we can wait to merge it if the system starts reporting today
15:11:35 <cknight> bswartz: So your intent is to merge those 2 removal patches today?
15:11:39 <bswartz> The other driver that has never submitted a result is HDS SOP
15:12:07 <bswartz> The maintainer for HDS SOP is Jason Bishop (jasonsb)
15:12:46 <bswartz> so far I've gotten no responses from him on IRC or by email so I will encourage you to merge the HDS SOP driver removal patch today
15:13:30 <xyang1> bswartz: and you plan to add them back if they get CI up by 9/24?
15:14:00 <bswartz> xyang1: that's a good question
15:14:10 <bswartz> that's up to the community really
15:14:35 <cknight> bswartz: There is precent from Cinder, where drivers removed at Kilo.3 were re-added later.
15:14:42 <cknight> *precedent
15:14:47 <bswartz> I'm going to push the removal patches and it's up to you to merge them
15:15:07 <bswartz> I would be okay with reverting the removal patches if the CI systems start working to our satisfaction
15:15:21 <bswartz> but the published deadline for that is Sept 24
15:15:40 <vponomaryov> bswartz: what is the difference between always failing existing CI and absent CI?
15:15:44 <vponomaryov> bswartz: for us?
15:16:03 <vponomaryov> bswartz: both cases have no real useful result
15:16:08 <bswartz> just to be clear, I hate removing drivers, but we want to reward the work of those vendors who have worked hard to meet the deadlines
15:16:15 <dustins> bswartz: What're the criteria for satisfaction that a CI is working nominally?
15:16:30 <bswartz> CI systems have a very positive impact on quality and community involvement
15:16:55 <casusbelli> @vponomaryov i think a reporting CI means that at least there's a working CI infrastructure...
15:17:00 <bswartz> vponomaryov: the difference is that the deadlines we agreed to required that the system be reporting by Sept 3 and passing by Sept 24
15:17:26 <bswartz> the first deadline has been passed and 14/16 drivers met that deadline
15:18:12 <vponomaryov> bswartz: and if second deadline is not met then more drivers are going to be deleted?
15:18:16 <bswartz> the 2 that did not meet it are at serious risk of not making the next deadline, and the lack of communication from HDS on the SOP driver is a serious concern
15:18:33 <bswartz> vponomaryov: that's what we agreed to
15:19:20 <vponomaryov> bswartz: what what criteria do we have for "success" reporting?
15:19:26 <bswartz> what I'm going to do is push some canary patches and check how the CI systems respond
15:19:39 <cknight> bswartz: Dustin's question is valid.  What constitutes a nominal pass rate?  It must be completely objective.
15:20:02 <bswartz> vponomaryov: I think we will follow the cinder team's lead on that question
15:20:18 <bswartz> everyone understands that sometimes CI systems have temporary problems, and that's okay
15:20:26 <dustins> I imagine there are guidelines somewhere, I think it'd be beneficial for anyone looking through the meeting notes to have a reference
15:20:34 <bswartz> but we're looking for voting from CI systems that mostly matches what jenkins votes
15:20:51 <bswartz> so CI systems should pass when I push a change with no bugs
15:20:59 <bswartz> and CI systems should fail when push a change that introduces a bug
15:21:02 <bswartz> I will do both
15:21:43 <ganso_> bswartz: if a vendor CI fails for a reason, due to a change, a bug ticket should be opened, right?
15:21:49 <xyang1> bswartz: our CI was reporting successfully for a while but ran into problems recently.  Still investigating
15:22:54 <bswartz> dustins: it's going to be by consensus of the core team -- I think we can tell if a CI system is generally working or totally broken
15:23:36 <bswartz> if the requirements aren't clear then it's a discussion we can have
15:23:46 <bswartz> we're not going to remove drivers just because the CI system had a hiccup
15:24:17 <bswartz> but it helps your case to have an established record of voting the same as jenkins that you can point to
15:24:25 <dustins> Indeed
15:25:23 <dustins> I think I'd want something on the Manila Wiki (if there isn't something already) to give some advice on what constitutes a "successful" CI system
15:25:27 <markstur_> some of us don't run CI until jenkins +1
15:26:20 <bswartz> just to answer cknight's earlier question, I believe the HDS SOP driver removal should proceed today, but I'm okay with giving IBM GPFS some time to vote, since guarangt is at least responsive
15:26:44 <bswartz> markstur_: I'll have to be clever with my canary patches to test the HP CI system then
15:27:08 <markstur_> should we all run on all jenkins -1?
15:27:08 <casusbelli> Quobyte does the same (wait for jenkins +1)
15:27:11 <bswartz> markstur_: I'll probably just push a patch that breaks your driver and nothing else
15:27:12 <cknight> bswartz: NetApp also doesn't run until Jenkins passes.  That's pretty standard.
15:27:22 <markstur_> bswartz, thanks for that
15:27:25 <bswartz> okay so negative testing CI systems will be a challenge for me
15:28:44 <bswartz> however the most basic thing is to at least pass when jenkins passes
15:28:54 <bswartz> if a CI system doesn't do that then it's broken
15:29:04 <bswartz> negative testing is just to ensure that nobody is cheating
15:29:14 <casusbelli> Unless one can verify that it has hit a real bug. :)
15:29:50 <bswartz> casusbelli: that's true
15:30:27 <bswartz> to answer ganso_'s question above, if there are real bugs preventing your driving from passing tests, those should be filed in LP and targetted for RC1
15:30:46 <bswartz> we won't release RC1 until all the bugs are fixed or pushed out to Mitaka
15:31:30 <bswartz> I'm hopeful that everyone can get all their bugs fixed in the next 2 weeks though -- that should be everyone top priority
15:31:42 <bswartz> it would be better to fix them all in the next week
15:32:09 <bswartz> starting Sep 17 I'm going to aggressively push bugs to Mitaka if they're not fixed and not essential
15:32:45 <bswartz> arg, it looks like we've drifted offtopic to agenda item #4
15:32:50 <bswartz> getting back to the agenda...
15:32:55 <bswartz> #topic Manila devref update
15:33:14 <bswartz> u_glide: you're up
15:33:38 <u_glide> bswartz: thanks
15:33:50 <bswartz> #link http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html#mapping-of-share-drivers-and-share-features-support
15:34:11 <u_glide> Valeriy added this table in docs
15:34:47 <u_glide> Driver maintainers should update this table with latest information
15:35:14 <u_glide> and remove ""supported operations"" pages from drivers sections
15:35:27 <bswartz> yes please update the devref docs
15:35:46 <bswartz> In addition to bugfixing, now is a great time to make docs updates
15:36:00 <u_glide> if you have any questions - please ask
15:36:41 <u_glide> also I have proposal to move DHSS support to another table
15:36:50 <bswartz> sounds like no questions for now
15:37:08 <bswartz> I'd like to have that table accurate by the RC1 date
15:37:14 <cknight> bswartz: I guess a driver removal patch should also include this doc.
15:37:31 <bswartz> because it will get branched along with stable/liberty when we release RC1
15:37:44 <bswartz> and updating stable branches is always more hassle
15:38:30 <bswartz> okay moving on
15:38:30 <bswartz> #topic Automount vs asset injection
15:38:45 <csaba> so this was my topic
15:38:45 <bswartz> csaba: this was your topic from last week I believe
15:38:49 <csaba> yes
15:39:17 <csaba> it's related to stuff that would go into mitaka ..
15:39:49 <bswartz> csaba: did you want to make a design summit session proposal?
15:39:49 <csaba> so not an urgent topic -- but something we (gluster folks) need to sort out and ask for suggestions
15:40:08 <bswartz> or is it something we can solve at this meeting? or should we have an ML thread?
15:40:33 <bswartz> there were 2 approaches for mount automation that we pursued in Liberty
15:40:41 <csaba> bswartz: well just let me quikly dratft the problem here and we'll see where to take it on
15:40:41 <bswartz> the nova-based approach was rejeted
15:40:54 <bswartz> but the zaqar-based approach is still ongoing
15:41:36 <cknight> bswartz: the nova-based approach wasn't completely rejected, but it needs more work
15:41:46 <csaba> my concern is not just auto-mount is what might be needed but more generally (my coinage) "asset injection"
15:41:47 <bswartz> cknight: that's good to know
15:41:49 <cknight> bswartz: and we'll need a nova sponsor
15:43:04 <csaba> ie. have the tenants to have some stuff deployed somewhere... it does not need to culminate in a mount right away
15:43:41 <csaba> for example in case of cert based allowance (as done by gluster native) tenants need to have some certificates deployed
15:43:45 <bswartz> csaba: I think a concrete example would help
15:44:06 <csaba> bswartz: that's the concrete example, cert setup
15:44:25 <bswartz> csaba: who creates the certs?
15:44:31 <bswartz> the admin or the tenant?
15:44:55 <csaba> bswartz: the driver automates cert creation (or fetching from eg Barbican)
15:45:43 <bswartz> and who consumes the certs?
15:45:49 <csaba> the tenant
15:45:55 <bswartz> the VMs doing the mounting?
15:46:02 <csaba> yep
15:46:12 <csaba> a needed asset for mounting
15:46:16 <bswartz> can't the VMs pull them out of the same place that the driver pulls them?
15:47:04 <bswartz> this seems conceptually similar to, in CIFS, where a client and a share server must be part of the same Active Directory domain in order to authenticate
15:47:04 <csaba> that's a viable scheme, with Barbican, but then the authentication problem is pushed further out
15:47:36 <csaba> and there should be an option of managing it in our scope
15:47:47 <bswartz> in that case the tenant tells Manila up front which service service to use (the security service contains the AD details) and manila ensure that share servers do the right thing
15:48:19 <bswartz> s/service service/security service/
15:49:14 <bswartz> without knowing more details, my gut feeling is that there needs to be a gluster type of security service, and we need to make sure users of gluster have a way to specify that
15:49:26 <csaba> hmm yeah but signing the cert (or getting the cert signed) might be out of the capabilities of the tenant
15:49:33 <bswartz> it may not work with the current architecture where security services are tied into share networks
15:50:01 <bswartz> csaba: the security service could simply be a pointer to a trusted 3rd party that could sign on behalf of both the clients and the servers
15:50:29 <bswartz> and that trusted 3rd party may in fact be barbican if you wish
15:50:40 <csaba> bswartz: the question is that on behalf of whom is that trusted 3rd party willing to perform signing
15:50:56 <bswartz> yeah I get there are some complexities
15:51:16 <bswartz> we don't need to design the feature right here in the meeting, since it feels glusterfs specific
15:51:21 <bswartz> but the mechanism is something we should agree to
15:51:55 <csaba> I just though that it would probably make sense to extend the idea of mount automation to "asset injection"
15:52:01 <bswartz> is there a problem with relying on something like manila security services to provide the tenant-facing part of this ?
15:52:19 <bswartz> oh okay
15:52:24 <csaba> actually the 3rpparty trusted service is a solution to the gluster problem, but another approach
15:52:52 <csaba> that is beyond these few minutes to discuss in detail, for sure..
15:53:19 <bswartz> well yeah I suggest making a proposal with a BP and spec and we can add it to the list for Tokyo
15:53:44 <csaba> but "asset injection" is just a more simplistic scheme -- allow the driver (in case of gluster: sighned cert files ) to push to tenant vms througfh some mechanism
15:54:00 <bswartz> yeah it's not a crazy idea
15:54:17 <bswartz> the challenge will be doing it in a way that doesn't damage the protocol or vendor neutrality of the Manila API
15:54:26 <bswartz> I think it can probably be done
15:54:35 <csaba> which I feel to be similar to automount, in that it is some driver generatied data that is to be obtained by the tenatt
15:54:40 <bswartz> yes
15:54:56 <csaba> would ssh injection be an option eg?
15:55:03 <bswartz> what do you mean by ssh injection?
15:55:51 <bswartz> nova has a way to inject key pairs into VMs, but it only works with cloud-init and with a non-buggy neutron
15:55:55 <csaba> manila has a pre-agreed public key ssh that tenant vms authorize to access
15:56:25 <csaba> hmm yeah could we push data into cloud init from mainla context?
15:56:34 <bswartz> if the glusterfs certificates are literally SSH keys then that might be easy to solve
15:56:45 <bswartz> if they're something else then you might have nova work to do to get them injected
15:57:14 <bswartz> since we're almost out of time I'm going to open up the floor
15:57:18 <bswartz> #topic open discussion
15:57:22 <csaba> bswartz: not literally ssh keys, but could init would work out for us. can a driver specify cloud init payload for the tenatn vms?
15:57:23 <bswartz> anything else we need to mention today?
15:57:39 <ganso> bswartz: is the deadline Sept 17th or Sept 24th?
15:57:41 <bswartz> csaba: no not currently
15:57:46 <ganso> bswartz: for RC1
15:58:02 <bswartz> ganso: CI deadline is consistently passing by Sep 24th
15:58:14 <bswartz> RC1 has no date
15:58:23 <bswartz> RC1 is when there are no open bugs
15:58:36 <bswartz> we achieve that by either fixing the bugs or targeting them to Mitaka
15:59:04 <ganso> bswartz: on Sept 17th you said you would be pushing lower priority bugs to Mitaka, correct?
15:59:20 <bswartz> starting Sept 17th I'm going to push unfixed bugs to Mitaka if they're not high priority
15:59:25 <bswartz> yes
15:59:40 <ganso> bswartz: ok, got it, thanks!
15:59:51 <bswartz> thanks everyone
15:59:58 <bswartz> #endmeeting