15:00:05 <bswartz> #startmeeting manila 15:00:06 <openstack> Meeting started Thu Sep 10 15:00:05 2015 UTC and is due to finish in 60 minutes. The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:10 <bswartz> hello all 15:00:10 <openstack> The meeting name has been set to 'manila' 15:00:10 <cknight> Hi 15:00:22 <bswartz> #agenda https://wiki.openstack.org/wiki/Manila/Meetings 15:00:24 <markstur_> hi 15:00:26 <ganso_> hello 15:00:34 <tbarron> hi 15:00:42 <xyang1> Hi 15:00:50 <Zhongjun> hi 15:01:09 <toabctl> hi 15:01:11 <u_glide> hi 15:01:12 <bswartz> so this is the first meeting since the debacle last week with the gate 15:01:26 <vponomaryov> Hello 15:01:43 <bswartz> I sent out some ML posts about that over the past week 15:02:05 <Shamail> Hi 15:02:07 <bswartz> the good news is that all of the BPs which were granted technical FFEs did get merged before Tuesday 15:02:32 <bswartz> so we were able to start our bugfix and stabilization period only 5 days late 15:02:51 <bswartz> more about that later in the agenda 15:02:58 <bswartz> first thing is 15:03:03 <bswartz> #topic 3rd-party CI status 15:03:11 <bswartz> #link http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000/?project=openstack/manila&user=&timeframe=72&start=&end=&page_size=500 15:03:15 <lpabon> o/ 15:03:17 <bswartz> #link http://ci-watch.tintri.com/project?project=manila 15:03:29 <dustins> \o 15:03:49 <bswartz> so these are 2 nice tools that give you a quick view of the health of various CI systems 15:04:03 <bswartz> thanks whoever added those links to the agenda 15:04:36 <cknight> bswartz: welcome 15:04:45 <bswartz> There was a deadline for CI systems last thursday, which did not get discussed because of the gate breakage and urgent need to close out remaining features 15:04:52 <bswartz> cknight: ty! 15:05:02 <casusbelli> hi 15:05:25 <bswartz> I've been tracking several drivers progress on CI and the good news is that most of driver do have reporting CI systems 15:05:51 <bswartz> some of them are reporting very sporadically, and some of them fail nearly always 15:05:59 <bswartz> those things need to be fixed 15:06:19 <bswartz> but the deadline for healthy CI systems is Sept 24 15:07:12 <bswartz> if anyone thinks there will be a problem with meeting that date please ask for help 15:07:44 <bswartz> the first round of driver removal patches will go up today, but we can still remove drivers that have non-passing CI systems on Sept 24 15:08:08 <xyang1> bswartz: do you have a list? 15:08:12 <bswartz> so there are 2 driver that currently have no reporting CI systems 15:08:17 <bswartz> xyang1: yes 15:08:31 <bswartz> IBM GPFS has never reported a result 15:08:53 <xyang1> bswartz: ok 15:09:50 <bswartz> The maintainer for IBM GPFS is Gaurang Tapase (gaurangt) 15:10:20 <bswartz> I've talked to him and he says it's mostly working now and will be fully working by the deadline of Sept 24 15:11:06 <bswartz> I'm going to submit the patch to remove the GPFS driver and we can wait to merge it if the system starts reporting today 15:11:35 <cknight> bswartz: So your intent is to merge those 2 removal patches today? 15:11:39 <bswartz> The other driver that has never submitted a result is HDS SOP 15:12:07 <bswartz> The maintainer for HDS SOP is Jason Bishop (jasonsb) 15:12:46 <bswartz> so far I've gotten no responses from him on IRC or by email so I will encourage you to merge the HDS SOP driver removal patch today 15:13:30 <xyang1> bswartz: and you plan to add them back if they get CI up by 9/24? 15:14:00 <bswartz> xyang1: that's a good question 15:14:10 <bswartz> that's up to the community really 15:14:35 <cknight> bswartz: There is precent from Cinder, where drivers removed at Kilo.3 were re-added later. 15:14:42 <cknight> *precedent 15:14:47 <bswartz> I'm going to push the removal patches and it's up to you to merge them 15:15:07 <bswartz> I would be okay with reverting the removal patches if the CI systems start working to our satisfaction 15:15:21 <bswartz> but the published deadline for that is Sept 24 15:15:40 <vponomaryov> bswartz: what is the difference between always failing existing CI and absent CI? 15:15:44 <vponomaryov> bswartz: for us? 15:16:03 <vponomaryov> bswartz: both cases have no real useful result 15:16:08 <bswartz> just to be clear, I hate removing drivers, but we want to reward the work of those vendors who have worked hard to meet the deadlines 15:16:15 <dustins> bswartz: What're the criteria for satisfaction that a CI is working nominally? 15:16:30 <bswartz> CI systems have a very positive impact on quality and community involvement 15:16:55 <casusbelli> @vponomaryov i think a reporting CI means that at least there's a working CI infrastructure... 15:17:00 <bswartz> vponomaryov: the difference is that the deadlines we agreed to required that the system be reporting by Sept 3 and passing by Sept 24 15:17:26 <bswartz> the first deadline has been passed and 14/16 drivers met that deadline 15:18:12 <vponomaryov> bswartz: and if second deadline is not met then more drivers are going to be deleted? 15:18:16 <bswartz> the 2 that did not meet it are at serious risk of not making the next deadline, and the lack of communication from HDS on the SOP driver is a serious concern 15:18:33 <bswartz> vponomaryov: that's what we agreed to 15:19:20 <vponomaryov> bswartz: what what criteria do we have for "success" reporting? 15:19:26 <bswartz> what I'm going to do is push some canary patches and check how the CI systems respond 15:19:39 <cknight> bswartz: Dustin's question is valid. What constitutes a nominal pass rate? It must be completely objective. 15:20:02 <bswartz> vponomaryov: I think we will follow the cinder team's lead on that question 15:20:18 <bswartz> everyone understands that sometimes CI systems have temporary problems, and that's okay 15:20:26 <dustins> I imagine there are guidelines somewhere, I think it'd be beneficial for anyone looking through the meeting notes to have a reference 15:20:34 <bswartz> but we're looking for voting from CI systems that mostly matches what jenkins votes 15:20:51 <bswartz> so CI systems should pass when I push a change with no bugs 15:20:59 <bswartz> and CI systems should fail when push a change that introduces a bug 15:21:02 <bswartz> I will do both 15:21:43 <ganso_> bswartz: if a vendor CI fails for a reason, due to a change, a bug ticket should be opened, right? 15:21:49 <xyang1> bswartz: our CI was reporting successfully for a while but ran into problems recently. Still investigating 15:22:54 <bswartz> dustins: it's going to be by consensus of the core team -- I think we can tell if a CI system is generally working or totally broken 15:23:36 <bswartz> if the requirements aren't clear then it's a discussion we can have 15:23:46 <bswartz> we're not going to remove drivers just because the CI system had a hiccup 15:24:17 <bswartz> but it helps your case to have an established record of voting the same as jenkins that you can point to 15:24:25 <dustins> Indeed 15:25:23 <dustins> I think I'd want something on the Manila Wiki (if there isn't something already) to give some advice on what constitutes a "successful" CI system 15:25:27 <markstur_> some of us don't run CI until jenkins +1 15:26:20 <bswartz> just to answer cknight's earlier question, I believe the HDS SOP driver removal should proceed today, but I'm okay with giving IBM GPFS some time to vote, since guarangt is at least responsive 15:26:44 <bswartz> markstur_: I'll have to be clever with my canary patches to test the HP CI system then 15:27:08 <markstur_> should we all run on all jenkins -1? 15:27:08 <casusbelli> Quobyte does the same (wait for jenkins +1) 15:27:11 <bswartz> markstur_: I'll probably just push a patch that breaks your driver and nothing else 15:27:12 <cknight> bswartz: NetApp also doesn't run until Jenkins passes. That's pretty standard. 15:27:22 <markstur_> bswartz, thanks for that 15:27:25 <bswartz> okay so negative testing CI systems will be a challenge for me 15:28:44 <bswartz> however the most basic thing is to at least pass when jenkins passes 15:28:54 <bswartz> if a CI system doesn't do that then it's broken 15:29:04 <bswartz> negative testing is just to ensure that nobody is cheating 15:29:14 <casusbelli> Unless one can verify that it has hit a real bug. :) 15:29:50 <bswartz> casusbelli: that's true 15:30:27 <bswartz> to answer ganso_'s question above, if there are real bugs preventing your driving from passing tests, those should be filed in LP and targetted for RC1 15:30:46 <bswartz> we won't release RC1 until all the bugs are fixed or pushed out to Mitaka 15:31:30 <bswartz> I'm hopeful that everyone can get all their bugs fixed in the next 2 weeks though -- that should be everyone top priority 15:31:42 <bswartz> it would be better to fix them all in the next week 15:32:09 <bswartz> starting Sep 17 I'm going to aggressively push bugs to Mitaka if they're not fixed and not essential 15:32:45 <bswartz> arg, it looks like we've drifted offtopic to agenda item #4 15:32:50 <bswartz> getting back to the agenda... 15:32:55 <bswartz> #topic Manila devref update 15:33:14 <bswartz> u_glide: you're up 15:33:38 <u_glide> bswartz: thanks 15:33:50 <bswartz> #link http://docs.openstack.org/developer/manila/devref/share_back_ends_feature_support_mapping.html#mapping-of-share-drivers-and-share-features-support 15:34:11 <u_glide> Valeriy added this table in docs 15:34:47 <u_glide> Driver maintainers should update this table with latest information 15:35:14 <u_glide> and remove ""supported operations"" pages from drivers sections 15:35:27 <bswartz> yes please update the devref docs 15:35:46 <bswartz> In addition to bugfixing, now is a great time to make docs updates 15:36:00 <u_glide> if you have any questions - please ask 15:36:41 <u_glide> also I have proposal to move DHSS support to another table 15:36:50 <bswartz> sounds like no questions for now 15:37:08 <bswartz> I'd like to have that table accurate by the RC1 date 15:37:14 <cknight> bswartz: I guess a driver removal patch should also include this doc. 15:37:31 <bswartz> because it will get branched along with stable/liberty when we release RC1 15:37:44 <bswartz> and updating stable branches is always more hassle 15:38:30 <bswartz> okay moving on 15:38:30 <bswartz> #topic Automount vs asset injection 15:38:45 <csaba> so this was my topic 15:38:45 <bswartz> csaba: this was your topic from last week I believe 15:38:49 <csaba> yes 15:39:17 <csaba> it's related to stuff that would go into mitaka .. 15:39:49 <bswartz> csaba: did you want to make a design summit session proposal? 15:39:49 <csaba> so not an urgent topic -- but something we (gluster folks) need to sort out and ask for suggestions 15:40:08 <bswartz> or is it something we can solve at this meeting? or should we have an ML thread? 15:40:33 <bswartz> there were 2 approaches for mount automation that we pursued in Liberty 15:40:41 <csaba> bswartz: well just let me quikly dratft the problem here and we'll see where to take it on 15:40:41 <bswartz> the nova-based approach was rejeted 15:40:54 <bswartz> but the zaqar-based approach is still ongoing 15:41:36 <cknight> bswartz: the nova-based approach wasn't completely rejected, but it needs more work 15:41:46 <csaba> my concern is not just auto-mount is what might be needed but more generally (my coinage) "asset injection" 15:41:47 <bswartz> cknight: that's good to know 15:41:49 <cknight> bswartz: and we'll need a nova sponsor 15:43:04 <csaba> ie. have the tenants to have some stuff deployed somewhere... it does not need to culminate in a mount right away 15:43:41 <csaba> for example in case of cert based allowance (as done by gluster native) tenants need to have some certificates deployed 15:43:45 <bswartz> csaba: I think a concrete example would help 15:44:06 <csaba> bswartz: that's the concrete example, cert setup 15:44:25 <bswartz> csaba: who creates the certs? 15:44:31 <bswartz> the admin or the tenant? 15:44:55 <csaba> bswartz: the driver automates cert creation (or fetching from eg Barbican) 15:45:43 <bswartz> and who consumes the certs? 15:45:49 <csaba> the tenant 15:45:55 <bswartz> the VMs doing the mounting? 15:46:02 <csaba> yep 15:46:12 <csaba> a needed asset for mounting 15:46:16 <bswartz> can't the VMs pull them out of the same place that the driver pulls them? 15:47:04 <bswartz> this seems conceptually similar to, in CIFS, where a client and a share server must be part of the same Active Directory domain in order to authenticate 15:47:04 <csaba> that's a viable scheme, with Barbican, but then the authentication problem is pushed further out 15:47:36 <csaba> and there should be an option of managing it in our scope 15:47:47 <bswartz> in that case the tenant tells Manila up front which service service to use (the security service contains the AD details) and manila ensure that share servers do the right thing 15:48:19 <bswartz> s/service service/security service/ 15:49:14 <bswartz> without knowing more details, my gut feeling is that there needs to be a gluster type of security service, and we need to make sure users of gluster have a way to specify that 15:49:26 <csaba> hmm yeah but signing the cert (or getting the cert signed) might be out of the capabilities of the tenant 15:49:33 <bswartz> it may not work with the current architecture where security services are tied into share networks 15:50:01 <bswartz> csaba: the security service could simply be a pointer to a trusted 3rd party that could sign on behalf of both the clients and the servers 15:50:29 <bswartz> and that trusted 3rd party may in fact be barbican if you wish 15:50:40 <csaba> bswartz: the question is that on behalf of whom is that trusted 3rd party willing to perform signing 15:50:56 <bswartz> yeah I get there are some complexities 15:51:16 <bswartz> we don't need to design the feature right here in the meeting, since it feels glusterfs specific 15:51:21 <bswartz> but the mechanism is something we should agree to 15:51:55 <csaba> I just though that it would probably make sense to extend the idea of mount automation to "asset injection" 15:52:01 <bswartz> is there a problem with relying on something like manila security services to provide the tenant-facing part of this ? 15:52:19 <bswartz> oh okay 15:52:24 <csaba> actually the 3rpparty trusted service is a solution to the gluster problem, but another approach 15:52:52 <csaba> that is beyond these few minutes to discuss in detail, for sure.. 15:53:19 <bswartz> well yeah I suggest making a proposal with a BP and spec and we can add it to the list for Tokyo 15:53:44 <csaba> but "asset injection" is just a more simplistic scheme -- allow the driver (in case of gluster: sighned cert files ) to push to tenant vms througfh some mechanism 15:54:00 <bswartz> yeah it's not a crazy idea 15:54:17 <bswartz> the challenge will be doing it in a way that doesn't damage the protocol or vendor neutrality of the Manila API 15:54:26 <bswartz> I think it can probably be done 15:54:35 <csaba> which I feel to be similar to automount, in that it is some driver generatied data that is to be obtained by the tenatt 15:54:40 <bswartz> yes 15:54:56 <csaba> would ssh injection be an option eg? 15:55:03 <bswartz> what do you mean by ssh injection? 15:55:51 <bswartz> nova has a way to inject key pairs into VMs, but it only works with cloud-init and with a non-buggy neutron 15:55:55 <csaba> manila has a pre-agreed public key ssh that tenant vms authorize to access 15:56:25 <csaba> hmm yeah could we push data into cloud init from mainla context? 15:56:34 <bswartz> if the glusterfs certificates are literally SSH keys then that might be easy to solve 15:56:45 <bswartz> if they're something else then you might have nova work to do to get them injected 15:57:14 <bswartz> since we're almost out of time I'm going to open up the floor 15:57:18 <bswartz> #topic open discussion 15:57:22 <csaba> bswartz: not literally ssh keys, but could init would work out for us. can a driver specify cloud init payload for the tenatn vms? 15:57:23 <bswartz> anything else we need to mention today? 15:57:39 <ganso> bswartz: is the deadline Sept 17th or Sept 24th? 15:57:41 <bswartz> csaba: no not currently 15:57:46 <ganso> bswartz: for RC1 15:58:02 <bswartz> ganso: CI deadline is consistently passing by Sep 24th 15:58:14 <bswartz> RC1 has no date 15:58:23 <bswartz> RC1 is when there are no open bugs 15:58:36 <bswartz> we achieve that by either fixing the bugs or targeting them to Mitaka 15:59:04 <ganso> bswartz: on Sept 17th you said you would be pushing lower priority bugs to Mitaka, correct? 15:59:20 <bswartz> starting Sept 17th I'm going to push unfixed bugs to Mitaka if they're not high priority 15:59:25 <bswartz> yes 15:59:40 <ganso> bswartz: ok, got it, thanks! 15:59:51 <bswartz> thanks everyone 15:59:58 <bswartz> #endmeeting