15:01:25 #startmeeting manila 15:01:25 Meeting started Thu Nov 14 15:01:25 2013 UTC and is due to finish in 60 minutes. The chair is bswartz. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:26 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:28 The meeting name has been set to 'manila' 15:01:43 good morning folks 15:01:53 good morning 15:01:56 Good Morning 15:02:09 I don't have an agenda today 15:02:24 today is my first day back since I extended my stay in Hong Kong 15:02:44 we can talk about the conference however 15:03:07 *conference was good" 15:03:27 yeah I enjoyed meeting many of you at the conference 15:03:30 glenng: +1 15:03:55 nice meeting everyone 15:03:56 I think that the unconference session was successful 15:04:23 for those of you who weren't able to be there, the unconference session was well attended, and the questions asked were good ones 15:05:40 the main thing that i realized after seeing how much interest there is in the project is that we need to get the project into a state where vendors can write backends very soon 15:06:07 +1 15:06:08 I suspect the best way to do that is to get the mutitenant support working ASAP, and to add support for that to the NetApp driver 15:06:12 bswartz: agree 15:06:20 bswartz: Agree. 15:06:26 +1 15:06:48 that means that the so-called Generic driver (or "LVM" driver) is less important in the short term 15:07:19 the generic driver has been sucking up a lot of time and attention and ultimately it's not critical to getting other backends implemented 15:07:32 bswartz: I can make update about multitenancy 15:07:34 LVM driver is of most interest to the TC - i suspect most of the people here want to build their own drivers. 15:07:40 it IS critical for an actual release, so we can't simply ignore it 15:08:10 1) Generic LXC share driver - https://review.openstack.org/#/c/55265/ 15:08:13 oh thanks for reminding me caitlin56 15:08:41 2) Neutron API module - https://review.openstack.org/#/c/55555/ 15:08:41 3) Network allocation in ShareManager - https://review.openstack.org/#/c/55558/ 15:08:54 evidently the TC considered our incubation request last tuesday, while I was on an airplane 15:08:56 I thought we are going with full blown VM? 15:09:33 I was under the impression that it would be considered next tuesday but they're considering it all this week 15:09:42 so the LVM driver is an issue for them 15:09:59 bswartz: even I was under the impression that it would be on 19th 15:10:03 there are a few important points about that 15:10:11 vbellur: I bet it's because I told you that date 15:10:23 vbellur: I was the one who was mistaken however 15:10:48 bswartz: possibly so, I never bothered to check either. 15:10:52 1) We must stop calling it the "LVM driver" -- it shall becalled the "Generic Driver" henceforth 15:11:54 2) I propose that the generic driver uses virtualization (based on nova) and a block device backend (based on cinder) 15:12:24 vponomaryov: I have yet to look at the LXC code that's been written, but if it works I don't want to ignore it 15:12:32 bswartz: and implements generic NFS/CIFS from stock linux? 15:12:55 bswartz: yes, it works, tested 15:13:02 vponomaryov: As I mentioned above, everything related to the generic driver is low priority in the short term though 15:13:20 for each implementation of protocols can be written class-helper 15:13:43 yponmaryov: but isn't setting up things like tenant specific DNS take extra work (even though I have seen scripts that demonstrate how). 15:13:44 but not tied yet with neutron code 15:13:46 vponomaryov: I'll look at it soon, the other 2 changes are more interesting however 15:14:10 vponomaryov: First code review link works, other 2 give permission errors. Is this expected? 15:14:37 yes it seems that the submissions are marked private currently 15:15:25 jcorbin: yes, it hat restricted access, at least, for now 15:16:10 vponomaryov: please add access to jcorbin as he's familiar with neutron and I'd like his feedback as well 15:16:38 anyone else who's curious please ping vponomaryov too 15:17:05 mee too 15:17:09 vponomaryov: please add me too 15:17:16 vponomaryov: me too 15:17:19 jcorbin: provide your username in launchpad 15:17:52 so the other thing that's clear from the conference is that there is strong interest in the "tunneled" style of access, expecially for backends like ceph and gluster 15:17:57 vponomaryov: My Launchpad name is ggobeli 15:18:04 Any IBM/GPFS folks here? 15:18:19 vponomaryov: My Launchpad name is osjcorbin. Thanks! 15:18:26 vponomaryov: also me - billowen@us.ibm.com 15:18:28 vponomaryov: it looks like *Everyone* is interested may as well mark it public 15:18:31 vponomaryov: my launchpad name is: xing-yang 15:18:34 And I'm IBM/GPFS 15:18:56 bill_az: have you given any thought to the multitenant access issues? 15:18:59 vponomaryov: my launchpad id is vbellur 15:19:08 bill_az: is GPFS going to be in the same boat as ceph and gluster? 15:19:18 bswartz: I can inform our devs, I am not the owner of gerrit commit 15:19:28 vponomaryov: ty 15:19:29 bswartz: I have been discussing it with others on the team 15:19:51 bswartz: we have some things coming in gpfs 4.1 in the spring that will help w/ multitenancy 15:19:52 bswartz: by "tunneled" do you mean some form of "virtFS"? 15:19:53 mine is singn 15:20:28 So just to recap, many clustered filesystems such as ceph and gluster doesn't have facilities to provide "virtual servers" on different tenant networks 15:20:56 These clustered filesystems assume full network connectivity between the client and every server node 15:21:29 bswartz: I believe the fundamental issue is who deals with tenant users. The "virtFS" approach puts this on the Hypervisor, the virtual server approach puts it on the file server. 15:21:54 caitlin56: yes 15:22:06 that's part 1, part 2 is the networking constraints imposed by the full connectivity bswartz mentioned 15:22:26 My evaluation is that relying on the hypervisor a) will add a minimum of one year to the project and b) doesn't truly secure the files even then. 15:22:43 bill_az, I assume gpfs also requires connectivity to every node, right? 15:22:50 so for clustered filesystems that aren't able to create a virtual-server-per-tenant, due to the number of network interfaces involved, we need a hypervisor-mediated approach to secure multitenancy 15:23:24 that could mean virtfs, or NFS-over-PPP, or NFS-over-vsock 15:23:26 Nobody is *unable* to do a virtual server per tenant, they just haven't done it yet. 15:23:41 gregsfortytwo1: yes -thats correct 15:24:06 caitlin56: if "server" — implying one — is nonsensical in your architecture you're unable 15:24:10 caitlin56: I think the issue is the number of network interfaces involved -- it's simply impractical 15:24:21 Tunneling NFS with less-than-a-tenant-network sounds like a simpler approach than virtFS. 15:24:26 we are all able to implement multienancy even if we haven't, but that's not the biggest issue from Manila's perspective 15:24:28 gregsfortytwo1: but we plan to limit access through isolated filesystems and encryption keys 15:24:37 if you have a 200 node ceph cluster, you can't create 200 additional virtual network interfaces for every tenant 15:25:26 But even with tunneled NFS, the server is going to have to do per-tenant authentication, which I presume they are also incapable of doing today. 15:25:28 bill_az: yeah, that's essentially the path Ceph is walking down, though not very quickly 15:26:09 caitlin56: the theory would be to create a NFS-to-ceph bridge, or an NFS-to-glusterFS bridge 15:26:25 gregsfortytwo1: it will be tough for us for icehouse - but should have that for j 15:26:25 the NFS itself would have no authentication 15:26:37 That's fine. That's a vendor provided proxy. Effectively a virtual server. 15:26:44 caitlin56: no, we can handle that already in most of the stack; not sure about gluster; sounds like gpfs can/will 15:26:44 fortunately bad security is something NFS is very good at 15:27:27 gregfortytwo: my main concern is avoiding an openstack managed common UID/GID pool that everyone maps to. 15:27:27 gregsfortytwo1: gluster can handle that too 15:29:03 caitlin56: ah, hadn't even gotten to that issue yet in my head :) 15:29:10 okay so I'm glad we're all thinking about the issue here -- tunneled FS support will need to come soon after the full network connectivity style of multitenant support 15:29:18 issues* 15:29:49 question: does it need to be a lib, or can it just be a NAS proxy? 15:29:51 those are the 2 main roadblocks to allowing various storage vendors to write functional backends 15:30:38 the actual implementation of the bridge/proxy will need to run on the compute node -- whether it's a manila-agent, or changes to nova, or some library -- I'm not sure about taht 15:30:56 for a prototype we should keep the code in manila 15:31:25 long term it might make sense to host it elsewhere -- especially if it's generically useful outside of manila 15:31:30 So this would be a pluggable library that ran in the same context as the manilla agent? 15:31:58 I want to be careful with the term "manila agent" because it could mean a lot of different things 15:32:18 You would want it to be on a path that made sense for bulk data transfer - preferably "close" to the guest. 15:32:43 the idea of automatic mounts inside the nova instances is still interesting to many -- and those may necessitate a different kind of agent 15:33:13 but absolutely it can/should run on the compute node 15:33:43 possibly "manila-bridge" or "manila-gateway" would be a more descriptive term 15:33:51 bswartz: proxying TCP connections is something compatible with the current infrastructure. Passing full file-systems through hypervisors will be a lot more work. 15:34:13 caitlin56: we're talking about the later not the former 15:34:37 I'm aware that it will be real work 15:34:39 bswartz: distinguishing between manila-bridge (control plane) vs. manila-gateway (user-plane) sounds like a good idea to me. 15:35:32 anyways today I just wanted to lay out the priorities 15:36:02 1) get the neutron code working so we can do full network multitenancy 15:36:25 I first looked at NAS proxies for VMs two years ago, and concluded that intercepting at the network layer was more feasible than at the FS-operation layer. 15:36:30 2) implement some tunneling/virtfs code to support mediated multitenancy 15:36:40 3) implement the generic driver 15:36:58 I feared that even if Xen and KVM adopted "virtFS" it would take two years to deploy. In those 2 years there has been no progress on adopting a virtFS API. 15:37:29 caitlin56: I'd be curious to know why that is -- let's take that discussion offline 15:37:51 regarding virtfs -- if that turns out to be a blocker than NFSv3-over-vsock may be a faster path to get something working 15:38:37 s/offline/to #openstack-manila? :) 15:38:46 Let each proxy decide whether to support NFSv3 and/or NFSv4. 15:38:56 there are lots of technologies that can be applied here -- I'm confident we'll find something that works 15:38:57 or for that matter CIFS. 15:40:14 caitlin56: we're going to have to write these proxies ourselves though -- I'm not eager to take on tons of work 15:40:39 Correct, these are pluggable proxies. Someone has to provide their own proxy. 15:40:54 If NFSv3 works as a bridge protocol, I think it will be hard to argue that there's a significant advantage to using something like cifs or v4 15:41:11 Am I correct that neither CEPH or Gluster would want soeone else writing the proxies to access their networks? 15:41:34 caitlin56: I think you're incorrect 15:41:52 I should be the job of manila to provide the glue 15:41:56 no reason to independently implement setting up an nfs server instance ;) 15:42:19 my vision is to implement one thing that works for everyone 15:42:44 So you want a manila coded proxy that maps UID/GIDs for NFSv3 to NFSv3? 15:42:52 once we have a gateway of sorts that runs on the compute node, it shouldn't matter what's on the actual backend -- as long as the compute node can mount it, it can be tunneled through to the guest 15:43:30 caitlin56: I don't want any UID/GID mapping to occur 15:43:47 if we do this correctly, it should be safe to do no mapping 15:43:48 Another form of proxy would translate Any-NAS-Protocol to My-backends-proprietary-protocol. 15:44:04 in lxc implementation we can mount directly to vms 15:44:37 we will can :) 15:44:58 okay so we have some conflicting ideas here 15:45:19 maybe my vision will not work out 15:45:26 can we at least agree on the priorities I mentioned? 15:46:00 those who are interested in the tunneling approach should probably form a working group and hash out the issues 15:46:09 If you aren't doing UID/GID mapping then you would have to trust the gateway to do mount authentication. Then all you need to map to a universal pool are the mounts. 15:47:06 I'm interested in the tunneled approach working correctly, so I'll organize those discussions 15:47:38 * caitlin56 is interested in both models - could see implementing either on our end. 15:47:51 I suspect that we might be making this harder than it needs to be though 15:48:29 My instinct is that we should write a prototype first, and then discover all the ways that it's wrong 15:48:31 bswartz: priorities seem fine to me 15:48:35 You can make it simpler if you just state the caveat that a gateway solution requires trusting the gateway. 15:48:58 caitlin56: oh yes, absolutely 15:49:23 bswartz: +1 to the prototype 15:49:34 caitlin56: the gateway approach is akin to how you must trust the hypervisor to faithfully present block devices to you 15:49:37 bswartz: +1 - get the simlest --realistic-- prototype working asap as a model and to help discussion 15:49:50 bswartz: +1 to the prototype 15:50:28 this "manila-gateway" thing would be running on the compute node, and it would be part of the cloud infrastructure -- hidden from the tenant's view 15:50:30 +1, but we should not wait onthe multi-tenant network solution. 15:50:33 the gateway would be essentially the generic implementation, plus some holes that different drivers can plug in order to set up their own backends, right? 15:50:39 the tenant would have no choice but to trust it 15:51:20 gregsfortytwo1: sounds like a good model 15:51:26 bswartz: the multi-tenant network approach with virtual servers avoids needing to trust the compute host. 15:51:40 gregsfortytwo1: the gateway is really a get-out-of-jail-free card for backends that can't connect themselves directly to tenant networks on account of the limitted-number-of-network-interfaces problem 15:52:06 caitlin56: yeah I believe that's the best method for backends than can support it -- it's just that not all can 15:53:06 in fact it's come to my attention that even some more traditional NFS-based storage devices (I'm thinking of Isolon) can't directly support multitenancy, and may need something like this to work in public cloud environments 15:53:17 bswartz: if the server trusts the gateway to mount only exports approved by the correct authentication server then you only need a single LAN netowrk between the compute hosts and the file servers. 15:53:56 caitlin56: yes that's the idea 15:54:03 okay since we're nearly out of time 15:54:09 #topic open discussion 15:54:23 does anyone else have another topic that I didn't bring up? 15:55:31 wow that quieted things down 15:55:31 bswartz: is there a summary of manila activities at summit? 15:55:49 bill_az: yes 15:56:03 as I mentioned at the beginning, we have a unconference session which went well 15:56:13 we had* a unconference session 15:56:47 and we also got together for drinks and I managed to get my boss to pay the bar tab 15:56:48 bswartz: sorry - i missed the first few minutes - is there minutes or summary published? 15:57:16 thanks to everyone who showed up -- it's always great to meet face to face before we go off and spend 6 months coding together 15:57:22 bswartz: thanks for the drinks! :) 15:57:22 bswartz: sorry I missed both! (especially drinks...) 15:58:14 and I can't tell you how many people grabbed me personally in the hallways to talk about manila 15:58:23 the interest level is high 15:58:34 we need to move fast while we have momentum 15:58:37 bswartz: +1 15:59:14 I'll be talking to TC members about the incubation request -- we should have an answer on tuesday 15:59:31 bswartz: that all sounds good - Thanks 15:59:33 and with that we're out of time 15:59:35 Let's hope, 15:59:36 thanks everyone 15:59:42 thanks all 15:59:51 #endmeeting