15:00:52 <tbarron> #startmeeting manila
15:00:53 * bswartz looks around for manila ppl
15:00:54 <openstack> Meeting started Thu Nov  8 15:00:52 2018 UTC and is due to finish in 60 minutes.  The chair is tbarron. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:55 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:57 <openstack> The meeting name has been set to 'manila'
15:01:00 <bswartz> .o/
15:01:02 <tbarron> hey ben
15:01:05 <gouthamr> o/
15:01:15 <tbarron> let's wait a min ...
15:01:28 <tbarron> ping ganso
15:01:30 <ganso> hello
15:01:42 <tbarron> ping vkmc
15:01:47 <tbarron> ping dustins
15:01:47 <vkmc> o/
15:02:35 <tbarron> ping zhongjun
15:02:36 <dustins> Ahoy
15:02:38 <tbarron> ping xyang
15:02:43 <tbarron> ping markstur
15:02:47 <tbarron> ping toabctl
15:02:48 <xyang> hi
15:02:50 <tbarron> ping erlon
15:02:54 <tbarron> piing tpsilva
15:02:57 <xyang> so early
15:02:58 <tbarron> ping amito
15:03:05 <tbarron> xyang: so late :)
15:03:17 <tbarron> I'm in Rome and it's time for an apertiv.
15:03:25 <tbarron> Hi all, let's get going.
15:03:38 <amito> Hey
15:04:08 <tbarron> #chair bswartz
15:04:09 <openstack> Current chairs: bswartz tbarron
15:04:15 <amito> tbarron: Oh in Rome already
15:04:15 <tbarron> just in case, and also
15:04:28 <tbarron> bswartz will be picking up chair part way through
15:04:36 <tbarron> I's even later for amito :)
15:04:45 <tbarron> Hello all!
15:04:50 <tbarron> #topic announcements
15:05:00 <tbarron> Summit is next week.
15:05:26 <tbarron> I think ganso, erlon, tpsilva, amito, vkmc, and myself will be there.
15:05:30 <tbarron> anyone else?
15:05:49 <tbarron> We have a pretty good set of sessions involving manila topics:
15:06:04 <tbarron> #link https://www.openstack.org/summit/berlin-2018/summit-schedule/global-search?t=manila
15:06:16 <tbarron> also one on Ceph day (monday)
15:06:41 <tbarron> #link https://ceph.com/cephdays/ceph-day-berlin/
15:07:02 <tbarron> amito may be arranging a  get together Tuesday night
15:07:19 <tbarron> amito: I note that cinder is thinking the same way if we want to see if we can piggy-back
15:07:30 <tbarron> Anything else on summit?
15:07:34 <amito> It seems like there's a RH party on Tuesday
15:07:44 <tbarron> amito: Monday
15:07:58 <vkmc> both
15:07:59 <amito> RH+Trilio
15:08:18 <tbarron> amito: ah, that's a difft thing.  We don't do manila bakup atm.
15:08:18 <amito> Maybe Wednesday and move Cinder as well?
15:08:30 <amito> lol
15:08:37 <tbarron> amito: Wednesday is pub crawl
15:08:42 <amito> Oh
15:08:50 <tbarron> amito: let's take it to #openstack-manila
15:09:01 <amito> Where do I sign up? (for the pub crawl, lol)
15:09:08 <tbarron> sounds like trilio and RH want inifinidat for cinder ...
15:09:22 <tbarron> amito: there's a link I think off the main summit page
15:09:30 <tbarron> amito: ping me if you don't find it
15:09:38 <amito> I'll check that, thanks tbarron
15:09:52 <tbarron> OK, next question.
15:09:55 <amito> So we'll have the dinner on Tue.
15:09:58 <tbarron> Do we meet next week?
15:10:27 <bswartz> I can meet
15:10:30 <tbarron> amito: just get Trilio to invite us all and we can grab a corner table :) not many of us.
15:10:35 <tbarron> I can too.
15:10:38 <bswartz> I'm pretty sure 2 weeks from now most of us cannot
15:10:53 <bswartz> So if we cancel next week we'll be going 3 weeks between meetings
15:11:00 <tbarron> And as I think we'll see ^^ and that we have unfinished spec review business.
15:11:12 <tbarron> OK, let's meet.
15:11:23 <tbarron> Anyone who can not attend next week?
15:12:05 <gouthamr> i can, virtual-beer'o'clock same time next week
15:12:14 <tbarron> #agreed we will meet next week as usual even though it's summit week
15:12:22 <tbarron> #topic spec reviews
15:12:36 <bswartz> gouthamr: beer-o-clock is 7am for you?
15:12:44 <tbarron> So it's spec deadline but we're really just now in the middle of all these reviews!
15:12:53 <gouthamr> bswartz: it can be if we want it to be :D
15:12:59 <tbarron> And I think the reviews are making progress.
15:13:11 <tbarron> But we started reviewing late -- again.
15:13:36 <tbarron> I honestly think there is too much work associated with these reveiws to complete it all this cycle but --
15:13:49 <tbarron> whether I'm right about that or not -- it seems a shame
15:14:07 <tbarron> to lose momentum and not get the design work that is ongoing completed.
15:14:15 <tbarron> There, that's what I have to say abou this.
15:14:23 <tbarron> Thoughts from others?
15:14:37 <ganso> tbarron: I agree that reviews should have started earlier
15:14:57 <bswartz> Merging a spec isn't a guarantee that we'll merge a feature in the same release
15:15:19 <tbarron> And some of them (not pointing at you ganso) should have been posted earlier, but they were posted by notable slackers :)
15:15:20 <bswartz> But not merging a spec is a guarantee we won't merge a feature
15:15:27 * tbarron is messing around obviously
15:15:38 <tbarron> bswartz: ++
15:16:07 <tbarron> So I want to propose that we extend the deadline until 29 November.
15:16:17 <bswartz> Blanket extension?
15:16:26 <tbarron> Some of us will be at summit next week and will have limited bandwidth.
15:16:42 <tbarron> bswartz: not for any new submissions, but for the four submitted.
15:16:58 <bswartz> Right, okay
15:17:02 <gouthamr> ~erm, notable slacker would like to add one more~
15:17:05 <tbarron> Because I think they are worth thinking about collectively and seeing if we can get agreement on them.
15:17:23 <tbarron> gouthamr: you already have two if you refresh the agenda
15:17:33 <tbarron> gouthamr: do you have three in mind?
15:17:45 <gouthamr> oh, nope, just two
15:18:09 <tbarron> gouthamr: it's only becasuse I honestly think that both of those you proposed are things that we
15:18:31 <tbarron> should think about collectively now, while we're doing it, that I'm not being a meanie about them.
15:19:00 <tbarron> No guarantee that I'll agree with them.  I haven't had time to study much more than the problem statement.
15:19:18 <gouthamr> sure, thank you for including the link, i will keep my elevator pitch ready :)
15:19:24 <tbarron> But I agree with the problem staements and think that if in the month of Novmember we could as a tem
15:19:29 <tbarron> as a team
15:19:56 <tbarron> get good design in place for the big residual DHSS=True issues and
15:20:29 <tbarron> for the manila AZ usage and storage-protocol/capability discovery
15:20:47 <tbarron> we'd actually have accomplished some good work in this spec cycle.
15:21:01 <tbarron> Contrary views?
15:21:45 <bswartz> You're arguing for getting all 4 specs in shape in the next 3 weeks?
15:21:49 <tbarron> The 29 November date is a bit out but we have summit and US Thanksgiving (big football holidy) in that window.
15:21:55 <tbarron> bswartz: Yes.
15:22:09 <bswartz> Is there any priority order we could put on them
15:22:10 <bswartz> ?
15:22:16 <tbarron> good question.
15:22:18 <bswartz> If push comes to shove and we can't get them all done?
15:22:58 <tbarron> ganso's forst priority was manage/unmanage for share servers, enabling that capability for DHSS=True
15:23:01 <tbarron> first
15:23:21 <ganso> tbarron: yes
15:23:27 <tbarron> I think that ones very close (even though it's not my personal first priotiry)
15:23:57 <tbarron> bswartz already +2ed it, then ganso had to tweak because I -1ed but
15:24:05 <tbarron> I'm pretty much OK with it now.
15:24:12 <bswartz> I'm inclined to follow the priorities of the spec authors, because they'll also be implementing in their own priority orders (presumably)
15:24:28 <tbarron> gouthamr: vkmc: can you look at this one this week?
15:24:32 <tbarron> bswartz: +1
15:24:37 <gouthamr> tbarron: ack
15:24:43 <vkmc> tbarron, sure thing
15:25:11 <tbarron> ok then his next prio was the tough one, share networks spanning subnets.
15:25:24 <tbarron> my next prio would be gouthamr's AZ spec
15:25:30 <tbarron> but it came in late
15:25:35 <bswartz> #link https://review.openstack.org/#/c/616123
15:26:02 <tbarron> bswartz: I'm going to turn over chair to you since you met with ganso and gouthmr on that one
15:26:10 <tbarron> And I have to drop the meeting soon.
15:26:24 <tbarron> I will read the log (bouncer and recorded) but
15:26:25 <bswartz> okay should we go over all the specs in teh agenda?
15:26:37 <bswartz> Or start with ^
15:26:42 <tbarron> bswartz: probably set priorities first as you suggested, then
15:26:55 <tbarron> maybe dig into the tough one given your meeting?
15:26:56 * bswartz sees 5 links
15:27:02 <bswartz> Is it 4 or 5 specs?
15:27:15 <tbarron> two are for the extend share networks work
15:27:22 <gouthamr> bswartz: i'm writing another one which makes it 5
15:27:23 <bswartz> Wait that makes 6
15:27:27 <tbarron> one is gouthamr's orignal from newton moved over
15:27:33 <bswartz> Okay
15:27:42 <bswartz> Let's just go in order, and try to go fast
15:27:50 <bswartz> Enjoy your vacation tbarron!
15:27:59 * bswartz salutes
15:28:06 <bswartz> #topic Manage/Unmanage Share Servers
15:28:15 <bswartz> #link  https://review.openstack.org/#/c/607342
15:28:24 <tbarron> bswartz: thank you!
15:28:44 <vkmc> babai tbarron :)
15:28:52 <bswartz> Okay as mentioned before, I'm okay with this as of PS11
15:28:57 <gouthamr> tbarron: ++ enjoy the wine/food + more wine
15:29:02 <vkmc> enjoy
15:29:09 <ganso> tbarron: and don't forget italian pasta
15:29:15 <bswartz> There's been a lot of discussion since my +2
15:29:21 <amito> enjoy tbarron!
15:29:28 <bswartz> Anything left to discuss on this or do we just need final reviews?
15:29:48 <ganso> bswartz: I think we only need the final reviews, no major concerns raised since the first revision
15:29:57 <bswartz> going once...
15:30:05 <gouthamr> nope, will take a look
15:30:07 <bswartz> twice...
15:30:20 <bswartz> #topic Share Networks Span Subnets
15:30:25 <bswartz> #link https://review.openstack.org/#/c/391805/
15:30:31 <bswartz> #link https://review.openstack.org/#/c/615947/1
15:30:45 <bswartz> So this is a topic that goes back years
15:31:02 <bswartz> It's a thorny topic that we've spend a lot of time discussion in front of whiteboards at various venues
15:31:15 <bswartz> Gouthamr proposed a spec long ago that we agreed to, but it was never implemented
15:31:51 <bswartz> Ganso wants to do that work, plus more, to complete the picture
15:32:24 <bswartz> I like the idea of having a comprehensive design, even if the implementation has to be phased over longer than 1 release
15:32:54 <bswartz> Because if we leave parts of the design undecided, it creates risk that the implementation might go in the wrong direction
15:33:00 <ganso> bswartz: correction, for now my commitment it to do just that work. I can propose the "plus more", but I will not commit
15:33:15 <ganso> bswartz: I will not commit *at this titme
15:33:20 <ganso> s/titme/time
15:33:37 <bswartz> ganso: I just meant you were volunteering to complete the design portion
15:33:48 <ganso> bswartz: oh ok
15:33:59 <bswartz> I really like where this design seems to have ended up, even though it probably a ton of work to make it reality
15:34:21 <bswartz> ganso: can you outline the relationship between the 2 specs so everyone understand why it's split?
15:34:29 <ganso> bswartz: sure
15:34:34 <ganso> so in summary
15:34:58 <ganso> When we introduced IPv6, we noticed that we weren't able to have share servers serving export locations in IPv4 and IPv6 at the same time
15:35:47 <ganso> it is not a very good user experience to not be able to serve your shares in IPv4+IPv6, if the hosts that could consume those shares can have IPv4+IPv6
15:36:28 <ganso> to accomplish that, we would need to allow a share network to be assigned to at least 2 subnets
15:36:49 <ganso> and if we can do 2, we could even do more, but that's another bonus use case
15:37:27 <ganso> in the first design of the Replication in DHSS=True spec, we don't change that share network subnet limitation, so the IPv4+IPv6 problem remains
15:38:05 <ganso> but we make changes to the share network APIs that could conflict with the work to improve the IPv4+IPv6 and we desire to do in the future
15:38:12 <bswartz> Yeah we're trying to address multiple problems with these specs
15:38:25 <bswartz> Cleaning up all the loose ends from the original share-networks design
15:39:28 <ganso> so our current plan of attack is to design what we plan to work on in the future to solve the IPv4+IPv6 share servers situation, so we can safely implement Replication in DHSS=True knowing we won't have problems in the future
15:40:23 <ganso> we've agreed to a phased implementation, but we need to design these phases first, before start implementing them
15:41:01 <ganso> <end of summary>
15:41:33 <bswartz> So IMO this spec should answer all the questions regarding how share networks will eventually work
15:41:54 <bswartz> Including how we'll get dual IPv4/IPv6 support for share servers
15:42:02 <bswartz> How share servers will support replication
15:42:15 <bswartz> And how share servers will support multiple subnets
15:42:35 <bswartz> In particular, changes to the driver interface are needed
15:43:03 <bswartz> And there will need to be a capability/versioning exercise similar to what happened with access rules
15:43:37 <bswartz> ganso has suggested that for this release the only thing we might get is the replication (multi-AZ) support which could be done without changing the driver interface yet
15:44:12 <bswartz> But I'd still like to have written down where we eventually plan to go with share networks so we can make sure that the stein phase of this work doesn't conflict with that
15:44:30 <bswartz> Seem reasonable?
15:45:15 * bswartz looks at clock
15:45:17 <ganso> bswartz: yep
15:45:20 <bswartz> Better keep moving
15:45:32 <bswartz> #topic Create share from snapshots in another pool or back end
15:45:37 <bswartz> #link https://review.openstack.org/#/c/609537/
15:45:52 <ganso> ok that is a good one
15:46:13 <ganso> most of the feedback have been incorporated into the spec
15:46:15 <ganso> except one
15:46:25 <ganso> "I would like to see a mechanism to tell Manila, on a per-share-create basis, that it should ignore the location of the snapshot when scheduling the new share. This would be a scheduler hint, which the scheduler could safely ignore, but would result in dramatically better load balancing in the situation where a large number of shares were created from the same snapshot, and the end user had some knowledge of the
15:46:25 <ganso> layout of the storage backends."
15:46:29 <bswartz> This is a solution to the practical issue we have where if you always create shares from snapshots, your shares all get crowded on the same pool
15:46:59 <bswartz> ganso: what is the likelihood this gets implemented in Stein?
15:47:18 <ganso> bswartz: if only this and manage/unmanage merges, it is very likely
15:47:44 <bswartz> okay but which is higher priority in your mind?
15:48:01 <ganso> 1) manage/unmanage, 2) replication, 3) this one
15:48:07 <ganso> if only 1) and 3) merge
15:48:10 <ganso> then they will be implemented
15:48:13 <ganso> if all 3 merge
15:48:15 <bswartz> Okay
15:48:23 <ganso> #3 will most likely not be implemented in Stein
15:48:30 <bswartz> This one seems like less of a bear to implement for sure
15:48:40 <bswartz> But there are some complex issues to consider
15:48:56 <ganso> yea it doesn't seem to be a lot of code
15:49:17 <bswartz> gouthamr: have you read this one?
15:49:28 <ganso> so in summary, the user will be able to choose the AZ to create a share from snapshot in
15:49:33 <bswartz> It might be only tbarron and I have any opinions so far
15:49:41 <gouthamr> bswartz: not yet, but familiar with the idea/motivation - so will do
15:49:48 <amito> Might be a stupid question but... how is create share in another pool different than manage share when we're talking about two different backends?
15:49:54 <ganso> and there will be capabilities that allow manila to determine a compatible destination host
15:50:07 <bswartz> gouthamr: the issue that we need to sort out is how the cross-pool scheduling gets triggered
15:50:13 <amito> (I read through the spec)
15:50:32 <bswartz> ganso and I were debating the extent to which end-users should be empowered to make that happen
15:50:47 <bswartz> because you can make the argument that only the administrator should know or care about this particular problem
15:50:50 <ganso> and, even if the same AZ is chosen, there is a balancing rule (suggested by tbarron) to allow the admin to have some control on whether a share is created in the same pool with COW efficiency, or in another backend, for the purpose of capacity balancing
15:50:52 <gouthamr> bswartz: ah, and the current direction is "AZ"?
15:51:35 <bswartz> The problem from my perspective is that, to make an ideal decision, you need to know if a particular snapshot will only be cloned a small number of times, or whether it will be cloned a large number of times
15:51:51 <bswartz> Only an end user can understand the usage patterns of the snapshots
15:52:16 <bswartz> If you err on the side of keeping things on the same pool, you have the same problem we have today
15:52:43 <ganso> bswartz suggests that a parameter in the "create share API" should be added to force load balancing. I don't like this idea because this parameter doesn't make sense for users that aren't supposed to know of backends (unless they are also admins, which is not generally the case).
15:52:49 <gouthamr> how many times are you cloning this snapshot - can we infer that without asking the question?
15:52:50 <bswartz> But if you're too aggressive about scheduling new shares to other pools you could end up needlessly lowering storage efficiency and wasting time copying data
15:53:49 <bswartz> My intuition is that the the end user should be empowered to give hints to the scheduler, to cover the cases when the end user and the admin are actually the same person or at least people who talk to eachother
15:54:26 <bswartz> But such hints would not make sense in a more traditional public cloud deployment
15:54:48 <ganso> gouthamr: if the user intends to send 100 requests, and send them individually, it is harder for us to decide whether to spread it or not. Although, if the user could mark the snapshot with a flag, we could know it. But why should a user mark it if it is not supposed to know or care about backends and balancing?
15:54:54 <bswartz> My solution there is to call them hints so it's clear that the system it welcome to ignore them
15:55:14 <gouthamr> ganso: +1
15:55:50 <ganso> bswartz: regarding efficiency, it could go both ways
15:56:09 <ganso> bswartz: if you have COW, then creating a 100 new shares from a snapshot is going to consume 0 bytes (in theory)
15:56:17 <bswartz> Anyways I'd like to answer that question before this spec merges
15:56:23 <bswartz> That's the crux of the issue
15:56:32 <bswartz> Let's move on though, I have a hard stop in 4 minutes
15:56:40 <bswartz> #topic Storage Availability Zone Improvements
15:56:42 <ganso> bswartz: if we spread them, we would consume ~50x the size of the original share
15:56:46 <bswartz> #link https://review.openstack.org/#/c/616123/
15:57:11 <bswartz> This is new as of yesterday
15:57:15 <bswartz> I started to take a look this morning
15:57:16 <gouthamr> bswartz: you're thought is to allow users to say `create --snapshot mysnap --okay-with-slow-creation` or `create --snapshot mysnap --quickly-if-possible`
15:57:24 <gouthamr> bswartz: don't yet
15:57:28 <bswartz> gouthamr: yes
15:58:10 * gouthamr reddit-shames himself with you're/your
15:58:23 <gouthamr> okay, sorry - i can quickly cover these two specs
15:58:25 <bswartz> Anything to say about this? I'm not done reviewing
15:58:25 <ganso> gouthamr: still, how about the COW concerns? slow and fast isn't the only concern
15:58:49 <bswartz> gouthamr and your other spec has a workflow -1
15:58:51 <gouthamr> ganso: hmm, lemme read previous comments and your spec
15:58:59 <gouthamr> bswartz: yes, it is not complete
15:59:07 <bswartz> still aiming for stein though?
15:59:11 <gouthamr> yep
15:59:19 * bswartz waves the shame stick at gouthamr
15:59:23 <gouthamr> :)
15:59:26 <bswartz> today was supposed to be the deadline!
15:59:36 <bswartz> Please get it in shape and ping people for reviews
15:59:53 <gouthamr> sure thing
15:59:54 <bswartz> that's all the time we have I'm afraid
16:00:05 <bswartz> We will meet next week, despite people being at summit
16:00:14 <bswartz> hopefully to make more progress on these specs
16:00:27 <bswartz> enjoy berlin
16:00:38 <bswartz> I'll miss you guys
16:00:42 <bswartz> #endmeeting