21:00:25 <nikhil> #startmeeting quotas-wg
21:00:26 <openstack> Meeting started Mon Mar 14 21:00:25 2016 UTC and is due to finish in 60 minutes.  The chair is nikhil. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:30 <openstack> The meeting name has been set to 'quotas_wg'
21:00:36 <nikhil> Courtesy quotas-wg meeting reminder: nikhil, vilobhmm, DuncanT, mc_nair, ninag
21:00:48 <ninag> o/
21:01:10 <vilobhmm111> o/
21:01:13 <nikhil> Hello everyone!
21:01:37 <vilobhmm111> hi nikhil
21:01:38 <nikhil> How's the DST treating you (whoever it affects)? :)
21:02:07 <vilobhmm111> haha may be the first day so didn;t realize it
21:02:08 <DuncanT> Hi
21:02:33 <mc_nair> hey
21:02:38 <nikhil> I know I thought to have driven 4.5 hours from CLT-Bburg but then realized it was Sunday where DST starts so it was merely 3.5 hours starting at 12am
21:02:49 <vilobhmm111> okies
21:03:05 <nikhil> fun DST experience!
21:03:07 <nikhil> anyways, I added a few items to the agenda
21:03:12 * thingee lurks
21:03:24 <nikhil> now that we've a few folks online and lurking :D
21:03:30 * nikhil pokes thingee
21:03:36 <nikhil> #topic agenda
21:03:39 <nikhil> #link https://etherpad.openstack.org/p/quotas-wg-meeting-agenda
21:03:53 <vilobhmm111> so about https://review.openstack.org/#/c/284454/ i have updated the patch with few comments…thinking more on the need for reservation..spoke with andrew laski on nova channel and i agree we can get rid of reservations overall
21:04:49 <nikhil> vilobhmm111: let's get to that in a just a bit. sorry, I had a outline for today's meeting predetermined ..
21:05:29 <nikhil> Sorry, I wasn't at the last meeting but saw the logs and they look pretty good. We are still pending on the discussion of service vs lib
21:05:37 <vilobhmm111> nikhil : sure..
21:05:57 <nikhil> I had a great F2F convo with sdague on that point in NYC last week.
21:06:15 <DuncanT> Can you explain a bit about how you get rid of reservations on an async system?
21:06:57 <ninag> How about quota as a micro service that each project can run?
21:07:22 <vilobhmm111> DuncanT : once nikhil is done will answer to your que
21:07:32 <nikhil> awesome, thanks vilobhmm111
21:07:49 <DuncanT> Thanks
21:08:22 <nikhil> SO, let's recap last meeting on the review front, these questions today and then go with action items follow up. I've kept the feedback discussion for the open discussion topic given the amount og time weget today.
21:08:49 <nikhil> #topic recap from last meeting & review quotas
21:09:00 <nikhil> #link http://eavesdrop.openstack.org/irclogs/%23openstack-meeting-cp/%23openstack-meeting-cp.2016-03-07.log.html#t2016-03-07T21:13:31
21:09:44 <nikhil> That's the ad-hoc meeting we'd but we don't have official logs unfortunately as we ran over in the on air hangout last week.
21:10:07 <nikhil> vilobhmm111: floor is all yours on the follow up and the above questions
21:10:57 <vilobhmm111> nikhil :ok
21:11:44 <vilobhmm111> so if we think about nova flow might look like irc://irc.freenode.net:6667/#1. nova boot cli irc://irc.freenode.net:6667/#2. commit the irc://irc.freenode.net:6667/#of resource depending on request (source of truth being the quota.usages table) irc://irc.freenode.net:6667/#3. go to scheduling pick up a host to boot the instance on irc://irc.freenode.net:6667/#4. on nova-api side claim the commited resources in ir
21:11:59 <vilobhmm111> a. nova boot cli
21:12:22 <vilobhmm111> b. commit resource depending on request (source of truth being the quota.usages table)
21:12:41 <vilobhmm111> c. go to scheduling pick up a host to boot the instance
21:12:59 <vilobhmm111> d. on nova-api side claim the commited resources
21:13:47 <vilobhmm111> right now nova and all other projects have concept of reservation and then commit (or rollback)
21:14:18 <vilobhmm111> reservation is just a guarantee that (within a certain timeout) a particular req owns the resouce
21:14:28 <vilobhmm111> which is redundant imho
21:14:45 <vilobhmm111> why not just rely on quota_usages
21:15:06 <vilobhmm111> and make commit / rollback accordingly
21:15:17 <nikhil> vilobhmm111: can you please elaborate more?
21:15:23 <vilobhmm111> does the team see any possible pitfalls with this design
21:16:04 <nikhil> what is the difference in the race condition during the reservation vs quotas_usages ?
21:16:06 <vilobhmm111> a reservation id is created for every resource request
21:16:42 <mc_nair> vilobhmm111: is the redundant part that we're splitting up reserved/in-use, but anytime we check the free resources we need to take the total (in-use + reserved)?
21:17:01 * alaski peeks in
21:17:10 <vilobhmm111> alaski : hi
21:17:13 <alaski> mc_nair: that's a big part of it
21:17:39 <alaski> I was questioning the benefit of reservations vs just committing the usage optimistically up front
21:18:02 <nikhil> I see and that's makes me curious ..
21:18:22 <mc_nair> alaski: yea I was curious about that also.  We did something similar with the most recent Cinder nested quota changes, where we did the reservations of "allocated" values up front (took it out immediately and rolled back if needed)
21:18:37 <mc_nair> but we still had it using a reservation
21:18:45 <nikhil> we've this option of either undercommit using reservation or overcommit using usages?
21:18:58 <mc_nair> just not split into reserved/in-use
21:19:41 <alaski> mc_nair: gotcha. that can work, but I do wonder if we can take it further
21:19:56 <vilobhmm111> mc_nair : reserved/in_use/allocated all 3 needs to be considered
21:19:58 <vilobhmm111> imho
21:20:08 <vilobhmm111> alaski : ^^
21:20:17 <mc_nair> but the reservation-id part of it still seems useful, because otherwise how do you keep track of when to timeout a reservation and roll it back, or group updates to multiple project reservations into one unit that's rolled back / committed as one unit
21:20:50 <mc_nair> vilobhmm111: considered as in decide if we need them at all?
21:20:56 <alaski> vilobhmm111: the benefit of distinguishing reserved/in-use is what I'm trying to figure out
21:21:55 <alaski> mc_nair: rather than timing out a reservation I was thinking along the lines of having usage tied to the consuming thing, and if that thing doesn't exist or gets cleaned up then the usage can be cleaned up
21:22:12 <alaski> group updates is still a question mark
21:23:02 <alaski> another way to put it might be don't time out the reservation, time out the consuming entity
21:23:21 <mc_nair> alaski: ok, that's interesting.  I don't have my head wrapped around how that'd actually work yet, but I'm slightly more following :)
21:23:45 <vilobhmm111> alaski : you have an example for it
21:23:59 <vilobhmm111> will help to understand better what you want to convey here
21:24:06 <nikhil> ok guys, I think this needs a brainstorming session. While we do that, we need to move forward on some of the action items too and get the requirements from other projects that people might be itching to give as per last meeting logs.
21:24:21 <mc_nair> we'd want to consider the nested quota design for that carefully, cause currently we're doing the grouped reservations
21:24:25 <alaski> mc_nair: cool. I realize I'm raising more questions than proposing answers, but this seems like the right time to think a bit
21:24:27 <mc_nair> nikhil: sounds good
21:24:41 <mc_nair> alaski: makes sense
21:25:00 <vilobhmm111> nikhil : it surely does…alaski : +1
21:25:15 <nikhil> ninag: you can go first if you want
21:25:30 <nikhil> #info action items follow up below
21:25:39 <ninag> nikhil: in terms of the Auto Scaling for VMs?
21:25:47 <nikhil> ninag: yes, abvout to ask
21:25:56 <nikhil> did we get feedback from them yet?
21:26:13 <nikhil> (heat/autoscaling use cases drivers)
21:26:22 <ninag> Yes, I did talk with them..there are really no new requirements..since the policies are being built at thr PaaS layer
21:26:30 <DuncanT> So the thing about reservations is that they go (time out) if the create creates
21:26:49 <ninag> They are interested in APIs to query quotas and actuals
21:27:24 <nikhil> ninag: I see, so sketching out an API first would help more?
21:27:29 <ninag> Yeah
21:27:40 <nikhil> excellent direction
21:27:57 <nikhil> #action all: provide feedback on the API front for the quotas
21:28:16 <nikhil> of-course, this ties into our discussion about service vs lib
21:28:46 <nikhil> but let's comment a bit about our perspective so far on the spec
21:29:11 <nikhil> vilobhmm111: you seem to have action item on migration plan
21:29:33 <nikhil> not sure if you wanted to add anything to it today
21:29:42 <vilobhmm111> migration plan still brainstorming…will have an update by tommorow
21:29:48 <nikhil> okay, we can say no-op for now
21:29:58 <vilobhmm111> yep
21:30:10 <nikhil> anything else on the action item follow up?
21:30:32 <vilobhmm111> i was working on getting this need for reservation thing sorted out + addressing feedback + adding details about quota for PaaS related details.
21:31:01 <nikhil> vilobhmm111: ++
21:31:12 <nikhil> we'd have a quick open discussion, couple of small things
21:31:17 <nikhil> #topic open discussion
21:31:18 <vilobhmm111> few things which we should clarify imho #1. what this entity should be (lib/service) #2. reservtion or no reservation
21:31:39 <nikhil> #starvote extend the meeting length to 1 hours? yes, no, maybe
21:31:49 <mc_nair> yes
21:31:50 <ninag> yes
21:31:52 <nikhil> vilobhmm111: exactly
21:32:01 <nikhil> #vote yes
21:32:19 <vilobhmm111> #vote yes
21:32:26 <nikhil> anyone else?
21:32:45 <nikhil> #endvote
21:33:28 <nikhil> okay, I misspelled startvote hence bot did not pick up
21:33:53 <nikhil> mc_nair: vilobhmm111 ninag should we continue for the rest of the hour today then?
21:34:06 <ninag> Sure
21:34:09 <mc_nair> nikhil: I'm fine with that
21:34:20 <mc_nair> Are there any design sessions planned for quotas in Austin summit?
21:34:25 <nikhil> silence would mean yes (for all)
21:34:46 <nikhil> mc_nair: not yet, once TC gets elected they will review the x-prj topics
21:35:05 <nikhil> we'd get the link to proposal soon-ish I think (usually up in the wiki)
21:35:33 <nikhil> #info we are continuing the meeting to the full hour as there were no objections raised today's meeting
21:37:06 <nikhil> A quick update to everyone, we'd a great talk from vilobhmm111 on quotas and scheduling as a hangout on air last week. It's got ton of views already so, I get a feeling that quotas is super important to openstack realm just that not everyone can/wants to impl.
21:37:16 <nikhil> #link https://www.youtube.com/watch?v=PTFww2RH21c
21:37:47 <nikhil> All in all, I think we're in the right time for impl quotas.
21:37:53 <nikhil> Now about the right place:
21:37:56 <nikhil> 1. service
21:37:58 <nikhil> 2. library
21:38:16 <ninag> Microservice?
21:38:38 <vilobhmm11> why ninag
21:38:40 <nikhil> My discussion with Sean gave me a understanding that it being a library that defines a bare-bone quotas DB table for all projects to source in should be a good starting point.
21:38:48 <nikhil> ok, let's add that
21:38:55 <nikhil> 3. Microservice
21:39:37 <ninag> It is not a full fledged service, and given that we want to define behavior with a backing db
21:40:20 <ninag> Seemed like a good fit for a micro service..
21:40:28 <ninag> but am open..wanted to put it out there..
21:40:32 <nikhil> ninag: what would be the critical defining point for it being called a micro service?
21:41:02 <ninag> In my mind, we are not providing quota as a service..
21:41:24 <ninag> it still exists in the context of another service (nova etc)
21:41:43 <nikhil> haha, then so is glance :D
21:41:46 <ninag> but is more than a library..
21:41:58 <ninag> :)
21:42:08 <nikhil> ++
21:42:17 <nikhil> I got the similar feedback
21:42:21 <ninag> Lets talk about what you were proposing...I think it is pretty similar
21:42:31 <nikhil> but not necessarily defining it as a separate service
21:42:34 <nikhil> ninag: right on
21:43:09 <nikhil> (I wanted to let you-ninag go first on micro service as this should open a new dimension of thinking separate from what we've already discussed on others)
21:43:17 <nikhil> On the feedback:
21:43:27 <vilobhmm11> nikhil, ninag : making it a service adss addional dependency about deployments issues etc as opposed to it being a lib
21:44:02 <vilobhmm11> micros-service we can think more
21:44:39 <nikhil> If we've a library that defines bare-bone structure for the quotas and has a backing of the tables that all projects should have on quotas. Similar to what oslo.db does on the models for DB tables.
21:45:16 <vilobhmm11> imho a lib with a db bare bones seems reasonable for what its purpose is
21:45:23 <nikhil> Sourcing into the project tree and then implementing custom logic seems like a good plausibility.
21:45:35 <vilobhmm11> +1
21:46:00 <nikhil> OF course, DB migrations would 'a' if not 'the' pain point
21:46:26 <vilobhmm11> but that you would have irrespective of it being a service or a lib
21:46:36 <vilobhmm11> DB migrations
21:46:41 <ninag> yup
21:47:03 <nikhil> well sure, but not necessarily so distributed
21:47:34 <vilobhmm11> what do you mean by that nikhil : ^^
21:47:52 <nikhil> it's easier to source out than source in information, imho
21:48:21 <nikhil> my presumptions being the DBs would be owned by this service and not by individual projects
21:48:44 <nikhil> and the migration plans would be part and release of this service release cycle
21:49:28 <ninag> nikhil: not sure I get you..even in the library model, i thought you talked of its being backed by tables
21:49:30 <nikhil> for example, if nova and cinder were two projects using this new service
21:50:14 <nikhil> the implementation for nova & cinder quotas and the increments of DB migration will be done as a  part of this service
21:50:48 <DuncanT> I'd suggest you're going to want to define a new table with a name not used by any project then drain the records across over time... you can't just copy data around during migrations anymore, due to live upgrades
21:51:42 <nikhil> yeah, and the idea is that for a library it will be a DB model and not DB table provided by this library
21:51:56 <ninag> Ah ok
21:52:11 <nikhil> that way when to do the (live) upgrade is dependent on the prj
21:52:29 <nikhil> so, I'm inclining towards a lib of that sorts
21:53:31 <nikhil> code that maps to a generic table that defines: Resource R has Usage S and is Nested (N or not S) that has a limit L etc.
21:54:06 <vilobhmm11> lib should provide quota engine and the db model only
21:54:08 <nikhil> R becomes instances for nova and volumes for Cinder
21:54:52 <nikhil> (per se)
21:55:08 <nikhil> Anyways, we've 5 mins and that was the feedback I'd.
21:55:31 <nikhil> DuncanT: had a good question earlier that we'd prolly get to before next week's meeting and possibly more brainstorming.
21:55:40 <vilobhmm11> all this the grammer "Resource R has Usage S and is Nested (N or not S) that has a limit L etc." can be taken care by the quota engine which can be the main component of the quota lib(entity)
21:56:01 <vilobhmm11> ok so nikhil i think it will be better to start an e-mail thread
21:56:12 <vilobhmm11> what do other think ?
21:56:16 <nikhil> Also, I wanted to ask quickly: Would people like to have a video conference session prior to summit? (~3 weeks from now)
21:56:37 <nikhil> vilobhmm11: that's a good idea. Let's begin with that.
21:57:00 <ninag> vilobhmn11, nikhil +1 to both
21:57:29 <nikhil> Also, let's try to bring this spec up in next CPL meeting. I think we'd had good structure by then.
21:57:46 <vilobhmm11> nikhil : yup
21:58:39 <nikhil> Just so that I ack this question and get a sense if we're waiting for more brainstorm or not:
21:58:47 <nikhil> "So the thing about reservations is that they go (time out) if the create creates - DuncanT"
21:59:03 <DuncanT> If the create crashes
21:59:08 <DuncanT> Is what I meant
21:59:13 <nikhil> any takers?
21:59:58 <vilobhmm11> need to brainstorm this more
22:00:12 <nikhil> DuncanT: IMO, that's what reservation wants to achieve but may not be what it does atm
22:00:48 <nikhil> So, I think reservation should help for non-flat quotas -- something that folks have been quite worries about. Correct?
22:00:53 <DuncanT> nikhil: That wouldn't entirely surprise me
22:01:10 <DuncanT> What are non-flat quotas?
22:01:22 <vilobhmm11> non-heirarchical
22:01:31 <vilobhmm11> is what you meant nikhil right
22:01:33 <vilobhmm11> by flat
22:02:04 <nikhil> vilobhmm11: eh, I think we need to get the terminology right there.
22:02:27 <nikhil> I meant hierarchal flat (non-floating).
22:02:34 <nikhil> sorry, and we're out of time...
22:02:42 <nikhil> Thanks all for joining!
22:02:49 <nikhil> #endmeeting