17:01:19 <vilobhmm11> #startmeeting quotas-wg
17:01:20 <openstack> Meeting started Tue May 10 17:01:19 2016 UTC and is due to finish in 60 minutes.  The chair is vilobhmm11. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:01:22 <vilobhmm11> hi all
17:01:22 <nikhil> ola
17:01:24 <openstack> The meeting name has been set to 'quotas_wg'
17:01:25 <nikhil> o/
17:01:32 <vilobhmm11> hi nikhil :)
17:01:33 <piet> o/
17:01:49 <flwang1> o/
17:01:50 <amrith> ./
17:02:08 <nikhil> #info agenda - https://etherpad.openstack.org/p/quotas-wg-meeting-agenda
17:02:14 <amrith> DuncanT was looking for meeting time a short while ago
17:02:26 <vilobhmm11> welcome everyone ..lets start
17:02:40 <vilobhmm11> #topic sync on status
17:03:03 <vilobhmm11> 1. last week we created a launchpad page for delimiter
17:03:04 <vilobhmm11> https://launchpad.net/delimiter
17:03:21 <vilobhmm11> 2. We also have source code here
17:03:22 <vilobhmm11> https://github.com/vilobhmm/delimiter
17:03:28 * DuncanT waves
17:04:01 <vilobhmm11> setting up new project in openstack takes some time so meanwhile code will reside here https://github.com/vilobhmm/delimiter and then we can migrate under the openstack repo
17:04:13 <vilobhmm11> itisha is working on the effort to set things up for delimiter
17:05:00 <vilobhmm11> #3. a summary of decisions made in the design summit was sent out to the ML
17:05:09 <nikhil> #info itisha is working on the effort to set things up for delimiter
17:05:29 <vilobhmm11> http://osdir.com/ml/openstack-dev/2016-05/msg00393.html
17:05:44 <vilobhmm11> link for the summary sent to ML
17:06:18 <vilobhmm11> that were the updates from last week…so spending time to get things going by doing some ground work
17:06:33 <vilobhmm11> anyone has any questions, suggestions till now ?
17:07:13 <vilobhmm11> for every meetings agenda we maintain this etherpad https://etherpad.openstack.org/p/quotas-wg-meeting-agenda
17:07:32 <vilobhmm11> we have detailed out today's agenda here https://etherpad.openstack.org/p/quotas-wg-meeting-agenda please check
17:07:48 <vilobhmm11> we will directly move on to #6 now as its kind of high priority
17:07:59 <vilobhmm11> #agenda Discussion on features for Newton
17:08:19 <vilobhmm11> so I have few suggestions ; need feedback from the team
17:08:41 <vilobhmm11> For Newton we should focus on :-
17:08:45 <vilobhmm11> #1. Enforce Quota : Get inputs from respective projects and enforce quota.
17:08:56 <vilobhmm11> which involves
17:08:59 <vilobhmm11> #1.1. Design Quota Engine
17:09:09 <vilobhmm11> #1.2. Basic Quota Engine Objects
17:09:14 <flwang1> do we need to talk about #5?
17:09:49 <vilobhmm11> flwang : sure we can but it will be a subpart in #6 imho
17:10:09 <vilobhmm11> since its more of a implementation detail and we will discuss this as part of features
17:10:17 <vilobhmm11> does that makes sense ?
17:10:39 <vilobhmm11> #1.3 Unit Tests
17:10:40 <flwang1> vilobhmm11: ok, cool
17:10:46 <vilobhmm11> #1.4  Functional Tests
17:10:53 <vilobhmm11> #1.5 Document changes
17:11:21 <vilobhmm11> #1.6 Seperate driver for different project models or single driver to handle all the different project models (flat, nested)
17:11:46 <vilobhmm11> we can discuss #1.6 first since we have seen interest in that
17:12:00 <nikhil> no, let's go one by one please
17:12:05 <vilobhmm11> ok
17:12:16 <nikhil> what is the purpose of this section?
17:12:28 <nikhil> are we trying to generate action items out of it?
17:12:36 <nikhil> or are we going to discuss use cases?
17:13:39 <vilobhmm11> both…for few you don't need use-cases for example designing quota objects..but for few you do need…1.6
17:13:54 <vilobhmm11> we would want to have some action items to get things done
17:14:03 <vilobhmm11> nikhil : ^^
17:14:13 <nikhil> ok, then I would encourage to split these two things
17:14:21 <vilobhmm11> ok
17:14:22 <nikhil> mostly as we've a lot of interest in this work
17:14:34 <nikhil> while one sub-team works on foundational stuff
17:14:50 <nikhil> others can focus on seeing the feasiblity of adoption
17:15:17 <nikhil> just trying to get a sense of whether the bare minimul will work for any one project?
17:15:37 <vilobhmm11> so first of all thats what i wanted to ask; does these features makes sense; is it too much; too less;  ?
17:15:39 <nikhil> basically, do we need to start discussing if any project wants to adopt it in newton?
17:15:55 <vilobhmm11> glance :)
17:16:06 <vilobhmm11> ha
17:16:12 <nikhil> ok, so for glance I think we've 3 people here :)
17:16:21 <vilobhmm11> cool
17:16:21 <nikhil> flwang1: sudipto and me
17:16:23 <DuncanT> I hope to do a POC patch for cinder using it, though I don't expect it to get merged in Newton
17:16:38 <nikhil> we would like to figure out what we can do in glance and in here to get started
17:16:50 <vilobhmm11> DuncanT : good to know that..that will be nice
17:17:00 <nikhil> that way we don't step on each others toes and also the progress of lib isn't halted
17:17:08 <nikhil> (that's the requirements part)
17:17:10 <DuncanT> My employer seems determined to fill my time with other things though, so it is a background task
17:17:39 <DuncanT> Mostly I just want to make sure the library isn't shaping up in a way that makes it impossible to use in cinder
17:17:55 <nikhil> the more important question is how can be craft the foundational work
17:17:56 <vilobhmm11> so nikhil, sudipto, flwang1 : what are your expectations; requirements from the lib for newton (that can be achieved in newton timeframe)?
17:18:10 <vilobhmm11> let me frame it in different way
17:18:14 <nikhil> I think DuncanT and I are saying the same thing
17:18:17 <vilobhmm11> the earlier question
17:18:18 <vilobhmm11> ok
17:18:56 <vilobhmm11> we would like to keep it more generic and not specific to a project as we discussed in design summit
17:18:57 <flwang1> i'm thinking if we(zaqar) can adopt as well for limiting queue number
17:19:29 <vilobhmm11> flwang1 : thats nice
17:19:50 <vilobhmm11> nikhil, DuncanT, others : because if you see the api's exposed by library
17:20:17 <vilobhmm11> check_quota(usage_details, objects/tables to operate upon, generation-id)
17:20:35 <vilobhmm11> which is quite generic and most of the things are provided by respective projects
17:20:41 <nikhil> ok
17:21:47 <DuncanT> I still a little fuzzy on the generation stuff, but I will read up some more before asking questions - I'll be clear by next week, one way or the other.
17:22:06 <DuncanT> Other than that, the design looks ok at a first look
17:22:20 <nikhil> in a gist, gen_id should work for you unless you want reservation
17:22:21 <amrith> DuncanT, drop me a note if you have q's about that
17:22:27 <vilobhmm11> DuncanT : https://review.openstack.org/#/c/283253/ - generation-id stuff details
17:22:29 <flwang1> vilobhmm11: : i'm repeating DuncanT's words. where to get the generation info? keystone?
17:22:38 <amrith> no
17:22:39 <flwang1> what happened if there is no HM
17:22:39 <Qijing> me too, I need to understand how generation to ensure sequencing
17:22:47 <amrith> generation infor should be in the library/delimiter
17:22:50 <amrith> and NEVER exposed
17:22:51 <nikhil> vilobhmm11: we need documentation on the generation_id work
17:23:02 <DuncanT> amrith: I plan too :-) Thanks
17:23:06 <vilobhmm11> flwang1 : https://review.openstack.org/#/c/283253/ - more details
17:23:11 <nikhil> looks like our first action item
17:23:13 <flwang1> vilobhmm11: cool, thanks
17:23:55 <nikhil> vilobhmm11: I think sudipto can help us get some official doc page on the gen_id stuff
17:24:06 <nikhil> he was looking at things from nova perspective
17:24:11 <vilobhmm11> amrith : shouldn't the generation-id info be passed from the caller ? since it is associated with a point in time view of resource that the project has seen
17:24:11 <nikhil> flwang1: and I can discuss it for glance
17:24:25 <flwang1> nikhil: awesome
17:24:35 <vilobhmm11> nikhil : that would be great; if not i can include one in the repo
17:24:38 <amrith> vilobhmm11, No
17:25:11 <Qijing> it looks the generation-id is used internally
17:25:16 <vilobhmm11> amrith : may be then i am missing something; please clarify
17:25:24 <nikhil> gen_id seems like something in the lib as it involves info/comput of the resource limits/usage
17:25:50 <amrith> rather than doing it here, maybe a short writeup that you can put in your doc?
17:26:22 <vilobhmm11> amrith : that works; will connect after the meeting
17:26:37 <nikhil> please create a etherpad
17:26:43 <nikhil> (at least)
17:26:56 <vilobhmm11> #action vilobhmm to come up with a doc explaining about generation-id and how it will be used by projects consuming delimiter
17:27:44 <vilobhmm11> #action vilobhmm send out for review the generation id stuff and get feedback…include it under docs section of https://github.com/vilobhmm/delimiter
17:27:58 <nikhil> I think we also need a etherpad explaining the foundational stuff
17:28:22 <nikhil> check_ enforce_ delete_ quotas
17:28:42 <vilobhmm11> nikhil : i thought thats already covered in the spec
17:28:52 <vilobhmm11> https://review.openstack.org/#/c/284454
17:29:13 <vilobhmm11> or if its not clear in spec i can update the spec with more details
17:29:34 <nikhil> ok, let's chat after the meeting on this.
17:29:34 <vilobhmm11> please suggest and i can have an action item for it; whatever is easier
17:29:55 <vilobhmm11> sure
17:30:39 <vilobhmm11> nikhil : do you still want to have a discussion on which project want to consume delimiter and then talk about features for newton ?
17:30:50 <vilobhmm11> nikhil, flwang1, sudipto : ^^
17:31:05 <vilobhmm11> i am open to options
17:31:10 <nikhil> vilobhmm11: I want to talk about #2, 3, 4, 5 from etherpad
17:31:22 <nikhil> that is all related to glance
17:31:37 <nikhil> and that is somewhat tied to foundational stuff
17:31:48 <nikhil> (bare min required to get things rolling in glance)
17:32:12 <vilobhmm11> nikhil sure…may be we can revisit #6 in next meeting then
17:32:25 <nikhil> works for me
17:32:53 <vilobhmm11> #action : discuss about #6 from 05/10 agenda on 05/17 https://etherpad.openstack.org/p/quotas-wg-meeting-agenda
17:33:13 <vilobhmm11> nikhil : lets talk about #2,3,4
17:33:23 <vilobhmm11> over to you
17:33:38 <nikhil> #topic updates
17:34:18 <nikhil> So, the update is that glance team thinks it's a 6 month job to implement quotas (nested) and 5mo (for simple -- extending existing logic)
17:34:35 <nikhil> I wanted to get feedback from other teams on that perspective.
17:35:05 <vilobhmm11> imho thats correct…if that includes detailed unit tests, functional tests
17:35:17 <nikhil> From the spec, it looks like the library will be a thin layer and we can implement it in 6 weeks. But the research for some projects like nova shows it may not be the case.
17:35:32 <nikhil> well, development ideally should be TDD
17:36:14 <vilobhmm11> nikhil : +2 for TDD
17:36:15 <nikhil> (I want to avoid unit tests as something separate from normal code proposals)
17:36:41 <flwang1> a silly question, will the lib talk to database layer?
17:36:46 <nikhil> so, do we think we can get the lib running in 6 weeks?
17:36:55 <nikhil> yes
17:37:02 <nikhil> flwang1: yes
17:37:17 <nikhil> (API <-> lib <-> DB)
17:37:27 <vilobhmm11> flwang1 : yes; althought it won't maintain any db of its own
17:37:30 <flwang1> nikhil: wow, if so, it may impact a lot and i think it's a little bit hard to complete in 6 weeks
17:37:37 <flwang1> since it will impact the api
17:37:46 <nikhil> I am talking about the POC
17:38:04 <nikhil> it should not impact API
17:38:07 <vilobhmm11> i agree with flwang1
17:38:08 <vilobhmm11> nikhil : for 6 weeks we need more people
17:38:25 <nikhil> vilobhmm11: we've at least 6 people here who can contribute!
17:38:53 <nikhil> and that's why I wanted to break down dependencies
17:39:40 <vilobhmm11> ok thats gr8 then if everyone can actively contribute and we have deliverable plans for every week it should be do-able
17:39:49 <vilobhmm11> nikhil : ^^
17:39:57 <nikhil> basically for glance to even try to do this in newton, we need the lib ready to be adopted before june 15
17:40:22 <vilobhmm11> ok
17:40:30 <nikhil> vilobhmm11: that would be awesome, if we can discuss the foundational stuff once done by you we can diversify the tasks
17:40:56 <vilobhmm11> nikhil : i didn't get that
17:41:09 <vilobhmm11> for me the foundational stuff include
17:41:14 <vilobhmm11> quota engine
17:41:26 <nikhil> vilobhmm11: we can chat more on the logistics after the meeting. I just want us to think us in terms of lego pieces and not the entire workflow atm.
17:41:33 <vilobhmm11> basic set of api's to be exposed to outside world check_quota, enforce_quota
17:42:15 <vilobhmm11> nikhil : sure need to define foundational stuff :) as it means different for different people…please carry on
17:42:23 <nikhil> thanks!
17:42:35 <nikhil> #topic     Complexity comparison of independent quota implementation and nested (non-floating) one
17:42:45 <DuncanT> Once there's enough defined for me to start bashing out code, I'll look at the db migration issue
17:43:02 <nikhil> So, glance (and may be other prjs) implement quota as aconfig value
17:43:09 <DuncanT> Sorry, should have left that until AoB
17:43:22 <nikhil> which is something I defined as a comment onthe spec
17:43:52 <nikhil> vilobhmm11: how complex do you think it will be to accomodate what exists for ops in prod env today?
17:44:14 <nikhil> basically, if you see the response from Tim Bell to the email thread started by you
17:44:50 <nikhil> he wants us to implement delimiter that will keep business logic consistent for different drivers
17:45:13 <nikhil> (and that is somewhat tied to the design question #5)
17:45:19 <vilobhmm11> sure
17:45:39 <DuncanT> If you use different drivers (nested, not nested, etc) then they need a consistent data model anyway, for when an operator wants to move from one to the other
17:46:11 <nikhil> exactly
17:46:34 <nikhil> and I think that's too complex
17:46:40 <nikhil> vilobhmm11: ?
17:47:10 <vilobhmm11> DuncanT, nikhil : trying to understand the problem here
17:47:23 <vilobhmm11> why having different drivers will be a problem
17:47:58 <nikhil> say if the operator today are using config to enforce quota (comsumption calculated using something like a "file" command)
17:48:13 <vilobhmm11> ok
17:48:18 <DuncanT> vilobhmm11: Operators using one driver (e.g. flat) will want to be able to move to e.g. nested in the future, without downtime (at least in cinder, nova)
17:48:33 <nikhil> if we move to nested, that logic will be different
17:48:48 <nikhil> :-)
17:48:58 <nikhil> see this ties with upgrade story
17:49:14 <nikhil> we need to break things down, now
17:49:23 <vilobhmm11> nikhil, DuncanT : the way i had implemented it was that the source of truth for both flat and nested remains the same
17:49:24 <nikhil> before we are stuck arguing later
17:49:53 <vilobhmm11> just that in case of nested we have few more attributes to the tables into consideration than the flat approach
17:50:27 <nikhil> but it's not just flat that people will be using
17:50:42 <vilobhmm11> so lets say if we admin wants to move from "flat" => "nested" then can something be done to take these additional attributes into consideration for computation
17:50:45 <nikhil> there's a ton of combinations that one can have while using quotas in production
17:51:36 <nikhil> the business logic should be opaque as far as library is concerned
17:52:36 <nikhil> like jay said at the summit, this is thin wrapper on top of DB tables
17:52:46 <DuncanT> vilobhmm11:  In the cinder case, we've also got values in the existing db tables for quota limits that we need a way of live migrating across on a running system
17:53:22 <vilobhmm11> or lets say if we have seperate drivers then changing the driver in config to use for quota computation can help for example for "flat" => dbquotadriver whereas for nested => nesteddbquotadriver and the driver will have the business logic….which will be opaque (since its already getting all the inputs from consuming project) …the lib won't be smart enough todistinguish this data is coming from which project
17:53:56 <DuncanT> vilobhmm11: Calling all the tables 'delimiter_*' and having a 'get_existing_limits' config controlled callback will take care of the live upgrade, among designs
17:54:32 <DuncanT> vilobhmm11: The different  logics need to be kept in rough sync htough, so one can consume the other's data
17:55:08 <vilobhmm11> DuncanT : delimiter won't own any value in db; all the db migration and stuff is individual project responsibility
17:55:33 <vilobhmm11> delimiter will consume from what exist in the project database and rely on that
17:55:37 <nikhil> vilobhmm11: can you please add a research action item here? we need to send out to resp folks to get inputs from their projects I think.
17:55:58 <DuncanT> vilobhmm11: It needs to be able to handle schema differences between projects then
17:56:00 <nikhil> I realize that piet is here and didn't want to consume all the time for my topic when he wants to discuss UX aspects
17:56:15 <piet> ;^)
17:56:21 <vilobhmm11> #action : start ML discussion about Seperate driver for different project models or single driver to handle all the different project models (flat, nested). - nikhil
17:56:36 <nikhil> works, thanks.
17:56:52 <vilobhmm11> hi piet
17:56:56 <piet> Hi!
17:57:04 <vilobhmm11> please go ahead
17:57:12 <vilobhmm11> we just have 5 more min before we wrap up
17:57:24 <vilobhmm11> and realized we don't have time for open discussion today
17:57:25 <piet> So, I’m the current PTL for the OpenStack UX project.
17:57:44 <piet> We’re going down the path of identifying research needs for the next six months.
17:57:55 <vilobhmm11> DuncanT : will chat with you on this after the meeting
17:58:30 <piet> I'd like to focus on cross-project quotas since it was raised as an issue by operators
17:59:05 <piet> Inlcuding operators from places like BestBuy and TWC
17:59:34 <piet> There are several different research methodologies we can use, eg interviews or usability studies, that can be used based on the project’s specific needs.
17:59:39 <vilobhmm11> piet : sure that would be great
17:59:58 <piet> We need to start hammering on ways to use the time
18:00:02 <nikhil> I would really like that input piet
18:00:29 <nikhil> basically -- are people happy with what's currently out there and if not, how can we make it better
18:00:44 <piet> For example, there is a ton of value in just watching operators complete tasks with quotas
18:01:16 <piet> nikhil Yep, helps to prioritize efforts.
18:01:48 <piet> I have a meeting with one of you this week, but will roll-out the research plan to the rest of the group for revieww
18:02:00 <piet> Is that cool with you?
18:03:10 <piet> Can also start a discussion through the ML
18:03:41 <nikhil> piet: would you be able to join us next week?
18:03:47 <piet> At this point, I just need to understand whether you would like a slot before the next summit - otherwise, I'll give it to another project
18:04:00 <piet> nikhil yep
18:04:11 <nikhil> we can discuss UX stuff first so that you don't have to wait the full hour.
18:04:28 <piet> ;^)
18:04:31 <nikhil> vilobhmm seems disconnected
18:04:42 <nikhil> piet: I think the research study seems essential
18:04:43 <piet> Does the group have an active ML?
18:05:05 <piet> nikhil k
18:05:19 <piet> I think we're five over...
18:05:19 <nikhil> piet:  subscription to [cross-project] [quota] [delimiter] on openstack-dev
18:05:28 <nikhil> yeah :)
18:05:30 <nikhil> thanks all
18:05:46 <piet> Cheers
18:05:49 <nikhil> #endmeeting