17:02:08 <vilobhmm11> #startmeeting quotas-wg
17:02:08 <openstack> Meeting started Tue May 17 17:02:08 2016 UTC and is due to finish in 60 minutes.  The chair is vilobhmm11. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:11 <openstack> The meeting name has been set to 'quotas_wg'
17:02:15 <vilobhmm11> Courtesy quotas-wg meeting reminder: nikhil, vilobhmm, DuncanT, mc_nair, ninag, amrith
17:02:16 <amrith> ./
17:02:29 <vilobhmm11> hi all
17:02:45 <thingee> o/
17:03:05 <DuncanT> Hi
17:03:28 <vilobhmm11> hi thingee, amrith, DuncanT
17:03:51 <vilobhmm11> we can get started
17:04:07 <vilobhmm11> #topic Updates
17:04:45 <vilobhmm11> lets wait 2 more min for nikhil, harlowja, itisha
17:05:06 <DuncanT> Do you have the link for the agenda, please?
17:05:44 <vilobhmm11> DuncanT : https://etherpad.openstack.org/p/quotas-wg-meeting-agenda
17:06:45 <DuncanT> Thanks
17:07:06 <vilobhmm11> alrite lets start
17:07:28 <vilobhmm11> #1. Last week Delimiter was added to CI system : https://review.openstack.org/#/c/315232/
17:08:31 <vilobhmm11> its a good start since this helps us to get the CI support and we can follow the same methods for pushing/revieing/merging code as other projects do..
17:09:05 <vilobhmm11> but even before that to make good progess we had set up travis to run nightly job https://github.com/vilobhmm/delimiter/commit/62b89b853b4e4a879e135bc8a24ec58ec5ea591c
17:09:19 <vilobhmm11> so now since we have openstack ci we don't need travis anymore
17:09:45 <vilobhmm11> #. The project launchpad page is all set up
17:09:46 <vilobhmm11> https://launchpad.net/delimiter
17:09:50 <vilobhmm11> #2. ^^
17:10:18 <vilobhmm11> #3. Delimiter library is now under openstack umbrella https://github.com/openstack/delimiter
17:10:59 <vilobhmm11> #4. To get things started we have basic interface and support for zookeeper based driver https://github.com/openstack/delimiter/blob/master/delimiter/drivers/zookeeper.py
17:11:25 <vilobhmm11> #5. For SQL based driver work is in progress and should be up for review this week
17:12:06 <vilobhmm11> #6. We had interestring discussion on generation-id and how it should be used by the consumers of delimiter library here http://lists.openstack.org/pipermail/openstack-dev/2016-May/095029.html just in case you missed it
17:12:40 <vilobhmm11> We will have a document under delimiter/docs short doc to explain it to the consumers of library
17:13:23 <amrith> vilobhmm11, I have two questions re: updates
17:13:24 <vilobhmm11> thanks to amrith, jaypipes for there clear and concise explinations for #6
17:13:38 <vilobhmm11> amrith : sure
17:13:47 <vilobhmm11> before the questions
17:13:53 <amrith> the first is about the discussion at the oslo meeting yesterday and the second is about the irc chat with jay yesterday.
17:13:55 <vilobhmm11> thats it about the updates for the library
17:13:59 <vilobhmm11> for this week
17:14:13 <vilobhmm11> thingee, amrith, DuncanT : any questions , suggestions, feedback ?
17:14:16 <amrith> yes, these two are updates.
17:14:26 <vilobhmm11> ok
17:14:33 <amrith> #7. oslo meeting yesterday
17:14:37 <vilobhmm11> amrith : please proceed will take suggestion, feedback later
17:14:41 <amrith> #8. irc chat with jaypipes yesterday
17:14:46 <DuncanT> I need to read the linked thread some more, but it certainly helps
17:14:59 <amrith> there appears to still be a signficant disconnect on the nature and scope of delimiter library
17:15:10 <amrith> for example, you ask about generation id's and things like that.
17:15:27 <amrith> yet projects (example. Nova) feel that all that will be in the projects scope, not delimiter scope
17:15:43 <amrith> so before going further, we need to all get on the same page about what the scope of delimiter is.
17:15:49 <vilobhmm11> irc://irc.freenode.net:6667/#8. irc chat with jaypipes yesterday on openstack-dev between 3-4 EST
17:15:56 <amrith> and whether consumers of the project would be interested in that converstaion.
17:16:28 <amrith> also whether consumers of the projectw would be interested in the outcome of the project.
17:16:58 <amrith> currently, as described by jay for example, delimiter is not a very interesting project (i.e. what does it really do?)
17:17:22 <amrith> how do we propose to address that.
17:18:40 <vilobhmm11> Thought about it yesterday
17:18:49 <vilobhmm11> #1. to answer that everyone agrees that quota infrastructure is not in a good shape in projects using quota
17:19:03 <DuncanT> amrith: It's in the project scope but a prerequisit for delimiter AFAICT
17:19:30 <vilobhmm11> #2. projects may be don't want to spend time in investigating time in going for "no-reservation" based approach which the library provides
17:19:50 <vilobhmm11> #3. support for quota calculation for hierarchical/non-hiearchical projects
17:20:27 <amrith> DuncanT, please elaborate. What is in the project scope and prerequisite for delimiter?
17:20:46 <vilobhmm11> amrith : +1
17:20:55 <vilobhmm11> DuncanT : please elaborate
17:21:17 <amrith> vilobhmm11, all of what you say fails to account for the fact that projects who want to use delimiter don't want to relinquish either the recording of the allocation or the actual process for consuming.
17:21:54 <DuncanT> amrith: Things like using generation ids
17:22:16 <amrith> DuncanT, what about them?
17:22:28 <amrith> they are, apparently, not in delimiter scope
17:22:42 <amrith> that is part of the 'how projects record allocations'
17:22:42 <vilobhmm11> amrith : will get back to that…listening to DuncanT now..
17:22:44 <DuncanT> amrith: But they do seem to be a prerequisit
17:22:59 <amrith> preprequisite for whom?
17:23:01 <amrith> for the projects
17:23:09 <amrith> delimiter neither sees nor touches them
17:23:11 <DuncanT> amrith: For projects that want to use delimiter
17:23:19 <amrith> absolutely not true
17:23:22 <amrith> that's an example
17:23:26 <amrith> of how nova wants to do things
17:23:32 <amrith> they aren't prescriptive about it
17:23:46 <amrith> for example, jay says there's no guarantee that projects will store this data in a relational database.
17:23:54 <DuncanT> amrith: I'm confused then about how else we can make things work, but I'm happy to wait to see code to a certain degree
17:23:59 <amrith> and very clearly, the whole generation concept depends on a transactional rdbms.
17:24:20 <amrith> DuncanT, did you read the chat with jay yesterday (3-4pm eastern)
17:24:20 <DuncanT> I think if we go too far into what is possible then we never get anything actually usable
17:24:56 <DuncanT> I have the log, I've not read it yet. I wouldn't take everything Jay says as gospel though.
17:25:32 <amrith> I don't take anything, anyone says, as gospel, even if it is printed in a book
17:25:35 <amrith> but that aside.
17:25:41 <amrith> he is illustrative of a consumer
17:25:42 <amrith> of the library
17:25:53 <amrith> and he describes what he views as delimiter.
17:26:02 <amrith> now, if he is one ... what about others?
17:26:07 <amrith> what do others expect of delimiter?
17:26:14 <amrith> I've seen no other 'consumer' tell what they want
17:26:21 <DuncanT> I'm one. What I most of all want right now is something less nebulous to talk about
17:26:44 <vilobhmm11> DuncanT : what does that mean ?
17:27:00 <DuncanT> I don't much care what scope delimiter chooses, I want to see if it produces something useful to the cinder project
17:27:31 <vilobhmm11> since its cross project it would be nice to limit t scope beforehand and see if it works in a cross project sense
17:27:54 <vilobhmm11> and not for a particular project otherwise we will have to go back-n-forth
17:28:51 <DuncanT> So what is the smallest possible scope we can start with? I'd say getting RDBMS working is absolutely needed for most existing deployments - cinder won't, I think, take anything that requires zookeeper, for example
17:28:52 <vilobhmm11> amrith : to answer your question ; in the current design that few of us have discussed delimiter can be seen as a proxy where it doesn't own any data, neither it directly manipulates any data but just acts as a mediator
17:29:52 <vilobhmm11> amrith : delimiter needs to provide something more than just being a proxy for the calls and doing some computation based of the data provided by the consumers is what you are pointing to right ?
17:30:06 <nikhil> hey guys, my previous meeting rolled over. sorry about the delay.
17:30:15 <vilobhmm11> np nikhil
17:30:22 <vilobhmm11> we are discussing about the scope for delimiter
17:30:53 <vilobhmm11> and going through the discussion that jay and amrith had yesterday on openstack-dev 3-4 pm about the scope of delimiter
17:31:32 <nikhil> Thanks!
17:32:08 <vilobhmm11> amrith : ^^
17:32:11 <vilobhmm11> you there ?
17:33:28 <vilobhmm11> DuncanT : you there ?
17:33:36 <DuncanT> Yes
17:33:40 <vilobhmm11> how will the solution proposed above will help
17:34:50 <nikhil> I think we should be okay with the rdbms assumption
17:35:08 <nikhil> most projects default to that
17:35:39 <DuncanT> I'm still trying to cartch up on the discussion yesterday and the email thread
17:35:46 <nikhil> and I echo DuncanT that the ZK won't be acceptable to most projects by default and it can be good to have
17:36:07 <DuncanT> But I think we're running round in circles at the moment, and will be until we've got a PoC
17:36:08 <nikhil> (I think good to have is just me)
17:36:16 <nikhil> DuncanT: +1
17:36:53 <vilobhmm11> just wanted to see an alternative as generation concept depends on a transactional rdbms.
17:37:24 <vilobhmm11> but for that delimiter should store generation-id
17:37:34 <vilobhmm11> which it is not doing at this moment
17:37:38 <nikhil> vilobhmm11: I think we should start with that approacha and then iterate over it
17:37:49 <DuncanT> From a cinder point of view, we're always going to have an rdbms, so I'm ok with that
17:37:54 <vilobhmm11> ok
17:38:01 <vilobhmm11> #agenda  Tasks for this week
17:38:09 <vilobhmm11> #1. Generation id documentation
17:38:30 <vilobhmm11> #2. Internal object structure for delimiter - vilobhmm
17:38:32 <DuncanT> We don't use generations as a concept yet, and I'm still trying to figure their cost/benefit outside of quota management
17:38:55 <vilobhmm11> DuncanT : is that targeted for Newton ?
17:39:03 <vilobhmm11> for cinder usage of generation id
17:39:10 <nikhil> I am wondering if sudipto and vilobhmm11 can pair on the gen_id documentation
17:39:21 <nikhil> as I won't be reviewing it :-)
17:39:27 <DuncanT> Not currently, we've no clear idea of the point, other than quota
17:39:28 <vilobhmm11> sure that would be gr8
17:39:32 <nikhil> my focus is the drivers
17:39:36 <vilobhmm11> sudipto : if you can help it would be nice
17:39:40 <sudipto> vilobhmm11, sure...
17:40:07 <vilobhmm11> by next week if we have these 2 things sorted we should be in good shape
17:40:29 <DuncanT> Migrating to generations on a live upgrade scenario looks challenging
17:40:34 <vilobhmm11> I have also started working on the sql based driver
17:40:37 * thingee has to disappear
17:40:40 <nikhil> I wonder if we can extend the basic engine that doesn't have transactional guarantees now?
17:41:01 <vilobhmm11> we have the initial template for zk based driver https://github.com/openstack/delimiter/blob/master/delimiter/drivers/zookeeper.py thanks to harlowja for that
17:41:43 <DuncanT> How can you use ZK without peppering ZK stuff all over the code? I've not got an understanding of that either
17:42:02 <nikhil> vilobhmm11: I think it would be nice if we can just have the most simple driver for now
17:42:10 <nikhil> DuncanT: it's a POC, we can iterate over it
17:42:33 <vilobhmm11> DuncanT, nikhil: zk is just a placeholder
17:42:37 <vilobhmm11> it won't be used
17:42:39 <nikhil> vilobhmm11: for example, get_quota, set_quota, and delete_quota
17:42:41 <vilobhmm11> it was something to startoff
17:43:03 <vilobhmm11> the only driver that will be used whenever delimiter is used by a project is the sql driver
17:43:05 <nikhil> vilobhmm11: without being transaction agnostic
17:43:16 <nikhil> sorry, with being*
17:43:20 <vilobhmm11> nikhil, DuncanT : ^^
17:43:37 <nikhil> ok
17:43:58 <vilobhmm11> so getting SQL driver up and running is our top most priority!
17:44:20 <vilobhmm11> and for that it will need the objects support etc which i plan to work on this week
17:44:46 <sudipto> vilobhmm11, you can pull me in for any work you want to get done there.
17:45:02 * sudipto is still catching up, but hopefully will be there soon
17:45:05 <vilobhmm11> sudipto : sure will need you help here as well
17:45:14 <vilobhmm11> thanks!
17:45:37 <vilobhmm11> feel free to ping me if you have any queries
17:45:57 <vilobhmm11> #topic open discussion
17:46:07 <nikhil> are there any reviews ? :)
17:46:38 <vilobhmm11> merged few of them yesterday
17:46:39 <vilobhmm11> https://review.openstack.org/#/q/project:openstack/delimiter
17:46:46 <vilobhmm11> everything that was out is merged now
17:46:46 <nikhil> looks like all josh and all merged :)
17:46:49 <nikhil> ok
17:46:53 <vilobhmm11> yup !
17:47:05 <nikhil> I want to propose DuncanT and sudipto for core if they are interested ?
17:47:42 <DuncanT> Sure, thanks
17:47:50 <vilobhmm11> cool
17:47:58 <sudipto> Sure, thanks!
17:48:06 <vilobhmm11> nice..welcome !
17:48:16 <vilobhmm11> so just to recap
17:48:18 <vilobhmm11> source : https://github.com/openstack/delimiter
17:48:30 <vilobhmm11> launchpad : https://launchpad.net/delimiter
17:48:37 <DuncanT> I suspect I won't provide much value until things have moved on slightly beyond where they are now and I've an actual picture in my head of what we're doing, but hopefully that is only a week or two away
17:48:47 <sudipto> +1 DuncanT
17:48:55 <nikhil> DuncanT: is your email first.last@gmail ?
17:49:07 <DuncanT> Yes
17:49:15 <nikhil> sudipto: DuncanT for cores you only get a free ping for review :)
17:49:30 <vilobhmm11> DuncanT : np..
17:50:00 <nikhil> done
17:50:03 <nikhil> #link https://review.openstack.org/#/admin/groups/1393,members
17:50:10 <vilobhmm11> thanks nikhil
17:50:13 <nikhil> and
17:50:14 <nikhil> https://review.openstack.org/#/admin/groups/1394,members
17:50:46 <nikhil> 👍
17:51:04 <nikhil> ping me for reviews, I would like to wrap my head around gen_id stuff this week
17:51:33 <DuncanT> I'm much the same
17:51:39 <vilobhmm11> Also to wrap up I would like to thank nikhil for representing delimiter and replying to all the ML thread and driving the CI related effort so precisely..thanks much nikhil..much appreciated
17:51:58 <vilobhmm11> any other open discussion
17:52:29 <amrith> sorry, I lost my internet connection for 20m
17:52:55 <amrith> DuncanT, vilobhmm11 I'm back. sorry abotu that
17:53:01 <vilobhmm11> amrith : np we are about to wrap up…the open discussion is going on..
17:53:01 <nikhil> 👍
17:53:05 <nikhil> my pleasure
17:53:16 <nikhil> I do want us to be on the same page with jay as much as possible
17:53:20 <tbarron> did you guys resolve yet whether delimiter will track resource usage or whether that will be done by the project who use delimiter?
17:53:33 <nikhil> but to do that, I need to understand him more than gen_id ;-)
17:53:38 <tbarron> apologies, but I am trying to catch up
17:53:52 <vilobhmm11> tbarron : good question
17:53:58 <nikhil> tbarron: usage is recorded in the project
17:54:04 <amrith> DuncanT, are you open to a phone call right after?
17:54:08 <nikhil> tbarron: calculated at the compute time by delimiter
17:54:13 <vilobhmm11> delimiter will be passed in with project usage record by the consuming project
17:54:25 <nikhil> vilobhmm11: ?
17:54:26 <DuncanT> amrith: Yes
17:54:45 <nikhil> vilobhmm11: if it's a simple number on the DB tables then I would agree.
17:54:55 <nikhil> vilobhmm11: do we need to have any more info on the usages?
17:55:01 <vilobhmm11> and delimiter will compute the quota…based on info provided as nikhil mentioned above
17:55:09 <nikhil> ok
17:55:31 <vilobhmm11> tbarron : does that answer your question
17:55:47 <tbarron> yes, at least at my current level of understanding
17:56:05 <vilobhmm11> in your case direct usage of # of volumes will be fetched from cinder.volumes
17:56:14 <vilobhmm11> rather than relying on cinder.quota_usages
17:56:23 <tbarron> and you are still thinking of using, say, keystone to hold the actual resource limits per project, subproject, etc.
17:56:47 <tbarron> ?
17:56:51 <vilobhmm11> nope the actual resource limits per projects will be fetched from respective projects and not keystone
17:57:00 <vilobhmm11> keystone does not want to be a single hub for all data
17:57:14 <nikhil> (no, all data lives in the project DB tables except the tree hierarchy from keystone)
17:57:43 <vilobhmm11> nikhil : +1
17:58:03 <tbarron> vilobhmm11: ok, so tree hierarchy from keystone, and delimiter would know how to compute based on that hierarchy?
17:58:14 <vilobhmm11> tbarron : yes
17:58:28 <vilobhmm11> it will compute even if there is no hierarchy
17:58:36 <tbarron> vilobhmm11: sure
17:58:50 <vilobhmm11> the objective is to comute irrespective of hiearchy or no hiearchy
17:59:02 <vilobhmm11> tbarron : was that helpful ?
17:59:13 <tbarron> yes, again at this level :)
17:59:16 <vilobhmm11> ok
17:59:25 <tbarron> i'm looking at manila quota reform :)
17:59:46 <vilobhmm11> ok shouldn't be different than other projects
17:59:48 <tbarron> so will be following with interest
17:59:56 <sudipto> I will dare look at it for Nova.
18:00:28 <vilobhmm11> please drop you suggestions here https://review.openstack.org/#/c/284454/ tbarron would be happy to see your requirement
18:00:34 <vilobhmm11> alrite thats it for this week
18:00:42 <vilobhmm11> #endmeeting