20:01:48 <markwash> #startmeeting glance
20:01:49 <openstack> Meeting started Thu Aug  8 20:01:48 2013 UTC and is due to finish in 60 minutes.  The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:50 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:01:52 <openstack> The meeting name has been set to 'glance'
20:01:56 <NobodyCam> hey :)
20:01:59 <markwash> brianr: hello
20:02:11 <markwash> let's do a quick item gathering again
20:02:19 <markwash> since I'm perpetually behind on setting up the agenda
20:02:28 <markwash> we want to talk about image_service in glanceclient
20:02:45 <markwash> https://review.openstack.org/#/c/33327/
20:02:49 <jbresnah> quota please
20:03:03 <markwash> quota, got it
20:03:11 <jbresnah> https://review.openstack.org/#/c/37993/
20:03:18 <brianr> new branch for tasks work?
20:03:21 <zhiyan> sorry, i'm little later.
20:03:22 <markwash> and revisit where we are on tasks
20:03:42 <markwash> anything else people want to mention first?
20:03:53 <markwash> I have to leave a tiny bit early today, probably at 50 minutes past the hour
20:04:42 <markwash> NobodyCam: you're here to talk about image_service in glance client, correct?
20:04:51 <markwash> NobodyCam: or did I make that up somehow
20:05:02 <NobodyCam> yep. deva cann't make it so you get me :)
20:05:05 <markwash> cool
20:05:12 <markwash> #topic image-tools
20:05:33 * NobodyCam notes he has not read all 1700 lines :(
20:05:51 <markwash> background: when cinder broke off from nova, it needed an image service, so it copied it from nova (like most other things)
20:06:00 <markwash> now ironic needs a similar service
20:06:20 <markwash> and it was pointed out that this layer could go in oslo
20:06:31 <NobodyCam> yes!
20:06:39 <markwash> but its so close to what is provided in glance client, it could go in glance client as well
20:07:20 <markwash> Looking through it again, I'd like to propose that we go ahead and put it in oslo rather than glance client, however
20:07:25 <jbresnah> what exactly is 'image service' here?
20:07:42 <jbresnah> nova.image.glance.GlanceImageService() ?
20:07:54 <zhiyan> yes, i think it is
20:07:57 <markwash> because I feel like the driver for this code is much more the needs of a service like nova and cinder, rather than what glance really is, or how it should be represented
20:08:00 <markwash> jbresnah: spot on
20:08:04 <jbresnah> ok cool
20:08:30 <jbresnah> the name is weird, because it is really a client, but i understand
20:08:35 <markwash> since this interface to me is bordering on legacy even in nova and cinder, I just don't want to take it on as a maintenance drag for glance client
20:09:03 <jbresnah> i just added a patch to that which could complicate it a bit
20:09:15 <markwash> ah indeed
20:09:17 <jbresnah> it now has a notion of loading download modules
20:09:20 <markwash> I saw that
20:09:22 <zhiyan> markwash: what's the oslo folks position for this port?
20:09:26 <jbresnah> which operate on the 'direct_url' info
20:09:37 <markwash> zhiyan: :-( I'm not sure
20:09:38 <jbresnah> and that all comes from entry points under the nova name space...
20:09:57 <zhiyan> jbresnah: yes, i has one also, you know, image handlers.
20:10:04 <jbresnah> but it does seem to make sense to me to have this in common code
20:10:12 <jbresnah> nod zhi, but i dont think that effects this case
20:10:25 <jbresnah> zhi: moving the image service
20:10:44 <jbresnah> markwash: if I have a vote I vote for glance client over oslo
20:10:59 <zhiyan> jbresnah: if you check his bp, you can see maybe they want to do some port work on 'fetch' functions.
20:11:01 <markwash> jbresnah: cool, can you go into that some more?
20:11:18 <jbresnah> markwash: into why i vote that?
20:11:36 <jbresnah> or into the patch that complicates it a bit?
20:11:55 <markwash> yeah, elaborating on your position of oslo vs. glanceclient
20:11:55 <flaper87> o/
20:11:59 <flaper87> sorry, I'm late
20:12:29 <jbresnah> well.... if it went into oslo it would be copied into other components
20:12:30 <jbresnah> (a model i dont like anyway)
20:12:38 <NobodyCam> markwash: what is the patch that may complicate things
20:12:42 <jbresnah> and it would have to keep in sync
20:12:52 <jbresnah> when what it is specifically doing is being a glance client
20:13:13 <jbresnah> it seems to me that the oslo model of syncing copied code shouldnt apply to clients
20:13:20 <jbresnah> they are less resource intensive
20:13:23 <markwash> NobodyCam: I believe it is https://review.openstack.org/#/c/37817/
20:13:27 <jbresnah> why not just use the client lib?
20:13:38 <jbresnah> yeah thats it
20:13:53 <flaper87> jbresnah: what client lib? Glance service ?
20:14:07 <jbresnah> NobodyCam: I also have a blog post about it, https://tropicaldevel.wordpress.com/2013/08/07/download-modules-in-nova/
20:14:27 <jbresnah> flaper87: why not put it into pythonglanceclient?
20:14:42 <zhiyan> NobodyCam: and https://review.openstack.org/#/c/33409/ potential.
20:14:43 <markwash> maybe I'm just being picky, but when I see that service interface, it just seems so hacky.  . detail() -> returns you a list of images?
20:15:05 <markwash> it seems like folks specifically just named the methods after the old v1 http paths
20:15:06 <jbresnah> markwash: i did not find it natural either
20:15:09 <flaper87> jbresnah: I'm a bit lost sorry, catching up. You're talking about moving nova's code into glanceclient ?
20:15:29 <jbresnah> flaper87: if the other option is to move it to oslo, then yes
20:15:38 <markwash> it really feels like folks just made something really quick in nova way back in the day
20:15:41 <markwash> and then it just never changed
20:15:58 <flaper87> jbresnah: if it is glance specific, I'm +1 for having it in glance
20:16:02 <jbresnah> markwash: what? in this industry?! no....
20:16:05 <flaper87> glanceclient, that is
20:16:08 <jbresnah> ;-)
20:16:11 <markwash> jbresnah: right :-)
20:16:16 <jbresnah> markwash: tech debt
20:16:42 <markwash> jbresnah: but, bringing tech debt like that into glanceclient implies we have to coordinate its changes with other projects and continue to provide support for it
20:16:53 <jbresnah> markwash: good point
20:17:11 <markwash> which is made harder by the fact that it seems to be an interface that is supposed to span versions
20:17:39 <jbresnah> markwash: but just architecturally
20:17:46 <markwash> I wish I understood why the projects can't "just use" glanceclient
20:18:09 <jbresnah> markwash: yeah... thats where i was going too
20:18:20 <flaper87> markwash: we just merged the service lib, and I think projects will now be able to just use glanceclient
20:18:23 <jbresnah> markwash: if the interfaces are broken, then lets fix them
20:18:26 <flaper87> that's the whole point about that code
20:18:45 <jbresnah> markwash: i mean, if there is an effort to do a refactor in this realm, we may as well do it right
20:19:08 <markwash> flaper87: I understand, but most of it seems cosmetic rather than real changes from the underlying
20:19:14 <jbresnah> i dont think oslo should have component client implementations
20:19:21 <jbresnah> that architecturally doesnt make sense to me
20:19:22 <markwash> like, detail, instead of client.images.list()
20:19:35 <jbresnah> but in full disclosure, i dont understand why oslo operates the way it does yet
20:19:40 <jbresnah> tho i am sure there are good reasons
20:19:41 <markwash> jbresnah: its not really a full client, just an interface adapter I think
20:20:01 <NobodyCam> was this not rejected by oslo a month or more ago?
20:20:11 <jbresnah> right, but that goes to the implication that the existing client has a less than usable interface
20:20:20 <flaper87> markwash: right, but that's just the first step. The idea is to clean-up that code
20:20:57 <flaper87> basically, w/ that code we can now make other modules use glanceclient directly, then we'll change it and upgrade those modules
20:21:04 <flaper87> they'll have to pin glanceclient version
20:21:06 <jbresnah> markwash: or i suppose it is more about felxibilty of subbing out glance
20:22:21 <jbresnah> markwash: in which case i guess it really isn't something for glance to decide
20:22:52 <markwash> flaper87: k, so maybe I just don't understand how its easier to do the changes in a central place when everyone is depending on it, vs when only one project is depending on it
20:23:25 <markwash> anyway, we should move on. if you all in favor could summarize again on the review
20:23:34 <markwash> I'll not be a hard blocker :-)
20:24:34 <zhiyan> markwash: maybe we can also ask oslo guys to make sure this, how about put this adapter to oslo.
20:24:35 <markwash> devananda: hi!
20:24:40 <devananda> hi!
20:24:58 <markwash> zhiyan: yes, at least we should find the review where it was rejected from oslo if it exists as well
20:25:06 <devananda> sorry i'm late. let me see if i can find that review
20:25:08 * NobodyCam is looking
20:25:18 <markwash> devananda: why can't ironic just use glanceclient directly, out of curiousity?
20:25:26 <markwash> what features are missing
20:25:54 <flaper87> markwash: mmh, I think I got confused. I was talking about having that stuff in glanceclient
20:26:08 <markwash> flaper87: no I think you were right on
20:26:15 <markwash> at least, I understood you I thought
20:26:30 <devananda> #link https://review.openstack.org/#/c/31186/
20:26:40 <devananda> #link https://review.openstack.org/#/c/30971/
20:26:58 <NobodyCam> https://review.openstack.org/#/c/30971/
20:27:01 <NobodyCam> :-p
20:27:03 <devananda> :)
20:27:29 <devananda> markwash: the goal is for ironic (and nova, and cinder) to use glanceclient! exactly
20:27:36 <zhiyan> aha, jbresnah, that my concern
20:27:39 <esheffield1> how important is the ability to flip between glance v1 and v2 for nova, cinder, etc.?
20:27:39 <devananda> we dont want the same code in all three projects, which is what we have today
20:27:46 <markwash> so why do we need this nova interface dinosaur? :-)
20:27:56 <devananda> nova and cinder already copy this code, and ironic needs the same functionality
20:28:08 <devananda> markwash: which dinosaur?
20:28:11 <jbresnah> it seems like the point of this thing is to have an abstraction layer to image services
20:28:12 <esheffield1> I was under the impression that was a driving factor, but if not then it does seem like major overkill
20:28:15 <markwash> I guess I'm trying to see, how is this functionality different from just using the client library?
20:28:16 <zhiyan> jbresnah: #31186
20:28:29 <jbresnah> the assumption there being that we may have image services other than glance
20:28:34 <devananda> markwash: aiui, the functionality does not exist today in the client lib
20:28:37 <jbresnah> in which case, it makes no sense to put that in something glance specific
20:28:58 <jbresnah> otherwise one could just use the glance client directly
20:29:04 <devananda> the glanceclient patch is adding the functionality that nova implements (and which cinder copied)
20:29:11 <jbresnah> the other point to this could be a new interface to glance client
20:29:24 <jbresnah> implying that the existing interface is not good
20:29:29 <markwash> it looks like its just list, get, download, upload, update, delete
20:29:35 <jbresnah> if so we should evaluate it on those points
20:29:37 <markwash> and all of that exists in v1, and is about to in v2 if not
20:29:39 <devananda> no... it's also open, inject, etc
20:29:50 <devananda> it's the "modify what's in an image and repack it" that we need
20:30:01 <devananda> and "open the image to copy certain things out of it"
20:30:21 <devananda> (in addition to list, get, etc...)
20:30:25 <markwash> oh, I didn't see that. .
20:30:31 <devananda> :)
20:31:05 <markwash> is that in the current review?
20:31:28 <devananda> i thought it was ...
20:32:00 <markwash> there's one feature in there that I know we don't support, namely we don't have the ability to proxy to multiple glance api servers
20:32:13 <markwash> basically, its like overriding the os-endpoint flag with a list
20:32:17 <markwash> but we could add that. .
20:33:18 <markwash> I just hate to see this interface further ensconced :-(
20:33:46 <markwash> we might need to move on...
20:34:02 <markwash> I think we can take up some more concerns in the review itself, but nevertheless this is a high priority for us to sort out
20:34:08 <markwash> so no more slacking from me!
20:34:11 <markwash> (on this issue)
20:34:26 <jbresnah> heh
20:34:40 <jbresnah> ok, i'll try to put together some coherant thoughts for the review as well
20:35:06 <NobodyCam> Thank you all :)
20:35:22 <markwash> thanks for joining, NobodyCam and devananda
20:35:25 <markwash> #topic quota
20:35:46 <markwash> jbresnah: are we still blocked on figuring out about where in the code the quota enforcement should land?
20:35:55 <jbresnah> yeah
20:36:11 <jbresnah> for reference: https://review.openstack.org/#/c/37993/
20:36:39 <devananda> thanks markwash :)
20:37:06 <jbresnah> markwash: you had some opinions on it because you did not like the db_api being introduced to the store/__init__
20:37:09 <jbresnah> markwash: iirc
20:37:11 <ameade> use the storage quota enforcement design pattern, duh
20:37:18 <jbresnah> heh
20:37:24 <jbresnah> ameade: can you send me that book?
20:37:24 <markwash> jbresnah: right. .
20:37:38 <markwash> as I understand our architecture, the stores and the db are fairly separate entities
20:37:49 <jbresnah> markwash: hard to argue with that
20:38:14 <markwash> I *think* that this could be a great tiny domain layer
20:38:33 <jbresnah> can you explain more?
20:38:41 <markwash> but it maybe hasn't escaped my attention that folks aren't meshing very well with the domain layers approach in glance today
20:38:58 <ameade> ya see, glance is like an onion
20:39:01 <jbresnah> heh
20:39:02 <zhiyan> markwash: also hard to agree with you, seems Scrubber-refactoring change also need db in store (adding location's status field).
20:39:02 <jbresnah> nod
20:39:10 <jbresnah> ameade: for sure
20:39:35 <markwash> yeah, basically inherit a proxy layer (image, image repo, image membership, image member repo), and then change just the functions you need to provide quota enforcement
20:40:01 <markwash> zhiyan: why does it need it? I think the scrubber in that case has a reference to both the db and the store, but we'd still not need the store code to talk to the db
20:40:38 <zhiyan> markwash: when using delay deletion, i need update 'status' in db, but queue-file.
20:40:44 <jbresnah> markwash: so you would like to add a new proxy layer for this?
20:40:52 <markwash> jbresnah: yes
20:40:55 <jbresnah> markwash: i can probably do that
20:41:13 <markwash> the idea of those layers is essentially that each captures a single cross cutting family of concerns
20:41:26 <flaper87> I can see the quota code being a proxy layer as well
20:41:29 <markwash> the division isn't quite right today, with policy and auth kind of separate
20:41:34 <flaper87> we've something similar for policies
20:41:35 <jbresnah> markwash: and have it be in a new module under glance/ ?
20:41:51 <markwash> jbresnah: module placement is my kryptonite
20:41:53 <markwash> :-)
20:41:58 <jbresnah> heh
20:42:01 <flaper87> lol
20:42:09 <markwash> well so is naming maybe
20:42:11 <markwash> :-)
20:42:22 <markwash> okay, so, wanna give that a try?
20:42:29 <jbresnah> yeah
20:42:32 <markwash> #topic tasks
20:42:37 <jbresnah> i'll figure out a place and run it though review
20:42:38 <jbresnah> thanks
20:42:42 <markwash> how's work going with asynchronous tasks?
20:42:52 * markwash looks at his watch. . stupid dentist appt :-(
20:43:18 <brianr> what did you decide about a branch for that?
20:43:22 <markwash> we had a meeting on tuesday to cover some of the design work and an attempt to divide up the work that needs to be done
20:43:48 <markwash> brianr: I'm not sure that I have a great answer there
20:43:59 <markwash> I like the idea of feature branches, because I think we need to all collaborate outside of master
20:44:06 <markwash> but there isn't a lot of good tooling to support that
20:44:21 <markwash> and we'd still want to leave things in small enough chunks to be reviewable by others
20:45:15 <markwash> brianr: so for now, it might be nice to start out working in github
20:45:40 <ameade> should we make an openstack-glance team in github then? with a glance fork?
20:45:54 <markwash> but it seems like we need another meeting, to figure out division of responsibility
20:45:58 <jbresnah> could do a stackforge project
20:46:03 <jbresnah> if gerrit is wanted
20:46:04 <markwash> brianr: ^^ do you buy that?
20:46:14 <jbresnah> and then bring it in once things are settled?
20:46:17 <markwash> jbresnah: ameade ooh neat ideas
20:46:32 <flaper87> mmh, I'd go with github org
20:46:52 <flaper87> stackforge might bring more complexity, it would have tests, though
20:47:22 <markwash> for this, given how little time is left in h3, it might make sense to just do whatever comes easiest
20:47:44 <brianr> markwash: i don't have a strong opinion
20:47:52 <brianr> +1 on what is easiest
20:47:53 <jbresnah> i think getting this into h3 is going to be a stretch
20:47:54 <markwash> brianr: nikhil what is most needed to help unblock progress? just a place to work?
20:48:02 <jbresnah> is that realistic?
20:48:13 <jbresnah> if not we may want to go with what is best instead of what is fastest
20:48:24 <jbresnah> but i dont have a ton of skin in that game so i will be quiet
20:48:44 <markwash> I"m concerned there as well
20:48:50 <markwash> which is one reason why I like a feature branch
20:49:03 <markwash> I don't want to have this feature half-landed
20:49:05 <flaper87> I don't think there's enough time to get this done before feature freeze
20:49:07 <jbresnah> nod
20:49:08 <markwash> very awkward to release note that :-)
20:49:34 <flaper87> I'd prefer to give this feature more time, more thoughts and make it right.
20:49:44 <flaper87> it'll be kind of important for glance once its done
20:49:57 <nikhil> markwash: from rax 2 of us are planning to work in this bp dedicated
20:49:58 <flaper87> so we better get it right and dedicate enough time on it
20:50:16 <markwash> okay, I have to run :-(
20:50:29 <markwash> nikhil, brianr: when can we check in again?
20:50:34 <nikhil> sure
20:50:39 <flaper87> nikhil: I'd love to help with that work, I didn't participate much in the design, sorry
20:50:54 <brianr> markwash: how about tomorrow p.m. ?
20:51:00 <nikhil> flaper87: sure, tomorrow we'r discussing a bit more in ET
20:51:25 <nikhil> so let try and follow up with some possible schedule
20:51:33 <markwash> let me know the time and I'll do my best :-)
20:51:36 <markwash> #endmeeting