20:02:43 <markwash> #startmeeting glance
20:02:44 <openstack> Meeting started Thu May 23 20:02:43 2013 UTC.  The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:02:45 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:02:48 <openstack> The meeting name has been set to 'glance'
20:03:06 <markwash> Agenda for today
20:03:09 <markwash> #link https://wiki.openstack.org/wiki/Meetings/Glance
20:03:44 <markwash> #topic new blueprints
20:04:18 <markwash> first thing I wanted to mention is a sheepdog store
20:04:35 <markwash> I don't know if folks are familiar, but sheepdog is a block/object store, some folks want to use for storing images
20:04:49 <markwash> I was looking around for a blueprint we could approve, but couldn't find one
20:05:05 <markwash> here is the code so far https://review.openstack.org/#/c/29961/
20:05:08 <ameade> i'm here now
20:05:08 <markwash> worth a look
20:05:32 <markwash> maybe we should ask Liu to make a bp, or maybe its not a big deal
20:05:47 <iccha> its good to have ppl interested in contributing and writing code for new stores, but +1 on bp required
20:05:49 <markwash> anybody see any reason not to add a sheepdog store? the code in the review looks like it is progressing nicely
20:06:10 <ameade> i'm cool with it, don't even care if there is a bp
20:06:39 <markwash> one thing I keep thinking about is
20:06:58 <markwash> in the long run, should we offer better support for drivers living outside of the code base?
20:07:15 <westmaas> yes
20:07:15 <jbresnah> it seems good to me
20:07:17 <ameade> markwash: ideally
20:07:18 <esheffield> I was about to say something along those lines
20:07:21 <jbresnah> one thing tho...
20:07:25 <markwash> these backends we're getting are great, but glance core doesn't necessarily have the resources to test them
20:07:31 <jbresnah> there is always the debate about having plugins part of the distro or not
20:07:43 <jbresnah> like, it gets contributed and then the expertise goes away
20:07:50 <jbresnah> and maintaining it is hard
20:08:18 <jbresnah> probably good to have an easy path to include your backend store without it being part of glance proper.
20:08:41 <esheffield> maybe openstack needs a "user contrib" area for collecting things like this for people to find, but aren't part of the official distro
20:08:49 <jbresnah> yeah something like that would be good
20:08:54 <markwash> I think we maybe could have Glance provide a testing framework for what-have-you backends
20:09:03 <ameade> what are the exact benefits and is it worth our time?
20:09:37 <markwash> so this might be a concern folks should start to chatter about
20:09:39 <jbresnah> that would be good.  to vet a driver
20:09:44 <markwash> I don't see it in our really near-term future
20:09:46 <jbresnah> nod
20:10:08 <markwash> cool, I've already asked for a blueprint for sheepdog, and I'll mark it approved when I see one (as long as its not somehow crazy)
20:10:41 <markwash> next item. . .
20:11:14 <markwash> https://blueprints.launchpad.net/glance/+spec/image-error-state-management
20:11:17 <markwash> error state management
20:11:26 <markwash> this one was recently added by our good friend ameade
20:11:44 <markwash> and triggered some interest from folks who are tired of being confused when their snapshot fails
20:12:02 <markwash> not trying to dig into the details, here, yet
20:12:13 <markwash> but I think it might make sense for more folks to take a look offline
20:12:41 <markwash> I'm inclined to accept things we can to improve the experience with failed uploads, while maintaining backwards compatibility
20:13:10 <markwash> one more pair of items for new blueprints
20:13:14 <nikhil> markwash: a comment on your reply to the bp ..
20:13:24 <markwash> nikhil: go for it
20:13:25 <nikhil> can we just check the checksum?
20:13:51 <nikhil> no?
20:14:02 <markwash> maybe in some cases, however, I think sometimes the image gets deleted immediately on a failure
20:14:09 <markwash> so there wouldn't be anything to check
20:14:10 <nikhil> migth be easier in case of copy_from (raw data) might get complicate
20:14:34 <nikhil> d in case of import on that (just completing thought)
20:14:55 <nikhil> markwash: ah k
20:15:05 <markwash> sure. . I think there's some possible synergy with the import export blueprints, which are next
20:15:23 <markwash> #link https://blueprints.launchpad.net/glance/+spec/new-download-workflow
20:15:33 <markwash> #link https://blueprints.launchpad.net/glance/+spec/new-upload-workflow
20:15:58 <ameade> markwash: really in this bp I care about the use cases and not the proposed solution
20:16:08 <ameade> does it make sense to change the bp to not propose a solution?
20:16:15 <ameade> or does that make it a really fancy bug?
20:16:35 <nikhil> tricky
20:16:58 <markwash> ameade, might be a good idea, but I'm not worried
20:17:36 <markwash> rosmaita sent out an email about the upload and download workflows (some might say import and export)
20:17:36 <ameade> i'm assuming we will touch on when we want to solve this in the upload/download discussion?
20:17:38 <markwash> #link http://lists.openstack.org/pipermail/openstack-dev/2013-May/009385.html
20:18:08 <markwash> ameade: its possible we can help out with the solution separately from upload/download changes
20:18:24 <markwash> I guess its kind of up to whoever writes the code and reviewers :-)
20:18:33 <ameade> markwash: that's true lol
20:18:34 <markwash> I just want to make sure we preserve backwards compat :-)
20:18:35 <nikhil> markwash: well, i was under the impression both were diff (where import was upload+convert)
20:19:04 <markwash> nikhil: I was under the same impression, but I think if you read into the specs its clear that import/export are whats being discussed
20:19:17 <markwash> nikhil: but I wouldn't be terribly surprised if I was wrong about that and missed something :-D
20:19:43 <nikhil> :)
20:19:47 <rosmaita> which specs are you talking about?
20:19:52 <rosmaita> up/down?
20:19:55 <markwash> yup
20:20:08 <nikhil> anything works, as long as we've some concensus on how we'r handling the image standards
20:20:23 <nikhil> i mean vhd qcow2 etc
20:20:25 <rosmaita> they don't require conversion, but they leave open an easy way to include that
20:20:39 <rosmaita> for import, as part of the "verification" step
20:20:57 <rosmaita> for export, whatever we want to call the "scrubbing"
20:21:00 <markwash> oh, yes
20:21:02 <markwash> sorry
20:21:09 <markwash> okay, maybe I'm on the same page as rosmaita, and not nikhil
20:21:14 <markwash> and was just confused a moment ago
20:21:21 <rosmaita> np, not sure how clear that was in the spec
20:21:30 <markwash> import -> bits & metadata can change
20:21:52 <iccha> so our plan is to let the idea soak in and see what else ppl have to say in mailing list?
20:21:53 <markwash> for whatever purpose (verification, conversion, something else)
20:22:08 <markwash> well, for me, I love these spec
20:22:12 <markwash> specs
20:22:27 <markwash> and I think it makes sense to start breaking ground
20:22:54 <markwash> I am interested to take a poll here in this meeting to see if folks have concerns about the specs, or if they want more time to consider things
20:23:36 <rosmaita> i have one more ... the image cloning one
20:23:59 <rosmaita> not complete though, but i use the "image-action" resource
20:24:04 <jbresnah> I am good with those specs, but so much of it depends on async workers (in my mind) and that does not yet seem worked out
20:24:25 <markwash> rosmaita: I was wondering, do we get cloning for free with image import?
20:24:31 <nikhil> (has anyone got a chance to look at the etherpad for asycn workers?)
20:24:40 <nikhil> pls ignore if this is not the topic of discussion
20:24:46 <rosmaita> not for free, i don't think
20:24:52 <markwash> rosmaita: if we define glance in another region as the source of the import
20:25:11 <markwash> perhaps I was wrong
20:25:25 <rosmaita> markwash: i hav eto think, i think it requires more glance-to-glance coordination
20:25:36 <markwash> nikhil: you and jbresnah bring up a good point. it seems like the first step here is still async workers
20:25:50 <nikhil> markwash: we'r just worried about the grusome implementation details for cloning being incorporated as copy_from
20:26:09 <markwash> nikhil: that's a good concern to have
20:26:14 <nikhil> :)
20:26:33 <markwash> nikhil: I think we're free to redefine how copy_from looks in the api for v2 (since it does not exist there yet TMK)
20:26:51 <nikhil> sounds good
20:27:07 <nikhil> so do you wanna start with that or from workers for that specifically?
20:27:23 <markwash> well, it sounds like we don't have any surpises here about upload/download, so I think we can move on
20:27:32 <nikhil> darn the stmnt looks complicated
20:27:35 <markwash> I'd appreciate it if any folks here want to chime in on the ML
20:27:56 <markwash> just to keep the late-game surprises to a minimum
20:28:26 <markwash> lets talk about bugs for a minute
20:28:30 <markwash> #topic bugs!
20:28:47 <markwash> folks, I have not been paying enough attention to bugs
20:29:05 <markwash> I think we might be out of the post-summit bp triage glut
20:29:14 <markwash> so maybe we can start taking on bugs as a group?
20:29:30 * jbresnah has been slacking on bugs lately
20:29:34 <jbresnah> +1
20:29:37 <iccha> +1
20:29:53 <ameade> you mean triaging as a group?
20:30:27 <markwash> ameade: just bringing up the difficult ones
20:30:41 <markwash> I think we can make our way through most as a matter of just personal attention
20:30:58 <markwash> with that in mind, does anyone have any bugs they're worried about right now?
20:31:30 <markwash> I have https://bugs.launchpad.net/glance/+bug/1155389
20:31:31 <ameade> https://bugs.launchpad.net/glance/+bug/1069979
20:31:41 <ameade> is the one I posted still true?
20:31:49 <markwash> ameade: I'm worried about that one too
20:31:57 <markwash> ameade: I think we need to double check to verify
20:32:35 <markwash> ameade: I assume that I originally marked it "invalid" for a good reason
20:32:40 <markwash> but that assumption may be wrong. .
20:32:44 * markwash marksmash
20:32:49 <ameade> hehe
20:33:10 <markwash> we need a bugbot in this channel
20:33:39 <markwash> ameade: can you look into that bug some more?
20:33:48 <ameade> sure you can action it to me
20:34:03 <markwash> #action ameade try to reproduce https://bugs.launchpad.net/glance/+bug/1069979
20:34:06 <markwash> ameade: thanks!
20:34:07 <ameade> i'll confirm it if it's true
20:34:16 <markwash> looks nasty if it is
20:34:29 <markwash> and we could theoretically get in quick fix before havana 1 is released
20:34:47 <iccha> hmm i think esheffield tried it recently tried it and it worked , but would be definitely worth checking through
20:35:00 <markwash> #action markwash try to do some bug triage for a change
20:35:00 <esheffield> I've been doing a lot of experimenting the last couple of days with image creates / uploads and they were working
20:35:04 <ameade> i think taking a look at what the root issue described is would be good
20:35:12 <esheffield> once I got the headers etc. right
20:35:19 <esheffield> but didn't see that specific error
20:35:26 <markwash> esheffield: we might need to ask him about what transport hes' using for notifications, too
20:35:27 <ameade> to make sure it doesnt sneak back in somehow
20:35:55 <esheffield> yeah, that was just a plain vanilla devstack setup I was using
20:36:16 <markwash> any other bugs people want to draw our collective attention to?
20:37:03 <markwash> cool, lets move on to tracking some existing work
20:37:11 <markwash> #topic blueprints in progress
20:37:24 <markwash> there has been a lot of discussion of https://blueprints.launchpad.net/glance/+spec/api-v2-property-protection
20:37:39 <markwash> continuing in the etherpad
20:37:43 * markwash searches for link
20:37:49 <iccha> https://etherpad.openstack.org/public-glance-protected-props
20:38:02 <iccha> markwash: ^^
20:38:36 <markwash> not sure if we're getting closer to breaking ground, but if not any delays are probably due to me being a jerk
20:39:18 <markwash> the main issues right now are how to deal with collisions between protected properties and existing free-form properties
20:39:31 <markwash> I think we'll probably have an update again next week
20:40:23 <iccha> the challenges being v2 api advocates flat hierarchy for properties
20:40:38 <markwash> which means, lots of collisions!
20:40:42 <markwash> :-(
20:40:56 <markwash> nikhil: do you want to talk about async workers?
20:41:26 <markwash> nikhil: and can you remind me of any links
20:41:35 <markwash> ameade: ^^ I think I randomly assigned you to the async workers bp
20:41:37 <jbresnah> there are some requirements on etherpad: https://etherpad.openstack.org/havana-glance-requirements
20:42:09 <nikhil> sure
20:42:14 <nikhil> jbresnah: thanks
20:42:19 <jbresnah> np
20:42:38 <nikhil> markwash: was wondering if you got a change to browse through a get a picture
20:42:53 <nikhil> tho i might have been a bit late in getting everything there
20:43:08 <markwash> nikhil: I don't think I quite follow what you are asking me
20:43:22 <nikhil> also, haven't added the granular opinions on the requirements
20:43:32 <ameade> man this async worker has a lot of responsibilities
20:43:33 <nikhil> i was talking about https://etherpad.openstack.org/havana-glance-requirements
20:43:36 <nikhil> markwash: ^^
20:43:43 <nikhil> ameade: yeah
20:43:49 <iccha> do we have a list of must haves vs nice to haves?
20:43:52 <markwash> nikhil: oh, okay
20:44:00 <nikhil> i thought flavio was doing some kinda poc
20:44:01 <markwash> nikhil what I'm seeing looks good to me
20:44:23 <nikhil> markwash: ah k , though I feel like adding more details
20:44:29 <jbresnah> iccha: must haves would be good
20:44:35 * ameade wishes we had an implemented workflow service
20:44:38 <nikhil> just dont want o step on peoples toes on what ideas are already out there
20:44:39 <markwash> I think some simple POCs would be great things for making the next step
20:44:52 <iccha> jbresnah: i agree that may help us focus
20:45:04 * markwash vaguely recalls promising some sort of POC of his own. . .
20:45:11 <iccha> did we reach any agreement about threads vs separate process..
20:45:12 <esheffield> there seems to be a lot of chatter related to workers in other projects as well
20:45:25 <esheffield> wondering if openstack needs a general worker service of some sort
20:45:39 * ameade switches to openstack-meeting to ask if they are done with it yet
20:45:42 <jbresnah> iccha: i think we didnt get that far into impl
20:45:49 <esheffield> feels like there may be a lot of "wheel reinventing" about to happen
20:46:05 <ameade> can threads handle some of these heavyweight usecases?
20:46:20 <markwash> iccha: I still feel great about making it a deployment config option (greenthreads vs other processes)
20:46:34 <nikhil> markwash: +1
20:46:36 <markwash> esheffield: this wheel might not be as old or universal
20:46:47 <markwash> but maybe it is and I just don't get it yet
20:47:15 <nikhil> markwash: may be even services (besides threads and processes)
20:47:23 <markwash> esheffield: though I guess it does all feel kind of like celery
20:47:27 * nikhil waits for a hurray from jbresnah
20:47:42 <jbresnah> nod celery
20:47:44 <esheffield> markwash: yeah, I've been looking more at celery
20:47:50 <jbresnah> but i would like to be able to have a simple impl too
20:47:51 <markwash> do we have any takers on a POC / straw man code?
20:47:59 <markwash> or should I just follow up with flaper87?
20:48:03 <jbresnah> it is not worth forcing everything through amqp messaging
20:48:21 <jbresnah> nd nikhil
20:48:29 <nikhil> markwash: i would like to try that, if we'r not focusing on getting it before h1
20:48:45 <markwash> jbresnah: agree, especially when it can save deployers the trouble of making a change if they don't particularly care about the new async features
20:48:54 <esheffield> celery does seem to support a "webhooks" approach as well, so maybe not everything thru amqp
20:48:55 <jbresnah> I would still like to see the first thing be an interface definition for a worker
20:49:13 <markwash> nikhil: cool, go for it. . it won't hurt to have more than one if flaper87 is already working on one too
20:49:27 <jbresnah> esheffield: but i would like to even support thtreads, fork, multiprocessing potentially
20:49:33 <nikhil> cool, i'll follow up with him too
20:49:36 <markwash> #action nikhil work on a proof of concept for processing
20:49:46 <markwash> I may look into that POC as well
20:49:50 <nikhil> #action nikhil what mark said
20:49:51 <markwash> no promises
20:49:56 <jbresnah> nikhil: can the first step be an interface deifnition for a service?
20:50:02 <nikhil> :)
20:50:04 <markwash> jbresnah: +1
20:50:13 * markwash <3s interfaces
20:50:18 <iccha> 10 minutes
20:50:26 <markwash> iccha makes a good point
20:50:27 <nikhil> oh yeah
20:50:40 <iccha> i want to bring up something thats why :p
20:50:58 <markwash> iccha for open discussion?
20:51:01 <iccha> yeah
20:51:06 <markwash> cool
20:51:10 <markwash> #topic open discussion
20:51:29 <markwash> iccha you go first
20:51:32 <jbresnah> I have something too, but I yield to the representative from rackspace
20:51:48 <iccha> we have oft and on talked about soft deletes, where we want to keep them if it would be great to take a look at alternative approaches than existing one
20:51:59 <iccha> *whther
20:52:01 <jbresnah> +1
20:52:04 <markwash> +1
20:52:26 <jbresnah> it seems that glance developers are on the same page there
20:52:31 <jbresnah> but not so much the users
20:52:33 <markwash> from where I'm sitting, I'd love to see us fix the admin uuid reuse issue first
20:52:47 <markwash> and then draw up a deprecation plan for getting rid of soft deletes
20:52:53 <jbresnah> +1 uuid
20:52:56 <ameade> someone please describe the use cases that caused soft deletes to happen
20:53:03 <ameade> in the first place
20:53:08 <jbresnah> ameade: agreed
20:53:25 <jbresnah> ameade: we tried to tease it out on the mailing lists but pretty much failed
20:53:27 <markwash> ameade: the use case is "as a glance developer, I want to copy and paste nova's (oslos) db models, to save work"
20:53:28 <iccha> rosmaita: had good use cases for why we need soft deletes
20:53:45 * markwash turns his sarcasm back off :-)
20:53:46 <iccha> so we can look at alternative approaches if he manages to convince us :)
20:53:51 <jbresnah> heh
20:54:00 <ameade> markwash: i figured that was why
20:54:06 <ameade> :P
20:54:08 <rosmaita> well, suppose i make an image and put a lot of package info into the metadata
20:54:11 <jbresnah> markwash: but if that is the reason it exists it is very helpful to know
20:54:31 <rosmaita> and i want to find out what that was, even though the image was deleted in the mean tim
20:54:36 <rosmaita> *time
20:54:47 <jbresnah> rosmaitia: i think that is a misuse tho
20:54:53 <rosmaita> why?
20:55:01 <jbresnah> rosmaitia: because glance only knows what the metadata is now
20:55:15 <jbresnah> rosmaitia: it can change throughout the lifetime
20:55:29 <jbresnah> looking up meta for a running instance give no gaurentee of consistancy
20:55:31 <markwash> rosmaita: seems like a reasonable use case to me
20:55:45 <jbresnah> markwash: all that will give you is the metadata at the time of delete
20:55:49 <markwash> rosmaita: we probably just need to figure out the right way to do that in the api, or through notifications or more structured logging
20:55:56 <jbresnah> markwash: and that is only if updates are not allowed after delete
20:56:05 <markwash> jbresnah: good points as well
20:56:17 <rosmaita> ok, another use case would be if glance does a "changes-since" list like nova has
20:56:24 <jbresnah> that is sort of the same usecase that was talked aobut on the mailing list
20:56:37 <jbresnah> and i think it is trying to use glance as a hammer when it was designed to be a wrench
20:57:01 * jbresnah is not familiar with changes since
20:57:11 <markwash> (3 minutes)
20:57:22 <rosmaita> lets you see your servers from a particular time, even deleted ones
20:57:23 <ameade> changes-since is still auditing right?
20:57:27 <iccha> here is another way to look at it, than getting rid of info; say if we err on side of keeping it and have a better way to soft delete. what do we lose?
20:57:49 <jbresnah> rosmaita: so it takes all the changes and lets you get a view from any particular time?
20:57:59 <rosmaita> right
20:58:06 <markwash> can we take this discussion offline and hit a few more quick items?
20:58:08 <jbresnah> rosmaita: sort of like version control (like git or whatever) would do?
20:58:13 <iccha> go for it markwash
20:58:15 <jbresnah> nod
20:58:23 <markwash> next weeks meeting time
20:58:26 <rosmaita> jbresnah: not as fine grained
20:58:55 <markwash> crap I forgot my suggested time options
20:59:08 <iccha> 7 pm EST
20:59:09 <nikhil> time buddy?
20:59:14 <rosmaita> while mark is thinking, also, check out spec for clonning BP: https://wiki.openstack.org/wiki/Glance-image-cloning
20:59:19 <iccha> 4 am EST :p
20:59:25 <markwash> oh yeah, that one
20:59:39 <nikhil> looks funky rosmaita
21:00:07 <markwash> I think 7 pm EST would be good for folks other than in europe
21:00:12 <markwash> but flaper tends to stay up late anyway
21:00:12 <iccha> *EDT
21:00:20 <markwash> well, I'll send out an email
21:00:24 <markwash> since we're runnig out of time
21:00:30 <nikhil> haha, prolly we'd talk in UTC
21:00:33 <markwash> any quick thoughts on another bug / review squash day?
21:00:44 <markwash> +1 ?
21:00:45 <jbresnah> fridays work well for me for such things
21:00:46 <jbresnah> +1
21:00:47 <iccha> +1
21:00:58 <rosmaita> +1, hope to help this time
21:01:09 <ameade> +1
21:01:16 <esheffield> +1
21:01:21 <markwash> let's try for friday, unless its conflicts with the havana-1 release in a silly way
21:01:38 <markwash> thanks everybody, in order not to anger the gods of timeliness, I'm going to close it out
21:01:40 <iccha> is anything due for h1?
21:01:52 <markwash> . . . nothing critical
21:02:01 <markwash> we should probably start pushing harder on h2 deadlines though
21:02:08 <markwash> as possible by the constraints of open source
21:02:29 <markwash> Thanks everybody!
21:02:31 <markwash> #endmeeting