20:06:34 #startmeeting glance 20:06:35 Meeting started Thu May 2 20:06:34 2013 UTC. The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:06:37 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:06:39 The meeting name has been set to 'glance' 20:06:58 what exactly IS glance? 20:07:03 nobody uses it 20:07:15 and, with the early troll lead, ameade 20:07:21 #link https://wiki.openstack.org/wiki/Meetings/Glance 20:07:56 wave 20:07:57 glance is like a box of chocolates 20:08:00 o/ 20:08:16 \o 20:08:43 so today we have several more blueprints to discuss 20:08:48 hi5 miss doesn't look good 20:08:51 tasty! 20:09:41 I've written out the list of the ones I'd like to consider on the agenda for today 20:09:59 but I"m happy to add other items as we go, to be sure 20:10:07 first one on the docket for me is https://blueprints.launchpad.net/glance/+spec/clone-image-across-regions 20:10:42 the way I see this going down is actually a few blueprints 20:10:49 1) add a "Glance" store 20:11:06 2) add the "import" functionality thats part of Upload/Download workflow 20:11:30 3) add a step to the "import" process that gathers metadata from the store location / image data 20:12:02 step 3 would also neatly accomodate https://blueprints.launchpad.net/glance/+spec/iso-image-metadata 20:12:26 nikhil: you own the cloning bp, do you agree with this breakdown 1-3 ? 20:12:28 i am not sure what #1 is 20:12:42 right, sorry meant to clarify, then forgot 20:12:55 the idea is a store that can use another glance endpoint as the source of image data 20:12:59 so basically glance/store/glance.py 20:13:00 markwash: works for me 20:13:05 how do you have 2 without 3? 20:13:12 parallel to glance/store/swift.py 20:13:27 So if we're dynamically pulling from another glance when needed, what if the original glance changes ACL's on the image? 20:13:29 ameade: think we gunna have dependecies here 20:13:51 #1 scares me a little 20:14:06 why 20:14:07 lindj: I think the idea is that the copy, clone operation is done once 20:14:20 so changes to ACLs after the clone succeeds don't really matter 20:14:22 but i have been dealing in test suite fork bombs lately, so perhaps i am just gun shy of recursion 20:14:47 markwash, so in that case, it's no longer a "glance store" of #1 20:15:00 once its copied 20:15:24 lindj: yeah I think you're right. . there would be a "store" for the purposes of import, but we could disallow it for register or for adding images 20:16:10 mmh, but #1 requires that the glance image willing to copy from the glance store to have access to the glance instance that has the image 20:16:31 or we're talking about public images here ? 20:16:38 flaper87: that's why we do a pull 20:16:54 flaper87: it could be a requirement to provide credentials to support the initial copy 20:16:56 pull model will auth user in first to avoid those issues 20:17:10 flaper87: but both glances could also be in the same auth zone 20:17:28 ok, that makes more sense 20:17:31 credentials -> token or trust 20:17:34 not passwords 20:17:36 I think we have to handle both cases where they could and could not be under the same keystone/credentials/ldap 20:17:53 lindj: sounds fine to me 20:18:20 might be easier to use download/upload in that situation 20:18:35 (if not under same cred system) 20:18:46 I guess passing in the remote temporary keystone credentials for the source-glance as a parameter to the destination-glance API is the way to do this 20:18:51 why is #1 needed? 20:18:59 jbresnah: I was about to ask the same thing 20:19:16 rosmaita: +1 20:19:44 lindj: yes - if you trust that you are sending to a friendly glance by looking at service catalog from keystone, etc 20:19:56 jbresnah: I'm sure alternative approaches would be fine, so long as they support getting image data / metadata out of a remote glance endpoint 20:20:02 "The regions a user has access to need be in the user's service catalog" is in the BP 20:20:48 just fyi 20:21:02 I definitely like #2 and #3 20:21:06 markwash: you would want to import from other stores too right? 20:21:09 westmaas, I think if we can't trust the destination glance, we have to use a "coordinator" process to do this instead of glance 20:21:15 jbresnah: yup 20:21:31 markwash: how would #3 work there? 20:21:44 some sort of store specific metadata plugin? 20:21:50 lindj: I think we have the service catalog and the source glance in the request and its a quick check 20:22:06 well, we get the service catalog as part of verifying the token 20:22:19 jbresnah: IMHO, should be more "container / image" oriented 20:22:19 by service catalog, we me one central keystone? 20:22:29 *mean 20:22:34 jbresnah: that could work, but I don't think it is required to work. . I would okay with there being a store-specific way to get extra data on an image and having it only in glance.store.glance 20:22:51 s/required to work/required for this bp to work/ 20:23:10 lindj: I mean we shouldn't send this token unless we are sure we are on the same keystone - it won't work anyway otherwise, and we would be leaking information 20:23:27 so I'd love to action nikhil and some other folks to refine the bp if needed 20:23:57 I'd like to take a look to the image metadata area 20:24:04 markwash: i think i am going to keep quiet on this one and try to figure it out in post 20:24:05 if nikhil takes a look to the pull thing 20:24:06 b/c I need to have the final bp targeted and assigned pronto :-) 20:24:40 markwash: i'm still trying to figure out whom to answer 20:24:44 westmaas: Okay, so then we're saying for the scope of this bp, that we only are doing transfers between glance's that are under the same keystone, and that if one wants to transfer images between different cloud providers, one must use a separate coordinator service 20:25:00 #action nikhil split up https://blueprints.launchpad.net/glance/+spec/clone-image-across-regions as discussed and propose any new bps for havana so they can be targeted and assigned 20:25:11 lindj: short term we can just make the call as if its public and requires no auth - if that fails, no luck 20:25:11 markwash: that works 20:25:30 implicit in that action item is to get some help from the folks here who have other ideas / concerns 20:25:39 not that I need to say that. . lol 20:25:52 markwash: nikhil I'd like to help on the image metadata step 20:26:01 flaper87: sounds good 20:26:03 works for me 20:26:06 markwash: yeah maybe we should keep impl detail discussion out of the meeting, like the potential for a glance store 20:26:13 flaper87: only if you recruit a few more Marconi contributors. ;) 20:26:22 lol 20:26:24 ameade: agree. . I'll keep that in mind for future 20:26:33 the image metadata will definitely have "restricted property" implications 20:26:48 kgriffs: LOOOOL 20:26:49 should restricted properties also replicate to the other glance? 20:26:54 kgriffs: I'm working on that 20:27:06 lindj: scary :-) 20:27:14 jbresnah: flaper87 it would be great if we could start detail discussion on the etherpad 20:27:15 lindj: i think they should 20:27:26 nikhil: sounds good 20:27:36 moving on to https://blueprints.launchpad.net/glance/+spec/remove-sensitive-data-from-locations, or objections? 20:27:37 to me it looks like different folks want/need different stuff atm 20:27:37 nikhil: do you mind puting that link in the bp? 20:27:47 sure 20:27:56 nikhil: count me in 20:27:56 do we need to mention protected properties in the bp? 20:28:01 #action nikhil add etherpad link to bp 20:28:12 markwash: can you add the action item above ^^ 20:28:15 ? 20:28:18 jbresnah: +1 20:28:33 ameade: for cloning across regions? 20:28:44 nikhil: ? I think anybody can #action 20:28:48 whoops I just did 20:28:53 yeah, that information needs to be retained i think 20:29:01 markwash: ohk 20:29:08 markwash: no objections 20:29:19 ameade: i'll link it as a dependency on the bp 20:29:32 who wants to tackle removing sensitive data from locations? 20:29:33 nikhil: if you need the whys to go in there we can talk 20:29:54 markwash: +1 20:29:58 ameade: sg, let's do it on openstack-glance channel offline meeting 20:30:21 markwash: +1 20:30:44 I'm thrilled we can do this, but we gotta be careful to make sure we don't mess up existing deployments 20:30:53 and provide some way to migrate away from the sensitive info 20:31:04 markwash: hey i dont wanna do all that, take it back! 20:31:05 I think swift is the only store we're worried about here, though 20:31:13 ameade: too late! 20:31:17 lol 20:31:24 haha 20:31:47 ameade: I'm sure flaper87 will help out, he had some work going on with this before 20:31:58 markwash: ameade +1 20:32:14 ameade will help keep meetings running on time 20:32:23 moving on to the related bp: https://blueprints.launchpad.net/glance/+spec/expose-image-locations 20:32:44 this is sort of the goal of the previously mentioned bp 20:33:21 is this something we can only do in v2? what followup steps are needed to start taking advantage of this blueprint for better performance in nova? 20:33:36 doesn that already happen? 20:33:50 jbresnah: sometimes, if you turn on a certain config setting 20:33:54 yeah 20:34:05 is that not enough? 20:34:28 there is code in nova to take advantage of it in specific situations 20:34:34 jbresnah: oh cool 20:34:50 it kind of relates to this: https://blueprints.launchpad.net/glance/+spec/direct-url-meta-data 20:34:55 IMHO, until the sensitive info bp isn't complete this shouldn't be done 20:35:01 and then this: https://blueprints.launchpad.net/glance/+spec/multiple-image-locations 20:35:09 doesn't that exposes the store location ? 20:35:13 nod 20:35:19 +1 20:35:26 flaper87: +1, and I'm not so H about it 20:35:30 but the problem is, you may need that sensitive info 20:35:33 s/exposes/expose/ 20:35:40 for it to be useful 20:35:58 markwash: yeah, I'd push it back, TBH 20:36:11 agree, I just think we should think about that a bit more 20:36:25 i would like to see what happens with multiple locations first 20:36:31 hmm, makes sense 20:36:32 it is really a special case of that 20:36:34 +1 20:36:45 where len(locations) == 1 20:37:01 ie: mult locations will have to solve all of the same problems 20:37:02 only we'd a way to give users temp swift url ;) 20:37:04 * flaper87 loves reading python 20:37:06 so I should probably just mark that bp as superseded by multiple locations and add a note? 20:37:19 +1 20:37:36 and also probably we should have https://blueprints.launchpad.net/glance/+spec/direct-url-meta-data depend on mult loca 20:37:47 jbresnah: +1 20:37:51 +1 20:37:52 or even be superseded by it 20:38:34 #action markwash mark direct-url-meta-data and expose-image-locations as superceded by multiple-image-locations and add a note to the latter bp 20:38:43 I love actioning myself to "mark" things 20:38:50 :) 20:38:59 briefly #topic quotas 20:39:10 i sent out a message to the list about quotas 20:39:15 20 mins to go 20:39:33 and while the quota work in keystone is targeted at H-2, I'm not super confident its going to make a lot of sense to wait on it 20:39:43 brb 2 seconds 20:39:58 markwash: does expose need to be set as superceded by removing creds too? 20:40:46 i mean, remove-sensitive-data supercedes exposing image locations...does that need to be set somewhere? 20:40:57 ameade: nope, I think we just need to make sure removing creds is appropriately a dep of multiple locations 20:41:03 unless I'm missing something 20:41:05 markwash: that works 20:41:16 so we gotta make sure that happens 20:41:16 re quotas, I'm not sure who really wants quotas and wants to work on it 20:41:39 but I'm of a mind that if there were a patch, we ought to consider it and not block on waiting for keystone unicorns 20:42:07 my gut feel is that anyone who really wants quota limitations for images will be using swift at back end and can use quotas at that layer 20:42:22 markwash: I think we should de-duplicate quotas code between nova and cinder and use that (by contributing back to oslo) 20:42:42 (or cinder for that matter) 20:42:59 flaper87: you should look at boson, it could serve for some of those purposes, and doesn't seem "too" crazy 20:43:12 markwash: I did already :P 20:43:26 ah, okay. . sry 20:43:35 thing is that, maybe quotas is something we shouldn't be querying on external services 20:43:40 or should we? 20:44:02 markwash: btw, hopefully that didn't sound jerkish, it wasn't meant to be like that 20:44:04 I'm still lending my weight to keeping quotas in with the service, for consistency and speed 20:44:12 flaper87: lol you don't have to worry about that with me :-) 20:44:37 markwash: :D 20:44:40 If we want to quota on a user's total # of images, then it has to be done by glance. If we care about the storage space used, seems more appropriate to control that at the cinder/swift side. 20:44:53 lindj: I'm cool with that approach too 20:45:21 i think our need for quotas is still undefined 20:45:22 i agree with the idea of not blocking a patch based on futures in other projects 20:45:22 can I pass off my action item from last week about having a quota pow-wow to somebody else? 20:45:27 lindj: I think that requires Sotres to be extended somehow 20:45:28 we just need to define what quotas we want first 20:45:41 I mean, making sure that stores can handle quotas 20:45:50 there are other stores besides cinder / swift 20:46:13 ameade: +1 20:46:15 its up to the cloud provider to use a backend that has quotas if they care about that.... 20:46:22 I can take on quota pow-wow if there are no other takes 20:46:22 but don't require them 20:46:24 +1 for making quota logic part of oslo 20:46:28 jbresnah: yay 20:46:42 jbresnah: I'd love to help with that 20:46:49 kgriffs: LOL :D 20:46:57 * kgriffs wants that for Marconi 20:46:57 cool 20:47:34 #action jbresnah quota pow-wow 20:47:51 so, https://blueprints.launchpad.net/glance/+spec/upload-download-workflow 20:47:55 we talked about this a bit 20:48:01 and I was supposed to sync up with rosmaita 20:48:18 but instead I talked with jbresnah 20:48:40 all I really care about here is that the bp needs to be broken down into two things, import and export, and I need volunteers and approx schedule 20:49:09 aren't we gunna talk/discuss on this on etherpad more? 20:49:16 but I think maybe in our conversation jbresnah thought it seemed less crazy of an idea than before? 20:49:22 nikhil: oh maybe, did I miss something? 20:49:32 markwash: i left that convo feeling good 20:49:35 trying to reload state 20:49:36 huzzah 20:49:57 right i recall 20:49:59 markwash: maybe not 20:50:14 the big thing was the distinction that upload is bits, and import is an image 20:50:23 conversion/verification/etc could happen on import 20:50:25 markwash: guess I was curious if this would be covered 20:50:37 in the etherpad discussion of the 3 points that you'd brought up 20:50:59 so i think the key work item there is making some sort of temp upload space, and then a means to have a work pipeline 20:51:07 is it time of Open Discussion 20:51:09 ? 20:51:19 don't wanna miss out on creative stuff 20:51:21 potentially calling out to other services to do work (like an image conversion service) 20:51:26 jbresnah: oh yes, I forgot, it might depend on async-workers 20:51:26 markwash: is that about right? 20:51:37 jbresnah: sounds reasonable to me 20:51:41 we haven't discussed async workers much 20:51:51 rosmaita: true, my bad 20:51:52 this is probably worthy of its own meeting 20:51:57 +1 20:52:06 let's sync a bit on etherpad firs 20:52:14 back 20:52:18 sorry guys, stupid ISP 20:52:28 flaper87: dude, give me my nick back 20:52:44 can we try to grab a meeting time in #openstack-glance for async workers? 20:52:52 #action nikhil to link etherpad in upload-download bp 20:53:08 sure 20:53:09 markwash: +1 20:53:11 markwash: +1 20:53:15 +1 20:53:30 #action flaper87 schedule a meeting for glance async workers discussion 20:53:48 with 7 mins to go, I wanna re-ask question from last week 20:53:52 if that's ok markwash ? 20:53:55 no! 20:53:57 sure :-) 20:53:58 hahahahaha 20:54:01 :) 20:54:13 have you heard back from scoot about the public glance bp 20:54:14 ? 20:54:19 scott 20:54:29 I thought I would hear back, but then I never asked him 20:54:50 hmm.. 20:55:00 has that been superseded by https://blueprints.launchpad.net/glance/+spec/exposing-glance-for-public-clouds 20:55:02 which is to say I forgot to do anything about that 20:55:43 #action markwash seriously this time ping smoser about public-glance 20:55:44 rosmaita: interesting 20:55:57 jbresnah: are you going to pick a time for the transfer service meeting? 20:56:06 markwash: the bp rosmaita just linked does not have smoser on it 20:56:07 i was just going to ask about that 20:56:20 ameade: any times that work for you? 20:56:40 i was hoping for something next week before thursday's meeting 20:56:46 nikhil: https://blueprints.launchpad.net/glance/+spec/public-glance does, doesn't it? 20:56:50 jbresnah: monday or tues work? 20:57:02 pretty open time-wise i think 20:57:17 markwash: wonder which one to follow? 20:57:19 Monday is very open for me 20:57:45 jbresnah: let's keep this in the evening ET 20:57:55 so that it's not tool late here :) 20:58:02 nikhil: ok 20:58:03 #info all blueprint folks be sure to propose your bps to havana by the beginning of next week so I can make sure to accept things before the project meeting 20:58:03 markwash: the public-glance is from folsom summit 20:58:12 rosmaita: good point 20:58:27 rosmaita: I think its possible that it already is implemented, or that nobody cares anymore, or something else 20:58:57 markwash: sorry i got a little out of order there 20:59:04 jbresnah: no worries 20:59:07 we can parallelize 20:59:12 cool 20:59:18 (I get a little frantic with 2 minutes to go) 20:59:21 (eek, 1!) 20:59:28 30 sec 20:59:32 does this same time (20:00 UTC) work on monday for interested people? 20:59:32 59,58,57,56.... 20:59:38 for a discussion on the image transfer service? 20:59:39 jbresnah: +1 20:59:42 jbresnah: +1 20:59:42 jbresnah: +1 20:59:43 +1 20:59:48 excellent 20:59:48 +1 20:59:53 that was pretty easy 20:59:58 hahaha 21:00:00 is Iccha still off? 21:00:06 back monday 21:00:07 she'll be back mon 21:00:08 at that point i mean 21:00:16 we didn't get a chance to talk about rolling db migrations 21:00:16 hmmm, i will email her to see if that works 21:00:17 no 21:00:23 I hope there aren't any disappointed lurkers. . . 21:00:23 yes, she gets back sat pm 21:00:50 as a lurker, I am not disappointed 21:00:55 haha 21:01:10 #action jbresnah send out email about transfer service discusion 20:00 UTC 5/6/2013 21:01:19 or may be calendar invite 21:01:22 jbresnah: ^^ 21:01:36 email is fine for me 21:01:40 but do as you will 21:02:11 markwash: i will work on up/down blueprint splitting 21:02:20 thanks! 21:02:42 will aim for tues for comments before next mtg 21:02:58 cool 21:03:04 any last items from folks? 21:03:18 (I don't *think* anybody has this room after us, but could be wrong) 21:03:45 * flaper87 will give a talk about glance going public at Openstack Israel 21:03:49 would love your feedback 21:04:15 when is that? 21:04:20 May 27th 21:04:27 cool 21:04:32 very cool 21:04:43 more exposure for exposing glance! 21:04:56 I will email you the slides as soon as they are ready 21:05:15 great 21:05:26 I guess that about does it for us guys 21:05:34 thanks! 21:05:38 #endmeeting