20:03:52 #startmeeting glance 20:03:53 Meeting started Thu Dec 12 20:03:52 2013 UTC and is due to finish in 60 minutes. The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:03:55 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:03:57 The meeting name has been set to 'glance' 20:03:58 hi glance folks 20:04:04 o/ 20:04:04 Hi markwash 20:04:10 hola 20:04:15 o/ 20:04:21 hi 20:04:26 hi 20:04:35 so folks I've been a bit absent this week, stuff at work and kitty health issues 20:04:39 Hi 20:04:43 hi 20:04:51 so i really appreciate whoever added stuff to the agenda 20:05:12 #link https://etherpad.openstack.org/p/glance-team-meeting-agenda 20:05:22 looks like we have some new folks 20:05:32 Hi guys! (looking around) is this a glance meeting? 20:05:39 yes this is 20:05:40 I suspect that is because of the exciting ML threads about glance and scope expansion 20:05:45 * markwash looks for links 20:06:22 #link http://lists.openstack.org/pipermail/openstack-dev/2013-December/021233.html 20:06:49 I propose we start off with some discussion of this 20:06:57 #topic glance and heatr 20:07:11 for those that aren't familiar, i suggest that you review the email link I posted 20:07:29 but the basic idea is to expand the idea of glance to store things like heat templates, IIUC 20:07:46 i am not opposed to expanding what glance contains, subject to keeping its basic philosophy intact (eg., immutable objects) 20:08:14 I guess there are some heatr folks here, care to intrroduce yourselves? 20:08:27 i support a more general catalog service 20:08:31 markwash: Hi, I am Georgy Okrokvertskhov form Mirantis 20:08:48 We are not from Heater team at all :-) 20:08:49 yes, it catalog-generalization to me 20:09:00 hello gokrokve 20:09:01 Hey guys, I'm Igor Marnat from Murano team (Mirantis) 20:09:17 Hi zhiyan 20:09:20 Hi, I am Alexander Tivelkov, Murano team 20:09:21 oh I see 20:09:22 I'm Stan Lagun - Murano & Mistral 20:09:31 sorry, I was confused, Hi Murano folks 20:09:39 hey folks 20:09:53 so, can you bring us up to speed on how Murano might fit into this integration? 20:10:09 Murano is not HeatR, but we are really interested in general-purpose metadata repository 20:10:41 Murano needs catalog for the same purpose, to store different objects 20:10:43 Murano is among those projects that have templates that can be stored in Glance 20:11:00 not only templates but scripts, UI definitions and workflows 20:11:05 We manipulate complex metadata packages, and we need to store them per-tenantly, index them, add tags, versions, authorships etc - in an immutable form, of course 20:11:31 We have an implementation of such catalog\repository in Murano 20:11:49 okay, there might be a little impedance mismatch but I think it doesn't sound like there would be any big problems 20:11:58 But Heater revealed that other projects also need repository 20:12:16 the one thing that's a little bit differnt in glance is that some of our metadata is mutable 20:12:22 And let's not miss the fact that Solum would also benefit from having the general metadata repo 20:12:38 and some of our metadata is semantically significant to Glance 20:12:44 so, not true metadata really 20:12:55 e.g. you can download images through glance 20:13:01 and Mistral too 20:13:08 and it will perform checksum verifications against the checksum attribute of the image 20:13:26 markwash: i think what gokrokve thinking is saving those files as our image entry 20:13:26 markwash: We need this too 20:13:45 gokrokve: ah okay 20:13:57 zhiyan: Kind of. At least in the first implementation. 20:14:00 Glance user is not able to modify an Image without changing ints id, right? 20:14:22 markwash: and also need a container... iirc, service metadata 20:14:33 ativelkov: it cannot modify the image data, it can modify metadata 20:14:40 *some metadata 20:14:49 its perfectly fine for us 20:14:58 markwash: i think if we want to pick this as a solid direction for glance it would make sense, we could start talking about a v3...of course there are more immediate needs we could jam into v2 20:15:01 Like, add tags, change description etc - right? 20:15:10 so what's our plan of attack? is there any sort of proposal already on the table? 20:15:13 markwash: We want to start submit BPs for new features required for genereic catalog 20:15:16 ativelkov: yeah stuff like that 20:15:17 I believe what is missing is a reach queriable metadata for all glance object that would allow implementing of catalogization - arranging object into hierarchies, groups etc. based on different criterias 20:15:28 markwash: We want to make sure that this is not a surprise for you :-) 20:15:42 heh sensible 20:16:16 ameade: you bring up a good point 20:16:26 markwash: Do you see a possibility to start this work in Icehouse? 20:16:33 would it make sense to do this stuff somewhat separtely as part of v2.x? 20:16:50 and try to munge the traditional view of images and these other aspects together in a v3 after Icehouse? 20:17:16 markwash: +1 on that 20:17:19 this sounds reasonable 20:17:21 markwash: We need this catalog to develop Murano, and Heat probably will be interested as well as they need this for HOT Software components 20:17:51 I kinda understand the complexity of adding stuff in domain layer so would recommed post I 20:18:42 gokrokve: okay cool. . so we shoudl jsut be on the lookout for some blueprints soon? 20:19:00 the domain model should be a separate convo we should def discuss about :) 20:19:04 gokrokve: to answer your question about Icehouse, its a bit scary but I think if we understand the bps well enough we might ahve some hopes 20:19:23 markwash: i kind of sense an agenda forming here for glance mini summit :) 20:19:26 gokrokve: is there a clear api definition for the metadata repo now? 20:19:33 Are HeatEr guys ok with this? So that there will not be 10 different private repository implementations by Icehouse release 20:19:39 I'll have a wiki-page with overall description of what we propose at about tomorrow 20:19:45 gokrokve: I think it may be possible to consider some slightly drastic options to make sure we can make progress 20:19:47 zhiyan: There is no final API defined yet. 20:19:59 gokrokve: for example, we could start out in a separate project under the same Glance program 20:20:00 Then we will make more detailed blueprints on a per-feature basis 20:20:34 markwash: do u mean, /templates like /images? 20:20:52 markwash: We can do this as a separate project. You are right. But are you ok to handle two projects. 20:21:00 I'm a bit concerned about adding specific code for the different objects that might be stored 20:21:08 iccha: that's one option, but just now I was saying something more like github.com/openstack/glance-template-api.git 20:21:15 markwash: iccha ameade also, more modular sqlaclchemy impl before adding this :) 20:21:15 gokrokve: from you old wiki, i can see a very draft define for poc, so i think if there has a clear define, i think it will be good to see the different for common metadata entry and image 20:21:33 could we think along the lines of a general metadata service with specific schemas defining the object types 20:21:40 gokrokve: handling two projects is a bit of a burden but it might still be an okay option 20:21:40 i see nikhil__ 's concern about strengthing what we currently have 20:21:43 zhiyan: Sure. We need Heater guys to contribute too. 20:21:52 esheffield: that makes a lot of sense 20:22:28 markwash: i'm not opposed to this evolving into maybe a glance replacement project either 20:22:29 esheffield: ameade +1 with a the option of drawing the line of cutomizations upfront 20:22:30 esheffield: +1 20:22:34 +1 20:22:38 ameade: +1 20:22:39 markwash: Can we discuss a separate project in ML? We will need input from TC on that too. 20:22:52 gokrokve: absolutely 20:22:54 too much customization will slow this down 20:23:16 gokrokve: I think we need a lot of clarity on the final api featureset though to talk about this intelligently 20:23:34 gokrokve: right now I don't know anywhere near enough to know which option would be best 20:24:06 markwash: Sure. I know that Randall is working on Heater BPs for Glance too. So we can sync up with him, to speed up the API discussion 20:24:14 there is much discussion to be had, especially about specific use cases...i think additional meetings are in order...or during the glance meetup 20:24:30 +1 20:24:44 ameade: I would rather use separate meeting. 20:25:00 was this just related to Murano or was there anything about Mistral too? 20:25:08 ameade: +1 definitely need additional discussion. gokrokve not sure if any interested folks from your side could manage a physical meetup? or if we should just schedule some separate IRC time 20:25:08 ameade: Otherwise we will consume whole glance meeting time. 20:25:40 gokrokve: yeah I think we'll wind this topic down for today here in a moment, just want to figure out the major next steps 20:25:42 Face 2 face will be very effective + some guys remote via hangout 20:25:55 Mistral would also probably need some repository in the future. Nothing specific for now 20:26:09 gotcha 20:26:10 We did this in Solum and it was efficient. 20:26:16 gokrokve: so there is a glance meetup in the works for late january near Washington DC. . not sure if that could possibly work? 20:26:30 markwash, gokrokve: i'm definitely interested in staying heavily in the loop on this 20:26:43 same here 20:27:05 markwash: Should work fine. We can have first meetings in IRC\ hangout and then f2f meeting for final decisions. 20:27:17 ashwini: yeah we might want more than 2 days if this does become part of the summit 20:27:25 gokrokve: okay, when should we meet again? 20:27:27 on irc 20:28:32 markwash: Lets meet on next week, say Tuesday 20:28:37 +1 20:28:46 +1 20:28:57 markwash: We will submit some BPs and create etherpads with drafts 20:28:59 markwash: yes we should have a more confirmed agenda for the mini summit to see how much time can be allocated to this discussion or we should consider 3 day options 20:29:13 gokrokve: okay sounds good 20:29:13 markwash: Also Heater team will have time to prepare 20:29:30 gokrokve: so we should look for an invite on the openstack-dev ML? 20:29:47 markwash: During next meeting we can also discuss how to do development \ separate project vs. branch 20:30:04 markwash: Sure. I will organize that. 20:30:10 great, thanks! 20:30:19 any other quick thoughts from folks? or should we move on? 20:31:05 Nothing from our side :-) 20:31:21 thanks for showing up 20:31:23 this is pretty exciting 20:31:35 #topic common version-agnostic api in glanceclient 20:32:11 that was mine 20:32:22 esheffield: go for it 20:32:32 #link https://etherpad.openstack.org/p/glance-client-common-api 20:32:46 basically wanted to get some more discussion on the idea of something like the image service layer being added to glanceclient 20:33:07 recall that Ghe had a patch doing some initial work in that direction a while back 20:33:31 is anybody against the idea, assuming we can work out the details of what the api should look liike? 20:33:37 this is all tied to supporting V2 in Nova as well, along with the requested api autodiscovery 20:34:30 I confess that *I'm* not 100% in support of it, but mostly because I was having trouble envisioning a good clean API 20:34:52 yeah, its a little rough 20:34:56 and what would happen in the case of mismatches (e.g. trying to use tags but only having a V1 backend) 20:35:05 at this point, it might just need to be "whatevers in both v1 and v2" 20:35:36 so probably tags wouldn't be part of v0.0, 20:35:48 image sharing would be really reduced or absent 20:36:23 etc 20:36:26 I know when we talked about it before it was mentioned that multiple locations was going to be required in Nova soon 20:36:55 so while I was hopeing on this helping with Nova -> Glance V2, that would be a blocker to this approach right away I think 20:37:14 esheffield: I think multiple locations might work 20:37:35 that would be great then 20:37:42 we just have to treat v1 as having only 1 20:38:07 there are some ways we can put in some info about whether or not adding locations is supported as well 20:38:10 if that is necessary 20:38:45 so I think what we need is someone who has time to commit to proposing an api 20:38:48 how big is the work on this? 20:39:50 * markwash is not sure 20:40:33 and we would have to maintain it for every feature we add as well 20:40:40 iccha: exactly 20:41:01 well, I did some work on the images layer in Nova which is kind of what this would be and refactoring that code into something similar to this took a couple of weeks 20:41:07 I think if we do it well, it will reduce support, because we can direct people to a lib api that we actually designed with the intention of supporting cross-version 20:42:10 esheffield: ok 20:42:43 we still have 2 other topics for this meeting btw :) 20:42:48 esheffield: looking at your quesitons in the etherpad 20:43:08 hmm, well let's try to respond in that etherpad to carry the discussion forward 20:43:08 I did reuse a lot of what was in Nova already, so it was a bit crufty - I'd want to take more time and do it cleanly with a well designed api this time so probably a bit more time 20:43:18 perhaps we can get to the other topics still, that way 20:43:33 yes, please add thoughts and comments there! 20:43:51 okay, anyone not ready to move on for now? 20:44:09 #topic glance versioning consistency 20:44:16 esheffield: is this also your topic? 20:44:29 heh, yes 20:45:14 we can probably hold off on the broader topic there for a bit, but in working on the bug linked there some concerns came up over the verionsing 20:45:20 and backward compatibility 20:45:47 the more immediate concern is if fixing that bug causes backward compat problems 20:45:58 ah 20:46:04 flwang was concerned esp. 20:46:17 well, we got out a little ahead of json patch 20:46:24 in draft 4, "add" on an existing member is an error 20:46:32 we were implementing support when it was still in draft form, not sure if that is the culprit here 20:46:37 in current standard, it acts like a replace 20:47:03 I think that sounds like it is not a problem for backwards compat 20:47:07 yes, when we went to api v2.2 we said we support draft 10, but in draft 10 (and current) it's as rosmaita says 20:47:13 well, at first consideration 20:47:14 but we still raise an error 20:47:33 we have already deprecated openstack-images-v2.0-json-patch 20:47:40 (or at least i have in the API docs) 20:48:30 it sounds like this is just a bugfix 20:48:40 I don't think depending on that erroring out is a behavior the client relies upon 20:49:12 flwang was worried about API users expecting that behavior, though 20:49:25 that was what I thought as well - if anything people would be having to work around it and worst case the workarounds wouldn't be needed now 20:50:09 hmm, flwang is not here to defend himself 20:50:15 so let's all attack! 20:50:16 j/k 20:50:19 :) 20:50:24 the glanceclient is unaffected too - it always fetches the image and generates a proper 'replace' or 'add' as needed 20:50:57 so, the only difference is that now if you use add and it already exists, you get an error? 20:51:07 yep 20:51:20 is there any chance we could implement so if you use the old content type it does the old behavior, and if you use the normal content type you get the bugfix? 20:51:25 probably a bit hacky, but 20:52:21 that kind of circles back to the bigger topic of version concistency 20:52:30 well, the code right now rewrites the 2.0 request to be like a 2.1 request 20:52:36 if you get a list of versions you get several, but they're all actually the same thing 20:52:51 esheffield: yeah that always seemed a bit weird to me 20:53:12 based on what we're doing, it would be easier to just say "we use semantic versioning" somewhere in the docs 20:54:07 * markwash is starting to worry about time for the next topic 20:54:20 next topic does not need much time 20:54:29 if there is a min time for open discussion I've a quick question 20:54:30 sorry, didn't mean to dominate things today! :-( 20:54:53 take this to the ML perhaps? 20:55:05 esheffield: I responded to the bug discussion 20:55:15 esheffield: I think we need a concrete proposal for what changes we want 20:55:20 not just "why is this weird" :-) 20:55:28 "because OpenStack" 20:55:32 lol 20:55:43 :-) 20:55:48 gotta love the honesty 20:56:09 there's probably lower, weirder fruit 20:56:11 :-) 20:56:13 okay 20:56:16 i agree we need a definite proposal for what we want to do before going to the ML 20:56:22 does it bother anyone else that OpenStack is camel case? 20:56:22 #topic image sharing 20:56:27 (sorry to steamroll) 20:56:43 so i promised to put together something to get the discussion going 20:56:51 looks like you got a #linkt here 20:56:53 ameade: +1 :) 20:56:54 well 20:56:57 s/linkt/link/ 20:57:29 #link https://etherpad.openstack.org/p/glance-image-sharing-discussion 20:57:54 anyway, we can discuss at next meeting 20:58:13 after everyone who's interested has looked over the etherpad 20:58:22 #topic open discussion 20:58:43 can I ask? 20:58:57 It seems a number of people did indeed follow my email about abandoned patches 20:59:11 i'm going to gather more stats on bugs and things that have been in progress for ages 20:59:32 then send another reminder email and later I will laser triage patches and bugs that nobody updates 20:59:56 nikhil__: did you have a note? 20:59:57 currently tasks response is of the form {} vs. openstack common usage {'task': } ? 21:00:13 let's be glance-consistent 21:00:20 +1 21:00:29 same as images then 21:00:35 cool, thanks 21:00:41 okay, thanks everybody! 21:00:45 wish me luck 21:00:51 (no reason) 21:00:52 good luck 21:00:55 good luck markwash ! (not sure for what though) 21:00:58 #endmeeting