20:03:13 <markwash> #startmeeting glance
20:03:14 <openstack> Meeting started Thu Jun 26 20:03:13 2014 UTC and is due to finish in 60 minutes.  The chair is markwash. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:03:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:03:17 <openstack> The meeting name has been set to 'glance'
20:03:22 <markwash> maybe just my client was looking funny
20:03:31 <SergeyLukjanov> markwash, I think that it was just a topic
20:03:45 <markwash> no worries, I think we're all set now
20:04:17 <rosmaita> o/
20:04:22 <ativelkov> o/
20:04:24 <hemanth_> o/
20:04:29 <jokke_> o/
20:04:30 <nikhil___> o/
20:04:42 <arnaud> o/
20:04:43 <TravT> o/
20:04:45 <markwash> agenda is here
20:04:47 <markwash> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda
20:05:13 <markwash> Looks like there is a gate-unblocking fix at the following link
20:05:18 <markwash> #link https://review.openstack.org/#/c/102915/
20:05:36 <markwash> also seems like a straightforward fix, if folks have some downtime soon to review
20:05:45 <markwash> let's dive into the other items, as everyone seems to be here
20:05:58 <markwash> #topic Update cinder volume-image-metadata  (rel. protected properties)
20:06:13 <TravT> #link http://openstack.10931.n7.nabble.com/cinder-glance-Update-volume-image-metadata-proposal-tt44371.html#a44523
20:07:20 <nikhil___> markwash: reviewing now
20:07:34 <markwash> can anyone give a bit of a summary? I'm still playing catchup on this topic
20:07:57 <TravT> Basically, we are adding the ability to update the volume image metadata stored in Cinder.  However, Cinder doesn't have property protections.  So, the question is what approach to take.
20:08:18 <TravT> The proposal is to allow cinder to lookup property protections in Glance.
20:08:49 <TravT> rosmaita asked about replicating the config file and functionality in cinder as an alternate.
20:09:01 <rosmaita> I'm wondering whether glance is the correct source of truth for this
20:09:57 <markwash> I'm assuming the volume-image-metadata properties were originally readonly becuase they're intended to track history?
20:10:09 <markwash> not track something that is "live" about that volume instance
20:10:28 <TravT> Those properties are used by nova just as if they came from glance
20:10:38 <TravT> Nova doesn't distinguish between them
20:11:03 <DuncanT> markwash: Sorry, late joinng. They're read-only because the idea of them changing never occurred to us... they're there at all because some bootable images didn't work without them
20:11:32 <TravT> So if they have things that affect scheduling or driver behavior, they are treated the same in nova whether the boot source in glance image or cinder volume
20:12:09 <TravT> The longer mailing list thread that Facundo included started with somebody pointing out that a bootable volume can change over time.
20:12:17 <TravT> And they may need to update the drivers.
20:12:22 <markwash> the idea of having every cinder update to those properties also require glance to be available to validate the update is pretty scary
20:12:38 <arnaud> can
20:12:43 <arnaud> cinder override
20:12:54 <arnaud> with another name for the property
20:13:10 <arnaud> instead of getting rid of the value coming from glance
20:13:44 <DuncanT> arnaud: Cinder has its own copy of them all, it doesn't currently consult glance for anything once the volume is created
20:13:53 <arnaud> yes
20:14:17 <DuncanT> arnaud: But cinder doesn't know what it is allowed to let the tennant change, so currently it allows no changes
20:14:32 <markwash> can the extension just make the nova-facing portion of this metadata mutable and assume that the rest of it must remain immutable?
20:14:46 <markwash> s/extension/cinder extension/ ?
20:15:21 <arnaud> markwash, iiic, you cannot make mutable based on the fact that some of them can be protected
20:15:32 <arnaud> s/iiic/iuic
20:15:38 <fmaldonado> there are properties that maybe we want to maintain immutable, like license key
20:15:52 <DuncanT> fmaldonado: +1
20:15:53 <markwash> well, but the ones that nova cares about are probably quite limited, right?
20:15:57 <markwash> kernel image id
20:16:01 <markwash> min_disk
20:16:04 <markwash> min_ram
20:16:31 <markwash> basically, I think cinder could have a completely static, hard-coded list of the volume-image-metadata keys that are mutable
20:16:35 <markwash> and make the rest of them immutable
20:17:32 <DuncanT> markwash: Isn't that pretty much exactly what giving cinder a list of the protected properties file does?
20:17:53 <markwash> not quite, at least how I'm seeing it
20:18:08 <markwash> becuase the protected properties files goes way into stuff that operators care about
20:18:09 <TravT> Well, AFAIK, property protections originated from operators wanting to be able to specify certain keys specific to them that they don't want users to be mucking with.
20:18:16 <markwash> where as the list I'm talking about only has to do with stuff that nova cares about
20:19:04 <DuncanT> markwash: So if we go that route, you can bypass protected properties by do image->cinder vol->edit properties->image
20:19:28 <markwash> my assumption is that every key that is ever protected in the property_protections config would be treated by cinder as immutable, by virtue of not being something that nova specifically knows or cares about
20:19:32 <markwash> I think I'm being unclear
20:19:38 <markwash> we also need to timebox this a little bit
20:19:51 <markwash> so perhaps, bringing it up, it would be helpful if more folks weighed in on the ML?
20:20:31 <DuncanT> markwash: I think people want to be able to edit any key... anything that you might want to put in a glance property can change if you change the content of the image
20:20:42 <jokke_> DuncanT: so instead of askingg Glance every time "Is it ok to change this property?" Rather asking Nova "What you really must be able to change?"
20:20:59 <markwash> when are we changing the content of the image? sorry I missed that somehow
20:21:03 <TravT> Property protections aren't necessarily about immutability.  It is about further restricting the people who can edit them.
20:21:21 <DuncanT> markwash: Sorry, volume in this case
20:21:48 <markwash> DuncanT: but if you want to make big changes, you can just create a new volume that copies the data from the existing one and then modify it, right?
20:22:06 <rosmaita> i really think we want a separate config for cinder protected props, it's up to the operator to keep them consistent (or not, depending on use case)
20:22:32 <DuncanT> markwash: Currently the only way to get any glance property type thing into cinder is from glance...
20:22:50 <fmaldonado_> What if I want to change the image kernel id of the volume?
20:23:18 <markwash> so what I'm really proposing is that we treat volume-image-metadata as immutable, but. . .
20:23:29 <markwash> we make a whitelist of properties that can be edited through this extension
20:23:34 <TravT> I think the two use cases got intermingled.  We want to be able to update the metadata on volumes.  We need to honor the specific property proections that the provider wants.
20:23:34 <DuncanT> rosmaita: That seems reasonable to me, the question is then is the protected property enforcement code in a suitable state to be shared with cinder, and if not does anybody object to that being changed
20:23:52 <DuncanT> markwash: How do you generate that white list and keep it up to date?
20:24:08 <markwash> its static in cinder code for now, and is edited through patches submitted to gerrit
20:24:11 <markwash> that's what I was thinking, anyway
20:24:33 <TravT> markwash: I think that misses the intent of property protections.
20:24:39 <markwash> its not elegant, but I guess I'm not feeling like elegant is a place we can get to from here :-)
20:25:10 <TravT> property protections are for operators who deploy openstack to be able to specify additional rbac on specific properties they don't want end users changing.
20:25:28 <DuncanT> markwash: I think that misses the point and provides a poor solution
20:25:58 <DuncanT> markwash: Sufficiently poor I'd -2 any attempt to put that into cinder
20:27:02 <markwash> I guess the place where I'm being a little confounded is on this particular type of collaboration between glance, cinder, and nova
20:27:29 <DuncanT> markwash: Not sure nova needs any chagnes?
20:27:50 <TravT> rosmaita: I'm starting to lean towards your proposal of just having the same config.
20:27:52 <markwash> and it feels more and more like cinder needs to either adopt a similar mechanism independently of glance, or we need a different way to handle the overall use case of image-attribute-like components on a mutable thing
20:28:20 <markwash> s/thing/volume/
20:28:49 <TravT> DuncanT: I don't see Nova needing to change.
20:29:21 <markwash> I'm definitely +2 on cinder adopting a similar mechanism to property protections and copying the config over operationally
20:29:30 <DuncanT> Cinder having a copy of the config file works, but then we'd like to share some of the enforcement code from glance (otherwise we're likely to be running into bugs galore), so is glance open to any code cleanup that might be needed to allow cinder to share the code? (periodic sync should do fine for sharing)
20:29:32 <markwash> I'd love to have a better answer, but that one incurs no extra debt from my perspective
20:29:51 <markwash> code cleanup is good :-)
20:29:59 <jokke_> +1 for cleanup
20:30:15 <markwash> its probably a good idea to have some reasonably good cross-openstack way of implementing some protections around such properties
20:30:33 <markwash> its really necessary in the world of putting semantic meaning in user-driven metadata
20:31:10 <DuncanT> Cinder gave users their own sematic free namespace for metadata, and operators a different one :-)
20:31:26 <markwash> yeah, I can't remember why we didn't do that in Glance v2
20:31:28 <markwash> :-)
20:31:42 <DuncanT> markwash: I know it was suggested
20:31:52 <DuncanT> markwash: But you'd ahve to ask stuart for details
20:31:58 <DuncanT> of the history
20:32:10 <TravT> i read the full specs, and i recall it saying "let's just do the simplest possible thing and see how far it gets us."
20:32:12 <fmaldonado_> The only scenario that nova needs any change is if we want to make volume metadata being available for scheduling purposes. But that is not what volume meta is for, right?
20:32:30 <DuncanT> Ok, cool so if code & conf sharing looks like a reasonable plan then we're good to try that
20:32:45 <rosmaita> +1
20:32:47 <markwash> cool
20:32:54 <DuncanT> fmaldonado_: volume glance meta is for all of the same things glance properties are for
20:33:22 <markwash> seems like a positive approach for everybody, and glance just needs to be ready for some patches that might move things around some
20:33:33 <markwash> okay
20:33:36 <markwash> next up, if I may
20:33:45 <markwash> #topic tempest tests improvements update
20:33:48 <TravT> minor question to DuncanT
20:34:01 <TravT> Should facundo put all of that in one blueprint?
20:34:01 <markwash> #link https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Glance_Gap_Coverage
20:35:05 <nikhil___> markwash: so, after some discussion on that in openstack-qa, the change seems straighforward unless that wiki asks for more coverage (which from the words of it, does not)
20:35:17 <markwash> nikhil___: great to hear!
20:35:30 <markwash> yeah I think the core thing is the one missing bit mentioned in the wiki
20:35:48 <markwash> nikhil___: however, it would be great if you could share some of your insights so that we can push forward with better v2 testing in tempest
20:35:51 <nikhil___> and the solution which we planned to implement is to use a file path with real image data in a CONF
20:36:04 <markwash> personally I couldn't even find the test that was using non-binary data
20:36:08 * markwash facepalms
20:36:40 <markwash> nikhil___: that solution sounds good
20:36:45 <markwash> okay, that was a quick one I think
20:36:54 <nikhil___> markwash: saw one test in a link shared bu mtreinish and it showed using image data as '*' X times
20:37:21 <markwash> again, the goal for this is juno-2
20:37:26 <markwash> let's check in on it again next week
20:37:38 <markwash> #topic Graffiti status update
20:37:47 <markwash> TravT: hi again! :-)
20:37:59 <TravT> Just want to say that we have code started in multiple areas: database schema, rest APIs, horizon, and the volume metadata discussion.
20:38:09 <TravT> So, we hope the spec goes through. We are hoping that in the process of code reviews that we're able to make changes as we go, because getting a perfect spec up front will just stall us out.
20:38:11 <markwash> I have this guilty feeling like we have not been providing you with enough -spec reviews yet
20:38:27 <TravT> in progress code should start showing up in gerrit soon.
20:38:41 <TravT> Lakshmi also can report on pecan and wsme.
20:38:50 <TravT> He had action item last week
20:39:03 <markwash> TravT: yeah, I think the spec is mostly about sharing vision, there are a large number of items we need to leave off for coding and code review
20:39:24 <TravT> markwash: great!
20:39:25 <markwash> rosmaita, arnaud, will you guys be available for this weeks drivers meeting?
20:39:32 <arnaud> yes
20:39:43 <lakshmiS> markwash: Regarding the action item from last week, I created an API in glance for metadata object which uses WSME by modifying the existing RequestSerializer and RequestDeserializer
20:39:45 <arnaud> let's focus on the big 2 (metadata+artifact) tomorrow
20:40:00 <markwash> arnaud: +1, I will try to give them some time tonight so we are ready
20:40:16 <rosmaita> markwash: arnaud: me too
20:40:18 <markwash> I think we know these specs will make it in some form, so I'd like to see us try to have them landed by next week
20:40:33 <arnaud> agreed
20:40:40 <rosmaita> +1
20:40:41 <markwash> maybe we find out we need a little more time, but we should be aggressive to try to best enable work to land in Juno
20:40:45 <markwash> :-) yay
20:40:48 <TravT> Thanks!  We certainly expect to make revisions based on feedback during the coding process.
20:41:16 <markwash> lakshmiS: how did the wsme api experiment go?
20:41:35 <markwash> was it painful? how do you like the result?
20:41:44 <lakshmiS> markwash: It went well. Was able to use the model objects and WSME succesfully
20:42:23 <markwash> lakshmiS: as I see it, we've got two concerns we're trying to optimize for
20:42:27 <markwash> which are kind of related
20:42:43 <markwash> 1) we want to ease as much as possible the process of wedging your existing grafitti stuff into glance
20:42:55 <markwash> 2) we want to ease as much as possible the eventual transition to pecan across glance
20:43:01 <lakshmiS> markwash: Its quite easy and reduces a lot of code and makes it simple since model objects live outside
20:43:05 <markwash> I guess #1 is higher priority because its more immediate
20:43:32 <lakshmiS> markwash: I think it will be a natural fit for pecan + wsme since that's where graffiti model came from.
20:43:54 <markwash> cool
20:43:57 <lakshmiS> and also fits well within current glance code.
20:44:01 <markwash> great to hear
20:44:05 <TravT> markwash: to try to be clear, WSME fit in fine with current Glance.  Pecan, didn't.
20:44:12 <markwash> right
20:44:31 <markwash> because pecan is a wsgi application framework, where as wsme is a serialization library/framework
20:44:36 <TravT> yep
20:44:37 <lakshmiS> right
20:44:54 <markwash> okay, any other notes about graffiti for us today?
20:45:17 <TravT> nope
20:45:19 <TravT> Thanks!
20:45:30 <markwash> great, thanks guys
20:45:35 <markwash> #topic broken gate
20:45:38 <markwash> I already mentioned this one
20:45:52 <markwash> but again, there is a review I think we might be almost already covered on
20:45:54 <jokke_> markwash: nikhil___ pushed it through, thanks for both of you
20:46:04 <markwash> oh good
20:46:07 <markwash> good work guys
20:46:13 <markwash> thanks for adding it to the agenda
20:46:20 <markwash> #topic keystoneclient integration with glanceclient
20:46:23 <hrybacki> o/
20:46:28 <markwash> hrybacki: hi there!
20:46:34 <jokke_> markwash: timing was so handy ;)
20:46:47 <markwash> can you give us an explanation of this topic? I guess we've seen it a bit before
20:46:51 <markwash> hrybacki: ^^
20:46:52 <hrybacki> so I'm the new kid on the block over in keystone and I'm working with ayoung and jamielennox to get keystoneclient integrated with all the other components
20:46:57 <hrybacki> okay sure
20:47:12 <hrybacki> Basically, I wanted to check in on https://blueprints.launchpad.net/python-glanceclient/+spec/support-keystone-v3-api
20:47:18 <hrybacki> and see how we can help
20:48:05 <hrybacki> they're trying to standardize interaction with keystone via the client, using sessions, and making ssl everywhere possible
20:48:29 <jokke_> hrybacki: have you seen this https://review.openstack.org/#/c/82126/
20:48:34 <jokke_> ?
20:48:49 <hrybacki> nope -- looking now
20:49:31 <jokke_> hrybacki: it's lots, but that's the current change to bring v3 support to glance client end
20:49:52 <markwash> what parts of v3 would glanceclient be using?
20:50:39 <markwash> I'm more familiar with the CRUD-y parts of the v3 identity api
20:50:53 <markwash> which I'm guessing glanceclient doesn't use very often
20:50:54 <hrybacki> ideally it would be talking to a keystoneclient instance using its api
20:51:02 <markwash> ah
20:51:12 <hrybacki> single point of failure/fixes
20:51:17 <markwash> principally for token creation?
20:51:31 <hrybacki> and sessions
20:52:14 <markwash> ah okay I see
20:52:50 <hrybacki> meant to post this link, sorry -- https://etherpad.openstack.org/p/keystoneclient_integration_with_component_clients
20:53:31 <markwash> so it looks like we have a blueprint that was not really ever approved, but we also have thomas leaman's patch which might help?
20:54:04 <hrybacki> okay, is there anything we can do on our end to assist?
20:54:34 <markwash> I think some review on that patch would be good, to see how its shaping up and to see if there is anything more that needs to be addressed
20:54:36 <hrybacki> jamie has been doing a lot of work with the session stuff and the other clients -- ayoung is also working on porting it into horizon atm
20:54:42 <hrybacki> ++
20:55:14 <markwash> hrybacki: do you think you could help with that? I think you understand the goals very well and could help us there
20:55:35 <hrybacki> markwash: yep, jamie, ayoung, and I will do that
20:55:39 <markwash> great
20:56:05 <hrybacki> that's all I had, just sort of feeling out where everything was and seeing where we could help :)
20:56:06 <markwash> and then bug us a bit in #openstack-glance to help keep it moving
20:56:15 <hrybacki> ++
20:56:18 <markwash> ^^ that step is probably very necessary, unfortunately :-)
20:56:21 <markwash> cool
20:56:26 <markwash> all right then
20:56:29 <markwash> #topic open discussion
20:57:28 * markwash hears some crickets
20:57:46 <TravT> chirp, chirp
20:57:52 * markwash wonders how group H is doing
20:57:56 <markwash> okay
20:58:05 <markwash> thanks everybody!
20:58:06 <jokke_> thanks
20:58:09 <TravT> thanks!
20:58:10 <hrybacki> thanks!
20:58:18 <arnaud> 0/0
20:58:37 <jokke_> arnaud: you shouldn't divide by 0 ;P
20:58:42 <arnaud> :)
20:59:01 <markwash> #endmeeting