Wednesday, 2011-08-03

*** medberry is now known as med_out00:29
*** heckj has quit IRC00:36
*** jakedahn has quit IRC00:56
*** danwent_ has joined #openstack-meeting00:57
*** danwent_ has quit IRC01:01
*** salv has quit IRC01:10
*** littleidea has joined #openstack-meeting01:23
*** dragondm has quit IRC01:33
*** jakedahn has joined #openstack-meeting01:47
*** Cyns has joined #openstack-meeting02:31
*** Cyns has quit IRC02:31
*** ewanmellor has quit IRC02:33
*** martine_ has joined #openstack-meeting03:29
*** shwetaap has quit IRC03:37
*** martine_ has quit IRC03:43
*** dolphm_ has joined #openstack-meeting03:51
*** dolphm_ has quit IRC03:52
*** littleidea has quit IRC04:20
*** adjohn has joined #openstack-meeting06:01
*** termie has quit IRC06:01
*** ke4qqq has quit IRC06:02
*** termie has joined #openstack-meeting06:02
*** ke4qqq has joined #openstack-meeting06:02
*** royh has joined #openstack-meeting06:57
*** ewanmellor has joined #openstack-meeting07:07
*** ewanmellor has quit IRC07:14
*** darraghb has joined #openstack-meeting08:58
*** yamahata has joined #openstack-meeting10:49
jaypipesyamahata: morning/evening :)11:27
yamahatajaypipes, morning11:40
jaypipesyamahata: so, I got confirmation last night that s1rp, bcwaldon, and I think jkoelker will be here (in 15 mins)11:44
*** cbeck has quit IRC11:45
yamahatajaypipes, Great. Thank you for arranging the meeting11:46
*** cbeck has joined #openstack-meeting11:46
jaypipesyamahata: no problem :) sorry it took so long to set it up!11:47
blamarjaypipes: morning11:52
blamaryamahata: evening :)11:52
jaypipesblamar: morning! :)11:53
*** bcwaldon has joined #openstack-meeting11:53
jaypipesbcwaldon: morning :)11:53
bcwaldonjaypipes: hey there11:53
blamarbcwaldon, thanks for the coffee11:53
bcwaldonbcwaldon: you're welcome?11:53
bcwaldonblamar ^11:54
jaypipesbcwaldon: too early :)11:54
bcwaldonI don't envy CDT right now (s1rp)11:55
jaypipesbcwaldon: indeed.11:56
jaypipesall: I'm keeping notes here:
bcwaldonjaypipes: discuss here, notes there? I don't want to mess it up, it looks so pretty in one color ;)11:58
jaypipesbcwaldon: hehe, no worries. what would folks prefer? we can discuss here or there. I just wanted to make sure we took notes on everything...11:59
bcwaldonLet's discuss here, kinda unclear if all through etherpad. We can follow up there12:00
jaypipesyamahata has done a good job summarizing the discussion so far; that's what I pasted in that etherpad.12:00
bcwaldonhey there, s1rp12:01
yamahatas1rp, morning12:01
jaypipesso, it sounds like yamahata doesn't really care which approach is taken (he just wants to get it done :) ). I'm on the side of natural filters/Atom subresource. s1rp is on the side of JSON blob in existing custom properties.12:01
jaypipesbcwaldon, blamar: just off the cuff, where do you guys stand?12:02
bcwaldonI'd prefer anything generic over specific. Give me a sec and I'll look at the Atom idea. I also had the thought that we could wait until we break apart the image mover and registry, then we get complex properties for free12:03
bcwaldon...since we don't have to send data through headers12:03
jaypipesbcwaldon: could you explain that last part a bit more?12:03
jaypipesbcwaldon: ah, yes.12:03
jaypipesbcwaldon: that thought occurred to me as well.12:03
s1rpfiltering is still complex though12:03
jaypipess1rp: right.12:03
bcwaldondefinitely, I think it will have to be complex12:04
s1rpis filtering arbitrarily nested data-structures a hard requirement though?12:04
jaypipess1rp: I like the "naturalness" of an API like /images/<ID>/manifest/devices/12:04
blamarjaypipes: is it really the purview of glance to handle that though?12:05
jaypipesblamar: I hear you.12:05
jaypipesblamar: not sure. that's why we're having this discussion ;)12:05
*** chuck_ has joined #openstack-meeting12:05
jaypipesblamar: put another way... if Glance does *not* handle that kind of information, what does?12:06
blamarjaypipes: yeah, I'm still wondering what the *ideal* place for this data12:06
bcwaldonHonestly, this feels like a horribly complex feature that may not be top priority right now. Would it be crazy to suggest we use json blobs and ignore filtering for now?12:06
*** chuck_ has quit IRC12:06
blamarjaypipes: I deal in ideals a lot, and then try and work my way towards that slowly12:06
*** chuck_ has joined #openstack-meeting12:06
*** zul has quit IRC12:07
*** chuck_ is now known as zul12:07
*** zul has joined #openstack-meeting12:07
jaypipesyamahata: what's your opinion on what is more important: a) ability to easily search for images that have a specific block-device-mapping through the API or b) just storing the information so that it is attached in output of GET /images/<ID>?12:08
jaypipesok, that's a good guide.12:08
jaypipesso, if b) is the case, I think the natural solution would be the more generic one -- i.e. we shove a JSON blob into image_properties with a key of 'manifest' (or similar)?12:09
yamahataWhat I wanted is b. When I implemented, I generalized it a bit.12:09
s1rpif only some of the structured metadata needs to be filtered on, we could denormalize it: have structured metdata stored as a blob, and split out any thing we need to filter on as separate metdata properties12:09
jaypipess1rp: good point.12:09
s1rporiginally when we were thinking of OVF this was an idea... basically read the whole manifest in as a JSON or XML blob... keep that around if needed, but anything we *really* use would be duplicated in easy to access fields like `os_type`12:10
*** jkoelker has joined #openstack-meeting12:10
s1rpthat provides filtering, but doesn't lose the structured aspect, allowing us to recreate the OVF manifest if needed12:11
jaypipess1rp: so, given yamahata's use case (priority b) ), do you think that bcwaldon's point above about the split of move and registry in the API should be prioritized?12:11
jaypipess1rp: in other words, splitting the mover and registry in the APIs would essentially solve yamahata's main use case, as complex JSON could be returned in the call to the registry GET /images/<ID> that would not need to be converted into headers...12:12
s1rpjaypipes: yeah, that may be the simplest/fastest way to get him what he needs12:12
s1rpit has the bonus of also getting us closer to where need to be as well12:12
jaypipesyamahata: let me give you some background... so this makes more sense :)12:12
yamahatajaypipes, yes please. In fact I lost about spliting mover and registry.12:13
jaypipesyamahata: so, we had a conversation a couple weeks ago about the future of the Glance architecture and API. One problem we've run into by having a Swift-style API (one that attached metadata about an object/image in HTTP headers) is that a) you are limited in the amount of metadata you can put in HTTP headers and b) you cannot cache static image data separately from non-static image metadata...12:14
jaypipesyamahata: so we discussed splitting out the unified Glance API into a "mover" API node and a registry API node, with the mover API returning the image file itself on GET /images/<ID> and the registry returning a serialized representation of the image's metadata on GET /images/<ID>12:15
jaypipesyamahata: and we would get rid of the API calls to HEAD /images/<ID> and the use of HTTP headers to store image metadata.12:16
jaypipesyamahata: this actually won't be all that difficult to do, since, as you know, the registry is already a separate server daemon and the unified API server simply speaks to the registry through the registry client.12:17
jaypipesyamahata: however, this would of course break backwards compatibility, to we would need to bump the Glance API to a v2.0.12:17
jaypipesyamahata: let me know if this makes sense or if you need more explanation.12:17
yamahataI see. So we want metadata to be 1st class citizon from http header.12:18
yamahataThat makes much sense.12:18
yamahataAPI compatibility would be an issue.12:18
jaypipesyamahata: yes, we want no more metadata in HTTP headers. and the glance client would speak directly to the registry node (instead of the API node) to get metadata about an image.12:19
jaypipesyamahata: so... due to the API compatibility thing, we've been wondering when a "good" time to do this would be? ;)12:19
jaypipess1rp, bcwaldon, blamar: thoughts?12:20
s1rps1rp: if that approach will work for yamahata (i can't think of a reason it wouldn't), i'd favor it12:21
s1rpheh, talking to myself (it's early!)12:21
yamahatathe API would be implemented in server code. But when for nova to use.12:21
jaypipesyamahata: Nova would continue to communicate with Glance via the glance client.12:21
bcwaldonjaypipes: I'd say let's break it up asap12:22
jaypipesyamahata: so most of the code changes will be around the client communicating with both a registry and a mover node directly (instead of only talking to the API node as it currently does)12:22
yamahatajaypipes, right.12:22
jaypipesbcwaldon, s1rp: what kind of timeframe do you think we need to copmlete this?12:23
jaypipesblamar: want to hear from you on this, too..12:23
blamarjaypipes: no worries, I'm tired and sitting next to bcwaldon so I'm letting him do the talking12:23
jaypipesblamar: :)12:23
blamarjaypipes: really, whatever he says is just me dictating12:23
bcwaldonI'd say it could absolutely get done for d4. I could take a look at it this evening12:23
bcwaldonblamar: excuse me?12:24
s1rpjaypipes: famous last words, but i don't think it would be too difficult12:24
bcwaldonits not complex, there would just be a lot of things to nail down12:24
s1rpit's going to be time consuming testing end-to-end...12:24
jaypipess1rp: ok. well clearly, this would block all other work that touched the API.12:24
bcwaldonjaypipes: what else is going on with the api?12:25
jaypipesshould we put this as a proposal to the community?12:25
bcwaldonAt least say it's happening ;)12:25
jaypipesbcwaldon: authentication is ongoing12:25
* jaypipes will type up a draft proposal for what we want to do12:25
* jaypipes will email all people here to review the draft12:26
bcwaldonsounds awesome12:26
jaypipesOnce good, we'll ask not for suggestions, but more as a notification of what is coming?12:26
jaypipesin other words, a warning vs. a proposal?12:26
bcwaldonprobably a good direction12:26
jaypipesotherwise, the discussion could seriously derail...12:27
bcwaldonwe all know how that goes12:27
s1rpyeah, really worried about drawing this out...12:27
s1rpwe know we need to do this for a bunch of reasons (caching, single glance registry, OVF, structured metadata)12:27
s1rplet's just do it12:27
jaypipesyamahata: ok, so you are good with this decision? It would essentially mean that much of your work would be scratched :(12:28
bcwaldonwe can go ahead and get started on it, assuming community approval12:28
bcwaldonanybody here want to take a lead on this?12:28
jaypipesyamahata: I feel bad since you put so much effort into it :(12:28
yamahataI'm quite fine with the conclusion.12:28
jaypipesyamahata: ok, thank you!12:28
jaypipesbcwaldon: I think s1rp should take the lead on this.12:28
jaypipesrick, you ok with that?12:29
yamahataAnyway we can reach much better solution.12:29
s1rpill need to run it by comstud first to make sure this won't impact our team priorities too much, but yeah, would love to12:29
jaypipesyamahata: yes, I think long-term this will definitely pay off.12:29
jaypipess1rp: k. could you get with him this morning on that?12:29
s1rpjaypipes: will do12:29
jaypipesOK, expect to see a draft of the email to the mailing list sometime before noonish, EDT, today.12:30
jaypipesanything else anyone want to bring up?12:30
jaypipesalright, see y'all later!12:30
bcwaldonthanks, Jay12:31
yamahatajaypipes, thank you.12:31
s1rpthanks guys12:31
* s1rp trying to decide whether to snooze for another hour or just stay up12:31
* blamar zzzzzzzzz12:32
*** jkoelker has quit IRC12:32
s1rphehe, seeing as it will be 108 today, think ill enjoy the relatively cool 80 degrees out on my balcony )12:32
*** jkoelker has joined #openstack-meeting12:32
*** yamahata has quit IRC12:36
*** sandywalsh has joined #openstack-meeting12:40
*** bcwaldon has left #openstack-meeting12:43
*** edconzel has joined #openstack-meeting13:45
*** toobulkeh has quit IRC13:54
*** dragondm has joined #openstack-meeting14:04
*** annegentle has joined #openstack-meeting14:10
*** toobulkeh has joined #openstack-meeting14:20
*** dprince has joined #openstack-meeting14:33
*** Cyns has joined #openstack-meeting14:56
*** rnirmal has joined #openstack-meeting14:57
*** msinhore has joined #openstack-meeting15:04
*** msinhore has quit IRC15:33
*** ewanmellor has joined #openstack-meeting15:48
*** heckj has joined #openstack-meeting15:49
*** nati has joined #openstack-meeting15:51
*** Cyns has quit IRC15:51
*** msinhore has joined #openstack-meeting16:29
*** ewanmellor has quit IRC16:31
*** clayg has left #openstack-meeting16:35
*** msinhore has quit IRC17:06
*** mattray has joined #openstack-meeting17:06
*** jakedahn has quit IRC17:11
*** joearnold has joined #openstack-meeting17:19
*** darraghb has quit IRC17:20
*** dprince has quit IRC17:22
*** heckj has quit IRC17:25
*** jakedahn has joined #openstack-meeting17:51
*** dendro-afk is now known as dendrobates17:59
*** rnirmal has quit IRC18:00
*** rnirmal_ has joined #openstack-meeting18:01
*** edconzel has quit IRC18:05
*** jakedahn_ has joined #openstack-meeting18:17
*** jakedahn has quit IRC18:19
*** jakedahn_ is now known as jakedahn18:19
*** msinhore has joined #openstack-meeting18:30
*** jakedahn has quit IRC18:33
*** KnuckleSangwich has joined #openstack-meeting18:36
*** jakedahn has joined #openstack-meeting18:39
*** msinhore has quit IRC18:45
*** jakedahn_ has joined #openstack-meeting18:47
*** jakedahn has quit IRC18:48
*** jakedahn_ is now known as jakedahn18:48
*** nati has quit IRC18:55
*** nati has joined #openstack-meeting18:56
*** rnirmal_ has quit IRC19:02
*** rnirmal has joined #openstack-meeting19:02
*** edconzel has joined #openstack-meeting19:05
*** dendrobates is now known as dendro-afk19:06
*** mdomsch has joined #openstack-meeting19:11
*** msinhore has joined #openstack-meeting19:16
*** nati has quit IRC19:23
*** Cyns has joined #openstack-meeting19:31
*** cynb has quit IRC19:33
*** cynb has joined #openstack-meeting19:34
*** nati_ has joined #openstack-meeting19:50
*** toobulkeh has quit IRC20:34
*** sandywalsh has quit IRC20:38
*** littleidea has joined #openstack-meeting20:42
*** cynb has quit IRC21:00
*** jjm has quit IRC21:06
*** _cerberus_ has quit IRC21:07
*** mdomsch has quit IRC21:09
*** _cerberus_ has joined #openstack-meeting21:10
*** jaypipes has quit IRC21:11
*** Cyns has quit IRC21:11
*** cynb has joined #openstack-meeting21:15
*** jjm has joined #openstack-meeting21:19
*** jaypipes has joined #openstack-meeting21:20
*** dolphm has quit IRC21:25
*** jkoelker has quit IRC21:41
*** sandywalsh has joined #openstack-meeting21:47
*** nati_ has quit IRC22:12
*** primeministerp1 is now known as primeministerp|h22:41
*** mattray has quit IRC22:43
*** edconzel has quit IRC22:53
*** Cyns has joined #openstack-meeting22:58
*** msinhore has quit IRC23:02
*** Cyns_ has joined #openstack-meeting23:17
*** Cyns has quit IRC23:18
*** Cyns_ is now known as Cyns23:18
*** joearnold has quit IRC23:18
*** jakedahn has quit IRC23:19
*** Cyns has quit IRC23:31
*** msinhore has joined #openstack-meeting23:39
*** rnirmal has quit IRC23:42
*** littleidea has quit IRC23:55

Generated by 2.14.0 by Marius Gedminas - find it at!