20:00:28 #startmeeting heat 20:00:29 Meeting started Wed Nov 13 20:00:28 2013 UTC and is due to finish in 60 minutes. The chair is stevebaker. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:30 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:33 The meeting name has been set to 'heat' 20:00:37 #topic rollcall 20:00:39 hola 20:00:45 heeey 20:00:50 booyaa 20:00:52 here 20:00:59 hi 20:01:09 o/ 20:01:10 o/ 20:01:11 o/ 20:01:12 Hi 20:01:15 o/ 20:01:16 o/ 20:01:16 o/ 20:01:29 evening 20:01:35 hello 20:01:46 hi 20:02:03 hope you've all recovered 20:02:17 no actions last meeting, so 20:02:17 not even close 20:02:24 mostly. more lagged coming back than I was going over. 20:02:26 hi 20:02:32 #topic Adding items to the agenda 20:02:59 Anything to add, so it doesn't get lost in the rolling-scrum of Open Discussion? 20:03:16 #link https://wiki.openstack.org/wiki/Meetings/HeatAgenda 20:03:23 do you want to do any ODS retrospective? 20:03:32 post it here and I will add it 20:03:41 adrian_otto: I put an item on the agenda for that 20:03:43 adrian_otto: we have Summit review, BPs/actions and prioritization 20:03:47 adrian_otto: I think that's already on the agenda 20:03:50 Summit review, BPs/actions and prioritization 20:04:00 I see, tx. 20:04:04 do we want to talk about mistral? 20:04:14 Change scope of Orchestration program to "OpenStack Control Program" 20:04:14 sure 20:04:24 can probably cover that there 20:04:46 is there much to talk about at this point? I thought its mostly design/planning stages 20:04:54 #topic Icehouse API policy (keep adding stuff, or freeze v1 + version bump for new stuff?) 20:05:04 So I added this 20:05:29 i'm not a big fan of rolling new features into an existing api 20:05:32 Wanted to discuss our policy on API additions, and also the possiblity of a new version without the tenant in the path 20:05:35 so i'd suggest freeze 20:06:04 sdake: I'm tending to agree with you, but we're already seeing patches adding stuff to the existing API 20:06:04 what's wrong with new features in a version? 20:06:05 +1 for revving the api to remove the tenant so we can have sane management -type calls in Icehouse 20:06:11 currently we're not planning on any breaking changes, just additions. This suggests v1_1 20:06:11 so we need core to agree the review policy 20:06:20 -1 (until I get a reason) 20:06:24 randallburt: That is where I'm headed :) 20:06:29 yeah, I think additions should go in v1_1 20:06:39 wfm 20:06:43 obviously changing an api is really bad 20:06:46 what's wrong with additions in v1? 20:07:01 but adding a standalone new thing? 20:07:02 -0 on removing the tenant thing, I am unconvinced by the arguments in favor 20:07:04 zaneb: and the removing tenants should be a major bump or also in v1_1? 20:07:09 however, "Build information API endpoint" is something deployers might want to turn off, which implies we need extensions if we do that 20:07:13 mspreitz some people will end up with v1 without additions, some people will end up with v1 wit hadditions 20:07:21 how are they to know which additions to use without versioning? 20:07:25 the current url scheme is sub-optimal for the managment/monitoring type stuff we want to pull from the api. that's by def a contract breaking change. 20:07:38 I thought the API already has a way to query for extensions 20:07:42 shardy: if we did it... dunno, in theory it's manageable by just changing the endpoint? 20:07:45 sdake not the job of the api for that 20:07:51 zaneb: How do you do a request which e.g lists multiple tenants stacks if the path impicitly scopes the request? 20:08:08 new endpoints could potentially break old endpoints, which is why a new version would be safer 20:08:28 shardy: /global/stacks ? 20:08:39 zaneb: I'd like the request scope to be purely based on context, not path 20:08:51 why? 20:08:54 zaneb: Maybe that would work, seems less clean than GET /stacks 20:09:12 true 20:09:13 and the scope is decided by the roles in the request context 20:09:20 roles/projects/domains 20:09:37 actually, there's nothing stopping us from adding that alongside the tenant-scoped stuff, since it's just for the management api 20:09:46 the tenant in the url shouldn't be trusted anyway, so it makes little sense to me that it is there in the first place 20:09:56 andersonvom: agreed 20:09:57 andersonvom: agree 20:10:00 agree 20:10:02 "I'd like the request scope to be purely based on context, not path" <- this is the opposite of why HTTP was invented 20:10:02 GET /stacks?scope=global <- ? 20:10:12 this needs to be inline with other openstack api 20:10:40 perhaps we need to discuss this with nova et al? 20:10:51 zaneb: well we're implicitly tied to scoping the data we return based on the RBAC profile provided by keystone token validation 20:11:05 so why encode a subset of the stuff in the path? 20:11:06 they have different paths for admin actions 20:11:20 The *heat specific* stuff should be encoded in the path IMHO 20:11:27 stevebaker: IIRC everyone wants to move away from that tho 20:11:31 not characteristics of the user making the request 20:12:03 so we're supposed to be talking about API versioning, not designing v2 ;) 20:12:07 stevebaker: the keystone devs said "don't do that" re admin specific endpoints 20:12:13 stevebaker: fair point ;) 20:12:14 has caused them pain apparently 20:12:25 can we get back on topic for a bit? 20:12:49 stevebaker: sure, I'll follow up with ML/wiki discussion 20:13:06 we're going to have minor additions within the scope of a v1_1 *or* extensions 20:13:16 if every time we add a feature we put it in a new version, we'll have v100 soon 20:13:21 stevebaker: agreed, but I think it matters as to *why* we'd rev the api. is the question if we *should* in Icehouse? 20:13:34 asalkeld: we have v1_1 for the duration of icehouse 20:13:48 It could be argued that just adding stuff is OK 20:13:58 since when is the version related to the cycle? 20:14:08 and that a bump is only required when existing interfaces change 20:14:21 I think just adding stuff doesn't require a contract revision, as long as all the old calls work 20:14:23 we have continuous deployers 20:14:43 shardy: how do API consumers know if the API they are calling has the feature they need? 20:15:04 stevebaker: That is a good point, but it's not specific to Heat 20:15:05 stevebaker, you publish features/capabilties 20:15:07 they know via the version #:) 20:15:09 via the version number. 20:15:24 asalkeld: like an extensions introspection call? 20:15:33 well Neutron for example have been adding stuff throughout Havana, haven't they? 20:15:37 yeah, just not extensions 20:15:55 adding stuff without breaking old stuff doesn't need a new version, though. extensions are different and should be versioned independently IMO. 20:16:05 +1 20:16:10 asalkeld: why not extensions? 20:16:32 well we are not talking about 3rd party extensions 20:16:37 stevebaker: alternate answer: suck it and see 20:16:51 how do users figure out the features 20:16:54 stevebaker: I think its a slippery slope of compatibility hell. We see it in nova/cloud servers. 20:16:59 zaneb: discovery via 404 ;) 20:17:03 lol 20:17:17 What about changes in the HOT language? 20:17:27 not changes in the API signature 20:17:28 that's template versioning 20:17:31 mspreitz: define "changes" 20:17:31 mspreitz: different topic 20:17:41 OK 20:17:41 mspreitz: that is a separate discussion, it's encoded in the template format 20:18:22 So I'm looking forward to the answer to the question about why extensions are different 20:18:29 so could a v1.1 rev remain with a /v1/ path or would we have to make it /v1_1/? 20:18:48 stevebaker, not sure 20:18:51 we'd have to have a /v1.1/ path IMO 20:19:00 stevebaker: surely the paths have to differ? 20:19:03 stevebaker: you're supposed to respond to both I believe 20:19:13 but the version is discoverable (not sure why features can't be) 20:19:22 stevebaker: but luckily there's some middleware that handles it all for you 20:19:27 or something 20:19:39 seems like abusing the version IMO 20:19:58 the version middleware should sort it so that a call to the root of the service shows you what versions are available and what paths they have. 20:20:28 so 20min in 20:20:28 yep 20:20:38 Maybe we should move this discussion to the ML and document a policy on API revisions for next week 20:20:41 indeed 20:20:43 I believe the keystone catalog supports different versions as well 20:20:44 agreed 20:20:45 +1 20:21:05 I'll do this next, because it is related 20:21:07 #topic Build information API endpoint (https://review.openstack.org/#/c/55434/) 20:21:29 So I'm still not sure what actual value this provides 20:21:45 seems like questionable value with potential for abuse 20:21:47 other than providing a shortcut which avoids folks querying their CM tools managing Heat 20:21:47 I would find it useful, as long as the call was at least authenticated 20:21:58 easily identify what version of heat is deployed even if you don't have access to the infra its running on 20:22:13 I believe the latest patch makes it optional as well as authenticated. 20:22:19 randallburt: Why do you care, unless you're the deployer or an attacker? 20:22:33 or someone submitting a bug report 20:22:36 If you're the deployer, you have the info already, via some means 20:22:36 ya, the features are exported via the versioning of the api :) 20:22:40 I'd like template_guide to document what version resources and their properties were added (or deprecated) 20:22:48 or a support technician 20:23:02 that doesn't have access to the infra 20:23:07 stevebaker: in that case, the template would fail to validate with an invalid prop 20:23:22 or invalid versioning of the template 20:23:23 stevebaker: that's a different bp, though 20:23:27 you know, apache used to report it's version number in error pages and stuff 20:23:28 randallburt: I'm not saying don't provide the info, I'm saying, why does heat have to provide it? 20:23:28 the published version should not be too fine-grained - and deployers would often carry their own patches too 20:23:33 and a little off topic ;) 20:23:42 and now they don't do that because people use that info to hack it 20:23:44 shardy customers care. 20:23:47 they consume the API. 20:23:54 randallburt: my point is that the version becomes an interesting piece of information that a user will want to know 20:24:10 Support employees care, as we put out a new build, they would like to track that. they don't have access to the infrastructure, only the API. 20:24:27 kebray: Ok, so what version do you report, if you have a deployement scaled out running multiple versions of things? 20:24:35 keybray in this case, the version of the api changes 20:24:42 stevebaker: agreed. but that sort of supports having it. As a service provider, the information is useful 20:25:06 shardy: each version of the service would report independently 20:25:08 sdake: the api doesn't necessarily change if you push out deploys with different configuration options 20:25:08 shardy: the version should be from whatever engine responded to the version rpc call 20:25:21 shardy, that's the implementer to decide. e.g. at Rackspace, we might manually bump a version number in a file. Others may automate the version from a CI tool. If we return it as a string, it can have whatever info we want. 20:25:24 shardy: if you have multiple versions, that would be a useful case to actually find out that different info is being reported. 20:25:53 perhaps a different approach is a config option, rather then the git sha 20:26:01 to handle arblyee's case 20:26:23 OpenStack is not monolithic, so the version is not monolithic 20:26:32 I think it's ok if the deployer wants it 20:26:33 sdake: that would also be acceptable; that way the deployer can define what it meands 20:26:36 sdake agreed, I don't want a git sha. This number needs to be understandable by humans, and for our setup, we may choose to have it be an incrementing number. 20:26:54 ok well that approach wfm :) 20:26:54 s/meads/means/ 20:26:54 what mspreitz said. I don't see how this can be useful 20:27:05 asalkeld: how do deployers turn it off if it is not an extension? maybe a config option. 20:27:06 andersonvom: but what version do you report, the API you hit, the engine which processes the request, both, versions of the platform it's all running on etc etc? 20:27:22 yeah, config option 20:27:29 shardy: yes. when would they be different? 20:27:29 zaneb it's useful for us. I promise. API users need to know when our code has changed, even if nothing about the contract has changed. 20:27:30 kebray: if you as a cloud operator don't know what version of Heat you've rolled out, we really can't help you 20:27:51 zaneb: wait, what? 20:27:54 * sdake giggles at zaneb 20:27:58 zaneb I know what version.. the other 4000 employees don't know. I want an API call that allows them to get it. 20:28:18 alright, lets move this to the review/ml 20:28:23 ok. 20:28:25 randallburt: you could be running slightly different versions of heat during a live upgrade couldn't you? 20:28:27 zaneb: I think you need to take some anti-troll pills or something :) 20:28:34 shardy: I would totally expect that. 20:28:36 #topic Summit review, BPs/actions and prioritization 20:28:38 * sdake for once agrees with lifeless 20:28:40 randallburt: assume you plan to chase trunk or do rolling updates of some sort 20:28:45 shardy: fair point, but that's a very small windo 20:28:47 window 20:28:56 depends on the cloud size 20:29:04 and update frequency 20:29:12 new topic, lets move on 20:29:19 versions are useful, they can get captured in logs for automated post mortems - fin. 20:29:21 randallburt: but it's that window when things are most likely to break 20:29:55 I need to triage the blueprint list, but here is where we are at https://launchpad.net/heat/+milestone/icehouse-1 20:29:59 lifeless: +1, capture in logs, return in error message to (authenticated) users 20:30:46 any general thoughts on summit? 20:30:55 too many 20:31:00 indeed. 20:31:25 maybe best to let them filter out into the implementation over the next 6 mo 20:31:40 we need to go through the etherpads 20:31:53 and make blueprint / fill in current ones 20:32:00 I'm a little worried that options for software config have expanded, not reduced. But I still think that arguments need to be augmented with actual code from this point 20:32:16 stevebaker: +9001 20:32:28 stevebaker: do we have an umbrella BP showing the dependent BPs for software-config to happen? 20:33:06 shardy: I think the blueprints need to be completely rejigged to reflect our refined understanding 20:33:34 yeah, time consuming stuff 20:33:43 #action stevebaker re-organise software config blueprints 20:33:48 heavy is the head... 20:33:49 stevebaker: Ok, cool, just wondered if we've started a task breakdown for that in particular, since it seems the biggest chunk of work 20:34:30 stevebaker: I'll raise some and do a wiki related to the request scoping, trusts, admin, in-instance auth related stuff 20:34:37 shardy: 20:34:47 shardy: thanks 20:35:22 stevebaker, I can help with bp pruning too 20:35:26 What are the other big items, autoscaling, ...? 20:35:30 stevebaker, for software config which is the master BP? is it hot-software-config? 20:35:35 I'm still not clear on how the software config stuff will bootstrap 20:35:54 stevebaker: Yeah, get volunteers to help get things into shape or you'll go insane clicking in launchpad :) 20:35:58 so as per normal, people should be writing their own blueprints, and setting the Milestone Target based on what they think is achievable 20:36:41 stevebaker: My concern was taking the big items, breaking them up and getting folks commited to working on the bits 20:36:43 shardy, there was "stack stop_on_error" 20:37:00 that enabled debugging, and retry 20:37:04 and you don't *need* something to be approved to work on it, but you carry the risk of being smacked with a banhammer at the end of the process 20:37:11 we also should consider some of the incoming requirements from trove and savannah 20:37:36 randallburt: they know to raise blueprints 20:37:37 if sdake will ever let me use the banhammer… :P 20:38:12 randallburt, yeah we probably need a way of prioritizing those features 20:38:23 stevebaker: k. I'll ping hub_cap next week when I'm in SFO 20:38:28 shardy: I think all the big items have owners who can handle that, apart from software config 20:38:50 shardy: are there others you are aware of that need breaking down and delegating? 20:38:57 stevebaker: Ok, cool, as long as you're happy that's the case and know who they are :) 20:39:36 radix, randallburt, autoscaling is big, let us know if help is needed 20:39:56 hi lo randallburt ill be around 20:40:01 yes we need to get rolling on autoscaling 20:40:04 stevebaker: No, we just discussed a lot last week so wanted to make sure we had owners for all the major items 20:40:21 #action stevebaker to go through summit etherpads and make sure all blueprints are raised https://wiki.openstack.org/wiki/Summit/Icehouse/Etherpads#Heat 20:40:27 maybe we can blog about summit? 20:40:34 to clarify ideas 20:40:40 stevebaker: will do 20:40:58 moving on? 20:41:02 yeah 20:41:06 yup 20:41:08 #topic Change scope of Orchestration program to "OpenStack Control Program" 20:41:23 here there be dragons. 20:41:23 where did this come from? 20:41:26 this was mostly SpamapS idea, but since he isn't here, I'll suggest it 20:41:31 yes dragons indeed :) 20:41:40 do tell, why is this a good idea? 20:41:41 Why? 20:41:42 my opinion: Orchestration already means all things to all people, so we just need to redefine what *we* mean by "Orchestration" ;) 20:41:54 his argument was mostly of the line that "we are doing autoscaling in heat, and we do orchestration" 20:41:54 stevebaker ++ 20:42:07 stevebaker: +1. Let solum be the new dumping ground 20:42:17 SpamapS, also was aiming at mistral I think 20:42:19 and there are other projects coming online such as qonos and mistral 20:42:37 i think his basic idea was to expand our scope to cover all the things ;) 20:42:37 asalkeld: so you think Mistral should be brought into our programme? 20:42:40 and "control program" is clearer? 20:43:15 stevebaker, if we are going to use it for our core flow control 20:43:22 it would be nice 20:43:22 mistral controls workflow 20:43:30 autoscaling api controls autoscale 20:43:31 wasn't the longer term goal to move autoscale *out* of heat, though? 20:43:36 heat controls the rest of the stack 20:43:46 they are all related, in some nebulous way 20:43:55 I am not sure on this move 20:43:55 randallburt: autoscaling will still be in the orchestration program though 20:44:03 we orchestrate autoscaling. autoscale doesn't necessarily have to be in orchestration. 20:44:03 one problem it would solve is "how to depend on mistral in an integrated program" like heat 20:44:05 stevebaker: fair point 20:44:29 #topic Mistral 20:44:33 autoscaling is going to be deeply connected to orchestration, they way we are planning to implement it 20:44:35 I don't want us to look like we are trying to take over openstack projects 20:45:02 but if those projects are keen, I have no problem 20:45:07 asalkeld: if they belong in orchestration we should make the case, like tusker merging with tripleo 20:45:19 I'm not convinced Mistral as a solid non-changing clear mission yet. So, shouldn't spin too many brain cycles on it just yet IMO. 20:45:24 I don't want us to be a shortcut to integrated status though 20:45:45 stevebaker: +1, but we shouldn't redefine our scope to include totally non-orchestration related projects 20:45:50 no that wouldn't be the intent 20:45:54 I think we can defer this for a while 20:45:54 clearly we don't want to present an opportunity to game the system 20:45:57 shardy: agreed 20:46:20 but mistral is related in some way i can't put my finger on to what heat does :) 20:46:33 asalkeld: anything technical about Mistral you can present yet? 20:46:35 randallburt: in theory new projects have to be incubated, even if part of established programs 20:46:44 so it's in design 20:46:45 zaneb: k 20:47:03 asalkeld: do you want to participate in that, so heat 20:47:06 we would need to build it (not head devs) 20:47:08 's needs are represented 20:47:20 sure 20:47:33 I can make sure we get what we need 20:47:44 not sure of the time line tho' 20:47:48 #topic Open discussion 20:47:53 they ahave a roadmap 20:48:08 long and winding road 20:48:12 https://blueprints.launchpad.net/heat/+spec/namespace-stack-metadata 20:48:21 so, does this latest resolution change our status in any way other than calling ourselve's Openstack Orchestration (™)? 20:48:22 looking for feedback on namespacing stack-metadata 20:48:36 well to do such a change, would require tc approval 20:48:41 which would require board approval 20:48:48 which means steve baker would have to sell it 20:48:50 tims: is this for horizion presentation hints? 20:48:57 and I assume we would all have to stand together to agree with the apporach 20:48:58 stevebaker: yes 20:49:06 For those interested https://etherpad.openstack.org/p/MistralRoadmap 20:49:07 as well as other things 20:49:09 lets not call it metadata, please! ;) 20:49:12 randallburt: not at all 20:49:15 stevebaker: lol 20:49:19 zaneb: k 20:49:23 I'm not attached to the name :) 20:49:46 just a place where we can safely put things that Heat should ignore in the template 20:49:47 Yeah metadata is already thrice overloaded 20:49:54 stevebaker extra data? 20:50:18 tims: Can it just be a comment block? 20:50:30 tims: or do you need it parsed and available via the API? 20:50:30 tims: so maybe a top-level template section which is ignored by heat engine 20:50:34 shardy: oh the looks tims gave me when I suggested that. 20:50:43 shardy: it needs to be structured, for things outside heat to parse 20:50:44 haha randallburt 20:50:52 OK 20:50:57 besides, comments are lost when safeloaded 20:51:00 yeah it should be a top level section 20:51:04 metadata is supposed to be overloaded... it means data about other data. It applies everywhere. 20:51:27 stack level Metadata? 20:51:37 imagine a visual template authoring tool, x,y coordinates of where resources are dragged need to be stored somewhere 20:51:44 to be inline with resource Metadata 20:51:45 ok, cube conversations just convinced me. Just call it data :-) 20:51:46 in the context of Horizon, its actual data though 20:51:50 kebray: but the template terminology should be distinct, because we already have metadata blocks which actually do stuff 20:52:10 kebray: if we had 50 sections called 'data', I would complain about that too ;) 20:52:12 asalkeld: there is no resource metadata in HOT (not exposed anyway) 20:52:29 stevebaker, cool:) 20:53:08 soo stack-metadata then? 20:53:11 zaneb what should we call it? 20:53:25 stevebaker: Are you sure? We have a Metadata block in CFN templates? 20:53:28 tims: so, I think the answer is yes, but we need to bikeshed the name. I think I'd prefer something top-level rather than sprinkling data throughout the template 20:53:39 stevebaker: agreed 20:53:44 stevebaker: agreed 20:53:58 stevebaker agreed 20:54:20 sounds ok 20:54:26 omg, we agreed on something! 20:54:36 :-O 20:54:42 6 minute4s 20:54:42 :) 20:54:52 https://github.com/openstack/heat-templates/blob/master/hot/F18/WordPress_NoKey.yaml#L93 20:54:53 tims is a consensus builder 20:54:55 stevebaker: ^^ 20:55:12 kebray: sorry, that was supposed to be a reply to the earlier comment about 'metadata' being intentionally overloaded 20:55:45 * kebray says no worries zaneb 20:55:48 shardy: HOT doesn't explicitly ban Metadata yet, but it is not part of the spec and there is no alternative until some bare-bones software config is implemented 20:55:57 oh no 20:56:12 *shakefist* time zoooooones! :( 20:56:15 radix: time change getcha didn't it 20:56:22 yes :( 20:56:33 DST can die in a fire 20:56:54 move to arizona - we roll on UTC :) 20:57:11 so, I'm prepping a proposed api spec (actually I started it a long time ago but never published it) for autoscaling 20:57:29 I'll post it to the list when I've put it up for comment 20:57:37 capital 20:58:26 lets mosey in a relaxed fashion back to #heat and have a friendly conversation 20:58:33 #endmeeting