20:00:02 #startmeeting heat 20:00:03 Meeting started Wed Nov 20 20:00:02 2013 UTC and is due to finish in 60 minutes. The chair is stevebaker. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:06 The meeting name has been set to 'heat' 20:00:13 #topic rollcall 20:00:14 o/ 20:00:15 greetings, y'all 20:00:17 o/ 20:00:20 o/ 20:00:20 hi all 20:00:27 o/ 20:00:37 hi 20:00:49 hi 20:00:50 o/ 20:00:53 o/ 20:00:57 o/ 20:01:32 #topic Review last meeting's actions 20:01:47 o/ 20:01:58 stevebaker re-organise software config blueprints 20:02:00 stevebaker to go through summit etherpads and make sure all blueprints are raised https://wiki.openstack.org/wiki/Summit/Icehouse/Etherpads#Heat 20:02:38 What I did to is triage every New blueprint. I closed some software config blueprints 20:02:49 well done 20:03:01 sounds like serious fun stevebaker 20:03:04 serious button pushing 20:03:08 :-) 20:03:10 ;) 20:03:24 mouse button broken imo :) 20:03:42 I made some arbitrary decisions on what to obsolete, what to target for icehouse-2, and what the implementation progress was for some of them, so feel free to correct anything you think was wrong 20:04:11 #topic Adding items to the agenda 20:04:19 anybody got anything to add? 20:04:34 #link https://wiki.openstack.org/wiki/Meetings/HeatAgenda#Agenda 20:04:36 not from me 20:04:55 oh man 20:04:57 5 min meeting? 20:04:57 meeting time, okay 20:05:19 radix: want to talk autoscaling? I'm sure you're not sick of that ;) 20:05:33 stevebaker: I think the discussion is going well in the mailing list 20:05:39 ok 20:05:41 I've punted the ball back over to zane's court ;) 20:05:41 we should have more bikeshedding imo :) 20:05:57 i have 1 q 20:06:07 radix: pretty sure it's back in your court ;) 20:06:10 have we all agreed on whether the as api will be a separate process? therve has already started the first patch 20:06:18 I saw a msg from steve b re integration into the main process vs separate process 20:06:22 was a decision made on this point? 20:06:39 zaneb: hmm, I thought I sent the last email 20:06:52 #topic autoscaling 20:06:54 radix: wait, you're right. I haven't hit send yet 20:06:58 zaneb: hah, ok :) 20:07:19 * zaneb hits send 20:07:19 sdake_: well, therve has already started writing a patch that puts it in different engine/api processes 20:07:28 radix: back in your court ;) 20:07:45 so it's not like it'll save us any time to integrate it into the main process at this point 20:07:47 zaneb: thanks :) 20:07:56 zaneb wants it to be a separate endpoint, separate everything is probably fine if therve is doing it that way 20:08:01 radix sounds like win to me 20:08:35 randall is here over shoulder 20:08:36 at least that gives us a little bit of room to keep working :) 20:09:02 anyway that's all I have for now, happy to continue discussing on the ML 20:09:12 btw, I think resource group is a useful thing, as scaling != autoscaling 20:09:19 what is reln with heat-native resource group? 20:09:36 sdake_: I have a question for randall about the resourcegroup 20:09:44 oh... one note, rackspace is launching their "auto scale" api today, the one based on Otter. just to assuage any fears, no plans have changed on basically dropping that in favor of heat auto scale once it solves those use cases 20:09:44 shoot (Randall) 20:09:58 (I just saw a tweet announcement come up 30 seconds ago) 20:10:19 will autoscaling use heat-native resource group? 20:10:22 zaneb: this is randall 20:10:36 sdake_: is that something that's supposed to be leading toward the autoscaling implementation, or... something else (please specify) 20:10:40 I think trove, savanna etc will be doing a lot of generating templates then running stack-update where the only thing that changes is the count of identical resources 20:10:45 mspreitz_too: the autoscaling API that we're working on for icehouse scales heat resources, if that's what you're asking 20:11:08 zaneb based on earlier discussions its what it says on the tin. an unopinionated group of similar resources 20:11:13 SpamapS: here? 20:11:37 it could certainly be used as the basis for a scaling group since its not opinionated about what those resources are 20:11:40 I am asking how https://blueprints.launchpad.net/heat/+spec/native-resource-group relates to autoscaling 20:11:56 randallburt: it seems like something we're just going to want to deprecate when we have the scaling API up and running... 20:12:04 mspreitz_too, not sure 20:12:09 mspreitz_too: I expect the existing Heat resources which do scaling groups to be reimplemented using the AS API, not sure about static groups such as resource-group 20:12:24 zaneb, why? what if I don't want that and just a static group of "things" 20:12:32 mspreitz_too: me too, that's what we're discussing at the moment 20:12:59 The multiplier of a static group can be changed with an UPDATE, right? So how is it static? 20:13:07 randallburt: I though we already agreed that it would be more about scaling in general and not just autoscaling 20:13:15 or AS group could be a special kind of resource group. it already does some of the things we talked about scaling group doing 20:13:16 randallburt: like we do with instancegroup 20:13:23 mspreitz_too: Each definition of it is static, rather than modified by heat 20:13:34 yeah, it's the policy that makes it "auto", you can certainly treat scaling groups just like resource groups I think 20:13:39 stack-update != autoscaling 20:13:48 radix yup 20:14:23 So I am getting the idea that autoscaling is auto mgmt of a 'static" resource group 20:14:59 mspreitz_too, could be, I haven't looked at therve's latest tbh 20:15:01 mspreitz_too: That may, possibly be an implementation option 20:15:05 randallburt: so, my concern is that this is a _new_ type of abstraction, and I am trying to push the idea of basing autoscaling on existing abstractions because past experience has shown us that there are a huge number of corner cases to deal with when you create a new abstraction for a nested stack 20:15:55 unfortunately I have to go (because I again forgot about the different meeting time in my timezone)... I'll read the log when I get back and keep up with the ML :) 20:16:11 see ya 20:16:12 zaneb understood, but there's no current native way to say "a group of all the same things" 20:16:26 so it was a missing implementation of a current abstraction 20:16:28 randallburt: right, but autoscaling will deliver that 20:16:39 without scaling semantics 20:17:12 randallburt: would you say that InstanceGroup is "without scaling semantics"? 20:17:12 can't this be a bit more hidden? 20:17:35 randallburt: because I wouldn't, but this seems the same 20:17:38 zaneb I suppose 20:17:48 like we have a generice resource attribute called "count" 20:17:53 do a stack update with a different size, and you have to scale it 20:18:29 zaneb, InstanceGroup is a server, his thing is generic 20:18:30 except that InstanceGroup handles all kinds of fancy cases, like rolling updates, correctly and this doesn't 20:18:41 I think we're arguing the semantics of "count" 20:18:46 asalkeld: I realise the reason for implementing it 20:18:59 update handles a different set of things then autoscaling 20:19:07 they seem like diffeferent things to me zaneb 20:19:12 so I am also a little uneasy about it 20:19:21 but not sure what 20:19:22 ^^ was from sdake 20:19:29 _ 20:20:27 november ftw 20:20:29 sdake_: if you have a count property and you change it during an update, you have to scale somehow 20:20:41 ya 20:20:52 unless you say replace on update only 20:21:02 yeah go make more 20:21:57 ok, but that's a result of stack update, not some other scaling mechanism 20:22:02 and its optional and could be static as a group 20:22:33 randallburt: right, and I think we want the autoscaling api to work well for that use case too 20:22:45 zaneb: I totally agree 20:23:14 what is a ZangMingjie? 20:24:07 uh, someones IRC handle. you just pinged him/her, so we'll soon find out :D 20:24:29 is it likely that the autoscaling api will use the native resource group? 20:25:04 seems a good fit to me, 20:25:04 that would make this argument moot 20:25:30 randallburt: from the perspective of today it looks like a useful feature, but from the perspective of end-of-icehouse it looks like a less featured/buggier version of a scaling API resource 20:25:32 stevebaker: I think it may be too inflexible, as ideally we want the complexity of rolling updates etc in the AS API, not the resource-group resource 20:25:45 how is it buggier? 20:26:13 so that's extending what's there shardy, it can be bult upon. 20:26:27 randallburt: we won't know until we know ;) 20:26:34 how can something be buggier than an thing that doesn't exist? ;) 20:26:50 randallburt: I thought the whole point of the AS API was to contain scaling related policy, rather than implementing it all in Heat 20:26:51 but it could evolve to provide the bits of rolling updates that make sense to put in that layer 20:26:58 zaneb my take is we dont really want autoscaling in heat directly if we can depend on a third party service for it 20:27:01 otherwise, what are we doing it for? 20:27:25 the main rationale why we couldn't in the past is because we needed autoscaling and there was no short path to integration with openstack 20:27:43 we have solved that problem, so lets make the software simpler by breaking out the autoscaling 20:27:49 sdake_, randallburt: seriously, don't you have two laptops? ;D 20:27:50 now that the political bs is out of the way 20:28:00 i have one :) 20:28:01 I say we park this for now. therve and radix need to be here, and the autoscaling implementation needs to be further down the track 20:28:02 stevebaker: if this were to become the basis for autoscaling, it would need to acquire all the features we have built into autoscaling, like rolling updates. it's a step backward in that sense 20:28:59 #topic icehouse-1 release 20:29:27 so i-1 is in 2 weeks! 20:29:35 eek 20:29:39 #link https://launchpad.net/heat/+milestone/icehouse-1 20:29:54 I've punted blueprints which have no sign of being started 20:30:24 looks pretty good 20:30:43 whats story with this https://blueprints.launchpad.net/heat/+spec/filter-stacks 20:30:45 102 bugs, lol 20:30:55 But more will need to be punted I think. I'm pretty sure all those blueprints have code in-flight, but some are quite dormant 20:31:06 can fix bugs tomorrow 20:31:09 always tomorrow :) 20:31:29 sdake_: I think there are patches up for that already 20:31:34 shardy cool 20:32:02 I think filter-stacks might need a bit of scrutiny. status is the only column that makes sense for exact match 20:32:14 created_at, updated_at needs a range 20:32:27 shardy and sdake_: we can update the status on that. patches were submitted. they need review, though 20:32:28 name needs a text search based index 20:32:41 ya just done 20:32:55 andersonvom: Ok, thanks, we can continue the discussion on the reviews then :) 20:33:30 #link https://blueprints.launchpad.net/heat/+spec/resource-support-status 20:33:31 andersonvom, arbylee: FYI the requirements of management-api were discussed at this weeks keystone meeting 20:33:32 #link https://blueprints.launchpad.net/heat/+spec/filter-resources-by-support 20:34:08 I'm trying to figure out the way forward related to tenantles or service-scoped role assignments to allow global requests 20:34:15 shardy: is there a summary/output of that meeting available? 20:34:30 (the conversation quickly escalated to scope creep galore, unfortunately) 20:34:46 zaneb: ^ are proposing that a resource's supported status be associated with a heat release version, eg DEPRECATED at 2013.1 20:34:46 #link https://blueprints.launchpad.net/keystone/+spec/tenantless-assignments 20:34:57 arbylee: see https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting for a link to the logs 20:35:08 * stevebaker wants less tentacles 20:35:11 shardy: thanks. we'll take a look at that 20:35:13 arbylee: checkout the keystone bp above 20:35:33 dolphm: Is your feeling that this is achieveable for Icehouse? 20:35:34 dolphm, shardy: thanks 20:35:41 stevebaker: yeah, I don't know about that. versions are pretty fluid. and operators can deploy any plugins they want 20:36:13 shardy: the bp above- yes. with scope creep- i'm not sure. it's a pile of complicated wishlist items that won't be consumed in icehouse 20:36:37 dolphm: Ok, lets see how it foes, thanks :) 20:36:43 s/foes/goes 20:36:48 zaneb: my objective is a single version of resource documentation, I can't think of any other way of documenting what works in what version without giving meaning to our release numbers 20:39:09 zaneb: an alternative will be a template guide maintained per major release. But in reality users will read the first template guide they find in google 20:39:41 from a downstream perspective separate docs per version would be preferrable 20:39:41 otoh they can query the api to get all of this info 20:39:53 zaneb++ 20:40:15 vijendar: there hasn't been a lot of action in https://blueprints.launchpad.net/heat/+spec/dbaas-trove-resource recently. Should I bump to i-2 20:40:35 sdake_: right, but ideally the latest docs should contain the history 20:40:38 ideally. 20:40:48 if latest docs contain history, wfm 20:40:55 but i'm not clear on how this is done 20:41:15 how can it contain the history if it doesn't mention release versions? 20:41:50 stevebaker: I was blocked on https://review.openstack.org/#/c/52137/ 20:41:52 stevebaker: yes, I was arguing your side of the case 20:42:01 * zaneb is on the fence 20:42:25 * stevebaker is about to tip zaneb back to the other side 20:42:58 zaneb: so if documentation mentions release versions, wouldn't that give value to a build_info API ;) 20:43:40 rofl 20:43:54 I'm out 20:43:57 * stevebaker drops the mic 20:44:04 I'm not against a build_info API 20:44:23 in fact, I think all of OpenStack should have one 20:44:25 I think documentation should relate to API or template versions, not a build 20:44:43 I have to be off - organize kids 20:44:54 later angus 20:44:57 as long as operators have to make a conscious choice as to what info they make available there 20:45:10 Or even a major version, but that's better than relying on a constantly changing build_info 20:45:37 ok, it looks like we can move forward on those in the review comments then 20:45:40 shardy: agreed 20:45:43 The thing is, when things stablize, we may not want to bump template format versions, or resource versions or whatever every major release 20:45:50 build_info is still pretty useful, though 20:45:55 Can anybody tell if this is complete? https://blueprints.launchpad.net/heat/+spec/oslo-db-support 20:45:55 shardy: but resource type docs are even *less* related to API and template versions than they are to build versions (where the link is already tenuous) 20:46:30 relying on buildinfo for resource info seems tenuous 20:46:30 andersonvom: now it's authenticated and optional, I'm not opposed to it, I just think for docs, we need something more concrete 20:46:50 After icehouse I may advocate putting our core resource types on a separate release cycle 20:46:50 zaneb: Sure, we have different versions for API, template, and resources 20:46:51 what is the downside of having multiple versions of the docs? 20:46:59 stevebaker brought up "google search" as one problem 20:47:02 any others? 20:47:08 those versions can have different granularity and cadence 20:47:54 zaneb: maybe "major version" is an appropriate cadence for resource versions, but what if a large subset doesn't change? 20:48:18 I certainly hope there won't be huge churn in resource schema every release 20:48:21 stevebaker: maybe we should just make the docs available through the api as HTML 20:48:30 stevebaker: that's the only reliable reference anyway 20:48:54 zaneb: on teh fly sphinx generation? 20:49:09 sounds like potential security issues with that 20:49:11 zaneb: That doesn't really help template authors who want/need to support things over time tho? 20:49:18 stevebaker: no, definitely not sphinx 20:49:23 ;) 20:50:03 writing html is easy, I have full confidence that somebody here could do it ;) 20:50:04 jamming the docs near the api is an interesting idea - maybe should be breought up as a cross-project object on ml 20:50:21 object/project 20:50:29 requiring that a browser does keystone auth may be tricky though 20:50:45 shardy: the only relevant information is "what will work on the clouds I want to deploy to". And the only way to reliably find that out is to ask them. 20:51:41 could just ask for content-type text/html on the current /resource_types 20:51:42 stevebaker: good point. we could use an unauthenticated endpoint, so long as it still does throttling &c. 20:51:44 zaneb: That is a major barrier to template portability tho, if folks can't specify a version when they author the template, and expect it to work, and keep working 20:51:46 seems simpler just to keep the docs offline tho zaneb 20:52:03 stevebaker: we'd like to support basic auth for the single domain deployments 20:52:19 stevebaker: or even support tokens in a basic auth header 20:52:43 we're almost out of time, any other blueprints in https://launchpad.net/heat/+milestone/icehouse-1 that people have concerns with? 20:52:53 stevebaker lgtm 20:53:11 shardy: agreed, but that problem is inevitable in OpenStack. AWS doesn't have that problem because there's only one of them. But we are deliberately giving operators the flexibility to make choices 20:53:19 #topic open discussion 20:53:26 I have a question about multi-region. Did you have time to read the discussion on ML? Are everybody okay with choosing the option (2) from Zane's diagram as a way to go? 20:53:29 7 minutes 20:54:17 bgorski: sounds good to me. btw I have locally implemented a clients.py heat() 20:54:25 sooo, build_info! while we also agree that the other openstack projects should/could have it too, isn't it possible that we pull it in and set the example for now? 20:54:39 zaneb: yeah, I just think, if we go to the effort of versioning resource schema, we probably have to give users a choice rather than a constantly expanding matrix of incompatibility 20:54:46 zaneb: it is a hard problem tho ;) 20:54:54 then, in the future, if other projects decide to join in, we can talk again and realign 20:55:05 stevebaker, What you mean by locally implemented? 20:55:24 bgorski: in my tree, no review yet 20:55:24 bgorski: he means he hasn't pushed the patch yet 20:55:46 ok good to hear that 20:56:05 when you will push it to review? 20:56:22 um, soon 20:56:42 hopefully before EOW 20:57:16 nice I'm looking forward to check it out 20:57:17 not that anything is gating currently anyway 20:58:02 zaneb, stevebaker, shardy: sooo, build_info! while we also agree that the other openstack projects should/could have it too, isn't it possible that we pull it in and set the example for now? and, if other projects decide to add it too, we can talk again an realign. how's that? 20:58:17 = ] 20:58:23 connectivity glitch hit last time I asked, so I will ask again. Steve Baker, are you implementing software config and, if so, how are you dealing with the circular dependency that Clint harps on? 20:58:42 andersonvom: didn't you just say that? :P 20:58:58 andersonvom: I think we're in favour, but zaneb's last review comment recommended getting the version from not heat.conf 20:59:04 yes, but I didn't ping anyone and no one seemed to have seen it =) 20:59:29 andersonvom: I think we can continue the discussion specific to the patch on the review 20:59:45 andersonvom: two concerns: 1) we might set a bad example, which would mean having to redo it, deprecate, migrate &c. 20:59:52 mspreitz_too: I'm working on a POC then will ask IBM for help. It will be implemented with a SoftwareConfig/SoftwareApplier resource pair 21:00:01 #endmeeting