20:00:02 <stevebaker> #startmeeting heat
20:00:03 <openstack> Meeting started Wed Nov 20 20:00:02 2013 UTC and is due to finish in 60 minutes.  The chair is stevebaker. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:06 <openstack> The meeting name has been set to 'heat'
20:00:13 <stevebaker> #topic rollcall
20:00:14 <asalkeld> o/
20:00:15 <zaneb> greetings, y'all
20:00:17 <shardy> o/
20:00:20 <sdake_> o/
20:00:20 <tspatzier> hi all
20:00:27 <tims1> o/
20:00:37 <vijendar> hi
20:00:49 <jpeeler> hi
20:00:50 <adrian_otto> o/
20:00:53 <mspreitz> o/
20:00:57 <lakshmi> o/
20:01:32 <stevebaker> #topic Review last meeting's actions
20:01:47 <bgorski> o/
20:01:58 <stevebaker> stevebaker re-organise software config blueprints
20:02:00 <stevebaker> stevebaker to go through summit etherpads and make sure all blueprints are raised https://wiki.openstack.org/wiki/Summit/Icehouse/Etherpads#Heat
20:02:38 <stevebaker> What I did to is triage every New blueprint. I closed some software config blueprints
20:02:49 <asalkeld> well done
20:03:01 <sdake_> sounds like serious fun stevebaker
20:03:04 <asalkeld> serious button pushing
20:03:08 <adrian_otto> :-)
20:03:10 <sdake_> ;)
20:03:24 <sdake_> mouse button broken imo :)
20:03:42 <stevebaker> I made some arbitrary decisions on what to obsolete, what to target for icehouse-2, and what the implementation progress was for some of them, so feel free to correct anything you think was wrong
20:04:11 <stevebaker> #topic Adding items to the agenda
20:04:19 <stevebaker> anybody got anything to add?
20:04:34 <stevebaker> #link https://wiki.openstack.org/wiki/Meetings/HeatAgenda#Agenda
20:04:36 <asalkeld> not from  me
20:04:55 <radix> oh man
20:04:57 <asalkeld> 5 min meeting?
20:04:57 <radix> meeting time, okay
20:05:19 <stevebaker> radix: want to talk autoscaling? I'm sure you're not sick of that ;)
20:05:33 <radix> stevebaker: I think the discussion is going well in the mailing list
20:05:39 <stevebaker> ok
20:05:41 <radix> I've punted the ball back over to zane's court ;)
20:05:41 <sdake_> we should have more bikeshedding imo :)
20:05:57 <sdake_> i have 1 q
20:06:07 <zaneb> radix: pretty sure it's back in your court ;)
20:06:10 <radix> have we all agreed on whether the as api will be a separate process? therve has already started the first patch
20:06:18 <sdake_> I saw a msg from steve b re integration into the main process vs separate process
20:06:22 <sdake_> was a decision made on this point?
20:06:39 <radix> zaneb: hmm, I thought I sent the last email
20:06:52 <stevebaker> #topic autoscaling
20:06:54 <zaneb> radix: wait, you're right. I haven't hit send yet
20:06:58 <radix> zaneb: hah, ok :)
20:07:19 * zaneb hits send
20:07:19 <radix> sdake_: well, therve has already started writing a patch that puts it in different engine/api processes
20:07:28 <zaneb> radix: back in your court ;)
20:07:45 <radix> so it's not like it'll save us any time to integrate it into the main process at this point
20:07:47 <radix> zaneb: thanks :)
20:07:56 <stevebaker> zaneb wants it to be a separate endpoint, separate everything is probably fine if therve is doing it that way
20:08:01 <sdake_> radix sounds like win to me
20:08:35 <sdake_> randall is here over shoulder
20:08:36 <radix> at least that gives us a little bit of room to keep working :)
20:09:02 <radix> anyway that's all I have for now, happy to continue discussing on the ML
20:09:12 <stevebaker> btw, I think resource group is a useful thing, as scaling != autoscaling
20:09:19 <mspreitz_too> what is reln with heat-native resource group?
20:09:36 <zaneb> sdake_: I have a question for randall about the resourcegroup
20:09:44 <radix> oh... one note, rackspace is launching their "auto scale" api today, the one based on Otter. just to assuage any fears, no plans have changed on basically dropping that in favor of heat auto scale once it solves those use cases
20:09:44 <sdake_> shoot (Randall)
20:09:58 <radix> (I just saw a tweet announcement come up 30 seconds ago)
20:10:19 <mspreitz_too> will autoscaling use heat-native resource group?
20:10:22 <sdake_> zaneb: this is randall
20:10:36 <zaneb> sdake_: is that something that's supposed to be leading toward the autoscaling implementation, or... something else (please specify)
20:10:40 <stevebaker> I think trove, savanna etc will be doing a lot of generating templates then running stack-update where the only thing that changes is the count of identical resources
20:10:45 <radix> mspreitz_too: the autoscaling API that we're working on for icehouse scales heat resources, if that's what you're asking
20:11:08 <randallburt> zaneb based on earlier discussions its what it says on the tin. an unopinionated group of similar resources
20:11:13 <stevebaker> SpamapS: here?
20:11:37 <randallburt> it could certainly be used as the basis for a scaling group since its not opinionated about what those resources are
20:11:40 <mspreitz_too> I am asking how https://blueprints.launchpad.net/heat/+spec/native-resource-group relates to autoscaling
20:11:56 <zaneb> randallburt: it seems like something we're just going to want to deprecate when we have the scaling API up and running...
20:12:04 <asalkeld> mspreitz_too, not sure
20:12:09 <shardy> mspreitz_too: I expect the existing Heat resources which do scaling groups to be reimplemented using the AS API, not sure about static groups such as resource-group
20:12:24 <randallburt> zaneb, why? what if I don't want that and just a static group of "things"
20:12:32 <zaneb> mspreitz_too: me too, that's what we're discussing at the moment
20:12:59 <mspreitz_too> The multiplier of a static group can be changed with an UPDATE, right?  So how is it static?
20:13:07 <zaneb> randallburt: I though we already agreed that it would be more about scaling in general and not just autoscaling
20:13:15 <randallburt> or AS group could be a special kind of resource group. it already does some of the things we talked about scaling group doing
20:13:16 <zaneb> randallburt: like we do with instancegroup
20:13:23 <shardy> mspreitz_too: Each definition of it is static, rather than modified by heat
20:13:34 <radix> yeah, it's the policy that makes it "auto", you can certainly treat scaling groups just like resource groups I think
20:13:39 <shardy> stack-update != autoscaling
20:13:48 <randallburt> radix yup
20:14:23 <mspreitz_too> So I am getting the idea that autoscaling is auto mgmt of a 'static" resource group
20:14:59 <randallburt> mspreitz_too, could be, I haven't looked at therve's latest tbh
20:15:01 <shardy> mspreitz_too: That may, possibly be an implementation option
20:15:05 <zaneb> randallburt: so, my concern is that this is a _new_ type of abstraction, and I am trying to push the idea of basing autoscaling on existing abstractions because past experience has shown us that there are a huge number of corner cases to deal with when you create a new abstraction for a nested stack
20:15:55 <radix> unfortunately I have to go (because I again forgot about the different meeting time in my timezone)... I'll read the log when I get back and keep up with the ML :)
20:16:11 <asalkeld> see ya
20:16:12 <randallburt> zaneb understood, but there's no current native way to say "a group of all the same things"
20:16:26 <randallburt> so it was a missing implementation of a current abstraction
20:16:28 <zaneb> randallburt: right, but autoscaling will deliver that
20:16:39 <randallburt> without scaling semantics
20:17:12 <zaneb> randallburt: would you say that InstanceGroup is "without scaling semantics"?
20:17:12 <asalkeld> can't this be a bit more hidden?
20:17:35 <zaneb> randallburt: because I wouldn't, but this seems the same
20:17:38 <randallburt> zaneb I suppose
20:17:48 <asalkeld> like we have a generice resource attribute called "count"
20:17:53 <zaneb> do a stack update with a different size, and you have to scale it
20:18:29 <asalkeld> zaneb, InstanceGroup is a server, his thing is generic
20:18:30 <zaneb> except that InstanceGroup handles all kinds of fancy cases, like rolling updates, correctly and this doesn't
20:18:41 <randallburt> I think we're arguing the semantics of "count"
20:18:46 <zaneb> asalkeld: I realise the reason for implementing it
20:18:59 <randallburt> update handles a different set of things then autoscaling
20:19:07 <randallburt> they seem like diffeferent things to me zaneb
20:19:12 <asalkeld> so I am also a little uneasy about it
20:19:21 <asalkeld> but not sure what
20:19:22 <sdkae> ^^ was from sdake
20:19:29 <sdkae> _
20:20:27 <sdake_> november ftw
20:20:29 <zaneb> sdake_: if you have a count property and you change it during an update, you have to scale somehow
20:20:41 <sdake_> ya
20:20:52 <zaneb> unless you say replace on update only
20:21:02 <angusss> yeah go make more
20:21:57 <randallburt> ok, but that's a result of stack update, not some other scaling mechanism
20:22:02 <randallburt> and its optional and could be static as a group
20:22:33 <zaneb> randallburt: right, and I think we want the autoscaling api to work well for that use case too
20:22:45 <randallburt> zaneb: I totally agree
20:23:14 <randallburt> what is a ZangMingjie?
20:24:07 <zaneb> uh, someones IRC handle. you just pinged him/her, so we'll soon find out :D
20:24:29 <stevebaker> is it likely that the autoscaling api will use the native resource group?
20:25:04 <randallburt> seems a good fit to me,
20:25:04 <stevebaker> that would make this argument moot
20:25:30 <zaneb> randallburt: from the perspective of today it looks like a useful feature, but from the perspective of end-of-icehouse it looks like a less featured/buggier version of a scaling API resource
20:25:32 <shardy> stevebaker: I think it may be too inflexible, as ideally we want the complexity of rolling updates etc in the AS API, not the resource-group resource
20:25:45 <randallburt> how is it buggier?
20:26:13 <randallburt> so that's extending what's there shardy, it can be bult upon.
20:26:27 <zaneb> randallburt: we won't know until we know ;)
20:26:34 <randallburt> how can something be buggier than an thing that doesn't exist? ;)
20:26:50 <shardy> randallburt: I thought the whole point of the AS API was to contain scaling related policy, rather than implementing it all in Heat
20:26:51 <stevebaker> but it could evolve to provide the bits of rolling updates that make sense to put in that layer
20:26:58 <sdake_> zaneb my take is we dont really want autoscaling in heat directly if we can depend on a third party service for it
20:27:01 <shardy> otherwise, what are we doing it for?
20:27:25 <sdake_> the main rationale why we couldn't in the past is because we needed autoscaling and there was no short path to integration with openstack
20:27:43 <sdake_> we have solved that problem, so lets make the software simpler by breaking out the autoscaling
20:27:49 <shardy> sdake_, randallburt: seriously, don't you have two laptops? ;D
20:27:50 <sdake_> now that the political bs is out of the way
20:28:00 <sdake_> i have one :)
20:28:01 <stevebaker> I say we park this for now. therve and radix need to be here, and the autoscaling implementation needs to be further down the track
20:28:02 <zaneb> stevebaker: if this were to become the basis for autoscaling, it would need to acquire all the features we have built into autoscaling, like rolling updates. it's a step backward in that sense
20:28:59 <stevebaker> #topic icehouse-1 release
20:29:27 <stevebaker> so i-1 is in 2 weeks!
20:29:35 <shardy> eek
20:29:39 <stevebaker> #link https://launchpad.net/heat/+milestone/icehouse-1
20:29:54 <stevebaker> I've punted blueprints which have no sign of being started
20:30:24 <sdake_> looks pretty good
20:30:43 <sdake_> whats story with this https://blueprints.launchpad.net/heat/+spec/filter-stacks
20:30:45 <shardy> 102 bugs, lol
20:30:55 <stevebaker> But more will need to be punted I think. I'm pretty sure all those blueprints have code in-flight, but some are quite dormant
20:31:06 <sdake_> can fix bugs tomorrow
20:31:09 <sdake_> always tomorrow :)
20:31:29 <shardy> sdake_: I think there are patches up for that already
20:31:34 <sdake_> shardy cool
20:32:02 <stevebaker> I think filter-stacks might need a bit of scrutiny. status is the only column that makes sense for exact match
20:32:14 <stevebaker> created_at, updated_at needs a range
20:32:27 <andersonvom> shardy and sdake_: we can update the status on that. patches were submitted. they need review, though
20:32:28 <stevebaker> name needs a text search based index
20:32:41 <sdake_> ya just done
20:32:55 <shardy> andersonvom: Ok, thanks, we can continue the discussion on the reviews then :)
20:33:30 <stevebaker> #link https://blueprints.launchpad.net/heat/+spec/resource-support-status
20:33:31 <shardy> andersonvom, arbylee: FYI the requirements of management-api were discussed at this weeks keystone meeting
20:33:32 <stevebaker> #link https://blueprints.launchpad.net/heat/+spec/filter-resources-by-support
20:34:08 <shardy> I'm trying to figure out the way forward related to tenantles or service-scoped role assignments to allow global requests
20:34:15 <arbylee> shardy: is there a summary/output of that meeting available?
20:34:30 <dolphm> (the conversation quickly escalated to scope creep galore, unfortunately)
20:34:46 <stevebaker> zaneb: ^ are proposing that a resource's supported status be associated with a heat release version, eg DEPRECATED at 2013.1
20:34:46 <dolphm> #link https://blueprints.launchpad.net/keystone/+spec/tenantless-assignments
20:34:57 <shardy> arbylee: see https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting for a link to the logs
20:35:08 * stevebaker wants less tentacles
20:35:11 <andersonvom> shardy: thanks. we'll take a look at that
20:35:13 <dolphm> arbylee: checkout the keystone bp above
20:35:33 <shardy> dolphm: Is your feeling that this is achieveable for Icehouse?
20:35:34 <arbylee> dolphm, shardy: thanks
20:35:41 <zaneb> stevebaker: yeah, I don't know about that. versions are pretty fluid. and operators can deploy any plugins they want
20:36:13 <dolphm> shardy: the bp above- yes. with scope creep- i'm not sure. it's a pile of complicated wishlist items that won't be consumed in icehouse
20:36:37 <shardy> dolphm: Ok, lets see how it foes, thanks :)
20:36:43 <shardy> s/foes/goes
20:36:48 <stevebaker> zaneb: my objective is a single version of resource documentation, I can't think of any other way of documenting what works in what version without giving meaning to our release numbers
20:39:09 <stevebaker> zaneb: an alternative will be a template guide maintained per major release. But in reality users will read the first template guide they find in google
20:39:41 <sdake_> from a downstream perspective separate docs per version would be preferrable
20:39:41 <zaneb> otoh they can query the api to get all of this info
20:39:53 <sdake_> zaneb++
20:40:15 <stevebaker> vijendar: there hasn't been a lot of action in https://blueprints.launchpad.net/heat/+spec/dbaas-trove-resource recently. Should I bump to i-2
20:40:35 <zaneb> sdake_: right, but ideally the latest docs should contain the history
20:40:38 <zaneb> ideally.
20:40:48 <sdake_> if latest docs contain history, wfm
20:40:55 <sdake_> but i'm not clear on how this is done
20:41:15 <stevebaker> how can it contain the history if it doesn't mention release versions?
20:41:50 <vijendar> stevebaker: I was blocked on https://review.openstack.org/#/c/52137/
20:41:52 <zaneb> stevebaker: yes, I was arguing your side of the case
20:42:01 * zaneb is on the fence
20:42:25 * stevebaker is about to tip zaneb back to the other side
20:42:58 <stevebaker> zaneb: so if documentation mentions release versions, wouldn't that give value to a build_info API ;)
20:43:40 <zaneb> rofl
20:43:54 <stevebaker> I'm out
20:43:57 * stevebaker drops the mic
20:44:04 <zaneb> I'm not against a build_info API
20:44:23 <zaneb> in fact, I think all of OpenStack should have one
20:44:25 <shardy> I think documentation should relate to API or template versions, not a build
20:44:43 <asalkeld> I have to be off  - organize kids
20:44:54 <sdake_> later angus
20:44:57 <zaneb> as long as operators have to make a conscious choice as to what info they make available there
20:45:10 <shardy> Or even a major version, but that's better than relying on a constantly changing build_info
20:45:37 <stevebaker> ok, it looks like we can move forward on those in the review comments then
20:45:40 <andersonvom> shardy: agreed
20:45:43 <shardy> The thing is, when things stablize, we may not want to bump template format versions, or resource versions or whatever every major release
20:45:50 <andersonvom> build_info is still pretty useful, though
20:45:55 <stevebaker> Can anybody tell if this is complete? https://blueprints.launchpad.net/heat/+spec/oslo-db-support
20:45:55 <zaneb> shardy: but resource type docs are even *less* related to API and template versions than they are to build versions (where the link is already tenuous)
20:46:30 <sdake_> relying on buildinfo for resource info seems tenuous
20:46:30 <shardy> andersonvom: now it's authenticated and optional, I'm not opposed to it, I just think for docs, we need something more concrete
20:46:50 <stevebaker> After icehouse I may advocate putting our core resource types on a separate release cycle
20:46:50 <shardy> zaneb: Sure, we have different versions for API, template, and resources
20:46:51 <sdake_> what is the downside of having multiple versions of the docs?
20:46:59 <sdake_> stevebaker brought up "google search" as one problem
20:47:02 <sdake_> any others?
20:47:08 <shardy> those versions can have different granularity and cadence
20:47:54 <shardy> zaneb: maybe "major version" is an appropriate cadence for resource versions, but what if a large subset doesn't change?
20:48:18 <shardy> I certainly hope there won't be huge churn in resource schema every release
20:48:21 <zaneb> stevebaker: maybe we should just make the docs available through the api as HTML
20:48:30 <zaneb> stevebaker: that's the only reliable reference anyway
20:48:54 <stevebaker> zaneb: on teh fly sphinx generation?
20:49:09 <sdake_> sounds like potential security issues with that
20:49:11 <shardy> zaneb: That doesn't really help template authors who want/need to support things over time tho?
20:49:18 <zaneb> stevebaker: no, definitely not sphinx
20:49:23 <stevebaker> ;)
20:50:03 <zaneb> writing html is easy, I have full confidence that somebody here could do it ;)
20:50:04 <sdake_> jamming the docs near the api is an interesting idea - maybe should be breought up as a cross-project object on ml
20:50:21 <sdake_> object/project
20:50:29 <stevebaker> requiring that a browser does keystone auth may be tricky though
20:50:45 <zaneb> shardy: the only relevant information is "what will work on the clouds I want to deploy to". And the only way to reliably find that out is to ask them.
20:51:41 <stevebaker> could just ask for content-type text/html on the current /resource_types
20:51:42 <zaneb> stevebaker: good point. we could use an unauthenticated endpoint, so long as it still does throttling &c.
20:51:44 <shardy> zaneb: That is a major barrier to template portability tho, if folks can't specify a version when they author the template, and expect it to work, and keep working
20:51:46 <sdake_> seems simpler just to keep the docs offline tho zaneb
20:52:03 <dolphm> stevebaker: we'd like to support basic auth for the single domain deployments
20:52:19 <dolphm> stevebaker: or even support tokens in a basic auth header
20:52:43 <stevebaker> we're almost out of time, any other blueprints in https://launchpad.net/heat/+milestone/icehouse-1 that people have concerns with?
20:52:53 <sdake_> stevebaker lgtm
20:53:11 <zaneb> shardy: agreed, but that problem is inevitable in OpenStack. AWS doesn't have that problem because there's only one of them. But we are deliberately giving operators the flexibility to make choices
20:53:19 <stevebaker> #topic open discussion
20:53:26 <bgorski> I have a question about multi-region. Did you have time to read the discussion on ML? Are everybody okay with choosing the option (2) from Zane's diagram as a way to go?
20:53:29 <stevebaker> 7 minutes
20:54:17 <stevebaker> bgorski: sounds good to me. btw I have locally implemented a clients.py heat()
20:54:25 <andersonvom> sooo, build_info! while we also agree that the other openstack projects should/could have it too, isn't it possible that we pull it in and set the example for now?
20:54:39 <shardy> zaneb: yeah, I just think, if we go to the effort of versioning resource schema, we probably have to give users a choice rather than a constantly expanding matrix of incompatibility
20:54:46 <shardy> zaneb: it is a hard problem tho ;)
20:54:54 <andersonvom> then, in the future, if other projects decide to join in, we can talk again and realign
20:55:05 <bgorski> stevebaker, What you mean by locally implemented?
20:55:24 <stevebaker> bgorski: in my tree, no review yet
20:55:24 <zaneb> bgorski: he means he hasn't pushed the patch yet
20:55:46 <bgorski> ok good to hear that
20:56:05 <bgorski> when you will push it to review?
20:56:22 <stevebaker> um, soon
20:56:42 <stevebaker> hopefully before EOW
20:57:16 <bgorski> nice I'm looking forward to check it out
20:57:17 <stevebaker> not that anything is gating currently anyway
20:58:02 <andersonvom> zaneb, stevebaker, shardy: sooo, build_info! while we also agree that the other openstack projects should/could have it too, isn't it possible that we pull it in and set the example for now? and, if other projects decide to add it too, we can talk again an realign. how's that?
20:58:17 <andersonvom> = ]
20:58:23 <mspreitz_too> connectivity glitch hit last time I asked, so I will ask again.  Steve Baker, are you implementing software config and, if so, how are you dealing with the circular dependency that Clint harps on?
20:58:42 <shardy> andersonvom: didn't you just say that? :P
20:58:58 <stevebaker> andersonvom: I think we're in favour, but zaneb's last review comment recommended getting the version from not heat.conf
20:59:04 <andersonvom> yes, but I didn't ping anyone and no one seemed to have seen it =)
20:59:29 <shardy> andersonvom: I think we can continue the discussion specific to the patch on the review
20:59:45 <zaneb> andersonvom: two concerns: 1) we might set a bad example, which would mean having to redo it, deprecate, migrate &c.
20:59:52 <stevebaker> mspreitz_too: I'm working on a POC then will ask IBM for help. It will be implemented with a SoftwareConfig/SoftwareApplier resource pair
21:00:01 <stevebaker> #endmeeting