20:00:06 <shardy> #startmeeting heat
20:00:07 <openstack> Meeting started Wed Sep 25 20:00:06 2013 UTC and is due to finish in 60 minutes.  The chair is shardy. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:11 <bgorski> o/
20:00:11 <openstack> The meeting name has been set to 'heat'
20:00:14 <shardy> #topic rollcall
20:00:20 <shardy> Hi all, who's around?
20:00:22 <adrian_otto> o/
20:00:26 <tspatzier> Hi
20:00:37 <randallburt> hithere
20:00:44 <spzala> Hi
20:00:46 <stevebaker> hi
20:00:48 <kebray> hello
20:00:56 <sdake> o/
20:01:06 <jpeeler> hey
20:01:16 <SpamapS> o/ but have to run in a few minutes to the dentist
20:01:17 <zaneb> o/
20:01:20 <sdake> pretty sure asalkeld is on pto
20:01:26 <SpamapS> made that appointment before my first commit to Heat I think. ;)
20:01:53 <shardy> Ok, lets get started
20:02:02 <shardy> #link https://wiki.openstack.org/wiki/Meetings/HeatAgenda
20:02:09 <mspreitz> I'm here too
20:02:15 <shardy> #topic Review last week's actions
20:02:27 <shardy> #link http://eavesdrop.openstack.org/meetings/heat/2013/heat.2013-09-18-20.00.html
20:02:43 <shardy> #info randallburt to repost review as WIP not draft
20:02:50 <randallburt> done
20:02:51 <shardy> So that got done, thanks randallburt
20:02:54 <randallburt> np
20:03:04 <randallburt> more input would be appreciated tho ;)
20:03:12 <shardy> anything else to raise from last week anyone?
20:03:29 <shardy> randallburt: Yeah, I think we'll have to work out the testr stuff in Icehouse
20:03:39 <randallburt> no issues for me there.
20:03:51 <shardy> #link https://review.openstack.org/#/c/46410/
20:04:00 <shardy> if anyone has any input ;)
20:04:13 <shardy> #topic RC1 bug status
20:04:19 <shardy> Soo
20:04:30 <shardy> We're planning RC1 at the end of this week ideally
20:04:41 <shardy> and we still have a pretty big backlog of bugs and reviews:
20:04:49 <shardy> https://launchpad.net/heat/+milestone/havana-rc1
20:05:16 <shardy> Unless we discover any more critical issues, that's the list, and when we get them all fix committed we'll branch RC1
20:05:26 <shardy> at which point master is open for Icehouse :)
20:05:32 <SpamapS> shardy: I'm assigning bug 1230191 to myself
20:05:34 <uvirtbot> Launchpad bug 1230191 in heat "Exceeding stack limits results in 500 errors" [High,Triaged] https://launchpad.net/bugs/1230191
20:05:41 <shardy> SpamapS: Ok, thanks!
20:05:50 <stevebaker> Do all those In Progress bugs have reviews in flight?
20:05:51 <SpamapS> shardy: will work on the patch post-dentist :)
20:06:16 <shardy> One thing to mention is the gate has been *really* flaky the last couple of days, so can we please nurse stuff through with reverify etc
20:06:24 <SpamapS> We need a status between "I'm looking at it" and "fix proposed"
20:06:25 <radix> oh, dang. sorry I missed the beginning of teh meeting.
20:07:18 <shardy> SpamapS: yeah, at this point pls mark in progress if you're actively working on it
20:07:31 <shardy> Anyone have anything to raise re RC1?
20:07:42 <stevebaker> here is a burndown chart http://old-wiki.openstack.org/rc/
20:07:58 <shardy> Other than the bugfixing and reviews, everyone please test all-the-things :)
20:08:06 <shardy> #link http://old-wiki.openstack.org/rc
20:08:10 <shardy> thanks stevebaker
20:08:27 <SpamapS> the Medium's may need to be dropped if we can't get patches landed
20:08:39 <stevebaker> yep
20:08:42 <SpamapS> IMO that is a _really_ big RC
20:08:50 <shardy> SpamapS: agree, but many of those have patches up, so ideally we'll just land them
20:09:06 <SpamapS> I'll try to make sure and run through some extra review cycles too
20:09:17 <shardy> SpamapS: Yeah, it's much bigger than ideal, as stuff ended up landing late and we've been behind testing as a result
20:09:20 <stevebaker> so is there an rc2 once rc1 is out?
20:09:29 <sdake_> if rc1 has bugs
20:09:32 <shardy> stevebaker: Only if we find critical bugs ;)
20:09:34 <sdake_> but generally no
20:09:50 <shardy> So the idea is RC1 is the release, unless it's not ;)
20:09:58 <stevebaker> ok
20:10:07 <sdake_> between rc1 and final release if critical bugs are found, an rc2 could be created from the rc1 branch
20:10:19 <shardy> Post RC1 any changes will be proposed via backports to milestone-proposed, I think
20:10:26 <stevebaker> ok, so its backports from rc1 on
20:10:28 <sdake_> scenario unlikley with our high level of test cases
20:10:37 <shardy> stevebaker: yup
20:10:53 <stevebaker> sdake_ I admire your optimism ;)
20:10:54 <shardy> sdake_: not that unlikely based on recent experience unfortunately
20:11:08 <sdake_> optimist is my middle name :)
20:11:15 <SpamapS> BTW if you are not fixing a bug right now, a good thing to do is to write tempest tests. Especially tests verifying error conditions... as we think we have botched that up a bit this cycle.
20:11:19 <sdake_> SOD
20:11:29 <shardy> Anyway, great work everyone we're nearly there, thanks for all being responsive to bugs and reviews etc :)
20:11:48 <stevebaker> SpamapS: +1!
20:11:58 <stevebaker> please try out tempest
20:12:03 <stevebaker> testr run orchestration
20:12:15 <shardy> SpamapS: yeah, the error paths (particularly between engine->API's have been poorly tested for a long time
20:12:31 <stevebaker> the autoscaling will probably fail, but that is most likely the test
20:13:24 <shardy> Ok any more concerns or questions re the Havana release before we move on?
20:13:56 * SpamapS must go now
20:14:01 <shardy> #topic Open discussion
20:14:08 <shardy> SpamapS: o/
20:14:39 <shardy> So one thing to mention, if you've not been following the ML is I'm stepping down as PTL
20:14:42 <mspreitz> Anyone want to follow up on the discussion I have been having in the ML?
20:15:01 <radix> wow that was quick
20:15:06 <shardy> stevebaker: has nominated himself as replacement, I've not yet seen any other candidates
20:15:30 <stevebaker> I think SpamapS might be?
20:15:34 <shardy> radix: The plan is to rotate the responsibility rather than one person being stuck with all the PTL overhead tasks ;)
20:15:37 <zaneb> shardy: SpamapS has also nominated himself
20:15:48 <radix> shardy: er, I meant the meeting, not the PTL position :)
20:15:54 <shardy> Ah I missed that, been battling trusts all day
20:16:03 <zaneb> still? ugh
20:16:20 <shardy> zaneb: yeah, unfortunately
20:17:03 <shardy> Ok, anyone else have anything to raise?
20:17:20 <mspreitz> The ML thread entitled "Bringing things together for Icehouse"
20:17:41 <radix> I don't have any input on that right now
20:18:02 <shardy> mspreitz: Did you generate the summary wiki page we discussed last week?
20:18:18 <shardy> I saw some google docs drafts
20:18:37 <mspreitz> Yeah, that and the ML traffic is all I have managed to get done so far.
20:18:43 <zaneb> I might reply on the ML at some point, not aware of anything that needs discussing here
20:18:58 <sdake_> ml seems like better forum for our essays on the topic
20:18:59 <mspreitz> Also made a design summit proposal in the nova track, but I think it's relavant to heat too.
20:19:13 <zaneb> I'm still not clear at all on why we are involved
20:19:31 <sdake_> zaneb because we know all the data
20:19:46 <shardy> mspreitz: I think continuing the discussion on the ML is fine for now, and we can discuss at summit, but FWIW, I'd like some concise, *specific* ideas about what you thing needs adding to heat
20:19:46 <sdake_> no other component has a systemic view
20:20:00 <shardy> rather than lots of very broad theories and ideas
20:20:16 <stevebaker> templates might need some holistic metadata, to be transformed into a normal heat template by the time it reaches heat
20:21:16 <mspreitz> OK, I will focus on writing more specifics.
20:22:02 <shardy> mspreitz: Ok, thanks, may help focus the discussion a little :)
20:22:25 <shardy> Anyone have anything else to discuss?
20:22:46 <stevebaker> I'll leave software-config talk until after rc1 is out
20:23:32 <shardy> stevebaker: k, sounds good, we can refocus on features/summit after rc1 is done
20:23:44 <mspreitz> I have another topic
20:23:54 <sdake_> maybe now would be a good time to write some docs ;)
20:23:56 <mspreitz> Can anyone give a TL;DR on the recent patches removing IDs?
20:24:09 <shardy> mspreitz: link?
20:24:21 <stevebaker> I haven't looked at them
20:24:28 <shardy> sdake_: we've been writing a lot of docs recently, but yeah more==good
20:24:31 <mspreitz> https://review.openstack.org/#/c/48239/ and many like that
20:25:08 <shardy> mspreitz: The id attribute is redundant, because it's the same value returned via Ref
20:25:19 <shardy> and it was causing some issues with template validation
20:25:43 <shardy> so we decided, before the release is final, it's best to remove the redundant attributes
20:26:01 <mspreitz> (has to read more before following up further)
20:26:48 <shardy> mspreitz: it is described in the linked bug
20:26:56 <mspreitz> yes, thanks on that for now
20:26:58 <shardy> #link https://bugs.launchpad.net/heat/+bug/1230228
20:27:01 <uvirtbot> Launchpad bug 1230228 in heat "Remove neutron resources id attributes" [High,In progress]
20:27:18 <shardy> sdake_: Did you have specific docs concerns to raise?
20:28:19 <sdake_> we have spoken in the past about taking 3-4 weeks and cranking out a set of docs
20:28:43 <sdake_> we have some technical design docs for devs and docs for the api
20:28:52 <sdake_> but beyond that, our docs aren't up to speed
20:29:11 * randallburt re-re-resurects his plug-in guide and frowns at himself.
20:29:38 <sdake_> each project has a user guide - we dont have that
20:29:45 <stevebaker> What are the general thoughts for a template writer's guide that describes the best way of doing everything now? It would be yaml cfn, with a mixture of aws and native resources.
20:30:00 <shardy> sdake_: It seems to have turned into a more gradual process, but I think the template guide is looking good now
20:30:02 <sdake_> we also dont have a template document that describes the properties and attributes of resources
20:30:11 <sdake_> link to template guide?
20:30:15 <randallburt> my vote is a template guide focused on HOT/YAML/native resources
20:30:17 <shardy> sdake_: Yes we do, that's what we've been doing!
20:30:17 <stevebaker> Alternatively we write a HOT template writers guide, which will only be useful when icehouse is out
20:30:26 <sdake_> i was traveling must hav emissed that
20:30:31 <randallburt> sdake_:  I thought that doc was generated now
20:30:56 <sdake_> anyone have a link?
20:31:00 <sdake_> so I can take a look pls
20:31:01 <stevebaker> we have this, which is all generated from source. Its not a guide, its a reference http://docs.openstack.org/developer/heat/template_guide/
20:31:23 <sdake_> cool when did that show up
20:31:27 <sdake_> nice work :)
20:31:27 <randallburt> I'm not sure there's a lot of value in re-documenting CFN is there?
20:31:31 <shardy> #link http://docs.openstack.org/developer/heat/template_guide/
20:32:01 <stevebaker> Which has this section, that looks like a good start for the HOT version of what I'm thinking of http://docs.openstack.org/developer/heat/template_guide/hot_guide.html
20:32:07 <radix> randallburt: I think there's a little bit of value. Not everyone knows about CFN, and there is still functionality that is only exposed as CFN-style resources
20:32:08 <shardy> stevebaker: tspatzier also wrote a HOT guide
20:32:23 <radix> randallburt: so if you're a newcomer to orchestration and you want to use Heat it's nice to have a single guide.
20:32:50 <randallburt> so insert link to CFN documentation? ;)
20:33:11 <mspreitz> Yes, I am looking for comprehensive and precise documentation of HOT, not just enough to understand the hello world example.
20:33:45 <radix> randallburt: well, I think the autogenerated-style stuff is good enough for most things. I wouldn't put *too* much effort into writing lots of prose for CFN resources
20:33:48 <stevebaker> so maybe as HOT and native features land in this cycle, we should insist that they come with additions to http://docs.openstack.org/developer/heat/template_guide/index.html
20:33:53 <tspatzier> shardy, the blog about writing HOT provider templates - would something like this be good input?
20:33:53 <shardy> #link https://github.com/openstack/heat/blob/master/doc/source/template_guide/hot_guide.rst
20:33:59 <radix> stevebaker: +1
20:34:08 <zaneb> mspreitz: HOT is a work in progress, there's not much more to it than the hello world example yet
20:34:08 <radix> stevebaker: I really like the idea of requiring doc updates with code updates.
20:34:11 <shardy> Where does that get generated to, anyone have a link?
20:34:30 <radix> shardy: stevebaker linked http://docs.openstack.org/developer/heat/template_guide/hot_guide.html
20:34:38 <tspatzier> radix, all the resources can already be used in HOT. So we could start promoting HOT already
20:34:46 <shardy> tspatzier: Yes, I'm writing a blog post which will cover HOT and Providers/Environments 101
20:34:48 <radix> tspatzier: yes, I understand.
20:34:53 <mspreitz> zaneb: the code disagrees; it handles lots of stuff
20:35:00 <shardy> tspatzier: should be next week, after we get rc1 done :)
20:35:22 <shardy> radix: thanks, I was getting the links for generated and written docs confused :)
20:35:43 <radix> hehe
20:35:44 <zaneb> mspreitz: it handles lots of stuff the same way as we have always handled it
20:35:47 <stevebaker> So maybe we should promote links to http://docs.openstack.org/developer/heat/template_guide/index.html since it is better than nothing
20:35:47 <tspatzier> shardy, let's see if we can bring something into the template guide. Maybe drive best practices and so on
20:36:03 <mspreitz> zaneb: sadly, it is only *mostly* like CFN handles it
20:36:13 <shardy> tspatzier: My plan is testing->blog->docs :)
20:36:18 <spzala> radix: Me too. I think doc should be get created in the same way, as as we create test with a patch, as needed
20:36:28 <tspatzier> btw, is there a way to see the actual documenation and not only the sources in github?
20:36:34 <zaneb> mspreitz: where you see incompatibilities, please do raise bugs
20:36:40 <tspatzier> I tried building it locally but was not successful
20:36:58 <mspreitz> zaneb: I think the differences are deliberate, not bugs.
20:37:04 <stevebaker> tspatzier: there is a docs job on check and gate, so you can always see what is generated
20:37:07 <zaneb> for instance?
20:37:31 <shardy> tspatzier: Yes, use rst for everything, not xml ;)
20:37:48 <tspatzier> stevebaker, do you have link?
20:37:57 <mspreitz> I last studied the code last week, before the constraint handling changed.  But at that time: constraint descriptions had no precise equivalent in CFN, and HOT allows multiple constraints of a given type for a given parameter whereas CFN does not
20:38:13 <stevebaker> tspatzier: click on gate-heat-docs Jenkins job for any gerrit review
20:38:14 <tspatzier> shardy, yeah, the rst is good. But it does not resolve e.g. references to other files
20:38:29 <tspatzier> stevebaker, thanks. got it :-)
20:38:43 <zaneb> mspreitz: right, understood. but those are the kinds of things that I thought appear in the Hello world example
20:39:07 <stevebaker> Finally I managed to watch an americas cup race, and we lost the cup
20:39:22 <zaneb> :(
20:39:24 <adrian_otto> mumble mumble Oracle mumble
20:39:49 <sdake_> if you have 40 billion you too can win the americas cup
20:40:11 <shardy> mspreitz: HOT is a rapidly evolving area, somewhere we can innovate, we're not restricting outselves to 1-1 mappings with CFN templates, otherwise what's the point in the new format?
20:40:39 <shardy> mspreitz: as you have noticed, things are still very much under development and not stablized yet ;)
20:41:06 <shardy> mspreitz: what we release for Havana should be basically useable tho, as a replacement for CFN templates
20:41:07 <stevebaker> I think it is too early to recommend HOT for anything yet, but I'd like to think that will change this cycle
20:41:31 <sdake_> once we recommend it - will be difficult to change
20:41:33 <mspreitz> My problem is that I set out to use HOT to get work done.  I needed to really understand it.  Even simple things like "and other XXX as in CFN are allowed here", if it is missing, is a problem for a user
20:41:39 <shardy> stevebaker: I think we promote it, for Havana, as a preview of a new template format
20:41:56 <randallburt> its been working fine for me so far
20:42:18 <tspatzier> stevebaker, I actually submitted a design session proposal to discuss a transition path CFN to HOT as _the_ primary format
20:42:18 <stevebaker> yeah, an early access feature, or somesuch
20:42:25 <shardy> mspreitz: If you need stablility, right now, using CFN template syntax (or the yaml rendered variant thereof) is a better plan
20:42:32 <stevebaker> tspatzier: cool
20:43:00 <zaneb> shardy: yep, that comment was due to a misunderstanding on my part
20:43:06 <mspreitz> Yeah, now that you mention it, stability is also good for users.  But the point I was trying to make is about documentation.
20:43:11 <stevebaker> I've been wondering if python-heatclient should grow a bunch of template manipulation tools cfn->hot, aws->native
20:43:27 <shardy> tspatzier: Yep, I think that's a good plan to aim for, it will just take a while :)
20:43:36 <stevebaker> plus offline template validation etc
20:43:49 <randallburt> +1 for that last one stevebaker
20:44:26 <randallburt> though not sure how it would work without access to the resource plugins
20:44:42 <tspatzier> stevebaker, that translation feature is an intersting thing to discuss. Key for some cleanup in the core engine
20:44:46 <shardy> stevebaker: we discussed such an idea before, IMO the client is probably not the right place for it
20:45:04 <stevebaker> randallburt: I guess it would be more of a sanity validation for syntax and structure
20:45:05 <sdake_> different repo makes sense to me
20:45:11 <shardy> stevebaker: but we should discuss where, when and how that could happen at summit :)
20:45:21 <sdake_> imo python-heatclient should just be the heat client lib
20:45:25 <randallburt> could be 'minimal' access to the server via resource_types and a few extensions
20:45:26 <stevebaker> shardy: another option is a heat-template-tools repo
20:45:38 <sdake_> stevebaker+1 there
20:45:38 <mspreitz> Deeper validation is good
20:45:49 <shardy> stevebaker: well we already have heat-templates/tools :)
20:45:52 <mspreitz> I mean resource-specific validation
20:46:19 <stevebaker> shardy: but make it a real python client, cli and maybe python lib
20:47:24 <shardy> stevebaker: Yeah, at some point maybe, but first we've got to transistion the engine to be completely decoupled from CFN, to that top-level translation to some native model format is possible
20:48:29 <shardy> stevebaker: IMO we've had a pretty big challenge handling HOT and CFN in the engine, I'm not sure pushing that up to the client really makes things easier, unless the aim is to remain just a cosmetic syntax translation
20:49:08 <zaneb> I think once we've finished refactoring the engine, we won't _want_ to move it out of there
20:49:22 <shardy> zaneb: I suspect the same
20:49:29 <zaneb> it should end up being pretty tidy
20:49:45 <shardy> zaneb: handle templates via a plugin interface like resources
20:49:54 <mspreitz> BTW, is all the stuff that the current code does with snake_to_camel accurately documented in the HOT spec?
20:49:54 <zaneb> and having two formats will help keep it that way
20:50:17 <stevebaker> I'm just thinking cosmetic translation, which will require manual finangling from the user to make a working template
20:50:47 <tspatzier> mspreitz, the hot spec does document exactly what you can use it HOT. there is not need to document HOT <--> CFN transformation  IMO
20:51:38 <zaneb> we may even want to add more formats, and having the abstractions in there will help with that
20:51:52 <shardy> stevebaker: we could have some script to facliltate that, but it's not a majore requirement IMO
20:51:58 <tspatzier> mspreitz, the current snake_to_camel and vice versa is just because both formats are so close today. I don't see this as a general way for going forward.
20:52:27 <shardy> having the template supported properly in our internal logical model is much higher priority
20:52:41 <mspreitz> If you want me to stop using HOT today, say so.  Otherwise, I need to know all the details.
20:52:42 <shardy> IMHO :)
20:53:02 * randallburt doesn't want you to stop using HOT.
20:53:20 <shardy> mspreitz: You can do whatever you want, that is the beauty of open source :)
20:53:20 <tspatzier> randallburt, +1 :-)
20:53:26 <tspatzier> We need to smoke test it
20:53:31 <stevebaker> mspreitz: if it works for you, go for it. But you may need to change your templates as it evolves
20:53:47 <adrian_otto> I've been faced with the decision about supporting multiple input formats multiple times. Each time, the simplest and cleanest solution is to support one input format, offer a sensible extensibility scheme for that format, and expect other formats to convert to the primary format, passing special stuff through as extensions.
20:53:56 <shardy> mspreitz: HOT is basically functional, and we welcome participation in testing, documenting, and improving it
20:54:27 <randallburt> I use HOT quite a bit and its working fine and matches the existing docs, so we basically need to start adding HOT stuff to the gates, yes?
20:55:01 <stevebaker> one final thing, can people do reviews please?
20:55:19 <adrian_otto> at our last design summit we reached a consensus that HOT could be that primary format, and that we had a desire to offer a compatibility story for CFN
20:55:27 <spzala> stevebaker: yes :)
20:56:06 <stevebaker> adrian_otto: eventually tempest tests should be all hot via python-heatclient, and cfn compatible tests via boto
20:56:11 <shardy> adrian_otto: We have existing users who rely on CFN template support, so it's more than a desire, it's a commitment
20:56:12 <tspatzier> adrian_otto, right. And I think we should pick that topic up again at the next summit and see how we get there.
20:56:39 <adrian_otto> tspatzier: agreed, thanks.
20:57:02 <radix> hmm
20:57:06 <zaneb> I vote for getting there one step at a time :)
20:57:18 <radix> we don't have a place in resource code to document the value of Ref for a resource
20:57:31 <shardy> adrian_otto: but HOT is certainly the primary development focus atm, and we're moving in the direction of it becoming the primary format, probably :)
20:58:06 <stevebaker> radix: If it doesn't already, it would be easy to take that from the FnGetRefId docstring
20:58:36 <zaneb> radix: yeah, we need that. stevebaker: good idea
20:58:39 <radix> hmm.
20:58:42 <randallburt> radix:  isn't that kinda covered here: #link http://docs.openstack.org/developer/heat/template_guide/functions.html#ref
20:58:43 <shardy> zaneb: +1, incremental progress
20:58:47 <radix> I don't generally like that pattern because it conflates developer documentation with user documentation.
20:59:05 <radix> randallburt: no, because the value of Ref is different for every resource
20:59:10 <randallburt> maybe that just needs to be expanded a bit
20:59:31 <shardy> Ok, out of time, thanks everyone!
20:59:33 <randallburt> oh, I see.
20:59:39 <radix> ok! seeya :)
20:59:41 <shardy> #endmeeting