20:00:16 <shardy> #startmeeting heat
20:00:17 <openstack> Meeting started Wed Jul 24 20:00:16 2013 UTC.  The chair is shardy. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:21 <openstack> The meeting name has been set to 'heat'
20:00:24 <shardy> #topic rollcall
20:00:35 <zaneb> o/
20:00:36 <stevebaker> yop
20:00:39 <shardy> Hi all, who's around :)
20:00:48 <randallburt> hi
20:00:49 <asalkeld> o/
20:00:50 <tspatzier> Hi
20:00:58 <timductive> Hi
20:01:42 <shardy> sdake, therve, jpeeler?
20:01:46 <therve> Hi!
20:01:58 <stevebaker> SpamapS?
20:02:09 <shardy> good catch stevebaker ;)
20:02:26 <shardy> Oh well, lets make a start
20:02:44 <shardy> #topic Review last week's actions
20:03:06 <shardy> So thanks to stevebaker for doing meetings etc while I was away :)
20:03:08 <radix> oh, heya
20:03:22 <jpeeler> oh hi
20:03:27 <shardy> #link http://eavesdrop.openstack.org/meetings/heat/2013/heat.2013-07-17-20.00.html
20:03:49 <shardy> #info action stevebaker to start mission discussion on list
20:04:16 <shardy> stevebaker: did that happen, I didn't spot it but behind on the ML backlog..
20:04:22 <stevebaker> I have a draft, should we have a quick bikeshed here https://etherpad.openstack.org/heat-mission
20:04:56 <shardy> I missed the context on this, who's asking for this?
20:05:23 <stevebaker> the board, heat is now the OpenStack Orchestration program
20:05:24 <topol> short and sweet :-)
20:05:44 <asalkeld> do we need to mention templates?
20:05:50 <stevebaker> all programs need to come up with a mission statement
20:05:53 <shardy> #link https://etherpad.openstack.org/heat-mission
20:06:05 <zaneb> stevebaker: orly? can we call ourselves OpenStack(TM) Orchestration now?
20:06:25 <stevebaker> which I'm interpreting to mean define "what we do", not "what value do we offer"
20:06:27 <randallburt> added some small input for clarity, but looks good to me
20:06:36 <topol> why couldnt we before?
20:06:54 <stevebaker> heat is a project within the OpenStack Orchestration program
20:07:07 <asalkeld> program is what people do, not what they produce
20:07:32 <stevebaker> I'd like to get something about the lifecycle of applications in there, heat is not just launch and forget
20:07:42 <topol> I like your mission statement
20:08:06 <randallburt> stevebaker:  how about that?
20:08:23 <stevebaker> maybe "configuration" is redundant?
20:08:52 <topol> I like that you explicitly call out configuration. Thats a big deal
20:08:53 <radix> stevebaker: what does "openstack resources" mean in this context?
20:09:03 <asalkeld> I think is ok
20:09:06 <asalkeld> it is
20:09:21 <topol> +1
20:09:41 <stevebaker> radix: anything that can be invoked via an openstack API
20:10:14 <stevebaker> that looks good to me
20:10:33 <stevebaker> arg
20:10:40 <randallburt> needs more commas
20:10:46 <shardy> "management of applications on those resources" could just be "management of those resources"
20:10:50 <stevebaker> ,,,,,,,,,,,,,,,,,
20:10:54 <randallburt> thanks ;)
20:10:57 <shardy> since we're IaaS focussed
20:10:58 <radix> shardy: different meaning of resources, I think
20:11:21 <shardy> I just want to avoid saying we're an application (PaaS) management solution
20:11:27 <radix> does heat orchestrate openstack, or applications on top of openstack?
20:11:33 <shardy> but otherwise looks good
20:11:36 <randallburt> one could argue that "configuration" would include software, but I'm not fussed
20:11:52 <shardy> radix: depends on your definition of applications I guess
20:11:56 <topol> radix, Im hoping both
20:12:03 <randallburt> radix:  if SpamapS and OOO has their way, the answer is "yes"
20:12:08 <radix> like when I see "enable the orchestration of OpenStack resources" it seems like that could be misinterpreted as what OOO is
20:12:14 <stevebaker> "support the..." implies that we can pass off to some other configuration mechanism
20:12:16 <radix> randallburt: well, yes, but isn't that a separate program?
20:12:46 <therve> +1 for sending and ML discussion
20:12:48 <shardy> the main think is, internally, Heat doesn't care about what you deploy, whereas a PaaS has to, due to e.g versions for the application containers it provides
20:12:49 <radix> :)
20:12:56 <stevebaker> yup, shardy want to send it?
20:12:57 <randallburt> radix:  well, yes. and Heat would just be a means to an end for them.
20:13:04 <shardy> Yeah, lets move on and pick it up on the ML
20:13:06 <randallburt> therve:  +1
20:13:08 <radix> randallburt: right, I understand. so we need our program descriptions to be distinct
20:13:09 <shardy> stevebaker: can do, sure
20:13:13 <radix> yeah, this conversation is getting sprawly
20:13:23 <shardy> #action shardy to send mission ML statement
20:13:36 <stevebaker> yay for etherpad though
20:13:55 <shardy> Yeah, thanks stevebaker for getting things rolling
20:13:59 <shardy> #topic Documentation
20:14:20 <stevebaker> today I will learn how to write a sphinx extension
20:14:25 <radix> woo
20:14:37 <shardy> So I see this is carried over from last meeting - need to get the docs sprint organised
20:14:39 <stevebaker> for resource doc generation
20:15:27 <shardy> Anyone got anything to raise re Docs other than we need to write lots of them very soon? ;)
20:15:39 <stevebaker> shardy: set it for after h-3, since we'll be in some form of feature freeze after that
20:15:43 <therve> Can we get rid of the docs directory?
20:15:51 <annegentle> o/
20:15:52 <randallburt> how is the wadl maintained? is it manual?
20:15:59 <shardy> stevebaker: agree, we've got too much in h3 already
20:16:05 <randallburt> we said "docs" too many times ;)
20:16:27 <stevebaker> randallburt: I believe so
20:16:29 <shardy> annegentle: hi
20:16:31 <annegentle> randallburt: manual is typical for all the other projects. Neutron had an attempt at automation but the manual way was quicker and more accurate
20:16:34 <annegentle> heh
20:16:58 <asalkeld> annegentle, any reason heat is missing from here: http://docs.openstack.org/developer/openstack-projects.html
20:17:11 <stevebaker> our api is too simple to justify automatic generation
20:17:19 <annegentle> asalkeld: just patch openstack-manuals/www/developer/index.html
20:17:25 <annegentle> stevebaker: +1
20:17:26 <asalkeld> ok
20:17:28 <randallburt> annegentle:  cool. figured that was the case. Just wondered if we needed a task to make sure its still accurate and new things were added
20:17:28 <shardy> It would be nice if there was some way to automatically generate API docs and resource schema documentation
20:17:36 <topol> annegentle, I noticed a lot of the architecture pictures have not been updated to have heat and ceilometer
20:17:40 <topol> as core
20:17:41 <stevebaker> shardy: that is what I am working on today
20:17:45 <annegentle> shardy: sure would be but then which would be truth? :)
20:18:16 <shardy> annegentle: If it's generated from code then the docs always match the code, surely?
20:18:18 <annegentle> topol: I have been thinking about that a lot, and I'm going to send an OpenStack program description for docs to the -dev list today
20:18:22 <shardy> stevebaker: awesome :)
20:18:26 <stevebaker> zaneb has written api docs, it just needs to be merged into api-site
20:18:41 <annegentle> in it, I have to draw a line between core and integrated. So I'm not sure how to address things like arch diagrams maintenance and updates.
20:18:52 <annegentle> shardy: but what if the code is wrong?
20:18:53 <zaneb> someone should probably compile them first too ;)
20:19:25 <shardy> annegentle: auto-generated docs != specification, but yea I see your point
20:19:32 <stevebaker> zaneb: maybe gating compiles it for you?
20:19:47 <shardy> all that java stuff is really hideous
20:19:56 <zaneb> stevebaker: let's go with that :)
20:20:32 <shardy> Anyway, any other docs stuff before we move on?
20:20:51 <annegentle> shardy: thanks for having docs on the agenda!
20:21:27 <shardy> annegentle: thanks for pointing us in the right direction, hopefully we'll start making progress after h3
20:21:47 <shardy> #topic h3 blueprint milestone and priority
20:21:50 <annegentle> oh one other note, there are changes to the Heat cli docs at https://review.openstack.org/#/c/37328/
20:22:11 <annegentle> nothing major, but want to point out they are being maintained/freshened
20:22:23 <shardy> #link https://launchpad.net/heat/+milestone/havana-3
20:22:30 <radix> I am pushing to have a WIP review up for my instance-group refactor today
20:23:10 <shardy> radix: sounds good
20:23:59 <shardy> so we really have a lot of BPs compared to h1 and h2
20:24:35 <shardy> guess we'll all be busy, but if things are going to slip, best to flag it early and start untargetting them
20:25:04 <shardy> Anyone have any issues re h3 bp or bugs they want to raise?
20:25:26 <randallburt> asalkeld, zaneb should we go another round on https://review.openstack.org/#/c/36744/ or can I move on to https://blueprints.launchpad.net/heat/+spec/update-stack-new-resource-state
20:25:27 <therve> I've been thinking about load balancer lately
20:25:40 <therve> I wonder if we could squeeze improvements to it in h3?
20:25:43 <SpamapS> o/
20:25:46 <SpamapS> sorry lunch went long
20:26:15 <shardy> therve: I thought we were going to move to the neutron lbaas rather than polishing what we have?
20:26:34 <zaneb> randallburt: you were going to fix up the Output.* stuff
20:26:35 <therve> shardy, That's one side of it, yeah. It's not in h3 either, though
20:26:36 <randallburt> +1 on neutron lbaas over tweaking AWS resource
20:27:09 <randallburt> zaneb:  oh, yes. I have that in a branch but forgot because I've been out/busy on other stuff. I'll get that submitted before COB today
20:27:12 <shardy> therve: If people are prepared to work on stuff, and they can realistically deliver it then we can consider putting it in h3
20:27:18 <therve> I've been looking at it, but neutron makes me angry
20:27:23 <therve> shardy, Cool, thanks
20:27:25 <randallburt> therve:  lol. indeed
20:27:38 <zaneb> randallburt: ok, when that is done I think I am happy
20:27:44 <stevebaker> I'd like something that can be tempest tested. building the image that the nested stack uses is a barrier right now
20:27:49 <randallburt> zaneb:  roger that, and thanks!
20:27:56 <asalkeld> randallburt, I'll work on this too sometime https://review.openstack.org/#/c/38285/
20:27:59 <shardy> the lbaas stuff I didn't put in havana because I was not really sure if it's mature enough to integrate with yet
20:28:18 <randallburt> asalkeld:  awesome. I really like the concept there.
20:28:24 <SpamapS> stevebaker: I believe we will be pushing to host several different images for infra as we move pieces of tripleo into the gate
20:28:25 <zaneb> randallburt: but quit with the double underscores anyway ;)
20:28:30 <therve> Yeah I don't know about the implementation. The interface ought to be, though
20:28:48 <asalkeld> we really need to have a way of marking resources (deprecated, prototype, supported)
20:28:48 <randallburt> zaneb:  understood ;D
20:29:15 <stevebaker> SpamapS: can you keep me up to date on that? I'll be getting devstack to generate images as soon as I can package dib
20:29:27 <randallburt> asalkeld would that be something to do with stevebaker's sphinx extension?
20:29:38 <asalkeld> not sure
20:29:40 <randallburt> or would you want it as something query-able via the api?
20:29:48 <randallburt> probably both, I'd assume
20:29:50 <asalkeld> I think both
20:30:24 <stevebaker> once the metadata is there it can be exposed however
20:30:37 <therve> As a result of API calls would be even better
20:30:41 <asalkeld> nice to have a an arg --only-use-production-ready-resources
20:30:57 <stevebaker> we need a way of flagging which type is the primary one too, OS::Quantum:: vs OS::Neutron
20:31:35 <randallburt> IIRC there are already bps around a "catalog" of resources; cba to remember specifics atm, but might be worth a look
20:31:46 <asalkeld> (maybe yet another blueprint)
20:32:30 <randallburt> and probably not h3, yes?
20:32:37 <randallburt> considering the existing backlog and all
20:32:42 <asalkeld> yea
20:32:49 <shardy> IMO there are more pressing things to do in h3
20:33:25 <asalkeld> everyone has their own priorities
20:33:44 <shardy> Ok, one last thing, if anyone has too much assigned to them, please unassign yourself, then if people are looking for stuff to do they can pick it up
20:33:52 <shardy> asalkeld: yea true
20:34:01 <randallburt> agreed, but I can take an action to groom the bp backlog around this topic if you like
20:34:31 <stevebaker> groom away!
20:34:44 <shardy> #action randallburt bp grooming
20:34:55 <shardy> anything else on h3?
20:34:59 <randallburt> around the resource catalog, I mean ;)
20:35:10 <shardy> #topic #undo
20:35:34 <shardy> #action randallburt resource catalog bp grooming
20:35:43 <randallburt> thanks ;)
20:35:53 <shardy> #topic Removal/moving of heat-boto/heat-cfn/heat-watch client tools
20:35:59 <stevebaker> so
20:36:02 <shardy> sdake: around?
20:36:40 <shardy> stevebaker: go for it ;)
20:36:42 <stevebaker> I've submitted that heat-cfnclient be a new repo to house heat-cfn, heat-boto, heat-watch
20:36:58 <stevebaker> and moved them out of the core repo
20:37:13 <asalkeld> I think sdake was against putting in a separate project
20:37:13 <shardy> stevebaker: I don't really mind where they live provided they still exist
20:37:28 <shardy> sdake had some strong opinions on the matter yesterday
20:37:31 <SpamapS> they're just tools for testing the cfn apis?
20:37:37 <jpeeler> what's the reason to move it?
20:37:38 <stevebaker> my position is that they should still exist, they should have python releases, but we should encourage distro packages not to package them
20:37:39 <shardy> SpamapS: yep
20:37:40 * SpamapS has never used them
20:38:26 <shardy> stevebaker: that sounds fine - sdake seemed to indicated if we moved them to a separate repo that downstream distros would be somehow obligated to package and document them
20:38:37 <shardy> I didn't quite get to the bottom of why
20:38:43 <SpamapS> they sound like decent things to have in heat proper as sort of contrib/test/helpful bits to make sure the cfn api is working.. ?
20:38:58 <stevebaker> tempest already has boto tests for ec2 compat. I think the primary way to confirm our cfn compatibility is to add to these. These would not require heat-cfnclient at all
20:39:13 <shardy> SpamapS: Yeah, that would work too, and just not package them
20:39:38 <shardy> stevebaker: Agree, there was a functional test previously which did exactly that
20:39:39 <SpamapS> Agreed to have tempest do api calls directly.
20:40:06 <zaneb> jpeeler: clients are generally not kept in the same repo, so it's incongruous to have those ones there
20:40:20 <stevebaker> to me, it is madness that clients were packaged in server packages anyway - but I don't know distro policies on removing executables from packages
20:40:26 <shardy> and there was an argument about removing heat-boto vs heat-cfn, but they are actually the same tool, one symlinks to the other, it just selects a different client wrapper
20:40:46 <shardy> ie the duplication is not significant and they can both be useful
20:41:08 <SpamapS> Yeah, I think its fine to have them in another repo, but I wouldn't call their presence in heat anything more than a low priority bug.
20:41:19 <shardy> I definitely agree (despite occasional version pain) that we should keep testing the cfn api with boto instead of our own client lib
20:41:23 <stevebaker> heat-boto is not working for me right now, but I may have old copies of keystoneclient
20:41:36 <zaneb> stevebaker: it made some sense to me because when you change the api you also have to change the client
20:41:48 <shardy> stevebaker: Yeah >=0.9.1 needs my patch to keystoneclient Ec2Signer
20:42:07 <shardy> https://review.openstack.org/#/c/35945/
20:42:30 <stevebaker> so how is heat-cfntools not broken?
20:42:40 <shardy> sorry boto 2.9.2
20:43:10 <shardy> stevebaker: the jeos images don't use bleeding edge boto
20:43:33 <shardy> e.g F18 is still 2.6.0
20:43:43 <stevebaker> ok
20:43:44 * SpamapS would just like to note that boto is the devil
20:43:55 <asalkeld> haha
20:44:03 <stevebaker> so, the delete change is here https://review.openstack.org/#/c/38228/
20:44:21 <stevebaker> import repo is here https://github.com/steveb/heat-cfnclient
20:44:31 <topol> boto 666.  the version never changes
20:44:37 <stevebaker> launchpad and pypi are set up and ready to go
20:44:49 <zaneb> SpamapS: but if anybody is ever going to use the cfn-api again, they'll almost certainly be using boto to do it
20:45:03 <SpamapS> zaneb: I wrote my own cfn calls for os-collect-config
20:45:04 <shardy> stevebaker: what do we gain by moving to a new repo, vs just moving to a subdir in heat master?
20:45:30 <SpamapS> and just used keystoneclient for ec2signer
20:45:35 <stevebaker> removing a bunch of client specific code from "common"
20:45:53 <stevebaker> the only thing really common between client, api and engine is exceptions
20:45:54 <zaneb> SpamapS: yeah, the in-instance side is a different story
20:46:05 <shardy> stevebaker: Ok, well if you can get past sdake's objections, I'm not opposed to it
20:46:15 <SpamapS> https://github.com/stackforge/os-collect-config/blob/master/os_collect_config/cfn.py
20:46:46 <zaneb> SpamapS: from the API perspective, you're either migrating from AWS and therefore using boto, or not and you should just jump straight to the native api
20:47:18 <stevebaker> using boto is not mandatory though
20:47:18 <shardy> Yeah, we've found a lot of CFN incompatibilities by using boto for testing
20:47:19 <asalkeld> ec2signer
20:47:47 <asalkeld> zane for in-instance calls
20:47:49 <shardy> but nobody is saying users have to use it, and the upstream is pretty unresponsive at times
20:48:03 <stevebaker> another thing, if we can outright kill heat-cfn then authtoken middleware can be removed from the cfn pipeline. <-- I would really like to do that
20:48:33 <zaneb> the point is, if we're going to have a compatibility api, it should be compatible
20:48:43 <topol> stevebaker, what would replace authtoken middelware if you remove it?
20:48:54 <stevebaker> either kill heat-cfn or implement an ec2token auth strategy for it
20:48:54 <asalkeld> stevebaker, you mean just use boto
20:49:28 <shardy> stevebaker: I'd be more in favor of killing heat-cfn than heat-boto
20:49:29 <zaneb> I'm all for killing heat-cfn and just keeping heat-boto
20:49:35 <stevebaker> topol: ec2token provides auth for ec2 signed requests, like boto and heat-cfntools does
20:49:41 <shardy> (even though they are the same code at the top level)
20:49:58 <topol> stevebaker, gotcha. thanks
20:50:15 <zaneb> shardy: right, but there's a whole client.py file we can delete
20:50:33 <shardy> zaneb: woohoo
20:50:52 <shardy> heat-watch will die after the ceilometer migration is complete too
20:51:16 <shardy> Ok, lets move on
20:51:24 <shardy> #topic open discussion
20:51:35 <shardy> 9 minutes, anyone have anything to raise
20:51:55 <asalkeld> someone on #heat didn't want to use *any* aws api
20:52:05 <asalkeld> including ec2signers
20:52:22 <asalkeld> alarming now depends on ec2signers
20:52:28 <asalkeld> is that a problem
20:52:30 <stevebaker> yes, deployers are disabling all ec2 features on advice of legal :/
20:52:34 <randallburt> asalkeld:  not an unreasonable request considering
20:52:38 <shardy> asalkeld: I think long term it would be nice to make the cfn API optional, but the signing scheme is separate from that
20:52:49 <SpamapS> I think until keystone has oauth, or we embrace trusts... ec2 signer is it.
20:52:56 <topol> stevebaker, wow when did that advice come down?
20:52:56 <shardy> ie couldn't we just allow the pre-signed URL mechanism to also work via the ReST API?
20:53:00 <stevebaker> shardy: not according to the lawyers
20:53:07 <SpamapS> stevebaker: s/deployers/a single deployer/
20:53:13 <stevebaker> SpamapS: yes
20:53:18 <stevebaker> to our knowledge
20:53:24 <randallburt> not true <.< >.>
20:53:39 <SpamapS> Many other orgs have come to the opposite conclusion regarding the legality of using AWS apis.
20:53:39 <shardy> stevebaker: sigh, so we have to reinvent something new and probably less secure, huurah
20:53:42 <asalkeld> at the moment there is not really a nice alternative
20:53:53 <SpamapS> s/a single deployer/a subset of deployers/
20:54:02 <stevebaker> renaming the endpoint and the class would probably do ;)
20:54:23 <randallburt> lols. they are only lawyers after all :D
20:54:24 <shardy> SpamapS: trusts still does not provide any mechanism for signing requests
20:54:25 <SpamapS> asalkeld: oauth or trusts will work fine.
20:54:36 <SpamapS> don't need to sign the req
20:54:44 <topol> +1 on oauth!
20:54:54 <asalkeld> is that ready yet?
20:54:59 <asalkeld> (oauth)
20:54:59 <SpamapS> You have been authorized for the API call.
20:55:10 <randallburt> the lack of os native key escrow is a pita too
20:55:22 <topol> asalkeld, getting close. still out for reviews
20:55:28 <zaneb> oauth sounds like the Right Thing
20:55:29 <shardy> SpamapS: the whole point of pre-signing is to avoid deploying credentials in the instance (or elsewhere, e.g to another service)
20:55:33 <topol> stevemar loves feedback
20:56:16 <asalkeld> so what I am getting at is do I have to rework my ceilo patches
20:56:23 <shardy> SpamapS: so trusts may work, but we'd need to use other means to lock down the request (role based policy, and even then we can't restrict the content of the request
20:56:24 <SpamapS> shardy: agreed that that is superior to oauth.
20:56:29 <asalkeld> (and ceilometer)
20:56:51 <stevebaker> asalkeld: not yet, there is no alternative yet
20:56:58 <shardy> I just think if we're going to reinvent a new solution it should be at least as secure, definitely not less secure, otherwise what's the point?
20:57:28 <asalkeld> ok, well I'll leave as is for now
20:57:38 <asalkeld> issue raised ...
20:57:50 <SpamapS> Sounds like a feature to push for soon. The right answer may be to copy whatever swift does for its signed urls.
20:57:57 <shardy> Yeah we can keep mulling the alternatives :)
20:58:49 <topol> is there a use case where heat benefits from oauth?
20:59:00 <shardy> Ok nearly out of time, anything else or are we done?
20:59:07 <topol> thought the plan was to use trust?
20:59:10 <topol> s
20:59:21 <shardy> out of time, thanks all!
20:59:26 <shardy> #endmeeting