20:02:07 #startmeeting tc 20:02:08 Meeting started Tue Feb 19 20:02:07 2013 UTC. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:02:09 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:02:10 o/ 20:02:12 The meeting name has been set to 'tc' 20:02:15 Agenda for today is: 20:02:19 #link http://wiki.openstack.org/Governance/TechnicalCommittee 20:02:20 hi 20:02:32 (on brand-new wiki) 20:02:37 #topic Joint Board / TC F2F meeting on April 14 20:02:50 The Board is proposing that we hold a common BoD/TC face-to-face meeting on the Sunday before the Summit 20:02:58 Something like 2pm to 5pm then dinner at 6pm. 20:03:12 This raises a few issues... some of us (including me) already have booked non-refundable conflicting plane tickets 20:03:13 ttx: hey hey 20:03:21 Also we don't really know who will be on the TC, as we are renewing 10/12 members in the upcoming elections 20:03:35 Comments ? Should we answer, we can try to be there, but best effort only ? 20:04:10 The goal was to wrap up the conclusions of the Incubation/Core committee 20:04:15 I think that best-effort is probably all we _can_ offer 20:04:22 which should hopefully be completed by then 20:04:32 if folks have tickets booked, that's bad news 20:04:36 especially ttx 20:04:42 could we do a breakfast together? 20:05:29 We could also do an evening thing, starting at 6pm and ending in dinner 20:05:45 probably the best if we actually want everyone there 20:06:08 OK, I'll answer depending on how needed people are, prefer a late or early thing during the week 20:06:16 if optional, then sunday is fine 20:06:22 o/ 20:06:25 * heckj nods 20:06:30 does that sound good ? 20:06:31 Sunday's tough, maybe another week day 20:06:48 the know that getting people in regular hours will be impossible 20:06:54 the rest of the week is out because of the summit 20:07:03 so that means doing early or late in the day 20:07:14 and keeping it short 20:07:16 and the board members yelled at bryce after the last time that we had a meeting scheduled over top of sessions 20:07:23 keeping it short is definitely ++ 20:07:41 OK, I'll come up with an answer, though I'm pretty sure AlanClark will see this log 20:08:01 #topic End-of-cycle graduation review (cont'd) 20:08:17 Last week both projects presented why they think they are ready to be integrated in the Havana release cycle 20:08:26 We also reviewed their release process alignment status, which was positive 20:08:34 Brian suggested we continue the review serially rather than in parallel 20:08:43 If there is no objection to that... 20:08:56 ...then I suggest that Heat goes first... since nijaba from Ceilometer is not around for this meeting 20:09:10 ttx, do we absolutely need to reach a decision today? 20:09:13 serially within a single meeting, or? 20:09:22 or accross multiple meetings? 20:09:23 serially within one or two meetings 20:09:26 k 20:09:33 depending how fast we go 20:09:50 objections ? 20:09:51 I can speak for ceilo in any case if we get to it today 20:10:07 eglynn: we should touch it, but maybe not finish it, today. 20:10:26 k 20:10:31 #topic Technical stability and architecture maturity assessment, Heat 20:10:50 In this section I'd like to make sure that (1) the project is currently usable and (2) the architecture is now stable (no full rewrite needed during Havana cycle) 20:11:10 sure 20:11:14 For Heat the only questions seem to be around template / API, and the need to support more than just AWS cloudformation. 20:11:33 sdake: could you give us your view on that ? 20:11:47 heat is basically a parser 20:11:57 which is contained in one file 20:12:07 if someone wants another template format, simply write another parser.py 20:12:13 so no rewrite required 20:12:39 api as far as I am concerned is in good shape 20:12:43 so the architecture is pretty simple... and stable ? 20:12:53 our architecture has not changed in 9 months 20:13:11 code base was stable when we went into incubation, but even more bugfree now ;) 20:13:23 one area where our code base will change.. 20:13:28 we have this directory called resources 20:13:38 it contains things that launch vms, eips, etc. 20:13:47 it contains something called nested stacks - ie: RDS 20:13:52 a relational database service 20:14:05 we would prefer those not be nested stacks and instead use openstack apis where avialable 20:14:21 atm there are no openstack apis for rds but if there are, we will merge to use those apis 20:14:34 there are other resource types as well 20:14:41 so to be clear, in the absence of a db service, you have code that knows how to set up an instance with a database on it, that kind of thing, right? 20:14:43 load balancer 20:14:49 right 20:14:53 k 20:15:09 but those are not major architectural changes, only changes in how we interface with openstack 20:15:11 makes sense 20:15:15 but you want to kill those off when an API is available ... makes sense 20:15:27 other questions on technical stability and architecture maturity, before we talk about the scope ? 20:15:34 would like to do so yes - and add more resources as projects like moniker hit the wire 20:15:42 what's the story with the cloudwatch impl? 20:15:45 sdake_z: does the architecture support switching those out without major rewrites as new things come available? 20:15:48 is there a deprecation plan? 20:15:48 s/as/if/ 20:15:57 or is it already optional and you can use another implementation? 20:15:59 yes, each resource is a .py file 20:16:08 with a standard programming api 20:16:12 markmc: we plan to move to using ceilometer when the features we need are there 20:16:42 shardy, will there be much work to make that move? 20:16:51 sdake_z: thanks 20:16:52 shardy, they should be compatible, so no major user impact right? 20:16:53 re cloudwatch, want to remove it from the code base as soon as ceilo is in place, have had discussions with ceilo team about alerting and that seems positive 20:17:06 cool 20:17:08 markmc: There will be a bit of rework in the engine to decouple things, but nothing major, no 20:17:35 how about the metadata server, heat still has its own read/write server? 20:17:40 markmc: I've been putting off that rework waiting for the ceilo stuff to be ready 20:18:00 markmc: No, all resource metadata is now served via the CFN or ReST API 20:18:08 we removed the metadata server 20:18:11 the metadata server would disappear once ceilo is in 20:18:18 how do you determine which resource modules to use? do you use keystone's service catalog, or config flags, or...? 20:18:35 gabrielhurley not sure i understand question 20:18:50 you were talking about trading out resource .py modules 20:18:50 shardy I think markmc was tlaking about cloudwatch server process 20:19:12 if there are competing implementations, how do you determine which ones to use? 20:19:15 no, I was asking about the metadata server that at one point you wanted to use nova's metadata server 20:19:20 but the issue was that it was readonly 20:19:27 sdake_z: well he mentioned CW then metadata - the CW stuff is in progress, but the separate metadata service has been removed now 20:19:39 gabrielhurley there is a name space - for example OS::HEAT::Reddwarf 20:19:41 the server that e.g. cfn-trigger or whatever talks to? 20:19:44 * markmc waves hands 20:20:01 hi 20:20:15 each resource.py contains the namespace resource it is responsible for 20:20:24 markmc: It all talks to either the CFN or Cloudwatch API now, there is no heat-metadata anymore 20:20:35 which means everything is authenticated/signed 20:20:36 shardy, ok, thanks 20:20:52 sdake_z so the resource files are namespaced for each impementor, but how do you determine which one to use? I'm trying to understand if this is an "operator must configure" or "dynamically determine what's available" situation. 20:21:15 gabrielhurley you put in the template file which resource you want to use and which parameters you want to pass it 20:21:34 gabrielhurley: there is a mapping of class name to template name in each resource implementation 20:21:35 what happens if I (as an end user) put in my template a resource which is not available 20:21:40 gabrielhurley, I don't think there are any competing implementations of a single resource type atm 20:21:45 or worse, a resource which is available from a different implementor 20:21:57 gabrielhurley you get a parse error if there is no resource available 20:22:00 gabrielhurley: I'd expect template validation to fail 20:22:30 in the example of databases... 20:22:34 okay. that's what I was trying to understand. I think that architecture is gonna need more work as the ecosystem expands, but that's fine for now. 20:22:45 ok sounds good ;) 20:22:55 any more questions before we discuss scope ? 20:23:27 gabrielhurley: the resource implementations are pluggable now, so the architecture is extensible 20:23:53 #topic Scope complementarity, Heat 20:24:02 In this section I'd like to discuss the desirability of integrate Ceilometer in the common OpenStack Havana release 20:24:15 We don't really have name guidelines yet that define what is off-limits for "OpenStack" resource focus 20:24:17 you mean Heat :) 20:24:19 s/Ceilometer/Heat/ 20:24:19 s/Ceilomter/Heat/ 20:24:19 lol 20:24:22 s/Ceilometer/Heat/ 20:24:22 oops 20:24:24 heh 20:24:26 So at this point we can only apply technical guidelines 20:24:33 Is the project complementary in scope, or overlapping with others ? 20:24:39 Are there other projects in our community covering the same scope ? 20:24:45 Does it integrate well with current integrated projects ? Does it form a coherent product ? 20:24:55 (that's what you get by reshuffling me) 20:24:56 we are only project in this space inside incubation/core 20:25:12 integrates extremely well with other projects including full keystone auth 20:25:28 ttx: "does it forma conherent product" get's into openstack in-general guidelines, not technical things 20:25:46 as far as coherent product, again, i'd like to see the rds and autoscaling and other features come out of heat into other projects so we could use those directly 20:25:56 personally I place it in the same category as Horizon, an integration point 20:25:59 notmyname: true 20:26:05 as a service which pulls together our APIs, I love it 20:26:13 it's a pretty natural expansion of scope, I think 20:26:23 ++ 20:26:34 notmyname: for some pretty large definition of "coherent" 20:26:34 obviously, one of our largest competitors has something similar 20:26:44 hehe 20:26:45 it provides a functionality which a lot of openstack consumers are clamoring for... but it's definitely an expansion more than a compliment. 20:26:46 and there's a lot of interest in application level orchestration 20:27:39 fwiw, we also had conversations with rob at dell about ways that heat can be complimentary to crowbar 20:27:51 how much do changes to the OpenStack APIs affect Heat's templates? Are templates versioned somehow? 20:28:19 * annegentle wonders about integrated releases 20:28:26 annegentle we generate a version against a specific version of openstack - ie heat for havana integrates against havana apis 20:28:31 my crowbar, you are looking fine today... 20:28:48 vishy: :) 20:28:50 heh 20:29:21 annegentle: but changes in the service api's heat uses will not change the template syntax (unless you're using a new feature specific to a release, e.g the new Quantum based resources) 20:29:33 more questions on scope ? 20:29:43 sdake_z: one of openstack's goals is to be version N-1 compatible. how does Heat feel on that front? 20:29:57 it feels the heat 20:29:58 we follow openstack processes - so that seems reasonable 20:30:02 shardy: sdake_z ok, thanks 20:30:05 cool 20:30:17 although atm that is not implemented in the architecture 20:31:06 gabrielhurley i would expect python-* libs to be backwards compatible for the most part though so should be straightforward 20:31:21 sdake_z: you'd think that, wouldn't you. ;-) 20:31:33 heh 20:31:35 gabrielhurley: :) 20:31:36 naive i guess :) 20:31:50 you'll learn :-D 20:31:54 see why I put horizon and heat in the same bag, they are already forming a group 20:31:59 gabrielhurley: sorry, do you mean python version? 20:32:08 shardy: no, openstack release version 20:32:19 gabrielhurley: k, thanks 20:32:31 #topic Final Q&A and vote, Heat 20:32:43 Final questions/discussion before we vote on Heat graduation ? 20:33:03 doubts, objections... 20:33:07 o/ 20:33:18 one more Q, are you documenting your own API somewhere? 20:33:41 that needs to be done - although there is some basic docs already in the source tree 20:33:47 but they need love 20:33:56 we should speak offline about your expectations re documentatoin 20:34:01 so we can deliver what you want 20:34:31 sdake_z: yes and I want to be sure we meet user expectations for docs 20:34:35 other final questions ? 20:34:39 i think something like heat is very valuable. To me the only question is whether it makes sense to put one such template/orchestration approach as the "official" one. 20:34:47 * mordred registers his support for both the concept and the codebase 20:34:58 sdake_z: is the size of the API like "16 calls" -- basically CRUD on templates? Want a ballpark 20:35:10 if no one has concerns, I think the team has done a great job building heat and integrating, so i'm generally supportive 20:35:11 I believe 9 - just a guess tho 20:35:29 especially since we've got projects moving towards figuring out how to integrate other orchestration system with heat, rather than just competitive 20:35:31 sdake_z: ok thanks, yeah that sounds right 20:35:48 ok, ready to vote ? 20:36:03 so from my end, it doesn't seem like adding heat will block other things from playing in the ecosystem - but rather will enable things 20:36:15 (that's related to danwent's concern above) 20:36:40 mordred: i agree, as long as others see it that way as well. 20:36:52 #startvote Approve graduation of Heat (to be integrated in common Havana release)? yes, no, abstain 20:36:53 Begin voting on: Approve graduation of Heat (to be integrated in common Havana release)? Valid vote options are yes, no, abstain. 20:36:54 Vote using '#vote OPTION'. Only your last vote counts. 20:37:02 #vote yes 20:37:04 #vote yes 20:37:06 #vote yes 20:37:23 #vote yes 20:37:24 #vote yes 20:37:24 #vote yes 20:37:25 #vote yes 20:37:35 #vote abstain 20:37:53 #vote no 20:38:17 30 more seconds 20:38:52 #vote yes 20:38:57 ttx: in general, it might be good to ping people at the start of a vote 20:39:07 hmm 20:39:13 for those who aren't looking at IRC, it's not always apparent that a timed vote is happening 20:39:13 who are we missing 20:39:14 gabrielhurley: do you mean you're not hanging on every word of the discussion? :P 20:39:18 vishy: ping 20:39:19 * annegentle sets up a notification for vote :) 20:39:19 jaypipes, vishy ^ 20:39:25 bcwaldon: ping 20:39:30 #vote yes 20:39:40 I'm looking! 20:39:43 lol 20:39:48 heh 20:39:49 tick tock 20:39:54 #vote yes 20:40:09 ok, 10 more seconds 20:40:21 #endvote 20:40:22 Voted on "Approve graduation of Heat (to be integrated in common Havana release)?" Results are 20:40:23 #vote yes 20:40:24 yes (10): markmc, bcwaldon, ttx, vishy, annegentle, russellb, jgriffith, mordred, gabrielhurley, danwent 20:40:25 abstain (1): heckj 20:40:26 no (1): notmyname 20:40:35 jaypipes: :P 20:40:36 jaypipes: just missed it ;-) 20:40:45 yeh, sorry, on yet another call :( 20:40:52 sdake: congrats 20:40:56 #topic Technical stability and architecture maturity assessment, Ceilometer 20:40:57 congrats sdake_z, shardy and co. 20:41:00 tx - blame the developers ;) 20:41:24 #link https://wiki.openstack.org/wiki/Ceilometer/Graduation#Is_our_architecture_stable.3F 20:41:26 so... making sure that (1) the project is currently usable and (2) the architecture is now stable (no full rewrite needed during Havana cycle) 20:41:42 During the Grizzly cycle, Ceilometer appeared to adapt its architecture to external pressure from new consumers 20:41:58 ttx, we made incremental changes but nothing major 20:42:02 * mordred has to drop off for a plane flight ... thinks ceilometer is great 20:42:03 But the linnked doc explains the architecture is now pretty flexible and shouldn't change, iirc 20:42:18 we could see that pressure as a positive (connoting a healthy, sustainable community attracting wide interest) 20:42:41 certainly 20:42:44 thanks mordred :) 20:42:46 * gabrielhurley appreciates that ceilometer has pushed to move useful functionality into oslo 20:42:50 I'm concerned that CM is generating monitoring data but claiming usefulness for billing calculations 20:43:15 gabrielhurley: +1 20:43:15 * annegentle will be back shortly 20:43:21 notmyname: we want that data acquisition for metering and monitoring to shared infrastructure 20:43:38 eglynn: but they have very different requirements 20:44:02 eglynn: specifically, you must be able to reliably recreate your numbers when used in a billing context 20:44:16 yes and the architecture is intended to be flexible enough to address these differing requirements 20:44:24 what I've heard from the ceilometer team is that they're doing metrics, and if some people try to use that for billing that's their choice (or folly, if you prefer) 20:44:36 and the logs are the persistent source of those numbers, but you are simply using messages sent in the course of the request 20:44:51 notmyname: and polling, and auditing notifications 20:45:07 * notmyname has mostly looked at the swift parts, since that's what I know 20:45:18 notmyname: we have a multi-publish framework that allows measurements for different backends to travel via different conduits 20:45:34 notmyname: so for example for metric, trade off currency versus completeness 20:45:51 notmyname: for metering/billing ... do the opposite trade-off 20:46:36 notmyname: the idea is not to force the metrics and metering data to be shoehorned into the same bucket 20:46:55 (with the exact same QoS etc.) 20:47:10 eglynn: is CM an aggregation point for whatever metering/etc you are using (like keystone for auth, cinder for blocks)? 20:47:55 notmyname: yes, CM can acquire and aggregate measurements from many different source including the one you mention 20:48:48 notmyname: and also publish to different backends 20:49:06 notmyname: e.g. one backend would be the CM metering store 20:49:19 notmyname: another would be the future CW service 20:49:30 (i.e. the integrated Synaps engine) 20:49:39 what backends do you provide as part of the code today? 20:50:06 notmyname: just one, the CM collector/metering store/CM API service 20:50:32 notmyname: but the archiecture was specifically evolved during G to accomodate others 20:50:59 and so, eg, if you wanted something for correctness (like billing), you'd provide your own? 20:51:37 notmyname: well we envisage the metering store mentioned above would be suitable for that purpose 20:52:22 notmyname: (i.e. we don't throw metering data on the floor, though another backend might do sample or discard delayed data etc.) 20:52:43 what scale has it been tested at? 20:52:51 it's actually already used by some to do that, like DreamHost 20:52:58 notmyname: its currently deployed in DreamHost 20:53:08 dhellmann can speak to the scale there 20:53:19 our current cluster is fairly small 20:54:00 small == 100s req/sec? 10s req/sec? 20:54:17 I understand if you can't really share that ;-) 20:54:28 I would share, but I don't have those numbers 20:55:08 we can agree though that its a non-trivial production deployment, or? 20:55:46 s/agree/surmise/ 20:55:48 if others have questions, please ask. /me is ready to vote 20:56:01 we need to talk about scope a bit first 20:56:09 eglynn: yes, it's a non-trivial deployment, just not seeing a lot of traffic at this point 20:56:12 ya, I know my thoughts there ;-) 20:56:18 any more question on Technical stability and architecture maturity N 20:56:20 ? 20:56:23 are the meters in https://wiki.openstack.org/wiki/EfficientMetering in production? 20:57:02 agentle_: yes. a better list is at http://docs.openstack.org/developer/ceilometer/measurements.html 20:57:23 why would you emit volume units not in bytes? 20:57:24 IIRC, we've updated the formal docs, and not gone back and updated that design document 20:57:32 * agentle_ sighs :) 20:57:33 notmyname: where? 20:58:06 dhellmann: on agentle_'s link 20:58:10 mostly I sigh because Google finds the wiki first 20:58:15 notmyname: ok, that's out of date 20:58:16 agentle_: note that adding a meter is a relatively straight forward task, the archiecture is highly extensible ... so we'd expect that list to grow 20:58:31 eglynn: yup, understood 20:58:31 http://docs.openstack.org/developer/ceilometer/index.html 20:58:38 notmyname: Cinder does everything in GB so makes sense to me 20:58:46 looks like we are running out of time -- I propose we do scope complementarity and vote next week, unless nobody has any question on scope 20:58:49 should add the up to date link to the top of the wiki page 20:58:51 that should avoid confusion 20:59:01 russellb: good idea, I'm doing that now 20:59:34 what is kwapi? 20:59:37 on units, we rationalized our units usage during G so that its now much more consistent & logical 20:59:52 eglynn: yay incubation. 20:59:53 +1 for GiB BTW :) 21:00:10 agentle_: an energy monitoring tool see https://github.com/stackforge/kwapi 21:00:11 ok, we'll continue (and finish) the ceilometer review next week, thanks everyone 21:00:19 big meeting ahead 21:00:24 #endmeeting