20:00:16 #startmeeting heat 20:00:17 Meeting started Wed Aug 26 20:00:16 2015 UTC and is due to finish in 60 minutes. The chair is stevebaker. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:19 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:22 The meeting name has been set to 'heat' 20:00:25 hey all 20:00:28 #topic rollcall 20:00:33 o/ 20:00:33 o/ 20:00:50 hi 20:00:57 who wants to be co-chair if I need to leave early? 20:01:13 I may try ;) 20:01:20 this would be a bad time to announce my presence with a o/ 20:01:20 o/ 20:01:21 o/ 20:01:25 #chair skraynev_ 20:01:26 Current chairs: skraynev_ stevebaker 20:01:27 o/ 20:01:45 #topic adding items to the agenda 20:01:51 #link https://wiki.openstack.org/wiki/Meetings/HeatAgenda#Agenda_.282015-08-26_2000_UTC.29 20:03:40 stevebaker: will you cover feature proposal freeze under l-3 release? 20:03:49 shardy: I will 20:04:37 #topic heat reviews https://etherpad.openstack.org/p/heat-reviews 20:04:40 stevebaker: cool, thanks 20:06:06 do we plan to land all convergence related patches befor l3 ? 20:06:17 this close to feature freeze we're probably better off following the milestone page rather than this etherpad 20:06:22 or it may be done later ? 20:06:31 I rebased the final part of PATCH updates https://review.openstack.org/#/c/205754/ 20:06:48 shardy: oh yeah, lets get that on the list 20:06:50 adding that to the list, or just any feedback would be great :) 20:07:16 I'll finish the test for the heatclient patch soon, but that should be the last heat patch AFAIK 20:07:20 skraynev_: I'm assuming any non-landed convergence changes can be framed as bug fixes after feature freeze 20:07:57 stevebaker: ok. sounds reasonable 20:08:13 we will need to figure out what we're doing with Refactor ASG and RG resources before freeze though 20:08:16 I'm currently reworking the scaling-library topic 20:08:37 and of course I reposted some reviews and forgot to re-specify the topic 20:08:58 #topic prep for liberty-3 release 20:09:00 so there's some under https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bug/1488971,n,z 20:09:06 feature freeze is a week away! 20:09:13 * stevebaker runs in circles 20:09:22 #link https://wiki.openstack.org/wiki/Liberty_Release_Schedule 20:09:29 #link https://launchpad.net/heat/+milestone/liberty-3 20:09:30 zaneb: do not forget about batching for different actions ;) 20:09:34 zaneb: did you and ramishra reach agreement, your patches appear to overlap and I wasn't sure which series to focus on 20:09:56 shardy: I asked him for reviews but never heard anything 20:09:58 I may be bugging you for blueprint status updates if it is not obvious 20:10:02 but I found some bugs on my own 20:10:13 hence currently reworking with mondo unit tests 20:10:48 zaneb: can you give us a quick summary on the difference between the series and why yours is better? ;) 20:10:59 by the time that's done hopefully my patches will be self-evidently correct ;) 20:11:40 zaneb: so far yours do look good and are definitely easier to review 20:11:45 stevebaker: well, the ResourceGroup implementation of rolling updates is really inefficient from a user point of view, because it comes with all the limitations of resource groups 20:12:03 the InstanceGroup implementation is also really inefficient because of a bug 20:12:11 fix for that is in https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bug/1488971,n,z 20:12:20 I just wanted to see if there was scope for collaboration vs wholesale abandonment of the work ramishra started 20:12:45 rabi wants to base ASG on RG, which is a mistake because it is a big regression 20:13:10 I want them all to use code from a common scaling library 20:13:26 (I've been saying this for ~2 years) 20:13:31 zaneb: Ok, cool, +1 on the common scaling library for sure 20:13:38 which I'm currently working on 20:13:43 yeah, that sounds sensible 20:13:44 I would also think that RG is just an ASG minus A 20:13:44 should have a lot more to show this week 20:13:50 I just wanted to avoid taking sides via review if you guys are mid-dispute/discussion ;) 20:13:51 depending on interruptions 20:13:53 which are legion 20:14:27 pas-ha: yes, that's the right way to think of it 20:14:41 stevebaker: when we plan to bump l3 tag and introduce FF ? I mean particular day ;) 20:14:43 pas-ha: not the other way round, which is how we have always organised things in the code 20:15:04 zaneb: wrt feature freeze this is probably a feature. i recall a bp for this 20:15:25 rabi had a bp I believe 20:15:31 \o (late) 20:15:39 I didn't want to try to steal that too ;) 20:15:53 zaneb: I'd be happy to ask for an extension for that if it is not all landed by ff 20:16:07 when is FF? 20:16:20 about sept 3 20:16:20 skraynev_: since we're hardly snowed under with feature reviews, I think FF on the day of l-3 is ok 20:16:36 ok, yeah, could be tight 20:16:42 stevebaker: yeah, I'm working on recursive-validation and wondering if I may have enought time wrt FPF 20:17:01 I'm trying to keep the patches either small + easy to review, or with lots of unit test coverage, or both 20:17:20 shardy: is there a lp bp for that? 20:17:57 stevebaker: There's a spec, https://review.openstack.org/#/c/197199/ 20:17:58 stevebaker: FYI, I know, that it's probably late, but we plan to upload on review all rich-network bp stuff... 20:18:05 It's been in review limbo for $months 20:18:12 * shardy sees if he raised the BP 20:18:33 shardy: ok, I'll push that through today 20:18:42 zaneb: what's about batching for RG? it has merged spec. 20:19:00 skraynev_: the code landed already 20:19:09 I'm refactoring it as we speak 20:20:08 stevebaker: Ok, thanks, I'll raise the LP BP, looks like there isn't one yet 20:20:08 zaneb: I meant this one 20:20:26 https://review.openstack.org/#/c/185385/ 20:20:34 zaneb, that's on_create batching 20:20:36 there is this https://blueprints.launchpad.net/heat/+spec/reorg-asg-code 20:20:52 skraynev_: oh, right 20:21:09 I don't think it should be a property, but once that is fixed I'm ok with landing it 20:21:19 it will conflict with what I'm doing though :/ 20:21:42 what is the proper place to add batch_create? 20:21:54 zaneb: I just think, that rolling update is a part of batching for create/update/delete .. 20:22:01 don't think this place is update_policy 20:22:21 ochuprykov: IMHO it is update_policy 20:22:55 stevebaker: raised https://blueprints.launchpad.net/heat/+spec/nested-validation 20:23:19 ochuprykov: if just that changes then that shouldn't be detected as a change to the resource, ergo it's an update_policy 20:24:01 zaneb: i mean it works on creation, why we add it to update_policy? 20:24:35 I freely admit the naming makes it a little odd, since we are in uncharted territory here 20:24:47 but it is a policy, not a property imo 20:24:51 ochuprykov: arguably, create is just a type of update which starts with an empty template ;) 20:24:52 shardy: I correctly understand, that re-authentification for expired token will be done as a bug and may be land after FF ? 20:25:08 * bug fix 20:25:12 zaneb: I don't think rabi's changes are against a bp. Could you develop against Qiming's old one? https://blueprints.launchpad.net/heat/+spec/reorg-asg-code 20:25:15 skraynev_, that is surely a bug 20:25:38 skraynev_: Yeah, I'm hoping to get time to work on that too, modulo everything else I'm hoping to get done 20:25:50 shardy: exactly my implementation of batch_create) 20:25:52 stevebaker: ok, if that's what floats your boat 20:26:07 zaneb: just to have something to track against 20:26:28 shardy: got it. thx 20:26:36 stevebaker: might it be easier to track if I created a new one? 20:26:37 zaneb: so i just add batching_actions to existiing rolling update in update_policy? 20:26:44 whiteboard is getting pretty long there 20:27:03 any other non-asg features to discuss? 20:27:08 zaneb: I would, you'll end up writing a new service if you follow that one ;) 20:27:25 ochuprykov: I actually think we should have separate policies for create and rolling_update 20:27:40 stevebaker: rich network. I hope to upload all related patches on review on Friday 20:27:48 on this week 20:27:53 zaneb: need to add create_policy to parent resource class? 20:27:58 skraynev_: cutting it fine ;) 20:28:04 ochuprykov: thinking about tripleo use case, I can can definitely see a case for them being different 20:28:15 skraynev_: but that would be good 20:28:44 stevebaker: ok. will spent all time on this work 20:28:46 zaneb: we have the third case actually, for full update 20:28:56 zaneb, even if they are the same, you can update it in the same updated template 20:28:58 ochuprykov: no, not a new section in the template. there can be multiple keys in the update_policy section. we just need another one in addition to rolling_update 20:29:18 stevebaker: probably we may go to the next topic about RG 20:29:20 ;) 20:29:31 pas-ha: true, because it's a policy! ;) still a major pita for tripleo though 20:29:36 #topic batching for create in ResourceGroup 20:29:54 stevebaker: because discussionalready in progress ;) 20:29:57 I thought we were already talking about this ;) 20:30:09 yep :) 20:30:10 zaneb: rolling_create would be fine? 20:30:31 y'all jumped the gun 20:30:47 ochuprykov: maybe 'batch_create'? but just bikeshedding at this point 20:30:48 let's make more - rolling_suspend :) 20:30:50 ochuprykov: wow. what's about actions in this case ? 20:31:09 separate policy for each action looks expansive and heavy, IMO 20:31:23 zaneb: batch_create looks fine tome too 20:31:30 zaneb: I'll leave it up to you whether you create a new bp or use reorg-asg-code, just target it to l-3 20:31:41 stevebaker: ack 20:32:04 zaneb: ochuprykov: what if we want to add batching for another actions ? 20:32:07 zaneb: i'm just wondering about repetition of max_batch_size and pause_time in theirs schemas 20:32:19 for "delete" it looks reasonable 20:32:47 skraynev: don't think about delete for now 20:33:00 zaneb: I'd prefer one batching policy with option for create/update or sonmething else 20:33:12 ochuprykov: easy to avoid that duplication in code. user has to duplicate that in their schema, but that is the price for being more flexible. I still think flexible wins 20:33:16 yes, deletes are usually lightweight on Nova (sans SC/SD with on_delete actions) 20:33:22 s/schema/template/ 20:33:50 skraynev_: What if you want to create everything as fast as possible, but keep your application running during update? 20:34:44 skraynev: ok, add batch_crete to update_policy and batching_actions to existing rolling update (i.g. update_existing and update) 20:34:47 shardy, as it is a policy - you update it in the same updated template, and use new value during update 20:35:02 shardy: you mean separate options (pause, min in service , etc) for each action ? 20:35:12 pas-ha: I guess, not very declarative though :\ 20:35:18 zaneb: ok, add batch_crete to update_policy and batching_actions to existing rolling update (i.g. update_existing and update) 20:35:33 skraynev_, shardy: what if you want to create your compute nodes in batches of 10, but then update them 1 at a time so you can migrate user workloads off <- potential tripleo use case 20:35:36 or we are talking on (auto)scaling actions? that might need different for create and update 20:35:39 skraynev_: yes, I think optionally supporting that (even if there's a way to specify common values) would be good 20:36:20 skraynev_: I need to go soon. can you topic Open Discussion, and endmeeting? 20:36:29 ochuprykov: I don't understand the batching_actions one 20:36:41 shardy: default batching parameters like for nested stacks :) 20:36:59 stevebaker: sure 20:37:29 zaneb: we have 3 actions: create (just discussed), update_existing (exactly what rolling_update do) and for full update 20:37:53 what does full update mean? how is that different from rolling_update? 20:38:34 if we need to create new resources, we will create in batches 20:38:40 in update 20:38:46 rolling update does that already 20:38:50 no 20:38:59 in only replace existing resources 20:39:19 and after that make resize 20:40:14 actually it depends 20:40:14 it may create extras to maintain the min_in_service 20:40:14 and if it does, it creates them in batches 20:40:24 but you're right, not if you change the size to > efft_capacity 20:40:34 so, for example, we have 10 res and want to have 20 res', ru updates 10 res in batches and then creates 10 res simultaneously 20:40:34 zaneb, ochuprykov: guys, do not you mind if we continue it in Open discussion ? 20:40:53 however, when I am done refactoring, that will be trivial to fix with a ~2-line change 20:41:06 skraynev_: go ahead 20:41:26 #topic Open Discussion 20:41:54 ochuprykov: I think we should see that as an enhancement to rolling_update, not a new policy. I'm planning to implement it anyway 20:42:29 zaneb: but i think rolling_update is only about updating existing resources 20:42:48 ochuprykov: my goal is to have all creates and updates go through the rolling update code :) 20:42:49 zaneb: so we need to distinguish this 2 cases 20:43:21 why should we? do we have to keep strict compat with something here? 20:43:39 pas-ha: not for RG 20:44:29 even for AWS ASG - does it really matter if new will be created in batches or not (like now)? 20:45:34 pas-ha: at least it has different time to update for this 2 cases 20:45:34 ochuprykov: so I guess the question is if the user increases the size of the RG and has differing update_policy for both batch_create and rolling_update, which would they expect to apply? 20:46:15 zaneb: rolling_update 20:46:25 zaneb: because they want to update 20:47:23 zaneb: and i'd like to add additional options, for applaying batching to replace or to full update 20:47:33 ochuprykov: I'd say rolling_update if the launch config/resource definition has also changed, and batch_create if it has not 20:47:57 zaneb, that's not too declarative 20:48:17 there should be one meaning, otherwise it is confusing 20:48:35 zaneb: i think about this actions in terms of a whole stack 20:48:52 or the RG itself 20:49:08 zaneb: create -- means stack create, not create of resource ( that may be performed on update) 20:50:28 the original purpose of rolling_update was to update to a new launch config (we corrupted the meaning when we adopted it) 20:50:40 zaneb: i;d like you to look at this spec https://review.openstack.org/#/c/183506/ 20:51:10 if you're rolling out new stuff you might have requirements to go slowly, stop and test that you might not have if just rolling out more of the same 20:51:15 I'm back 20:51:33 You could have a series of options, with the first taking precedence if the others aren't defined, batch_size->update_batch_size->rolling_update_batch_size 20:52:00 Then, you can either just define batch_size, or all the options if you want very granular control 20:52:06 stevebaker: cool. we are in Open discussion, but still talk about RG batching ;) 20:52:14 heh 20:52:53 shardy: that's actually sounding pretty good 20:53:01 zaneb: don't know the history of appearance of rolling_update 20:53:36 ochuprykov: it came from CloudFormation originally 20:53:53 or EC2, to be more precise 20:54:33 actually for that it would be cool to be able to pause the update until explicit signal from user (like it is possible with stack breakpoints) 20:54:35 AWS allows you to change the launch config for future instances *without* doing a rolling update 20:54:45 of existing ones 20:55:10 pas-ha: yes, we desperately need that for tripleo 20:55:18 I thought there was a review up at some point 20:55:48 zaneb: agree. I saw something similar... 20:55:50 not that I remeber of. I mean inf pause between batches 20:56:04 until user signal 20:56:09 zaneb: what is the conclusion? add batch_create to update_policy now? 20:56:36 pas-ha: I think ryansb put a review up for hooks between batches (i.e. same impl as breakpoints) 20:56:53 oh, cool, missed that 20:58:35 zaneb: we discussed the hook for update breakpoint in one of ramishras RG patches IIRC 20:58:39 anything else ? or we may finish ? 20:58:59 2 mins 20:59:02 but it's not actually been implemented 20:59:08 1 ;) 20:59:14 skraynev_: Ok to finish IMO :) 20:59:23 +1 20:59:31 ok cool. thx everyone ! 20:59:36 #endmeeting