Thursday, 2014-02-27

*** e0ne has quit IRC00:02
*** kevinbenton is now known as kevin00:05
*** kevin is now known as kevinbenton00:05
*** zns has quit IRC00:06
*** radez_g0n3 is now known as radez00:10
*** tango has quit IRC00:12
*** rcleere has quit IRC00:15
*** david_lyle_ has quit IRC00:23
*** matsuhashi has joined #heat00:23
*** saurabhs1 has quit IRC00:29
*** lnxnut has joined #heat00:32
*** saurabhs has joined #heat00:32
*** saurabhs has quit IRC00:35
*** shakayumi has joined #heat00:41
*** shakayumi has quit IRC00:41
*** shakayumi has joined #heat00:42
*** shakayumi has quit IRC00:42
*** pvaneckw has quit IRC00:44
*** nati_uen_ has joined #heat00:46
*** nati_uen_ has quit IRC00:47
*** nati_uen_ has joined #heat00:47
*** saurabhs has joined #heat00:49
*** nati_ueno has quit IRC00:49
*** ZZpablosan is now known as pablosan00:52
*** saurabhs has left #heat00:53
*** e0ne has joined #heat00:57
openstackgerritlvdongbing proposed a change to openstack/heat: Rename HARestarter to HARebuilder  https://review.openstack.org/7580300:58
*** jcru has quit IRC01:00
*** e0ne has quit IRC01:02
*** harlowja_away is now known as harlowja01:05
*** therve has quit IRC01:15
*** therve has joined #heat01:15
*** gokrokve_ has quit IRC01:16
*** gokrokve has joined #heat01:16
*** gokrokve has quit IRC01:20
*** lnxnut has quit IRC01:27
*** lnxnut has joined #heat01:27
*** lnxnut has quit IRC01:31
*** nati_ueno has joined #heat01:35
*** nati_uen_ has quit IRC01:38
*** nati_ueno has quit IRC01:44
*** nati_ueno has joined #heat01:44
*** nosnos has joined #heat01:46
*** e0ne has joined #heat01:57
*** gokrokve has joined #heat02:01
*** e0ne has quit IRC02:02
*** fandi has joined #heat02:03
*** DaveJ__ has quit IRC02:07
*** erkules_ has joined #heat02:22
*** erkules has quit IRC02:25
*** radez is now known as radez_g0n302:25
*** nati_uen_ has joined #heat02:32
*** nati_ueno has quit IRC02:36
*** rpothier_ has quit IRC02:39
*** achampion has joined #heat02:40
*** rpothier has joined #heat02:40
*** radez_g0n3 is now known as radez02:41
*** erkules_ has quit IRC02:44
*** erkules_ has joined #heat02:47
*** cody-somerville has joined #heat02:47
*** radez is now known as radez_g0n302:55
*** e0ne has joined #heat02:57
*** derekh is now known as derekh_afk02:57
openstackgerrithuangtianhua proposed a change to openstack/heat: Fix typo and remove unused code in nova_utils.py  https://review.openstack.org/7674002:59
*** e0ne has quit IRC03:01
*** nati_uen_ has quit IRC03:02
*** LiangC has joined #heat03:05
*** WinnieTsang has joined #heat03:15
*** zhiyan_ is now known as zhiyan03:16
*** pablosan has quit IRC03:30
*** daneyon_ has joined #heat03:32
*** pablosan has joined #heat03:34
*** ramishra has joined #heat03:34
*** david-lyle has joined #heat03:36
*** coolsvap has quit IRC03:42
*** tspatzier has joined #heat03:50
*** e0ne has joined #heat03:57
*** e0ne has quit IRC04:01
*** Linz has joined #heat04:08
*** tspatzier has quit IRC04:18
*** pablosan is now known as ZZpablosan04:20
*** lnxnut has joined #heat04:29
*** saurabhs has joined #heat04:35
*** tspatzier has joined #heat04:43
*** asalkeld has quit IRC04:44
*** tspatzier has quit IRC04:50
*** e0ne has joined #heat04:57
*** matsuhashi has quit IRC05:00
*** lnxnut has quit IRC05:02
*** e0ne has quit IRC05:02
*** coolsvap has joined #heat05:04
sdakeannoying I just got a 1760$ gift card for amazon05:13
sdakeand I spent 130$ there this morning05:13
* sdake grumbles05:13
sdakenight all05:13
*** Linz has quit IRC05:13
Slowergnight sdake05:15
*** coolsvap has quit IRC05:16
*** coolsvap has joined #heat05:17
*** sdake_ has joined #heat05:19
*** sdake_ has quit IRC05:19
*** sdake_ has joined #heat05:19
*** cmyster has joined #heat05:21
*** cmyster has quit IRC05:21
*** cmyster has joined #heat05:21
*** andersonvom has joined #heat05:32
*** maxskew_ has joined #heat05:42
*** maxskew has quit IRC05:45
*** nkhare has joined #heat05:54
*** killer_prince has quit IRC05:54
*** sballe has joined #heat05:55
*** e0ne has joined #heat05:57
openstackgerritSteve Baker proposed a change to openstack/heat: SignalResponder move signed URL deleting to its own method  https://review.openstack.org/7420505:58
openstackgerritSteve Baker proposed a change to openstack/heat: Resource type implementations for structured software config  https://review.openstack.org/7420605:58
openstackgerritSteve Baker proposed a change to openstack/heat: REST deployment metadata method  https://review.openstack.org/7420305:58
openstackgerritSteve Baker proposed a change to openstack/heat: RPC method to fetch deployments metadata  https://review.openstack.org/7420205:58
openstackgerritSteve Baker proposed a change to openstack/heat: OS::Nova::Server support for software config  https://review.openstack.org/6762505:58
openstackgerritSteve Baker proposed a change to openstack/heat: Resource type implementation for software deployment  https://review.openstack.org/6762405:58
openstackgerritSteve Baker proposed a change to openstack/heat: Nova server to ref cloud-config resources in user_data  https://review.openstack.org/6923805:58
*** sballe has quit IRC05:59
*** e0ne has quit IRC06:01
openstackgerritJenkins proposed a change to openstack/heat: Imported Translations from Transifex  https://review.openstack.org/7256606:09
*** lazy_prince has joined #heat06:11
*** sdake_ has quit IRC06:13
SpamapSstevebaker: o/ :)06:28
stevebaker\o06:28
stevebakerSpamapS: take a look at https://review.openstack.org/#/c/72533/306:28
*** andersonvom has quit IRC06:29
SpamapSstevebaker: poring over it right now06:30
SpamapSstevebaker: I like the clean separation we end up with between things we're setting at deploy time vs. things that are just set that way in the template.06:31
stevebakerSpamapS: the biggest hand-wavy bit is probably how the oac templates can be adapted to the new structure, especially splitting out the bm config/deployment. At least you can control the order of configs now so you can adopt a merge strategy on your image06:32
SpamapSstevebaker: we already have a merge strategy in oac for the sources...06:35
SpamapSstevebaker: so I think we can just extend that over this as well06:35
stevebakernice06:35
SpamapSstevebaker: it might even be worth it to just designate a key as one that gets merged.06:36
SpamapSso "deployments" is expected to be a list and just always gets merged by sorting the keys06:36
SpamapSerr06:37
SpamapSmerged as the list06:37
SpamapS... long day06:37
cmystermorning06:37
stevebakerSpamapS: here is the actual results from a metadata poll from a real (useless) template http://paste.openstack.org/show/70114/06:37
SpamapShm06:38
SpamapSso two levels of indirection, deployments and inputs06:38
SpamapSstevebaker: I really don't like that we have schema leaking in there06:38
stevebakerSpamapS: ignore the inputs, you can just use the config since the inputs have already been substituted into it by then06:38
SpamapSstevebaker: ah, and the config will just be part of the json with structuredconfig right?06:39
SpamapSI see escaped json in a string06:39
SpamapSwhich is... weird06:39
stevebakerhmm06:39
SpamapSI don't really see any situation where you'd want that.06:40
SpamapSstevebaker: anyway, unfortunately I have reached the cognitive wall for today06:41
stevebakerits because config is always a string, I'll have to think about that one06:43
stevebakerSpamapS: i too have reached making-sense threshold06:43
SpamapSstevebaker: with structureconfig, it makes a lot more sense to have config always be .. well.. structured. :)06:45
SpamapSanyway.. -> sleep06:45
*** saju_m has joined #heat06:48
*** lindsayk has joined #heat06:49
openstackgerritChenZheng proposed a change to openstack/heat: Sort requirement files in alphabetical order  https://review.openstack.org/7677506:50
*** amritanshu_RnD has joined #heat06:52
*** amritanshu_RnD is now known as Guest7567906:52
*** gokrokve has quit IRC06:53
*** chandan_kumar has joined #heat06:54
*** e0ne has joined #heat06:57
*** LiangC has quit IRC07:02
*** e0ne has quit IRC07:02
*** LiangC has joined #heat07:14
*** topol has quit IRC07:17
*** harlowja is now known as harlowja_away07:23
*** gokrokve has joined #heat07:23
*** gokrokve_ has joined #heat07:25
*** gokrokve has quit IRC07:28
*** daneyon_ has quit IRC07:29
*** gokrokve_ has quit IRC07:30
*** sirushti is now known as shortstop07:32
openstackgerritChenZheng proposed a change to openstack/heat: Sort requirement files in alphabetical order  https://review.openstack.org/7677507:32
*** jprovazn has joined #heat07:32
*** yogesh has joined #heat07:40
*** nati_ueno has joined #heat07:46
openstackgerritMitsuru Kanabuchi proposed a change to openstack/heat: Implement OS::Neutron::ExtraRoute as /contrib  https://review.openstack.org/7489907:52
*** e0ne has joined #heat07:57
*** e0ne has quit IRC08:02
*** skraynev_afk is now known as skraynev08:06
skraynevMorning08:06
cmystermorning08:07
*** saju_m has quit IRC08:10
*** saju_m has joined #heat08:10
*** chandan_kumar has quit IRC08:16
*** aignatov_ is now known as aignatov08:17
*** ramishra has quit IRC08:23
*** yogesh has quit IRC08:23
*** ramishra has joined #heat08:23
*** saju_m has quit IRC08:25
*** gokrokve has joined #heat08:26
*** saju_m has joined #heat08:26
*** LiangC has quit IRC08:28
*** jistr has joined #heat08:30
*** gokrokve has quit IRC08:30
*** chandankumar_ has joined #heat08:32
*** giulivo has joined #heat08:39
*** lindsayk has quit IRC08:40
*** LiangC has joined #heat08:41
*** jrist has quit IRC08:41
*** saurabhs has quit IRC08:41
*** shardy_afk is now known as shardy08:46
shardymorning all08:47
*** e0ne has joined #heat08:47
openstackgerritSergey Kraynev proposed a change to openstack/heat: Make OS::Nova::Server networks property updatable  https://review.openstack.org/7429908:48
*** aignatov is now known as aignatov_08:48
*** saju_m has quit IRC08:52
*** che-arne has quit IRC08:55
*** jrist has joined #heat08:55
*** aignatov_ is now known as aignatov08:55
*** aignatov is now known as aignatov_08:56
*** aignatov_ is now known as aignatov08:57
cmystermorning shardy08:58
cmystershardy: I have a question about https://code.engineering.redhat.com/gerrit/#/c/19960/ if error 2013 was removed so connection retries can be used, why not remove the other two ?09:00
cmyster3 even09:01
cmysterthose are all standard server connection issues09:01
*** ifarkas has joined #heat09:01
*** nati_ueno has quit IRC09:03
cmysterhttps://review.openstack.org/#/c/73314/ even :)09:04
shardycmyster: tbh the commit message and associated bug contain all the info I know09:05
shardycmyster: it's adding an error code, just backporting an oslo commit09:06
shardyhttps://bugs.launchpad.net/heat/+bug/127583809:06
cmysterI see, I got it the other way around first time I read it09:07
cmysterbut that was 7 am and before coffee :)09:08
shardy:)09:08
cmysteralso, I retested ./stack on a clean machine again (F20 updated to latest) and there were many issues with it.09:08
cmysterI had to give it a few tries before it worked, and the problem was always with starting Neutron service for the first time09:09
cmysterwent home pissed at it, did nothing, came today, and unstack stack fixed it09:09
shardySuch are the pitfalls of living life on the bleeding edge with devstack ;)09:10
cmysterbleeding edge--09:10
cmysterits why I run slack09:10
shardyWhen I get devstack working I tend to leave it alone and just selectively update stuff09:11
shardyand I have another box running RDO for a stable test platform09:11
cmystersame09:11
cmysterI just copt tempest's config locally and then I only need to change the IP09:11
shardypackstack has some options to install and configure tempest for you IIRC09:12
cmysterwas told that its broken ATM09:12
cmysteralso packstack will be replaced soon...09:12
cmysterno idea if it will be fixed09:13
*** asalkeld has joined #heat09:13
shardycmyster: it was broken a while back, but the bug I raised got closed, so I assumed it was now working:09:14
shardyhttps://bugzilla.redhat.com/show_bug.cgi?id=101671209:14
openstackgerritYongli He proposed a change to openstack-dev/heat-cfnclient: Exception message should not be localize  https://review.openstack.org/7680409:15
cmysterright, OK then, next time I need it09:15
*** jdag_ has quit IRC09:15
*** nati_ueno has joined #heat09:16
*** jdag_ has joined #heat09:19
*** gokrokve has joined #heat09:26
*** ramishra has quit IRC09:29
*** gokrokve has quit IRC09:31
*** e0ne has quit IRC09:34
*** e0ne has joined #heat09:34
*** derekh_afk is now known as derekh09:34
*** jamieh has joined #heat09:35
*** lindsayk has joined #heat09:40
*** lindsayk has quit IRC09:45
*** ramishra has joined #heat09:45
*** che-arne has joined #heat10:02
openstackgerritChenZheng proposed a change to openstack/python-heatclient: Sort requirement files in alphabetical order  https://review.openstack.org/7681210:04
*** tomek_adamczewsk has joined #heat10:04
*** alexpilotti has joined #heat10:12
*** aignatov is now known as aignatov_10:17
*** denis_makogon has quit IRC10:19
*** denis_makogon has joined #heat10:19
openstackgerritChenZheng proposed a change to openstack/heat: Sort requirement files in alphabetical order  https://review.openstack.org/7677510:23
*** gokrokve has joined #heat10:26
*** LiangC has quit IRC10:27
*** gokrokve_ has joined #heat10:28
*** gokrokve has quit IRC10:31
*** gokrokve_ has quit IRC10:32
*** mkollaro has joined #heat10:33
*** saju_m has joined #heat10:35
*** lindsayk has joined #heat10:41
*** faramir has joined #heat10:43
*** lindsayk has quit IRC10:45
*** jprovazn has quit IRC10:47
*** mkollaro has quit IRC10:52
*** mkollaro has joined #heat10:52
*** jufeng has joined #heat10:57
*** aignatov_ is now known as aignatov11:03
*** julienvey1 has quit IRC11:03
*** jufeng has quit IRC11:05
*** aignatov is now known as aignatov_11:05
*** aignatov_ is now known as aignatov11:07
*** alexpilotti has quit IRC11:08
*** Tross has quit IRC11:12
openstackgerritVyacheslav Vakhlyuev proposed a change to openstack/heat: Fix comparison with singletons in unit tests  https://review.openstack.org/7620911:12
*** jprovazn has joined #heat11:14
*** nosnos has quit IRC11:14
*** jamieh has quit IRC11:16
*** jamieh has joined #heat11:19
*** jufeng has joined #heat11:19
*** fandi has quit IRC11:20
*** Tross has joined #heat11:20
*** gokrokve has joined #heat11:26
*** ramishra has quit IRC11:28
*** ramishra has joined #heat11:28
*** gokrokve has quit IRC11:31
*** lindsayk has joined #heat11:41
*** lindsayk has quit IRC11:45
*** alexpilotti has joined #heat11:46
*** jufeng has quit IRC11:57
*** blomquisg has quit IRC11:59
*** jufeng has joined #heat12:09
*** gokrokve has joined #heat12:26
*** gokrokve has quit IRC12:30
cmysterdang, just hit that non zero exit code again12:34
*** dims_ has quit IRC12:36
*** nkhare has quit IRC12:38
*** lindsayk has joined #heat12:42
*** lindsayk has quit IRC12:46
*** rbuilta has joined #heat12:49
*** saju_m has quit IRC12:53
*** rpothier has quit IRC12:54
openstackgerritSteven Hardy proposed a change to openstack/heat: migrate User/AccessKey resources to StackUser base class  https://review.openstack.org/7276313:09
openstackgerritSteven Hardy proposed a change to openstack/heat: StackUser add _delete_keypair function  https://review.openstack.org/7276213:09
openstackgerritSteven Hardy proposed a change to openstack/heat: engine: allow stack_user_project users to retrieve stack  https://review.openstack.org/7130013:09
openstackgerritSteven Hardy proposed a change to openstack/heat: Add test for StackUser._create_keypair  https://review.openstack.org/7276113:09
openstackgerritSteven Hardy proposed a change to openstack/heat: Add config options to specify stack domain admin  https://review.openstack.org/7603513:09
openstackgerritSteven Hardy proposed a change to openstack/heat: StackUser add suspend/resume support  https://review.openstack.org/7193013:09
openstackgerritSteven Hardy proposed a change to openstack/heat: migrate StackUser base class to stack domain users  https://review.openstack.org/7121013:09
openstackgerritSteven Hardy proposed a change to openstack/heat: heat_keystoneclient add delete_stack_domain_user_keypair  https://review.openstack.org/7192913:09
openstackgerritSteven Hardy proposed a change to openstack/heat: Modify stack_user_domain config option to take domain ID  https://review.openstack.org/7397813:09
*** jufeng has quit IRC13:09
*** LiangC has joined #heat13:15
*** david-lyle has quit IRC13:16
*** achampion has quit IRC13:20
*** jdob has joined #heat13:25
*** gokrokve has joined #heat13:26
sdakemorning13:27
shadowerafternoon13:28
shardyhi sdake13:29
*** gokrokve has quit IRC13:30
pscheie_I'13:31
pscheie_I've got some heat templates I inherited.  In them, for some instances (resources), files are created on the instances.13:32
pscheie_The files are constructed using the Fn::Join function.13:33
*** blomquisg has joined #heat13:33
pscheie_But in some (most, actually) cases, the Fn::Base64 function is called on those files as well.13:34
pscheie_What would be the point of the Base64 encoding?  The files are just text files on the instances.13:34
shardypscheie_: Fn::Base64 doesn't do anything atm13:34
shardyso there's no point in using it at all13:35
pscheie_shardy, oh, I like that answer!  I was hoping to remove the Base64 calls.13:35
*** che-arne has quit IRC13:36
shardyhttps://bugs.launchpad.net/heat/+bug/107295513:36
shardypscheie_: ^^ looks like we've decided not to fix it13:36
shardynobody has complained since the bug was raised in 2012, so probably reasonable :)13:37
sdakehey shardy13:37
pscheie_What would be a use case for base64?13:37
shardypscheie_: passing a non-text file in userdata13:38
sdakeAWS requires data to be base64 encoded to passs it to the userdata13:38
sdakeHeat base64 encodes all data already when passed to the userdata13:38
sdakeso essentially, the Base64 step is unnecessary, and will result in non-usable userdata ;=)13:38
pscheie_So, in heat it's redundant?13:38
sdakeright13:39
sdakeand if it were actually implemented hte result would be a nonworking system13:39
*** ramishra has quit IRC13:40
pscheie_Is it necessary if one wants to maintain compatibility with AWS?13:40
sdakeyes13:40
sdakethe base64 intrinsic in heat is a noop13:41
pscheie_Ah, that's probably why the calls are in these templates I inherited.13:41
*** ramishra has joined #heat13:42
*** lindsayk has joined #heat13:42
shadowerhey, this is probably a stupid question, but I'm at my wits' end. I'm looking at this: https://github.com/openstack/heat/blob/master/heat/engine/parser.py#L547 which in turn calls this: https://github.com/openstack/heat/blob/master/heat/engine/update.py#L35. The __init__ method excepts 3 positional arguments (excluding self), but the caller only gives two (i.e. previous_stack is not passed). How does this not raise TypeError every time?13:42
pscheie_sdake, I see that the Update Failure Recovery blueprint has been assigned to (none).13:43
pscheie_I suppose that means its implementation is now further out on the horizon (?)13:43
sdakeit got dropped from i313:44
shardyshadower: the first argument is self13:44
sdakeI suspect our next fearless ptl leader will address it in early Juno :)13:44
shardyi.e the caller self==existing_stack13:44
sdakeshardy it has 4 arguments13:44
sdakeprevious_stack is the last argument13:44
shadowershardy: of course it is, I'm an idiot. Thanks13:45
sdakenm ;-)13:45
shardysdake: yeah, we're passing 4 arguments13:45
*** rpothier has joined #heat13:45
*** andersonvom has joined #heat13:47
*** lindsayk has quit IRC13:47
*** rbuilta has quit IRC13:48
*** rbuilta has joined #heat13:51
*** zns has joined #heat13:52
*** aignatov is now known as aignatov_13:58
*** tomek_adamczewsk has quit IRC13:59
*** sballe has joined #heat14:02
*** aweiteka has joined #heat14:04
*** fandi has joined #heat14:05
*** che-arne has joined #heat14:05
*** aignatov_ is now known as aignatov14:06
*** che-arne has quit IRC14:07
openstackgerritAnderson Mesquita proposed a change to openstack/heat: Alter stack_count_all_by_tenant to stack_count_all  https://review.openstack.org/7085314:16
openstackgerritAnderson Mesquita proposed a change to openstack/heat: Change Stack timestamps to save correct info  https://review.openstack.org/7451914:16
openstackgerritAnderson Mesquita proposed a change to openstack/heat: Unscoped List Stacks  https://review.openstack.org/6304114:16
openstackgerritAnderson Mesquita proposed a change to openstack/heat: Fix stack_get_all call on stack watcher  https://review.openstack.org/7549514:16
openstackgerritAnderson Mesquita proposed a change to openstack/heat: Change Resource timestamps to save correct info  https://review.openstack.org/7664414:16
openstackgerritAnderson Mesquita proposed a change to openstack/heat: Add project to unscoped stack list response  https://review.openstack.org/7278914:16
*** ramishra has quit IRC14:23
*** arbylee has joined #heat14:26
*** gokrokve has joined #heat14:26
sgranhi14:28
sgranjust a question that's not clear from the docs: can you use a waitcondition and an autoscaling group?14:28
sgranI'd like to do rolling scaling group updates, and have the iteration wait for the first to be 'in service' before moving to the second.  This seems like a good way to do it, but it's not clear what the behavior will be14:29
sgranI'm using the havana code base right now14:30
shardysgran: I don't think you can, because there's no way to specify the waitcondition via the LaunchConfig14:30
sgranI was going to put it in the UserData in the LaunchConfig14:31
shardysgran: however, there's RollingUpdate interfaces which provide a rolling update capability including a PauseTime14:31
shardyunfortunately that just missed Havana..14:31
*** gokrokve has quit IRC14:31
*** jamieh has quit IRC14:31
shardysgran: https://review.openstack.org/#/c/43571/14:32
shardyOr rather all of https://review.openstack.org/#/q/status:merged+project:openstack/heat+branch:master+topic:bp/as-update-policy,n,z14:34
* IgorYozhikov is now away: went away...14:34
*** IgorYozhikov is now known as IYozhikov_away14:34
shardysgran: there is work going on which should enable native autoscaling resources, so in future you should be able to define a nested stack containing an instance, software configuration, network, wait condition, whatever & scale that nested stack out14:35
shardybut that work has not yet landed (radix posed an initial patch)14:36
shardyposted even14:36
radixyeah, please try it :)14:36
sgranwhat does the 'count' parameter do if it's greater then one?14:36
sgrangreater than one*14:36
shardyhttps://review.openstack.org/#/c/74229/14:36
*** dims has joined #heat14:37
*** jamieh has joined #heat14:37
*** fandi has quit IRC14:38
shardysgran: it means you need to recieve $count signals before the WaitCondition is declared complete14:39
shardySo you could for example launch an AutoScaling group then wait for $count instances to be up before continuing14:40
sgranso if you then modify the LaunchConfig to be a new image id, what would happen?14:41
sgranwould it again wait on the condition for each one?  Or would it cycle the whole lot and wait for all of them to check in before declaring completion ?14:42
sgranif it does the latter, this might not be what I'm looking for14:43
shardysgran: Yeah it's the latter14:43
sgranoh well, never mind :)14:43
shardysgran: Although ignoring waitconditions for a second, the RollingUpdates does do what you want I think14:43
sgranI'll take a look at the rolling update stuff in due course14:43
sgranalmost - it sounds like it has a hardcoded timer14:44
shardyIt will do batched replacement of the instances not destroy the whole lot14:44
*** Guest75679 has quit IRC14:44
sgranI'd rather let it proceed if it finishes sooner14:44
*** jcru has joined #heat14:44
*** radez_g0n3 is now known as radez14:44
shardysgran: Yeah, we don't have a good solution for that, but we're working on it14:44
sgranso I could set a timer of an hour, but mostly expect an instance replacement to take a few minutes14:44
*** e0ne_ has joined #heat14:46
*** jcru_ has joined #heat14:47
*** jcru has quit IRC14:47
*** jcru_ has quit IRC14:49
*** spzala has joined #heat14:49
*** e0ne has quit IRC14:49
*** jcru has joined #heat14:49
*** ramishra has joined #heat14:54
*** cmyster has quit IRC14:56
*** ramishra has quit IRC14:56
*** ramishra has joined #heat14:57
*** lindsayk has joined #heat14:57
*** che-arne has joined #heat15:01
*** sabeen has joined #heat15:01
*** faramir has quit IRC15:01
*** ramishra has quit IRC15:02
*** che-arne has quit IRC15:04
*** tomek_adamczewsk has joined #heat15:06
*** ramishra has joined #heat15:07
*** jmckind has joined #heat15:08
*** gokrokve has joined #heat15:11
*** ZZpablosan is now known as pablosan15:11
*** gokrokve_ has joined #heat15:12
*** maxskew has joined #heat15:12
*** che-arne has joined #heat15:13
*** lnxnut has joined #heat15:14
*** jcru_ has joined #heat15:14
*** nati_uen_ has joined #heat15:14
*** gokrokve has quit IRC15:15
*** tspatzier has joined #heat15:16
*** blamar_ has joined #heat15:16
*** arbylee1 has joined #heat15:16
*** SpamapS_ has joined #heat15:16
*** _jmp___ has joined #heat15:17
*** nkhare has joined #heat15:17
*** zns has quit IRC15:18
*** _jmp___ is now known as _jmp_15:19
*** john-n-seattle has joined #heat15:20
*** jcru has quit IRC15:21
*** arbylee has quit IRC15:21
*** aweiteka has quit IRC15:21
*** rbuilta has quit IRC15:21
*** jprovazn has quit IRC15:21
*** mkollaro has quit IRC15:21
*** nati_ueno has quit IRC15:21
*** giulivo has quit IRC15:21
*** chandankumar_ has quit IRC15:21
*** maxskew_ has quit IRC15:21
*** _jmp__ has quit IRC15:21
*** swygue has quit IRC15:21
*** blamar has quit IRC15:21
*** SpamapS has quit IRC15:21
*** john-n-s- has quit IRC15:21
*** blamar_ is now known as blamar15:21
*** ramishra has quit IRC15:21
*** achampion has joined #heat15:22
*** aignatov is now known as aignatov_15:22
*** ramishra has joined #heat15:22
*** funzo has quit IRC15:22
*** funzo_ has joined #heat15:22
*** aignatov_ is now known as aignatov15:24
*** vijendar has joined #heat15:28
*** jprovazn has joined #heat15:28
*** david-lyle has joined #heat15:29
*** zns has joined #heat15:29
*** chandankumar_ has joined #heat15:30
*** swygue has joined #heat15:30
*** giulivo has joined #heat15:30
*** aweiteka has joined #heat15:30
*** mkollaro has joined #heat15:32
*** mkollaro has quit IRC15:32
*** mkollaro has joined #heat15:32
*** rbuilta has joined #heat15:32
*** tspatzier has quit IRC15:32
*** funzo_ is now known as funzo15:33
*** topol has joined #heat15:35
*** fandi has joined #heat15:38
*** alexheneveld has joined #heat15:38
*** Tross has quit IRC15:39
*** lazy_prince is now known as killer_prince15:41
*** dims has quit IRC15:41
*** ramishra has quit IRC15:41
*** ramishra has joined #heat15:42
*** LiangC has quit IRC15:42
*** dims has joined #heat15:42
*** rcleere has joined #heat15:47
*** nati_uen_ has quit IRC15:49
*** ramishra has quit IRC16:00
*** jmckind has quit IRC16:01
*** tims1 has joined #heat16:04
*** nkhare has quit IRC16:06
*** tims has quit IRC16:07
openstackgerritJason Dunsmore proposed a change to openstack/heat: Handle API limit exception in nova_utils.refresh_server  https://review.openstack.org/7166016:08
*** zhiyan has quit IRC16:09
*** coolsvap has quit IRC16:10
*** zhiyan has joined #heat16:11
*** aignatov is now known as aignatov_16:11
*** Linz has joined #heat16:13
sergmelikyanGuys, why Router Interface may be deleted before floating IP? I am getting this error during stack deletion:16:15
sergmelikyan[8:11:28 PM] Alexander Tivelkov: NeutronClientException: 409-{u'NeutronError': {u'message': u'Router interface for subnet 24a73454-7264-4c03-9464-22050794f679 on router 416488de-d38a-4d43-b475-8ecdf57202c5 cannot be deleted, as it is required by one or more floating IPs.', u'type': u'RouterInterfaceInUseByFloatingIP', u'detail': u''}}16:15
*** zhiyan has quit IRC16:20
*** jmckind has joined #heat16:20
larskssergmelikyan: Did you assign any floating ips outside of heat?16:25
larsks(That is, did you ever run "nova add-floating-ip" or the neutron equivalent)?16:25
*** julienvey has joined #heat16:30
*** tims1 has quit IRC16:33
shardycheck-tempest-dsvm-neutron-heat-slow SUCCESS in 22m 01s16:34
shardy\o/16:35
*** tomek_adamczewsk has quit IRC16:40
*** aweiteka has quit IRC16:41
*** mkollaro has quit IRC16:42
*** aignatov_ is now known as aignatov16:43
*** lindsayk has quit IRC16:44
*** radez is now known as radez_g0n316:44
*** lindsayk has joined #heat16:44
*** radez_g0n3 is now known as radez16:47
*** daneyon has joined #heat16:48
*** zns has quit IRC16:50
*** zns has joined #heat16:50
*** tomek_adamczewsk has joined #heat16:52
*** jistr has quit IRC17:00
*** killer_prince is now known as lazy_prince17:01
*** pablosan has quit IRC17:01
*** pablosan has joined #heat17:02
*** jmckind has quit IRC17:04
*** e0ne_ has quit IRC17:06
*** Linz_ has joined #heat17:07
*** alexheneveld has quit IRC17:07
*** mkollaro has joined #heat17:08
*** Linz has quit IRC17:10
daneyonshardy: I made the changes that you requested to this patch: https://review.openstack.org/#/c/74450/  When you have a moment, do you mind reviewing?17:12
shardydaneyon: lgtm, but I think you actually want stevebaker as he -1'd it ;)17:14
shardyLooks like his comment was addressed so I'll go ahead and approve17:15
daneyon:-)17:15
openstackgerritA change was merged to openstack/heat-templates: Adds Support for OpenShift Origin v3.0 on Fedora 19  https://review.openstack.org/7445017:20
openstackgerritA change was merged to openstack/heat-templates: Added HOT Template for 2 Servers Without Floating IPs  https://review.openstack.org/6555217:21
*** sdake_ has joined #heat17:24
*** aweiteka has joined #heat17:26
*** fandi has quit IRC17:28
*** fandi has joined #heat17:29
*** aignatov is now known as aignatov_17:35
*** gokrokve_ has quit IRC17:37
*** radez is now known as radez_g0n317:37
*** gokrokve has joined #heat17:37
daneyonshardy: Thanks again for your help.17:39
*** radez_g0n3 is now known as radez17:40
shardydaneyon: np17:40
*** julienvey has left #heat17:41
*** gokrokve has quit IRC17:41
*** zns has quit IRC17:43
*** dtalton has joined #heat17:45
*** dtalton has left #heat17:46
*** derekh has quit IRC17:46
*** WinnieTsang has quit IRC17:47
*** SpamapS_ is now known as SpamapS17:48
*** SpamapS has quit IRC17:48
*** SpamapS has joined #heat17:48
SpamapSg'morning17:51
*** jmckind has joined #heat17:52
*** pvaneck has joined #heat17:54
*** aignatov_ is now known as aignatov17:57
shadowerhey17:59
*** cadenzajon has joined #heat18:00
*** tomek_adamczewsk has quit IRC18:03
*** harlowja_away is now known as harlowja18:12
*** lindsayk has quit IRC18:13
*** alexheneveld has joined #heat18:15
*** tango has joined #heat18:21
*** shakayumi has joined #heat18:22
*** alexheneveld has quit IRC18:25
*** lindsayk has joined #heat18:25
*** WinnieTsang has joined #heat18:27
*** lindsayk1 has joined #heat18:28
*** bvandenh_ has quit IRC18:29
*** lindsayk has quit IRC18:29
*** shakayumi has quit IRC18:29
*** gokrokve has joined #heat18:30
*** gokrokve has quit IRC18:30
*** gokrokve_ has joined #heat18:30
*** shakayumi has joined #heat18:30
*** shakayumi has quit IRC18:30
*** saurabhs has joined #heat18:30
*** lindsayk1 has quit IRC18:31
*** cadenzajon_ has joined #heat18:33
openstackgerritSteven Hardy proposed a change to openstack/heat: Make user_creds_id a parser.Stack attribute  https://review.openstack.org/7692818:33
openstackgerritSteven Hardy proposed a change to openstack/heat: fix DB API user_creds_get for non-existent ID  https://review.openstack.org/7692918:33
openstackgerritSteven Hardy proposed a change to openstack/heat: Add user_creds_delete to the DB API  https://review.openstack.org/7693018:33
openstackgerritSteven Hardy proposed a change to openstack/heat: Delete user_creds on stack delete  https://review.openstack.org/7693118:33
sdake_SpamapS any word on the retry idea18:33
sdake_SpamapS the blueprint shadower pointed out18:34
*** chandan_kumar has joined #heat18:35
*** cadenzajon has quit IRC18:35
SpamapSsdake_: I'm still working through morning emails, and handing off a tripleo issue. :-P18:35
sdake_if you could take a quick look so shadower can head to bed that would rock :)18:35
shadowersdake_: I'm waiting for stevebaker anyways18:35
shadowerno worries18:35
sdake_shadower ok cool18:36
*** e0ne has joined #heat18:36
shadowerlike I won't be here *all* night but I can be aronud another 2-3 hrs18:36
sdake_stevebaker is not on a mon-fri schedule - hopefully he is in today :)18:37
*** saju_m has joined #heat18:37
shadoweroh18:38
sdake_aweiteka any luck with the instance group feature18:39
sdake_I think jpeeler could use your work in the origin template or you two could collaborate18:40
aweitekasdake, i haven't had a chance to attempt18:40
SpamapSshadower: ah you're up here :)18:40
sdake_aweiteka cool just curious18:40
aweitekasdake, yeah, i'm happy to collab on this stuff.18:40
SpamapSsdake_: ok so caught up now ..18:40
SpamapSshadower: the "escape hatch" is exactly what I'm pursuing now.18:41
*** saju_m has quit IRC18:41
aweitekajpeeler, ping me if you're stuck or anything. no sense in making this too painful :) funzo helped me a bunch18:41
jpeeleraweiteka: what work is sdake referring to?18:42
SpamapSBasically instead of using state in the db to keep track of completed updates, I'm just going to tell stack-update to short-circuit to the resource that failed.18:42
SpamapSzaneb: ^18:42
zanebwhy does launchpad insist on sending me emails from myself?18:42
tangoSpamapS: Hi Clint, I am available to help on the retry update if you need a hand with looking at code or coding something.18:43
SpamapSzaneb: launchpad is inherently narcissistic.18:43
aweitekajpeeler, here's my contrib to openshift heat https://github.com/openstack/heat-templates/tree/master/openshift-enterprise/heat/neutron/highly-available18:43
SpamapStango: thank you!18:43
tangojust let me know what I can work on18:43
jpeelerah the HA stuff, ok cool18:43
*** yogesh has joined #heat18:44
zanebSpamapS: by 'short-circuit' you mean 'kill and replace'?18:44
SpamapSzaneb: tell me if you think this will work. If I can push down hints to stack-update to tell it where in the graph to pick up the update.. we don't have to store state.. we can just continue from wherever the admin says to continue from.18:44
zanebwait, no I misread that18:44
SpamapSzaneb: It puts the automation off, and is not a long term solution.18:45
SpamapSzaneb: but we can basically say what resources to consider "already done" and then the update will continue diffing from there.18:45
zanebSpamapS: so, your issue will be that after an update we store the 'new' template in the database... so you won't know how to update the subsequent resources as it stands18:46
zanebthis may be fixable by keeping the old template in the db18:46
SpamapSthat's after a successful update though, right?18:46
SpamapSIf it fails we lose the new template we were working on.18:47
SpamapSwhich is "the problem"18:47
zaneband I guess if you skip ahead to the failing ones, you won't have the problem of trying to re-update stuff that has been updated18:47
zanebSpamapS: depends if rollback is enabled. if not then it goes to the new one18:47
zanebeven if the update failed18:48
SpamapSoh, ok that does make things more complicated18:48
zanebhttps://github.com/openstack/heat/blob/master/heat/engine/parser.py#L59018:48
SpamapSzaneb: well I'm poking at the state-storing-and-loading stuff now.. but .. it is Thursday.... :-P18:48
SpamapSsome places it is already Friday :-P18:49
*** shakayumi has joined #heat18:50
*** shakayumi has quit IRC18:50
tangoSpamapS: would it help to consider the case of rollback disabled first and get retry working, then consider the case of rollback enabled?18:50
zanebwe do have the backup stack in the db now... maybe we can do something clever with the template18:50
*** shakayumi has joined #heat18:51
*** shakayumi has quit IRC18:51
*** Linz_ has quit IRC18:51
*** saju_m has joined #heat18:51
*** Linz has joined #heat18:51
*** shakayumi has joined #heat18:51
SpamapStango: we can't use rollback ever.18:52
*** shakayumi has quit IRC18:52
SpamapStango: the orchestration of rolling back a database schema change is really complex. We have to roll forward, with old software and new software, to make it work.18:52
*** shakayumi has joined #heat18:52
*** shakayumi has quit IRC18:53
*** shakayumi has joined #heat18:53
tangoSpamapS: got it. Then should we say that if the user wants to be able to retry failed update, make sure to disable rollback?18:55
SpamapSshadower: I'll have some conclusions from my efforts today by the time you are back online.18:55
shadowerSpamapS: okay, cool. Let me know if I can help anything18:55
shadower*with anything18:55
SpamapStango: rollback must be explicitly enabled, so I think we're o-k there. There's nothing to retry if rollback is enabled anyway, as things are put back in place (in theory.. unless rollback fails too.. doh)18:55
SpamapSshadower: much appreciated18:55
*** dtalton has joined #heat18:56
shadowerSpamapS: not sure it's feasible for everyone to work on the same thing but if the work can be split out, I'm happy to help18:56
SpamapSshadower: that is what I'm hoping I can do18:59
*** mkollaro has quit IRC18:59
shadowercool18:59
SpamapSshadower: if you wanted to spike on my short-circuit idea.. by all means go for it. I'm going to push a little further on recovering automatically before switching to that.18:59
openstackgerritJeff Peeler proposed a change to openstack/heat: Document schema properties for Neutron router resources  https://review.openstack.org/7666518:59
SpamapSshadower: but I suspect you are about done for the next 12 hours or so :)19:00
*** dtalton has quit IRC19:00
*** che-arne has quit IRC19:01
shardyIf anyone feels like passing on some review-love, I finally got the gate tests (including the non-voting slow job) working for the instance-users remaining patches:19:01
openstackgerritJason Dunsmore proposed a change to openstack/heat: Make the first line of every file consistent.  https://review.openstack.org/7659119:01
shardyhttps://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:bug/1089261,n,z19:01
openstackgerritJeff Peeler proposed a change to openstack/heat: Document schema properties for Neutron subnet resource  https://review.openstack.org/7665519:01
shardystevebaker: ^^19:01
shadowerSpamapS: yea I'm no longer thinking straigth19:03
*** kebray has joined #heat19:13
sdakeSpamapS would the retry of the lifecycle operations mitigate your problems?19:13
stevebakermorning19:13
sdakemorning stevebaker19:14
shadowermorning19:14
* stevebaker hasn't read any backscroll yet19:14
SpamapSsdake: I'm not sure I understand what you mean.19:15
SpamapSsdake: any retrying automatically at an individual operational level is insufficient. "shit happens"19:16
*** randallburt has joined #heat19:16
*** randallburt has quit IRC19:16
sdakeit solves the overload problem doesn't it?19:16
*** randallburt has joined #heat19:16
SpamapSmaybe19:16
*** tspatzier has joined #heat19:17
SpamapSsdake: and what about the next neutron bug or novaclient bug which renders us dead?19:17
sdakeis overload the only problem you see in practice?19:17
SpamapSsdake: no, I've also seen network issues cause a timeout.19:17
sdakewell I get its not perfect, thats why I used the word mitigate :)19:17
sdakewouldn't a retry mitigate network issues?19:18
SpamapSsdake: one thing that has happened, for instance, is that we reboot a machine to update its software (with rebuild) and the machine takes longer than usual to come back up and ping its' wait condition. Oops.. stack dead.19:18
SpamapSWe could just have days long timeouts.19:19
*** randallburt1 has joined #heat19:19
SpamapSsdake: mitigation is really not what I'm looking for. There is a massive hole that mitigation makes smaller.. but we can only shrink it to the size of one stack + one problem.19:19
sdakethe problem you just raised - waitconditions not being responded to in a specific amount of time can be mitigated by changing the timeout value of the wait conditions19:19
SpamapSsdake: I don't think you're understanding me. We need a plan to recover from the problems we don't know about.19:20
sdakeSpamapS I understand mitigation is not what you want, but its better then nothing :)19:20
stevebakershardy: yay for passing heat-slow :)19:20
SpamapSIts really not better than nothing.19:20
*** randallburt has quit IRC19:20
SpamapSIt just delays the impending doom.19:21
sdakeso some large percentage of time it mitigates, and some small percentage of time it delays impending doom19:21
SpamapSlike sending terminators back in time to save john conner.19:21
sdakei think this is better then all the time impending doom19:21
*** alexheneveld has joined #heat19:21
SpamapSsdake: I think it is a false sense of "better".19:22
SpamapSanyway, I think I need to drop off irc/email/etc. so I can make progress.. so.. many... distractions.. ;)19:22
sdakeSpamapS enjoy19:23
*** aignatov is now known as aignatov_19:23
stevebakersdake: school has been back for a month now, so I'm back to working gentlemen's hours19:23
sdakestevebaker sounds good19:23
sdakeschool still in session on the other side of the planet :(19:23
sdakeI already went through school once19:24
sdakeI dont see a great need to do it two more times19:24
*** chandan_kumar has quit IRC19:24
stevebakershardy: do you run tempest locally against domain users?19:24
*** zhiyan_ has joined #heat19:25
*** jcru_ has quit IRC19:34
*** kebray has quit IRC19:35
*** saju_m has quit IRC19:35
*** mkollaro has joined #heat19:37
*** zns has joined #heat19:40
*** akuznetsov has quit IRC19:43
*** akuznetsov has joined #heat19:43
*** chandan_kumar has joined #heat19:50
*** WinnieTsang has quit IRC19:53
*** WinnieTsang has joined #heat19:55
openstackgerritJeff Peeler proposed a change to openstack/heat: Add host_routes property to Neutron subnet resource  https://review.openstack.org/7695019:56
*** radez is now known as radez_g0n319:57
*** nati_ueno has joined #heat19:59
*** radez_g0n3 is now known as radez20:01
*** fandi has quit IRC20:06
*** zns has quit IRC20:11
andersonvomshardy: ping. =]20:11
*** TonyBurn has quit IRC20:12
andersonvomshardy, sdake: does randallburt1's comments address your concerns regarding https://review.openstack.org/#/c/72745/5/heat/engine/resources/server.py ?20:12
*** tspatzier has quit IRC20:16
*** cmyster has joined #heat20:17
openstackgerritA change was merged to openstack/heat: Replace '+' with string interpolation operation  https://review.openstack.org/7579420:17
cmysterevening20:17
*** zns has joined #heat20:18
*** therve_ has joined #heat20:20
*** alexheneveld has quit IRC20:21
*** kebray has joined #heat20:22
*** therve_ has quit IRC20:25
*** kebray has quit IRC20:32
*** kebray has joined #heat20:33
*** rbuilta has quit IRC20:37
*** randallburt1 has quit IRC20:39
*** daneyon has quit IRC20:43
*** zns has quit IRC20:48
SpamapSzaneb: around?20:50
zanebyo20:50
SpamapSzaneb: so I'm poking at this.. and it ocurrs to me that I need to store not just the snippet, but the _resolved_ snippet with all the runtime data...20:50
zanebSpamapS: yes, because the parameters and whatnot change too :(20:51
SpamapSyeah20:51
SpamapSwhich also leads to zomg 3 way merge.. :-P20:51
zanebthat also occurred to me only after I wrote that blueprint20:51
SpamapSwhich makes me think.. hmm.. maybe a retry command is in order..20:51
SpamapStango: ^20:51
tangoSpamapS: Hi Clint20:52
SpamapSzaneb: so perhaps what is needed is to store the new "to be updated to" stack in a state that makes it invisible to the user.20:52
SpamapStango: I believe you had suggested that we need a separate retry and update. :)20:53
zanebbackup stack?20:53
SpamapSHaven't looked closely at that.20:53
SpamapSbut that seems backwards20:53
zanebtrue20:54
SpamapSalso.. zomg.. swallowing DB exceptions in resource code is making me twitch20:54
zanebwe do that?20:54
SpamapSzaneb: ok so if I store the new target stack in a similar way to the backup stack though, that may be what I need...20:54
SpamapSzaneb: yes a lot20:54
tangoSpamapS: you mean the difference between changing the template and using the previous template?20:55
zanebSpamapS: you mean how we just fail the resource rather than bailing out of the whole operation?20:55
SpamapSzaneb: https://git.openstack.org/cgit/openstack/heat/tree/heat/engine/resource.py#n73620:56
SpamapSand.. many more20:56
*** ifarkas has quit IRC20:56
SpamapSzaneb: no, we log error and then put our hands on our ears and go "LALALALALALALA"20:56
zanebick20:56
SpamapSzaneb: https://git.openstack.org/cgit/openstack/heat/tree/heat/engine/resource.py#n757 worse20:56
SpamapSalso it is blanket Exception20:56
zanebI'm pretty sure that's only there because hald of the unit tests would fail without it20:56
SpamapSso we miss things like "oh thats not actually a resource object you're trying to call methods on"20:57
SpamapSanyway, focus... must focus..20:57
SpamapSzaneb: I think in the grander scheme of things.. this all goes back to taskflow...20:58
SpamapSzaneb: if we start thinking in terms of active workflows.. and not so much "stacks" ... we can reason about this easier.20:58
SpamapSbecause what I'm doing is facilitating an active workflow by saving states and then providing resume capability.. :-P20:59
SpamapSok.. back to the cone of silence20:59
*** zns has joined #heat20:59
*** cmyster has quit IRC21:02
*** Tross has joined #heat21:05
*** e0ne has quit IRC21:05
*** zns has quit IRC21:12
sdakeandersonvom re 72745 - if you don't pass any password to the nova client, does a root password get set?21:12
sdakeandersonvom what I was thinking was not actually passing the parameter would result in no password being set by nova21:13
sdakethis is how I would expect it would behave21:13
sdakenot create an empty password for root21:13
andersonvomsdake: apparently that can be setup differently, so it will depend on your specific setup21:15
andersonvomsdake: in my tests here, if you pass None or '' to nova (on create) it just generates a new password21:15
sdakewhat if you dont pass the argument at all?21:16
shardysdake: it's defaulted to None in the client21:16
shardyandersonvom: ugh :(21:16
sdakeso by adding this, we aren't actually changing anything21:16
sdakenova behaves as the user configures it21:16
*** gokrokve has joined #heat21:17
andersonvomsdake: precisely. just giving the user the opportunity to set their own passwords21:17
shardyandersonvom: seems like something you should be able to give the user a choice about to me :(21:17
sdakein the current master(without patch) nova would create a password21:17
sdakeIMNSHO nova is broken21:17
sdakeit needs to not do that if None is passed21:17
sdakeso if you file a bug and link it in the review, I'll +2 the patch21:17
andersonvomshardy: choice about what?21:18
sdake(and also make the patch pass None if none is actually specified)21:18
shardyandersonvom: users should have the choice to not set a password, and rely on a keypair instead21:18
sdakewith that course of action we can assume nova will either do one of two things - fix the bug - or not fix the bug - in either case Heat doesn't make the security situation worse21:18
*** erkules_ is now known as erkules21:19
arbylee1fyi, nova code that generates the pass https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L2601-L260221:19
arbylee1(i believe)21:19
*** gokrokve_ has quit IRC21:20
sdakeandersonvom as the code is, since None is not passed, if nova later fixed this problem (which I do indeed feel is a problem) we would end up loading empty passwords into the root user21:20
sdakemight put a link to the bug in the commit log as well ;-)21:21
andersonvomshardy: we're not really changing behavior here. If no password is specified, it will be as if the option didn't even exist, so the user does have a choice, right?21:21
andersonvomsdake: agreed. I can change it to pass None if it's am empty string21:21
shardyandersonvom: my problem was that we're pulling the random passwords back from nova, storing them, and exposing them as resource attributes21:21
shardybut I also believe this is a nova bug, None should not create a root password at all21:22
shardycloud-init disables root ssh logins, so that minimises the risk I guess, but it still seems wrong to me21:22
sdakeI hadn't thought about the resource attirbutes21:23
*** denis_makogon has quit IRC21:23
*** radez is now known as radez_g0n321:23
andersonvomshardy: worst case scenario, I guess we could store and display the password only once to the user. nova does give you the password back, it's only fair that the user have access to it at some point21:23
sdakethat creates an attack vector through heat21:23
*** dmakogon_ has joined #heat21:23
*** radez_g0n3 is now known as radez21:24
jasond``wouldn't they just say to disable the "enable_instance_password" option?21:24
*** Michalik has quit IRC21:24
shardyandersonvom: I think we should only allow users to specify a password explicitly, then there's no attribute required, because it's a property21:24
*** spzala has quit IRC21:24
sdakeshardy+121:24
shardyjasond``: Can you do that on a per-instance basis though?21:25
shardyfrom the review it sounds like a global nova thing, which seems like a bug21:25
jasond``shardy: yeah, it's global21:25
zanebshardy: +121:25
shardyif the server create took a create_random_rootpw arg, and the user had to choose it I'd have less of an issue with it21:25
*** sabeen has quit IRC21:26
sdakethe property approach exposes no security threats that wouldn't already be there with nova, so I could live with that21:28
sdakewe don't want to make openstack *less* secure by our actions21:28
andersonvomshardy: when you say property, do you mean parameter? i.e. there's no need for the user to retrieve the information from the attribute because they're the ones who set the pwd in the first place?21:28
sdakeandersonvom bingo21:29
sdakebut the property can be retrieved via get_prop21:29
shardyandersonvom: yes, I mean resource property, which defaults to None21:29
sdakethen if you really wanted to display it to the user, you could abuse the outputs section :)21:29
arbylee1sdake: I don't think that's true for admin_pass.  the value is only returned after create21:29
shardyso if the user sets  it explicitly (even if it's via the RandomString resource), they already know the value21:30
shardyarbylee1: that's what we're arguing is a nova bug21:30
shardypassing admin_pass=None should not set an admin password at all21:30
sdakeif nova wants to autocreate a password, that is fine, but that should not be the default just because of some global config option!21:30
sdakethought exercise:21:31
shardymaybe there should also be a random_admin_pass=True arg to nova, but doing that for admin_pass=None is just wrong IMO21:31
sdake10k vms deployed in a data center on nova21:31
sdakesomeone figures out magically how to predict the passwords created21:31
arbylee1shardy: even if you explicitly set a password, admin_pass doesn't come back on the server object afaik.  it's not just the autogenerated passwords21:31
sdakenow what?21:31
shardysdake: or we have a bad vulnerability in heat, then we really really don't want to be storing those 10k passwords21:32
shardy(or any credentials for anything IMO)21:32
sdakeshardy points out a second  thought experiment :)21:33
sdakenow if the user were to abuse get_prop in an output section to display the property that would be their prerogative - and they would have to deal with the consequences :)21:33
sdakesorry get_param21:34
*** shakayumi has quit IRC21:35
andersonvomok... so we can make it as a param and allow the user to set it. if they don't, just pass None to nova and file a bug with nova to fix that. would that work?21:36
sdakeI think you would want to add it as a property to the resource21:36
sdakeand use get_param in the hotness21:36
sdakebut ya, that wfm21:36
sdakeand put a link to the nova bug in the commit log pls :)21:37
*** Michalik has joined #heat21:37
sdakeshardy that sound good?21:37
*** shakayumi has joined #heat21:41
andersonvomsdake: so, the only thing missing here (https://review.openstack.org/#/c/72745/5/heat/engine/resources/server.py) is to remove it as an attribute, right? since it's already a property21:41
*** shakayumi has quit IRC21:41
sdakeI suspect default should be None rather then ''21:41
andersonvomtrue, that as well21:42
sdakebut I'm not sure if that would work - need to test it21:42
shardyandersonvom, sdake: +1 sounds good21:43
*** zns has joined #heat21:43
sdakeandersonvom and the removal of the db_api insertion of admin_pass as well21:43
jasond``sdake: i was looking at the python-novaclient source.  None or '' would work21:44
andersonvomsdake: also, it can't default to None, because it's a String property, so it won't pass the validation21:44
shardyandersonvom: You just don't specify a default21:44
andersonvomI knew something forced us to default it to empty string21:44
shardyand make the property optional21:45
andersonvomshardy: but that will make it required21:45
shardyit's not a mandatory property21:45
*** chandan_kumar has quit IRC21:45
shardywe pass None (which IIRC Properties gives us anyway) unless the user sets it to something21:45
sdake_jasond'' for some reason I have a preference for None21:45
*** kebray has quit IRC21:46
jasond``sdake_: yeah, None eliminates the question about whether or not Nova will create a server with an empty root password21:47
*** randallburt has joined #heat21:47
andersonvomshardy: humm... I'll take another look into it. I could swear trying to do without the default and required false but it didn't work. maybe I missed something21:47
*** randallburt has quit IRC21:47
sdake_jasond'' well it doesn't eliminate the question, but from a heat perspective it does since we don't pass any optoins for it :)21:47
*** randallburt has joined #heat21:47
jasond``well yeah :)21:47
sdake_andersonvom a comment around the resource property might be handy so people don't muck with it with a link to the review21:48
sdake_typically it wouldn't matter, but changing the code is dangerous and not obvious21:49
stevebakershardy: Here is a tempest change which switches to using standard credentials. Works locally for me with your patch series https://review.openstack.org/#/c/76981/21:50
*** wchrisj has joined #heat21:50
*** lnxnut has quit IRC21:51
shardystevebaker: awesome!  So what user will the tests get run as now?21:51
sdake_shardy are parameters stored in the db?21:52
stevebakerdevstack demo user, demo tenant21:52
*** pablosan is now known as ZZpablosan21:52
andersonvomsdake_: they are. in plain text, if I'm not mistaken21:53
shardystevebaker: Ok cool, I'm planning to submit a patch flipping deferred_auth_method to trusts, but we'll need another devstack patch adding heat_stack_owner role to the demo user first21:53
*** jprovazn has quit IRC21:53
*** lnxnut has joined #heat21:53
andersonvomsdake_: (even if the param is marked as hidden)21:54
*** lnxnut has quit IRC21:54
stevebakershardy: my only concern with these landing now is they might break the tripleo cloud several ways. You may need to work with lifeless and SpamapZ to make sure all the config changes are in place first21:54
*** lnxnut has joined #heat21:54
shardystevebaker: Yeah, I was planning to mark the patch WIP and get the gate working, then solicit feedback from folks as to whether we're happy to make the switch21:54
shardywould be good to kill stuff like the horizon password field21:55
*** kebray has joined #heat21:55
sdake_andersonvom that still exposes us to attacks on the heat db then, and I'm not quite sure what to do about it21:57
sdake_maybe shardy has some ideas21:57
andersonvomsdake_: this is already happening, since we pass all our passwords as params currently21:57
sdake_yup21:58
andersonvom=\21:58
*** lnxnut_ has joined #heat21:58
shardywell if you use ssh keys, you have no need to have any credentials stored in the heat db21:58
*** lnxnut has quit IRC21:58
randallburtsdake_: , andersonvom sorry to be late to the party, but what I'm hearing is that we have two issues: the general one about admin_pass which *is* being encrypted, and a separate issue altogether around the fact that we don't encrypt parameters marked as "hidden"21:59
*** rpothier has quit IRC21:59
randallburtshardy:  the *if* is the important part though.21:59
sdake_shardy vm information password is still passed even in our sample templates - eg things like the db password21:59
randallburtIMO, we should still handle the "if you don't" part21:59
shardyrandallburt: we could definitely look at encrypting more stuff in the DB22:00
*** jmckind has quit IRC22:00
randallburtshardy:  totally agreed. seems we have the info already to know "when" to do it, at least in the parameter case.22:00
shardyrandallburt: sure, it's just a question of making incremental steps in a more secure direction :)22:00
shardyrandallburt: letting nova create random passwords for every instance and storing them all seems like an incremental step in the wrong direction tho ;)22:01
sdake_randallburt I would put the credentials of the root account at higher priority then the credentials of the applications they export22:01
sdake_the second they being the vms sorry for bad grammar :)22:02
randallburtshardy:  I'd agree except there's no way to get them out of Nova and its swallowed and lost if you create your instances via Heat.22:02
shardyrandallburt: sure, if the user had to explictly select a random_admin_pass option I'd have not problem with it at all22:02
randallburtsdake_:  fair enough, but again, the patch around admin pass does encrypt the data22:02
sdake_the only reason they are not swallowed by nova is because it doesn't have to permently record them in its db22:03
shardyI just think the current nova functionality is broken22:03
*** Tross has quit IRC22:03
sdake_I suspect if nova had to record the passwords, it would swallow them :)22:03
randallburtshardy:  true, but that's not under our control, is it? or are you saying add that to the resource and *skip* Nova's functionality unless its selected?22:03
shardyrandallburt: I'm saying lets fix nova22:03
shardyor at least ask if they're willing to entertain the idea :)22:04
sdake_the fact that nova behaves in a broken way is orthogonal to encrypting and storing root credentials in the heat db22:04
sdake_nova doesn't store private credentials, only public credentials22:05
sdake_eg id_rsa.pub22:05
randallburtshardy:  not sure I agree Nova is "broken" in this respect, they have functionality that works in a certain way. As an orchestration service, I think it behoves us to support that functionality. That being said, this can be a stop-gap until Nova is fixed and we have Barbican to handle all the other sensitive data we're already keeping around in plain text.22:06
shardyrandallburt: having any interface do stuff when you pass a None argument is broken IMO22:06
sdake_shardy i agree22:07
*** jdob has quit IRC22:07
shardyespecially when the API defaults that argument to None which implies no side-effects22:07
shardy(the novaclient python API)22:07
randallburtshardy:  oh, I see. I thought you were referring to the password generation in general, not the conditions under which it does it.22:07
randallburtsorry, its what I get for missing the scollback ;)22:08
shardyrandallburt: It's the interface and the default-ness I'm arguing against22:08
shardyif nova grows a random_admin_pass boolean option that's totally cool with me :)22:08
randallburtshardy:  gotcha. So my question then is accept this now with some changes that ignore None and don't pass that arg on to Nova at all, and then work on geting Nova to do something "better"?22:09
sdake_i think storing the root password in the db, even ecrypted is suboptimal as well22:10
shardyrandallburt: lets just add the property, optional, which will default to None anyway, but not the attribute and db resource_data stuff22:10
randallburtsdake_:  I don't disagree, but what's the option (that allows password retrieval if needed) other than to wait for barbican.22:10
shardythen we can see if nova will fix (or accept a patch) for their stuff22:10
randallburtshardy:  I still lose the password if its generated which kinda blows away half the use case for the feature.22:11
shardyand argue about the attributes stuff later :)22:11
randallburtshardy:  ok, so we add the attribute in server.py and then we can add the retrieval stuff in cloud_server.py since its our users that have asked for and need this functionality.22:12
shardyrandallburt: well if the nova interface is likely to change, we don't want to merge something which will break folks later?22:12
shardys/likely/I'd like to see it/ ;)22:12
randallburtshardy:  that's something we face every day. we'd never change any resource ever.22:12
randallburtshardy:  oh, gotcha.22:12
sdake_randallburt I get that our views on the security implications don't fit the use case, but I'd need someone more versed in security to convince me22:12
sdake_like folks from the openstack SRT for example22:13
shardyrandallburt: well lets have a discussion with the nova folks, and revisit this heat discussion in a few days time22:13
randallburtshardy:  I still disagree though. The interfaces could change and expand every six months. We should still support the features that will be available in Icehouse, even if we know it will change in Juno (for example)22:13
sdake_shardy is there a global openstack srt list this could be taken to?22:14
randallburtshardy:  even if we move the password store/retrieve to contrib?22:14
shardyrandallburt: there are aspects of the neutron API we don't support because it's wrong too22:14
shardyrandallburt: if you submit a separate patch for the contrib resource I imagine it will be looked at with considerably less scrutiny ;)22:15
randallburtshardy:  I understand. Not our call, IMO, but those reasons are quite different in that this *is* a user-facing functionality, but I won't debate that to death :)22:15
randallburtshardy:  I thought so. ;)22:16
randallburtandersonvom:  sound good to you?22:16
*** e0ne has joined #heat22:16
sdake_randallburt I'm open to having real security experts analyze the problem - I'm just not sure who in openstack to contact for that22:16
shardyrandallburt: my concern, particularly for the core resources, is stability, so it makes no sense to merge a new iterface if it's behavior may change in six weeks time22:16
andersonvomrandallburt: I'm on it!22:17
shardysdake_: I think a post to openstack-dev may suffice, if this functionality is documented?22:17
sdake_shardy I mean storing the root password in the heat db22:17
sdake_not the fact that nova is broke n:)22:18
randallburtshardy:  no worries, then. andersonvom can move the patch from server to contrib for now and we can start talking to Nova and whoever we figure out has some more expertise in these things. thanks andersonvom!22:18
*** alexheneveld has joined #heat22:18
randallburtwhoo. just in time. I got to run the youngest to fencing. bbiab22:18
shardysdake_: I would much prefer we did not do that, so yeah if that happens in the contrib patch maybe we should ask for some wider review22:19
sdake_shardy I'm not on the srt for heat - maybe stevebaker knows who leads up that effort and can point some other reviewers on the submission22:19
shardysdake_: with the move to trusts, we'll finally remove the need to store any credentials inside the heat DB, I'm really opposed to starting to do it again unless it's really unavoidable22:20
stevebakerwhat change are we talking here?22:20
*** e0ne has quit IRC22:21
shardystevebaker: https://review.openstack.org/#/c/72745/522:21
sdake_shardy we do store the parameters (which often contain credentials) in the db22:22
sdake_not as an argument for why we should do so22:22
sdake_but as pointing it out as another place that needs addressing22:22
shardysdake_: sure, we can look at that as a seperate issue22:23
sdake_agree22:23
* shardy needs some sleep22:23
shardynight all22:23
sdake_later shardz22:23
*** shardy is now known as shardy_afk22:23
*** lnxnut_ has quit IRC22:24
*** lnxnut has joined #heat22:29
*** randallburt has quit IRC22:29
openstackgerritJason Dunsmore proposed a change to openstack/heat: Don't install cloud-init on Rackspace images  https://review.openstack.org/7699322:33
openstackgerritJason Dunsmore proposed a change to openstack/heat: Ensure that the NoCloud data source is loaded  https://review.openstack.org/7699422:33
*** lnxnut has quit IRC22:33
*** spzala has joined #heat22:35
*** achampion has quit IRC22:37
*** radez is now known as radez_g0n322:40
*** vijendar has quit IRC22:41
*** lnxnut has joined #heat22:44
*** Tross has joined #heat22:44
*** kebray has quit IRC22:45
*** aweiteka has quit IRC22:45
*** jcru has joined #heat22:48
*** jcru has quit IRC22:48
*** lnxnut has quit IRC22:48
*** jcru has joined #heat22:48
*** zns has quit IRC22:49
*** ZZpablosan is now known as pablosan22:50
*** rpothier has joined #heat22:51
*** alexheneveld has quit IRC22:55
*** wchrisj has quit IRC22:57
*** kebray has joined #heat22:59
*** pvaneckw has joined #heat23:01
*** pvaneck has quit IRC23:01
*** blomquisg has quit IRC23:04
*** kebray has quit IRC23:06
*** pvaneck has joined #heat23:07
SpamapSWhat if we just never failed stack-update ?23:07
SpamapSretry _forever_23:07
SpamapSlifeless: ^^ isn't that basically what you were thinking?23:08
*** pvaneckw has quit IRC23:08
SpamapSzaneb: ^^ Thoughts on this crazy idea?23:08
zanebwhat could possibly go wrong23:10
lifelessSpamapS: that would be ok, but the 500 thing is different iMNSHO23:10
lifelessSpamapS: we're not timing out today, we're stopping early because heat has unreasonable expectations about 5xx HTTP status codes23:10
lifelessSpamapS: simple rate limits like most public clouds have will emit 500s on over-rate users.23:11
lifelessSpamapS: iz bug :)23:11
SpamapSlifeless: so, backing up a bit23:11
*** rcleere has quit IRC23:11
SpamapSlifeless: even if we chase down every call to every external library and make them retry indefinitely...23:11
zaneblifeless: that implies that we should retry in e.g. novaclient, no?23:11
SpamapSlifeless: we may have something happen that causes heat to fail and thus leaves a heat stack in a FAILED state..23:12
SpamapSlifeless: which currently has only one remedy.. delete the entire stack.23:12
*** WinnieTsang has quit IRC23:12
SpamapSlifeless: I am searching for an answer to that problem, not the 500 problem specifically.. which is only one of many problems we are likely to encounter because of the FAILED state.23:13
*** pvaneck has quit IRC23:14
*** pvaneck has joined #heat23:14
*** WinnieTsang has joined #heat23:16
lifelessSpamapS: so I don't think we're connecting23:18
lifelessSpamapS: I mean - I agree that a failed update has to be recoverable23:19
lifelessSpamapS: but, *normal* situations should never lead to manual intervention.23:19
SpamapSYeah, two separate things23:19
SpamapSI want to give us the actual ability _to manually intervene_23:19
SpamapSsince without that, we're talking about removing the entire datacenter of machines if we missed even one hiccup23:20
lifelessright23:20
lifelessso retry forever as a strategy23:20
lifelesswhere is the manual intervention there?23:20
SpamapSlifeless: the implementation is challenging23:21
SpamapShttps://etherpad.openstack.org/p/update-failure-recovery23:21
SpamapSlifeless: retry forever is basically an alternative to this big state-resolving monster that you see there.23:22
lifelesslooking23:23
lifelessmaybe its time to move all this stuff into concoord distributed state machines23:23
*** topol has quit IRC23:23
*** zns has joined #heat23:24
SpamapSlifeless: yes. feature freeze is Monday.23:24
lifelessah yes23:25
SpamapSlifeless: so I'm looking for a short term escape hatch so we can handle problems like this in the interim.23:25
SpamapSlifeless: one thought I had was to just allow administrators to forcibly short-circuit the update process.23:25
lifelessassert updated?23:30
*** maxskew_ has joined #heat23:33
*** maxskew has quit IRC23:37
SpamapSlifeless: the problem is that we do updates in parallel. So while we know what failed, we don't actually know what succeeded.23:37
*** spzala_ has joined #heat23:37
*** sdake__ has joined #heat23:40
*** sgran_ has joined #heat23:40
*** pablosan_ has joined #heat23:43
*** SpamapS_ has joined #heat23:43
*** spzala has quit IRC23:45
*** sdake_ has quit IRC23:45
*** _jmp_ has quit IRC23:45
*** sgran has quit IRC23:45
*** pablosan has quit IRC23:45
*** SpamapS has quit IRC23:45
*** openstackgerrit has quit IRC23:45
*** yogesh has quit IRC23:45
*** arbylee has joined #heat23:45
*** mkollaro has quit IRC23:46
*** Linz has quit IRC23:46
*** pablosan_ has quit IRC23:46
*** pablosan_ has joined #heat23:46
*** _jmp_ has joined #heat23:47
*** _jmp_ is now known as Guest7680623:47
*** arbylee1 has quit IRC23:48
*** arbylee has quit IRC23:49
*** SpamapS_ is now known as SpamapS23:53
*** SpamapS has quit IRC23:53
*** SpamapS has joined #heat23:53
lifelessSpamapS: so what do you mean precisely by short-circuit the update process?23:54
zaneblifeless: he means jump ahead past all the stuff that already succeeded, and carry on where we left off23:58
SpamapSin pondering this a bit more..23:59
SpamapSI wonder if we can keep a journal23:59
SpamapSwhich would give us the exact ones to ignore23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!