Friday, 2014-05-23

*** rsmitty has quit IRC00:05
*** mwheckmann has quit IRC00:08
*** TravT has quit IRC00:10
*** nati_ueno has joined #heat00:12
*** gokrokve has joined #heat00:12
*** IlyaE has joined #heat00:19
*** matsuhashi has joined #heat00:23
*** killer_prince has quit IRC00:25
*** nati_ueno has quit IRC00:29
*** nati_ueno has joined #heat00:37
*** saurabhs has quit IRC00:40
*** gokrokve has quit IRC00:41
*** gokrokve has joined #heat00:44
*** gokrokve_ has joined #heat00:45
*** gokrokve has quit IRC00:49
*** m_22 has quit IRC00:54
*** m_22 has joined #heat01:01
*** nati_ueno has quit IRC01:02
*** pablosan is now known as zz_pablosan01:02
*** nati_ueno has joined #heat01:03
*** IlyaE has quit IRC01:10
*** m_22 has quit IRC01:12
*** nati_ueno has quit IRC01:14
*** tiantian has joined #heat01:17
tiantianHi all, would you please to review my patch:https://review.openstack.org/#/c/94021/ . migrate to glanceclient to get image id. Thanks.01:26
*** blamar_ has joined #heat01:30
*** blamar has quit IRC01:31
*** blamar_ is now known as blamar01:31
*** lazy_prince has joined #heat01:32
*** lazy_prince is now known as killer_prince01:32
*** nosnos has joined #heat01:35
*** nati_ueno has joined #heat01:45
*** rpothier has joined #heat01:47
*** nati_ueno has quit IRC01:49
*** pack3t has joined #heat01:51
*** nati_ueno has joined #heat01:51
*** nati_ueno has quit IRC01:54
*** lazy_prince has joined #heat01:58
*** killer_prince has quit IRC01:58
*** sgordon has quit IRC01:58
*** jpeeler has quit IRC01:58
*** packet has quit IRC01:58
*** sergmelikyan has quit IRC01:58
*** pas-ha has quit IRC01:58
*** lazy_prince is now known as killer_prince01:58
*** pack3t is now known as packet01:58
*** jpeeler has joined #heat02:00
*** jpeeler has quit IRC02:00
*** jpeeler has joined #heat02:00
*** pas-ha has joined #heat02:00
*** m_22 has joined #heat02:01
*** sgordon has joined #heat02:04
*** andersonvom has joined #heat02:06
*** IlyaE has joined #heat02:06
*** packet has quit IRC02:12
*** nati_ueno has joined #heat02:14
*** achampion has joined #heat02:18
*** zhiyan_ is now known as zhiyan02:18
*** IlyaE has quit IRC02:18
openstackgerritWilliam C. Arnold proposed a change to openstack/heat: Implements stack lifecycle plugpoints  https://review.openstack.org/8936302:28
*** alexpilotti has quit IRC02:29
*** nati_ueno has quit IRC02:32
*** killer_prince has quit IRC02:44
*** killer_prince has joined #heat02:48
*** tiantian has quit IRC02:54
*** harlowja_ is now known as harlowja_away02:55
*** lazy_prince has joined #heat03:02
*** killer_prince has quit IRC03:02
*** lazy_prince is now known as killer_prince03:02
*** nosnos has quit IRC03:03
*** tiantian has joined #heat03:07
tiantianzhiyan: in?03:13
*** radez is now known as radez_g0n303:14
*** matsuhashi has quit IRC03:29
*** matsuhashi has joined #heat03:36
*** m_22 has joined #heat03:38
*** piyush has joined #heat03:38
*** matsuhashi has quit IRC03:43
*** matsuhashi has joined #heat03:43
*** matsuhas_ has joined #heat03:45
*** matsuhashi has quit IRC03:45
*** nosnos has joined #heat03:45
*** andersonvom has quit IRC03:47
*** swygue_ has joined #heat03:58
*** akuznetsov has joined #heat03:59
*** IlyaE has joined #heat03:59
*** killer_prince has quit IRC04:04
*** lazy_prince has joined #heat04:05
*** lazy_prince is now known as killer_prince04:05
*** swygue_ has quit IRC04:08
*** killer_prince has quit IRC04:14
*** sgordon has quit IRC04:19
*** sab has joined #heat04:19
*** nati_ueno has joined #heat04:28
*** edmund has quit IRC04:28
*** edmund has joined #heat04:28
*** piyush has joined #heat04:31
*** matsuhas_ has quit IRC04:34
*** piyush has quit IRC04:38
*** killer_prince has joined #heat04:41
openstackgerritWilliam C. Arnold proposed a change to openstack/heat: Implements stack lifecycle plugpoints  https://review.openstack.org/8936304:42
*** matsuhashi has joined #heat04:43
*** nkhare has joined #heat04:56
*** akuznetsov has quit IRC05:01
*** lazy_prince3 has joined #heat05:08
*** edmund has quit IRC05:11
*** edmund has joined #heat05:11
*** sab has quit IRC05:12
*** tiantian has quit IRC05:19
*** edmund1 has joined #heat05:21
*** edmund has quit IRC05:23
*** shakayumi has joined #heat05:30
*** ramishra has joined #heat05:31
skraynevmorning all ;)05:32
*** jprovazn has joined #heat05:34
*** tspatzier has joined #heat05:37
*** keyvan is now known as zz_keyvan05:38
*** IlyaE has quit IRC05:41
*** lazy_prince3 has quit IRC05:51
*** edmund has joined #heat05:54
*** edmund1 has quit IRC05:54
*** akuznetsov has joined #heat05:56
*** sab has joined #heat05:57
*** ramishra has quit IRC05:57
*** alexpilotti has joined #heat06:01
*** saju_m has joined #heat06:03
openstackgerritOpenStack Proposal Bot proposed a change to openstack/heat: Imported Translations from Transifex  https://review.openstack.org/8975006:03
*** alexpilotti has quit IRC06:06
*** tspatzier__ has joined #heat06:06
*** akuznetsov has quit IRC06:07
*** nati_ueno has quit IRC06:08
*** tiantian has joined #heat06:10
*** tspatzier has quit IRC06:10
*** akuznetsov has joined #heat06:13
*** IlyaE has joined #heat06:14
*** jstrachan has joined #heat06:14
*** sergmelikyan has joined #heat06:17
*** gokrokve_ has quit IRC06:21
*** alexheneveld has joined #heat06:23
*** che-arne has quit IRC06:24
mattoliverauMorning skraynev, sorry for the late greeting, am having a busy Friday arvo :)06:26
*** edmund has quit IRC06:29
*** mikir has joined #heat06:30
skraynevmattoliverau: hey, np ;) I understand.06:36
*** aniket has joined #heat06:40
anikethello all :)06:40
*** nati_ueno has joined #heat06:41
*** alexpilotti has joined #heat06:42
*** yogeshmehra has joined #heat06:43
*** yogeshmehra has quit IRC06:46
mattoliverauHi aniket06:46
*** yogeshmehra has joined #heat06:46
*** lazy_prince3 has joined #heat06:48
*** alexheneveld has quit IRC06:49
aniket<mattoliverau>while launching a stack i am getting Error: Resource CREATE failed: Error: Creation of server St-LoadBalancer-uo6ahjfiawep-LB_instance-on55iwga2kfa failed: No valid host was found. (500)06:49
aniketdo you have any idea,how to resolve this?06:50
*** yogeshmehra has quit IRC06:50
*** zhiyan is now known as zhiyan_06:52
*** gokrokve has joined #heat06:52
*** gokrokve_ has joined #heat06:54
*** gokrokve has quit IRC06:56
*** e0ne has joined #heat06:57
*** gokrokve_ has quit IRC06:59
*** yogeshmehra has joined #heat07:00
*** lazy_prince3 is now known as lazy_prince07:02
*** jamie_h has joined #heat07:05
*** e0ne has quit IRC07:09
*** zhiyan_ is now known as zhiyan07:10
therveGood morning!07:12
*** killer_prince has quit IRC07:12
*** e0ne has joined #heat07:12
therveaniket, It means nova was unable to schedule the creation of your server07:14
therveGenerally because of not enough memory/cpu/disk07:14
therveTry to look at nova logs07:14
openstackgerritA change was merged to openstack/heat: Order imports in alphabetical order (9/9)  https://review.openstack.org/8536107:17
*** alexpilotti has quit IRC07:17
*** mikir has quit IRC07:20
*** killer_prince has joined #heat07:21
*** e0ne_ has joined #heat07:22
aniket<therve>thanks :)07:24
*** e0ne has quit IRC07:25
*** alexpilotti has joined #heat07:25
*** tspatzier__ is now known as tspatzier07:25
tspatzierGood morning therve! I reworked the allowed_values for numbers patch a bit. I'm not completely happy with it but it's the best I can think of to be backwards compatible. And at least I added a lot of tests. Will post in a moment.07:28
*** alexpilotti has quit IRC07:28
*** ifarkas has joined #heat07:29
pas-hamorning07:30
openstackgerritHaiyang DING proposed a change to openstack/heat: Change Volume metadata attribute from json to raw dict format.  https://review.openstack.org/9485807:30
skraynevpas-ha: hi man :)07:30
skraynevpas-ha: my congratulation with merging last patch with imports :007:31
pas-hayay!07:31
skraynevpas-ha: :)07:31
*** IlyaE has quit IRC07:32
*** nati_ueno has quit IRC07:39
*** SushillKM has joined #heat07:40
*** SushillKM has left #heat07:41
*** jistr has joined #heat07:43
*** IlyaE has joined #heat07:49
*** gokrokve has joined #heat07:55
*** yogeshmehra has quit IRC07:57
*** rwsu has quit IRC07:58
*** yogeshmehra has joined #heat07:58
*** IlyaE has quit IRC07:59
*** gokrokve has quit IRC08:00
*** zns has joined #heat08:00
*** derekh_ has joined #heat08:00
*** zns has quit IRC08:01
*** tomek_adamczewsk has joined #heat08:01
*** zns has joined #heat08:01
*** yogeshmehra has quit IRC08:02
*** tiantian has quit IRC08:03
*** nkhare has quit IRC08:04
*** zz_keyvan is now known as keyvan08:05
*** zns_ has joined #heat08:07
*** zns has quit IRC08:08
*** akuznetsov has quit IRC08:09
*** zns_ has quit IRC08:15
*** akuznetsov has joined #heat08:27
*** ifarkas has quit IRC08:27
*** tiantian has joined #heat08:30
andrearosa ls08:30
*** gokrokve has joined #heat08:32
*** gokrokve has quit IRC08:38
*** sab has quit IRC08:46
*** saju_m has quit IRC08:46
*** matsuhashi has quit IRC08:48
andrearosawrong window :(08:53
openstackgerritThomas Spatzier proposed a change to openstack/heat: Respect schema data type in constraint validation  https://review.openstack.org/9464108:55
*** nkhare has joined #heat08:56
tspatziertherve: ^^08:57
thervetspatzier, Will look later08:58
tspatziertherve: thanks, and no hurry.08:58
*** shakayumi has quit IRC09:01
*** e0ne_ has quit IRC09:08
*** e0ne has joined #heat09:09
*** e0ne_ has joined #heat09:11
*** e0ne has quit IRC09:15
*** shardy_afk is now known as shardy09:15
*** matsuhashi has joined #heat09:16
*** julienvey has quit IRC09:17
*** dmakogon_ is now known as denis_makogon09:23
*** julienvey has joined #heat09:27
*** gokrokve has joined #heat09:33
*** gokrokve has quit IRC09:38
*** keyvan is now known as zz_keyvan09:45
*** nosnos has quit IRC09:45
elynnHi all, could you help to review this patch? https://review.openstack.org/#/c/75800/10:01
*** ifarkas has joined #heat10:01
*** mkollaro has joined #heat10:02
shardyelynn: I looked at it yesterday, but we need https://review.openstack.org/#/c/75798/12 in first10:03
shardyyou can help by looking at the logs and figuring out what bug we need to recheck against, so we can get rid of the jenkins -110:04
*** akuznetsov has quit IRC10:05
*** matsuhashi has quit IRC10:12
*** matsuhas_ has joined #heat10:15
*** openstackstatus has quit IRC10:20
*** openstack has joined #heat10:21
*** sorantis has joined #heat10:22
*** andreaf has joined #heat10:32
*** gokrokve has joined #heat10:34
*** akuznetsov has joined #heat10:36
*** gokrokve has quit IRC10:39
*** vpanizo has quit IRC10:46
*** sgordon has joined #heat10:52
*** matsuhas_ has quit IRC10:55
*** matsuhashi has joined #heat10:57
*** sorantis has quit IRC10:58
*** matsuhashi has joined #heat10:58
*** matsuhashi has quit IRC10:58
*** zhiyan is now known as zhiyan_10:58
*** matsuhashi has joined #heat10:59
*** asalkeld has quit IRC11:01
*** tiantian has quit IRC11:07
*** rpothier has quit IRC11:14
openstackgerritSergey Kraynev proposed a change to openstack/heat: Adding own child_template for AutoScalingGroup  https://review.openstack.org/9514211:20
*** nkhare has quit IRC11:29
*** sgordon has quit IRC11:29
*** saju_m has joined #heat11:33
*** gokrokve has joined #heat11:34
*** gokrokve has quit IRC11:39
*** andreaf_ has joined #heat11:39
*** gpocentek has quit IRC11:52
*** andreaf_ has quit IRC11:55
openstackgerritSergey Kraynev proposed a change to openstack/heat: Adding own child_template for AutoScalingGroup  https://review.openstack.org/9514212:00
*** plomakin has quit IRC12:04
*** e0ne_ has quit IRC12:10
*** tspatzier has quit IRC12:10
*** plomakin has joined #heat12:15
*** matsuhashi has quit IRC12:20
*** matsuhashi has joined #heat12:21
*** andersonvom has joined #heat12:23
*** shakayumi has joined #heat12:23
*** andreaf has quit IRC12:25
*** matsuhashi has quit IRC12:26
*** blomquisg has joined #heat12:27
*** dims has joined #heat12:28
*** dims has quit IRC12:28
*** piyush has joined #heat12:29
*** lazy_prince has quit IRC12:32
*** dims has joined #heat12:33
*** gokrokve has joined #heat12:35
*** sgordon has joined #heat12:35
openstackgerritPavlo Shchelokovskyy proposed a change to openstack/heat: Support mutually exclusive properties in schema  https://review.openstack.org/9462212:36
*** jdob has joined #heat12:37
*** matsuhashi has joined #heat12:37
*** andreaf_ has joined #heat12:38
*** matsuhashi has quit IRC12:38
*** matsuhashi has joined #heat12:38
*** matsuhashi has quit IRC12:38
*** matsuhashi has joined #heat12:39
*** gokrokve has quit IRC12:40
*** saju_m has quit IRC12:41
*** e0ne has joined #heat12:41
*** matsuhashi has quit IRC12:43
andreaf_stevebaker: ping12:44
andreaf_shardy: ping12:45
shardyandreaf_: Hi!12:50
shardyandreaf_: stevebaker is probably asleep atm12:50
andreaf_oh, ok is he on the West Coast?12:51
shardyandreaf_: He's in new zealand12:51
andreaf_ok most likely alseep :D12:51
shardyandreaf_: If needed I can rebase his test patch, I don't think he'll mind :)12:51
*** radez_g0n3 is now known as radez12:51
*** erecio has joined #heat12:51
andreaf_shardy: ok12:52
shardyandreaf_: what is your preferred solution to get the various manager.py things in?12:52
shardywe seems to have a bit of a dependency loop going on with the various patches12:52
*** radez is now known as radez_g0n312:52
*** radez_g0n3 is now known as radez12:52
shardyand it would be good to get the base-class stuff in soon so we can focus on getting new testcases done12:52
andreaf_shardy: right so the policy we have is that you should not merge code which is not used12:52
shardyOk, so even reworking the autoscaling test is no good then, as it's currently skipped?12:53
shardyI commented the skip and tested it locally, fwiw12:53
shardyIf you prefer I can squash everything into stevebaker's patch, and we can focus on getting that in12:53
andreaf_shardy: yes that was my idea12:56
shardyandreaf_: Ok, I'll do that now then12:57
andreaf_shardy: great thanks12:58
andreaf_shardy: it would be great to get those patches through :)12:58
shardyandreaf_: +1 :)12:58
andreaf_sdague: are you around?12:59
*** piyush has left #heat12:59
sdagueandreaf_: a little12:59
andreaf_:D12:59
*** alexheneveld has joined #heat12:59
andreaf_ok I just wanted to ask you if you agree on ^^^13:00
andreaf_sdague:  you have seen the heat patches before right?13:00
*** mwheckmann has joined #heat13:01
andreaf_sdague: so the idea to squash the _wait_on_stack_status function into https://review.openstack.org/#/c/88137/ and get it merged so the other heat patches can progress from there13:01
sdagueyeh, but I've lost context on them13:01
*** arbylee has joined #heat13:02
andreaf_sdague: I -1 https://review.openstack.org/#/c/92565/5 because it introduces a function which is not used - it is in the latest patchset but the test is skipped13:03
andreaf_sdague: so I proposed to move that function to another patch (https://review.openstack.org/#/c/88137/)  where the same function is used so we can see that in action.13:04
sdagueandreaf_: yeh, I think that's fair13:04
andreaf_sdague: ok thanks13:04
sdagueI think that's one of the culture differences here. I know the heat team tends to do deeper patch stacks with non exercised bits in the bottom, and we tend to want to see code executed before landing it13:04
shardyFWIW stevebaker was initially -1 on _wait_on_stack_status in 92565, but I believe I've now justified the response to his "Does this provide anything which status_timeout doesn't" remark13:05
* shardy answers "yes" :D13:05
shardysdague: that's definitely the case, yes13:06
sdaguehonestly though, my primary concern is that the heat-slow job remains non-voting. I'm not sure who is working on the stability issues there. I'd really like to see some sketch (even if just an etherpad) of how that gets solved.13:06
therveHow often is it failing nowadays?13:06
shardysdague: I've been looking at the autoscaling job - I think we can simplify that test a lot and get it working with cirros images13:07
*** andersonvom has quit IRC13:07
sdaguetherve: I don't know, once it passed the 50% failure threshold we turned it off. I've now been ignoring it.13:07
therveIt actually caught a nova problem recently13:08
therveBut yeah the waitcondition issue should be improved by shardy's work13:08
sdagueanyway, I'm mostly not working today, so I'm just going to largely go back to "not working" :)13:09
andreaf_sdague: have a nice day!13:12
EmilienMshardy: hey, is someone already working on grenade/heat?13:13
shardyEmilienM: I started a patch, but haven't made much progress because I couldn't get a local test environment working13:14
shardyhttps://review.openstack.org/#/c/86978/13:14
EmilienMshardy: ah ok.13:15
EmilienMshardy: need help?13:15
shardyEmilienM: If you have time to help then yes, definitely :)13:15
*** achampion has quit IRC13:17
*** sjmc7 has joined #heat13:27
*** jrist has quit IRC13:28
*** aweiteka has joined #heat13:30
*** SushilKM has joined #heat13:30
*** vijendar has joined #heat13:31
*** snk has quit IRC13:34
*** gokrokve has joined #heat13:36
*** jrist has joined #heat13:40
*** gokrokve has quit IRC13:41
*** rpothier has joined #heat13:45
*** gokrokve has joined #heat13:49
*** SushilKM has quit IRC13:49
*** arbylee has quit IRC13:49
*** gokrokve has quit IRC13:52
*** zns has joined #heat13:59
*** pafuent has joined #heat14:01
*** cdent has joined #heat14:04
*** denis_makogon has quit IRC14:06
*** shakayumi has quit IRC14:07
*** rwsu has joined #heat14:08
*** pafuent has left #heat14:12
*** achampion has joined #heat14:13
*** zns has quit IRC14:13
*** zz_gondoi is now known as gondoi14:15
*** jstrachan has quit IRC14:16
*** piyush has joined #heat14:16
therveshardy, Can you look at https://review.openstack.org/#/c/94702/ ?14:21
therveI should be able to fix the config generation with that14:21
shardytherve: sure14:22
*** edmund has joined #heat14:23
*** arbylee has joined #heat14:25
*** pafuent has joined #heat14:26
*** arbylee1 has joined #heat14:26
*** arbylee1 has quit IRC14:26
*** arbylee1 has joined #heat14:26
*** tims has joined #heat14:28
*** arbylee has quit IRC14:29
*** arbylee1 has quit IRC14:30
*** arbylee has joined #heat14:31
*** sabeen has joined #heat14:34
*** david-lyle has joined #heat14:34
*** tomek_adamczewsk has quit IRC14:36
*** andersonvom has joined #heat14:39
*** IlyaE has joined #heat14:40
*** pasquier-s has quit IRC14:43
*** jprovazn is now known as jprovazn_afk14:46
openstackgerritThomas Herve proposed a change to openstack/heat: Use entry points for config generation  https://review.openstack.org/9517814:47
*** gokrokve has joined #heat14:49
*** gokrokve has quit IRC14:54
*** tims1 has joined #heat14:54
*** tims has quit IRC14:55
*** piyush has quit IRC14:58
*** blomquisg has quit IRC15:00
*** packet has joined #heat15:03
*** tomek_adamczewsk has joined #heat15:06
*** andrew_plunk has joined #heat15:07
*** pas-ha has quit IRC15:08
*** tims has joined #heat15:09
*** zz_pablosan is now known as pablosan15:10
*** pablosan has quit IRC15:10
*** tims1 has quit IRC15:11
*** tomek_adamczewsk has quit IRC15:12
*** tomek_adamczewsk has joined #heat15:13
*** rsmitty has joined #heat15:15
*** piyush has joined #heat15:16
*** ifarkas has quit IRC15:17
*** tomek_adamczewsk has quit IRC15:18
*** randallburt has joined #heat15:18
*** gokrokve has joined #heat15:18
jasondshardy: are you there?15:19
shardyjasond: hey, I am15:19
jasondhey, i was looking at your response in https://review.openstack.org/#/c/94736/215:20
jasondhow can you share threads between forked processes?15:20
jasondi mean, i didn't think that was possible15:20
shardyjasond: well they all have a reference to the same ThreadGroupManager created before the fork15:21
*** arbylee has quit IRC15:21
*** saurabhs has joined #heat15:21
*** arbylee has joined #heat15:21
shardyIt is all really confusing and non-obvious, and I'm still not 100% clear on how it's supposed to work15:22
*** zns has joined #heat15:22
jasondhm, everything i've read says forking python processes with threads is problematic15:22
shardyI would far prefer that services only supported one worker and you just had to run multiple processes instead15:22
jasondshardy: +115:22
zanebshardy: what is the difference between a worker and an engine here?15:22
jasondthere should just be a service that spawns multiple engines15:23
* zaneb is confused by the distinction15:23
shardyzaneb: The engine forks multiple workers, but you can also run multiple engine (parent) processes15:23
shardybasically it's internal fork vs explicitly starting multiple processes15:23
shardyI'd far prefer we only did the latter, but atm we don't15:23
zanebsooo... workers share a single input queue?15:23
jasondzaneb: https://github.com/openstack/heat/blob/master/heat/openstack/common/service.py#L31715:23
shardyzaneb: I believe so, but the oslo abstractions make the rpc side of it really hard to follow15:24
zanebit seems like we should find that out15:25
shardyzaneb: we were getting hung up w/qpid when multiple workers were trying to share a common EngineListener object, so I just moved that to after the fork15:25
zanebshardy: so basically they're independent engines but started with fork()15:26
shardybasically yes15:26
shardybut because all the requests are serialized per-stack by the global lock, I added the ThreadGroupManager before the fork, because it manages per-stack ThreadGroups15:27
zanebI'm not against that, but to the extent that multi-engine stuff is involved, we have to treat the workers as separate engines15:27
zanebshardy: but there's no shared memory, so it shouldn't matter whether you do that part before or after the fork, surely?15:28
shardyzaneb: well the issue is, you don't want a periodic task running for every sub-process, or they'll all race15:29
jasondbbiab, meeting15:30
shardyMaybe we should just disable periodic watchers for all situations where more than one worker is specified15:30
shardybut then SpamapS's patch will break Alarms for every existing user, because it starts a worker per cpu15:30
shardyand the ceilometer alarm provider resource doesn't currently work15:31
shardyzaneb: the before/after fork thing is just whether something happens once, or $num_workers times15:31
*** randallburt has quit IRC15:32
zanebok, but what you seem to be saying is that because something is created before the fork, it doesn't exist in the child processes15:32
zanebwhich is not how fork() works ;)15:33
therveFor the dangerous version of "just"15:33
*** blomquisg has joined #heat15:33
*** TravT has joined #heat15:35
shardyzaneb: Not really, I'm saying that the observed behavior with eventlet based timers started via ThreadGroup is one timer run per engine parent process15:44
shardywhereas if you actually run heat-engine multiple times you get alarms evaluated multiple times in different engines15:45
zanebtranslation: "something fancy in Oslo makes sure only the parent process runs timers, but we don't know what"?15:46
shardysomething weird in eventlet makes unfathomable things happen, situation normal15:47
shardyeventlet is the bit I'm unsure of, not the fork itself15:47
shardyoslo doesn't seem to do anything fancy afaict15:48
*** sjmc7_ has joined #heat15:48
shardyI'll just put everything back into the start()15:49
zanebrelying on eventlet behaviour we have no explanation for makes me nervous :/15:49
*** sjmc7 has quit IRC15:51
shardyI'd love to just rip all the periodic task stuff out completely, but I guess we have to deprecate CWLiteAlarm first15:51
SpamapSshardy: so.. another option..15:59
SpamapSshardy: is to fork/re-exec everything with a --no-timers for each worker15:59
shardySpamapS: possibly, although that's not actually needed atm16:04
shardyas zaneb has highlighted, I probably need to write off a day and go down the rabbit hole to figure out why forking breaks EngineListener but doesn't fire duplicate timer tasks16:04
*** jistr has quit IRC16:05
shardyprobably, the answer in both cases is monkey_patch16:06
*** blomquisg has quit IRC16:07
*** blomquisg has joined #heat16:07
zanebshardy: I assume that forking after creating EngineListener breaks because it's not valid to have the file descriptor that is listening to qpid open twice16:07
*** aweiteka has quit IRC16:08
shardyyup, but for some reason it works OK for rabbit16:08
jasondshardy: what would be the disadvantage to writing a service that spawns/keeps track of multiple engines?  seems a lot simpler16:08
shardyjasond: depends on your definition of simpler16:08
jasondshardy: easier to understand & therefore less buggy.  not lines of code16:09
zanebI believe that's called systemd :D16:09
shardyjasond: that is exactly what we have now, and as this discussion proves, it's not simple at all16:09
shardyzaneb: +116:09
shardyI'd far rather we had a single heat-engine process which $operating_system_scripts could spawn multiple times16:09
zanebme too16:10
shardyThis oslo/eventlet forking makes my head hurt16:10
jasond+116:10
zanebSpamapS: is memory usage the main reason we want to fork?16:10
shardySpamapS: what is your opinion on that?16:11
SpamapSzaneb: simplicity of the patch is the main reason I just used the num_workers bits16:12
*** gokrokve has quit IRC16:12
SpamapSzaneb: also "it works for nova-api"16:12
SpamapSbut we're not nova-api :)16:13
zanebnova-api is a very different beast16:13
*** e0ne has quit IRC16:14
zanebit may be a small patch, but it is not a small implementation16:14
shardyIt's kind of unfortunate that by using the oslo Service abstractions, you end up dealing with this complexity whether you want it or not16:14
zanebI would be strongly +1 on just launching multiple engines with systemd/upstart16:15
*** ativelkov_away is now known as ativelkov16:18
*** sjmc7_ is now known as sjmc716:18
*** sergmelikyan has quit IRC16:19
*** sergmelikyan has joined #heat16:22
openstackgerritA change was merged to openstack/heat: Sync oslo incubator  https://review.openstack.org/9470216:27
*** akuznetsov has quit IRC16:31
shardyzaneb: there must be a shared descriptor somewhere in the eventlet Hub, because if you move the ThreadGroupManager after the fork I get 5 periodic tasks fired (with 4 workers specified), but leaving it in __init__ I only get one16:33
shardyI'm digging through the eventlet code trying to figure out exactly why16:34
*** pablosan has joined #heat16:35
zaneb"The child does not inherit timers  from  its  parent" <- man 2 fork16:36
zanebso eventlet will have the data structures, but the underlying timer is never triggered16:39
zanebshardy: ^16:39
zanebI find that a bit icky16:39
shardyBut the "timer" implemented by oslo ThreadGroup is just a loop with a sleep in it, spawned to a greenthread16:39
shardyhttps://github.com/openstack/oslo-incubator/blob/master/openstack/common/loopingcall.py#L6216:39
zanebok, there goes that theory16:39
shardySo my guess is whatever is referenced by the Hub impelementation is common between the forked processes16:40
zanebcommon as in "shared memory"?16:40
shardyI'm not sure what implementation eventlet is using on my system atm16:41
shardythere are several16:41
zanebgood :/16:41
*** derekh_ has quit IRC16:42
*** IlyaE has quit IRC16:43
*** radez is now known as radez_g0n316:46
*** bnemec is now known as beekneemech16:49
*** radez_g0n3 is now known as radez16:49
*** nati_ueno has joined #heat16:50
openstackgerritSergey Kraynev proposed a change to openstack/heat: Replacing variable logger on LOG  https://review.openstack.org/9435616:54
openstackgerritchrisroberts proposed a change to openstack/heat: Fixes personality support in Rackspace ASG resource  https://review.openstack.org/9521716:54
*** harlowja_away is now known as harlowja_16:55
*** tomek_adamczewsk has joined #heat16:56
*** akuznetsov has joined #heat16:57
*** andrew_plunk has quit IRC16:57
*** IlyaE has joined #heat17:02
*** yogeshmehra has joined #heat17:02
*** nati_uen_ has joined #heat17:03
*** nati_ueno has quit IRC17:06
*** randallburt has joined #heat17:07
*** nati_uen_ has quit IRC17:07
*** alexheneveld has quit IRC17:08
*** nati_ueno has joined #heat17:08
*** akuznetsov has quit IRC17:10
*** SushilKM has joined #heat17:12
*** randallburt has quit IRC17:12
*** tomek_adamczewsk has quit IRC17:12
*** tomek_adamczewsk has joined #heat17:13
*** alexheneveld has joined #heat17:16
*** SushilKM has left #heat17:19
*** tomek_adamczewsk has quit IRC17:20
*** piyush has quit IRC17:22
*** zns has quit IRC17:24
*** tomek_adamczewsk has joined #heat17:26
*** SushillKM has joined #heat17:30
*** SushillKM has left #heat17:30
*** jprovazn_afk has quit IRC17:30
*** arbylee has quit IRC17:34
*** nati_ueno has quit IRC17:35
*** tomek_adamczewsk has quit IRC17:37
*** andersonvom has quit IRC17:41
*** randallburt has joined #heat17:44
*** e0ne has joined #heat17:49
*** SushillKM has joined #heat17:51
*** zns has joined #heat17:51
*** e0ne has quit IRC17:52
*** e0ne has joined #heat17:52
*** e0ne has quit IRC17:55
*** andrew_plunk has joined #heat18:04
*** piyush has joined #heat18:05
*** gokrokve has joined #heat18:06
*** radez is now known as radez_g0n318:10
*** jamie_h has quit IRC18:10
*** e0ne has joined #heat18:11
*** radez_g0n3 is now known as radez18:12
*** aweiteka has joined #heat18:13
*** jamie_h has joined #heat18:13
*** andrew_plunk has quit IRC18:16
*** andrew_plunk has joined #heat18:16
openstackgerritchrisroberts proposed a change to openstack/heat: Fixes personality support in Rackspace ASG resource  https://review.openstack.org/9521718:20
*** SushillKM has left #heat18:20
*** piyush1 has joined #heat18:25
*** sabeen has quit IRC18:26
*** sabeen has joined #heat18:27
*** radez is now known as radez_g0n318:27
*** piyush has quit IRC18:28
*** e0ne has quit IRC18:30
*** e0ne has joined #heat18:31
*** john3213 has joined #heat18:33
*** arbylee has joined #heat18:35
*** e0ne has quit IRC18:35
*** alexheneveld has quit IRC18:37
*** packet has quit IRC18:38
*** pscheie has quit IRC18:38
*** john3213 has left #heat18:38
*** swygue_ has joined #heat18:38
*** packet has joined #heat18:39
*** tspatzier has joined #heat18:40
*** arbylee has quit IRC18:42
*** akuznetsov has joined #heat18:43
SpamapSzaneb: btw, I'm -1 on "Just run them with systemd".. that puts the onus on deployers/packagers. We can do this really easily in oslo's service code as an alternate implementation that forks/execs with a short-circuit argument that makes this work quite automatically.18:44
*** andreaf_ has quit IRC18:48
*** IlyaE has quit IRC18:51
*** randallburt has quit IRC18:52
*** IlyaE has joined #heat18:53
*** zns has quit IRC18:54
*** arbylee has joined #heat18:55
*** IlyaE has quit IRC18:55
*** arbylee has quit IRC18:55
*** arbylee has joined #heat18:55
*** IlyaE has joined #heat18:56
*** radez_g0n3 is now known as radez18:58
*** ativelkov is now known as ativelkov_away18:59
*** mkollaro has quit IRC19:01
*** jdob has quit IRC19:08
*** zz_keyvan is now known as keyvan19:09
*** yogeshmehra has quit IRC19:11
*** yogeshmehra has joined #heat19:11
*** IlyaE has quit IRC19:12
*** akuznetsov has quit IRC19:14
*** randallburt has joined #heat19:18
*** lsmola has quit IRC19:20
*** cdent has quit IRC19:24
*** tspatzier has quit IRC19:27
*** jamie_h has quit IRC19:30
*** IlyaE has joined #heat19:31
*** alexheneveld has joined #heat19:34
*** zns has joined #heat19:35
*** erecio_1 has joined #heat19:36
*** zns has quit IRC19:37
*** zns has joined #heat19:37
*** erecio has quit IRC19:39
*** e0ne has joined #heat19:40
*** radez is now known as radez_g0n319:46
*** tspatzier has joined #heat19:46
*** sjmc7_ has joined #heat19:47
stevebakersdague, shardy, hai19:49
*** sjmc7 has quit IRC19:50
stevebakersdague, shardy, my plan for heat-slow stability is to get the custom image in devstack working, then we can switch to ubuntu as the test image, or a fedora with increased timeouts19:50
*** nati_ueno has joined #heat19:54
*** pablosan is now known as zz_pablosan19:54
*** erecio_2 has joined #heat19:58
*** piyush1 has quit IRC20:00
*** aniket has quit IRC20:00
*** erecio_1 has quit IRC20:01
*** zns has quit IRC20:04
*** zns has joined #heat20:10
*** erecio_2 has quit IRC20:14
*** tspatzier has quit IRC20:16
*** metral is now known as metral_zzz20:16
*** piyush has joined #heat20:19
*** rsmitty has quit IRC20:20
*** yogeshme_ has joined #heat20:21
*** zz_pablosan is now known as pablosan20:23
*** yogeshmehra has quit IRC20:24
*** yogeshme_ has quit IRC20:27
*** yogeshmehra has joined #heat20:28
*** randallburt has quit IRC20:30
-openstackstatus- NOTICE: Gerrit will be offline for about 20 minutes in order to rename some projects starting at 21:00 UTC.20:32
*** sabeen has quit IRC20:33
*** nati_uen_ has joined #heat20:36
*** piyush1 has joined #heat20:39
*** nati_ueno has quit IRC20:39
*** piyush has quit IRC20:41
*** arbylee has quit IRC20:42
*** arbylee has joined #heat20:43
*** nati_ueno has joined #heat20:43
*** yogeshmehra has quit IRC20:45
*** nati_uen_ has quit IRC20:45
*** nati_uen_ has joined #heat20:46
*** nati_ueno has quit IRC20:48
*** sjmc7_ has quit IRC20:48
*** gondoi is now known as zz_gondoi20:52
*** achampion has quit IRC20:52
*** nati_uen_ has quit IRC20:53
*** zz_gondoi is now known as gondoi20:54
*** nati_uen_ has joined #heat20:55
*** nati_ueno has joined #heat20:58
*** pafuent has left #heat20:59
*** nati_uen_ has quit IRC21:00
*** mattoliverau has quit IRC21:08
*** arbylee has quit IRC21:09
*** mwheckmann has quit IRC21:10
*** mattoliverau has joined #heat21:11
*** vpanizo has joined #heat21:21
*** edmund has quit IRC21:21
*** arbylee has joined #heat21:26
*** zns has quit IRC21:26
*** aweiteka has quit IRC21:27
openstackgerritJason Dunsmore proposed a change to openstack/heat: Implement context manager for stack lock  https://review.openstack.org/9527421:29
*** vijendar has quit IRC21:30
openstackgerritJason Dunsmore proposed a change to openstack/heat: Fix exceptions in update_stack  https://review.openstack.org/8940921:31
-openstackstatus- NOTICE: Gerrit is offline in order to rename some projects. ETA: 22:00.21:33
*** ChanServ changes topic to "Gerrit is offline in order to rename some projects. ETA: 22:00."21:33
*** nati_ueno has quit IRC21:37
*** zns has joined #heat21:42
Lotus907efican anyone tell me where the data that is supposed to be available at http://169.254.169.254/2009-04-04/meta-data/instance-id   should come from?21:44
Lotus907efiI have a group of overcloud systems where the cloud-init runs are not successful because although that IP is on the systems, the data that is supposed to be there is not there21:44
Lotus907efiI am trying to diagnose why the heat create of the overcloud failed21:44
Lotus907efior better yet how I can trace back in heat to see why this failed?21:44
Lotus907efiwhen I do a "heat event-list overcloud" on the undercloud system the only error I see is an authentication error21:45
Lotus907efiand when I try to re-run cloud-init on one of the failed overcloud nodes by hand I see a lot of errors about: url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [0/120s]: bad status code [404]21:46
*** piyush1 has quit IRC21:47
Lotus907efii.e. the data that is supposed to be there is not there21:48
Lotus907efithe IP address is in the routing table21:48
*** packet has quit IRC21:49
*** ativelkov_away is now known as ativelkov21:55
*** harlowja_ is now known as harlowja_away21:59
*** arbylee has quit IRC22:00
*** ativelkov is now known as ativelkov_away22:00
*** gondoi is now known as zz_gondoi22:01
*** pablosan has quit IRC22:03
*** pablosan has joined #heat22:06
zanebLotus907efi: 169.254.169.254 is the nova metadata server22:07
greghaynesSounds like it was a keystone issue22:08
greghaynes(giving some support offline)22:08
*** IlyaE has quit IRC22:09
*** randallburt has joined #heat22:09
*** alexheneveld has quit IRC22:12
Lotus907efiFrom the keystone.log file on the undercloud:  ERROR keystone.common.wsgi [-] object of type 'NoneType' has no len()22:13
*** openstackgerrit has quit IRC22:14
*** openstackgerrit has joined #heat22:15
*** blomquisg has quit IRC22:16
*** david-lyle has quit IRC22:16
*** openstackstatus has quit IRC22:18
*** openstack has joined #heat22:19
*** sgordon has quit IRC22:22
*** nati_ueno has joined #heat22:23
*** randallburt has quit IRC22:25
*** nati_ueno has quit IRC22:26
*** tims has quit IRC22:26
*** ChanServ changes topic to "support @ https://ask.openstack.org | developer wiki @ https://wiki.openstack.org/wiki/Heat | development @ https://launchpad.net/heat | logged @ http://eavesdrop.openstack.org/irclogs/%23heat/"22:26
*** gokrokve has quit IRC22:26
*** gokrokve has joined #heat22:27
openstackgerritMonty Taylor proposed a change to openstack/heat-specs: Updated gitreview file for repo rename  https://review.openstack.org/9529222:28
*** andrew_plunk has quit IRC22:31
*** gokrokve has quit IRC22:32
openstackgerritMonty Taylor proposed a change to openstack/heat-specs: Updated gitreview file for repo rename  https://review.openstack.org/9529722:34
*** akuznetsov has joined #heat22:37
*** IlyaE has joined #heat22:47
*** akuznetsov has quit IRC22:47
*** IlyaE has quit IRC22:51
*** piyush has joined #heat22:55
*** TravT has quit IRC22:56
*** dims has quit IRC22:56
*** nati_ueno has joined #heat23:09
*** zns has quit IRC23:09
*** nati_ueno has quit IRC23:13
*** achampion has joined #heat23:14
*** piyush has left #heat23:24
*** nati_ueno has joined #heat23:26
*** e0ne has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!