| *** cadenzajon has quit IRC | 00:00 | |
| *** lindsayk has quit IRC | 00:01 | |
| sdake__ | zaneb you sure if isinstance(params[k], collections.Iterable): | 00:02 |
|---|---|---|
| sdake__ | is right for a list? | 00:02 |
| sdake__ | it seems to be triggering on everything | 00:02 |
| zaneb | note that strings are also iterable | 00:02 |
| sdake__ | (strings included) | 00:02 |
| zaneb | you have to check for the strings first | 00:02 |
| zaneb | you can use isinstance(x, list) | 00:03 |
| zaneb | better is isinstance(x, (list, tuple)) | 00:03 |
| *** zns has joined #heat | 00:05 | |
| *** zns has quit IRC | 00:05 | |
| sdake__ | thanks thats better | 00:06 |
| sdake__ | fixed a bug and run_tests.sh reports 2 more | 00:07 |
| sdake__ | imo run_tests.sh is unreliable :( | 00:07 |
| openstackgerrit | Steven Dake proposed a change to openstack/heat: Use olso.messaging https://review.openstack.org/72798 | 00:08 |
| sdake__ | new oslo.messaging added a serilaizer for hte context which is handy | 00:23 |
| sdake__ | wonder why that didn't go in oslo-incuabot | 00:23 |
| *** alexpilotti has quit IRC | 00:31 | |
| *** gokrokve_ has quit IRC | 00:32 | |
| *** cody-somerville has joined #heat | 00:34 | |
| sdake__ | calling into messaging._drivers | 00:36 |
| sdake__ | sounds evil | 00:36 |
| sdake__ | I guess we can deal with it later | 00:36 |
| *** saurabhs1 has left #heat | 00:39 | |
| *** david-lyle has joined #heat | 00:41 | |
| *** gokrokve has joined #heat | 00:41 | |
| *** nati_ueno has quit IRC | 00:45 | |
| SpamapS | ok.. spike failing.. I think I'll have to investigate abandon/fix/adopt | 00:47 |
| SpamapS | sdake__: you're not supposed to be using run_tests.sh | 00:48 |
| SpamapS | sdake__: tox ftw | 00:48 |
| SpamapS | I mean, people use it.. its fine.. but. don't use. | 00:48 |
| sdake__ | i use nosetest acutally | 00:48 |
| sdake__ | so i can get printout | 00:49 |
| SpamapS | tox prints out | 00:50 |
| SpamapS | well some :) | 00:50 |
| SpamapS | IMO prints too much | 00:50 |
| sdake__ | i'll dig into that when i have more time | 00:51 |
| *** gokrokve has quit IRC | 00:56 | |
| sdake__ | zaneb what is the difference between * and ** | 00:59 |
| *** nati_ueno has joined #heat | 01:04 | |
| *** nati_ueno has quit IRC | 01:07 | |
| *** openstackgerrit has quit IRC | 01:10 | |
| *** openstackgerrit has joined #heat | 01:11 | |
| *** nati_ueno has joined #heat | 01:18 | |
| SpamapS | ugh | 01:27 |
| SpamapS | we have abandon in heatclient, but not adopt yet? | 01:27 |
| SpamapS | ahh unreleased | 01:29 |
| SpamapS | stevebaker: I can haz python-heatclient releazzz? | 01:29 |
| *** erkules has joined #heat | 01:36 | |
| Slower | heh | 01:38 |
| *** pvaneck has quit IRC | 01:42 | |
| *** achampion has joined #heat | 01:43 | |
| *** spzala has joined #heat | 01:45 | |
| *** gokrokve has joined #heat | 01:46 | |
| tango | SpamapS: Hi Clint, still around? | 01:54 |
| *** jufeng has joined #heat | 01:59 | |
| *** mkollaro has quit IRC | 02:00 | |
| *** lnxnut has joined #heat | 02:10 | |
| *** tango has quit IRC | 02:15 | |
| *** erkules_ has joined #heat | 02:18 | |
| *** spzala has quit IRC | 02:20 | |
| *** jufeng has quit IRC | 02:20 | |
| *** erkules has quit IRC | 02:21 | |
| *** ramishra has joined #heat | 02:31 | |
| *** lazy_prince is now known as killer_prince | 02:41 | |
| *** jufeng has joined #heat | 02:41 | |
| *** dtalton2 has quit IRC | 02:49 | |
| *** faramir1 has joined #heat | 02:52 | |
| *** lnxnut has quit IRC | 02:52 | |
| *** lnxnut has joined #heat | 02:55 | |
| *** ramishra has quit IRC | 02:58 | |
| *** lindsayk has joined #heat | 03:08 | |
| *** annegentle has joined #heat | 03:10 | |
| *** jufeng has quit IRC | 03:34 | |
| *** jufeng has joined #heat | 03:39 | |
| *** faramir1 has quit IRC | 03:43 | |
| *** jasond has joined #heat | 03:45 | |
| *** jasond has left #heat | 03:46 | |
| *** andersonvom has quit IRC | 03:56 | |
| *** sballe_ has quit IRC | 04:12 | |
| *** andersonvom has joined #heat | 04:12 | |
| *** sballe_ has joined #heat | 04:13 | |
| *** lindsayk1 has joined #heat | 04:13 | |
| *** jufeng has quit IRC | 04:14 | |
| *** jufeng has joined #heat | 04:14 | |
| *** lindsayk has quit IRC | 04:17 | |
| *** lnxnut has quit IRC | 04:18 | |
| *** sballe_ has quit IRC | 04:20 | |
| *** lnxnut has joined #heat | 04:22 | |
| *** harlowja is now known as harlowja_away | 04:31 | |
| *** Michalik has quit IRC | 04:32 | |
| *** andersonvom has quit IRC | 04:33 | |
| *** andersonvom has joined #heat | 04:36 | |
| *** ramishra has joined #heat | 04:37 | |
| *** chandan_kumar has joined #heat | 04:48 | |
| *** wchrisj has quit IRC | 04:49 | |
| *** wchrisj__ has joined #heat | 05:16 | |
| *** lindsayk1 has quit IRC | 05:25 | |
| *** lindsayk has joined #heat | 05:28 | |
| *** ramishra has quit IRC | 05:36 | |
| *** lindsayk has quit IRC | 05:39 | |
| *** chandan_kumar has quit IRC | 06:01 | |
| *** killer_prince has quit IRC | 06:05 | |
| *** killer_prince has joined #heat | 06:08 | |
| *** killer_prince has quit IRC | 06:08 | |
| *** killer_prince has joined #heat | 06:08 | |
| openstackgerrit | Jenkins proposed a change to openstack/heat: Imported Translations from Transifex https://review.openstack.org/72566 | 06:10 |
| *** chandan_kumar has joined #heat | 06:14 | |
| *** yogesh has joined #heat | 06:19 | |
| *** wchrisj__ has quit IRC | 06:22 | |
| *** chandan_kumar has quit IRC | 06:28 | |
| *** yogesh has quit IRC | 06:37 | |
| *** ramishra has joined #heat | 06:40 | |
| *** chandan_kumar has joined #heat | 06:41 | |
| *** nati_ueno has quit IRC | 06:47 | |
| *** chandan_kumar has quit IRC | 06:48 | |
| *** nati_ueno has joined #heat | 06:48 | |
| *** ramishra has quit IRC | 06:57 | |
| *** ramishra has joined #heat | 06:58 | |
| *** chandan_kumar has joined #heat | 06:59 | |
| *** faramir has joined #heat | 06:59 | |
| *** nati_ueno has quit IRC | 07:02 | |
| *** ramishra has quit IRC | 07:02 | |
| *** faramir has quit IRC | 07:05 | |
| *** bada_ has quit IRC | 07:08 | |
| *** akuznetsov has quit IRC | 07:12 | |
| *** killer_prince has quit IRC | 07:12 | |
| *** yogesh has joined #heat | 07:14 | |
| *** chandan_kumar has quit IRC | 07:22 | |
| *** ramishra_ has joined #heat | 07:25 | |
| *** jufeng has quit IRC | 07:35 | |
| *** yogesh_ has joined #heat | 07:37 | |
| *** yogesh has quit IRC | 07:39 | |
| *** killer_prince has joined #heat | 07:46 | |
| *** killer_prince has quit IRC | 07:46 | |
| *** killer_prince has joined #heat | 07:46 | |
| *** jufeng has joined #heat | 07:53 | |
| *** cody-somerville has quit IRC | 07:57 | |
| *** killer_prince is now known as lazy_prince | 08:00 | |
| *** cfriesen_ has quit IRC | 08:02 | |
| *** ramishra_ has quit IRC | 08:15 | |
| *** yogesh_ has quit IRC | 08:22 | |
| openstackgerrit | A change was merged to openstack/heat: Change Stack timestamps to save correct info https://review.openstack.org/74519 | 08:23 |
| *** kfox1111 has quit IRC | 08:35 | |
| *** akuznetsov has joined #heat | 08:35 | |
| *** gokrokve has quit IRC | 08:38 | |
| *** gokrokve_ has joined #heat | 08:41 | |
| *** gokrokv__ has joined #heat | 08:43 | |
| *** gokrokve_ has quit IRC | 08:45 | |
| openstackgerrit | Jesse proposed a change to openstack/heat: Fix python 3.x compatibility for print function https://review.openstack.org/77344 | 09:05 |
| *** david_lyle_ has joined #heat | 09:10 | |
| *** akuznetsov has quit IRC | 09:12 | |
| *** david-lyle has quit IRC | 09:13 | |
| *** david-lyle has joined #heat | 09:13 | |
| *** david_lyle_ has quit IRC | 09:15 | |
| openstackgerrit | Jesse proposed a change to openstack/heat: Fix python 3.x compatibility for metaclass https://review.openstack.org/77345 | 09:15 |
| *** gokrokv__ has quit IRC | 09:17 | |
| *** mkollaro has joined #heat | 09:29 | |
| *** sdake__ has quit IRC | 09:36 | |
| *** tspatzier has joined #heat | 09:42 | |
| *** tspatzier has quit IRC | 09:47 | |
| *** gokrokve has joined #heat | 09:48 | |
| *** gokrokve_ has joined #heat | 09:49 | |
| *** chandan_kumar has joined #heat | 09:51 | |
| *** gokrokve has quit IRC | 09:52 | |
| *** jufeng has quit IRC | 09:54 | |
| *** chandan_kumar has quit IRC | 10:02 | |
| *** david_lyle_ has joined #heat | 10:19 | |
| *** david-lyle has quit IRC | 10:23 | |
| *** chandan_kumar has joined #heat | 10:25 | |
| *** alexpilotti has joined #heat | 10:40 | |
| *** lazy_prince has quit IRC | 10:57 | |
| *** jamieh has joined #heat | 10:58 | |
| *** asalkeld has quit IRC | 11:03 | |
| *** Michalik has joined #heat | 11:12 | |
| *** lazy_prince has joined #heat | 11:16 | |
| *** lazy_prince is now known as killer_prince | 11:16 | |
| *** killer_prince has quit IRC | 11:28 | |
| *** chandan_kumar has quit IRC | 11:36 | |
| *** chandan_kumar has joined #heat | 11:54 | |
| *** alexpilotti has quit IRC | 11:54 | |
| *** killer_prince has joined #heat | 11:59 | |
| *** killer_prince has quit IRC | 12:10 | |
| *** lazy_prince has joined #heat | 12:16 | |
| *** lazy_prince is now known as killer_prince | 12:16 | |
| *** e0ne has joined #heat | 12:24 | |
| *** killer_prince has quit IRC | 12:30 | |
| *** lazy_prince has joined #heat | 12:36 | |
| *** lazy_prince is now known as killer_prince | 12:37 | |
| *** jamieh has quit IRC | 12:37 | |
| *** chandan_kumar has quit IRC | 12:38 | |
| *** alexpilotti has joined #heat | 12:45 | |
| *** rpothier has quit IRC | 13:01 | |
| *** rpothier has joined #heat | 13:02 | |
| *** erkules_ is now known as erkules | 13:02 | |
| *** akuznetsov has joined #heat | 13:03 | |
| openstackgerrit | A change was merged to openstack/python-heatclient: Remove None for dict.get() https://review.openstack.org/74694 | 13:11 |
| openstackgerrit | A change was merged to openstack/python-heatclient: Update apiclient.base and reuse new functional https://review.openstack.org/72877 | 13:11 |
| openstackgerrit | A change was merged to openstack/python-heatclient: Do not use the '+' operation dict_items() https://review.openstack.org/74710 | 13:12 |
| *** christophk_ has quit IRC | 13:15 | |
| *** alexpilotti has quit IRC | 13:27 | |
| *** fandi has joined #heat | 13:29 | |
| *** mkollaro has quit IRC | 13:38 | |
| *** e0ne has quit IRC | 14:24 | |
| *** blamar has quit IRC | 14:52 | |
| *** lnxnut has quit IRC | 14:58 | |
| *** e0ne has joined #heat | 15:03 | |
| *** lnxnut has joined #heat | 15:04 | |
| *** e0ne has quit IRC | 15:12 | |
| *** akuznetsov has quit IRC | 15:12 | |
| *** fandi has quit IRC | 15:15 | |
| *** akuznetsov has joined #heat | 15:18 | |
| *** e0ne has joined #heat | 15:22 | |
| *** e0ne has quit IRC | 15:26 | |
| *** lnxnut has quit IRC | 15:27 | |
| *** akuznetsov has quit IRC | 15:32 | |
| *** lnxnut has joined #heat | 15:35 | |
| *** alexpilotti has joined #heat | 15:37 | |
| *** lnxnut has quit IRC | 15:39 | |
| *** e0ne has joined #heat | 15:43 | |
| *** jamieh has joined #heat | 15:43 | |
| *** sdake__ has joined #heat | 15:44 | |
| *** sdake__ has quit IRC | 15:47 | |
| *** sdake_ has joined #heat | 15:48 | |
| sdake_ | hey guys | 15:48 |
| sdake_ | anyone know why nosetest would run the same test 5 times when it i s told only once to run it? | 15:48 |
| *** akuznetsov has joined #heat | 15:51 | |
| *** e0ne has quit IRC | 15:53 | |
| *** mestery_ has quit IRC | 15:54 | |
| *** mestery has joined #heat | 15:54 | |
| *** tomek_adamczewsk has joined #heat | 15:55 | |
| *** blamar has joined #heat | 15:56 | |
| *** aignatov_ is now known as aignatov | 16:00 | |
| *** e0ne has joined #heat | 16:02 | |
| *** fandi has joined #heat | 16:10 | |
| *** e0ne has quit IRC | 16:21 | |
| *** alexpilotti has quit IRC | 16:22 | |
| *** mspreitz has joined #heat | 16:23 | |
| mspreitz | Is anybody here able to make Ceilometer work? | 16:26 |
| mspreitz | radix: are you there? | 16:30 |
| *** spzala has joined #heat | 16:34 | |
| *** lnxnut has joined #heat | 16:35 | |
| *** aignatov2 has joined #heat | 16:38 | |
| *** ruhe- has joined #heat | 16:39 | |
| *** SergeyLukjanov2 has joined #heat | 16:40 | |
| *** ruhe has quit IRC | 16:40 | |
| *** ruhe- is now known as ruhe | 16:40 | |
| *** lnxnut has quit IRC | 16:40 | |
| *** aignatov has quit IRC | 16:41 | |
| *** aignatov2 is now known as aignatov | 16:41 | |
| *** SergeyLukjanov has quit IRC | 16:41 | |
| *** SergeyLukjanov2 is now known as SergeyLukjanov | 16:41 | |
| radix | mspreitz: i am out at breakfast... should be back at a computer in an hour or so | 16:59 |
| *** lnxnut has joined #heat | 17:00 | |
| *** fandi has quit IRC | 17:14 | |
| *** wchrisj has joined #heat | 17:22 | |
| *** wchrisj has quit IRC | 17:26 | |
| *** fandi has joined #heat | 17:26 | |
| *** wchrisj has joined #heat | 17:28 | |
| *** wchrisj has quit IRC | 17:31 | |
| *** killer_prince is now known as lazy_prince | 17:32 | |
| *** e0ne has joined #heat | 17:32 | |
| *** e0ne has quit IRC | 17:37 | |
| *** gokrokve_ has quit IRC | 17:43 | |
| *** gokrokve has joined #heat | 17:43 | |
| therve` | mspreitz, Yeah I made it work relatively recently | 17:48 |
| *** spzala has quit IRC | 17:49 | |
| *** lnxnut has quit IRC | 17:52 | |
| *** lnxnut has joined #heat | 17:53 | |
| sdake_ | victory test_api_openstack_v1.py is passing all tests | 17:59 |
| *** jamieh has quit IRC | 18:00 | |
| sdake_ | down to 24 errors getting closer | 18:13 |
| *** lnxnut has quit IRC | 18:20 | |
| SpamapS | sdake_: is this still trying to get from oslo-incubator rpc to oslo.messaging? | 18:20 |
| *** lnxnut has joined #heat | 18:21 | |
| *** lnxnut has quit IRC | 18:21 | |
| *** lnxnut has joined #heat | 18:22 | |
| sdake_ | SpamapS yup | 18:26 |
| *** lnxnut has quit IRC | 18:26 | |
| sdake_ | should have sometihng ready for review today | 18:26 |
| sdake_ | I finally solved the last hard problem (I think) | 18:26 |
| sdake_ | serialization of exceptions | 18:26 |
| SpamapS | sdake_: cool. I gave up on the retry spike. | 18:26 |
| mspreitz | therve`: Is the Resource Usage / stats display in Horizon showing proper data? | 18:27 |
| SpamapS | Which is.. devastating frankly. | 18:27 |
| sdake_ | I am really close with oslo.messging | 18:27 |
| SpamapS | If I can't figure out how to use abandon/adopt .. we will likely have to rip Heat out | 18:27 |
| SpamapS | I'm still a bit shocked that we're the only people who want updates to not kill stacks. | 18:28 |
| *** cfriesen_ has joined #heat | 18:30 | |
| sdake_ | SpamapS I think what went wrong in the icehouse timeframe is that blueprint was set to high | 18:31 |
| mspreitz | SpamapS: what do you mean by only people? | 18:31 |
| sdake_ | it sounds like it should have been set to critical | 18:31 |
| mspreitz | SpamapS: in what way to updates kill stacks? | 18:32 |
| mspreitz | s/to/do | 18:32 |
| SpamapS | mspreitz: any error -> dead stack | 18:32 |
| sdake_ | zaneb had alot of high prio blueprints on his plate and he cut one that allowed him to work on the others | 18:32 |
| mspreitz | SpamapS: out of what would you rip Heat if you ca not figure out how to use abandon/adopt? | 18:32 |
| sdake_ | SpamapS I'll make sure in juno1 we make it a priority atleast from the rht side to get update in a working state - we need it for tuskar and a hos tof other projects | 18:33 |
| SpamapS | sdake_: well I think I should have looked deeper into the problem. I thought we could resume more reliably, but the devil is in the way updates are implemented | 18:33 |
| SpamapS | mspreitz: TripleO | 18:33 |
| mspreitz | that would be bad | 18:33 |
| mspreitz | In what way is the stack dead if there is an error in update? | 18:33 |
| SpamapS | yeah, but the stateful nature is scaring the bejezus out of me | 18:33 |
| SpamapS | one slip up and your state is screwed | 18:33 |
| SpamapS | and the state is poorly recorded | 18:33 |
| sdake_ | sounded lik cypher for a minut ethere ;) | 18:34 |
| SpamapS | basically now that I"m actually trying to _USE_ heat, I am seeing that it is really far from being production ready | 18:34 |
| sdake_ | i think far is an overstatement | 18:34 |
| sdake_ | It definately has some kinks to be worked out | 18:34 |
| SpamapS | sdake_: a blueprint that requires an entire rewrite of a major function == far in my book. | 18:34 |
| sdake_ | I think icehosue is the first version of heat I'd feel comfortable deploying in production | 18:35 |
| sdake_ | update works, just not in the way that is useable for you spamaps | 18:35 |
| SpamapS | mspreitz: if you encounter any error while updating a stack, and you cannot rollback, the stack is dead. You cannot update it anymore, and its state in the database is completely inconsistent. | 18:35 |
| sdake_ | its not like update falls over | 18:35 |
| *** cfriesen_ has quit IRC | 18:35 | |
| SpamapS | sdake_: it falls over _25% of the time_ in our cloud. | 18:36 |
| sdake_ | clearly it could be better | 18:36 |
| sdake_ | SpamapS yup I get the scale issue | 18:36 |
| mspreitz | SpamapS: that sounds pretty bad. Do you understand what has to change to fix this? | 18:36 |
| SpamapS | mspreitz: yes. update must record incremental state so that it can be resumed. | 18:36 |
| sdake_ | like a journal | 18:36 |
| mspreitz | yeah, makes sense | 18:37 |
| sdake_ | tbh I guess I dont understand what is so hard about making a journal | 18:37 |
| sdake_ | but if zane says 2 months | 18:37 |
| sdake_ | something could probably be hacked togehter in 1 mo | 18:37 |
| sdake_ | but to meet zane's standards 2 mos is right on target | 18:37 |
| SpamapS | sdake_: my current spike is a journal | 18:37 |
| SpamapS | it might be made to work | 18:37 |
| mspreitz | If I could get some IBM colleagues to help, could this go into Icehouse-3? | 18:38 |
| SpamapS | and then I'll be less pessimistic about the other Heat debt we have to repay. | 18:38 |
| SpamapS | mspreitz: Monday. | 18:38 |
| mspreitz | OK | 18:38 |
| *** e0ne has joined #heat | 18:38 | |
| sdake_ | mspreitz need to speak with stevebaker about jamming new stuff into i3 - 4th is the dealdine | 18:38 |
| mspreitz | deadline for BPs or bug fixes? | 18:38 |
| SpamapS | If it isn't in a working state by Monday, it will require exceptions. | 18:39 |
| sdake_ | I actually added a cat fro mthe rht side to assist spamaps | 18:39 |
| mspreitz | I see | 18:39 |
| sdake_ | but the timeline (5 days) just too short | 18:39 |
| SpamapS | mspreitz: this is an invasive enough change I think there is concern that it would de-stabilize other bits. | 18:39 |
| sdake_ | i wonder if working mean sreviewed and in master :) | 18:39 |
| mspreitz | I see | 18:39 |
| *** wchrisj has joined #heat | 18:39 | |
| sdake_ | oslo.messaging may need an exception i nthis case | 18:39 |
| SpamapS | working means it has done the thing it should do and has at least basic test coverage | 18:40 |
| SpamapS | and it is in the review queue.. | 18:40 |
| sdake_ | I dont htink I can get it reviewed by i3 | 18:40 |
| sdake_ | mspreitz daedline for blueprint implementations is march 4th | 18:40 |
| sdake_ | bug fixes can come for another mo or so | 18:40 |
| SpamapS | I did try to get that done .. but I hit the wall yesterday because at this point I need the engine scheduler to write out the journal and then in the update case, I need to read that journal to recover the state of the update.. | 18:41 |
| mspreitz | I think this could be called a bug fix, but if it's really invasive I understand the concern | 18:41 |
| SpamapS | mspreitz: the spike to use a journal is an attempt at being less invasive. | 18:42 |
| sdake_ | its definately a feature | 18:42 |
| SpamapS | the appropriate architectural change has us letting go of state and just keeping track of the change in user intentions. | 18:42 |
| sdake_ | who manages state then? | 18:42 |
| SpamapS | sdake_: talk to zaneb .. the bug it fixes is that when you have a failed update, it is entirely possible when you delete that stack you are not deleting everything. | 18:43 |
| SpamapS | sdake_: nova.. neutron.. etc. | 18:43 |
| sdake_ | spamaps that part is a bug fix | 18:43 |
| sdake_ | but my understanding was you want to be able to update stacks after they have failed | 18:43 |
| sdake_ | that is a feature | 18:44 |
| mspreitz | Seems to me like a focus on state is the direct approach. User gives desired state, Heat migrates actual state to match desired state. | 18:44 |
| SpamapS | basically if I look at the template I have, and I compare it to what is in nova, neutron, cinder, etc, then I can just make what I have expressed happen. | 18:44 |
| mspreitz | SpamapS: yeah, that's what I mean | 18:44 |
| SpamapS | sdake_: you don't think it is a bug that on any hiccup, any problem whatsoever ... your only recourse is rollback (just as scary) or delete the entire stack? | 18:44 |
| sdake_ | SpamapS what is your recourse during a create operation? | 18:45 |
| SpamapS | sdake_: same. | 18:45 |
| sdake_ | right, so update and create are lifecycle operations | 18:46 |
| SpamapS | sdake_: we could easily resume create, though, because we don't have a previous delta to consider. | 18:46 |
| sdake_ | agree, and we probably want to intoroduce that | 18:46 |
| sdake_ | but we haven't yet - need more minerals :) | 18:46 |
| SpamapS | sdake_: I'm also just frustrated because I've known about this problem for a year.. and I haven't managed to get it fixed. :-/ | 18:47 |
| sdake_ | SpamapS just so you are aware, I am getting alot of feedback not just from you but from rht customers that update is a huge model for them | 18:47 |
| sdake_ | I think when we originally tackled update it was more of like a lets get it to work | 18:47 |
| SpamapS | right... | 18:47 |
| SpamapS | we have one internal customer who uses create / delete only... | 18:47 |
| sdake_ | the viewpoint on the value of update is changing and resources will flow accordingly | 18:47 |
| SpamapS | everybody else evaluates update.. has it fail on them.. and then abandons using Heat | 18:47 |
| SpamapS | (everybody else internally I mean) | 18:48 |
| *** wchrisj has quit IRC | 18:48 | |
| sdake_ | yup its a problem | 18:48 |
| sdake_ | we also need a way to autoscale native instance os::Nova::servers | 18:48 |
| SpamapS | I get why we're here. I just really really need to have a fix. | 18:48 |
| sdake_ | right now can only autoscale ec2 | 18:48 |
| sdake_ | clunky | 18:48 |
| SpamapS | sdake_: autoscale.. meh ;) | 18:48 |
| sdake_ | well I've commited to you that we are going to make that a priority and address in early i | 18:49 |
| sdake_ | atleast if I can convince the cats that work on it to :) | 18:49 |
| sdake_ | I've explained I've got alot of other pressue to make it work better then it does as well | 18:49 |
| sdake_ | what I can't do is figure out how to solve it in icehouse because it wasn'tclear to me at the time it was a problem | 18:49 |
| SpamapS | sdake_: that is really too late. I need it addressed immediately.. so we will likely have to run a branch of heat until i releases. | 18:50 |
| mspreitz | OS::Heat::AutoScalingGroup can scale any resource | 18:50 |
| sdake_ | I think next time if we hve any of these critical bugs, we shoudl mark them in critical in launchpad | 18:50 |
| SpamapS | sdake_: well it wasn't clear to me exactly how critical it was until we started using Heat in a real system with real issues. | 18:51 |
| mspreitz | stack update definitely figures in IBM's plans too | 18:51 |
| sdake_ | right which is the same thing that our customers are hitting with it | 18:51 |
| SpamapS | I figured we would just kick it in the head or something if we ever hit these corner cases. | 18:51 |
| sdake_ | thing is the info is just hitting this month from multiple sources | 18:51 |
| SpamapS | At this point I need to test abandon/--edit abandon file to reflect reality--/adopt | 18:52 |
| sdake_ | SpamapS stevebaker is responsible for heat, I just try to help coordinate people from the rht side | 18:52 |
| SpamapS | sdake_: this is nobody's fault. I'm just venting. | 18:52 |
| sdake_ | cool | 18:52 |
| sdake_ | I just dont want you to tihnk we let you down :) | 18:52 |
| SpamapS | I look really bad right now. It is my own doing. | 18:52 |
| sdake_ | you cant fix what you dont know until you know it | 18:52 |
| SpamapS | Because I _also_ have to land graceful updates so we can live migrate instances while doing an update. | 18:53 |
| sdake_ | how do you live migrate with heat? | 18:53 |
| SpamapS | you don't | 18:53 |
| SpamapS | you do it in instance | 18:53 |
| sdake_ | mspreitz is there an example hot template - tha would be really hot :) | 18:53 |
| mspreitz | sdake_: example that does what? There are trivial examples in heat-templates already. | 18:54 |
| sdake_ | lsat itme i used autoscaling group it wouldn't scale an OS::Nova::Server instance | 18:55 |
| sdake_ | complains about image ids and such | 18:56 |
| sdake_ | is that fixed in icehouse? | 18:56 |
| sdake_ | iirc the autoscaling code was looking for an ec2 resources not an OS resource | 18:56 |
| *** wchrisj has joined #heat | 18:56 | |
| sdake_ | mspreitz I got people bitching about that nonstop, if you havesome ibm cats to fix it that would rock :) | 18:57 |
| SpamapS | I believe there's a review up to make scaling any resource work | 18:57 |
| mspreitz | Yes, I am testing it now | 18:58 |
| mspreitz | Let me review my test cases... | 18:58 |
| sdake_ | basically if you take the autoscaling sample for f17and change the resource type of the scaling group to OS::Nova::Server, it no worky | 18:59 |
| sdake_ | of the instance of the scaling group | 18:59 |
| sdake_ | I have not tried in a couple months | 18:59 |
| sdake_ | maybe something has landed | 18:59 |
| SpamapS | sdake_: no I think it is in review now | 19:00 |
| SpamapS | not landed yet | 19:00 |
| mspreitz | http://paste.openstack.org/show/71090/ | 19:00 |
| mspreitz | https://review.openstack.org/#/c/74229/ --- it is in review.. I am testing. That template I pasted works with this patch | 19:00 |
| mspreitz | If/when I can wrangle my Ceilometer into behaving I can finish testing the "auto" part | 19:02 |
| mspreitz | I now have a Ceilo that produces plausible samples, but at odd times. Have not tried aggregation yet. And the display in Horizon is messed up. | 19:03 |
| mspreitz | Gotta take care of some other things now, but hope to get back to it soon. | 19:03 |
| *** wchrisj has quit IRC | 19:06 | |
| sdake_ | SpamapS any ide ahow to debug this test cawse or what this error means: http://paste.fedoraproject.org/81561/00720139 | 19:07 |
| radix | mspreitz: heya, I'm back | 19:09 |
| mspreitz | radix: thanks. I'm trying to get a decent Ceilometer so I can finish testing OS::Heat::AutoScalingGroup. But right now I'm doing other things, hope to get back to it later today. | 19:10 |
| radix | okie doke | 19:10 |
| mspreitz | What I have right now is decent data but not at regular times. Is this what I should expect? | 19:11 |
| radix | mspreitz: honestly I don't know hardly anything about ceilometer | 19:11 |
| sdake_ | radix time to learn thats what triggers autoscaling :) | 19:17 |
| radix | sdake_: hitting URLs triggers autoscaling | 19:18 |
| mspreitz | good point | 19:18 |
| sdake_ | radix what converts the ceilometer alarms to hit the urls? | 19:20 |
| sdake_ | radix you mean in the new autoscaling model, not the existing | 19:20 |
| radix | sdake_: no, even the existing one. anything can hit a signal URL | 19:20 |
| sdake_ | radix shows how much I know | 19:21 |
| sdake_ | just for your edification, pick a random file out of heat, and git log | grep Author | sor t | uniq -d | wc -l | 19:21 |
| sdake_ | everything has been rewritten like 30 time s:) | 19:22 |
| *** tspatzier has joined #heat | 19:22 | |
| *** lnxnut has joined #heat | 19:23 | |
| mspreitz | Woa, tspatzier, you need to know the bad news about stack update | 19:23 |
| mspreitz | I'll send a copy of recent IRC discussion | 19:24 |
| sdake_ | i think reqally its a matter of scale | 19:25 |
| sdake_ | stack update works pretty well up to a limited scale | 19:25 |
| sdake_ | people are starting to push its limits on scale and running into problems tha tneed addressing | 19:25 |
| sdake_ | its not that the idate of update is fundamentally broken, it just needs more minerals :) | 19:25 |
| mspreitz | sent | 19:26 |
| SpamapS | sdake_: it isn't really about scale.. | 19:26 |
| SpamapS | sdake_: it is about the cost of deleting | 19:26 |
| sdake_ | good point | 19:26 |
| *** tspatzier has quit IRC | 19:27 | |
| *** lnxnut has quit IRC | 19:27 | |
| *** lnxnut has joined #heat | 19:35 | |
| therve` | mspreitz, I didn't look into Horizon. But I had cpu alarms working | 19:37 |
| *** therve` is now known as therve | 19:37 | |
| mspreitz | therve: were you getting CPU data at regular intervals? | 19:40 |
| *** lnxnut has quit IRC | 19:40 | |
| therve | mspreitz, Yes. You may need to change the internal in the pipeline, though | 19:42 |
| therve | By default it's 10 minutes or something | 19:42 |
| *** daneyon has joined #heat | 19:44 | |
| mspreitz | therve: interesting. My CPU data is indeed coming evey 10 minutes, while other data (e.g. disk.root.size) that Horizon says are also from nova come at odd times | 19:47 |
| therve | mspreitz, http://openstack.redhat.com/CeilometerQuickStart has some good content | 19:50 |
| radix | fwiw, cpu time, load etc are generally pretty bad things for autoscaling. more often things like loadbalancer load, request time, or internal message queus are the best thing to base autoscaling on | 19:52 |
| radix | queue lengths, rather | 19:52 |
| therve | radix, They are pretty great for demo :) | 19:53 |
| radix | heh, yes | 19:53 |
| therve | Also maybe using cpu for scaling down is not totally bad | 19:55 |
| mspreitz | I agree. But I am also interested in server metrics. | 19:57 |
| SpamapS | response time | 19:58 |
| SpamapS | Google has published some papers on this. | 19:59 |
| SpamapS | Basically nothing else matters. If you can spend money to bring down response time, do it. | 19:59 |
| SpamapS | and most host based metrics are just bad proxies for response time | 19:59 |
| openstackgerrit | Thomas Herve proposed a change to openstack/heat: Add a validation step to parameters schema https://review.openstack.org/69655 | 20:00 |
| *** topol has joined #heat | 20:04 | |
| therve | Well, except that you can use host based metrics to predict that your response time is going to increase | 20:04 |
| therve | Whereas if you react on response time becoming bad it's already a bit too late | 20:04 |
| *** topol_ has joined #heat | 20:06 | |
| SpamapS | therve: that is a proxy | 20:07 |
| SpamapS | therve: too late is too late with CPU as well. Your response time will have an ideal number, and then at max utilization, you'll be just a bit above that. So you should scale up when your response times start to rise, and down when they are ideal. | 20:08 |
| *** topol has quit IRC | 20:09 | |
| *** topol_ is now known as topol | 20:09 | |
| SpamapS | therve: so if your app, without any contention, responds in 250ms, you should scale up at a percentage of that response time increase.. so 20% increase would be 300ms .. thats a good time to scale up... if it drops down to 250ms you are at a point where you _can_ scale down. | 20:10 |
| SpamapS | therve: and then the magic thing to do is to find out whether your app runs well in smaller and smaller servers.. and then you get more granular scaling and thus in theory cost savings. :) | 20:11 |
| SpamapS | There's also the response time of non-scalable resources to consider... that does complicate things. | 20:12 |
| *** topol has quit IRC | 20:20 | |
| * SpamapS decides to try another spike.. because who needs Saturdays anyway? | 20:20 | |
| * SpamapS closes irc, email, and the door to office.. | 20:20 | |
| mspreitz | radix: what would I write in a template to extract the URL to hit to trigger a scaling action? | 20:21 |
| mspreitz | therve: thanks for the pointer, I will study it when I get a chance | 20:21 |
| radix | mspreitz: you need the "AlarmUrl" attribute of the AWS::AutoScaling::ScalingPolicy resource | 20:24 |
| *** lnxnut has joined #heat | 20:35 | |
| *** daneyon has quit IRC | 20:37 | |
| *** rwsu has quit IRC | 20:40 | |
| *** lnxnut has quit IRC | 20:40 | |
| *** daveed has quit IRC | 21:13 | |
| *** daveed has joined #heat | 21:14 | |
| *** alexheneveld has joined #heat | 21:24 | |
| *** mkollaro has joined #heat | 21:30 | |
| *** wchrisj has joined #heat | 21:34 | |
| *** lnxnut has joined #heat | 21:35 | |
| *** kfox1111 has joined #heat | 21:39 | |
| *** lnxnut has quit IRC | 21:40 | |
| *** cody-somerville has joined #heat | 21:48 | |
| *** cody-somerville has quit IRC | 21:48 | |
| *** cody-somerville has joined #heat | 21:48 | |
| *** wchrisj has quit IRC | 21:54 | |
| *** e0ne has quit IRC | 22:24 | |
| *** e0ne has joined #heat | 22:25 | |
| *** lnxnut has joined #heat | 22:35 | |
| *** lnxnut has quit IRC | 22:40 | |
| *** cody-somerville has quit IRC | 22:45 | |
| openstackgerrit | A change was merged to openstack/heat: Change access to ceilometerclient https://review.openstack.org/72321 | 22:45 |
| *** alexpilotti has joined #heat | 22:55 | |
| *** fandi has quit IRC | 22:57 | |
| *** mspreitz has quit IRC | 23:08 | |
| *** yogesh has joined #heat | 23:21 | |
| *** lnxnut has joined #heat | 23:35 | |
| *** che-arne has quit IRC | 23:36 | |
| *** lnxnut has quit IRC | 23:40 | |
| *** che-arne has joined #heat | 23:49 | |
| *** sballe has joined #heat | 23:58 | |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/heat: WIP: Record resource actions in a journal https://review.openstack.org/77374 | 23:59 |
| SpamapS | alright, there is my spike | 23:59 |
| SpamapS | need to take a break for a while and try to do "Saturday" | 23:59 |
| *** yogesh has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!