| *** sdake has joined #heat | 00:03 | |
| *** tochi has quit IRC | 00:05 | |
| *** vijendar has joined #heat | 00:13 | |
| *** rajpatel has joined #heat | 00:15 | |
| *** tochi has joined #heat | 00:16 | |
| *** vijendar1 has joined #heat | 00:22 | |
| *** wangbo has quit IRC | 00:25 | |
| *** lamt has quit IRC | 00:25 | |
| *** lamt has joined #heat | 00:30 | |
| *** osoriano has quit IRC | 00:30 | |
| *** spzala has joined #heat | 00:32 | |
| *** vijendar has quit IRC | 00:36 | |
| *** spzala has quit IRC | 00:37 | |
| *** vijendar has joined #heat | 00:37 | |
| *** vijendar has quit IRC | 00:43 | |
| *** vijendar1 has quit IRC | 00:43 | |
| *** osoriano has joined #heat | 00:49 | |
| *** jcoufal has quit IRC | 00:50 | |
| *** rajpatel has quit IRC | 00:53 | |
| *** hoangcx has joined #heat | 00:54 | |
| *** bdeschenes has quit IRC | 00:55 | |
| openstackgerrit | Merged openstack/heat-templates: Correct tags option position in ansible hook https://review.openstack.org/397879 | 01:17 |
|---|---|---|
| *** yanyanhu has joined #heat | 01:23 | |
| *** yanyanhu has quit IRC | 01:30 | |
| *** zhurong has joined #heat | 01:30 | |
| *** yanyanhu has joined #heat | 01:33 | |
| *** yanyanhu has quit IRC | 01:40 | |
| openstackgerrit | huangtianhua proposed openstack/heat: Give correct example for translation_rules definition https://review.openstack.org/387763 | 01:55 |
| *** links has joined #heat | 01:56 | |
| *** lamt has quit IRC | 01:57 | |
| *** elynn has joined #heat | 01:58 | |
| *** vijendar has joined #heat | 02:04 | |
| *** vijendar_ has joined #heat | 02:05 | |
| *** links has quit IRC | 02:07 | |
| *** wangbo has joined #heat | 02:09 | |
| *** vijendar_ has quit IRC | 02:09 | |
| *** wangbo has quit IRC | 02:10 | |
| *** vijendar_ has joined #heat | 02:10 | |
| *** wangbo has joined #heat | 02:10 | |
| openstackgerrit | huangtianhua proposed openstack/heat: Refactor _needs_update() for resource https://review.openstack.org/389117 | 02:18 |
| *** elynn_ has joined #heat | 02:21 | |
| *** elynn has quit IRC | 02:24 | |
| *** EricGonc_ has quit IRC | 02:24 | |
| *** wangbo has quit IRC | 02:25 | |
| *** EricGonczer_ has joined #heat | 02:27 | |
| *** vijendar_ has quit IRC | 02:34 | |
| *** vijendar has quit IRC | 02:34 | |
| *** vijendar has joined #heat | 02:35 | |
| *** wangbo has joined #heat | 02:38 | |
| *** vijendar has quit IRC | 02:39 | |
| *** yanyanhu has joined #heat | 02:48 | |
| *** yuanying has quit IRC | 02:52 | |
| *** wangbo has quit IRC | 02:56 | |
| *** yanyanhu has quit IRC | 02:57 | |
| openstackgerrit | Merged openstack/heat-templates: docker-cmd hook https://review.openstack.org/393553 | 03:07 |
| *** wangbo has joined #heat | 03:08 | |
| *** steveg_afk has quit IRC | 03:10 | |
| *** vijendar has joined #heat | 03:15 | |
| *** spzala has joined #heat | 03:16 | |
| *** spzala has quit IRC | 03:20 | |
| *** EricGonczer_ has quit IRC | 03:20 | |
| ramishra | Morning all. Some reviews for https://review.openstack.org/#/q/status:open+project:openstack/python-heatclient+branch:master+topic:goal-remove-incubated-oslo-code and https://review.openstack.org/#/q/topic:remove_apiclient would help remove the oslo-incubator code. | 03:26 |
| openstackgerrit | Spencer Yu proposed openstack/heat: Remove unused pylintrc https://review.openstack.org/398066 | 03:26 |
| *** spzala has joined #heat | 03:30 | |
| *** ctayal has joined #heat | 03:32 | |
| *** vijendar1 has joined #heat | 03:33 | |
| *** sdake_ has joined #heat | 03:33 | |
| *** vijendar2 has joined #heat | 03:35 | |
| *** vijendar1 has quit IRC | 03:35 | |
| *** vijendar1 has joined #heat | 03:36 | |
| *** vijendar2 has quit IRC | 03:36 | |
| *** vijendar has quit IRC | 03:37 | |
| *** sdake has quit IRC | 03:37 | |
| *** vijendar1 has quit IRC | 03:38 | |
| *** Drago has quit IRC | 03:40 | |
| *** fragatin_ has joined #heat | 03:47 | |
| *** adrian_otto has joined #heat | 03:50 | |
| *** yuanying has joined #heat | 03:50 | |
| *** fragatina has quit IRC | 03:51 | |
| *** fragatin_ has quit IRC | 03:52 | |
| *** flwang1 has quit IRC | 04:00 | |
| *** spzala has quit IRC | 04:03 | |
| *** cwolferh has quit IRC | 04:04 | |
| *** spzala has joined #heat | 04:04 | |
| *** yanyanhu has joined #heat | 04:04 | |
| *** spzala has quit IRC | 04:05 | |
| *** KanagarajM has joined #heat | 04:08 | |
| openstackgerrit | Merged openstack/heat: Replace retrying with tenacity https://review.openstack.org/367638 | 04:14 |
| *** adrian_otto has quit IRC | 04:22 | |
| *** misty-cloud has joined #heat | 04:24 | |
| *** bks has joined #heat | 04:24 | |
| *** adrian_otto has joined #heat | 04:24 | |
| *** adrian_otto has quit IRC | 04:26 | |
| *** adrian_otto has joined #heat | 04:27 | |
| *** cwolferh has joined #heat | 04:33 | |
| *** elynn_ has quit IRC | 04:35 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/heat: Updated from global requirements https://review.openstack.org/397906 | 04:44 |
| *** adrian_otto has quit IRC | 04:47 | |
| *** adrian_otto has joined #heat | 04:49 | |
| openstackgerrit | Rabi Mishra proposed openstack/heat: Use apache config templates from local tree https://review.openstack.org/398098 | 04:53 |
| *** sdake_ is now known as sdake | 04:54 | |
| *** jprovazn has joined #heat | 05:07 | |
| *** ctayal has quit IRC | 05:08 | |
| *** sdake_ has joined #heat | 05:10 | |
| *** sdake has quit IRC | 05:14 | |
| *** wangbo has quit IRC | 05:23 | |
| *** sdake_ has quit IRC | 05:24 | |
| *** yanyanhu has quit IRC | 05:26 | |
| *** yanyanhu has joined #heat | 05:28 | |
| *** flwang1 has joined #heat | 05:29 | |
| *** gongysh has joined #heat | 05:32 | |
| *** yanyanhu has quit IRC | 05:34 | |
| *** wangbo has joined #heat | 05:37 | |
| *** yanyanhu has joined #heat | 05:38 | |
| *** yanyanhu has quit IRC | 05:38 | |
| *** elynn has joined #heat | 05:45 | |
| *** adrian_otto has quit IRC | 05:46 | |
| *** elynn_ has joined #heat | 05:48 | |
| *** bana_k has joined #heat | 05:48 | |
| *** elynn has quit IRC | 05:49 | |
| *** adrian_otto has joined #heat | 05:51 | |
| *** harlowja has quit IRC | 05:55 | |
| *** gongysh has quit IRC | 05:56 | |
| *** flwang1 has quit IRC | 05:58 | |
| *** adrian_otto has quit IRC | 05:58 | |
| *** elynn has joined #heat | 06:11 | |
| *** elynn_ has quit IRC | 06:13 | |
| *** jaosorior has joined #heat | 06:16 | |
| *** bana_k has quit IRC | 06:22 | |
| *** yanyanhu has joined #heat | 06:29 | |
| *** yanyanhu has quit IRC | 06:29 | |
| *** jprovazn has quit IRC | 06:32 | |
| *** harlowja_at_home has joined #heat | 06:32 | |
| *** yanyanhu has joined #heat | 06:33 | |
| *** harlowja_at_home has quit IRC | 06:33 | |
| *** vijendar has joined #heat | 06:39 | |
| *** vijendar has quit IRC | 06:40 | |
| *** jaosorior has quit IRC | 06:41 | |
| *** jaosorior has joined #heat | 06:42 | |
| *** yanyanhu has quit IRC | 06:44 | |
| *** cwolferh has quit IRC | 06:48 | |
| *** yanyanhu has joined #heat | 07:09 | |
| *** yanyanhu has quit IRC | 07:09 | |
| openstackgerrit | Merged openstack/heat: Updated from global requirements https://review.openstack.org/397906 | 07:20 |
| *** cwolferh has joined #heat | 07:32 | |
| *** jaosorior has quit IRC | 07:34 | |
| *** yprokule has joined #heat | 07:35 | |
| *** jprovazn has joined #heat | 07:35 | |
| ramishra | hi all, weekly irc meeting in 20 mins #openstack-meeting agenda: https://wiki.openstack.org/wiki/Meetings/HeatAgenda#Agenda_.282016-11-16_0800_UTC.29 | 07:40 |
| ramishra | please feel free to add to it. | 07:40 |
| jprovazn | ramishra: hi | 07:45 |
| ramishra | jprovazn: hi | 07:46 |
| jprovazn | ramishra: I see you are having mtg in few mins, one idea to discuss: any chance to have a fix/workaround for the concurrent db transaction error we discussed earlier? | 07:46 |
| ramishra | jprovazn: I think most of us aware of the issue(we discussed during the summit). However, it would be better if you can report a bug with the logs. | 07:48 |
| jprovazn | ramishra: ok, will do | 07:49 |
| ramishra | jprovazn: you can add the topic and join the meeting, if you want. | 07:49 |
| jprovazn | ramishra: ack, I don't have the backtrace right now, I'll see if I can reproduce before the mtg, could you please repaste me the link to the patch which bumps number of retries? | 07:50 |
| ramishra | jprovazn: hmm. you mean this one https://review.openstack.org/#/c/321783/. | 07:52 |
| jprovazn | thanks | 07:52 |
| ramishra | this does not increase the retries but fixes the retry delay. | 07:53 |
| *** jaosorior has joined #heat | 07:54 | |
| *** shardy has joined #heat | 07:56 | |
| *** ricolin has joined #heat | 08:00 | |
| ramishra | therve, stevebaker, skraynev, shardy, tiantian meeting? | 08:01 |
| ramishra | KanagarajM ^^ | 08:01 |
| *** dbecker has joined #heat | 08:05 | |
| *** wangbo has quit IRC | 08:07 | |
| *** ccamacho has joined #heat | 08:07 | |
| *** wangbo has joined #heat | 08:08 | |
| *** amoralej|off is now known as amoralej | 08:20 | |
| *** wangbo has quit IRC | 08:22 | |
| jprovazn | ramishra: what channel are you meeting at? | 08:24 |
| jprovazn | ramishra: well, I created bug https://bugs.launchpad.net/heat/+bug/1642166 , you mentioned you all areaware of this issue, so probably no reason to raise it on the mtg? | 08:25 |
| openstack | Launchpad bug 1642166 in heat "Concurrent DB transaction when deleting resources" [Undecided,New] | 08:25 |
| openstackgerrit | Rico Lin proposed openstack/heat-specs: Implement Placeholder https://review.openstack.org/392499 | 08:26 |
| ricolin | jprovazn: #openstack-meeting this week | 08:28 |
| *** jpich has joined #heat | 08:31 | |
| *** wangbo has joined #heat | 08:34 | |
| skraynev | ramishra: yeah. thank you for the reminder. zaneb asked about it. I wanted to dig this issue. | 08:44 |
| skraynev | because looks like retry with 2 ms works better then retry with range in 2 s | 08:44 |
| jprovazn | ricolin: getting http://paste.openstack.org/show/589374/ | 08:46 |
| skraynev | ramishra; also looks like this fix in case of convernegence does not work as expected. | 08:46 |
| ricolin | jprovazn: oh, looks like some dummy failure | 08:47 |
| ramishra | skraynev: yeah, that's the reason we tweaked the tenacity patch for it to merge. | 08:47 |
| ramishra | skraynev: https://review.openstack.org/#/c/367638/8/heat/objects/resource.py | 08:48 |
| ricolin | jprovazn: I will update a new one for you | 08:48 |
| jprovazn | ricolin: thanks | 08:48 |
| ramishra | skraynev: we changed it to max=0.002 as it uses seconds not milliseconds | 08:48 |
| skraynev | yes. I see. However looks like person who reported bug, has not last mentioned patch with tenacity | 08:49 |
| therve | jprovazn, Do you have the original traceback outside of launchpad? | 08:50 |
| therve | It messes it up pretty badly | 08:50 |
| jprovazn | therve: yes, sec | 08:50 |
| openstackgerrit | Merged openstack/heat: Don't try to delete trust if trustor not found https://review.openstack.org/388561 | 08:51 |
| skraynev | ramishra: it's really strange why retrying lib works not so good as tenacity | 08:51 |
| jprovazn | therve: http://paste.openstack.org/show/589377/ | 08:51 |
| skraynev | ok. I read the commit message.. retrying is just outdated and tenacity is a fork | 08:52 |
| therve | jprovazn, Thanks | 08:52 |
| ramishra | skraynev: I don't think it's an issue with retrying library other than it uses milliseconds. | 08:52 |
| jprovazn | therve: hm, this one might not be the original one though, me checks for older one | 08:52 |
| ramishra | the reason retrying is replaced with tenacity is, retrying is not maintained any more. | 08:53 |
| jprovazn | therve: this is the original - http://paste.openstack.org/show/589378/ | 08:54 |
| therve | jprovazn, So I think I understand the contention. All nodes try to push a change to the single bastion server | 08:55 |
| jprovazn | therve: yes | 08:55 |
| therve | jprovazn, Even scale down failed? like a few node scale down? | 08:55 |
| jprovazn | therve: this works fine on small stacks (e.g. 4-5 nodes), it occurs on larger scale | 08:56 |
| ramishra | therve: it uses delete software config, so changes are pushed during scaledown. | 08:56 |
| openstackgerrit | Sergey Kraynev proposed openstack/heat: Use intended retry delays https://review.openstack.org/321783 | 08:56 |
| therve | Right | 08:56 |
| therve | It feels like a flaw in the design | 08:57 |
| skraynev | ramishra: ok . I have rebased patch from zaneb. To increase it to 2 sec. Let's wait test results, before I will enjoy debugging this issue :) | 08:57 |
| ramishra | skraynev: it failed with 2 secs, hence we modified it to 0.002 | 08:58 |
| jprovazn | therve: the reason it's done on single server is that both scale up and scale down is handled by ansible | 08:58 |
| skraynev | ramishra: hm... and I suppose, that it failed for convergence only. | 08:59 |
| therve | jprovazn, Yes, I see that. But the way it's done now, it's bound to have contention | 08:59 |
| therve | Whereas there should be a way to batch those, in theory | 08:59 |
| *** zzzeek has quit IRC | 09:00 | |
| skraynev | ramishra: aha. yes. it's | 09:00 |
| skraynev | ramishra: honestly I have not tested it on convergence, when added it, because... it was before convergence :) | 09:01 |
| jprovazn | therve: AutoScalingGroup is used for the scaled nodes, there is now: rolling_updates: {min_in_service: 0, pause_time: 0, max_batch_size: 50} | 09:01 |
| therve | jprovazn, I guess diminishing the batch size could alleviate the issue | 09:01 |
| jprovazn | therve: it might be this batch size is applied on scale down too, but I can not set it to 2-3 because then scale up is extra slow | 09:02 |
| * therve nods | 09:02 | |
| therve | Probably doesn't help for delete too | 09:03 |
| *** zzzeek has joined #heat | 09:03 | |
| therve | jprovazn, Could ansible run on the node instead of the master? | 09:04 |
| jprovazn | therve: negative | 09:07 |
| ricolin | jprovazn: weird, `wait_jitter_max` should be accepted by retrying since 2014 | 09:07 |
| jprovazn | ricolin: I have this version: python-retrying-1.2.3-4.el7ost.noarch | 09:07 |
| jprovazn | ricolin: briefly checked retrying.py but don't see a jitter param there | 09:08 |
| jprovazn | therve: only workaround I could think of to avoid writing metadata for single node was that each host would have to ssh to the bastion host and execute an acction, but this would be a potential security issue of providing access of "compute" ndoes to the management node | 09:10 |
| therve | jprovazn, Yeah that makes sense | 09:10 |
| therve | It's kind of bad that we have contention here though | 09:11 |
| therve | There is no good reason for it | 09:11 |
| *** yanyanhu has joined #heat | 09:14 | |
| *** tochi has quit IRC | 09:15 | |
| ricolin | jprovazn: 1.2.3 is builded two years ago, so my patch might not be a good fit for you(need at least 1.3.2) | 09:19 |
| ricolin | jprovazn: could you kindly tell me your python-tenacity version, maybe I can change that patch with it | 09:20 |
| jprovazn | ricolin: I'm afraid I don't have python-tenacity avail at all | 09:22 |
| jprovazn | ricolin: this is RHEL 7.3 FWIW | 09:22 |
| ricolin | jprovazn: Got it | 09:24 |
| jprovazn | ricolin: lame question, would it make sense if I tried your patch but instead of: | 09:24 |
| jprovazn | wait_jitter_max=wait_jitter_max, | 09:24 |
| jprovazn | used: | 09:24 |
| *** bdeschenes has joined #heat | 09:24 | |
| jprovazn | wait_random_max=2000 | 09:24 |
| *** tochi has joined #heat | 09:25 | |
| ricolin | the jitter is about keep increase with index time | 09:25 |
| ricolin | like x*2^n (n=1,.....) | 09:25 |
| *** yassine has joined #heat | 09:26 | |
| jprovazn | ack | 09:26 |
| *** yassine is now known as Guest45910 | 09:26 | |
| ricolin | using wait_random is the way in zane's patch which you already tried | 09:26 |
| ricolin | jprovazn: there is another dummy way to test this algorithm if you're intrested | 09:28 |
| jprovazn | ricolin: sure | 09:28 |
| ricolin | use a while loop function to replace retrying and implement index logic inside | 09:29 |
| ricolin | or find way to replace retrying.py in python package to use the one in 1.3.3 | 09:29 |
| *** derekh has joined #heat | 09:41 | |
| *** sorantis has joined #heat | 09:44 | |
| *** zhurong has quit IRC | 10:02 | |
| *** elynn has quit IRC | 10:03 | |
| *** dbecker has quit IRC | 10:04 | |
| openstackgerrit | Thomas Herve proposed openstack/heat: Fix get all attributes with cache data https://review.openstack.org/398240 | 10:06 |
| *** wangbo has quit IRC | 10:13 | |
| *** hoangcx has quit IRC | 10:16 | |
| *** yanyanhu has quit IRC | 10:17 | |
| *** yanyanhu has joined #heat | 10:17 | |
| *** yanyanhu has quit IRC | 10:18 | |
| *** liusheng has quit IRC | 10:29 | |
| *** liusheng has joined #heat | 10:30 | |
| *** wangbo has joined #heat | 10:32 | |
| *** bdeschenes has quit IRC | 10:37 | |
| *** dbecker has joined #heat | 10:50 | |
| *** misty-cloud has quit IRC | 10:58 | |
| *** bks has quit IRC | 11:04 | |
| *** sorantis has quit IRC | 11:04 | |
| *** jaosorior is now known as jaosorior_lunch | 11:25 | |
| *** sorantis has joined #heat | 11:25 | |
| *** sorantis_ has joined #heat | 11:30 | |
| *** sorantis has quit IRC | 11:32 | |
| *** sorantis_ is now known as sorantis | 11:32 | |
| openstackgerrit | Thomas Herve proposed openstack/heat: Fix get all attributes with cache data https://review.openstack.org/398240 | 11:35 |
| *** gongysh has joined #heat | 11:36 | |
| *** gongysh has quit IRC | 11:45 | |
| *** elynn has joined #heat | 11:59 | |
| *** om2 has quit IRC | 12:00 | |
| *** dprince has joined #heat | 12:01 | |
| *** elynn_ has joined #heat | 12:03 | |
| *** vgridnev has joined #heat | 12:03 | |
| *** elynn has quit IRC | 12:04 | |
| *** om2 has joined #heat | 12:08 | |
| *** jaosorior_lunch is now known as jaosorior | 12:09 | |
| *** shardy is now known as shardy_lunch | 12:11 | |
| *** psanchez has quit IRC | 12:13 | |
| *** vgridnev has left #heat | 12:14 | |
| *** dprince has quit IRC | 12:15 | |
| *** rook-out has quit IRC | 12:15 | |
| *** psanchez has joined #heat | 12:15 | |
| *** dprince has joined #heat | 12:16 | |
| *** elynn_ has quit IRC | 12:17 | |
| *** rook has joined #heat | 12:18 | |
| *** rook is now known as Guest45665 | 12:18 | |
| *** steveg_afk has joined #heat | 12:19 | |
| *** catintheroof has joined #heat | 12:23 | |
| *** EricGonczer_ has joined #heat | 12:35 | |
| *** zhurong has joined #heat | 12:35 | |
| *** EricGonczer_ has quit IRC | 12:37 | |
| *** shardy_lunch is now known as shardy | 12:45 | |
| *** sdake has joined #heat | 12:53 | |
| *** pradk has joined #heat | 12:55 | |
| *** vgridnev has joined #heat | 13:02 | |
| *** ccamacho is now known as ccamacho|lunch | 13:04 | |
| *** sdake_ has joined #heat | 13:05 | |
| *** coreyob has quit IRC | 13:06 | |
| *** EricGonczer_ has joined #heat | 13:08 | |
| *** EricGonczer_ has quit IRC | 13:08 | |
| *** sdake has quit IRC | 13:08 | |
| *** elynn_ has joined #heat | 13:11 | |
| *** lamt has joined #heat | 13:23 | |
| *** jamielennox is now known as jamielennox|away | 13:25 | |
| *** coreyob has joined #heat | 13:26 | |
| *** coreyob has quit IRC | 13:48 | |
| *** KanagarajM has quit IRC | 13:49 | |
| *** amoralej is now known as amoralej|lunch | 13:52 | |
| *** eliqiao has quit IRC | 13:56 | |
| *** eliqiao has joined #heat | 13:57 | |
| *** coreyob has joined #heat | 13:59 | |
| *** coreyob has quit IRC | 14:03 | |
| *** ctayal has joined #heat | 14:08 | |
| *** Guest45665 is now known as rook | 14:13 | |
| *** jcoufal has joined #heat | 14:14 | |
| *** coreyob has joined #heat | 14:15 | |
| *** jprovazn has quit IRC | 14:20 | |
| *** vgridnev has quit IRC | 14:22 | |
| *** lamt has quit IRC | 14:23 | |
| *** ccamacho|lunch is now known as ccamacho | 14:28 | |
| *** Drago1 has joined #heat | 14:28 | |
| *** amoralej|lunch is now known as amoralej | 14:32 | |
| *** Drago1 has quit IRC | 14:33 | |
| openstackgerrit | Rico Lin proposed openstack/heat-specs: Add snapshot ability to stack https://review.openstack.org/243015 | 14:38 |
| *** elynn_ has quit IRC | 14:38 | |
| therve | zaneb, Can we move forward with https://review.openstack.org/#/c/388188/ ? | 14:40 |
| therve | It's not perfect, but it fixes the bug for now | 14:40 |
| *** jaosorior has quit IRC | 14:41 | |
| *** lamt has joined #heat | 14:41 | |
| *** jaosorior has joined #heat | 14:42 | |
| *** coreyob has quit IRC | 14:44 | |
| *** Drago1 has joined #heat | 14:45 | |
| *** chris_hultin|AWA is now known as chris_hultin | 14:45 | |
| *** jaosorior has quit IRC | 14:47 | |
| *** elynn_ has joined #heat | 14:50 | |
| *** zhurong has quit IRC | 14:51 | |
| *** sabeen has joined #heat | 14:52 | |
| *** coreyob has joined #heat | 14:55 | |
| *** yprokule_ has joined #heat | 14:57 | |
| *** adrian_otto has joined #heat | 14:57 | |
| *** ricolin has quit IRC | 14:58 | |
| *** yprokule has quit IRC | 14:58 | |
| *** yprokule_ is now known as yprokule | 14:58 | |
| *** sdake has joined #heat | 14:58 | |
| *** jcoufal has quit IRC | 14:59 | |
| Drago1 | shardy: ping | 14:59 |
| *** jcoufal has joined #heat | 15:01 | |
| *** sdake_ has quit IRC | 15:02 | |
| *** coreyob has quit IRC | 15:02 | |
| shardy | Drago1: Hi! | 15:04 |
| *** coreyob has joined #heat | 15:05 | |
| Drago1 | shardy: Hey | 15:05 |
| Drago1 | shardy: I have some questions about db migrations with rolling upgrades in Heat | 15:06 |
| *** adrian_otto has quit IRC | 15:06 | |
| *** spotz_zzz is now known as spotz | 15:06 | |
| openstackgerrit | Rico Lin proposed openstack/python-heatclient: Add `--no-validate` in heat client https://review.openstack.org/397200 | 15:07 |
| openstackgerrit | Rico Lin proposed openstack/python-heatclient: Add `--no-validate` in osc stack https://review.openstack.org/397528 | 15:07 |
| shardy | Drago1: sure, I can try to help, but I may not be the best contact tbh - not been working on that myself | 15:07 |
| * shardy been too busy with TripleO things | 15:07 | |
| Drago1 | shardy: Do you know who would? Or what libraries besides ovo help with it? | 15:08 |
| shardy | therve, zaneb: ^^ who's most familiar with this? | 15:08 |
| therve | Hum :) | 15:09 |
| therve | Drago1, Maybe I can help | 15:09 |
| zaneb | I think we're all equally clueless ;) | 15:09 |
| shardy | lol | 15:09 |
| Drago1 | Awesome, zaneb | 15:09 |
| therve | Yeah that'd be the consensus | 15:09 |
| Drago1 | So I saw a talk in Austin that laid out a method for doing DB migrations while still allowing services at different versions talk to the DB | 15:10 |
| zaneb | Drago1: hit us with the questions. we'll make something up | 15:10 |
| Drago1 | It was basically: Let the old version be N. N+1 writes to old and new, reads from old. N+2 writes to old and new, reads from new. N+3 can write and read to only new. | 15:11 |
| zaneb | Drago1: was it the triggers thing? | 15:11 |
| zaneb | actually, cwolferh knows quite a bit about this | 15:11 |
| zaneb | he is in Pacific tz | 15:12 |
| Drago1 | I don't remember anything about triggers, but that could be my memory | 15:12 |
| therve | Drago1, THat's one of the strategy. ovo doesn't really help for that | 15:13 |
| zaneb | ah, ok | 15:13 |
| *** coreyob has quit IRC | 15:14 | |
| zaneb | I think triggers was the proposed solution to it taking 2 years to change a db schema | 15:14 |
| Drago1 | Yeah, if we're talking about each version being in one cycle, then it's a long process to migrate over. | 15:14 |
| zaneb | although there are other ways to compress that | 15:14 |
| Drago1 | I was curious about what other ways are out there | 15:14 |
| *** jasond has joined #heat | 15:14 | |
| zaneb | config options would be one | 15:15 |
| strigazi | I'm here too | 15:15 |
| zaneb | in theory ovo could help with that, but in practice I haven't been able to figure out how it works | 15:16 |
| Drago1 | zaneb: Meaning the operator sets an option for what version they want to upgrade to? | 15:16 |
| Drago1 | Or what | 15:16 |
| zaneb | so for RPC you can set a version cap and it uses older versioned objects until you remove the cap... | 15:17 |
| strigazi | zaneb, heat does this? | 15:17 |
| zaneb | if you could do the same thing with db versioned objects then you could compress the N+2 and N+3 into a single release iirc | 15:17 |
| zaneb | strigazi: no | 15:17 |
| zaneb | strigazi: it's not clear to me that this even _can_ work with heat. ovo seems to be designed around Nova's particular architecture and is not much applicable to what we're doing | 15:19 |
| Drago1 | zaneb: Yeah, if the whole migration could be done in a single release, it'd be great | 15:19 |
| strigazi | it is doable to do the all migrations in one go, how does this help? | 15:20 |
| strigazi | we still need to version the db api | 15:21 |
| therve | strigazi, Why? | 15:21 |
| *** bana_k has joined #heat | 15:22 | |
| strigazi | therve, how the a conductor will talk to a db in a newer release? | 15:22 |
| therve | strigazi, I don't want to know about nova :) | 15:22 |
| therve | If you're talking about the engine, that's relatively easy: don't make breaking changes in the schema | 15:23 |
| Drago1 | strigazi: single release != migrating all in one go. To me it sounds like the upgrade is done in multiple rounds | 15:24 |
| Drago1 | *For the scenario that zaneb was talking about | 15:24 |
| strigazi | In this cycle with cluster_template versions we'll break the schema | 15:25 |
| *** coreyob has joined #heat | 15:25 | |
| zaneb | for the record, I said compress N+2 & N+3 into just N+2 | 15:26 |
| therve | I don't know what that is | 15:26 |
| Drago1 | strigazi: Will we? | 15:26 |
| strigazi | Drago1, the new cluster_template verions will be significantly reduced in number of fields | 15:27 |
| zaneb | strigazi: I feel like we're missing some context here. maybe you should start at the beginning? because you keep bringing up stuff that is nothing to do with heat afaik | 15:27 |
| strigazi | true | 15:28 |
| *** EricGonczer_ has joined #heat | 15:28 | |
| Drago1 | zaneb: We are looking at Magnum problems by trying to understand what Heat does :) | 15:28 |
| zaneb | e.g. we don't have a conductor and nobody here has any idea what a cluster_template is | 15:28 |
| zaneb | ok, that helps a lot :) | 15:28 |
| Drago1 | zaneb, strigazi: We want to move fields from table A to table B | 15:29 |
| strigazi | it's better to dicsuss it in our chanell i guess. Heat is not doing what we are looking for | 15:29 |
| zaneb | so, short answer, we haven't solved the problem either | 15:29 |
| zaneb | heat still requires all engines to be upgraded in step | 15:29 |
| zaneb | which effectively means downtime | 15:29 |
| Drago1 | zaneb: So Heat's grenade gate is upgrade without rolling. I've been thinking it's rolling but I guess that's only nova and neutron? | 15:30 |
| *** jprovazn has joined #heat | 15:30 | |
| *** EricGonc_ has joined #heat | 15:30 | |
| zaneb | the "[heat] Rolling Upgrades" thread on the ML and https://etherpad.openstack.org/p/heat-ocata-hitless-upgrades would be the best source of info we do have | 15:31 |
| zaneb | afaik grenade never does rolling upgrades for any project, but I might be out of date | 15:32 |
| *** EricGonczer_ has quit IRC | 15:32 | |
| *** annegentle has joined #heat | 15:33 | |
| Drago1 | zaneb: That is an incredible subject line | 15:33 |
| Drago1 | "We converted all of our DB interfaces to use oslo.versionedobjects to no effect whatsoever except to add a pointless and poorly-designed layer of indirection" | 15:33 |
| zaneb | it seemed like such a good idea at the time | 15:34 |
| Drago1 | zaneb, therve, shardy: Thanks for the help! My understanding of upgrades in Heat was wrong :) | 15:39 |
| strigazi | Thanks for your time guys | 15:39 |
| *** spzala has joined #heat | 15:39 | |
| zaneb | sorry to disappoint ;) if you figure it out let us know too! | 15:40 |
| *** sorantis has quit IRC | 15:43 | |
| Drago1 | Will do, you too | 15:44 |
| *** annegentle has quit IRC | 15:45 | |
| *** annegentle has joined #heat | 16:11 | |
| *** rajpatel has joined #heat | 16:15 | |
| *** ramishra has quit IRC | 16:19 | |
| *** EricGonc_ has quit IRC | 16:20 | |
| *** ramishra has joined #heat | 16:20 | |
| *** EricGonczer_ has joined #heat | 16:23 | |
| *** adrian_otto has joined #heat | 16:26 | |
| openstackgerrit | Thomas Herve proposed openstack/heat: Remove extra resource definition loading https://review.openstack.org/398476 | 16:35 |
| therve | zaneb, ^^^ | 16:35 |
| *** yprokule has quit IRC | 16:39 | |
| *** strigazi is now known as strigazi_AFK | 16:46 | |
| *** jistr is now known as jistr|biab | 16:48 | |
| *** rajpatel has quit IRC | 16:51 | |
| *** Zer0Byte__ has joined #heat | 17:01 | |
| *** ctayal has quit IRC | 17:02 | |
| *** dbecker has quit IRC | 17:04 | |
| *** ctayal has joined #heat | 17:04 | |
| *** sdake has quit IRC | 17:06 | |
| *** sdake has joined #heat | 17:09 | |
| cwolferh | looking for thoughts https://review.openstack.org/#/c/361570/ . all easy solutions, just need to pick one. | 17:19 |
| *** rajpatel has joined #heat | 17:27 | |
| *** jistr|biab is now known as jistr | 17:36 | |
| *** EricGonczer_ has quit IRC | 17:37 | |
| *** rajpatel has quit IRC | 17:37 | |
| *** fragatina has joined #heat | 17:41 | |
| *** derekh has quit IRC | 17:47 | |
| *** david-lyle has quit IRC | 17:48 | |
| *** david-lyle has joined #heat | 17:48 | |
| *** chris_hultin is now known as chris_hultin|AWA | 17:51 | |
| *** coreyob has quit IRC | 17:57 | |
| *** randallburt has joined #heat | 17:58 | |
| *** randallburt has quit IRC | 18:02 | |
| *** fragatin_ has joined #heat | 18:04 | |
| *** coreyob has joined #heat | 18:05 | |
| *** fragatina has quit IRC | 18:08 | |
| *** coreyob has quit IRC | 18:10 | |
| *** randallburt has joined #heat | 18:12 | |
| *** catinthe_ has joined #heat | 18:13 | |
| *** harlowja has joined #heat | 18:14 | |
| *** catintheroof has quit IRC | 18:15 | |
| *** catintheroof has joined #heat | 18:15 | |
| *** randallburt has quit IRC | 18:17 | |
| *** catinthe_ has quit IRC | 18:18 | |
| *** dbecker has joined #heat | 18:20 | |
| *** coreyob has joined #heat | 18:20 | |
| *** coreyob has quit IRC | 18:25 | |
| *** bana_k has quit IRC | 18:27 | |
| *** bana_k has joined #heat | 18:27 | |
| *** randallburt has joined #heat | 18:29 | |
| *** EricGonczer_ has joined #heat | 18:31 | |
| *** dbecker has quit IRC | 18:32 | |
| *** coreyob has joined #heat | 18:36 | |
| *** jpich has quit IRC | 18:36 | |
| *** coreyob has quit IRC | 18:40 | |
| *** ctayal has quit IRC | 18:45 | |
| *** vijendar has joined #heat | 18:50 | |
| *** vijendar1 has joined #heat | 18:51 | |
| *** coreyob has joined #heat | 18:51 | |
| *** chlong has joined #heat | 18:53 | |
| *** coreyob has quit IRC | 18:56 | |
| *** vijendar has quit IRC | 18:59 | |
| *** vijendar has joined #heat | 19:00 | |
| *** evilt0ne_ has joined #heat | 19:00 | |
| *** jcoufal has quit IRC | 19:01 | |
| *** shardy is now known as shardy_afk | 19:01 | |
| *** rajpatel has joined #heat | 19:03 | |
| *** evilt0ne has quit IRC | 19:04 | |
| *** coreyob has joined #heat | 19:06 | |
| *** chris_hultin|AWA is now known as chris_hultin | 19:10 | |
| *** amoralej is now known as amoralej|off | 19:12 | |
| *** spotz is now known as spotz_zzz | 19:15 | |
| *** chris_hultin is now known as chris_hultin|AWA | 19:16 | |
| *** shardy_afk is now known as shardy | 19:16 | |
| *** jcoufal has joined #heat | 19:16 | |
| *** shardy is now known as shardy_afk | 19:17 | |
| *** chris_hultin|AWA is now known as chris_hultin | 19:17 | |
| *** flwang1 has joined #heat | 19:22 | |
| *** rajpatel has quit IRC | 19:24 | |
| *** jamielennox|away is now known as jamielennox | 19:26 | |
| *** flwang1 has quit IRC | 19:33 | |
| *** fragatin_ has quit IRC | 19:41 | |
| *** fragatina has joined #heat | 19:41 | |
| *** rajpatel has joined #heat | 19:45 | |
| *** coreyob has quit IRC | 19:46 | |
| *** EricGonczer_ has quit IRC | 19:48 | |
| *** EricGonczer_ has joined #heat | 19:54 | |
| *** strobelight has joined #heat | 19:57 | |
| *** coreyob has joined #heat | 19:58 | |
| *** blomquisg_gone is now known as blomquisg | 20:00 | |
| *** vijendar has quit IRC | 20:02 | |
| *** coreyob has quit IRC | 20:02 | |
| *** jcoufal has quit IRC | 20:05 | |
| *** vijendar has joined #heat | 20:11 | |
| *** jcoufal has joined #heat | 20:12 | |
| *** strobelight has quit IRC | 20:14 | |
| *** strobelight has joined #heat | 20:14 | |
| *** coreyob has joined #heat | 20:15 | |
| *** annegentle has quit IRC | 20:15 | |
| *** strobelight has quit IRC | 20:17 | |
| *** annegentle has joined #heat | 20:17 | |
| *** blomquisg is now known as blomquisg_gone | 20:28 | |
| *** vijendar has quit IRC | 20:28 | |
| *** vijendar1 has quit IRC | 20:28 | |
| *** sabeen has quit IRC | 20:28 | |
| *** evilt0ne has joined #heat | 20:29 | |
| *** flwang1 has joined #heat | 20:30 | |
| *** spotz_zzz is now known as spotz | 20:31 | |
| *** sdake has quit IRC | 20:35 | |
| *** strobelight has joined #heat | 20:36 | |
| *** sdake has joined #heat | 20:36 | |
| *** coreyob has quit IRC | 20:38 | |
| *** sabeen has joined #heat | 20:38 | |
| *** evilt0ne_ has quit IRC | 20:49 | |
| *** coreyob has joined #heat | 20:51 | |
| *** vijendar has joined #heat | 20:54 | |
| *** vijendar1 has joined #heat | 20:57 | |
| *** rcernin has joined #heat | 20:58 | |
| *** vijendar has quit IRC | 21:00 | |
| *** coreyob has quit IRC | 21:00 | |
| *** rajpatel has quit IRC | 21:05 | |
| *** coreyob has joined #heat | 21:05 | |
| *** vijendar has joined #heat | 21:05 | |
| *** rajpatel has joined #heat | 21:06 | |
| *** coreyob has quit IRC | 21:08 | |
| *** Guest45910 has quit IRC | 21:11 | |
| *** dprince has quit IRC | 21:13 | |
| *** coreyob has joined #heat | 21:20 | |
| *** rajpatel has quit IRC | 21:22 | |
| *** coreyob has quit IRC | 21:23 | |
| *** rajpatel has joined #heat | 21:25 | |
| *** vijendar1 has quit IRC | 21:26 | |
| *** vijendar has quit IRC | 21:26 | |
| *** rajpatel has quit IRC | 21:29 | |
| *** EricGonczer_ has quit IRC | 21:31 | |
| *** catinthe_ has joined #heat | 21:32 | |
| *** Drago1 is now known as Drago | 21:34 | |
| *** coreyob has joined #heat | 21:35 | |
| *** catintheroof has quit IRC | 21:36 | |
| *** coreyob has quit IRC | 21:38 | |
| *** rajpatel has joined #heat | 21:39 | |
| *** sdake_ has joined #heat | 21:44 | |
| *** sdake has quit IRC | 21:47 | |
| *** catintheroof has joined #heat | 21:51 | |
| *** catinthe_ has quit IRC | 21:51 | |
| *** coreyob has joined #heat | 21:51 | |
| *** ayoung has quit IRC | 21:55 | |
| *** EricGonczer_ has joined #heat | 21:57 | |
| openstackgerrit | Julian proposed openstack/heat-specs: Heat Resource Type Versions https://review.openstack.org/398601 | 22:00 |
| *** EricGonczer_ has quit IRC | 22:01 | |
| openstackgerrit | Crag Wolfe proposed openstack/heat: Refactor purge_deleted, operate on batches of stacks https://review.openstack.org/361570 | 22:03 |
| *** coreyob has quit IRC | 22:04 | |
| *** rajpatel has quit IRC | 22:05 | |
| *** rajpatel has joined #heat | 22:06 | |
| *** adrian_otto has quit IRC | 22:08 | |
| *** lifeless has quit IRC | 22:11 | |
| *** lifeless has joined #heat | 22:12 | |
| *** annegentle has quit IRC | 22:14 | |
| *** vijendar has joined #heat | 22:16 | |
| *** jprovazn has quit IRC | 22:17 | |
| *** vijendar1 has joined #heat | 22:20 | |
| *** vijendar has quit IRC | 22:21 | |
| *** vijendar has joined #heat | 22:28 | |
| openstackgerrit | Tin Lam proposed openstack/heat-specs: Heat Resource Type Versions https://review.openstack.org/398601 | 22:29 |
| *** jcoufal has quit IRC | 22:34 | |
| *** coreyob has joined #heat | 22:35 | |
| *** adrian_otto has joined #heat | 22:35 | |
| *** rajpatel has quit IRC | 22:40 | |
| *** coreyob has quit IRC | 22:41 | |
| *** vijendar has quit IRC | 22:43 | |
| *** steveg_afk has quit IRC | 22:44 | |
| *** rajpatel has joined #heat | 22:44 | |
| *** ayoung has joined #heat | 22:45 | |
| *** pradk has quit IRC | 22:48 | |
| *** coreyob has joined #heat | 22:50 | |
| *** vijendar has joined #heat | 22:50 | |
| *** catintheroof has quit IRC | 22:54 | |
| *** jasond has quit IRC | 22:55 | |
| *** vijendar1 has quit IRC | 22:55 | |
| *** vijendar has quit IRC | 22:55 | |
| *** steveg_afk has joined #heat | 22:59 | |
| *** chlong has quit IRC | 22:59 | |
| *** spzala has quit IRC | 23:03 | |
| *** sdake has joined #heat | 23:06 | |
| *** sdake_ has quit IRC | 23:10 | |
| *** annegentle has joined #heat | 23:15 | |
| *** randallburt has quit IRC | 23:16 | |
| *** sdake has quit IRC | 23:16 | |
| *** chlong has joined #heat | 23:17 | |
| *** dfflanders has joined #heat | 23:25 | |
| *** chlong has quit IRC | 23:27 | |
| *** chris_hultin is now known as chris_hultin|AWA | 23:29 | |
| *** bana_k has quit IRC | 23:31 | |
| *** lamt has quit IRC | 23:33 | |
| *** sdake has joined #heat | 23:41 | |
| *** david-lyle_ has joined #heat | 23:50 | |
| *** david-lyle has quit IRC | 23:51 | |
| *** rcernin has quit IRC | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!