| SlickNik | glucas: I found out what the issue was. | 00:05 |
|---|---|---|
| SlickNik | glucas: I was running into another timing condition. | 00:06 |
| SlickNik | glucas: Basically the timeout for instance-create (400s) was not high enough for the restore to happen successfully in my environment | 00:06 |
| glucas | Oh? | 00:07 |
| SlickNik | glucas: Which meant that the attach_replication_slave message was being sent to the guest too early (before the prepare call had had a chance to finish). | 00:07 |
| glucas | SlickNik: ah, ok | 00:08 |
| SlickNik | And since prepare hadn't finished, the access for os_admin hadn't been set up correctly yet | 00:08 |
| glucas | SlickNik: Right - that makes sense. | 00:08 |
| glucas | SlickNik: So for now is increasing timeouts sufficient? | 00:09 |
| *** nosnos has joined #openstack-trove | 00:11 | |
| *** julienvey has joined #openstack-trove | 00:18 | |
| openstackgerrit | Auston McReynolds proposed a change to openstack/trove: Clusters API Implementation https://review.openstack.org/113675 | 00:21 |
| SlickNik | glucas: it probably makes sense to have a different (higher) timeout for a create based on a backup (i.e. the restore scenario). | 00:22 |
| SlickNik | I can't think of many real DBs you'd be able to restore within 5 mins. | 00:23 |
| *** julienvey has quit IRC | 00:23 | |
| SlickNik | (given that the prepare call itself also has some overhead) | 00:23 |
| glucas | Yeah. | 00:23 |
| *** eguz has joined #openstack-trove | 00:24 | |
| *** eguz has quit IRC | 00:24 | |
| SlickNik | https://bugs.launchpad.net/trove/+bug/1356645 filed | 00:26 |
| *** johnma has quit IRC | 00:26 | |
| glucas | Cool. And thanks for the follow up. | 00:27 |
| *** eghobo has quit IRC | 00:28 | |
| glucas | Creating a slave may need even more time since it has to both backup and restore. | 00:30 |
| *** boblebauce has quit IRC | 00:36 | |
| *** boblebauce has joined #openstack-trove | 00:36 | |
| *** boblebauce has joined #openstack-trove | 00:37 | |
| *** boblebauce has quit IRC | 00:38 | |
| *** boblebauce has joined #openstack-trove | 00:39 | |
| *** boblebauce has joined #openstack-trove | 00:39 | |
| *** boblebauce has joined #openstack-trove | 00:39 | |
| *** boblebauce has joined #openstack-trove | 00:40 | |
| *** boblebauce has joined #openstack-trove | 00:40 | |
| *** boblebauce has joined #openstack-trove | 00:41 | |
| *** rwsu has quit IRC | 00:42 | |
| *** achampion has joined #openstack-trove | 00:44 | |
| *** nosnos has quit IRC | 00:46 | |
| openstackgerrit | Auston McReynolds proposed a change to openstack/trove: Clusters TaskManager Implementation https://review.openstack.org/113692 | 00:46 |
| *** IanGovett has joined #openstack-trove | 00:47 | |
| openstackgerrit | Ramashri Umale proposed a change to openstack/trove: Clusters Guest Implementation https://review.openstack.org/113728 | 00:48 |
| *** IanGovett has quit IRC | 00:50 | |
| *** IanGovett has joined #openstack-trove | 00:53 | |
| *** rwsu has joined #openstack-trove | 00:55 | |
| *** IanGovett has quit IRC | 00:55 | |
| *** rwsu has quit IRC | 00:57 | |
| *** rwsu has joined #openstack-trove | 00:57 | |
| *** saurabhs has quit IRC | 00:58 | |
| *** eghobo has joined #openstack-trove | 01:02 | |
| openstackgerrit | Nikhil Manchanda proposed a change to openstack/trove: Look up trove instance by ID instead of name https://review.openstack.org/114095 | 01:05 |
| *** eghobo has quit IRC | 01:07 | |
| *** amcrn has quit IRC | 01:10 | |
| *** Krast has quit IRC | 01:17 | |
| *** Krast has joined #openstack-trove | 01:17 | |
| *** demorris has joined #openstack-trove | 01:19 | |
| *** julienvey has joined #openstack-trove | 01:19 | |
| *** julienvey has quit IRC | 01:24 | |
| openstackgerrit | Ramashri Umale proposed a change to openstack/trove: Unit tests for clusters guest code https://review.openstack.org/114096 | 01:28 |
| *** tvoran has quit IRC | 01:28 | |
| *** annashen has quit IRC | 01:32 | |
| *** Riddhi has quit IRC | 01:34 | |
| *** Riddhi has joined #openstack-trove | 01:37 | |
| *** nosnos has joined #openstack-trove | 01:38 | |
| *** ramashri has quit IRC | 01:38 | |
| *** haomaiw__ has quit IRC | 01:53 | |
| *** haomaiwang has joined #openstack-trove | 01:53 | |
| *** rushiagr_away is now known as rushiagr | 02:11 | |
| *** julienvey has joined #openstack-trove | 02:20 | |
| *** ramishra has joined #openstack-trove | 02:20 | |
| *** jcru has quit IRC | 02:23 | |
| *** julienvey has quit IRC | 02:25 | |
| *** rushiagr is now known as rushiagr_away | 02:30 | |
| *** jdonalds has joined #openstack-trove | 02:35 | |
| *** ramashri has joined #openstack-trove | 02:38 | |
| *** haomaiw__ has joined #openstack-trove | 02:40 | |
| *** Riddhi has quit IRC | 02:42 | |
| *** ramashri has quit IRC | 02:42 | |
| *** haomaiwang has quit IRC | 02:44 | |
| *** ramashri has joined #openstack-trove | 02:46 | |
| *** annashen has joined #openstack-trove | 02:48 | |
| *** demorris has quit IRC | 02:48 | |
| *** ViswaV_ has joined #openstack-trove | 02:59 | |
| openstackgerrit | Ramashri Umale proposed a change to openstack/trove: Unit tests for clusters guest code https://review.openstack.org/114096 | 02:59 |
| *** ramishra has quit IRC | 03:00 | |
| *** ramishra has joined #openstack-trove | 03:01 | |
| *** ViswaV has quit IRC | 03:03 | |
| *** tylerhannan has quit IRC | 03:07 | |
| *** tylerhannan has joined #openstack-trove | 03:07 | |
| *** ramashri has quit IRC | 03:17 | |
| *** julienvey has joined #openstack-trove | 03:21 | |
| *** julienvey has quit IRC | 03:25 | |
| *** Barker has joined #openstack-trove | 03:26 | |
| *** ViswaV_ has quit IRC | 03:30 | |
| *** nosnos has quit IRC | 03:32 | |
| *** jdonalds has quit IRC | 03:51 | |
| *** erik_x1 has joined #openstack-trove | 03:51 | |
| *** dloi_ has quit IRC | 03:52 | |
| *** radez_g0n3 has quit IRC | 03:52 | |
| *** coolsvap has quit IRC | 03:52 | |
| *** erik_x1_ has quit IRC | 03:52 | |
| *** peterstac has quit IRC | 03:52 | |
| *** jdonalds has joined #openstack-trove | 03:53 | |
| *** peterstac has joined #openstack-trove | 03:53 | |
| *** dloi has joined #openstack-trove | 03:53 | |
| *** radez_g0n3 has joined #openstack-trove | 03:53 | |
| *** jdonalds has quit IRC | 03:59 | |
| *** nosnos has joined #openstack-trove | 04:01 | |
| *** coolsvap has joined #openstack-trove | 04:01 | |
| *** dmakogon_ has quit IRC | 04:01 | |
| *** achampion has quit IRC | 04:09 | |
| *** julienvey has joined #openstack-trove | 04:21 | |
| *** julienvey has quit IRC | 04:26 | |
| *** juantwo has quit IRC | 04:40 | |
| *** amcrn has joined #openstack-trove | 04:46 | |
| *** jdonalds has joined #openstack-trove | 05:05 | |
| *** jdonalds has quit IRC | 05:07 | |
| *** jdonalds has joined #openstack-trove | 05:10 | |
| *** rushiagr_away is now known as rushiagr | 05:19 | |
| *** amcrn has quit IRC | 05:21 | |
| *** julienvey has joined #openstack-trove | 05:22 | |
| *** arborism has joined #openstack-trove | 05:24 | |
| *** julienvey has quit IRC | 05:27 | |
| *** haomaiw__ has quit IRC | 05:31 | |
| *** rushiagr is now known as rushiagr_away | 05:31 | |
| *** rushiagr_away is now known as rushiagr | 05:31 | |
| *** haomaiwang has joined #openstack-trove | 05:31 | |
| *** arborism has quit IRC | 05:35 | |
| *** arborism has joined #openstack-trove | 05:37 | |
| *** haomai___ has joined #openstack-trove | 05:42 | |
| *** haomaiwang has quit IRC | 05:46 | |
| *** ajayaa has joined #openstack-trove | 06:01 | |
| *** arborism has quit IRC | 06:03 | |
| openstackgerrit | GuanYu, lin proposed a change to openstack/trove: Guest Agent goes active before connecting to message queue https://review.openstack.org/114131 | 06:06 |
| *** ramishra has quit IRC | 06:14 | |
| *** ramishra has joined #openstack-trove | 06:14 | |
| *** julienvey has joined #openstack-trove | 06:23 | |
| *** tshefi_ has joined #openstack-trove | 06:24 | |
| *** julienvey has quit IRC | 06:28 | |
| *** k4n0 has joined #openstack-trove | 06:30 | |
| *** ViswaV has joined #openstack-trove | 06:34 | |
| *** iartarisi has joined #openstack-trove | 06:35 | |
| *** ViswaV has quit IRC | 06:35 | |
| *** ramishra has quit IRC | 06:40 | |
| *** sgotliv has joined #openstack-trove | 06:50 | |
| *** arborism has joined #openstack-trove | 06:57 | |
| *** jdonalds has quit IRC | 07:05 | |
| *** jdonalds has joined #openstack-trove | 07:08 | |
| *** jdonalds has quit IRC | 07:08 | |
| *** jdonalds has joined #openstack-trove | 07:08 | |
| *** jdonalds has quit IRC | 07:10 | |
| *** haomai___ has quit IRC | 07:12 | |
| *** haomaiwang has joined #openstack-trove | 07:12 | |
| *** Longgeek has joined #openstack-trove | 07:13 | |
| *** haomaiw__ has joined #openstack-trove | 07:15 | |
| *** haomaiwang has quit IRC | 07:18 | |
| *** julienvey has joined #openstack-trove | 07:24 | |
| *** boblebauce has joined #openstack-trove | 07:26 | |
| *** julienvey has quit IRC | 07:28 | |
| *** arborism has quit IRC | 07:30 | |
| *** annashen has quit IRC | 07:43 | |
| *** denis_makogon has joined #openstack-trove | 07:44 | |
| *** Longgeek has quit IRC | 07:46 | |
| *** Longgeek has joined #openstack-trove | 07:46 | |
| *** mrsnivvel has joined #openstack-trove | 07:51 | |
| *** mrsnivvel has left #openstack-trove | 07:53 | |
| *** Longgeek has quit IRC | 08:00 | |
| *** Longgeek has joined #openstack-trove | 08:00 | |
| *** Longgeek has quit IRC | 08:04 | |
| *** haomaiw__ has quit IRC | 08:04 | |
| *** Longgeek has joined #openstack-trove | 08:05 | |
| *** haomaiwang has joined #openstack-trove | 08:05 | |
| *** iartarisi has quit IRC | 08:06 | |
| *** haomai___ has joined #openstack-trove | 08:11 | |
| *** iartarisi has joined #openstack-trove | 08:13 | |
| *** haomaiwang has quit IRC | 08:14 | |
| *** julienvey has joined #openstack-trove | 08:24 | |
| *** boden has joined #openstack-trove | 08:27 | |
| *** julienvey has quit IRC | 08:29 | |
| *** haomai___ has quit IRC | 09:11 | |
| *** haomaiwa_ has joined #openstack-trove | 09:11 | |
| *** julienvey has joined #openstack-trove | 09:25 | |
| *** julienvey has quit IRC | 09:29 | |
| *** julienvey has joined #openstack-trove | 09:37 | |
| *** ramishra has joined #openstack-trove | 09:45 | |
| *** sgotliv has quit IRC | 10:02 | |
| *** denis_makogon has quit IRC | 10:09 | |
| *** Longgeek has quit IRC | 10:10 | |
| *** Longgeek has joined #openstack-trove | 10:11 | |
| *** Longgeek has quit IRC | 10:14 | |
| *** nosnos has quit IRC | 10:16 | |
| *** ramishra has quit IRC | 10:19 | |
| *** ramishra has joined #openstack-trove | 10:21 | |
| *** haomaiw__ has joined #openstack-trove | 10:24 | |
| *** sgotliv has joined #openstack-trove | 10:25 | |
| *** haomaiwa_ has quit IRC | 10:27 | |
| *** ramishra has quit IRC | 10:30 | |
| *** ramishra has joined #openstack-trove | 10:31 | |
| *** ajayaa has quit IRC | 10:34 | |
| *** Longgeek has joined #openstack-trove | 10:37 | |
| *** haomaiw__ has quit IRC | 10:50 | |
| *** haomaiwang has joined #openstack-trove | 10:51 | |
| *** ramishra has quit IRC | 10:51 | |
| openstackgerrit | GuanYu, lin proposed a change to openstack/trove: Guest Agent goes active before connecting to message queue https://review.openstack.org/114131 | 10:51 |
| *** ajayaa has joined #openstack-trove | 11:01 | |
| *** sgotliv_ has joined #openstack-trove | 11:05 | |
| *** sgotliv has quit IRC | 11:07 | |
| *** ramishra_ has joined #openstack-trove | 11:18 | |
| *** sgotliv_ has quit IRC | 11:21 | |
| *** ramishra_ has quit IRC | 11:31 | |
| *** Krast has quit IRC | 11:31 | |
| *** ramishra_ has joined #openstack-trove | 11:35 | |
| *** denis_makogon has joined #openstack-trove | 11:38 | |
| *** ramishra_ has quit IRC | 11:47 | |
| *** julienvey has quit IRC | 11:48 | |
| *** juantwo has joined #openstack-trove | 12:09 | |
| *** juantwo has quit IRC | 12:10 | |
| *** juantwo has joined #openstack-trove | 12:11 | |
| *** IanGovett has joined #openstack-trove | 12:13 | |
| *** sgotliv has joined #openstack-trove | 12:26 | |
| *** pdmars has joined #openstack-trove | 12:31 | |
| *** sgotliv has quit IRC | 12:42 | |
| *** tkatarki has joined #openstack-trove | 12:42 | |
| *** tkatarki has quit IRC | 12:44 | |
| *** robertmyers has joined #openstack-trove | 12:47 | |
| *** robertmyers has quit IRC | 12:51 | |
| *** IanGovett has quit IRC | 12:53 | |
| *** tkatarki has joined #openstack-trove | 12:59 | |
| *** ajayaa has quit IRC | 13:01 | |
| *** kevinconway has joined #openstack-trove | 13:08 | |
| *** achampion has joined #openstack-trove | 13:11 | |
| *** tkatarki has quit IRC | 13:20 | |
| *** tkatarki has joined #openstack-trove | 13:23 | |
| openstackgerrit | GuanYu, lin proposed a change to openstack/trove: Guest Agent goes active before connecting to message queue https://review.openstack.org/114131 | 13:35 |
| *** rushiagr is now known as rushiagr_away | 13:39 | |
| *** pdmars has quit IRC | 13:43 | |
| openstackgerrit | A change was merged to openstack/trove: Adjusted audit logging for taskmanager module https://review.openstack.org/108020 | 13:51 |
| *** pdmars has joined #openstack-trove | 13:51 | |
| *** coolsvap has quit IRC | 14:07 | |
| *** coolsvap has joined #openstack-trove | 14:11 | |
| *** jasondotstar has joined #openstack-trove | 14:13 | |
| *** IanGovett has joined #openstack-trove | 14:14 | |
| *** tkatarki has quit IRC | 14:20 | |
| *** Barker has quit IRC | 14:23 | |
| *** tkatarki has joined #openstack-trove | 14:26 | |
| *** IanGovett has quit IRC | 14:26 | |
| *** ruebenramirez has joined #openstack-trove | 14:27 | |
| *** tkatarki has quit IRC | 14:27 | |
| *** grapex_ has joined #openstack-trove | 14:29 | |
| *** Riddhi has joined #openstack-trove | 14:30 | |
| *** tkatarki has joined #openstack-trove | 14:37 | |
| *** ramashri has joined #openstack-trove | 14:38 | |
| *** rushiagr_away is now known as rushiagr | 14:41 | |
| *** k4n0 has quit IRC | 14:45 | |
| *** mattgriffin has joined #openstack-trove | 14:50 | |
| *** mattgriffin has quit IRC | 14:50 | |
| *** mattgriffin has joined #openstack-trove | 14:51 | |
| *** johnma has joined #openstack-trove | 14:51 | |
| *** kevinconway has quit IRC | 14:55 | |
| *** jcru has joined #openstack-trove | 14:59 | |
| *** Barker has joined #openstack-trove | 15:00 | |
| *** kevinconway has joined #openstack-trove | 15:04 | |
| *** sgotliv has joined #openstack-trove | 15:08 | |
| *** kevinconway has quit IRC | 15:10 | |
| *** robertmyers has joined #openstack-trove | 15:15 | |
| *** dkehnx has quit IRC | 15:16 | |
| *** iartarisi has quit IRC | 15:17 | |
| *** ramashri has quit IRC | 15:20 | |
| *** boblebauce has quit IRC | 15:21 | |
| openstackgerrit | Simon Chang proposed a change to openstack/trove: Adjusted audit logging for trove.instance module https://review.openstack.org/105294 | 15:22 |
| *** dkehn_ has joined #openstack-trove | 15:31 | |
| *** achampio1 has joined #openstack-trove | 15:31 | |
| *** dkehn_ is now known as dkehnx | 15:32 | |
| *** tkatarki1 has joined #openstack-trove | 15:32 | |
| *** tkatarki has quit IRC | 15:38 | |
| *** achampion has quit IRC | 15:38 | |
| *** dkehnx has quit IRC | 15:46 | |
| *** sgotliv has quit IRC | 15:49 | |
| *** ramashri has joined #openstack-trove | 15:50 | |
| *** Longgeek has quit IRC | 15:52 | |
| *** dkehn_ has joined #openstack-trove | 15:52 | |
| *** dkehn_ is now known as dkehnx | 15:52 | |
| *** jmontemayor has joined #openstack-trove | 15:53 | |
| *** mattgriffin has quit IRC | 15:59 | |
| *** tkatarki1 has quit IRC | 15:59 | |
| *** georgelorch has quit IRC | 16:00 | |
| *** pdmars has quit IRC | 16:01 | |
| *** PeterSchwaller has quit IRC | 16:01 | |
| *** georgelorch has joined #openstack-trove | 16:01 | |
| *** kevinconway has joined #openstack-trove | 16:02 | |
| *** ramashri has quit IRC | 16:11 | |
| *** tshefi_ has quit IRC | 16:14 | |
| *** shayneburgess has joined #openstack-trove | 16:23 | |
| *** ruebenramirez has quit IRC | 16:25 | |
| *** tvoran has joined #openstack-trove | 16:29 | |
| openstackgerrit | Felipe Reyes proposed a change to openstack/trove: Add support to detect SUSE https://review.openstack.org/108394 | 16:32 |
| openstackgerrit | Felipe Reyes proposed a change to openstack/trove: Add SUSE support in mysql datastore https://review.openstack.org/108703 | 16:33 |
| *** annashen has joined #openstack-trove | 16:34 | |
| *** PeterSchwaller has joined #openstack-trove | 16:35 | |
| *** ramishra has joined #openstack-trove | 16:37 | |
| *** ruebenramirez has joined #openstack-trove | 16:37 | |
| *** kevinconway has quit IRC | 16:37 | |
| *** PeterSchwaller has quit IRC | 16:40 | |
| *** tkatarki has joined #openstack-trove | 16:41 | |
| *** ramishra has quit IRC | 16:41 | |
| *** kevinconway has joined #openstack-trove | 16:41 | |
| *** saurabhs has joined #openstack-trove | 16:51 | |
| *** georgelorch has quit IRC | 16:53 | |
| *** georgelorch has joined #openstack-trove | 16:53 | |
| *** georgelorch has quit IRC | 17:01 | |
| *** kevinconway has quit IRC | 17:03 | |
| *** PeterSchwaller has joined #openstack-trove | 17:03 | |
| *** georgelorch has joined #openstack-trove | 17:03 | |
| *** radez_g0n3 is now known as radez | 17:10 | |
| *** Barker has quit IRC | 17:31 | |
| openstackgerrit | Greg Lucas proposed a change to openstack/trove: Mysql guest agent functionality for replication https://review.openstack.org/98147 | 17:33 |
| *** Barker has joined #openstack-trove | 17:35 | |
| *** PeterSchwaller has quit IRC | 17:36 | |
| *** tkatarki has quit IRC | 17:36 | |
| *** ramishra has joined #openstack-trove | 17:37 | |
| openstackgerrit | Greg Lucas proposed a change to openstack/trove: Add detach-replication support https://review.openstack.org/101981 | 17:41 |
| *** ramishra has quit IRC | 17:42 | |
| *** tkatarki has joined #openstack-trove | 17:46 | |
| *** PeterSchwaller has joined #openstack-trove | 17:52 | |
| *** pdmars has joined #openstack-trove | 17:57 | |
| *** pdmars has quit IRC | 17:59 | |
| *** pdmars has joined #openstack-trove | 17:59 | |
| *** tkatarki has quit IRC | 18:02 | |
| *** radez is now known as radez_g0n3 | 18:06 | |
| *** johnma_ has joined #openstack-trove | 18:17 | |
| *** johnma has quit IRC | 18:19 | |
| *** johnma_ is now known as johnma | 18:19 | |
| *** NehaV1 has joined #openstack-trove | 18:32 | |
| *** ramishra has joined #openstack-trove | 18:38 | |
| *** georgelorch has quit IRC | 18:40 | |
| iccha | glucas: I think I am running into timeout issues too | 18:42 |
| *** georgelorch has joined #openstack-trove | 18:42 | |
| *** ramishra has quit IRC | 18:42 | |
| *** sgotliv has joined #openstack-trove | 18:54 | |
| openstackgerrit | Greg Lucas proposed a change to openstack/trove: Snapshot component for replication https://review.openstack.org/109687 | 18:56 |
| *** sgotliv has quit IRC | 18:57 | |
| *** sgotliv has joined #openstack-trove | 18:57 | |
| *** Barker has quit IRC | 18:59 | |
| *** Barker has joined #openstack-trove | 18:59 | |
| amrith | hmmm ... now that rdjenkins has become "Deprecated Trove CI", how does one kick it? "rdjenkins no bug" doesn't seem to work. is it "Deprecated Trove CI no bug"? | 19:10 |
| *** grapex_ has quit IRC | 19:11 | |
| amrith | that doesn't work ... | 19:11 |
| *** grapex has joined #openstack-trove | 19:11 | |
| *** kevinconway has joined #openstack-trove | 19:15 | |
| openstackgerrit | Greg Lucas proposed a change to openstack/trove: Augment Mysql status checking for replication https://review.openstack.org/112689 | 19:20 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/trove: Updated from global requirements https://review.openstack.org/111529 | 19:23 |
| openstackgerrit | Greg Lucas proposed a change to openstack/trove: Unit Tests for Mysql replication functionality https://review.openstack.org/109313 | 19:33 |
| *** miqui has quit IRC | 19:33 | |
| *** denis_makogon_ has joined #openstack-trove | 19:35 | |
| *** kevinconway has quit IRC | 19:37 | |
| glucas | iccha: is it the same agent high timeout that SlickNik hit? | 19:37 |
| glucas | iccha: dougshelley66 was saying he had to increate the usage_timeout as well. | 19:37 |
| *** jmontemayor has quit IRC | 19:37 | |
| glucas | s/increate/increase/ | 19:37 |
| *** jmontemayor has joined #openstack-trove | 19:38 | |
| *** ramishra has joined #openstack-trove | 19:39 | |
| SlickNik | iccha / glucas: There were two timeouts I needed to increase: the timeout for the get_replication_snapshot, and the usage_timeout. | 19:39 |
| iccha | glucas: i also randomly from time to time strangely run into an error which says xtraback_binlog_info file is not found. i have no clue why. digging in. dont knoe if its related to time outs | 19:40 |
| *** miqui has joined #openstack-trove | 19:40 | |
| *** jdandrea has quit IRC | 19:44 | |
| *** ramishra has quit IRC | 19:44 | |
| *** kevinconway has joined #openstack-trove | 19:45 | |
| *** sgotliv has quit IRC | 19:46 | |
| openstackgerrit | Simon Chang proposed a change to openstack/trove: Adjusted audit logging for trove.instance module https://review.openstack.org/105294 | 19:54 |
| *** tkatarki has joined #openstack-trove | 19:56 | |
| *** tkatarki has quit IRC | 20:05 | |
| vgnbkr | iccha: When I was developing the feature, I couldn't find the xtraback_binlog_info file either - I wonder if there's some state that causes it not to appear. | 20:09 |
| iccha | vgnbkr: what did u do it make it work then | 20:11 |
| iccha | vgnbkr: i ran it multiple times and only once the slave went activae. other times there was issue finding the binlog info file | 20:12 |
| *** jcru_ has joined #openstack-trove | 20:15 | |
| *** robertmy_ has joined #openstack-trove | 20:15 | |
| *** jcru has quit IRC | 20:18 | |
| *** robertmyers has quit IRC | 20:18 | |
| SlickNik | iccha: There's a race condition where the attach_replication_slave message makes it to the guestagent even before prepare is finished — if the usage_timeout in the models is set too low. | 20:19 |
| *** jasondotstar has quit IRC | 20:19 | |
| iccha | gotcha that makes sense SlickNik | 20:19 |
| iccha | SlickNik: do we want to first try to push out and merge https://review.openstack.org/#/c/98147/25 ? | 20:21 |
| SlickNik | iccha: yes, I think that one has had adequate review and test coverage - so let's try and get that merged. | 20:22 |
| iccha | SlickNik: totally agree with that | 20:22 |
| iccha | and on the snapshot code I agree with some of denis_makogon_ 's comments. it could do with a little bit more polishing | 20:23 |
| SlickNik | iccha: rdjenkins died last night — so I'm looking at getting another jenkins box up and running to vet the tests before I +2. | 20:23 |
| denis_makogon_ | iccha, nice to hear =) | 20:23 |
| *** tkatarki has joined #openstack-trove | 20:23 | |
| iccha | SlickNik: you have been battling jenkins for a long time. thanks for keeping that going for us. | 20:24 |
| SlickNik | iccha: can you call out which of the comments you agree with as part of your review (if you haven't already done so — I haven't had a chance to look yet). | 20:24 |
| SlickNik | And thanks for the reviews, guys :) | 20:24 |
| iccha | ok sure will do SlickNik | 20:26 |
| *** tkatarki has quit IRC | 20:26 | |
| vgnbkr | iccha: re binlog - I didn't do anything - Greg found the file, then it just worked for me | 20:27 |
| iccha | glucas: SlickNik vgnbkr for some future concerns https://bugs.launchpad.net/trove/+bug/1357065 | 20:28 |
| iccha | SlickNik: what did u increase the time out too? I increased the usage timeout and agent high timeout and still having issues | 20:31 |
| SlickNik | iccha: I only increased those two values you mentioned. | 20:33 |
| SlickNik | iccha: Can you gist your guestagent-log? | 20:33 |
| SlickNik | iccha: Also, thanks for the bug. +1 to re-iterating and fixing some of these bugs/issues we find once we have the basic functionality merged (but before juno is cut). | 20:34 |
| iccha | SlickNik: https://gist.github.com/isethi/e47dcdec32000ada48ae its related to what you said i think. its trying to attach slave even before the restore is done | 20:35 |
| dougshelley66 | SlickNik, I think that makes the most sense as well. We can have more folks engaged in hardening it after the base commits are merged | 20:35 |
| *** tkatarki has joined #openstack-trove | 20:36 | |
| *** georgelorch has quit IRC | 20:36 | |
| *** georgelorch has joined #openstack-trove | 20:37 | |
| *** tkatarki has quit IRC | 20:37 | |
| SlickNik | iccha: yup looks like it. Just a silly question, but did you ensure to restart the trove-services (esp taskmanager) after you changed the timeouts? | 20:38 |
| iccha | yes SlickNik restarted all services. maybe i didnt sufficiently increase timeouts. will try with higher values | 20:39 |
| *** ramishra has joined #openstack-trove | 20:40 | |
| *** boden has quit IRC | 20:40 | |
| iccha | btw SlickNik did the trove community already have options discussed for failover earlier? are there any contenders? glucas vgnbkr dougshelley66 | 20:43 |
| *** ramishra has quit IRC | 20:44 | |
| SlickNik | iccha: I know vipul was looking into a few designs for auto-failover earlier — he probably has a lot more info regarding that. | 20:44 |
| iccha | vipul: any thoughts? | 20:45 |
| iccha | just trying to do my background work for the midcycle :) | 20:45 |
| *** rushiagr is now known as rushiagr_away | 20:47 | |
| denis_makogon_ | iccha, i've got question to you | 20:47 |
| iccha | yes denis_makogon_ | 20:47 |
| denis_makogon_ | iccha, any clue how to fix use of hardcoded repliation password? | 20:47 |
| vgnbkr | I suspect that failover will involve support beyond Trove - I suspect there will need to be some way to change the address that the user uses to connect to the db. First, I expect we would need to change the way the user addresses the database (i.e, not by direct ip address). | 20:48 |
| vgnbkr | iccha: ^^ | 20:48 |
| iccha | denis_makogon_: what the mysql dbas i spoke to suggest that we geenrate a random password that we do not need to store. if we need to reset replication we can always create a new user with a new random password | 20:48 |
| iccha | vgnbkr: yeah and there are so many ways this can be done. virtual floating ip, load baalncers, etc | 20:49 |
| denis_makogon_ | vgnbkr, when you'll try to fix it, leave some place for barbican integration | 20:50 |
| denis_makogon_ | vgnbkr, sorry | 20:50 |
| denis_makogon_ | iccha, ^^ | 20:50 |
| denis_makogon_ | iccha, when you'll try to fix it, leave some place for barbican integration | 20:50 |
| iccha | denis_makogon_: if we find a concrete need to store passwords then sure | 20:50 |
| denis_makogon_ | iccha, not store - deliver | 20:51 |
| iccha | why would we need to deliver passwords denis_makogon_ ? | 20:52 |
| denis_makogon_ | iccha, as you said, for replication user | 20:52 |
| vipul | iccha: No, so the last time we brought this up, the consensus was let's not invent our own mechanism for monitoring mysql and detecting when to fail over | 20:52 |
| denis_makogon_ | vipul, ++ | 20:52 |
| vipul | the folks from Percona said we need to be looking at existing tools that do this | 20:53 |
| vipul | but i haven't gotten around to actually looking at any of them | 20:53 |
| vipul | and what's unclear is how do we integrate those tools with trove | 20:53 |
| iccha | vipul: i think thats a good idea. | 20:53 |
| iccha | haha | 20:53 |
| vipul | because we want Trove to know when this tools has actually done a fialover | 20:53 |
| iccha | i typed the same things vipul | 20:53 |
| denis_makogon_ | vipul, existing tools are out of OpenStack, but we can use ceilometer once it'll have in-application monitoring | 20:53 |
| iccha | vipul: and some of these tools require setup outside trove infrastructure | 20:54 |
| vipul | iccha: yep exactly -- so i am thinking we need a reference implemenation that is openstack/trove specific. | 20:54 |
| vipul | and we need to make it pluggable so we can use other off the shelf things if an operator wants | 20:54 |
| vipul | imo Trove standalone shoudl be able to do some level of failover.. | 20:55 |
| vipul | it might not be as good as a third-party tool might.. but i don't know how to make it available out of the box otherwise | 20:55 |
| denis_makogon_ | vipul, what about initially implement periodic notifications from guest (datastore specific)? | 20:55 |
| iccha | vipul: so pluggable home grown basic solution? | 20:55 |
| vipul | denis_makogon_ yea it could be that.. something that relies on the guest notifications as the reference impl | 20:56 |
| vipul | iccha: yea | 20:56 |
| denis_makogon_ | vipul, take a look http://lists.openstack.org/pipermail/openstack-dev/2014-July/041356.html | 20:57 |
| denis_makogon_ | iccha, please take a look too, http://lists.openstack.org/pipermail/openstack-dev/2014-July/041356.html | 20:57 |
| vipul | a guest notification based implemenation won't be that great.. won't be robust, and likely will mean a longer time before we actually faiilover a customer database.. but it's something that Trove can provide without requiring operators to pick a tool | 20:57 |
| iccha | will need to dig in and see how these solutions (failover) can be remotely pluggable | 20:59 |
| denis_makogon_ | vipul, yes, i was looking to monitoring systems, and seems that there's no transparent way to integrate them into some sort of monitoring service (ceilometer or something else) | 20:59 |
| iccha | stuff like mha seems so different from say mysqlfailover | 20:59 |
| denis_makogon_ | iccha, there are tons of ways to do failover | 21:00 |
| denis_makogon_ | iccha, personally, i'd like to have failover with hidden node that's ready for substitution of any failed node | 21:01 |
| denis_makogon_ | returning to guest notifications | 21:02 |
| denis_makogon_ | we can add notification to guest and once DB is down due to certain reasons, the closest notification to ceilometer will raise alarm that can be used by side tools | 21:03 |
| vipul | iccha: yea, a lot of them also require the ability to ssh onto these boxes | 21:03 |
| vipul | which Trove has tried to avoid opening up.. (although an operator can do that if they still choose) | 21:04 |
| denis_makogon_ | for example, heat does autoscaling taking into account ceilometer alarm | 21:04 |
| vipul | while the ceiolometer based approach seem good long term, i don't know how many of us actually run Ceiolometer | 21:05 |
| vipul | and it would be adding a new dependency to run Trove | 21:05 |
| iccha | denis_makogon_: yeah this is gonna be a hard one to solve cause everyone is going to have their own preferences | 21:05 |
| iccha | vipul: yeah there are solutions liek opening specific ports only to the sentinel nodes etc, but i dont know why but i feel that this is going to be something hard to get into trove | 21:06 |
| iccha | bbiab happy hour time here | 21:08 |
| vipul | woah cool -- enjoy! | 21:09 |
| *** jmontemayor has quit IRC | 21:10 | |
| *** robertmy_ has quit IRC | 21:11 | |
| glucas | iccha: nice. I'm enjoying happy hour too, but that's only because I'm working at home today. :D | 21:13 |
| *** tkatarki has joined #openstack-trove | 21:15 | |
| *** tkatarki has quit IRC | 21:27 | |
| *** annashen has quit IRC | 21:30 | |
| *** georgelorch has quit IRC | 21:30 | |
| *** georgelorch has joined #openstack-trove | 21:30 | |
| *** juantwo has quit IRC | 21:34 | |
| *** ruebenramirez has quit IRC | 21:35 | |
| *** openstackgerrit has quit IRC | 21:46 | |
| *** openstackgerrit has joined #openstack-trove | 21:47 | |
| *** ramishra has joined #openstack-trove | 21:54 | |
| *** georgelorch has quit IRC | 21:58 | |
| *** georgelorch has joined #openstack-trove | 21:58 | |
| *** johnma has quit IRC | 21:58 | |
| *** ramishra has quit IRC | 21:59 | |
| *** ramashri has joined #openstack-trove | 22:05 | |
| *** achampio1 has quit IRC | 22:07 | |
| *** arist has quit IRC | 22:09 | |
| *** arist has joined #openstack-trove | 22:10 | |
| *** NehaV1 has quit IRC | 22:10 | |
| *** Barker has quit IRC | 22:10 | |
| amrith | oh, thanks glucas. I was wondering where you were at lunch. | 22:18 |
| openstackgerrit | Simon Chang proposed a change to openstack/trove: Fixed database migration script issues https://review.openstack.org/113582 | 22:25 |
| *** jcru_ has quit IRC | 22:40 | |
| *** georgelorch has quit IRC | 22:41 | |
| *** georgelorch has joined #openstack-trove | 22:43 | |
| denis_makogon_ | glucas, are you there ? | 22:53 |
| *** amcrn has joined #openstack-trove | 22:54 | |
| *** ramishra has joined #openstack-trove | 22:55 | |
| *** pdmars has quit IRC | 22:56 | |
| *** juantwo has joined #openstack-trove | 22:58 | |
| *** juantwo has quit IRC | 22:59 | |
| *** juantwo has joined #openstack-trove | 22:59 | |
| *** ramishra has quit IRC | 22:59 | |
| denis_makogon_ | SlickNik, are you around? | 23:00 |
| *** kevinconway has quit IRC | 23:02 | |
| *** denis_makogon has quit IRC | 23:06 | |
| *** denis_makogon_ has quit IRC | 23:06 | |
| *** denis_makogon has joined #openstack-trove | 23:06 | |
| SlickNik | denis_makogon: Around now. | 23:07 |
| openstackgerrit | Nikhil Manchanda proposed a change to openstack/python-troveclient: Add new command: detach-replica https://review.openstack.org/101771 | 23:12 |
| *** tvoran has quit IRC | 23:35 | |
| *** tvoran has joined #openstack-trove | 23:37 | |
| *** ramishra has joined #openstack-trove | 23:56 | |
| *** Riddhi has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!