*** kevinconway has quit IRC | 00:03 | |
*** yogesh has quit IRC | 00:10 | |
openstackgerrit | Michael Basnight proposed a change to openstack/trove: Removes XML api from trove https://review.openstack.org/68333 | 00:16 |
---|---|---|
*** mattgriffin has joined #openstack-trove | 00:35 | |
openstackgerrit | Auston McReynolds proposed a change to openstack/trove: Root_on_create per datastore https://review.openstack.org/77275 | 00:50 |
openstackgerrit | Auston McReynolds proposed a change to openstack/trove: Root_on_create per datastore https://review.openstack.org/77275 | 00:56 |
*** Barker has joined #openstack-trove | 01:04 | |
*** ramashri has joined #openstack-trove | 01:22 | |
*** nosnos has joined #openstack-trove | 01:37 | |
*** nosnos has quit IRC | 01:38 | |
*** nosnos has joined #openstack-trove | 01:40 | |
*** achampion has quit IRC | 01:45 | |
*** Ranjitha has joined #openstack-trove | 01:46 | |
*** rhodgin has joined #openstack-trove | 01:51 | |
openstackgerrit | A change was merged to openstack/trove: Rename Openstack to OpenStack https://review.openstack.org/73209 | 01:59 |
*** ramashri has quit IRC | 02:00 | |
*** mattgriffin has quit IRC | 02:04 | |
*** Ranjitha has quit IRC | 02:05 | |
*** haomaiwa_ has joined #openstack-trove | 02:06 | |
*** erkules_ has joined #openstack-trove | 02:06 | |
*** haomaiwang has quit IRC | 02:06 | |
*** erkules has quit IRC | 02:09 | |
*** erkules_ is now known as erkules | 02:09 | |
openstackgerrit | Auston McReynolds proposed a change to openstack/python-troveclient: Adding datastore_version column to instances list table https://review.openstack.org/74491 | 02:13 |
*** achampion has joined #openstack-trove | 02:15 | |
*** haomaiwang has joined #openstack-trove | 02:41 | |
*** haomaiwa_ has quit IRC | 02:45 | |
*** Barker has quit IRC | 03:04 | |
openstackgerrit | Auston McReynolds proposed a change to openstack/trove: Remove Min/Max for Configuration Group Booleans https://review.openstack.org/78499 | 03:55 |
*** grapex_ has joined #openstack-trove | 04:14 | |
*** grapex_ has quit IRC | 04:16 | |
*** shivam_ has joined #openstack-trove | 04:17 | |
*** grapex has quit IRC | 04:18 | |
*** shivamshukla has quit IRC | 04:21 | |
*** haomaiwa_ has joined #openstack-trove | 04:33 | |
*** haomaiwang has quit IRC | 04:34 | |
*** rhodgin has quit IRC | 04:36 | |
openstackgerrit | Auston McReynolds proposed a change to openstack/python-troveclient: Fix JSON In Output for Configuration Groups https://review.openstack.org/79235 | 04:37 |
*** atomic77 has joined #openstack-trove | 04:48 | |
*** saju_m has joined #openstack-trove | 04:59 | |
*** grapex has joined #openstack-trove | 05:17 | |
*** grapex has quit IRC | 05:25 | |
*** nosnos has quit IRC | 05:39 | |
*** nosnos has joined #openstack-trove | 05:39 | |
*** saju_m has quit IRC | 05:54 | |
*** saju_m has joined #openstack-trove | 05:55 | |
*** atomic77 has quit IRC | 05:57 | |
*** saju_m has quit IRC | 06:00 | |
*** saju_m has joined #openstack-trove | 06:02 | |
*** SushilKM has joined #openstack-trove | 06:07 | |
*** 7JTAAJM33 has quit IRC | 06:09 | |
*** grapex has joined #openstack-trove | 06:17 | |
*** SushilKM has quit IRC | 06:25 | |
*** grapex has quit IRC | 06:25 | |
*** SushilKM has joined #openstack-trove | 06:25 | |
*** osherov has joined #openstack-trove | 06:28 | |
*** nosnos_ has joined #openstack-trove | 06:51 | |
*** nosnos has quit IRC | 06:51 | |
*** yogesh_ has joined #openstack-trove | 06:53 | |
*** nosnos has joined #openstack-trove | 06:56 | |
*** nosnos_ has quit IRC | 06:58 | |
*** osherov has quit IRC | 07:04 | |
*** nosnos has quit IRC | 07:05 | |
*** nosnos has joined #openstack-trove | 07:08 | |
*** nosnos has quit IRC | 07:12 | |
*** nosnos has joined #openstack-trove | 07:12 | |
*** grapex has joined #openstack-trove | 07:18 | |
*** grapex has quit IRC | 07:22 | |
*** SnowDust has joined #openstack-trove | 07:33 | |
*** matsuhashi has joined #openstack-trove | 07:39 | |
openstackgerrit | Auston McReynolds proposed a change to openstack/trove: Changes Volume Prefix From mysql To datastore https://review.openstack.org/71829 | 07:40 |
*** flaper87|afk is now known as flaper87 | 08:14 | |
*** iartarisi has joined #openstack-trove | 08:17 | |
*** grapex has joined #openstack-trove | 08:18 | |
*** saju_m has quit IRC | 08:22 | |
*** grapex has quit IRC | 08:22 | |
*** tattabbum has joined #openstack-trove | 08:25 | |
*** saju_m has joined #openstack-trove | 08:28 | |
*** ihrachys|afk is now known as ihrachys | 08:28 | |
*** yogesh_ has quit IRC | 08:35 | |
*** matsuhashi has quit IRC | 08:50 | |
*** saju_m has quit IRC | 08:50 | |
*** matsuhashi has joined #openstack-trove | 08:54 | |
*** saju_m has joined #openstack-trove | 09:04 | |
*** saju_m has quit IRC | 09:10 | |
*** saju_m has joined #openstack-trove | 09:11 | |
*** grapex has joined #openstack-trove | 09:19 | |
*** grapex has quit IRC | 09:23 | |
openstackgerrit | Ionut Artarisi proposed a change to openstack/trove: rename and fix the db_wipe command https://review.openstack.org/73229 | 09:28 |
openstackgerrit | SlickNik proposed a change to openstack/python-troveclient: Get rid of XML related trove client bindings https://review.openstack.org/69999 | 09:34 |
openstackgerrit | Giuseppe Galeota proposed a change to openstack/trove: Trove Manual Installation guide https://review.openstack.org/78608 | 09:37 |
*** fghaas has joined #openstack-trove | 10:00 | |
*** SnowDust has quit IRC | 10:01 | |
fghaas | amcrn: I wasn't sure whether you were still up, but seeing as you are, we might as well have our conversation here :) | 10:11 |
amcrn | well, i'm just about to hit the hay unfortunately | 10:12 |
fghaas | no worries. was just about to write this: | 10:12 |
fghaas | I've already been chatting with some folks about getting this set up on StackForge, and Chris Hoge and Emilien Macchi have offered to help | 10:13 |
amcrn | ah, excellent | 10:13 |
fghaas | it would be great if you could let us know when you reckon you could push your stuff up on GitHub | 10:14 |
fghaas | but feel free to do that in the morning, by all means :) | 10:14 |
amcrn | we've got to strip out one or two internal puppet resources, and update it to handle icehouse, but it generally shouldn't be too burdensome | 10:14 |
fghaas | sounds great. I take it you're familiar with the process of setting up a StackForge project? if not, either I or any of the other two guys I mentioned can help | 10:15 |
amcrn | i've never done it personally, so i definitely wouldn't mind if you got the ball rolling on that | 10:16 |
fghaas | roger that, can do — but we need an initial repo to pull from. :) hence my question of when this will be up on GitHub | 10:16 |
amcrn | ah, you're asking me to put it up on a personal repo as a first step | 10:17 |
amcrn | alright, give me about a week, i've got some competing interests | 10:17 |
amcrn | we can circle back next monday? | 10:17 |
fghaas | either personal or one that your organization owns, yes | 10:18 |
fghaas | or, if some of the commit history is confidential/proprietary, we can also set up an empty repo and you copy your code in | 10:19 |
*** grapex has joined #openstack-trove | 10:19 | |
fghaas | whichever works better for you | 10:19 |
amcrn | sounds good, let me do some internal housekeeping on the modules and get back to you next week | 10:20 |
fghaas | that works for me. I'll write a quick email so everyone who expressed interest is informed, does that work for you? | 10:20 |
*** IvanZ has joined #openstack-trove | 10:20 | |
amcrn | that works | 10:21 |
fghaas | perfect, thanks! | 10:22 |
*** grapex has quit IRC | 10:23 | |
*** achampio1 has joined #openstack-trove | 10:32 | |
*** achampion has quit IRC | 10:32 | |
*** achampio1 has quit IRC | 10:36 | |
*** achampion has joined #openstack-trove | 10:40 | |
*** arist has quit IRC | 10:54 | |
*** arist has joined #openstack-trove | 10:54 | |
*** saju_m has quit IRC | 11:05 | |
*** grapex has joined #openstack-trove | 11:20 | |
tattabbum | denis_makogon: Hi denis. Does your mysql.cloudinit (https://gist.github.com/denismakogon/9410248) execute into a partial pre-build image or into a vanilla ubuntu image (like this: http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img)? | 11:22 |
*** grapex has quit IRC | 11:24 | |
*** IvanZ has quit IRC | 11:34 | |
*** fghaas has quit IRC | 11:35 | |
*** kevinconway has joined #openstack-trove | 11:42 | |
*** kevinconway has quit IRC | 11:45 | |
*** achampion has quit IRC | 11:59 | |
*** achampion has joined #openstack-trove | 11:59 | |
*** kevinconway has joined #openstack-trove | 12:01 | |
*** achampion has joined #openstack-trove | 12:01 | |
*** kevinconway_ has joined #openstack-trove | 12:06 | |
*** kevinconway has quit IRC | 12:06 | |
*** kevinconway_ is now known as kevinconway | 12:06 | |
*** achampion has quit IRC | 12:06 | |
*** SnowDust has joined #openstack-trove | 12:09 | |
*** matsuhashi has quit IRC | 12:11 | |
*** fghaas has joined #openstack-trove | 12:11 | |
*** matsuhashi has joined #openstack-trove | 12:13 | |
*** achampion has joined #openstack-trove | 12:14 | |
*** radez_g0n3 is now known as radez | 12:19 | |
*** grapex has joined #openstack-trove | 12:20 | |
*** grapex has quit IRC | 12:24 | |
*** SnowDust has quit IRC | 12:28 | |
*** SushilKM has quit IRC | 12:37 | |
*** grapex has joined #openstack-trove | 12:47 | |
*** grapex has quit IRC | 12:47 | |
*** pdmars has joined #openstack-trove | 12:51 | |
*** jcru has joined #openstack-trove | 13:14 | |
*** grapex has joined #openstack-trove | 13:14 | |
*** rhodgin has joined #openstack-trove | 13:17 | |
*** grapex has quit IRC | 13:20 | |
*** grapex has joined #openstack-trove | 13:21 | |
*** rustlebee is now known as russellb | 13:26 | |
*** matsuhashi has quit IRC | 13:32 | |
*** Barker has joined #openstack-trove | 13:36 | |
*** nosnos has quit IRC | 13:38 | |
*** iartarisi has quit IRC | 13:52 | |
*** iartarisi has joined #openstack-trove | 13:55 | |
*** freyes has quit IRC | 14:16 | |
*** mattgriffin has joined #openstack-trove | 14:32 | |
*** rwsu has joined #openstack-trove | 14:36 | |
*** iartaris` has joined #openstack-trove | 14:48 | |
*** juice has quit IRC | 14:49 | |
*** iartarisi has quit IRC | 14:49 | |
cp16net | amcrn: you around? | 14:49 |
fghaas | cp16net: I spoke with him earlier at about 2am his local time and he was just about to hit the hay — as that was only 4 hours ago, I suspect he's fast asleep now :) | 14:49 |
cp16net | hah | 14:49 |
cp16net | ok | 14:49 |
cp16net | thx | 14:49 |
*** juice has joined #openstack-trove | 14:51 | |
*** iartaris` is now known as iartarisi` | 14:51 | |
*** iartarisi` has quit IRC | 14:52 | |
*** iartarisi` has joined #openstack-trove | 14:52 | |
openstackgerrit | Jenkins proposed a change to openstack/trove: Updated from global requirements https://review.openstack.org/76705 | 14:53 |
*** freyes has joined #openstack-trove | 14:53 | |
*** demorris has joined #openstack-trove | 14:55 | |
*** iartarisi` is now known as iartarisi | 14:57 | |
*** thedodd has joined #openstack-trove | 14:58 | |
*** datsun180b has joined #openstack-trove | 14:59 | |
*** Barker has quit IRC | 15:24 | |
juice | amcrn: thanks for the behemoth push this weekend to review/approve patches | 15:43 |
*** jmontemayor has joined #openstack-trove | 15:46 | |
*** ViswaV has joined #openstack-trove | 15:47 | |
*** ViswaV_ has joined #openstack-trove | 15:49 | |
*** ViswaV_ has quit IRC | 15:50 | |
*** ViswaV_ has joined #openstack-trove | 15:50 | |
*** ViswaV has quit IRC | 15:52 | |
*** jmontemayor has quit IRC | 15:56 | |
*** Barker has joined #openstack-trove | 15:59 | |
*** jmontemayor has joined #openstack-trove | 16:00 | |
*** jmontemayor has quit IRC | 16:22 | |
*** jmontemayor has joined #openstack-trove | 16:32 | |
*** fghaas has quit IRC | 16:33 | |
openstackgerrit | Mat Lowery proposed a change to openstack/trove: Test restore full and restore incremental https://review.openstack.org/73736 | 16:37 |
*** harlowja has joined #openstack-trove | 16:49 | |
*** khyati has joined #openstack-trove | 17:01 | |
amcrn | juice: no worries, happy monday :) | 17:03 |
amcrn | cp16net: be back in a bit, will ping you | 17:03 |
*** amcrn has quit IRC | 17:04 | |
*** Ranjitha has joined #openstack-trove | 17:11 | |
*** michael-yu has joined #openstack-trove | 17:12 | |
*** eghobo has joined #openstack-trove | 17:23 | |
*** amcrn has joined #openstack-trove | 17:27 | |
*** Ranjitha has quit IRC | 17:30 | |
*** SnowDust has joined #openstack-trove | 17:31 | |
*** demorris has quit IRC | 17:36 | |
*** yogesh has joined #openstack-trove | 17:45 | |
*** yidclare has joined #openstack-trove | 17:46 | |
*** demorris has joined #openstack-trove | 17:47 | |
*** SnowDust has quit IRC | 17:47 | |
*** tanisdl has joined #openstack-trove | 17:55 | |
*** robertmyers has joined #openstack-trove | 17:57 | |
*** Ranjitha has joined #openstack-trove | 18:01 | |
*** Ranjitha has quit IRC | 18:02 | |
*** Ranjitha has joined #openstack-trove | 18:03 | |
*** fghaas has joined #openstack-trove | 18:07 | |
grapex | o/ | 18:11 |
grapex | Sorry, I just like making this emoticon | 18:12 |
vipul | \o | 18:12 |
amcrn | \o/ | 18:12 |
*** SnowDust has joined #openstack-trove | 18:12 | |
amcrn | Kicking it off with https://wiki.openstack.org/w/index.php?title=Trove/Blueprints/Trove-v1-MySQL-Replication | 18:12 |
* vipul reading | 18:13 | |
vgnbkr | o/ | 18:13 |
SlickNik | o/ | 18:13 |
amcrn | the key point in that blueprint is: "Phase I implementation, targeted for the Juno release, will implement Use Case A core functionality and the MySql datastore replication support. Replication support for other datastores beyond MySQL are beyond the scope of this blueprint and will be covered elsewhere." | 18:13 |
yogesh | o/ | 18:13 |
vipul | amcrn: +1 | 18:14 |
vgnbkr | I filled in some of the other sections, but probably got some of the details wrong. Feel free to point such deficiencies out. | 18:14 |
vipul | we shoudl first focus on just MySQL, and let other datastores follow | 18:14 |
grapex | I wonder, should phase 1 be it's own blueprint? It's it possibly to nest them? | 18:14 |
vipul | grapex: I think doug was going to file a separate BP as well | 18:15 |
SlickNik | I agree with the scope and use-cases mentioned in the bp. | 18:15 |
grapex | vipul: Ok, I support this then if that's doug's intent | 18:15 |
amcrn | vipul: i believe this *is* the separate bp | 18:15 |
vipul | vgnbkr: I think there will be some configuration changes | 18:15 |
amcrn | "There should be no changes to configuration files required." => this shouldn't be correct, because there was a verbal consensus that the server_id should be removed from configuration-groups and handled by trove to avoid conflicts | 18:15 |
vipul | amcrn: well i mena on LP | 18:15 |
*** iartarisi has quit IRC | 18:15 | |
vgnbkr | grapex: yes, it could be broken out, but my thought would be to stick with baby steps for now. | 18:16 |
amcrn | ah, well "configuration file" vs. "configuration group"; but anyway, my point above is still valid | 18:16 |
abramley | The intent was to set the stage for the full feature - but then have this blueprint focus on the phase 1 mysql deliverables | 18:16 |
SnowDust | vipul : focus on mysql means atleast the configs should be pluggable .. so that it may be adopted freely | 18:16 |
vipul | amcrn grapex SlickNik https://blueprints.launchpad.net/trove/+spec/replication-v1 | 18:16 |
vgnbkr | amcrn: I didn't realize that server_id was already there - I thought it was to be added, and we agreed not to add it. my mistake. | 18:16 |
vipul | SnowDust: Sure, it shoudl be built with the realization that there are other datastores that will support it | 18:17 |
vipul | Can we talk about Use Case 1? | 18:17 |
grapex | Let's | 18:17 |
yogesh | I am sorry if I missed out...has the discussion on container vs join approach done...? | 18:17 |
vipul | 4. When master fails, a slave can be chosen to be promoted to new master, with other slaves switched to follow new master | 18:17 |
vgnbkr | vipul: Yes, I think the idea is to concentrate on MySQL, but keep the others in the back of our minds. | 18:17 |
vipul | does this mean some promotion API? or some automated way | 18:17 |
*** yidclare has quit IRC | 18:18 | |
vipul | yogesh: this is the BP review meeting.. the container vs join isn't this yet | 18:18 |
grapex | I feel like A4 is phase 2 | 18:18 |
amcrn | agreed | 18:18 |
yogesh | vipul: sure...thanks | 18:18 |
grapex | Or should be- A1-A3 seems like a good phase 1 so we can get some traction on this | 18:18 |
vipul | well i think we should support an API call for slave promotion.. | 18:19 |
amcrn | phase 1 should allow you to break-off a slave so that it becomes an isolated instance | 18:19 |
vgnbkr | So is the intention that the user would use root access to do A4 manually? | 18:19 |
grapex | amcrn vipul: Agreed | 18:19 |
SlickNik | amcrn: I think that's explicitly called out in 5 | 18:19 |
amcrn | ah, good catch SlickNik: so yeah, i'd vote to strike 4 from phase 1 | 18:19 |
*** yidclare has joined #openstack-trove | 18:19 | |
vipul | SlickNik amcrn is that the same thing as 'promote a slave' via API ? | 18:20 |
vipul | 5 that is | 18:20 |
amcrn | vipul: well, rds uses "promote" in this context, so yes | 18:20 |
abramley | so it sounds like move #4 out of the phase 1 deliverables | 18:21 |
vgnbkr | So we're agreed on A6 - any existing site, even with data, can be made into a master? | 18:21 |
SlickNik | vipul: depends on the definition of "promote" I guess. It's RDS' definition, but not what 4 lays down as the definition of "promote" (since that entails switching other slaves over as well). | 18:21 |
amcrn | if "promote" means that a slave becomes a standalone instance, then yes, we should support a promote operation for a slave | 18:22 |
vgnbkr | A4 is different that RDS' 'promote'. The latter is like A5. | 18:22 |
vipul | So to make things simpler.. if we enforced a replication depth initially == 1 | 18:22 |
abramley | I think we would want #5 and #6 in order to transition sites into and out of replication | 18:22 |
vgnbkr | vipul: by which you mean no slaves of slaves? | 18:22 |
vipul | vgnbkr: yes, then 4 isn't an issue correct? | 18:23 |
vipul | or is this not a depth issue | 18:23 |
SlickNik | I don't think it's a depth issue. | 18:23 |
amcrn | +1 SlickNik | 18:23 |
vgnbkr | A4 is still an issue, just a nomenclature issue. It can be moved to phase II, but that means admins will need to do that manually. | 18:23 |
cp16net | what up party people | 18:23 |
vipul | ok yea i think i understand | 18:24 |
SlickNik | It's talking about switching other slaves that were replicating from the busted master to now replicate from the promoted slave. | 18:24 |
amcrn | vgnbkr: correct | 18:24 |
vipul | SlickNik: so if we implemented only 5, the other slave would potentially be just dangling | 18:24 |
grapex | Request to cores: can we move talk of the guest agent upgrade to another time? I should be free 1:30 onward PST. | 18:25 |
SlickNik | Yes, the user would have to manually switch slaves over. | 18:25 |
vipul | got it | 18:25 |
vgnbkr | A5 is detaching a slave - all the other slaves would continue to work. | 18:25 |
vipul | vgnbkr: ok makes sense.. | 18:25 |
vipul | 7. All slaves should be in the same zone. (is this necessary or desired?) | 18:26 |
vgnbkr | If the master was detached, or otherwise became unavailable, the slaves would be left dangling. | 18:26 |
amcrn | for #7, that just means trove remains as-is in respect to the availability_zone behavior, no changes correct? | 18:26 |
vgnbkr | I take it we should disallow the master to be 'detached'/promoted? | 18:26 |
amcrn | vgnbkr: yes | 18:26 |
vipul | amcrn: I think we want to ideally place them on differnt AZs if multiple AZs are available | 18:27 |
SlickNik | I don't think we care about 7. | 18:27 |
amcrn | vipul: correct, but that should work as-is today | 18:27 |
*** tattabbum has quit IRC | 18:27 | |
amcrn | assuming you have multiple az's in the same nova deployment | 18:27 |
vgnbkr | Multiple zones is important, but the bp is suggesting that that is a different use case. | 18:27 |
grapex | SlickNik: Agreed | 18:27 |
vipul | amcrn: sure.. for V1, i think we need to look at actually dong that scheduleing for the user in future revs | 18:27 |
amcrn | vipul: totally agree | 18:28 |
vgnbkr | I don't think the intention is to do anything to disallow multi AZ deployment, just that it's not a requirement for that use case. | 18:29 |
SlickNik | So what I'm hearing is that we're good with all points except 4, and 7 (we might address these incrementally in a future bp, but not for v1) | 18:29 |
*** denis_makogon_ has joined #openstack-trove | 18:29 | |
vipul | SlickNik: +1 | 18:29 |
amcrn | +1 | 18:29 |
vgnbkr | So you are saying that multi AZ deployment is a requirement for phase I? | 18:29 |
vipul | vgnbkr: so Trove alread supports specifying the AZ in a create call, we'd just leave it as-is | 18:30 |
vipul | vgnbkr: if the user specified separate AZs for the two instances, then they get that.. if they don't.. they dont | 18:30 |
vipul | so do we vote ? | 18:31 |
vgnbkr | vipul: OK, but could there potentially be issues in a multi AZ deployment. If that's a requirement, then we would need to address that. | 18:31 |
grapex | Sorry guys, I've got to go. Something's come up. | 18:31 |
abramley | amcrn - previously we talked about putting status of replication into the /topology requiest - do you think we need that in this phase and to specifically call it out here ? | 18:32 |
vipul | grapex please yay or nay before you go :) | 18:32 |
*** grapex has quit IRC | 18:32 | |
amcrn | abramley: the slight variations of schema are tbd, but the general approach is the same | 18:32 |
vgnbkr | abramley: I think that's an implementation issue for the spec doc. | 18:32 |
SlickNik | grapex: no worries, we'll go through the bps and fill you in when you get back. :) | 18:32 |
amcrn | one thing i don't see addressed is the level of visibility of the replication to (1) the user and/or (2) the admin | 18:33 |
amcrn | so, should mgmt-show have information about replication-lag | 18:33 |
cp16net | vipul: grapex said yay if not accepting A4 and A7 now | 18:33 |
abramley | amcrn - that is sort of where I was heading | 18:33 |
*** denis_makogon has quit IRC | 18:33 | |
*** denis_makogon_ is now known as denis_makogon | 18:33 | |
vipul | cp16net: cool | 18:33 |
amcrn | or should the user be able to see it? or should the user see some general level of "health" of replication in phase 1? | 18:33 |
denis_makogon | o/ | 18:34 |
vgnbkr | amcrn: I thought we had discussed that and the answer was 'No' | 18:34 |
*** dmakogon_ has joined #openstack-trove | 18:34 | |
amcrn | vgnbkr: it's not recorded in the blueprint, so i'm confirming with everyone here ;) | 18:34 |
vipul | well ideally we would push up the replication status via guest agent heartbeats.. and use that info to affect the 'state' of the instance | 18:34 |
vgnbkr | amcrn: no worries - personally, I think there should be some indication, but the group seemed to think not. | 18:35 |
vipul | so it might say something like 'degraded', not necessarily show you the details | 18:35 |
abramley | vgnbkr - I agree - some support for state / heartbeat seems desirable in phase 1 | 18:36 |
vipul | or if it's broken.. then indicate so | 18:36 |
vgnbkr | vipul: should that be in the bp or the spec? | 18:36 |
amcrn | it's a high-level user-facing change, so some mention of it should be in the bp (imho) | 18:36 |
vipul | i don't personally think we ned that level of detail in BP.. maybe jsut call it out as a 1 liner | 18:36 |
SlickNik | vipul: +1. | 18:36 |
vgnbkr | vipul: will do. | 18:36 |
vipul | +2 approved | 18:37 |
SlickNik | I've added the comments (in italics) in the bp at https://wiki.openstack.org/wiki/Trove/Blueprints/Trove-v1-MySQL-Replication#Use_Case_Requirements | 18:38 |
SlickNik | I'm good with approving the bp, based on this. | 18:38 |
vgnbkr | SlickNik: thanks. | 18:38 |
*** fghaas has left #openstack-trove | 18:38 | |
vgnbkr | SlickNik: (for the edits) | 18:39 |
abramley | SlickNik - we will do a quick update to the doc and then move forward | 18:39 |
cp16net | nice | 18:39 |
SlickNik | abramley/vgnbkr: Feel free to reword them as you see fit. | 18:39 |
SlickNik | okay moving on? | 18:39 |
SlickNik | amcrn? | 18:40 |
amcrn | let's do it | 18:40 |
amcrn | since we're deferring on discussing guest-agent until tim comes back | 18:40 |
amcrn | https://blueprints.launchpad.net/trove/+spec/dbinstance-log ? | 18:41 |
amcrn | https://wiki.openstack.org/wiki/Trove/DBInstanceLogOperation | 18:41 |
denis_makogon | i'm in | 18:41 |
denis_makogon | some pre-history | 18:41 |
amcrn | a long time ago, in a galaxy far away | 18:42 |
denis_makogon | RDS has the same feature | 18:42 |
denis_makogon | since Trove is a Database as A service | 18:42 |
denis_makogon | and there are no possible ways to log in to the instance | 18:42 |
denis_makogon | and no way to take a look at database log files | 18:43 |
denis_makogon | directly | 18:43 |
denis_makogon | my suggestion was to give to the end-user and ability to pull logs from instance through Swift | 18:43 |
denis_makogon | iteration one was implemented | 18:44 |
denis_makogon | links you could see inside the BP | 18:44 |
denis_makogon | next iteration will give an ability to retrieve N lines of given log file | 18:44 |
denis_makogon | it's something like polling | 18:45 |
denis_makogon | the question is how to store or not store retrieved lines from log | 18:45 |
denis_makogon | in terms of iteration 2 | 18:46 |
amcrn | well, we can defer that conversation when phase 2's bp presents itself | 18:46 |
amcrn | let's focus on phase 1 | 18:46 |
denis_makogon | sounds reasonable | 18:46 |
denis_makogon | ok | 18:47 |
*** freyes has quit IRC | 18:47 | |
denis_makogon | any questions about Phase 1 ? | 18:47 |
amcrn | does someone have any high-level questions about phase 1, otherwise i'll kick it off with a few | 18:47 |
vgnbkr | Can I ask a point of order? Is discussion open to everyone, or just core and stakeholders? | 18:47 |
SlickNik | everyone | 18:47 |
SlickNik | go for it vgnbkr | 18:47 |
vgnbkr | Ok, thanks. | 18:47 |
vgnbkr | So if the logs go into swift, how are they managed? Who deletes them? | 18:48 |
denis_makogon | vgnbkr, we don't manage them | 18:48 |
denis_makogon | vgnbkr, only user responsible for those files | 18:49 |
vgnbkr | So why not just download the files to the user's browser, then? Save blowing out the swift repository with temporal data. | 18:50 |
denis_makogon | vgnbkr, user owns the container that created in Swift, so user can delete them | 18:50 |
vgnbkr | So what is the reason for putting them is Swift? | 18:51 |
vgnbkr | s/is/in/ | 18:51 |
robertmyers | vgnbkr: the api sends an rpc message to the guest which performs the actions | 18:51 |
denis_makogon | vgnbkr, if we're working through Horizon dashboard, you could go to the Containers pannel and download them manually | 18:51 |
robertmyers | so the api can not respond directly with the content | 18:52 |
denis_makogon | vgnbkr, Swift is a consistent storage for all possible data across cloud | 18:52 |
vgnbkr | It just seems onerous to expect the user to manage their log files. | 18:52 |
amcrn | denis_makogon: is there a plan to limit the # of log files, as is done with backups? | 18:53 |
denis_makogon | amcrn, i think no/ | 18:53 |
denis_makogon | amcrn, unless there's huge need in quotas | 18:54 |
denis_makogon | amcrn, logs are not the backups | 18:54 |
amcrn | lol, of course they're not | 18:54 |
amcrn | the reason i ask is that by having a limit, users are indirectly influenced to prune | 18:54 |
denis_makogon | amcrn, i mean they have different signification | 18:54 |
cp16net | denis_makogon: are the files streamed to swift as they are written to or uploaded each time requested? | 18:54 |
denis_makogon | cp16net, eachtime | 18:55 |
denis_makogon | cp16net, they have timestamp in their name | 18:55 |
cp16net | ok that was my next question | 18:55 |
denis_makogon | cp16net, hope i answered | 18:56 |
denis_makogon | any other questions ? | 18:56 |
amcrn | denis_makogon: how will the rotation of the logs be handled across datastores, and how will this be presented to the user? | 18:56 |
amcrn | example: user asks for a log, then has no idea why it doesn't include Day X | 18:57 |
cp16net | or time X | 18:57 |
amcrn | correct | 18:57 |
cp16net | because the logs get rotated too quickly.. | 18:57 |
denis_makogon | amcrn, didn't thought about that | 18:57 |
SlickNik | denis_makogon: Or does the user have an option to ask for a log file from time X to Y? | 18:58 |
denis_makogon | amcrn, i guess that if user can enable log rotation, probably he'll know that logs will contain only the part from X to Y | 18:59 |
denis_makogon | SlickNik, this could be as part of iteration 3 | 18:59 |
amcrn | denis_makogon: well, that's not true; i believe mysql auto log rotates right now, and without an api for the user to ask what the rotation schedule is, they have no idea | 18:59 |
amcrn | so unless we agree on a single rotation schedule for all datastores, then publish this in the docs, you'll have to present *something* | 19:00 |
amcrn | even then, users asking for "log3.tar.gz" because they've done napkin math to figure out which one has the data they're looking for seems a bit janky | 19:00 |
SlickNik | denis_makogon: How does this work with different log locations? (eg. As a provider, I deploy a different my.cnf specifying a different log location) | 19:00 |
amcrn | SlickNik: good point, was just about to bring that up. in short, i think we'll have to remove the ability to change the log location in configuration-groups | 19:01 |
denis_makogon | SlickNik, for now it looks like https://review.openstack.org/#/c/64302/27/trove/dbinstance_log/models.py | 19:02 |
amcrn | yeah, aka remove the ability to change it in the configuration-group | 19:02 |
denis_makogon | amcrn, yes i thought about that, just wanted to ask it | 19:02 |
vgnbkr | Should the feature be up-scoped to provide somethnig beyond retrieving log files? Something like including log data in the heartbeat so that the user can see log entries in delayed-time? | 19:02 |
amcrn | vgnbkr: not sure i follow | 19:03 |
denis_makogon | vgnbkr, now its orientied only for logs, and i think that there's no reason for polling something else than logs | 19:04 |
vgnbkr | On every heartbeat, include changes to the long since the last heartbeat. Save a single log for each instance in (say) Swift. User could retrieve that, or watch near-live in Horizon. | 19:04 |
vgnbkr | s/long/log/ | 19:04 |
denis_makogon | vgnbkr, heartbeat happens each 3 sec | 19:05 |
vgnbkr | denis_makogon: yes, I'm just referring to logs. | 19:05 |
vgnbkr | denis_makogon: right, hence 'near-live'. | 19:05 |
amcrn | vgnbkr: by "changes", do you mean the actual incremental content, or a summary of said content, or ? | 19:05 |
cp16net | sounds like a period task or scheduled task to pump out the logs | 19:06 |
denis_makogon | vgnbkr, i don't think we need to pull to server side part of log each time heartbeat happens | 19:06 |
vgnbkr | amcrn: Really I'm just wondering if we need something more than just retrieving log files. | 19:06 |
denis_makogon | vgnbkr, i guess no | 19:06 |
vgnbkr | denis_makogon: I was thinking a push, not a pull. | 19:06 |
denis_makogon | vgnbkr, feature designed to push logs to Swift, instead of Trove server-side | 19:07 |
SlickNik | vgnbkr: Personally I think that might be a tad overkill. But it really depends on the use-case. denis_makogon: What's the specific use case(s) we're trying to address with this? | 19:08 |
amcrn | vgnbkr: aren't you really referencing [11:54:58] <cp16net> denis_makogon: are the files streamed to swift as they are written to or uploaded each time requested? | 19:08 |
denis_makogon | SlickNik, use-case is an audit | 19:08 |
denis_makogon | SlickNik, i worked with database security audit | 19:08 |
juice | denis_makogon: rather than use swift and a custom log file manager, have you considered using syslog to push the files to a remote server of the user's choice? | 19:08 |
denis_makogon | SlickNik, but there saw an ability to access the instance via SSH | 19:09 |
vgnbkr | My alternate suggestion was really just an idea of the top of my head. My real point was to suggest that just pulling log files adhoc might not be enough. | 19:09 |
juice | they could even setup another vm that is their central log server and then there they can do what they want with them and have full access to the history? | 19:09 |
vgnbkr | ju | 19:09 |
vgnbkr | juice: nice idea | 19:09 |
cp16net | i was just thinking about if you could always have the log files in swift up to date. | 19:10 |
amcrn | juice: with swift being an option of a remote-server? | 19:10 |
juice | plus you'd get all the headache of log ration taken care of by you | 19:10 |
amcrn | or in replacement of. | 19:10 |
denis_makogon | amcrn, + Swift is enough | 19:10 |
juice | amcrn: well maybe from the central log server you can set that up yourself or something | 19:10 |
juice | amcrn: I was thinking in replacement of mostly | 19:10 |
cp16net | something along the lines of scheduling the log files to append each time the task is fired | 19:10 |
juice | db instance 1-100 send your log files to syslog server 10.0.0.101. | 19:11 |
juice | ssh into 10.0.0.101 and go to town | 19:11 |
denis_makogon | cp16net, we could talk about appending log data inside the iteration 2 or 3 | 19:11 |
cp16net | ok | 19:11 |
juice | putting log files in swift is not going to be THAT convenient for users to manage | 19:11 |
SlickNik | denis_makogon: wouldn't you want _all_ the logs in the case of an audit? Will a one-time API call to copy existing logs over work? | 19:12 |
robertmyers | juice: syslog is a good idea | 19:12 |
SlickNik | denis_makogon: And how are you going to stitch log-files together to get a complete view? | 19:12 |
robertmyers | juice: that could work with loggly too | 19:12 |
denis_makogon | SlickNik, yes it is | 19:12 |
denis_makogon | SlickNik, audit features allows to analyze set of files | 19:13 |
juice | robertmyers: indeed: seems to keep the options open for more robust logging in a complex deployment (ie. not just one vm) | 19:13 |
denis_makogon | juice, but for now it's too complicated | 19:14 |
robertmyers | denis_makogon: how is syslog too complicated? | 19:14 |
juice | denis_makogon: I would disagree - building our own logging manager is going to be much more so | 19:14 |
*** tattabbum has joined #openstack-trove | 19:14 | |
juice | and that wheel has already been invented | 19:14 |
denis_makogon | juice, what do you mean under "custom log manager" ? | 19:15 |
juice | denis_makogon: perhaps log shipper is a better description | 19:15 |
openstackgerrit | Ranjitha Vemula proposed a change to openstack/trove-integration: MySQL 5.6 disk-image-builder element https://review.openstack.org/79413 | 19:15 |
denis_makogon | juice, there's no log sniphers inside the trove | 19:15 |
juice | denis_makogon: aren't we talking about building one now? | 19:16 |
denis_makogon | juice, noone analyze the data | 19:16 |
denis_makogon | juice, no | 19:16 |
denis_makogon | juice, could you please take a look at BP an WIKI page | 19:17 |
SlickNik | Okay, I get a feeling we're rat-holing a bit too much on this. | 19:17 |
juice | denis_makogon: the basic goal is to get the data off of the db instance since the user does not have access to it. that has been solved with syslog already | 19:17 |
juice | slicknik: you mean rabbit hole | 19:17 |
juice | :) | 19:18 |
juice | carry-on - just putting the idea out there | 19:18 |
amcrn | in general, i think the real use-case needs to be explained more. taking the "audit" scenario, to SlickNik's point, wouldn't you want all of the logs all of the time? outside of an audit scenario, how often would a user actually request to see a log? without hard answers (or gut feels) for those questions, ascertaining the right approach becomes more difficult. | 19:19 |
SlickNik | lol, I'm fully on-board with the intent of this bp; but I'm not convinced that the current proposed implementation addresses the use-case it's trying to solve. | 19:19 |
denis_makogon | juice, since inside the could we're only relaying only on existing services - the only way for Trove is to store logs to existed storage | 19:19 |
amcrn | my gut tells me juice's approach makes more sense | 19:19 |
SlickNik | I mean, I can see that this implementation may be okay if, say I set some config param that messed something up, and I want to take a quick look at the current logs. | 19:20 |
denis_makogon | amcrn, we cannot relay to "syslog" server that might exist or not | 19:21 |
denis_makogon | amcrn, the only way is to use Swift and only | 19:21 |
SnowDust | denis_makogon, +1 | 19:21 |
juice | denis_makogon: true, you would need to setup an additional vm or pay for swift storage or both | 19:21 |
denis_makogon | juice, Swift is our everything | 19:22 |
juice | denis_makogon: swift is definitely a good repository to store old archived log files | 19:22 |
SlickNik | But if we're talking of gathering logs for a database security audit , I'm not sure this would be an effectual approach. | 19:22 |
amcrn | but current, in-flight logs, not so much | 19:22 |
juice | amcrn: exactly - perhaps the last weeks worth | 19:23 |
amcrn | SlickNik: 100% agree with your assessment, so to borrow a phrase from our friend dougshelley66: what problem are we actually trying to solve | 19:23 |
*** ViswaV_ has quit IRC | 19:23 | |
amcrn | so denis_makogon, i think you need to update the blueprint with said information | 19:23 |
denis_makogon | amcrn, the proble is that user cannot access log files of database that is the backend of his application | 19:23 |
openstackgerrit | Ranjitha Vemula proposed a change to openstack/trove-integration: MySQL 5.6 disk-image-builder element https://review.openstack.org/79413 | 19:24 |
denis_makogon | amcrn, audit is the next approach | 19:24 |
denis_makogon | amcrn, perfomance tuning is the third approach (only logs could tell how database feels on currect hardware) | 19:25 |
amcrn | i think we can summarize that: the actual use-case(s) being solved for need to be elaborated on, including the expected access patterns (when, how often, etc.) and (2) a unified rotation strategy needs to be considered, or how to present to the user the rotation strategy in place so they can request for a log that's relevant to the timeframe they're interested in | 19:25 |
amcrn | without those, i'm of the opinion we're at an impasse. | 19:26 |
SlickNik | agreed. amcrn: +1. I'd vote that we defer the bp and revisit it later when we have more clarity around those data points. | 19:26 |
SnowDust | amcrn, juice, esp : can we do a quick discussion on this https://bugs.launchpad.net/trove/+bug/1288767 | 19:27 |
amcrn | not right now SnowDust, no. | 19:28 |
SnowDust | k .. will wait | 19:28 |
esp | I'll try to comment on this in a bit SnowDust :) | 19:28 |
amcrn | are we good to move on to the next blueprint? does someone else believe we need more information than the aforementioned to get clarity on the dblog blueprint? | 19:29 |
SnowDust | esp: welcome .. need more discussions anyway ;) | 19:29 |
denis_makogon | amcrn, lets move on | 19:30 |
SlickNik | sounds good to me. | 19:30 |
amcrn | https://blueprints.launchpad.net/trove/+spec/datastore-type-version-to-backp-model | 19:30 |
denis_makogon | Approach: avoid wrong restoring | 19:30 |
amcrn | yeah, aka handle the various problems seen @ https://bugs.launchpad.net/trove/+bug/1285876 | 19:31 |
denis_makogon | The way it'll be implemented: base backup model will be extended with two fields: type and version | 19:32 |
SlickNik | Question: We already have strategy as part of the backup model. Is that insufficient? (if so, I'm trying to understand why) | 19:33 |
denis_makogon | we should not relay on _deleted_ instances | 19:33 |
SlickNik | eg. If your cassandra instance is trying to backup with an innobackup strategy, that would point to a mis-configuration, right? | 19:33 |
denis_makogon | SlickNik, stragegy could be changed in conf file | 19:33 |
denis_makogon | SlickNik, but old backup could have old strategy | 19:34 |
cp16net | i dont see why we need both type and version when you all you really care about is the version | 19:34 |
cp16net | or can you restore a different version to a type? | 19:34 |
cp16net | err the same type to a different version? | 19:34 |
denis_makogon | SlickNik, example: backup (strategy: MySQLDump), but the guest works only with Xtrabackup stuff | 19:35 |
SlickNik | denis_makogon: can you clarify, I don't fully understand that example. | 19:36 |
denis_makogon | SlickNik, suppose we backuped the instance with xtrabackup strategy | 19:37 |
denis_makogon | SlickNik, and now we want to restore new one, but trove-api doesnt know anything about possible strategies | 19:37 |
denis_makogon | SlickNik, same for the trove-taskmanager - it doesn't know is the strategy correct or not | 19:38 |
vgnbkr | I don't see anything about strategies in the bp? Is it called something else? | 19:38 |
amcrn | i'm a bit lost as well, i thought this bp was solving something else entirely | 19:38 |
SlickNik | Why does trove not know about the strategies? There's a mapping between the datastore and strategy types in the config file. | 19:39 |
denis_makogon | amcrn, the approach to avoud incorrent provisioning-restoring | 19:39 |
denis_makogon | *avoud | 19:39 |
denis_makogon | avoid | 19:39 |
amcrn | based on a datastore-version incompatibility, or a strategy incompatibility, or both? | 19:40 |
denis_makogon | SlickNik, by telling "config file" you mean cfg.py ? | 19:40 |
denis_makogon | amcrn, i guess datastore-version | 19:41 |
SlickNik | Yes, cfg.py or the sample conf. | 19:42 |
denis_makogon | SlickNik, we cannot relay on config file since backup_strategy is the config attribute is the part of guest config | 19:42 |
denis_makogon | SlickNik, i wouldn't suggest to duplicate options through all trove services | 19:43 |
SlickNik | But if you store this in the database, wouldn't you be duplicating the info in the models instead? | 19:44 |
vgnbkr | The bp says the purpose is to prevent the user from restoring a mysql backup to mongo. What would be wrong with that? Or are you just saying to prevent the error? Is the guest-agent not capable of detecting bad input? | 19:44 |
amcrn | vgnbkr: as of now, it doesn't detect it and the instance is perma-stuck in BUILD | 19:44 |
denis_makogon | SlickNik, we're not duplicating data inside trove backend | 19:45 |
*** Ranjitha has quit IRC | 19:45 | |
vgnbkr | So wouldn't it be better to fix the guest agent? | 19:45 |
denis_makogon | SlickNik, strict validation requires special attribute | 19:46 |
*** Ranjitha has joined #openstack-trove | 19:46 | |
denis_makogon | SlickNik, we have two options: 1) version attr in backup mode. 2) backup_strategy - inappropriate since instance provisioning will go deeper | 19:47 |
denis_makogon | SlickNik, we need to block restore call at API part | 19:47 |
SlickNik | denis_makogon: Tell me this. Should I be able to restore a backup taken from mysql (using innobackup) on percona instance (using innobackup)? | 19:47 |
vgnbkr | It seems like it would be good to prevent the user from restoring to the wrong db, but there would be cases where the user validly wishes to restore to a different instance. We would need to allow all of those cases, no? | 19:47 |
vgnbkr | What SlickNik said :-) | 19:48 |
denis_makogon | SlickNik, we talked about this, it's very complicated since it would not work from time to time | 19:49 |
denis_makogon | SlickNik, that is why i suggested to have datastore type and version | 19:49 |
vgnbkr | denis_makogon: as long as the user got a reasonable error from the guest-agent, wouldn't that be OK? | 19:49 |
denis_makogon | SlickNik, user will at least try to create percona instane from mariadb backup | 19:49 |
denis_makogon | vgnbkr, we should not let the provisioning | 19:50 |
SlickNik | denis_makogon: When would it not work? | 19:50 |
vgnbkr | denis_makogon: why? | 19:50 |
denis_makogon | vgnbkr, quotas | 19:51 |
SlickNik | denis_makogon: I see what you are saying about error'ing out eagerly in the API call. If we need to expose the mapping in the API conf, for this purpose, I think that's an acceptable solution. | 19:51 |
*** Ranjitha has quit IRC | 19:51 | |
denis_makogon | SlickNik, my guess is that it would not work when migrating from one major version to another | 19:51 |
denis_makogon | SlickNik, i'd better keep trove.conf clean | 19:52 |
denis_makogon | SlickNik, backup strategies is the part of the guest, not he APU | 19:52 |
denis_makogon | *trove-api | 19:52 |
denis_makogon | lets do not mix everything | 19:53 |
SlickNik | Wait so you want to error out eagerly in the API, but not have it be part of the API? :) | 19:54 |
denis_makogon | SlickNik, yes | 19:55 |
denis_makogon | Something like "backup doesn't fit" | 19:55 |
SlickNik | denis_makogon: But it does fit. What we're saying is that it might be completely valid to have one datastore backup restore to a _different_ datastore type. | 19:56 |
SlickNik | The limiting factor here isn't the datastore type / version, but the strategy, and that's configurable by the operator. | 19:57 |
*** decora has joined #openstack-trove | 19:57 | |
denis_makogon | SlickNik, so, you say that it'll be better to have same options inside the api service and the guest | 19:58 |
denis_makogon | i mean: | 19:58 |
SlickNik | If you wanted to move the map of datastore -> strategies to the db so it's not repeated in the conf, we can talk about that. | 19:58 |
SlickNik | But that's not what this bp is about. | 19:58 |
denis_makogon | [datastore] backup strategy= blah-blah | 19:58 |
vgnbkr | SlickNik: but with time, don't you think there might be multiple strategies per datastore? | 19:59 |
SlickNik | vgnbkr: That's a good point. We probably need a one-to-many mapping instead of a one-to-one mapping. | 20:00 |
*** ViswaV has joined #openstack-trove | 20:00 | |
denis_makogon | SlickNik, i wanted to avoid provisioning with inappropriate backups | 20:01 |
SlickNik | But I'm still unconvinced that we need to force restores such that they can only happen on a datastore that has the same type and version as the original backup. | 20:01 |
denis_makogon | SlickNik, and i'm searching the way how to o thi | 20:01 |
denis_makogon | *to do this | 20:01 |
vgnbkr | If the guest-agent is physically able to restore a dataset, shouldn't we allow it? In cases where it can't, shouldn't it be robust enough to give a reasonable error and recover itself? | 20:02 |
denis_makogon | vgnbkr, guest cannot recover itself on restoring | 20:02 |
denis_makogon | vgnbkr, because "i want server with _this_ data, not just the server" | 20:03 |
denis_makogon | SlickNik, then lets stick to the strategies | 20:03 |
vgnbkr | denis_makogon: I'm simply saying that the guest-agent shouldn't crash or corrupt the database due to detectably bad data sent to it. | 20:04 |
denis_makogon | vgnbkr, main approach to guest is to stay simple as it possible, so i'd rather block provisioning instead of getting decreased quota | 20:06 |
denis_makogon | and making guest smart enough to rule the world | 20:06 |
SlickNik | denis_makogon: agreed, I think it makes sense sticking to strategies. | 20:06 |
SlickNik | As the bp is written now, I don't think it makes sense to approve it. | 20:07 |
denis_makogon | SlickNik, what if backup model will have datastore_manager ? | 20:07 |
denis_makogon | nevermind, strategies is enough | 20:08 |
denis_makogon | i guess | 20:08 |
*** ViswaV has quit IRC | 20:08 | |
*** ViswaV has joined #openstack-trove | 20:09 | |
vgnbkr | denis_makogon: if `file <fn>` = 'xtradb' is pretty simple. | 20:09 |
*** ViswaV_ has joined #openstack-trove | 20:10 | |
SlickNik | Okay, I think that's all we have for bps for now. | 20:10 |
SlickNik | As a heads up, we'll chat about the guest upgrade bp once grapex is back. :) | 20:10 |
SlickNik | I think we need to be a bit more rigid on how much time we spend on each of these bps. | 20:10 |
SnowDust | ok .. seems i have some time here :) | 20:11 |
SnowDust | may i ? | 20:11 |
amcrn | SlickNik: agreed, some sort of timeboxing | 20:11 |
SlickNik | So perhaps for the next bp review meeting, I propose a time limit on each bp discussion? | 20:11 |
SlickNik | amcrn: +1 | 20:11 |
SnowDust | https://bugs.launchpad.net/trove/+bug/1288767 | 20:11 |
*** ViswaV__ has joined #openstack-trove | 20:11 | |
amcrn | SlickNik: let me think about it a bit more, and hit up you + the channel on some thoughts (on the # of blueprints, how much time per blueprint, avoiding trying to solve implementation specific details) | 20:12 |
amcrn | time to grab some lunch | 20:12 |
SnowDust | :D | 20:12 |
denis_makogon | there's another one | 20:12 |
SnowDust | https://bugs.launchpad.net/trove/+bug/1288767 | 20:12 |
SlickNik | SnowDust: yes, I'm reading it. | 20:13 |
*** Ranjitha has joined #openstack-trove | 20:13 | |
*** mattgriffin has quit IRC | 20:13 | |
*** ViswaV has quit IRC | 20:13 | |
SlickNik | Did you have a question? | 20:13 |
amcrn | SnowDust: feel free to talk about your bug right now, i'll re-read the chat logs after i stuff my belly | 20:13 |
SnowDust | SlickNik: tx ! | 20:13 |
SnowDust | so .. the reviewers are different on the commit | 20:13 |
SnowDust | 1st solution proposed in the bug details is favored by amcrn | 20:14 |
SnowDust | the last patchset ( approach) is after discussion with juice, esp | 20:14 |
*** ViswaV_ has quit IRC | 20:14 | |
SnowDust | but juice commented on more enhancement | 20:14 |
SnowDust | so i need to get to a resolution on this .. acceptable by all :) | 20:15 |
denis_makogon | SnowDust, again, i don't see the problem of extending you custom trove patch with another oslo group | 20:16 |
juice | denis_makogon: do you mean a config group? | 20:16 |
denis_makogon | juice, yes | 20:16 |
yogesh | dnis_makogon: i think having a fallback will make things generic from trove perspective | 20:17 |
denis_makogon | juice, SnowDust 's caught the problem of the custom trove | 20:17 |
yogesh | denis_makogon: i think having a fallback will make things generic from trove perspective | 20:17 |
esp | SnowDust: I think the underlying problem is that we expect to have a configuration value set (datastore.mount_point) but we did not make it clear anywhere. | 20:17 |
SnowDust | until the implementation is in cfg.py there is no way to keep running an experimental datastore | 20:18 |
denis_makogon | yogesh, fallbacks is not required, please patch you taskmanager | 20:18 |
denis_makogon | SnowDust, write the new oslo group | 20:18 |
denis_makogon | easy as hell | 20:18 |
yogesh | conceptually | 20:18 |
SnowDust | [DEFAULT] group in conf was so easy .. to keep this working | 20:18 |
juice | I thought someone shot down the default | 20:19 |
SnowDust | juice, u r right ! | 20:19 |
yogesh | if there is something required by each of the datastores, then why to have it per group, it can be within the default group | 20:19 |
denis_makogon | yogesh, no, again, no, because we have N datastores that have their own mount points | 20:20 |
denis_makogon | so, we don't need one mount point in whole trove deployments | 20:20 |
SnowDust | denis_makogon, cfg.py binding is a big NO too ! | 20:20 |
yogesh | and do we have more than one datastores available from a single api? | 20:20 |
denis_makogon | yogesh, yes | 20:20 |
denis_makogon | yogesh, cassandra, mongo, redis, couchbase | 20:21 |
esp | SnowDust: if you supply the correct configuration in your taskmgr.conf does this get you around the issue? | 20:21 |
*** grapex has joined #openstack-trove | 20:21 | |
denis_makogon | esp, oslo group need to be registered | 20:21 |
SnowDust | esp, no .. it will only take the route to the coded datastore groups in cfg.py | 20:21 |
esp | hmm... ok | 20:21 |
SnowDust | until ur experimental datastore group is part of config group in cfg.py .. u are not entertained | 20:22 |
yogesh | deni_makogon: what if 3 out of these 4 have the same mount poin config value | 20:22 |
esp | ah ok I understand now | 20:22 |
yogesh | i mean, why not get it handled like a base class | 20:22 |
juice | is there any enforcement to ensure that every custom group has this setting? | 20:22 |
denis_makogon | yogesh, take more precise look | 20:22 |
yogesh | denis_makogon: yeah sure, just trying to understand batter... | 20:22 |
denis_makogon | yogesh, https://github.com/openstack/trove/blob/master/trove/common/cfg.py#L286-L288 | 20:23 |
denis_makogon | yogesh, https://github.com/openstack/trove/blob/master/trove/common/cfg.py#L326-L328 | 20:23 |
denis_makogon | yogesh, https://github.com/openstack/trove/blob/master/trove/common/cfg.py#L346-L348 | 20:23 |
denis_makogon | yogesh, https://github.com/openstack/trove/blob/master/trove/common/cfg.py#L368-L370 | 20:23 |
denis_makogon | yogesh, https://github.com/openstack/trove/blob/master/trove/common/cfg.py#L388-L390 | 20:24 |
esp | I wish I would have looked at the cfg.py changes earlier, sorry about that. | 20:24 |
*** mattgriffin has joined #openstack-trove | 20:24 | |
SnowDust | esp, any difference after u saw :) | 20:24 |
denis_makogon | SnowDust, register new oslo group and be happy | 20:24 |
denis_makogon | SnowDust, that is so easy | 20:24 |
SnowDust | denis_makogon, give me a solution without touching the trove checkout | 20:25 |
esp | to me it seems like it would have been nice to abstract the OptGroup stuff out into the property file itself rather than cluttering up the code with every datastore we might support one day. | 20:25 |
yogesh | denis_makogon: thanks for the pointers... | 20:25 |
SnowDust | i had that working .. when it was config driven [DEFAULT] group option kept that running for me | 20:25 |
esp | not sure if oslo config supports this tho | 20:25 |
SnowDust | esp, thats the problem | 20:26 |
yogesh | i am just trying to think the harm or design-flaw the default config is causing | 20:26 |
denis_makogon | esp, oslo.config cannot do this | 20:26 |
SnowDust | even if u define your custom groups .. they need to be loaded by a change in cfg.py | 20:26 |
denis_makogon | esp, this is how paste-deploy works | 20:26 |
yogesh | it is keeping this reusable | 20:26 |
esp | SnowDust: and everyone, how do yo feel about allowing for a generic datastore Group in the cfg.py for the purpose of testing experimental datastores for now? | 20:26 |
*** decora has quit IRC | 20:26 | |
amcrn | unless you polymorphically loaded subheaders by requiring some sort of prefix | 20:27 |
SnowDust | esp, no harm .. i am back running with that | 20:27 |
denis_makogon | SnowDust, you're patching guestagent, then patch the trove | 20:27 |
SnowDust | denis_makogon, i dont patch | 20:27 |
SnowDust | i load it as a paste filter | 20:27 |
esp | denis_makogon: that but that's kind of gross to have to mess with the api-paste (if that is what you mean) | 20:27 |
SnowDust | bulls eye :) | 20:27 |
denis_makogon | SnowDust, you wrote your own manager i guess | 20:27 |
SnowDust | the way trove was designed | 20:28 |
SlickNik | amcrn: Yeah, I think that's the gist of the problem. | 20:28 |
denis_makogon | esp, oslo.config works over paste deploy lib | 20:28 |
amcrn | SlickNik: should be doable, although it has downsides | 20:28 |
*** grapex has quit IRC | 20:29 | |
yogesh | denis_makogon: what problem are we solving taking the setting out from default | 20:29 |
kevinconway | esp: wouldn't those simply be API/Acceptance tests that we put in tempest? | 20:29 |
*** ViswaV__ has quit IRC | 20:29 | |
openstackgerrit | Mat Lowery proposed a change to openstack/trove: Get service endpoints from catalog https://review.openstack.org/68015 | 20:29 |
*** grapex has joined #openstack-trove | 20:29 | |
*** ViswaV has joined #openstack-trove | 20:29 | |
amcrn | if we go back to the specific problem at hand: every single datastore should have mount_point defined in its config-opts, and relying on a "default" to handle the scenario in which an operator does something silly, is in my opinion, overreaching. | 20:29 |
SnowDust | amcrn: ok | 20:30 |
esp | kevinconway: sorta I think SnowDust wants to test a new datastore in a live env. I could be wrong though. | 20:30 |
amcrn | all of the current datastores have a defaulted value for mount_point in cfg.py | 20:30 |
*** ViswaV has quit IRC | 20:30 | |
amcrn | esp: right, but let's make sure we're not clumping all of these requirements together | 20:30 |
denis_makogon | yogesh, we're dealing with the problem of specific parameters that are differenet fro all datastores | 20:30 |
amcrn | so, to say that there's an issue with the current set of datastores as it relates to mount_point is incorrect | 20:31 |
kevinconway | esp: i mean in terms of generic tests, some thing simply executes api calls without caring about what's inside | 20:31 |
*** ViswaV has joined #openstack-trove | 20:31 | |
esp | amcrn: k | 20:31 |
yogesh | denis_makogon: but then how about abstraction of the ones which are common? | 20:31 |
denis_makogon | yogesh, they are already in default section | 20:31 |
amcrn | now if we're talking about how do you support another experimental datastore without checking out cfg.py: how are you adding a custom manager or custom logic without checkout out the code? | 20:31 |
amcrn | you can't, so i fail to see a real life scenario that requires you modifying cfg.py and nothing else | 20:32 |
esp | kevinconway: yeah dude, I think you are correct I was thinking of a way to allow for kind of dump datastore group that would help bypass the problem. | 20:32 |
SnowDust | amcrn: conf modifications .. is the only change | 20:32 |
SnowDust | amcrn: no *.py change | 20:32 |
amcrn | SnowDust: what's the specific, real life example | 20:32 |
denis_makogon | amcrn, ++ to impossible scenarious | 20:32 |
yogesh | mount_point is needed for all datastores | 20:32 |
openstackgerrit | Mat Lowery proposed a change to openstack/trove: Test restore full and restore incremental https://review.openstack.org/73736 | 20:32 |
amcrn | yogesh: yes, it's set already as a default in cfg.py | 20:32 |
denis_makogon | yogesh, we already have mount point for all supported datastores | 20:33 |
amcrn | i'm asking SnowDust for a real scenario in which he needs to modify cfg.py to support an experimental datastore, and absolutely modify no other .py files | 20:33 |
denis_makogon | amcrn, i curious to, SnowDust please tell | 20:33 |
SnowDust | ? | 20:33 |
SnowDust | real scenario is mysql_foo | 20:33 |
SnowDust | thats configured as default_datastore in trove.conf | 20:33 |
denis_makogon | what is "mysql_foo" | 20:34 |
denis_makogon | ? | 20:34 |
SnowDust | denis_makogon which u never tried | 20:34 |
denis_makogon | is it datastore manager ? | 20:34 |
SnowDust | denis_makogon, yes | 20:34 |
amcrn | SnowDust: let's avoid abstractions, I'd like to hear a real life example | 20:34 |
SnowDust | amcrn ? | 20:34 |
amcrn | MySQL 5.6 vs. 5.5? | 20:34 |
denis_makogon | SnowDust, then you need to have mysql/mysql_foo.py | 20:34 |
SnowDust | amcrn its mysql_foo version 1.1 | 20:34 |
openstackgerrit | Andrew Bramley proposed a change to openstack/trove: Improve Datastore Not Implemented exceptions and remove traceback from messages https://review.openstack.org/78054 | 20:34 |
amcrn | mysql_foo isn't based in reality, please explain the real problem you're trying to solve | 20:35 |
SnowDust | amcrn as i said | 20:35 |
amcrn | why do you need mysql_foo, what changed in the manager | 20:35 |
SnowDust | the datastore has the guestagent manager written in a separate package beautifully loaded by the guest.py | 20:35 |
SnowDust | but .. fails to function as cfg.py has no config group for mysql_foo | 20:35 |
denis_makogon | SnowDust, you need to keep in the codebase mysql_foo manager | 20:36 |
amcrn | i'm aware of your abstract; explain to me a real life example | 20:36 |
SnowDust | denis_makogon, i will never do that .. i dont want to touch trove code ( due to license) | 20:36 |
yogesh | Snowdust: is it only about mount_point or other setting as well? | 20:36 |
SnowDust | but i will keep developing my extension and have right to make profit .. :) | 20:36 |
SnowDust | yogesh, bare minimum mount point | 20:37 |
yogesh | Snowdust: why can't it just use it from default settings? | 20:37 |
SnowDust | yogesh: there is _no_ default mount point setting now | 20:37 |
SnowDust | thats the point of bug | 20:37 |
amcrn | as there shouldn't be | 20:37 |
esp | yogesh: I was under the impression that the code that is there now isn't making use of the default settings | 20:37 |
esp | amcrn: yeah it was removed during the refactor/intro of datastores | 20:38 |
*** ViswaV has quit IRC | 20:38 | |
*** ramashri has joined #openstack-trove | 20:38 | |
esp | and a default that would makes sense for all datastores probably doesn't exist | 20:38 |
SnowDust | esp, yes ! | 20:39 |
SnowDust | thats my point | 20:39 |
denis_makogon | SnowDust, you are not able to add new datastore manager by changing only the conf file =) | 20:39 |
denis_makogon | SnowDust, send you config files and actual code to your github acc | 20:39 |
amcrn | here let me do it for you SnowDust: I need mysql_foo as a new manager, because I need to be able to deploy MySQL vanilla with the manager as 'mysql', and then mysql_foo for my Facebook fork of MySQL that requires very different configuration values. By using CONF.datastore_registry_ext, i'm able to point mysql_foo to trove.guestagent.datastore.mysql.manager.Manager, meaning I have no other .py file changes. | 20:40 |
SnowDust | amcrn: slight modifications | 20:40 |
amcrn | that's the only scenario i can cook up that would necessitate your requirements | 20:40 |
amcrn | which would have the cfg.py problem | 20:41 |
SnowDust | mysql_foo is loaded from mysql_foo: mysql_foo.abc.def.ghi.manager.Manager | 20:41 |
SnowDust | a separate entity | 20:41 |
yogesh | esp: isee | 20:41 |
SnowDust | amcrn: i said the same thing | 20:41 |
amcrn | that's additional .py files | 20:41 |
esp | amcrn: nice are you doing a side project for facebook too ;) | 20:41 |
denis_makogon | amcrn, ++ =)) | 20:41 |
amcrn | so if you added additional .py files, i don't see why modifying cfg.py is so heinous | 20:41 |
amcrn | you've already effectively forked the project | 20:42 |
denis_makogon | agreed | 20:42 |
*** ViswaV has joined #openstack-trove | 20:42 | |
amcrn | you could theoretically write some code to polymorphically load various [datastore-<x>] subheaders in the CONF, but that seems extremely overkill given your very unique situation | 20:42 |
*** ViswaV_ has joined #openstack-trove | 20:43 | |
*** ViswaV has quit IRC | 20:43 | |
amcrn | plus, are you really going to deploy 'mysql' vanilla? just re-purpose mysql to use your facebook fork. | 20:43 |
SnowDust | amcrn: u understood my purpose | 20:44 |
amcrn | esp: nope, was just taking a sherlock stab in the dark ;) | 20:44 |
*** demorris has quit IRC | 20:44 | |
*** ViswaV_ has quit IRC | 20:44 | |
SnowDust | also, the changes how difficult they have made the generic solution into | 20:44 |
*** demorris has joined #openstack-trove | 20:44 | |
yogesh | amcrn: it can make things more flexible...polymorphically loading the config... | 20:44 |
SnowDust | and just a fallback can bring this back | 20:44 |
*** ViswaV has joined #openstack-trove | 20:44 | |
SnowDust | its either through falling back to /var/lib/mysql as the mount point for taskamanger | 20:45 |
amcrn | yogesh: that has quite a few downsides, one of which is ascertaining what parameters are required for datastores | 20:45 |
SnowDust | or, from the default group mount_point | 20:45 |
*** ViswaV has quit IRC | 20:45 | |
SnowDust | which may be configured in the taskmanager conf file | 20:45 |
yogesh | amcron: some of the configs are mandatory for all datastores... | 20:45 |
*** ViswaV has joined #openstack-trove | 20:45 | |
yogesh | amcrn: sorry for isspelling ur alias.. | 20:46 |
amcrn | SnowDust: you're blinded a bit by your particular scenario; having a default is bad in nearly every other circumstance. If you add the default, then if someone misconfigures a new datastore, they're going to fallback to /var/lib/mysql, which is completely nonsensical. | 20:46 |
yogesh | amcrn: i agree to an extent | 20:46 |
SnowDust | amcrn: thats why we have neutral values :) /var/lib/datastore | 20:46 |
SnowDust | than /var/lib/mysql | 20:46 |
amcrn | lol, you're abusing a feature to get the desired result | 20:46 |
amcrn | you need to fork cfg.py, plain and simple | 20:46 |
*** ViswaV_ has joined #openstack-trove | 20:47 | |
yogesh | Snowdust: i think u need to think more | 20:47 |
amcrn | unless someone writes a blueprint and a rough poc proving a polymorphic load would work, and wouldn't have detrimental side-effects | 20:47 |
SnowDust | amcrn, if it was cfg.cfg :) i shall had with no complains | 20:47 |
amcrn | and to juice point, we need to look into how to *mandate* that a particular conf has a value, vs. defaulting it and hoping the user doesn't accidentally set it to "" in the configuration file | 20:47 |
*** ViswaV_ has quit IRC | 20:47 | |
*** ViswaV_ has joined #openstack-trove | 20:48 | |
esp | amcrn: ++ SnowDust may want to fork his work | 20:48 |
SnowDust | finally: should we make cfg.py pluggable itself :D | 20:50 |
*** ViswaV has quit IRC | 20:50 | |
SnowDust | if thats .. to be tied to datastore implementation | 20:50 |
yogesh | for a setting which is mandatory for all the datastores...having a setting in default group gives an option to provide a single value which is applicable to all the datastores....in case those datastores are running behiand the same api... | 20:51 |
esp | SnowDust: hopefully we can put in some time to clean up the datastore config stuff. | 20:51 |
yogesh | thats the only use.. | 20:51 |
amcrn | SnowDust: so for now, no defaults. as a side project, you can look into what it means to make it pluggable. | 20:53 |
SnowDust | amcrn : yep .. it can be like the config templates | 20:54 |
*** radez is now known as radez_g0n3 | 20:56 | |
*** Ranjitha has quit IRC | 20:57 | |
yogesh | Snowdust: best solution will be to push your datastore implementation upstream :-) | 20:57 |
*** ViswaV_ has quit IRC | 20:58 | |
*** michael-yu has quit IRC | 20:58 | |
yogesh | i mean after all the due diligence | 20:58 |
SnowDust | yogesh: let me think ;) | 20:58 |
esp | yep, seems reasonable. I like the idea of config templates too. but I think folks will want to take a hard look at it. | 20:59 |
SnowDust | but i guess .. we can have a template driven datastore loading | 20:59 |
*** ramashri has quit IRC | 21:02 | |
*** Ranjitha has joined #openstack-trove | 21:02 | |
*** ViswaV has joined #openstack-trove | 21:03 | |
SnowDust | amcrn, denis_makogon : so i think settings.template shall be fine to think as alternative | 21:04 |
SnowDust | that will keep it pluggable | 21:04 |
SnowDust | but .. need to think how early it can be called :) | 21:05 |
*** amcrn has quit IRC | 21:06 | |
denis_makogon | It can't be the setting template | 21:07 |
SnowDust | denis_makogon, reasons ? | 21:07 |
*** michael-yu has joined #openstack-trove | 21:08 | |
denis_makogon | SnowDust, because its the base confi files | 21:08 |
denis_makogon | SnowDust, they cannot be template | 21:09 |
*** ramashri has joined #openstack-trove | 21:09 | |
*** Barker has quit IRC | 21:11 | |
*** amcrn has joined #openstack-trove | 21:12 | |
*** demorris has quit IRC | 21:17 | |
*** demorris has joined #openstack-trove | 21:20 | |
*** tanisdl has quit IRC | 21:23 | |
esp | denis_makogon: would be nice someday to have template driven way to configure datastores? not sure what consensus on that idea is but I like it :) | 21:24 |
denis_makogon | esp, need more clean elaboration for that | 21:25 |
esp | so, we'd have json/yaml or whatever file for each datastore | 21:25 |
esp | sort of a serialized version of the object that would be loaded into the code | 21:26 |
esp | that way, you could abstract that from living inside the code itself | 21:26 |
*** tanisdl has joined #openstack-trove | 21:28 | |
esp | makes sense denis_makogon? again I'm just thinking out loud. | 21:28 |
denis_makogon | esp, you could try to disgn it | 21:30 |
esp | lol | 21:30 |
esp | sure I can do a blue print or something | 21:31 |
esp | but I'm open to other suggestions as well | 21:31 |
esp | I just don't see shoving all the datastore implementations in code as maintainable way to go | 21:32 |
*** esp has left #openstack-trove | 21:37 | |
*** tanisdl has quit IRC | 21:37 | |
*** esp has joined #openstack-trove | 21:42 | |
*** robertmyers has quit IRC | 21:45 | |
openstackgerrit | Dan Nguyen proposed a change to openstack/trove: Removes extra initialization from config https://review.openstack.org/79467 | 21:48 |
*** demorris has quit IRC | 21:54 | |
*** pdmars has quit IRC | 22:12 | |
SnowDust | esp, i said in similar line | 22:12 |
*** rhodgin has quit IRC | 22:12 | |
esp | SnowDust: yep. seems like it would be a nice thing. I'm just hesitant to say it's going to fit with the rest of the code base and all. Probably need to see what code currently supports. | 22:14 |
SnowDust | :) | 22:14 |
SnowDust | will surely give some time to that | 22:15 |
*** datsun180b has quit IRC | 22:16 | |
*** SnowDust has quit IRC | 22:19 | |
*** mattgriffin has quit IRC | 22:27 | |
*** yogesh has quit IRC | 22:27 | |
*** thedodd has quit IRC | 22:35 | |
*** rhodgin has joined #openstack-trove | 22:36 | |
*** yidclare has quit IRC | 22:43 | |
*** denis_makogon has quit IRC | 22:54 | |
*** jmontemayor has quit IRC | 23:01 | |
*** Ranjitha has quit IRC | 23:05 | |
*** amcrn has quit IRC | 23:07 | |
*** ramashri has quit IRC | 23:10 | |
*** Ranjitha has joined #openstack-trove | 23:10 | |
*** amcrn has joined #openstack-trove | 23:10 | |
*** yidclare has joined #openstack-trove | 23:12 | |
*** yidclare has quit IRC | 23:14 | |
*** michael-yu has quit IRC | 23:18 | |
*** ViswaV has quit IRC | 23:20 | |
*** kevinconway has quit IRC | 23:20 | |
*** flaper87 is now known as flaper87|afk | 23:22 | |
*** ViswaV has joined #openstack-trove | 23:22 | |
*** tenaglia has quit IRC | 23:25 | |
openstackgerrit | Giuseppe Galeota proposed a change to openstack/trove: Trove Manual Installation guide https://review.openstack.org/78608 | 23:27 |
*** tattabbum has quit IRC | 23:28 | |
*** michael-yu has joined #openstack-trove | 23:31 | |
*** ViswaV has quit IRC | 23:32 | |
*** grapex has quit IRC | 23:39 | |
*** ViswaV has joined #openstack-trove | 23:41 | |
*** ViswaV_ has joined #openstack-trove | 23:42 | |
*** ViswaV has quit IRC | 23:42 | |
*** jcru has quit IRC | 23:54 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!