Friday, 2013-12-13

amcrncp16net demorris: just replied to the configuration-group mailing-list thread, fyi.00:05
*** grapex has quit IRC00:05
openstackgerritKhyati Sheth proposed a change to openstack/trove: Mask Database user's password in trove api log  https://review.openstack.org/5885800:06
*** dougshelley66 has quit IRC00:15
*** dougshelley66 has joined #openstack-trove00:24
*** openstackgerrit has quit IRC00:24
*** openstackgerrit has joined #openstack-trove00:24
*** reed has quit IRC00:28
*** dougshelley66 has quit IRC00:29
openstackgerritKhyati Sheth proposed a change to openstack/trove: Mask Database user's password in trove api log  https://review.openstack.org/5885800:30
*** demorris has quit IRC00:31
*** vipul is now known as vipul-away00:35
*** vipul-away is now known as vipul00:35
*** vipul is now known as vipul-away00:51
*** vipul-away is now known as vipul00:53
*** yogesh has joined #openstack-trove01:02
*** haomaiwang has joined #openstack-trove01:09
*** plodronio has quit IRC01:09
*** haomaiwang has quit IRC01:14
*** ashestakov has joined #openstack-trove01:17
*** ashestakov has quit IRC01:22
*** yogesh has quit IRC01:46
openstackgerritSteve Leon proposed a change to openstack/trove: Instance details view shows hostname (if it has it) or IP  https://review.openstack.org/6191001:53
*** greghill has joined #openstack-trove02:06
*** erkules_ has joined #openstack-trove02:22
*** erkules has quit IRC02:25
*** reed has joined #openstack-trove02:37
*** achampion has joined #openstack-trove02:43
*** plodronio has joined #openstack-trove02:47
*** plodronio has left #openstack-trove02:48
*** amcrn has quit IRC02:52
*** SushilKM has joined #openstack-trove02:57
*** haomaiwang has joined #openstack-trove03:11
*** haomaiwang has quit IRC03:16
SushilKMhey slicknik, hub_cap, vipul kindly review https://review.openstack.org/#/c/58834/    this updates troveclient readme to latest help and also updates typos in shell.py, this was earlier -1d for troveclient bug03:26
*** greghill has quit IRC03:37
openstackgerritSushil Kumar proposed a change to openstack/trove: Adds support for running PyPy,py33 tests with tox  https://review.openstack.org/6191704:06
*** haomaiwang has joined #openstack-trove04:11
SushilKMneeded help for re-running redwarf on https://review.openstack.org/#/c/61853/ it failed due to unexpected reasons :)04:11
*** SushilKM has quit IRC04:15
*** SushilKM has joined #openstack-trove04:17
*** jcooley_ has joined #openstack-trove04:17
*** SushilKM has quit IRC04:20
*** SushilKM has joined #openstack-trove04:21
*** SushilKM has quit IRC04:25
*** jasonb365 has joined #openstack-trove04:36
*** haomaiwang has quit IRC04:43
*** haomaiwa_ has joined #openstack-trove04:44
*** haomaiwa_ has quit IRC04:47
*** haomaiwang has joined #openstack-trove04:47
*** abramley has quit IRC04:57
*** jcooley_ has quit IRC05:06
*** jcooley_ has joined #openstack-trove05:15
*** AndroUser2 has joined #openstack-trove05:20
*** SergeyLukjanov has joined #openstack-trove05:27
*** shalini has joined #openstack-trove05:33
*** shalini has quit IRC05:34
*** reed has quit IRC05:36
*** erkules_ is now known as erkules05:45
*** jcooley_ has quit IRC05:54
*** jcooley_ has joined #openstack-trove05:54
*** jcooley_ has quit IRC05:59
*** jaishanker has joined #openstack-trove05:59
*** jcooley_ has joined #openstack-trove06:02
*** jcooley_ has quit IRC06:04
*** jcooley_ has joined #openstack-trove06:04
*** jaishanker_ has joined #openstack-trove06:11
*** ashestakov has joined #openstack-trove06:14
*** jaishanker has quit IRC06:14
*** ashestakov has quit IRC06:14
*** jcooley_ has quit IRC06:19
*** denis_makogon has joined #openstack-trove06:20
*** jasonb365 has quit IRC06:22
*** jaishanker_ has quit IRC06:26
*** SushilKM has joined #openstack-trove06:39
*** yogesh has joined #openstack-trove06:52
*** yogesh has quit IRC06:57
*** SergeyLukjanov is now known as _SergeyLukjanov07:05
*** _SergeyLukjanov has quit IRC07:06
*** yogesh has joined #openstack-trove07:06
*** AndroUser2 has quit IRC07:11
*** SergeyLukjanov has joined #openstack-trove07:19
*** yogesh has quit IRC07:41
*** SergeyLukjanov is now known as _SergeyLukjanov07:44
*** _SergeyLukjanov has quit IRC07:45
*** flaper87|afk is now known as flaper8708:06
*** yogesh has joined #openstack-trove08:10
*** SergeyLukjanov has joined #openstack-trove08:12
*** denis_makogon has quit IRC08:25
*** rongze has joined #openstack-trove08:29
*** yogesh has quit IRC08:42
*** yogesh has joined #openstack-trove08:46
*** yogesh has quit IRC08:46
*** yogesh_ has joined #openstack-trove08:48
*** nosnos has joined #openstack-trove08:55
*** yogesh has joined #openstack-trove08:55
*** yogesh_ has quit IRC08:56
openstackgerritJenkins proposed a change to openstack/trove: Updated from global requirements  https://review.openstack.org/6193508:56
*** rongze has quit IRC09:05
*** yogesh has quit IRC09:16
*** yogesh has joined #openstack-trove09:21
*** SergeyLukjanov has quit IRC09:22
*** rongze has joined #openstack-trove09:34
*** yogesh has quit IRC09:41
*** yogesh has joined #openstack-trove09:42
*** yogesh has quit IRC09:45
*** yogesh has joined #openstack-trove09:46
openstackgerritSushil Kumar proposed a change to openstack/trove-integration: Removes log generation parameters from EXTRA_OPTS  https://review.openstack.org/6194809:49
*** nosnos has quit IRC09:51
*** SergeyLukjanov has joined #openstack-trove10:10
*** yogesh has quit IRC10:14
*** yogesh has joined #openstack-trove10:15
*** flaper87 is now known as flaper87|afk10:33
*** yogesh has quit IRC10:34
*** yogesh has joined #openstack-trove10:34
*** yogesh has quit IRC10:39
*** rongze has quit IRC11:06
*** rongze has joined #openstack-trove11:38
openstackgerritDenis M. proposed a change to openstack/trove: Block delete instance while buckup  https://review.openstack.org/6196911:41
*** yogesh has joined #openstack-trove11:45
*** rongze has quit IRC11:49
*** yogesh has quit IRC11:49
*** rongze has joined #openstack-trove12:15
*** SergeyLukjanov is now known as _SergeyLukjanov12:24
*** _SergeyLukjanov has quit IRC12:25
*** dougshelley66 has joined #openstack-trove12:26
*** SergeyLukjanov has joined #openstack-trove12:33
*** abramley has joined #openstack-trove12:52
*** dmakogon_ is now known as denis_makogon13:02
*** pdmars has joined #openstack-trove13:05
*** OpenStack1 has joined #openstack-trove13:10
openstackgerritDenis M. proposed a change to openstack/trove: Block delete instance while buckup  https://review.openstack.org/6196913:17
openstackgerritIonut Artarisi proposed a change to openstack/trove: add sql_connection option to the sample config for guestagent  https://review.openstack.org/6198913:20
*** shakayumi has joined #openstack-trove13:43
*** OpenStack1 has quit IRC13:51
*** PradeepChandani has quit IRC13:57
*** rongze has quit IRC13:59
*** radez_g0n3 is now known as radez13:59
*** robertmyers has joined #openstack-trove14:18
*** SushilKM has quit IRC14:20
*** robertmyers has quit IRC14:20
*** robertmyers has joined #openstack-trove14:21
*** OpenStack1 has joined #openstack-trove14:21
*** russellb is now known as rustlebee14:25
*** flaper87|afk is now known as flaper8714:26
*** glucas has joined #openstack-trove14:35
*** rongze has joined #openstack-trove14:44
*** rnirmal has joined #openstack-trove14:44
*** abramley_ has joined #openstack-trove14:46
*** OpenStack1 has quit IRC14:46
*** abramley has quit IRC14:48
*** abramley_ is now known as abramley14:48
*** demorris has joined #openstack-trove14:48
*** shakayumi has quit IRC14:49
*** Flying_Bond has joined #openstack-trove14:52
*** Barker has joined #openstack-trove14:54
*** plodronio has joined #openstack-trove14:56
*** mrsnivvel has quit IRC14:56
*** vgnbkr has joined #openstack-trove14:57
*** jcooley_ has joined #openstack-trove15:00
*** jcru has joined #openstack-trove15:00
*** kevinconway has joined #openstack-trove15:03
*** shakayumi has joined #openstack-trove15:03
*** shakayumi has quit IRC15:08
*** shakayumi has joined #openstack-trove15:10
*** rongze has quit IRC15:11
*** Flying_Bond has quit IRC15:16
*** Flying_Bond has joined #openstack-trove15:17
*** datsun180b has joined #openstack-trove15:26
*** jasonb365 has joined #openstack-trove15:26
*** grapex has joined #openstack-trove15:28
*** greghill has joined #openstack-trove15:29
*** amytron has joined #openstack-trove15:30
*** Sushil_GL has joined #openstack-trove15:31
*** Sushil_GL has quit IRC15:31
*** rwsu has joined #openstack-trove15:31
*** Sushil_GL has joined #openstack-trove15:31
*** SushilGL has joined #openstack-trove15:36
*** jcooley_ has quit IRC15:37
*** SushilKM has joined #openstack-trove15:37
*** jcooley_ has joined #openstack-trove15:38
*** rongze has joined #openstack-trove15:39
*** Sushil_GL has quit IRC15:40
SushilKMhi hub_cap15:40
SushilKMur review required on https://review.openstack.org/#/c/58834/15:40
SushilKMhi slicknik, vipul, hub_cap, denis needed review on https://review.openstack.org/#/c/61948/ :)15:42
*** yidclare has quit IRC15:43
*** jcooley_ has quit IRC15:43
SushilKMthats for u hub_cap http://lonbronsonband.com/images/Friday-2.gif15:44
*** OpenStack1 has joined #openstack-trove15:49
*** rnirmal_ has joined #openstack-trove16:02
*** rnirmal has quit IRC16:02
*** rnirmal_ is now known as rnirmal16:02
*** shakayumi has quit IRC16:08
*** jcooley_ has joined #openstack-trove16:09
*** SnowDust has joined #openstack-trove16:11
*** rongze has quit IRC16:14
*** Flying_Bond has quit IRC16:15
*** Flying_Bond has joined #openstack-trove16:20
*** rongze has joined #openstack-trove16:21
*** jcooley_ has quit IRC16:21
*** jcooley_ has joined #openstack-trove16:21
*** jcooley_ has quit IRC16:25
*** OpenStack1 has quit IRC16:28
*** Flying_Bond has quit IRC16:37
*** openstackgerrit has quit IRC16:37
*** juice has quit IRC16:37
*** extrovert has quit IRC16:37
dougshelley66r0utemap16:38
*** OpenStack1 has joined #openstack-trove16:38
*** 17WAAKZAN has joined #openstack-trove16:38
*** Flying_Bond has joined #openstack-trove16:38
*** openstackgerrit has joined #openstack-trove16:38
*** juice has joined #openstack-trove16:38
*** extrovert has joined #openstack-trove16:38
*** haomaiwang has quit IRC16:39
*** SushilKM has quit IRC16:40
*** SergeyLukjanov has quit IRC16:43
*** haomaiwang has joined #openstack-trove16:44
*** OpenStack1 has quit IRC16:47
*** juice- has joined #openstack-trove16:49
*** SushilGL has quit IRC16:49
*** 17WAAKZAN has quit IRC16:49
*** Flying_Bond has quit IRC16:49
*** openstackgerrit has quit IRC16:49
*** juice has quit IRC16:49
*** extrovert has quit IRC16:49
*** juice- is now known as juice16:49
*** Flying_Bond has joined #openstack-trove16:51
*** SushilKM has joined #openstack-trove16:51
*** jcooley_ has joined #openstack-trove16:55
cp16netSushilKM: thats an awesome dancing garfield17:02
SnowDustwhats that cp16net ?17:02
cp16nethttp://lonbronsonband.com/images/Friday-2.gif17:02
cp16netheh17:02
SushilKMbtw noone seems here, big silence17:04
SnowDustcp16net .. where is it pointing to ?17:05
*** SushilKM has quit IRC17:08
SnowDustSushilKM :) lost his cool .. when everything was so cold :D  and quit17:09
SnowDustlets see .. if someone speaks in the silence17:09
*** haomaiwang has quit IRC17:11
cp16netyeah everyone must be hung over :-P17:12
SnowDustif anyone interested we are discussing the remaining to be done to implement tempest for trove at #openstack-ds-trove17:13
SnowDustkeeping it noise free ... here :)17:13
SnowDustall interested please join there17:13
kevinconwaySnowDust: that seems like you should have that conversation in here17:14
kevinconwaythat way channel logging can record it17:14
SnowDustok ..17:14
SnowDustlets do that no issues17:14
SnowDustam bringing the guys here17:14
greghillsince the bot didn't seem to announce it - https://review.openstack.org/#/c/62039/  for the datastore backwards compatibility bug.  review!  DO IT NOW!!117:15
*** SushilGL has joined #openstack-trove17:20
SnowDusthi all17:21
SnowDustdiakunchikov17:21
SnowDustFlying_bond17:21
hub_caphi back to this room?17:21
SnowDustLets restart the tempest progress17:21
SnowDustyes ..17:21
SnowDustkevinconway is missed out :)17:21
hub_cap<317:21
hub_caplets definitely not move rooms again :)17:21
SnowDustsure hub_cap17:21
*** openstackgerrit has joined #openstack-trove17:21
hub_capit happened for the horizon stuff too and its hard to track17:22
hub_capi had no idea what was goin on w it hehe17:22
diakunchikovI'm here ^)17:22
kevinconwayhub_cap: well doh. if i had known you were cool with convos in another room i wouldn't have said anything17:22
kevinconwayi figured you wanted everything logged17:22
SnowDustwe switched .. and then thought to invite more .. and lastly came back with our <3 here !17:22
*** SergeyLukjanov has joined #openstack-trove17:23
hub_caphehe17:23
hub_capkevinconway: i _want_ them all here17:23
SnowDustatleast in a week i saw hub_cap in public convos LOL !17:23
hub_capi said "lets def not move rooms again" kevinconway, i think maybe u misread me :p17:23
SnowDustok lets begin17:23
hub_capdo it17:23
SnowDusttwo part of the work17:23
Flying_BondHi all17:24
SnowDust1. cli tests ( all wrote ) https://review.openstack.org/#/c/61937/17:24
SnowDust2. api tests ( in progress)17:24
SnowDustnow .. we need additional overhauls to get trove in tempest17:24
SnowDustwhich are17:24
SnowDust1.  devstack changes17:24
diakunchikov2.api tests https://blueprints.launchpad.net/tempest/+spec/add-trove-api-tests and https://review.openstack.org/#/c/60254/17:25
SnowDust2. configuration change in tempest17:25
SnowDustdiakunchikov: thanks !17:25
diakunchikovwhat do u mean in configuration change in tempest17:26
*** OpenStack1 has joined #openstack-trove17:26
*** amcrn has joined #openstack-trove17:27
SnowDusti meant this : <dkranz> Flying_Bond: You need to add something for trove in the feature_enabled section of the tempest config17:27
hub_capand we have to make sure we have a valid image in trove/glance17:27
hub_capthats the work that slicknik is doing ^ ^17:27
hub_capits in a blueprint somewhere17:28
*** SushilKM has joined #openstack-trove17:28
SnowDustyes .. thats good17:28
SnowDustso we are separate from each others progress which is the best part :)17:28
SnowDustanything .. that is left out ? hub_cap ?17:29
SnowDustas it seems all the necessary breakup is being worked on ...17:29
*** Flying_Bond has quit IRC17:30
SnowDustanyone to ask or share ?17:30
hub_capnope thats all of it17:30
hub_capnow everyone get to work!!!!!!!!!!!!!!!!17:31
hub_caphehehehe17:31
*** Flying_Bond has joined #openstack-trove17:31
SnowDustyes .. sure ..17:31
SnowDustwill get it to level 1 very soon :)17:31
hub_capSnowDust: can u create a master blueprint in trove and make sure all the "pieces" that we talked about are dependencies on it?17:31
hub_capsorry, a master "tempest test" blueprint17:31
hub_capa epic :)17:31
SnowDustok .. will take the ownership of it .. its in my <3 now .. and diakunchikov / Flying_bond are supporting it in awesome way17:32
hub_capthx SnowDust17:33
openstackgerritSushil Kumar proposed a change to openstack/trove-integration: Fixes typos in project files  https://review.openstack.org/6205017:33
SushilKMhey hub_cap need ur review on this one, https://review.openstack.org/#/c/58834/17:35
*** OpenStack1 has quit IRC17:36
*** freyes has joined #openstack-trove17:36
*** Flying_Bond has joined #openstack-trove17:37
*** Flying_Bond has joined #openstack-trove17:39
*** Flying_Bond has quit IRC17:40
*** SnowDust has quit IRC17:40
*** freyes has quit IRC17:41
*** Flying_Bond has joined #openstack-trove17:41
*** Flying_Bond has quit IRC17:41
*** amytron has quit IRC17:42
*** Sushil_GL has joined #openstack-trove17:42
*** SushilKM has quit IRC17:43
*** reed has joined #openstack-trove17:43
hub_capdoes SushilKM just dissapear right after he speaks? lol17:44
*** Flying_Bond has joined #openstack-trove17:45
*** SergeyLukjanov_ has joined #openstack-trove17:48
*** SergeyLukjanov has quit IRC17:48
*** OpenStack1 has joined #openstack-trove17:50
*** SnowDust has joined #openstack-trove17:53
hub_capSnowDust: can u shoot me a link to it when u get it all set up so i can check it out17:54
SnowDustsure .. doing it17:59
*** demorris_ has joined #openstack-trove17:59
openstackgerritSushil Kumar proposed a change to openstack/python-troveclient: Updates README with new trove help options  https://review.openstack.org/5883418:00
*** demorris has quit IRC18:02
*** demorris_ is now known as demorris18:02
Sushil_GLHey hub_cap vipul slicknik, updated the commit message .... :)18:03
*** freyes has joined #openstack-trove18:04
*** harlowja has quit IRC18:05
*** SushilKM has joined #openstack-trove18:07
*** SushilGL has quit IRC18:08
*** Sushil_GL has quit IRC18:08
*** SushilKM has quit IRC18:09
*** SushilKM has joined #openstack-trove18:09
SushilKMHey hub_cap if u can review this one too https://review.openstack.org/6194818:12
amcrndenis_makogon: is ashestakov out for the day?18:14
*** kevinconway has quit IRC18:15
*** freyes has quit IRC18:16
*** amytron has joined #openstack-trove18:17
*** harlowja has joined #openstack-trove18:24
openstackgerritSteve Leon proposed a change to openstack/trove: Instance details view shows hostname (if it has it) or IP  https://review.openstack.org/6191018:25
*** yidclare has joined #openstack-trove18:27
*** yogesh has joined #openstack-trove18:31
*** rwsu has quit IRC18:32
*** OpenStack1 has quit IRC18:34
*** SushilKM has quit IRC18:38
*** rwsu has joined #openstack-trove18:40
*** SushilKM has joined #openstack-trove18:45
*** abramley has quit IRC18:54
*** abramley has joined #openstack-trove18:55
*** OpenStack1 has joined #openstack-trove18:59
*** OpenStack1 has quit IRC19:03
*** Flying_Bond has quit IRC19:08
*** kevinconway has joined #openstack-trove19:08
robertmyersSlickNik: hub_cap: would like to get your reviews on https://review.openstack.org/#/c/56702/19:12
robertmyershub_cap: and https://review.openstack.org/#/c/59283/19:13
*** kanzaros has joined #openstack-trove19:14
SlickNikrobertmyers: sure, will look in about an hour'ish.19:17
robertmyersSlickNik: thanks!19:18
*** jmontemayor has joined #openstack-trove19:18
*** rongze has quit IRC19:20
openstackgerritGreg Hill proposed a change to openstack/trove: add support for working with pre-datastore instances  https://review.openstack.org/6203919:23
cp16netamcrn: i replied to your concerns wrt validation rules19:30
cp16netwith some more concerns.. :)19:30
amcrnyeah, i'm not particularly happy with the workaround i presented, but19:30
amcrn"if a provider creates a datastore of "mysql" with two versions, "5.5" and "5.6", both versions must share the same manager, meaning there is no means of having separate validation-rules for the configuration-groups."19:31
amcrnthat's a big problem19:31
hub_capi think we hoped / assumed people would make mysql5.5 a datastore19:31
hub_capsince the 5.X's are so drastically different19:31
cp16netyeah i think thats the draw back of having the DS stuff so general19:31
hub_capyup.. its configurable, but its not recommended ;)19:32
amcrnlol19:32
amcrni don't think we should lend people enough rope to hang themselves though19:32
hub_capexactly19:32
hub_capi was writing something like that19:32
cp16netyeah the more i think about it you should really make it mysql5.[1,5,6]19:32
hub_capso there comes a point where we have to "draw a line in the sand"19:32
hub_capand this is kinda that right? like we should help make a decision somehow19:33
amcrnif we think about this in the context of migrations, you'll need version to version migration "managers" anyway19:33
hub_capbut how can we really?19:33
cp16netand then the wind blows the sand around19:33
hub_caplol cp16net19:33
cp16netyeah version to version would not work if you are upgrading from 5.1 to 5.519:34
amcrnif the manager concept is moved to the datastore-version (from the datastore), this largely solves the problem19:34
amcrnthen each version can choose to have a different manager, which in turn means it can have a different set of validation-rules19:34
cp16netbut if you are saying that we can have validation rules for versions as well as types19:35
amcrnwell, right now you have validation-rules per manager19:35
cp16netdepending on how you create a CG19:35
cp16netwe would need to merge those in the code19:35
amcrnthat would remain the same, the only change would be to permit a different manager per version instead of per type19:36
hub_capbut do we need to solve this for all ds' for one bastard one amcrn? :P19:36
cp16netbut which to you apply if they override each other19:36
cp16netheh19:36
hub_capdo we have to deal w/ this with other datastores? like would people want to run a specific version of cassandra thats _not_ latest (i assume so?)19:36
amcrncp16net: sorry, let me re-explain. kill the whole <type>-<version> multiple lookup idea19:36
cp16netok19:36
amcrnkeep it as-is, except, move the manager column from the datastore table to the datastore_version table19:37
cp16nethub_cap: or redis maybe? (i dont know)19:37
cp16netso instances are tied to a version which should ahve a manager19:37
hub_capamcrn: but then if we use it like mysql5.5, with 5.5.32 or whatever as teh version, i have to keep putting that manager in that version?19:37
cp16netthen the CG uses the manager field to know what to validate for19:38
amcrnwell, we can work around the tediousness of setting the values up via trove-manage, but let's first vet whether the idea works19:38
hub_capwell the theory works19:38
amcrncp16net: correct19:38
cp16neti mean *anything* could work :-P19:39
amcrnalright, now if we try and anticipate the datastore migration scenario19:39
amcrnit's *possible* that a datastore has a different migration strategy from version 1.x to 1.y, and a different one for 1.x to 1.z19:39
amcrndepending on their versioning scheme, whether they accidentally released a version with an incompatible change and reverted days later, etc.19:40
amcrnso in short, migrations will need a manager on a version to version basis, not type to type, because the level of granularity required will vary19:40
amcrngranted certain migration-managers will be shareable19:40
amcrnso from a configuration-group scenario, there has to be a way to convert or validate or clone during a migration19:41
amcrne.g. if i have an instance w/ a configuration-group against mysql-5.5, and i do a migration to mysql-5.7 in 2014, how does the validation work for the configuration19:42
amcrndoes the cloud provider explicitly set up a "compatibility" list of version-to-version ok's, or does each version have it's own validation-rules.json and during a migration they're cross-checked19:43
kevinconwayso i'm trying to catch up on this conversation. are you talking about configs or actual migrations of data ?19:43
amcrnhow does a user in general assert whether an existing configuration-group against mysql-5.5 is valid against 5.6?19:43
amcrnkevinconway: configs, but i'm attempting to anticipate migrations, so we don't have to revisit at what granularity should validation-rules exist19:44
cp16nethttp://img.pandawhale.com/post-22106-Cup-and-Ball-perfect-loop-gif-pOdK.gif19:44
kevinconwayi guess i'm confused about mapping compatibility between version of the same type19:45
kevinconwayif you can't guarantee a migration from one version to the next wouldn't those be, by definition, a new type?19:45
amcrnit depends on whether there's a hard stance on what is a "datastore", is it a logical grouping that the provider chooses, or is it by definition a datastore of various versions that are all migratable?19:48
*** OpenStack1 has joined #openstack-trove19:48
amcrnnothing stops a datastore provider from adding a new configuration-parameter in a minor version, they merely aren't allowed to remove any (to be backwards compatible)19:49
amcrnnow you've got a situation in which a validation-rules set for the datastore is incorrect, because you'll either reject saying the new parameter isn't real, or you'll accept it on an older version that didn't support it19:50
kevinconwayamcrn: you're making assumptions about versioning schemes here though19:50
kevinconwayi think most sane products would use major.minor.patch, but we already have an obvious exception with mysql19:50
kevinconwaywhether or not two versions of a product are compatible is totally separate from its version number19:51
amcrngood point. so given your last statement, we can't guarantee that 2 versions under a single datastore are "compatible"19:52
kevinconwayi think it's the responsibility of the cloud provider to only place entries in a datastore version that they know are compatible19:53
kevinconwayotherwise they create a new datastore type19:53
SnowDusthub_cap : updated SlickNic's trove-tempest blueprint with work items but rest i will be following with him to get updated19:53
SnowDustits SlickNik oopzz19:53
kevinconwayi think we're trying to add features like config compat and data migrations that require an opinionated view of what a datastore type and datastore version are19:54
kevinconwaybut at the same time keeping those metadata unopinionated19:54
amcrnkevinconway: the tricky part will be documenting this in bold, explaining you shouldn't do things like "mysql" => "5.5", "5.6", even though the system will let you19:54
amcrnwhich means, as good citizens, the redstack scripts and other artifacts should follow the known-good path19:55
amcrnto avoid leading users down the wrong path19:55
cp16netamcrn: yes i agree with that19:55
cp16neti think thats why i thought it would be originally mysql - 5.1, 5.519:55
cp16netwhich enlighted me that its a bad idea19:56
cp16netafter all these convos19:56
amcrnone thing to point out though, is that a well known cloud provider has a flat datastore naming scheme, which means implicitly each version has it's own validation19:56
amcrni.e. instead of type + version, it's a single label19:57
*** kanzaros has quit IRC19:57
kevinconwayamcrn: that's what i thought we wanted when i was suggesting to use glance image names instead of new metadata19:57
amcrnso we should be aware we're going against the grain here by choosing validation against a grouping/type19:57
kevinconwaya flat space that is19:57
*** OpenStack1 has quit IRC19:58
*** kanzaros has joined #openstack-trove19:58
kevinconwaybut i think the versioning scheme we have could be beneficial when it comes to doing update/migrations19:59
amcrnagreed19:59
amcrnso i think we're all in consensus on how we should guide users on setting up their datastore heirarchies20:00
*** OpenStack1 has joined #openstack-trove20:01
*** kanzaros has quit IRC20:01
amcrnbut i still don't see the harm in moving 'manager' to datastore-version. the only downside is that you have an extra argument to pass to 'trove-manage', but you buy the ability to have different validations per version if you so choose20:01
*** OpenStack1 has quit IRC20:01
amcrnby default the same value can be passed to each version, so it will behave the same as it is now20:01
amcrnhave to jump to a meeting, will come back later20:02
*** kanzaros has joined #openstack-trove20:05
*** kanzaros has quit IRC20:06
*** kanzaros has joined #openstack-trove20:06
*** harlowja has quit IRC20:07
*** SnowDust has quit IRC20:13
greghillDOCS!!!!  I thought that got sorted out20:13
kevinconwaygreghill: just use the old trove motto: "If it doesn't do right the first time just keep doing it again until it does."20:15
cp16nethub_cap: what do you mean "add the test of the scripts cowboy!"20:15
cp16net#link https://review.openstack.org/#/c/54225/20:16
hub_capcp16net: that makes no sense20:23
hub_capi think i passted something wronge20:23
hub_capwrongie20:23
hub_capi meant just update he other bin/ scripts20:23
hub_capLOL20:23
hub_capsweet just got done talkin w/ a structural engineer about our remodel20:24
cp16netok cool20:24
cp16net:-P20:24
hub_caphe says we arent doing anything that requires special headers20:24
*** jcooley_ has quit IRC20:24
hub_capwe wont need a engineer approval horray!20:24
hub_capand he was good buddies w/ the guy who owned the house HAH20:24
hub_capalso cp16net fix that cmmit msg20:25
hub_capi agree w amcrn20:25
cp16netyeah i'm workin on it20:25
cp16nettrue20:25
hub_capbut now i must go to lunch, says wife :)20:25
hub_capamcrn: u know acme and semifreddi bread companies?20:25
demorrishey so I just caught up on the discussion about configs and datastore types20:26
hub_caphttp://www.acmebread.com/bread20:26
demorrisdid we land somewhere, it was not clear20:26
datsun180bhub_cap: i'm confused why none of this bread features rocket skates or flying suits20:28
demorrisone way we can simplify the config stuff is to limit what it can do in the first rev20:28
datsun180bor gigantic catapults or slingshots20:28
openstackgerritCraig Vyvial proposed a change to openstack/trove: make the bin scripts called with main()  https://review.openstack.org/5422520:28
cp16netmo betta20:28
demorriswe could just say, you can't use configurations across db types20:29
*** yogesh has quit IRC20:30
*** rongze has joined #openstack-trove20:31
*** SushilGL has joined #openstack-trove20:32
*** SushilKM has quit IRC20:33
*** SushilGL has quit IRC20:33
*** SushilKM has joined #openstack-trove20:33
*** rongze has quit IRC20:36
*** jcooley_ has joined #openstack-trove20:37
demorrishub_cap: yt?20:38
demorrisI am wondering if we maybe need to put some restrictions into the types / versions code20:38
cp16netdemorris: i agree we should not try to emcompass everything in v1 of CG20:38
demorrisI was originally thinking that the default way to use this would be something like Type (MySQL 5.5) and Versions (X.X.X)20:38
demorrisbut now I am thinking that does not make sense20:38
*** jcooley_ has quit IRC20:39
kevinconwaydemorris: i think amcrn and i agreed earlier that we need to clearly define and restrict was is a type and what is a version20:39
kevinconwayi'm of the opinion that anything the cloud provider knows is compatible is a version of the same type otherwise it needs a new type defined20:40
hub_cap++20:40
hub_capdemorris: not really here, bout to eat lunch20:40
hub_capbut i have a pageup key demorris so ask away20:41
demorrisyeah, I think to start, one restriction needs to be that a type is just the db type (MySQL)20:41
demorrisand versions is a major version, X.X20:42
kevinconwayi think that's too general20:42
kevinconwaythat would put mysql 5.1 and 5.5 in the same pool20:42
*** SushilGL has joined #openstack-trove20:42
demorrisbecause?20:42
cp16neti think the fall out of doing that would be that we need another table for the patch version that you suport then for each version20:43
kevinconwayi would rather see 5.1 as the type and 5.1.1, 5.1.2 as the versions20:43
demorriswhat does the table currently contain?20:44
cp16netother wise you will fall into the current idea of not knowing what patch version is currently installed and being used on the instances20:44
kevinconwaydemorris: i don't understand the question20:44
cp16net#link https://github.com/openstack/trove/blob/master/trove/db/sqlalchemy/migrate_repo/versions/016_add_datastore_type.py20:45
demorrisyou mentioned you would need another table20:45
demorrisso I wanted to know what the current table has in it and why you would need another table20:45
kevinconwayi think he's talking about the table(s) you would need to create compatibility matrices20:45
hub_capversions also imply "upgradeability"20:45
kevinconwaybecause if 5.1 and 5.5 are just mysql versions20:45
kevinconwaythen you need to mark them as super incompatible some how20:45
kevinconwayi'm looking at it from the perspective that anything that is a version of the same type should be compatible from both a config and data migration standpoint20:46
kevinconwayif it breaks one of those then it should be a new type20:46
demorrisdoenst a version id just point to a package so if it was 5.5 that would actually be some 5.5.X version20:46
*** SushilKM has quit IRC20:47
kevinconwaydemorris: version id and package are totally separate20:47
kevinconwayi can have TYPE: mysql VERSION: 5.5 PACKAGE: redid-server if i wanted20:47
kevinconway*redis20:47
demorriswell thats silly, why would you allow that20:47
kevinconwaywell nobody sane would do that to their users20:48
kevinconwayor themselves for that matter20:48
kevinconwayit's just possible20:48
kevinconwaybut that's kind of what i keep trying to say. we made datastore types and versions VERY non-opinionated.20:49
kevinconwayi think we need to opinionated them to support some of the features we want, though.20:49
kevinconwayman, this spell check is killing me20:49
demorrisyeah, its making it very hard to build on20:49
demorrisnear impossible to build on20:50
*** radez is now known as radez_g0n320:50
demorristoo many what if scenarios20:50
kevinconwayand it's not just config edits, we have to think about migrations/updates too20:50
openstackgerritCraig Vyvial proposed a change to openstack/trove: make the bin scripts called with main()  https://review.openstack.org/5422520:54
*** OpenStack1 has joined #openstack-trove20:55
demorriskevinconway: right20:55
demorrisso curious how we get to resolution on this20:55
cp16netwell you could put PACKAGES: xtra-backup, mysql5.1, my-funny-agent, cpython, puppet20:55
*** harlowja has joined #openstack-trove20:56
cp16netand it would install all those packages in the prepare20:56
cp16netjust an example of possiblities20:56
demorrispackages is part of the versions table, and assuming it installs the latest20:56
kevinconwayi like PACKAGES: mysql5.1, nsa-prism20:56
cp16netlol20:56
kevinconwaydemorris: packages is whatever the cloud provider wants it to be20:57
cp16netor... packages: mysql5.1==5.1.3320:57
cp16netbam you get that version20:57
kevinconwayjust like datastore type and datastore version. it's a string field20:57
demorrisright, so lets say I want to support 5.1.63 and 5.1.69 versions of MySQL at major version 5.120:58
demorrishow would I set that up20:58
cp16netits suppose to be the name of the package in the repo you are using. apt for debain/ubuntu or in yum for redhat20:58
cp16netyou could not separate that in a single version of 5.120:58
cp16netBUT...20:58
cp16netits only newly provisioned instances that this mattesr for20:59
demorristhen there is no point to ever have the minor version20:59
cp16netif you want customers to be able to provision both of them then you could not do that20:59
demorrisunless I am missing something20:59
kevinconwaycp16net: right, so you could have a version of mysql5.1 where you flip the packages so new instances have the latest20:59
cp16netsure20:59
*** glucas has quit IRC20:59
cp16netyou could even mark it to a certain version at that point as well20:59
kevinconwayor make 5.1.63 a version of the TYPE: mysql5.120:59
kevinconwayor have a TYPE: MySQL5.6.63 and VERSION: WHATEVER_I_WANT21:00
cp16netyeah21:00
cp16netLOL true...21:00
demorrisyeah but that solves nothing21:00
demorristhere is no real point to do that21:00
hub_capthats not true that it solves nothing21:00
cp16netmaybe not with mysql.21:00
kevinconwaydemorris: i agree21:00
hub_capit simplifies the code21:00
hub_capand it deliniates them as being _different_21:01
demorrisif you just restrict to version is MySQL, Redis, MongoDB and version is x . x, then you can still be just as flexible21:01
cp16netmaybe something else could have that kind of setup21:01
cp16netwe dont know21:01
cp16netwe cant future proof everything...21:01
cp16net:-P21:01
demorrisand for upgrades, the guest just needs to know what version it is at and check that against the latest package available in the list of supported versions to know if it needs to be upgraded21:01
demorristhat means though that you peg the code to only support the latest available version you have installed21:02
kevinconwaydemorris: hang on, you said list of supported versions21:02
kevinconwaywhere does that come from?21:02
demorrisfrom the versions table21:02
demorristhere is a flag on if it is active or inactive21:02
kevinconwaybut you're saying 5.1 and 5.5 are versions right?21:02
cp16netwell i think the upgrades should be defined in a different way than relying on the datastore version table21:02
demorrisyes21:02
kevinconwayyou can't upgrade between those though21:02
cp16netright..21:03
kevinconwaythat would be a migration, yes?21:03
demorrisyou can migrate between 5.1 and 5.5, its tricky and there are tons of pitfalls, but you can technically do it21:03
kevinconwayright, but it's much more involved than applying a security patch21:03
kevinconwayand there are backwards incompatibilities21:03
demorriscorrect21:04
demorriscorrect21:04
cp16netyea21:04
kevinconwayso why wouldn't those be different types21:04
kevinconwayi can describe going from mysql to postgresql the same way21:04
hub_capexactly21:04
kevinconwayits involved and incompatibile21:04
cp16netthats more like a integration than just a migration/upgrade21:05
demorrisRDS supports major version upgrades21:05
demorrisfor MySQL21:05
*** jcooley_ has joined #openstack-trove21:05
kevinconwaydemorris: i've done migration services between most versions and vendors of relational databases, that's not a super hard problem to solve21:06
kevinconwaybut we also support nosql21:06
hub_capdemorris: and we might too one day21:06
hub_capbut that doesnt mean it makes things easier to deal w/ a major/minor version21:07
demorrishub_cap: kevinconway right, and I am not saying it something that has to work day 1, point is that it IS possible to automate it for certain cases21:07
demorristhat was all21:07
kevinconwaywell let me ask this, whats the difference between update and migration?21:07
kevinconwayis update in place and migration not in place?21:07
cp16netcan i go from mysql 5.1 to mongo?21:08
kevinconwayand i mean within the same trove instance for in-place and requiring a new trove instance for not-in-place21:08
cp16netthat would be crazy21:08
demorrisI would say to the user it does not matter…mincing words with upgrade and migration21:08
cp16netlol21:08
kevinconwayi think it's an important distiction21:08
kevinconwayif update is in place then versions must support in place updates between them21:08
kevinconwayotherwise it may as well be a new datastore type21:09
demorrisI would consider an upgrade to be within the context of a type (MySQL, Redis, etc.)21:09
demorriswhether its major or minor21:09
kevinconwayi think we're talking about two different cases here. you're talking about end user.21:09
kevinconwayyou can lie to end users and call everything an update21:10
cp16netlol21:10
kevinconwaybut from a trove perspective i don't think you can update 5.1 to 5.5. that's a migration.21:10
demorriskevinconway: http://dev.mysql.com/doc/refman/5.5/en/upgrading-from-previous-series.html21:11
demorrisit's an upgrade21:11
demorris:)21:11
kevinconwaylol21:11
cp16netthats alot of word21:11
cp16net+s21:12
cp16netyou basically have to maek sure users undertand those issues21:12
cp16netbut that should NOT be inplace21:12
cp16netincase it screws everything up and a user needs to go back to the original system21:13
cp16neti think its should be similar-esk to the restore21:13
cp16neton a new instance and the user can decide if they want to go on with it or not21:13
kevinconwaycp16net: yeah, it's pretty easy to stream data from one relational database to another21:14
cp16netsure21:14
demorristhere are a lot of ways to do it21:14
demorrisuser can create a backup21:14
demorristry the upgrade (in place)21:14
demorrisif it fails..restore to backup21:15
kevinconwaydemorris: then what?21:15
kevinconwaythey just can't upgade?21:15
demorriscall support? :)21:15
kevinconwaylol, why not just call them at the beginning and have a dba do the whole thing?21:15
demorrisjoking aside, I am simply saying that just because something could go wrong is not a reason that it would never work21:16
*** OpenStack1 has quit IRC21:16
demorristhe question at the root of this is if you should allow for major version upgrades21:16
demorrisI think the system should and the provider decides if they want to support that21:16
demorristhe API should not restrict it outright21:17
demorrislooking more at the code I feel like it already does this21:17
kevinconwayi think i get what you're saying21:18
kevinconwaytarget compatibility to a datastore type and assume everything a provider puts as a version of that type is compatible with configs and can easily update in-place21:18
demorrismore or less, but amcrn has an interesting approach on this documented here - https://gist.github.com/amcrn/dfd493200fcdfdb61a2321:20
demorrisI think that allows a provider to decide what paths to enable and a specific manager written to take care of them21:20
kevinconwayyeah he's all about that compat matrix21:22
kevinconwayi think it's unnecessary21:22
kevinconwayif we define version updates to be in-place package changes then we just add a new guest agent method to handle it21:22
kevinconwayand type migrations get handled by something else21:23
demorrisI see21:24
demorriswell in that case, if the guest agent just handles minor version updates21:25
demorrisnm, incomplete thought21:25
kevinconwaywell and minor version updates may still be MySuperDB6 to MySuperDB721:26
kevinconwaydepending on the version numbers used21:26
* amcrn is back21:26
kevinconwayits whatever can be done in place21:26
*** OpenStack1 has joined #openstack-trove21:29
demorrisright21:30
demorrisso lets go back to the original problem21:30
demorrisI have a bunch of configurations and I want to associate them with a type (MySQL, Redis, MongoDB, Cassandra, etc.) and a major versions (X . X)21:31
kevinconwayso the question was do we target config compat at types or individual version right?21:31
demorrisI do not want to associate them with minor versions, or I would say, we can leave that out of scope21:32
kevinconwayagain, you're distinguishing between major and minor versions21:32
demorrisyes21:32
kevinconwaywhat's the distinction?21:32
kevinconwaytrove doesn't know what a major versus minor version is21:33
kevinconwayit just knows type and version21:33
amcrnagreed21:33
demorrisokay well what if I say it should21:33
kevinconwaythen what is the distinction between major and minor version?21:33
kevinconwaybecause it can't be the numbers. those are meaningless21:34
demorrisokay so let me back up21:35
hub_capdemorris: do u take two steps forward while kevinconway takes two steps back?21:35
hub_capyou come together cuz opposites attract?21:35
hub_capfrom what i hear, "it aint fiction, its a natural fact"... so we dont mind21:36
demorrisoh so you are breaking out the Paula Abdul21:36
amcrnlol21:36
demorrisI see how it is21:36
hub_capdemorris: :)21:36
*** jmontemayor has quit IRC21:37
amcrndemorris: example: mongodb introduced noautosplit in 2.0.7. even if you modeled your datastores after major.minor, there's no way of validating that noautosplit shouldn't be in a 2.0.6 configuration-group.21:38
demorrisamcrn: gotcha, and I was just thinking of the following example…5.1 - 5.5 MySQL21:38
demorrisinnodb_buffer_pool_shm_key and innodb_buffer_pool_shm_checksum were removed from 5.1 to 5.521:39
amcrnyour assumption is that library maintainers follow the rules, but that's a dangerous presumption ;)21:39
demorrisbut I believe in Percona distributions they kept them around for compatibly reasons and issue a warning…not sure how MySQL handles if they are present21:39
demorrishehe yeah21:39
demorrisperhaps I am over thinking21:39
*** jcooley_ has quit IRC21:41
*** yogesh has joined #openstack-trove21:43
amcrnanother example: require_client_auth for client_encryption_options was introduced in cassandra 1.2.321:44
amcrnit could be said that these are few and far between, but they do exist21:44
amcrnwhich leads me to believe that configuration-group parameter validation should be on a datastore-version level, not datastore-type21:45
amcrnthere are many ways that can be accomplished, but i think getting a consensus on this fact is what will drive it forward21:46
*** OpenStack1 has quit IRC21:47
*** harlowja has quit IRC21:50
* amcrn starts chatting with the crickets, asking how their day was21:51
cp16netlolz21:51
demorrisamcrn: thinking21:51
cp16netchirp chirp21:51
amcrn;)21:51
*** jcooley_ has joined #openstack-trove21:52
*** SergeyLukjanov_ has quit IRC21:53
*** mmcdaris has joined #openstack-trove21:56
*** jcooley_ has quit IRC21:57
*** harlowja has joined #openstack-trove22:00
*** jcooley_ has joined #openstack-trove22:01
*** greghill has quit IRC22:03
demorrisamcrn: good example, but even with that, why not still extend it to be associated with both the type and the version id22:03
amcrnthe version-id is fk'd to the type, so it's effectively the same thing22:04
amcrnthe version-id is effectively the type+version22:04
demorrisi see...22:05
amcrnnow, whether that's exposed to the provider as version-id, or as type+version-name, that's a choice that can be made22:06
demorrisswhat about allowing tying it to multiple version id's22:07
amcrntying a validation-rules set to multiple version-ids? sure22:07
*** pdmars has quit IRC22:08
amcrnright now, the abstraction for the sinew is the "manager". the manager is tied to a validation-rule set, which implicitly means that we need to tie a version-id to a manager to achieve the goal22:09
amcrnand many version-ids can tie to a single manager22:09
amcrnbut is the manager the right connector?22:09
demorrisnot sure, I am not at all familiar with the code, so I am getting a tad lost22:11
demorris:)22:11
cp16neta tad is an understatment22:12
cp16net:-P22:12
demorristhanks cp16net22:12
hub_capshould we have this convo 1) on the ml22:12
hub_capor 2) on monday?22:12
hub_cap:)22:12
amcrncp16net: have i lost you as well?22:13
hub_capsince its like almost knocking off time for the austinites and the mirantis guys are gone22:13
cp16neti dont know as much as lost me as i just lost myself22:13
amcrnhrm, sorry :/22:13
cp16net:-P22:13
amcrnare we in agreement that validation should be on a version basis vs. a datastore basis? if so, we have progress, we just need to determine *how*22:14
datsun180b4:15 better not be "almost knocking off time" for us22:14
kevinconwayspeaking of ML, have you folks been following the universal guest agent thread?22:15
cp16net5 min :-P22:15
kevinconwaythey seem to know a lot about what we want and need22:15
kevinconwayand by that i mean they say "This would be great for Trove" a lot22:15
datsun180bhttp://www.youtube.com/watch?v=mSy5mEcmgwU kevinconway22:15
* amcrn away22:16
*** harlowja has quit IRC22:16
*** harlowja has joined #openstack-trove22:24
*** jcooley_ has quit IRC22:25
*** harlowja has quit IRC22:27
*** kevinconway has quit IRC22:27
*** harlowja has joined #openstack-trove22:28
*** amytron has quit IRC22:29
*** robertmyers has quit IRC22:31
*** denis_makogon_ has joined #openstack-trove22:35
*** demorris has quit IRC22:40
*** Barker has quit IRC22:51
*** Barker has joined #openstack-trove22:52
*** Barker has quit IRC22:56
*** mmcdaris has left #openstack-trove23:04
*** Barker has joined #openstack-trove23:07
*** OpenStack1 has joined #openstack-trove23:08
*** datsun180b has quit IRC23:10
*** Barker has quit IRC23:14
openstackgerritA change was merged to openstack/python-troveclient: Updates README with new trove help options  https://review.openstack.org/5883423:17
*** yogesh has quit IRC23:20
*** rnirmal has quit IRC23:20
*** SushilGL has quit IRC23:24
*** jasonb365 has quit IRC23:27
*** greghill has joined #openstack-trove23:31
*** OpenStack1 has quit IRC23:50
*** flaper87 is now known as flaper87|afk23:50
*** denis_makogon_ has quit IRC23:51
*** grapex has quit IRC23:52
*** jcru has quit IRC23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!