Wednesday, 2014-03-12

vipulI hear this YAML thing is pretty hip00:00
*** michael-yu has quit IRC00:01
espxml hurts me00:01
juiceuntil it is formatted properly, they all hurt me00:02
juicedon't tell me some of the big blobs of json coming back from openstack don't make you feel a bit nauseous00:02
amcrnxml is like violence, if it doesn't solve your problem, you're not using enough of it.00:02
juiceamcrn: I'm getting the tattooed on my arm00:02
amcrnhaha00:03
espI feel like if you are going to make me use xml you might as well throw ninja stars in my face while you are at it.00:05
espk, sorry for the drama.  back to work.00:06
*** tattabbum has quit IRC00:07
*** matsuhashi has joined #openstack-trove00:26
*** esp has left #openstack-trove00:40
*** isviridov has quit IRC00:50
*** juantwo has joined #openstack-trove00:52
*** dmakogon_ has quit IRC00:58
*** zigo has quit IRC00:58
*** isviridov has joined #openstack-trove00:58
*** erkules has quit IRC00:58
*** dmakogon__ has joined #openstack-trove00:58
*** openstack has joined #openstack-trove01:04
*** erkules has joined #openstack-trove01:05
*** ramashri has joined #openstack-trove01:10
*** eghobo has quit IRC01:13
*** Barker has joined #openstack-trove01:18
*** achampion has joined #openstack-trove01:19
*** zigo has quit IRC01:27
*** zigo has joined #openstack-trove01:33
*** michael-yu has joined #openstack-trove01:33
*** michael-yu has quit IRC01:33
*** nosnos has joined #openstack-trove01:39
*** khyati_ has quit IRC01:56
openstackgerritA change was merged to openstack/trove: Root_on_create per datastore  https://review.openstack.org/7727501:56
*** matsuhashi has quit IRC01:58
*** erkules has quit IRC02:07
*** matsuhashi has joined #openstack-trove02:17
*** zigo has quit IRC02:19
*** erkules has joined #openstack-trove02:20
*** zigo has joined #openstack-trove02:23
*** michael-yu has joined #openstack-trove02:24
*** khyati has joined #openstack-trove02:27
*** michael-yu has quit IRC02:37
*** harlowja is now known as harlowja_away02:48
*** haomaiwang has quit IRC02:51
*** haomaiwa_ has joined #openstack-trove02:51
*** SushilKM has joined #openstack-trove03:02
*** amcrn has quit IRC03:05
*** haomaiwa_ has quit IRC03:05
*** mattgriffin has joined #openstack-trove03:19
*** rwsu has quit IRC03:19
*** haomaiwang has joined #openstack-trove03:20
*** Barker has quit IRC03:23
*** matsuhashi has quit IRC03:23
*** tanisdl has quit IRC03:26
cp16netoh phely03:31
cp16netfoowey03:32
cp16netfound another bug fixing the other one... :-P03:32
*** haomaiwang has quit IRC03:36
*** haomaiwang has joined #openstack-trove03:37
*** haomaiw__ has joined #openstack-trove03:40
*** SushilKM has quit IRC03:42
*** haomaiwang has quit IRC03:43
openstackgerritCraig Vyvial proposed a change to openstack/trove: Parses default configuration of an instance  https://review.openstack.org/7983803:44
*** eghobo has joined #openstack-trove03:45
*** rwsu has joined #openstack-trove03:50
*** achampio1 has joined #openstack-trove03:51
*** achampion has quit IRC03:54
*** achampio1 has quit IRC03:55
*** achampion has joined #openstack-trove03:56
*** esp has joined #openstack-trove03:58
*** mattgriffin has quit IRC04:05
*** matsuhashi has joined #openstack-trove04:25
*** achampion has quit IRC04:32
*** juantwo has quit IRC04:40
*** juantwo has joined #openstack-trove04:44
*** timfreund has quit IRC04:52
*** ramashri has quit IRC05:04
*** juantwo has quit IRC05:05
openstackgerritSam Morrison proposed a change to openstack/trove: Make hostnames in designate driver all lower case  https://review.openstack.org/7843905:19
*** timfreund has joined #openstack-trove05:21
*** ramashri has joined #openstack-trove05:42
*** yidclare has joined #openstack-trove05:46
*** ramashri has quit IRC05:48
*** ViswaV_ has quit IRC05:55
*** SushilKM has joined #openstack-trove05:57
openstackgerritCraig Vyvial proposed a change to openstack/trove: Datastore Configuration Parameters stored in db  https://review.openstack.org/7985006:03
openstackgerritCraig Vyvial proposed a change to openstack/python-troveclient: adding configuration paramters mgmt api  https://review.openstack.org/7985106:03
openstackgerritCraig Vyvial proposed a change to openstack/trove-integration: Configurations stored in the database  https://review.openstack.org/7985206:04
cp16netthere we go... hub_cap a little late i guess06:04
*** yidclare has quit IRC06:11
openstackgerritCraig Vyvial proposed a change to openstack/trove: Datastore Configuration Parameters stored in db  https://review.openstack.org/7985006:21
openstackgerritCraig Vyvial proposed a change to openstack/trove: Parses default configuration of an instance  https://review.openstack.org/7983806:23
*** khyati has quit IRC06:29
openstackgerritshivam shukla proposed a change to openstack/trove: Tests for heat based instance workflow  https://review.openstack.org/6649906:29
openstackgerritCraig Vyvial proposed a change to openstack/trove-integration: Configurations stored in the database  https://review.openstack.org/7985206:34
openstackgerritCraig Vyvial proposed a change to openstack/python-troveclient: adding configuration paramters mgmt api  https://review.openstack.org/7985106:34
openstackgerritCraig Vyvial proposed a change to openstack/trove: Datastore Configuration Parameters stored in db  https://review.openstack.org/7985006:34
*** denis_makogon has joined #openstack-trove06:50
*** PradeepChandani has joined #openstack-trove06:51
*** saju_m has joined #openstack-trove06:52
*** flaper87|afk is now known as flaper8707:24
*** eghobo has quit IRC07:28
*** matsuhashi has quit IRC08:21
*** amcrn has joined #openstack-trove08:35
*** saju_m has quit IRC08:41
*** tattabbum has joined #openstack-trove08:47
*** iartarisi has joined #openstack-trove08:50
*** amcrn_ has joined #openstack-trove08:50
*** amcrn has quit IRC08:53
*** amcrn_ is now known as amcrn08:53
*** saju_m has joined #openstack-trove09:09
*** ashestakov_ has joined #openstack-trove09:16
*** dukhlov has quit IRC09:24
*** dukhlov has joined #openstack-trove09:30
*** amcrn has quit IRC09:38
*** SushilKM has quit IRC09:39
*** SushilKM has joined #openstack-trove09:44
*** denis_makogon has quit IRC09:51
*** ashestakov_ has quit IRC10:04
*** iartarisi has quit IRC10:33
*** saju_m has quit IRC10:49
openstackgerritDenis M. proposed a change to openstack/trove: Add timeout on prepare call per datastore  https://review.openstack.org/7716010:58
openstackgerritSascha Peilicke proposed a change to openstack/python-troveclient: Add py33 trove classifier  https://review.openstack.org/7189511:05
openstackgerritDenis M. proposed a change to openstack/python-troveclient: Add point in time recovery  https://review.openstack.org/7722311:05
*** IvanZ has joined #openstack-trove11:13
*** ChanServ changes topic to "test/gate jobs are queuing now in preparation for gerrit maintenance at 12:00 utc (eta to resume is 12:30 utc)"11:28
*** SushilKM has quit IRC11:34
openstackgerritshivam shukla proposed a change to openstack/trove: Tests for heat based instance workflow  https://review.openstack.org/6649911:35
*** SushilKM has joined #openstack-trove11:37
*** IvanZ has quit IRC11:48
*** ChanServ changes topic to "gerrit on review.openstack.org is down for maintenance (eta to resume is 12:30 utc)"12:07
*** ChanServ changes topic to "gerrit on review.openstack.org is down for maintenance (revised eta to resume is 13:00 utc)"12:24
*** IvanZ has joined #openstack-trove12:36
*** saju_m has joined #openstack-trove12:37
*** yidclare has joined #openstack-trove12:47
*** openstackgerrit has quit IRC12:54
*** openstackgerrit has joined #openstack-trove12:54
*** Ranjitha has joined #openstack-trove13:04
*** achampion has joined #openstack-trove13:08
*** openstackgerrit has quit IRC13:08
*** openstackgerrit has joined #openstack-trove13:08
*** Ranjitha has quit IRC13:08
*** Ranjitha has joined #openstack-trove13:09
*** iartarisi has joined #openstack-trove13:17
*** robertmyers has joined #openstack-trove13:20
*** robertmyers has quit IRC13:21
*** robertmyers has joined #openstack-trove13:22
*** jcru has joined #openstack-trove13:26
*** radez_g0n3 is now known as radez13:38
*** nosnos has quit IRC13:43
*** mattgriffin has joined #openstack-trove13:49
*** matsuhashi has joined #openstack-trove13:50
*** matsuhashi has quit IRC13:56
*** demorris has joined #openstack-trove13:56
*** SushilKM has quit IRC13:58
*** IvanZ has quit IRC13:59
*** mattgriffin has quit IRC13:59
*** mattgriffin has joined #openstack-trove14:00
*** matsuhashi has joined #openstack-trove14:00
*** openstack has joined #openstack-trove14:04
*** IvanZ has joined #openstack-trove14:08
*** iartarisi has quit IRC14:09
*** grapex has joined #openstack-trove14:17
*** demorris has quit IRC14:18
*** demorris has joined #openstack-trove14:20
*** matsuhashi has quit IRC14:23
*** matsuhashi has joined #openstack-trove14:30
*** Barker has joined #openstack-trove14:34
*** mattgriffin has quit IRC14:35
*** datsun180b has joined #openstack-trove14:37
*** grapex has quit IRC14:37
*** grapex has joined #openstack-trove14:38
*** yidclare has quit IRC14:39
*** ChanServ changes topic to "Trove Openstack Database as a Service | Docs https://github.com/openstack/trove | Channel Logs http://eavesdrop.openstack.org/irclogs/ | IRC must read http://sackheads.org/~bnaylor/spew/away_msgs.html"14:42
*** Ranjitha has quit IRC14:43
*** shivamshukla has joined #openstack-trove14:44
*** mattgriffin has joined #openstack-trove14:49
*** thedodd has joined #openstack-trove14:50
*** kevinconway has joined #openstack-trove14:53
*** jmontemayor has joined #openstack-trove14:55
*** jmontemayor has quit IRC14:55
*** jmontemayor has joined #openstack-trove14:57
*** saju_m has quit IRC15:03
*** demorris has quit IRC15:04
*** saju_m has joined #openstack-trove15:04
*** demorris has joined #openstack-trove15:06
*** demorris has quit IRC15:10
*** rramirez has joined #openstack-trove15:14
*** IvanZ has quit IRC15:15
rramirezHello everyone!  I'm one of the newest additions to the Rackspace Trove team.15:16
*** IvanZ has joined #openstack-trove15:17
datsun180bhiya rueben15:17
datsun180bnice silence, rest of the channel15:18
*** rramirez has left #openstack-trove15:18
*** matsuhashi has quit IRC15:18
*** dmakogon__ is now known as denis_makogon15:20
denis_makogonrramirez, welcome =)15:20
denis_makogondatsun180b, we are out at least its too early for showing up15:21
*** rramirez has joined #openstack-trove15:26
*** ViswaV has joined #openstack-trove15:35
*** ViswaV_ has joined #openstack-trove15:36
*** PradeepChandani has quit IRC15:39
*** ViswaV has quit IRC15:39
*** kpom is now known as k-pom15:39
*** saju_m has quit IRC15:40
*** shivamshukla has quit IRC15:40
*** PradeepChandani has joined #openstack-trove15:40
*** shivamshukla has joined #openstack-trove15:41
*** SushilKM has joined #openstack-trove15:44
*** tanisdl has joined #openstack-trove15:46
*** Barker has quit IRC15:48
*** Barker has joined #openstack-trove15:52
*** tanisdl has quit IRC15:53
*** tanisdl has joined #openstack-trove15:54
*** eghobo has joined #openstack-trove15:56
*** eghobo has quit IRC15:58
SlickNikHey guys15:58
SushilKMHello @SlickNik15:58
SlickNikReminder that we have our trove meeting in about an hours time, so update the wiki page at https://wiki.openstack.org/wiki/Meetings/TroveMeeting if you want to add items to the agenda.15:59
grapexSlickNik: Don't you mean two hours?15:59
SlickNikgrapex: good point. We're tied to UTC, so we don't observe DST.16:00
SlickNikSo yes, two hours :)16:00
grapexSlickNik: I know because all of the Texans are looking forward to lunch16:01
grapexbecause everything is bigger in Texas, including our waistlines16:01
* grapex begins firing two revolvers into the air while yelling "yehaw"16:01
*** rramirez has quit IRC16:02
SlickNiklol16:03
*** eghobo has joined #openstack-trove16:04
*** IvanZ has quit IRC16:06
*** demorris has joined #openstack-trove16:14
*** digambar has joined #openstack-trove16:16
*** SushilKM has quit IRC16:20
*** khyati has joined #openstack-trove16:41
*** amcrn has joined #openstack-trove16:42
*** Barker has quit IRC16:44
*** Barker has joined #openstack-trove16:45
*** ViswaV_ has quit IRC16:45
*** ViswaV has joined #openstack-trove16:45
*** eghobo has quit IRC16:47
*** eghobo has joined #openstack-trove16:48
*** rramirez has joined #openstack-trove17:02
*** sballe has joined #openstack-trove17:03
*** harlowja_away is now known as harlowja17:04
*** jmontemayor has quit IRC17:04
*** jmontemayor has joined #openstack-trove17:05
*** atomic77 has joined #openstack-trove17:08
*** shivamshukla has left #openstack-trove17:08
*** shivamshukla has joined #openstack-trove17:08
*** sballe has quit IRC17:10
*** demorris has quit IRC17:16
*** saju_m has joined #openstack-trove17:16
*** rramirez has quit IRC17:18
*** saurabhs has joined #openstack-trove17:19
denis_makogonhttps://github.com/openstack/trove/blob/master/trove/tests/unittests/guestagent/test_backups.py#L106-L114 ||| http://logs.openstack.org/60/77160/5/check/gate-trove-python27/199b1e7/console.html17:21
denis_makogonguys merged patch doesn't fix all issues17:21
*** digambar has quit IRC17:21
denis_makogonit still tries to execute real shell command on the slave environment17:22
*** digambar has joined #openstack-trove17:22
*** demorris has joined #openstack-trove17:24
openstackgerritRamashri Umale proposed a change to openstack/trove: Added checks for backup-create API call to ensure datastore supports backup strategy Also for trove-create API call (when restoring from backup) added checks to ensure that datastore and datastore version of backup being restored match with datastore and   https://review.openstack.org/8001017:24
*** yidclare has joined #openstack-trove17:25
*** michael-yu has joined #openstack-trove17:28
*** rramirez has joined #openstack-trove17:30
*** michael-yu has quit IRC17:30
*** michael-yu has joined #openstack-trove17:31
*** openstackstatus has quit IRC17:38
*** amytron has joined #openstack-trove17:38
*** openstackstatus has joined #openstack-trove17:39
*** yogesh has joined #openstack-trove17:42
openstackgerritDenis M. proposed a change to openstack/trove: Refactor mysqla and redis apps  https://review.openstack.org/8002017:42
openstackgerritDenis M. proposed a change to openstack/trove: Refactor mysql and redis apps  https://review.openstack.org/8002017:43
abramleydenis_makogon - is this refactor basically the same thing I was looking at as part of https://bugs.launchpad.net/trove/+bug/1276860 ?17:45
denis_makogonabramley, yes, you was right, different scopes17:45
denis_makogonabramley, i guess, now we can me this patches dependent (your and mine)17:46
denis_makogonabramley, since they are following the almost the same goal17:46
denis_makogonabramley, is it ok ?17:46
abramleydenis_makogon - just surprised since only yesterday you suggested I refactor the code along these lines. Wasn't aware you were already working on a bp for it17:46
denis_makogonabramley, i should've notify you, that i took this17:47
denis_makogonabramley, now we can co-work17:48
digambarGuys looking into this bug https://bugs.launchpad.net/trove/+bug/113092917:49
digambartrove-mgmt-cli17:49
denis_makogonabramley, my patch addresses to memory usage and refactoring, your to the hiding stack traces17:49
denis_makogondigambar, rd* are rackspace specific mgmt apis. They need to push that functionality into nova before these calls will actually work.17:50
denis_makogondigambar, this is rax specific code, as hub_cap said17:51
digambarok17:51
abramleydenis_makogon - if I understand the new process, your bp will need to be approved before you can move forward?17:51
digambarthen what is the workaround for this or should skip this bug ?17:51
denis_makogonabramley, i guess no, but the patch probably would go through several iteration17:52
denis_makogondigambar, lets ask hub_cap or grapex about that17:52
denis_makogonhub_cap, are you with us ?17:52
abramleydenis_makogon - I will hopefully have mine pushed soon (transitioning to a new dev machine so taking longer than I wanted) - and then you can make yours depend on mine ?17:53
denis_makogonabramley, i guess, we should do opposite thing17:53
denis_makogonabramley, your patch should be dependent on mine17:54
denis_makogonabramley, or even they could stay separate17:54
denis_makogonabramley, as far as i can see, your patch looks like a "feature" and it could land after march 2717:55
abramleydenis_makogon - but then we would need yet another one to identify further methods that need to be moved down into the base class. I prefer to keep mine focused on adding in the improved exception handling - and part of that was to add a base class. Anything beyond that to refactor the datastores seems more related to your 'refactor the datqstores' blueprint - and part of that refactoring would include pushing17:55
abramley common code into the base class17:55
*** SnowDust has joined #openstack-trove17:56
denis_makogonabramley, methods are not changed in my patch17:56
denis_makogonabramley, your patch totally fine, mine, i guess, too, but they follow different approaches17:57
*** shivamshukla has quit IRC17:58
denis_makogonabramley, the difference is that i added constructor to each class that instantiate the main datastore app, status app17:59
*** NehaV has joined #openstack-trove17:59
amcrndenis_makogon abramley: i'll reply to this once the weekly meeting is over18:00
digambar@denis_makogon - I think hub_cap is not there18:01
denis_makogonamcrn, fine18:01
*** Ranjitha has joined #openstack-trove18:05
digambardenis_makogon - want to contribute on clientside as well as securitygroup side18:05
digambarshould I assign bug from that category to me ?18:05
denis_makogondigambar, we have weekly meeting now18:05
digambaryes18:05
denis_makogondigambar, ping me in the 1h18:05
digambarok18:05
*** ramashri has joined #openstack-trove18:06
*** eghobo has quit IRC18:07
*** eghobo has joined #openstack-trove18:09
*** yidclare has quit IRC18:17
*** saju_m has quit IRC18:30
*** digambar has quit IRC18:41
*** tattabbum has quit IRC18:52
*** sriram_tesora has joined #openstack-trove18:53
denis_makogonmerged patch doesn't fix anything, gate still failing https://bugs.launchpad.net/trove/+bug/129151618:56
*** datsun180b has quit IRC18:59
SlickNikwhat review is that for?19:02
denis_makogonSlickNik, https://review.openstack.org/#/c/77160/19:02
denis_makogonSlickNik, i though submitted patch fixed that19:03
denis_makogonSlickNik, but as i can see, its not19:04
SlickNikdenis_makogon: What's the problem then, do you know?19:04
denis_makogonSlickNik, its incremental backup tests19:05
denis_makogonSlickNik, i aslo added link at bug description19:05
SlickNikdenis_makogon: I can't investigate now, I need to run to a meeting.19:05
SlickNikdenis_makogon: Will look when I get out.19:05
denis_makogonSlickNik, cool19:05
*** NehaV1 has joined #openstack-trove19:07
*** NehaV has quit IRC19:08
denis_makogonamcrn, you said you would replay to our convo19:08
amcrnabramley denis_makogon: as to your conversation above19:08
denis_makogonamcrn, would be cool to hear your thoughts19:08
amcrnhttps://review.openstack.org/#/c/78054/ should be amended to use raise exception.DatastoreOperationNotSupported() in the existing managers, but the refactoring part should be reverted as it's overstepping the bounds of the problem statement. after that has merged, any refactoring requires a new blueprint, and said blueprint must be approved.19:09
*** rramirez has quit IRC19:09
denis_makogonamcrn, how https://review.openstack.org/#/c/80020/ overstepping the bounds of https://review.openstack.org/#/c/78054/ ?19:10
amcrnyou misunderstand my point19:10
amcrnthe bug is to have the guestagent properly handle an unsupported operation19:11
denis_makogonyes19:11
amcrnhttps://review.openstack.org/#/c/78054/ fixes said bug, but it goes a bit too far in terms of refactoring19:11
amcrnalthough the intentions were good, it seems there is some conflict between how to approach the refactoring19:11
denis_makogonyes, thats what i said19:11
denis_makogoni mean wrote at comments19:12
amcrnand therefore, the simple solution to this is to amend https://review.openstack.org/#/c/78054/19:12
abramleyamcrn - I wouldn't say it was 'refactoring'- it was simply the best and least error prone way of adding these improved exceptions into the datastores19:12
amcrnto revert the refactoring, but keep the essential raise exception.DatastoreOperationNotSupported() logic19:12
denis_makogonamcrn, agreed19:12
denis_makogonamcrn, there's another one patch https://review.openstack.org/#/c/80020/19:13
amcrnabramley: i agree with you, but since there's a differing of opinions on how to handle the base class, etc, the easiest way to ensure the bug fix lands is to keep it as minimally viable as possible19:13
amcrndenis_makogon: https://review.openstack.org/#/c/80020/ will be deferred until Juno19:13
abramleyamcrn I would suggest then that the improvements to the exceptions should wait until after a base class exists - since anything else is throw away work19:13
denis_makogonamcrn, this one make code a bit easy and makes mysql/redis guest use less memory19:13
denis_makogonamcrn, of course19:14
denis_makogonamcrn, because it doesn't fits to FF request19:14
amcrnabramley: not sure i follow, the basis of your patch still fixes the issue19:14
denis_makogonabramley, i think you should split patches into the two19:15
abramleyamcrn - it does fix the bug with a lot of copy / paste of code across all the datastores - if that is what you want I will revert my changes and do it that way - just seems wrong somehow though19:15
denis_makogonabramley, one for the exception handling, another one for the manager base class19:16
vgnbkramcrn: I think what abramley  is saying is that it doesn't make sense to do the exception fixes, if we will then do the refactoring and remove the exception fixes.19:16
denis_makogonabramley, dependent patches is everything you need19:16
amcrnabramley: given the point of the lifecycle we're at, and given there is a bit of contention on how baseline manager functionality should be shared, the least controversial option is to avoid changing the structure of the code.19:17
denis_makogonvgnbkr, that's why manager base class should come firts19:17
abramleyI am clearly new to the process and have a lot to learn - I can raise another patch request for 'reduce duplication of code in datastores with a manager baseclass' or something like that19:17
amrithabramley: +<several> for that19:17
vgnbkrI suspect the issue is in calling it a "base class".  It's really just a mixin class to make error handling more consistent.19:18
denis_makogonabramley, AFAIK, it will work for all of us19:18
denis_makogonvgnbkr, i think naming it base class is fine since it contains possible API implementation19:19
denis_makogonvgnbkr, my point that BaseManager wasn't made only for the making exception handling easier19:20
denis_makogonvgnbkr, it describes API contract for the new income managers19:20
vgnbkrMy Python is still rudimentary, but I don't really see the point of API defining base classes in a dynamically typed language.19:21
denis_makogonvgnbkr, there a good word for such classes in Java - interface, but since its the python is duck-typing lang, you don't need to implement all methods19:21
vgnbkrIf you don't have a compiler that enforces the API, it's just extra code to maintain.19:22
denis_makogonvgnbkr, agreed, but its very good document for writing new managers19:22
denis_makogonvgnbkr, it was not so easy to write cassandra/mongo manager from the mysql manager19:22
vgnbkrThen that is a major change that needs bp discussion.19:22
denis_makogonvgnbkr, probably you're right19:23
denis_makogonvgnbkr, but i still stand for such change, its required19:23
denis_makogonIMHO19:23
denis_makogonViswaV, ping19:23
denis_makogonViswaV, hope you saw my comments at https://blueprints.launchpad.net/trove/+spec/config-paramters-mongodb19:24
denis_makogonViswaV, please do better search before submitting new blueprint19:24
ViswaVdenis_makogon: ?19:25
ViswaVWhere do you see my name in that BP link you posted above?19:26
denis_makogonViswaV, sorry wrong ping19:26
denis_makogonViswaV, tried to ping Khyati19:26
ViswaVnp.19:26
denis_makogonViswaV, i've got question to you also19:27
ViswaVyes19:27
denis_makogonViswaV, whats the status of mongo backup/restore ?19:27
ViswaVhttps://review.openstack.org/#/c/78339/19:27
ViswaVWaiting for Juno…19:27
*** datsun180b has joined #openstack-trove19:27
khyati denis_makogon as for https://blueprints.launchpad.net/trove/+spec/config-paramters-mongodb I've replied to your comment19:28
khyaticould you please re-frame the description and ill dup my bp19:28
khyatirephrase**19:28
denis_makogonkhyati, done19:29
denis_makogonViswaV, i'm gonna review it soon, and response to it19:29
ViswaVok.19:29
khyatiokay thanks denis_makogon19:29
*** michael-yu has quit IRC19:30
denis_makogonkhyati, so, do you want to pick that task ?19:30
khyatiyes I'm working on it actively19:30
*** SnowDust has quit IRC19:31
khyatidenis_makogon could you assign it to me ?19:33
SlickNikdenis_makogon: Looks like I'm going to be in this meeting for a while. Can you take a look at what might be the issue with https://bugs.launchpad.net/trove/+bug/1291516?19:34
SlickNikThanks!19:36
denis_makogonkhyati, yes, i'll do that19:40
denis_makogonViswaV, done19:41
denis_makogonViswaV, the main problem is that you place whole command into the one exec, command should be divided as it done at https://review.openstack.org/#/c/77551/6/trove/guestagent/strategies/backup/mysql_impl.py19:43
*** tanisdl has quit IRC19:43
denis_makogonViswaV, and also, i'd not suggest you to add another one conf value into the each oslo group, this makes configuration more complext, Trove should have single module for all backup/restore strategies19:44
*** rramirez has joined #openstack-trove19:45
ViswaVdenis_makogon:  I tried both ways and submitted the one with single cmd since I saw a precedent for that (https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/mysql_impl.py#L34-L42)  .. either way , any particular reason why it SHOULD NOT be one command?19:45
denis_makogonViswaV, that is why it _should_ be divided https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/base.py#L135-L13919:46
denis_makogonViswaV, and only19:47
denis_makogonViswaV, backup base class was re-factored to support non-streaming backup strategies (cassandra, mondo, couchbase, redis)19:48
denis_makogonViswaV, in _pre_backup you collect data into the one object, run_backup streams it via "cat file", _post_backup deletes that file19:49
denis_makogonViswaV, easy19:49
ViswaVyeah…that had not landed when I submitted...19:49
ViswaVAlso, I see it as a convenient hook to use (if you need it) and not enforced ….but yes, good to break up the flow (the cmd) that way. Will look at it when I have to rebase.19:49
denis_makogonViswaV, now i'd suggest you to update implementation, and also take a deep look at comments about gate error19:49
denis_makogonViswaV, you need to mock correctly subprocess19:50
ViswaVAlso,  https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/base.py#L71   that  _run_post_backup()  should be moved out of there to https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/base.py#L7519:50
openstackgerritRamashri Umale proposed a change to openstack/trove: Added Backup/Restore validations  https://review.openstack.org/8001019:50
ViswaVYes,  denis_makogon, I ran into some mock erros when some python commands were trying to be executed on the Jenkins and their subsystem caught it and threw error.19:51
denis_makogonViswaV, no, it should not be moved19:51
ViswaVI modified my tests to mock some more additional methods to avoid that.19:51
denis_makogonViswaV, but the gate failed19:51
*** saurabhs has quit IRC19:52
denis_makogonViswaV, it means that that something still mocked incorrect19:52
ViswaVyup.. will look into it.19:53
ViswaV _run_post_backup()  should be called when the "with runner…."  has completed and is about to exit.19:55
ViswaVwhich is intercepted by the __exit__ method  in https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/base.py#L7519:55
denis_makogonViswaV, no, it shouldn't19:56
ViswaVThat is why the _run_post_backup() should be moved into the __exit__  method. It's not enough to simply wrap the run() method with pre and post19:56
denis_makogonViswaV, it somehow breaks the percona backuping19:57
denis_makogonViswaV, so, _pre_backup and _post_backup are called in execute_backup19:57
denis_makogonand its fine19:57
amcrndenis_makogon: on a different note before i go to lunch, since https://review.openstack.org/#/c/77160/ requires a rebuild anyway, change reqiurest => requires and propesed => proposed in your commit msg.19:58
*** demorris has quit IRC19:58
amcrnthanks :)19:59
ViswaVdenis_makogon:  https://github.com/openstack/trove/blob/master/trove/guestagent/backup/backupagent.py#L102  ,  the "with runner"  bounds the backup begin and backup ended.  and not just the https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/base.py#L7020:00
ViswaVSo until the "with runner" execution is done ,  you can't be sure that backup phase is done and post_backup () can be called.20:00
denis_makogonViswaV, in this case - yes, but it should be done as separate patch20:01
ViswaVWhich is why post_backup should be called in the __exit__() method.  since  the __enter__ and  __exi()___ bound the "with runner" code.20:01
openstackgerritDenis M. proposed a change to openstack/trove: Add timeout on prepare call per datastore  https://review.openstack.org/7716020:01
*** NehaV1 has quit IRC20:02
denis_makogonViswaV, IMHO, https://github.com/openstack/trove/blob/master/trove/guestagent/strategies/backup/base.py#L67-L72 - its fine20:02
*** NehaV has joined #openstack-trove20:02
denis_makogonViswaV, i don't see the problem, guest will lauch _post_backup only if run() will finish successfully20:03
denis_makogonViswaV, so, its fine, 100%20:03
*** Ranjitha has quit IRC20:04
ViswaVDenis,  run() will only generate the data that needs to be uploaded to swift, that run() gets executed as soon as the "with runner…" block is entered (which calls __enter__ which invokes pre_backup() and run() )20:05
ViswaVBut the save to swift part is here https://github.com/openstack/trove/blob/master/trove/guestagent/backup/backupagent.py#L10620:05
ViswaVpost_backup() cannot happen until all that is finished ....20:06
*** ramashri has quit IRC20:06
ViswaVas it exists today per your changes  the sequence would be  enter the "with runner" ---> triggers __enter__() method -->  pre, run, post --->   then move further down in the "with runner" block where save() happens ---> block ends and __exit__() gets invoked.20:07
ViswaVWhat if in your post_backup() you clean up the data generated by backup cmd that needs to be uploaded to swift….you will now be deleting that before the save() step is reached.20:08
ViswaVIT has to be __enter__() {  pre_backup,  run() } ,  ……remaining logic in  "with runner" block …….block ends ,  __exit__() { post_backup}20:09
ViswaVIf you do otherwise, you will run into serious race conditions possibly.20:09
ViswaVdenis_makogon: ^^20:09
*** saurabhs has joined #openstack-trove20:20
*** datsun180b has quit IRC20:26
*** NehaV has quit IRC20:32
*** radez is now known as radez_g0n320:43
*** michael-yu has joined #openstack-trove20:45
*** Ranjitha has joined #openstack-trove20:46
*** datsun180b has joined #openstack-trove20:46
*** atomic77 has quit IRC20:47
*** khyati has quit IRC20:57
*** NehaV has joined #openstack-trove20:57
*** saurabhs has quit IRC20:59
*** NehaV has quit IRC20:59
*** NehaV has joined #openstack-trove20:59
*** saurabhs has joined #openstack-trove21:02
*** yidclare has joined #openstack-trove21:02
*** amytron has quit IRC21:08
*** amytron has joined #openstack-trove21:08
*** denis_makogon_ has joined #openstack-trove21:14
*** datsun180b has quit IRC21:19
*** denis_makogon has quit IRC21:21
*** denis_makogon_ is now known as denis_makogon21:21
*** dmakogon_ has joined #openstack-trove21:21
*** datsun180b has joined #openstack-trove21:23
denis_makogonViswaV, at the run() data is being streamed into the remote storage, _pre_backup - prepares the data, _post_backup drops it21:24
*** Esteban has joined #openstack-trove21:25
denis_makogonViswaV, so, i still don't see the problem in the way its being implemented, i don't think if you move _post_backup into the __exit__ it'll give a huge valueadd into the guest perfomance21:25
denis_makogonViswaV, got your email, this chat is being logged, so i can see all messages,21:26
denis_makogonViswaV, i don't think that this is the problem in the way backup execution organized.21:27
ViswaVthe point is when the post_backup drops it… it can't drop until save() (upload to storage) finishes…and not just wait only until run() finishes21:27
denis_makogonmat-lowery, ping21:32
*** yidclare has quit IRC21:32
mat-lowerydenis_makogon: hi21:33
denis_makogonmat-lowery, hi21:33
denis_makogonmat-lowery, saw your comments at https://review.openstack.org/#/c/80010/21:33
denis_makogonmat-lowery, i think you should copy them into the actual patchset, since commiter doesn't took into account your suggestions21:34
denis_makogonViswaV, please provide the example21:35
denis_makogonViswaV, real-life example21:35
mat-lowerydenis_makogon: I commented on an old patch set but I work with the committer so I just pointed them out to her. Thanks though.21:36
mat-lowerydenis_makogon: Is that what you mean? That I commented on a now out-of-date patch set?21:37
*** saurabhs has quit IRC21:37
denis_makogonmat-lowery, yeah, thats all21:38
denis_makogonmat-lowery, also, could please point the commiter that we have already registered blueprint for this patch, so, bug-report should be marked as duplicated21:39
denis_makogonmat-lowery, https://blueprints.launchpad.net/trove/+spec/validation-on-restore-by-the-backup-strategy21:39
denis_makogonmat-lowery, would be nice if you help with that21:40
ViswaVdenis_makogon:  We are talking about logic flow (based on how python "with…block" operates) and what is the appropriate bounds to place _pre and _post at… What real life example do you want me to offer? This is about potential race conditions21:40
ViswaVhttp://docs.python.org/2/whatsnew/2.6.html#pep-343-the-with-statement21:40
ViswaVThe object’s __enter__() is called before with-block is executed and therefore can run set-up code. It also may return a value that is bound to the name variable, if given. (Note carefully that variable is not assigned the result of expression.)21:40
denis_makogonViswaV, i know how "with" works21:41
denis_makogonViswaV, i asked for the example when current implementation would not work21:41
*** saurabhs has joined #openstack-trove21:41
mat-lowerydenis_makogon: Good point. I saw that. The bug addresses a bit more than just the blueprint. I'll let amcrn decide on the dupe or no dupe since he created the bug.21:41
ViswaVSo do you see how for example  ,  https://github.com/openstack/trove/blob/master/trove/guestagent/backup/backupagent.py#L105  can not be reached before everything in __enter__() finishes?21:41
ViswaVmeaning pre, run and post() would have to finish before you get to storage.save()  as implemented currently...21:42
ViswaVthat is why you have to split  pre,run before with block (in __enter__)   and  post after with block (in __exit__)21:43
*** thedodd has quit IRC21:44
*** khyati has joined #openstack-trove21:47
denis_makogonViswaV, somehow i agree, but from the other side, it depends how to interpretate whole flow21:47
denis_makogonbasically, the mail question when to clean-up21:48
denis_makogon*main21:48
ViswaVdenis_makogon:  as it exists today, my vote would be in the __exit__method of the runner.21:49
ViswaVAlso look at https://github.com/openstack/trove/blob/master/trove/guestagent/backup/backupagent.py#L17021:49
ViswaVthe restore implementation is cleaner because there is no complexity of "with block " and enter exit methods and the whole flow of when something happens or not.21:49
*** yidclare has joined #openstack-trove21:49
ViswaVNot sure why backup was implemented this way. May be there are reasons that I am not aware of. May be thats why pre and post steps weren't even there until you added them.  To me the restore implementation looks cleaner.21:50
denis_makogonViswaV, "with" was removed because it causes percona incremental restore errors21:50
ViswaVGot it. May be similar flow interoperation errors were made there.21:51
*** achampion has quit IRC21:51
ViswaVBut if you use with, which the backup logic is using, then we got to align with __enter__() and __exit__()21:51
denis_makogonViswaV, as for me its fine21:51
denis_makogonViswaV, its just the another way to interact with objects21:52
denis_makogonnothing else21:52
ViswaVdenis_makogon:  'fine' meaning you are ok moving the _run_post_backup() to  __exit__() method?21:52
ViswaVI see no harm in doing that. But potential harm in keeping the current mechanism as is.21:52
denis_makogonViswaV, i guess yes21:52
denis_makogonViswaV, its the easiest way to make clean up when it should happen21:53
*** Barker has quit IRC21:53
denis_makogonViswaV, i'll take it21:53
denis_makogonViswaV, thanks for the convo21:53
ViswaVCool. We can tackle it during any of the backup patch sets submission (cassandra, mongo etc). No one else is using them yet.21:53
ViswaVSure. np21:54
denis_makogonViswaV, it'll be the separate patch21:54
ViswaVsure21:54
denis_makogonViswaV, anyway this way of backuping is required by the non-streaming backup tools21:55
denis_makogonlike mongoumn21:55
denis_makogon*mongodump21:55
denis_makogonor nodetool21:55
ViswaVyup21:55
ViswaVwhere you have to simulate streaming via tar..21:56
*** kevinconway has quit IRC21:57
*** NehaV has quit IRC21:58
denis_makogonamcrn, thanks for updating the BP whiteboard21:58
amcrnnp21:59
*** Esteban has quit IRC22:01
*** NehaV has joined #openstack-trove22:01
amcrndenis_makogon: yeah, so if you wouldn't might writing up the specs (taking the comments into consideration), then we could review the bp on an upcoming monday22:01
denis_makogonamcrn, review which one ?22:02
amcrnhttps://blueprints.launchpad.net/trove/+spec/validation-on-restore-by-the-backup-strategy22:03
denis_makogoncool22:04
amcrnthe scope of this blueprint being the long-term solution22:04
*** yidclare has quit IRC22:04
amcrn(as compared to the in-flight https://review.openstack.org/#/c/80010/ which will only check datastore-version-id)22:05
denis_makogonamcrn, yes, agreed, unfortunately i'm going to skip BP-meeting and weelky meeting due to traveling22:05
amcrnah, vacation or work?22:05
denis_makogonamcrn, vacation22:05
amcrnniiice22:05
denis_makogonamcrn, far-far away from the internet22:06
denis_makogoni hope22:06
amcrndenis_makogon: that's always a good plan every once in awhile, decompress and enjoy the natural surroundings :)22:06
*** sriram_tesora has quit IRC22:06
denis_makogonamcrn, yeah, agreed =)22:06
denis_makogonamcrn, if you have time would be nice to hear your feedback on my feature designs (dblog, point in time recovery, data volume snapshot)22:07
denis_makogonamcrn, links are available at wiki.openstack.org/wiki/Trove#Development22:08
amcrni'll do my best to give them a look by friday22:08
*** timfreund has quit IRC22:08
amcrncoffee break, bbiab22:09
denis_makogonamcrn, have a nice day, Auston =))22:09
*** yidclare has joined #openstack-trove22:17
*** yogesh has quit IRC22:27
*** harlowja has quit IRC22:30
*** harlowja has joined #openstack-trove22:31
*** NehaV has quit IRC22:41
*** timfreund has joined #openstack-trove22:42
*** Ranjitha has quit IRC22:47
*** jcru has quit IRC22:49
*** eguz has joined #openstack-trove22:56
*** datsun180b has quit IRC22:57
*** grapex has quit IRC22:58
*** eghobo has quit IRC23:00
*** eguz has quit IRC23:00
*** jmontemayor has quit IRC23:11
*** rramirez has quit IRC23:17
*** amytron has quit IRC23:24
*** saurabhs has quit IRC23:24
*** achampion has joined #openstack-trove23:25
*** saurabhs has joined #openstack-trove23:27
*** denis_makogon has quit IRC23:32
*** yidclare has quit IRC23:40
*** atomic77 has joined #openstack-trove23:40
*** saurabhs has quit IRC23:47
*** saurabhs has joined #openstack-trove23:48

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!