Friday, 2015-08-07

« Thursday, 2015-08-06 Index Saturday, 2015-08-08 »
*** sharikapongubala has quit IRC00:02
*** esmute_ has quit IRC00:18
*** chlong has joined #openstack-trove00:27
*** sharikapongubala has joined #openstack-trove00:59
openstackgerritIshita Mandhan proposed openstack/trove: Code refactoring for couchdb  https://review.openstack.org/21017001:01
*** sharikapongubala has quit IRC01:04
*** sharikapongubala has joined #openstack-trove01:05
*** imandhan has joined #openstack-trove01:07
*** sharikapongubala has quit IRC01:09
openstackgerritoctopuszhang proposed openstack/trove: Allow user to select zone for their trove volume  https://review.openstack.org/20986401:14
*** imandhan has quit IRC01:15
*** sharikapongubala has joined #openstack-trove01:17
*** sharikapongubala has quit IRC01:18
*** erkules has joined #openstack-trove01:21
*** erkules_ has quit IRC01:23
*** sharikapongubala has joined #openstack-trove01:31
*** sharikapongubala has quit IRC01:36
*** sharikapongubala has joined #openstack-trove01:39
*** sharikapongubala has quit IRC01:43
openstackgerritoctopuszhang proposed openstack/trove: Allow user to select zone for their trove volume  https://review.openstack.org/20986401:44
*** sharikapongubala has joined #openstack-trove01:46
*** sharikapongubala has quit IRC01:51
*** sharikapongubala has joined #openstack-trove01:52
*** sharikapongubala has quit IRC01:54
*** mingdang1 has joined #openstack-trove01:54
*** sharikapongubala has joined #openstack-trove01:56
*** sharikapongubala has quit IRC01:58
*** sharikapongubala has joined #openstack-trove02:03
*** sharikapongubala has quit IRC02:04
*** sharikapongubala has joined #openstack-trove02:09
*** sharikapongubala has quit IRC02:12
*** pmackinn_ has quit IRC02:16
*** sharikapongubala has joined #openstack-trove02:28
*** sharikapongubala has quit IRC02:29
*** sharikapongubala has joined #openstack-trove02:39
*** sharikapongubala has quit IRC02:42
*** sharikapongubala has joined #openstack-trove03:24
*** rjaiswal has quit IRC03:40
*** sharikapongubala has quit IRC03:40
*** sharikapongubala has joined #openstack-trove04:01
*** sharikapongubala has quit IRC04:05
*** sgotliv_ has joined #openstack-trove04:11
*** sharikapongubala has joined #openstack-trove04:21
*** sharikapongubala has quit IRC04:24
*** sharikapongubala has joined #openstack-trove04:36
*** sharikapongubala has quit IRC04:40
*** sharikapongubala has joined #openstack-trove04:40
*** sgotliv_ has quit IRC04:48
*** sharikapongubala has quit IRC04:50
*** imandhan has joined #openstack-trove05:06
*** imandhan has quit IRC05:10
*** haomaiwang has quit IRC05:11
*** haomaiwa_ has joined #openstack-trove05:11
*** sharikapongubala has joined #openstack-trove05:16
*** sharikapongubala has quit IRC05:20
*** imandhan has joined #openstack-trove05:21
*** sgotliv has joined #openstack-trove05:49
openstackgerritNikhil Manchanda proposed openstack/trove-integration: Fix openstack endpoint list usage for stable branches  https://review.openstack.org/20719305:55
*** k4n0 has joined #openstack-trove06:11
*** imandhan has quit IRC06:19
*** nexusz99 has joined #openstack-trove07:15
*** dbennett has quit IRC07:59
*** chlong has quit IRC08:04
*** dbennett has joined #openstack-trove08:07
*** erkules has quit IRC08:29
*** erkules has joined #openstack-trove08:29
*** dbennett has quit IRC08:54
*** dbennett has joined #openstack-trove08:54
*** tomblank has joined #openstack-trove09:00
*** tosky has joined #openstack-trove09:32
*** IanGovett has joined #openstack-trove09:45
*** kmARC_ has joined #openstack-trove10:08
*** kmARC has joined #openstack-trove10:08
*** k4n0 has quit IRC10:43
*** k4n0 has joined #openstack-trove10:43
*** openstackgerrit has quit IRC10:45
*** raginbajin has quit IRC10:45
*** dhellmann has quit IRC10:45
*** abramley has quit IRC10:45
*** mingdang1 has quit IRC10:47
*** openstackgerrit has joined #openstack-trove10:49
*** raginbajin has joined #openstack-trove10:49
*** dhellmann has joined #openstack-trove10:49
*** abramley has joined #openstack-trove10:49
*** kmARC has quit IRC11:00
*** kmARC_ has quit IRC11:01
*** pmackinn has joined #openstack-trove11:03
*** kmARC_ has joined #openstack-trove11:05
*** kmARC has joined #openstack-trove11:05
*** kmARC has quit IRC11:18
*** kmARC_ has quit IRC11:20
*** k4n0 has quit IRC11:23
*** haomaiwa_ has quit IRC11:37
*** haomaiwa_ has joined #openstack-trove11:38
*** saurabhs has joined #openstack-trove11:55
*** chlong has joined #openstack-trove12:16
*** radez is now known as radez_g0n312:23
*** bapalm has joined #openstack-trove12:39
*** logan2 has quit IRC13:11
*** logan2 has joined #openstack-trove13:12
*** radez_g0n3 is now known as radez13:18
openstackgerritZhao Lei proposed openstack/trove: Remove quotes from subshell call in tools/with_venv.sh  https://review.openstack.org/21040913:20
*** _crobertsrh is now known as crobertsrh13:46
*** tomblank has quit IRC13:57
*** mingdang1 has joined #openstack-trove14:27
*** edmund has joined #openstack-trove14:29
*** radez is now known as radez_g0n314:33
*** edmund1 has joined #openstack-trove14:50
*** edmund1 has quit IRC14:50
*** edmund1 has joined #openstack-trove14:51
*** edmund has quit IRC14:52
openstackgerritPeter Stachowski proposed openstack/trove: Add Redis backup/restore functionality  https://review.openstack.org/18589214:56
openstackgerritPeter Stachowski proposed openstack/trove: Add support for Redis replication  https://review.openstack.org/20327814:58
pmackinnanyone care to explain test_volume_found to my small mind?15:01
*** sriram_tesora has joined #openstack-trove15:11
*** mingdang1 has quit IRC15:21
*** saurabhs has quit IRC15:26
*** tomblank has joined #openstack-trove15:28
SlickNikpmackinn: are you asking about the int-test?15:31
pmackinnSlickNik, yes...sems to fail for pgsql unless my setup is wrong somehow15:32
SlickNikpmackinn: interesting — not sure why that'd happen. Do you have a stack trace of the failure?15:34
SlickNikMaybe, it's possible that you have volume support turned off in your trove config under [pgsql], but not in your test config.15:35
pmackinnSlickNik, paste.openstack.org/show/412196/15:35
pmackinnSlickNik, http://paste.openstack.org/show/412196/15:36
*** logan2 has quit IRC15:38
*** logan2 has joined #openstack-trove15:41
SlickNikpmackinn: interesting - from looking at the code, this should be getting set here https://github.com/openstack/trove/blob/master/trove/guestagent/dbaas.py#L10315:42
pmackinnretrying now15:43
* pmackinn pushes on the pull door15:43
pmackinnsame deal15:44
SlickNikBut I wonder if there is a bug which is causing this test to fail — can you check your guestagent (or taskmanager) logs to see if there is an error corresponding to the call in https://github.com/openstack/trove/blob/d8d2d9f03aeac46e0e83c36c0f2487b4119371a5/trove/guestagent/datastore/experimental/postgresql/manager.py#L109 ?15:44
openstackgerritPeter Stachowski proposed openstack/trove: Add Redis backup/restore functionality  https://review.openstack.org/18589215:50
pmackinnSlickNik, whiich conf should have volume_support = True? taskmanager?15:51
SlickNikpmackinn: Yes, although from looking at the stack trace you sent me that doesn't seem to be the issue.15:54
pmackinnSlickNik, are we saying it should be turned off for env+test for pgsql? or pgsql tests should work with it enabled?15:55
*** esmute_ has joined #openstack-trove16:09
*** sgotliv has quit IRC16:12
*** saurabhs has joined #openstack-trove16:27
*** imandhan has joined #openstack-trove16:29
SlickNikpmackinn: The tests should work with pgsql + volumes enabled.16:30
*** bapalm has quit IRC16:31
*** esmute_ has quit IRC16:33
*** esmute_ has joined #openstack-trove16:33
*** ashleighfarnham has joined #openstack-trove16:35
saurabhsSlickNik: upstream for trove integration projects http://logs.openstack.org/86/181186/15/check/gate-trove-functional-dsvm-mysql-kilo/84e4e58/ and http://logs.openstack.org/86/181186/15/check/gate-trove-functional-dsvm-mysql-juno/89c137e/ tests are failing is that a known problem or am I doing something wrong16:37
saurabhsparticularly this patch : https://review.openstack.org/#/c/181186/16:37
SlickNiksaurabhs: They are failing on the stable branch tests16:37
SlickNikYou're not doing anything wrong.16:37
SlickNikI'm looking into fixing the stable juno / kilo tests.16:38
SlickNiksaurabhs: https://review.openstack.org/#/c/207193/ still WIP16:38
saurabhsSlickNik: thanks16:39
openstackgerritSushil Kumar proposed openstack/trove-specs: Datastore Registration API spec  https://review.openstack.org/18807216:48
sushilkmSlickNik, cp16net, vkmc I have updated the https://review.openstack.org/#/c/188072/ Datastore Registration API spec16:49
sushilkmplease review the same, thank you16:49
*** esmute_ has quit IRC16:51
*** esmute_ has joined #openstack-trove16:53
openstackgerritPeter Stachowski proposed openstack/trove: Add Redis backup/restore functionality  https://review.openstack.org/18589216:54
*** sriram_tesora has quit IRC16:55
*** esmute_ has quit IRC16:58
*** esmute_ has joined #openstack-trove16:58
*** exploreshaifali has joined #openstack-trove17:09
*** radez_g0n3 is now known as radez17:15
*** sharikapongubala has joined #openstack-trove17:15
sharikapongubalacan i please have some reviews on these? :) https://review.openstack.org/#/c/196913/ https://review.openstack.org/#/c/197360/17:16
*** rjaiswal has joined #openstack-trove17:19
*** ashleighfarnham has quit IRC17:29
edmondkvkmc, we could go ahead and close this bug out, as it hasn't been reproduced: https://bugs.launchpad.net/trove/+bug/139523117:33
openstackLaunchpad bug 1395231 in Trove "Database Create Instance Limit does not work, shows Quota exceeded though quota is available" [Low,Invalid] - Assigned to Edmond Kotowski (ekotowski)17:33
vkmcedmondk, yup :)17:33
vkmcsushilkm, reviewed, thanks!17:33
*** radez is now known as radez_g0n317:42
*** radez_g0n3 is now known as radez17:46
*** ashleighfarnham has joined #openstack-trove17:47
*** sriram_tesora has joined #openstack-trove18:03
*** sharikapongubala has quit IRC18:03
*** tosky has quit IRC18:03
sushilkmthank u vkmc18:03
*** saurabhs has quit IRC18:05
crobertsrhAny chance that any of you have managed to have the trove services start up in devstack by adding enable_plugin trove git://git.openstack.org/openstack/trove  to local.conf?  I have added it, it gets cloned, but I see no evidence of the processes trying to start.18:05
*** sharikapongubala has joined #openstack-trove18:06
*** saurabhs has joined #openstack-trove18:07
*** bhunter71 has joined #openstack-trove18:09
*** kmARC has joined #openstack-trove18:11
*** esmute_ has quit IRC18:11
*** kmARC_ has joined #openstack-trove18:12
*** bhunter71 has quit IRC18:13
edmondkpmackinn, I am getting the exact error on test_volume_found for mysql, looking into it now18:15
pmackinnoh cool thx edmondk18:15
*** pmackinn is now known as pmackinn-afk18:16
*** sharikapongubala has quit IRC18:18
*** radez is now known as radez_g0n318:19
*** radez_g0n3 is now known as radez18:21
*** bapalm has joined #openstack-trove18:26
*** sharikapongubala has joined #openstack-trove18:28
openstackgerritPeter Stachowski proposed openstack/trove: Add generic int-test classes  https://review.openstack.org/20327418:32
openstackgerritPeter Stachowski proposed openstack/trove: Add Redis backup/restore functionality  https://review.openstack.org/18589218:34
openstackgerritPetr Malik proposed openstack/trove: Add user actions int-tests  https://review.openstack.org/21009418:36
*** bapalm has quit IRC18:38
*** ashleighfarnham has quit IRC18:38
SlickNikcrobertsrh: I believe you need to enable the services explicitly as well — one sec, let me check on it.18:41
SlickNiki.e. enable the plugin + enable the trove services.18:41
openstackgerritMerged openstack/trove-specs: Datastore Registration API spec  https://review.openstack.org/18807218:42
*** sriram_tesora has quit IRC18:47
*** bapalm has joined #openstack-trove18:48
*** bapalm has quit IRC18:48
crobertsrhah, thanks SlickNik18:49
*** esmute_ has joined #openstack-trove18:50
SlickNikcrobertsrh: you'll want to add something like https://github.com/openstack/trove-integration/blob/master/scripts/localrc.rc#L19 your localrc / local.conf18:53
crobertsrhRight.  Somehow, that wasn't clear from the docs I was looking at.18:54
*** sharikapongubala has quit IRC18:54
*** sharikapongubala has joined #openstack-trove18:54
*** kmARC has quit IRC18:55
*** kmARC_ has quit IRC18:55
*** radez is now known as radez_g0n318:56
*** radez_g0n3 is now known as radez19:00
*** ashleighfarnham has joined #openstack-trove19:00
openstackgerritPeter Stachowski proposed openstack/trove: Add support for Redis replication  https://review.openstack.org/20327819:02
openstackgerritPeter Stachowski proposed openstack/trove: Add generic int-test classes  https://review.openstack.org/20327419:02
*** sharikapongubala has quit IRC19:07
*** sharikapongubala has joined #openstack-trove19:11
*** bhunter71 has joined #openstack-trove19:11
*** sharikapongubala has quit IRC19:14
SlickNikpmackinn-afk: Was looking at the volume bug we were talking about earlier with edmondk. Looks like we might have a bug upstream (race condition) that's causing this. Basically the instance may go to ACTIVE before the 'prepare' call has been run on the guest. Will19:19
cp16netyikes19:22
cp16netSlickNik: oh... i bet thats because the datastore is set to start on boot19:23
cp16netmaybe?19:23
cp16neti noticed something where on boot percona would start up after the perpare had started and shutdown mysql.19:24
SlickNikcp16net: Yes, that's what my guess was.19:24
cp16netit would start it back up19:24
SlickNikThe heartbeat goes out as "Up" even before prepare has completed.19:24
cp16netso in the image elements i added the line to disable mysql from starting19:24
cp16neti could see that happening19:24
SlickNikCausing the guest to flip to ACTIVE for a short time even before the volume is ready.19:25
cp16netthe guest starts mysql in the prepare any way so i figure thats good enough19:25
cp16netif mysql isnt started on boot19:25
cp16netits started at the end of install_nessesary_packages19:26
cp16netafter that i stopped running into that race condition19:26
vkmcSlickNik, did ya run those tests in redstack?19:27
SlickNikvkmc: Yes, in redstack.19:28
vkmcSlickNik, I ran them and hit another issues, but not that one in particular19:29
vkmcmaybe it is not always reproducible?19:29
*** exploreshaifali has quit IRC19:34
SlickNikvkmc: Yes, it's a race condition — the issue manifests only sometimes.19:35
vkmcops yes19:36
*** esmute_ has quit IRC19:36
SlickNikbasically dependent if the loopingcall which checks for ACTIVE happens to check at the time when the instance flips into ACTIVE erroneously (which happens for a short window)19:36
pmalikHi. Could we possibly push the configuraion race fix forward. I think it's already in a pretty good shape, just needing some pluses. :-) Thanks. P.19:51
pmalikhttps://review.openstack.org/#/c/198891/19:51
pmalik^^^^^ SlickNik peterstac cp16net vkmc _amrith_19:51
openstackgerritMorgan Jones proposed openstack/python-troveclient: Redis Clustering Initial Implementation  https://review.openstack.org/20370119:56
openstackgerritMorgan Jones proposed openstack/trove: Redis Cluster Initial Implementation  https://review.openstack.org/20370219:56
*** chlong has quit IRC20:06
*** pmackinn-afk is now known as pmackinn20:10
*** sharikapongubala has joined #openstack-trove20:10
pmackinnSlickNik, i found my race cond w/ kilo pgsql...reproduces everytime with a home-rolled pgsql image but never saw it before w/ mariadb or mongodb20:19
saurabhssushilkm, cp16net, amrith Please review https://review.openstack.org/#/c/181186/. I think I have addressed most of the recent/old comments on this patch and was able to test it end to end.20:33
saurabhsI added a seperate element to treat DATASTORE_PKG_LOCATION as file/directory/url so it should address the concerns everybody had about treating it as a file/directory20:34
*** logan2 has quit IRC20:56
edmondkpmackinn, we think we nailed down the issue that is happening20:57
edmondkpmackinn, for mysql case mysql is started before prepare is finished which causes a fake ACTIVE state to appear, then mysql is stopped in prepare and prepare sets the status back to BUILD, then prepare finished and sets the status back to ACTIVE20:59
edmondkpmackinn, for the trove volume test it is run when it the first fake ACTIVE status appears but when the test does the instances.get(id) the status is back to actual BUILD state which the volume['used'] won't show up21:00
pmackinnedmondk, volume['used'] won't persist across these states i take it21:01
edmondkSlickNik and I think this issue has been in since the conductor was added but has only shown up recently because of the oslo messaging addition21:01
edmondkpmackinn, the volume does indeed exist but since it's in BUILD the trove API will not return the volume information21:02
edmondksince it is a race condition the speed of your environment also will matter21:03
edmondkI am running on a powerful system that is faster than our upstream integration test system which could be the reason we both see the issue locally and not on the gate21:03
pmackinnedmondk, so none of http://bit.ly/1OVUJRM can come back through the API even though it must have been previously computed when ACTIVE21:04
edmondkyeah I think so21:07
pmackinnedmondk, gotcha21:08
*** crobertsrh is now known as _crobertsrh21:09
edmondkbecause we check http://paste.openstack.org/show/412234/ on the view21:09
edmondkI have to check but when its in build volume_used maybe false21:09
pmackinnyeah trying to find where that logic is21:16
pmackinnSlickNik, edmondk: shouldn't the volume attributes ahead of 'used' also thrown an error? http://bit.ly/1OVXH8D21:21
SlickNikpmackinn: No it's completely possible that they don't.21:25
SlickNikpmackinn: The 'size' comes from the actual cinder volume which is created and provisioned.21:26
SlickNikpmackinn: It's only 'used' (i.e. the actual used space on the volume) which needs to be queried from the guest and sent back.21:26
SlickNikpmackinn: ANd since the guest is not fully active at this point, it's missing.21:26
pmackinnSlickNik, considered missing :-)21:27
pmackinnSlickNik, ok makes more sense21:27
pmackinn'used' is special in the API21:27
SlickNikyeah, used'  is special in that dict since we need to actively query the guest to get the value for it.21:28
pmackinnso ACTIVE->BUILD and thrown away by ga, set to None, etc.21:29
*** logan2 has joined #openstack-trove21:39
openstackgerritCraig Vyvial proposed openstack/trove: Percona Xtradb Cluster implementation  https://review.openstack.org/20807321:59
openstackgerritCraig Vyvial proposed openstack/trove: Percona Xtradb Cluster implementation  https://review.openstack.org/20807322:02
openstackgerritPeter Stachowski proposed openstack/trove: Add generic int-test classes  https://review.openstack.org/20327422:03
openstackgerritIshita Mandhan proposed openstack/trove: Code refactoring for couchdb  https://review.openstack.org/21017022:04
openstackgerritPeter Stachowski proposed openstack/trove: Add Redis backup/restore functionality  https://review.openstack.org/18589222:05
openstackgerritPeter Stachowski proposed openstack/trove: Add user actions int-tests  https://review.openstack.org/21009422:06
openstackgerritPeter Stachowski proposed openstack/trove: Add support for Redis replication  https://review.openstack.org/20327822:07
pmackinnwhoa, peterstac ftw22:16
openstackgerritIshita Mandhan proposed openstack/trove: Code refactoring for couchdb  https://review.openstack.org/21017022:24
openstackgerritSharika Pongubala proposed openstack/trove: Modifying Vertica clusters to have a master node  https://review.openstack.org/20427122:35
*** radez is now known as radez_g0n322:36
*** edmund1 has quit IRC22:39
*** ashleigh_ has joined #openstack-trove22:42
*** ashleighfarnham has quit IRC22:49
*** imandhan has quit IRC22:49
*** georgelorch has quit IRC22:49
*** pmackinn has quit IRC22:49
*** nexusz99 has quit IRC22:49
openstackgerritSharika Pongubala proposed openstack/trove: Modifying Vertica clusters to have a master node  https://review.openstack.org/20427122:50
*** nexusz99 has joined #openstack-trove22:52
*** georgelorch has joined #openstack-trove22:53
openstackgerritPetr Malik proposed openstack/trove: Redis Cluster Initial Implementation  https://review.openstack.org/20370222:55
edmondkpmackinn, our suspicion was correct. I wrote a quick fix in the set_status on the guest to check when in building if it was the trove prepare method calling heartbeat of the datastore and I was able to get the volume test to pass22:57
edmondkHere is the current quick fix I made: http://paste.openstack.org/show/412242/22:58
edmondkthe first if status == instance.ServiceStatuses.Building: section22:58
edmondkDiscussing with SlickNik for a cleaner fix and I am going to try adding a check for if /var/lib/data exists send a heartbeat otherwise do not23:01
SlickNikedmondk: Yeah, I think that's a bit cleaner23:03
SlickNikYou can read the data dir from the CONF file, so we don't have to explicitly rely on /var/lib/data23:04
SlickNikBut that way the hearbeat won't be explicitly sent out until the prepare call runs.23:05
edmondkCreated bug for issue: https://bugs.launchpad.net/trove/+bug/148279523:10
openstackLaunchpad bug 1482795 in Trove "On create instance the instance status alternates between BUILD->ACTIVE->BUILD->ACTIVE" [High,New] - Assigned to Edmond Kotowski (ekotowski)23:10
edmondkI will test the /var/lib/data fix23:12
openstackgerritSharika Pongubala proposed openstack/trove: Root enablement for Vertica clusters/instances  https://review.openstack.org/19691323:15
*** logan2 has quit IRC23:15
openstackgerritSharika Pongubala proposed openstack/trove: Modifying Vertica clusters to have a master node  https://review.openstack.org/20427123:16
openstackgerritNikhil Manchanda proposed openstack/trove-integration: Fix openstack endpoint list usage for stable branches  https://review.openstack.org/20719323:26
*** nexusz99 has quit IRC23:35
*** imandhan has joined #openstack-trove23:42
openstackgerritNikhil Manchanda proposed openstack/trove-integration: Fix openstack endpoint list usage for stable branches  https://review.openstack.org/20719323:53
« Thursday, 2015-08-06 Index Saturday, 2015-08-08 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!