Friday, 2015-07-24

« Thursday, 2015-07-23 Index Saturday, 2015-07-25 »
*** chlong has joined #openstack-trove00:10
*** pmackinn has joined #openstack-trove00:18
*** annashen has joined #openstack-trove00:32
*** mingdang1 has quit IRC00:34
*** pmackinn has quit IRC00:35
*** _amrith_ is now known as amrith00:37
*** annashen has quit IRC00:37
*** david-lyle has quit IRC00:40
*** esmute_ has quit IRC00:48
*** esmute_ has joined #openstack-trove00:53
*** pmackinn has joined #openstack-trove00:55
*** erkules_ has joined #openstack-trove01:08
*** erkules has quit IRC01:11
*** mingdang1 has joined #openstack-trove01:12
*** haomaiwang has quit IRC01:26
*** pmackinn has quit IRC01:30
*** amrith is now known as _amrith_01:33
*** annashen has joined #openstack-trove01:33
*** esmute_ has quit IRC01:38
*** pmackinn has joined #openstack-trove01:38
*** pmackinn has quit IRC01:42
*** mingdang1 has quit IRC01:46
*** erkules_ has quit IRC01:52
*** erkules_ has joined #openstack-trove01:53
SlickNiksabeen3: I think I know what you're running into01:58
SlickNiksabeen3: Please set use_heat = true in /etc/trove/trove-taskmanager.conf as well.01:59
SlickNiksabeen3: the taskmanager service (which is where you're running into that problem) uses this config.02:00
SlickNikcp16net: How go the travails with the pxc clustering work?02:02
*** esmute_ has joined #openstack-trove02:20
*** sabeen has joined #openstack-trove02:36
*** sgotliv has quit IRC02:37
*** rjaiswal has quit IRC02:44
*** ashuk has quit IRC02:59
*** ashuk has joined #openstack-trove03:00
*** ByPasS has quit IRC03:01
*** david-lyle has joined #openstack-trove03:17
openstackgerritMatthew Van Dijk proposed openstack/trove: Mongodb Cluster Scaling  https://review.openstack.org/20535503:26
*** sgotliv has joined #openstack-trove03:29
openstackgerritPetr Malik proposed openstack/trove: Improve the guestagent configuration manager  https://review.openstack.org/19859503:37
*** ashuk has quit IRC03:38
*** coolsvap|away is now known as coolsvap03:40
cp16netSlickNik: so i have not made it past the grant missing issue03:57
cp16netSlickNik: i have written all the other code that should make the instance become a cluster if the prepare will ever finish.03:58
cp16netSlickNik: i've tracked down the issue to when the guest is putting the configs in place. https://gist.github.com/cp16net/54b59b34579ca851968104:13
cp16netyou can see the debugs i added to show the mysql.user table.04:13
cp16netafter the prepare fails i was able to run through the cluster setup steps and it works... https://gist.github.com/cp16net/a22c1286ba94d494d7a004:14
*** esmute_ has quit IRC04:16
*** annashen has quit IRC04:24
*** annashen has joined #openstack-trove04:33
*** pmackinn has joined #openstack-trove04:50
*** sabeen has quit IRC05:00
*** coolsvap is now known as coolsvap|away05:00
*** coolsvap|away is now known as coolsvap05:06
*** mingdang1 has joined #openstack-trove05:08
*** coolsvap is now known as coolsvap|away05:12
*** esmute_ has joined #openstack-trove05:17
*** esmute_ has quit IRC05:21
*** bhunter71 has quit IRC05:21
*** ashleighfarnham has joined #openstack-trove05:27
*** mingdang1 has quit IRC05:45
*** coolsvap|away is now known as coolsvap05:54
*** ashleighfarnham has quit IRC06:17
*** pmackinn has quit IRC06:22
*** exploreshaifali has joined #openstack-trove06:24
*** pmackinn has joined #openstack-trove06:29
*** annashen has quit IRC06:31
*** annashen has joined #openstack-trove06:36
*** annashen has quit IRC06:36
*** pmackinn has quit IRC06:38
*** ashleighfarnham has joined #openstack-trove06:53
*** pcaruana has joined #openstack-trove07:10
*** pcaruana is now known as centos07:13
*** centos is now known as Guest5973007:13
*** Guest59730 is now known as pcaruana07:13
*** annashen has joined #openstack-trove07:24
*** k4n0 has joined #openstack-trove07:26
*** chlong has quit IRC07:33
cp16netlefred: you around?07:35
cp16neti just made an interesting break through... on this grant issue07:35
cp16netlefred: dbennett https://gist.github.com/cp16net/12db3f8544c8b604903a07:39
cp16netlooking at these new logs you can see that the files are NEVER updated after the call was made07:39
*** annashen has quit IRC07:52
lefredcp16net: I'm sure there is a configuration issue somewhere like using different path for datadir or something similar, have you tried without changing the my.cnf (like not copying the template) before restart just to verify ?07:55
cp16netso i've done that all manaully and its always fine07:56
cp16netwould the tmpdir matter?07:57
cp16neti see its changed from /tmp to /var/tmp07:58
cp16nettrying again without changing the installed my.cnf08:02
cp16netso here is something that is weird as well...08:06
cp16neti made the grant call again later in the process and it worked08:06
cp16netits like making that grant call as soon as the service started made it not happy08:07
cp16netwtf08:07
cp16netthats soooooooo weird08:07
lefredare you sure the grant succeeded ? do you have an output ?08:18
cp16netyeah08:18
cp16neti've run it a few times and its getting through08:19
cp16neti'll get a new log of it all08:19
cp16neti need to goto sleep since its 320am but i'm excited.... :-P08:20
lefredhehe08:20
lefredyou said you put it later in the process... after the change of my.cnf ?08:20
cp16nethttps://gist.github.com/cp16net/9031db0e41629575213e08:24
cp16netyeah08:24
cp16netwhich is strange to me08:25
cp16netbecause this has never been a problem before08:25
*** mingdang1 has joined #openstack-trove08:38
*** Quack has left #openstack-trove08:42
openstackgerritdangming proposed openstack/trove: correct annotation  https://review.openstack.org/20545109:04
*** coolsvap is now known as coolsvap|away09:39
openstackgerritPetr Malik proposed openstack/trove: Improve the guestagent configuration manager  https://review.openstack.org/19859510:03
*** mingdang1 has quit IRC10:33
*** logan2 has quit IRC10:47
*** logan2 has joined #openstack-trove10:50
*** openstackgerrit has quit IRC11:01
*** openstackgerrit has joined #openstack-trove11:01
*** mingdang1 has joined #openstack-trove11:07
*** mingdang_ has joined #openstack-trove11:35
*** mingdang1 has quit IRC11:36
*** sgotliv has quit IRC12:13
*** tomblank has joined #openstack-trove12:39
*** mingdang_ has quit IRC13:02
*** mingdang1 has joined #openstack-trove13:03
*** _crobertsrh is now known as crobertsrh13:29
*** exploreshaifali has quit IRC13:30
*** tristanC has quit IRC13:31
*** annashen has joined #openstack-trove13:32
*** tristanC has joined #openstack-trove13:32
*** radez_g0n3 is now known as radez13:43
*** annashen has quit IRC13:46
dbennettcp16net: can you post the output of `mysqld --verbose --help`  ?13:48
*** sgotliv has joined #openstack-trove13:51
*** tomblank has quit IRC13:53
*** lexholden has joined #openstack-trove13:55
openstackgerritMatthew Van Dijk proposed openstack/trove: Mongodb Cluster Scaling  https://review.openstack.org/20535514:16
*** k4n0 has quit IRC14:26
*** sabeen has joined #openstack-trove14:27
*** sabeen2 has joined #openstack-trove14:32
*** sabeen has quit IRC14:36
*** mingdang1 has quit IRC14:41
*** esmute_ has joined #openstack-trove14:42
*** saurabhs has joined #openstack-trove14:49
*** georgelorch has quit IRC14:57
openstackgerritMatthew Van Dijk proposed openstack/trove: MongoDB user management - access grant/revoke/show  https://review.openstack.org/19082115:00
openstackgerritPetr Malik proposed openstack/trove: Improve the guestagent configuration manager  https://review.openstack.org/19859515:15
*** georgelorch has joined #openstack-trove15:17
*** david-lyle has quit IRC15:44
openstackgerritMorgan Jones proposed openstack/python-troveclient: Redis Clustering Initial Implementation  https://review.openstack.org/20370115:44
*** esmute_ has quit IRC15:52
*** pmackinn has joined #openstack-trove15:53
*** bitblt has joined #openstack-trove15:56
*** david-lyle has joined #openstack-trove16:20
*** esmute_ has joined #openstack-trove16:27
*** radez is now known as radez_g0n316:38
*** pmackinn has quit IRC16:39
*** haomaiwang has joined #openstack-trove16:48
*** pmackinn has joined #openstack-trove17:02
*** annashen has joined #openstack-trove17:05
cp16netdbennett: https://gist.github.com/cp16net/e27c5f9256e98596efcd17:07
dbennettcp16net: ack,   I am testing some lower settings for you...17:30
cp16netdbennett: k these are from pxc 5.5 running on 512 instance17:31
cp16netthis is using the config template we have for mysql and percona17:32
dbennett512M will be a tight fit,  you are at runlevel 3 (no GUI) right?17:33
cp16netye17:41
*** mingdang1 has joined #openstack-trove17:42
*** esmute_ has quit IRC17:43
cp16netdbennett: heres a snapshot of processes running https://gist.github.com/cp16net/f9a05bfe4926f460304217:44
*** pmackinn has quit IRC17:46
cp16netnot to mention there are other processes that can be fired like backup that will use memory17:46
*** mingdang1 has quit IRC17:47
*** pmackinn has joined #openstack-trove17:47
*** esmute_ has joined #openstack-trove17:48
*** bhunter71 has joined #openstack-trove18:05
*** pmackinn has quit IRC18:08
*** pmackinn has joined #openstack-trove18:16
dbennettcp16net:  https://gist.github.com/dbpercona/f0495c2888faee292e0c18:17
dbennett"running" and synced in 128m RES18:18
*** annashen has quit IRC18:19
*** annashen has joined #openstack-trove18:22
cp16netthats awesome this is pxc 5.5 or 5.6?18:23
*** ashleighfarnham has quit IRC18:24
*** ashleigh_ has joined #openstack-trove18:24
dbennett5.6 x86_6418:24
dbennett# rpm -qa | grep -i percona18:25
dbennettPercona-Server-shared-51-5.1.73-rel14.12.624.rhel6.x86_6418:25
dbennettPercona-XtraDB-Cluster-56-5.6.24-25.11.1.el6.x86_6418:25
dbennettPercona-XtraDB-Cluster-galera-3-3.11-1.rhel6.x86_6418:25
dbennettpercona-xtrabackup-2.2.11-1.el6.x86_6418:25
dbennettPercona-XtraDB-Cluster-client-56-5.6.24-25.11.1.el6.x86_6418:25
dbennettPercona-XtraDB-Cluster-shared-56-5.6.24-25.11.1.el6.x86_6418:25
dbennettPercona-XtraDB-Cluster-server-56-5.6.24-25.11.1.el6.x86_6418:25
dbennettpercona-release-0.1-3.noarch18:25
dbennett---18:25
dbennettcp16net:  make sure you read through the my.cnf carefully and make sure to comment out any duplicate definitions of the settings given.18:26
cp16netok18:29
cp16neti dont see any dups in yours18:30
cp16netbut for the existing cnf i get it.18:30
openstackgerritMorgan Jones proposed openstack/python-troveclient: Redis Clustering Initial Implementation  https://review.openstack.org/20370118:37
*** radez_g0n3 is now known as radez18:38
*** esmute_ has quit IRC18:49
*** esmute_ has joined #openstack-trove18:53
*** esmute_ has quit IRC19:06
*** esmute_ has joined #openstack-trove19:07
*** pmackinn has quit IRC19:09
*** esmute_ has quit IRC19:12
*** pmackinn has joined #openstack-trove19:17
openstackgerritMatthew Van Dijk proposed openstack/trove: MongoDB user management - access grant/revoke/show  https://review.openstack.org/19082119:21
*** bitblt has quit IRC19:22
*** tomblank has joined #openstack-trove19:44
*** tomblank has quit IRC20:02
*** lexholden has quit IRC20:04
*** annashen has quit IRC20:10
*** esmute_ has joined #openstack-trove20:12
*** annashen has joined #openstack-trove20:23
*** pmackinn has quit IRC20:35
cp16netdbennett: so if you start mysql and you can use 'mysqladmin ping' it doesnt mean that the service is really ready to be used20:37
cp16neti think this might be part of the problem for the grant20:37
cp16netbecause i removed all my loging statements and now its failing to grant again20:37
cp16netits like as soon as the service is "alive" according to mysqladmin ping we start doing stuff20:38
cp16netand this doesnt seem to be really be ready...20:38
*** sgotliv_ has joined #openstack-trove20:51
*** crobertsrh is now known as _crobertsrh20:53
*** sgotliv has quit IRC20:54
*** pmackinn has joined #openstack-trove21:05
dbennettcp16net:  yes,  mysqladmin ping is a 'service ready' report.  A PXC node would need to synced with the cluster in order for 'mysqladmin ping' to return 0 (OK)21:19
dbennettcp16net: following these instructions to bootstrap (start) the cluster:   https://www.percona.com/doc/percona-xtradb-cluster/5.6/manual/bootstrap.html21:25
dbennetts/following/follow/21:27
*** rjaiswal has joined #openstack-trove21:27
cp16netdbennett: is that even if pxc is not setup with the cluster options?21:28
cp16netbecause this is before i put the cluster.cnf file in place21:28
dbennettcp16net:   if you boot a node with 'wsrep_cluster_address=gcomm://' (empty) then 'mysqladmin ping' should exit 0 almost immediately.  Check to see if you can `mysql -uxxx -pxxx -e select 1"`21:30
cp16nethmm yeah i'm thinking i need to wait to verify with the 'select 1;'21:31
dbennett><  /s/-e select 1"/-e "select 1"/21:32
cp16netyeah i gotcha21:32
dbennettthis is what  'service mysql bootstrap-pxc' does.21:33
dbennettit's different on Centos 7 BTW21:33
cp16neti see the note on that page21:33
cp16netdbennett: on another note...21:35
dbennetton Centos 7 it's 'systemctl start mysql@bootstrap.service' .   It doesn't actually set the address empty but includes  the '--wsrep-new-cluster' cli parameter which essentially does the same thing.21:35
cp16netdbennett: you can use the pxc-5.6 in place of percona5.6 without any cluster stuff setup?21:36
dbennettthen bring other nodes up normally,  check them with `SHOW GLOBAL STATUS LIKE '%wsrep%stat%'` and you should see 'Synced'.   On they are all synced you can restart the bootstrap node.21:37
dbennetts/On they/Once they/     :)21:37
dbennettcp16net:  if you leave the wsrep_cluster_address=gcomm:// then it should run just fine by itself.21:38
cp16netok21:39
cp16netbecause we were figuring out if we can use a single image for both the single instance percona and for the cluster21:41
dbennettyes that should work.21:41
cp16nettite21:41
georgelorchcp16net, generally that should work but I do not think it is a good idea at all21:41
cp16netwhy would that be bad?21:42
*** esmute_ has quit IRC21:42
georgelorchtoo many ways for an end user to shoot themselves in the foot or a support person to mistakenly assume that "Ohh I see that this is PXC so it is obviously running as part of a cluster". It us just all around not an expected configuration.21:43
cp16nethmm i see21:43
dbennettthat is a fair assessment21:44
georgelorchbut YMMV, do it if it saves you some kind of headache for now, just be prepared to get an eventual "I told you so" later on21:44
cp16nethah21:44
cp16netthats good advice21:44
cp16neti'll keep it in mind21:45
georgelorchalso know that PS and PXC operate on different release schedules and PXC is almost ALWAYS behind PS, so if  critical update comes along, PS will almost ALWAYS be released first with PXC hours, days or weeks behind21:45
cp16netbecause it would be somuch easier to use just a single image....21:45
dbennettalso,  MyISAM tables are not (will never be) supported with PXC replication.21:45
cp16netok21:45
* cp16net writes these notes down21:45
georgelorchdbennett, do you know if that is also true for a PXC node doing traditional async/binlog replication or does PXC effectively kill MyISAM repl at the root21:46
dbennettNo,  you can do traditional async rep off any node,  PXC will not prevent you from using MyISAM tables,  it just won't replicate them 'fully'21:47
dbennettbut MyISAM async (binlog) rep will work.21:48
dbennettMyISAM DDL statements are replicated and MyISAM DML statements that occur inside of a (InnoDB) transaction will replicate so it'21:48
dbennetts deceiving.21:48
*** annashen has quit IRC21:49
georgelorchyeah, that's what I was getting at, knowing that it's not supported by wsrep, but would still work on a PXC binary vua async21:49
dbennettBasically, the Defacto answer is not to use MyISAM at all in a PXC/cluster (aside from the sys table requirement)21:49
georgelorchyup, that much is understood for sure, but if they want to use this somewhat dubious "PXC as a vanilla MySQL" case then MyISAM in a traditional async will still work, but you're screwed if you then decide to bootstrap this bad boy into a cluster21:51
dbennettgeorgelorch:  that is a good point, using a PXC node by itself does not make you 'cluster ready'21:52
dbennettPlus, you also would not be able to upgrade the server to TokuDB.  Percona Server is usually a week or two ahead on upstream (Oracle) patches too.21:55
*** esmute_ has joined #openstack-trove21:55
georgelorchaahhh jajaja, and I should know all about Toku :) yeam AFAIK we aren't even going to ship TokuDB in PXC because it is completely unsupported in PXC21:56
georgelorchgood call dbennett, that alone for me would be enough to settle the debate ;)21:56
*** esmute_ has quit IRC21:59
*** esmute_ has joined #openstack-trove22:07
*** annashen has joined #openstack-trove22:09
dbennettalso serializable isolation is not across the entire cluster so certain unique sequence operations will not work as expected.   Basically (like any Enterprise tech) it requires a planning and audit phase.22:14
*** bhunter71 has quit IRC22:21
annashenthanks georgelorch to clarify the diff :)22:49
*** esmute_ has quit IRC22:58
*** esmute_ has joined #openstack-trove23:08
*** sabeen2 has quit IRC23:09
dougshelley66cp16net, just catching up...so unlike clustering in mongo and redis the database software isn't same for PXC23:09
dougshelley66it seems unnatural to force percona and percona clusters to be a single image23:10
dougshelley66also, you won't be able to cluster mysql or mariadb using this mech (i assume)23:10
*** annashen has quit IRC23:11
*** esmute_ has quit IRC23:14
*** haomaiwang has quit IRC23:16
*** esmute_ has joined #openstack-trove23:19
*** esmute_ has quit IRC23:56
*** esmute_ has joined #openstack-trove23:58
*** esmute_ has quit IRC23:59
« Thursday, 2015-07-23 Index Saturday, 2015-07-25 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!