Friday, 2013-10-04

*** matsuhashi has joined #openstack-trove00:28
*** amcrn has quit IRC00:42
*** nosnos has joined #openstack-trove00:45
*** jrodom has quit IRC00:55
*** jrodom has joined #openstack-trove00:59
openstackgerritMike Metral proposed a change to openstack/trove-integration: Sets correct MYSQL_HOST needed to run stack.sh  https://review.openstack.org/4965901:06
*** yogeshmehra has joined #openstack-trove01:10
*** yogeshmehra has quit IRC01:14
*** jrodom has quit IRC01:25
*** erkules has quit IRC01:27
*** erkules has joined #openstack-trove01:45
*** showka has joined #openstack-trove01:51
openstackgerritA change was merged to openstack/trove-integration: Sets correct MYSQL_HOST needed to run stack.sh  https://review.openstack.org/4965902:12
*** jrodom has joined #openstack-trove02:20
*** jrodom has quit IRC02:41
*** vipul has quit IRC02:45
*** vipul has joined #openstack-trove02:46
*** haomaiwang has quit IRC03:03
hub_capvipul: around?03:06
vipulhub_cap: yea03:07
hub_cap Why is a slave node not treated as a node, what's so special about it that it needs to be different03:07
hub_capdid u mean for replication?03:07
hub_capor clustering03:07
vipulYea for replication03:07
hub_capfor replication, a slave is different03:07
hub_capu want to know the ip03:07
hub_caplike a read slave03:07
hub_capu also may want to shirnk the ram03:07
hub_capcuz it doesnt do a bunch of updates03:07
hub_capu also might want to push a custom config for better read performance03:08
hub_capthose things make u want to make it a independent isntance03:08
vipulBut I would argue that you'd want those things for clusters too03:09
vipulI want to know the IPs of other nodes in a cluster03:09
hub_capnot normally03:09
vipulbecause I could potentially use them as read only salves03:10
hub_capnot for a cluster03:10
hub_capcassandra has no read slaves03:10
hub_capgalera, if used in active-active (true cluster) doesnt either03:10
vipulGalera.. I may only want to write to a single node03:11
vipulbut read from the other nodes03:11
vipulOr put them behind haproxy03:11
hub_capsure and u can get their ips03:11
hub_capitll likely be in the data u get back from a instance get03:11
hub_capbut u wuldnt want to make one bigger or smaller than the rest03:11
vipulBut how is that different from a replication slave03:11
hub_capor put a different configuration on it03:12
hub_capu _just_ want its ip03:12
vipulSure..03:12
hub_capthats the difference03:12
hub_capa cluster is homogenous03:12
hub_capreplication is not03:12
hub_capa cluster resize = resize all nodes03:12
hub_capread replica resize = just shrink node X03:12
vipulOk, that's fair.. configuration and flavor might be homogenous03:13
hub_capi think everything is03:13
hub_capexcept u might want to know the IPs03:13
hub_capand sure ill give u that, that makes sense03:13
hub_capbut wrt replication, _most_ of the time its not homogenous03:14
vipulI just feel like we're bastardizing the replication concept.. besides it not being homogenious.. it's still managed03:14
vipullike it still supports failover, etc03:14
hub_capnot automatic03:14
hub_capclusters self promote03:14
hub_capmysql master-maste does not03:14
vipulWhy not.. why couldn't we make replication do that too03:14
hub_capthe differnce is03:14
hub_cap_we_ vs builtin cluster software03:15
hub_capreplication is not HA / fault tolerant by design03:15
hub_capits HA but not fault tolerant03:15
vipulsure.. but it's very likely that Trove would make it fault tolerant03:15
hub_capeventually sure03:15
vipuland when we do03:15
vipulwhy does it feel different03:16
hub_caplets figure taht out when we do03:16
vipulwhy is it a completely differnt experience provisioning it03:16
hub_capbecause clustering != replication03:16
vipulI feel like you could still put it in the same instace->node concept..03:16
vipulthrowing out the homogenious factor03:16
hub_capya but then you have a scenario where u cant muck w/ nodes of a particular service_type03:17
hub_capso the nodes are no longer equal03:17
hub_capin terms of what u can do to them across diff service types wrt clusters03:17
vipulOk.. let's talk about the service type aspect..03:17
hub_capsure03:17
vipulare we going to allow somone to migrate between service types?03:17
hub_capeventually i think its fine03:17
vipulas a user.. i may choose to start small.. single instance03:17
*** demorris has joined #openstack-trove03:18
hub_capright off the bat i dont think so03:18
vipulhowever to upgrade, it's really a different service type03:18
hub_caphell the codership guys recommend creating a new cluster and migrating to it03:18
hub_caprahter than doing it in place03:18
hub_capand codership wrote galera :)03:18
vipulBut you couldn't do a cluster of 1...03:19
hub_capso i think its perfectly sane to say03:19
hub_capmake a backup03:19
vipulso you really can't add nodes if you start small03:19
hub_cappass in a backupUUID to a create03:19
hub_capservice_type=galera, backupUUID=blah03:19
hub_capand it spins up a new cluster given that uuid03:19
hub_capif u just want master/slave, u can just add a slave via POST /instance03:19
*** Barker has quit IRC03:24
vipulstill digesting the etherpad.. just seems slightly odd to me that we're treating it differnt03:26
vipuljust the fact that it's something done by US vs. built-in shouldn't be a reason to expose it differently03:27
vipulif i choose to do master-master-master.. it's likely that i'm going to make it homogenious03:27
vipulyet.. it's treated as a diffrent animal in Trove03:28
vipulBecause in either case.. as a user.. i just may want to add nodes03:29
vipulhowever doing so is very differnt depending on whether i chose Galera or not03:29
vipulDo you see other use cases (other service types) that would behave like master/slave.. besides mysql03:35
hub_capi believe redis deos teh same03:35
hub_cap*Does03:35
hub_capbut i think thats it03:35
hub_capim open to suggestions03:36
hub_capwe just assumed that for the most part u wouldnt care about the ips of each node for clustering03:37
vipulI do like tying everything to /instances .. making 'node' a subresource of an instance03:37
hub_capfor sure03:37
vipuli dislike treating replication different03:37
vipulsince to the end user.. does it really matter?03:37
vipulthat they did replication vs chose a cluster03:37
vipul(if both support failover, etc)03:38
hub_caphomogenous vs heterogenous matters03:38
hub_capwell they dont both support failover03:38
hub_capthats one of the main things03:38
vipulTrove does..03:38
vipulregardless of the underlying tech03:38
hub_captrove _may_ in the future03:38
vipulmaybe not on day1 ..but it iwll03:38
hub_capi think we all say that, but im not entirely sold on it being doable03:39
hub_cap:P03:39
hub_capbut, if you need to use galera like a replica, instead of a cluster03:39
hub_capthe difference between them is that w galera u _only_ need to get the ips03:40
hub_capbut lets chat w/ daniel tomrorow about it03:40
hub_capif hes feeling ok03:40
hub_capi think he got a stomach bug03:40
hub_capbut effectively if u set up haproxy, youre telling me "gimme a loadbalancer on top of my cluster"03:41
hub_capits still a cluster03:41
hub_capand you are using haproxy to do the work03:41
vipulsure.. but i do feel that a lot of cluster will require knowledge of every node03:42
hub_capim not sure thats the case03:43
vipuljust like master/slave/read replication will03:43
hub_capthat will, for sure03:43
vipulI could see use cases that use a cluster for sharding or something03:43
vipuljust target a shard to a node in a cluster03:44
vipuland you could say the same about multi-master03:44
hub_capis that a cluster tho?03:45
hub_capif its got different data03:45
hub_capand you have tos hard03:45
hub_cap*shard03:45
hub_capor just a bunch of machines w/ your own hashing algoritms03:45
hub_caplike what happens when your shard dies? if it was a cluster, it was writen to some other node03:46
hub_capand a rebalance fixes it03:46
hub_caplike cassandra, you write to any node03:46
vipulYep, exactly.. every shard is replicatd to every other node03:47
hub_capwell03:47
hub_capright thats a cluster, and internally the tech does it03:47
vipulbut you still want to leverage the toehr nodes in a cluster03:47
hub_capnot w cassandra03:47
hub_capnot w a true cluster03:47
vipulso they aren't sitting there idle.. and waiting to be promoted03:47
vipulif i had a write heavy app, i'd probably shard, and span my writes amongst all nodes03:48
vipulas long as i'm not doing anything that would cause the same rows to be written to on differnt nodes.. the cluster hadnles it all03:48
vipulit replicates those shards across to the toher nodes..and i could read from anywhere03:49
hub_capexactly, so u shouldnt care about what node to read from03:49
hub_capu wouldnt want to hardcode ips into your client app for that03:49
vipulbut how do i configure the IPs to write to03:50
vipulunless i know them all03:50
vipulin cassandra.. you said you could write anywhere03:50
vipulso wouldn't you want to know all the nodes.. and just round-robin ?03:51
vipulor are we assuming there is always a single endpoint exposed.. and the logic to choose a node is hidden from the user03:51
hub_capi was assuming a single endpoint03:52
hub_capbe it haproxy, or round robin dns03:52
hub_capim nto sure how apps typically write to these tho03:52
hub_capsometimes client side connector code does it too03:52
hub_capim not saying u dont need to know all the ips... at least i dont thnk i am lol03:53
hub_capim trying to figure out if we need to enact upon cluster nodes in a different ways from replication intances03:53
hub_capim trying to simplify a cluster, but give you replication flexibility03:53
vipulSure.. so if it's true that you might want to know about nodes in a cluster.. it's similar to needing to know about replica nodes03:53
vipuli'm trying to make that connection03:54
hub_capbut you acdess the replicas differently03:54
hub_capu dont read from nodes 6~8 in cassandra and write to nodes 1~303:54
hub_capu read and write to any o them03:54
hub_capand, in theory, same w/ galera (if it didnt have limitations and need haproxy)03:55
hub_capwhereas in replication, you very often have small memory, tuned read slaves03:55
hub_capdiff configs03:55
hub_capdiff ram03:55
vipulYea, i think the thing that makes sense to me is.. if we're exposing a single endpoint with cluster.. then it sorta makes sense to spearate the replica instances03:55
openstackgerritSteve Leon proposed a change to openstack/trove: Add tenant id to guest_info file  https://review.openstack.org/4967103:55
vipulbecause then it's like you have a uber-instance with one entrypoint03:56
vipulwhich can't be the case with replicas03:56
hub_capthats what a cluster is tho right?03:56
vipulbut if we're opneing up the uber-instance... and exposing individual nodes..03:56
vipulthen what makes it different03:56
hub_capthe actions you do to them03:56
hub_capnever do u want different nodes in cluster, always do u in read replicas03:57
hub_capmaybe master-master is a cluster... and we an think of it as such03:57
hub_capand replication is really just "read replicas"03:57
hub_capbut im not sure, at least in the beginning, i want to be adding in haproxy stuff too lol03:58
hub_capwe can use quantum tho for  lbaas03:58
vipulOk.. so now if we say that even read-replicas could be fail'd over to..03:59
hub_caphttp://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-replication-multi-master.html <-- calls multi-master a cluster03:59
*** adrian_otto has joined #openstack-trove03:59
hub_capread replicas can be promoted, yes03:59
vipulthen is it that much different besides maybe the flavors03:59
hub_capwell the config will have to be updated, the flavor, and the relation to the other slave04:00
hub_capif there is > 1 slave04:00
hub_capor read replica04:00
vipulWhat if that slave if a hidden slave..that's used for HA only04:00
vipulwould 'hidden slave' be a service type?04:00
*** adrian_otto has quit IRC04:01
vipulbecasue then there is a single endpoint04:01
vipulyou only write to the master.. we just promote the slave04:01
vipulit seems in that scenario.. master/slave falls into the cluster group04:02
vipulIt's just confusing when the user has to know about the salve04:02
hub_capwould the user know abotu that slave even?04:02
hub_capim not sure it applies if its not known to them or in their /instances in any way04:03
vipulBut it is a service-level they are buying.. they want only a single node.. but with soem security blanket04:04
vipulit could be known in /instances as a node.. but maybe not it's IP or something04:04
hub_caphmm04:04
vipulit seems that would make sense as a new service-type04:04
hub_capya04:04
vipulSo far the distinction of an 'Instance' = a single endpoint.. makes sense.. but the more we start exposing node details04:06
vipulthen i question why we separate read replicas04:06
hub_capsry sec04:08
hub_capputting wife/baby to sleep04:08
hub_capok04:11
vipulMaybe that's what we gotta try to group stuff into.. if we indeed think they are separate beasts04:12
vipulinstance is one endpoint.. period04:12
hub_capa read replica in a diff AZ04:12
hub_capnot sure thatll be easy to grok in a single instance04:12
hub_capits doable yes04:12
hub_capbut i see it as this (let me gist)04:13
hub_capbut i guess you mgith want to distribute a cluster over AZs too04:15
vipuldefinitely.. but is that something that you'd expose to user.. the ability to choose04:16
vipuli could see the use case wehre replica they choose04:16
vipulbut cluster has a scheduler tat decides for them04:16
*** haomaiwang has joined #openstack-trove04:19
hub_caphttps://gist.github.com/hub-cap/682088404:19
hub_capyes that makes sense vipul04:20
hub_capcluster = hands off, id think04:20
hub_capreplica = you choose all the details04:20
hub_capi think im ok w/ saying multi-master is a cluster04:22
hub_capeven tho its replication lol04:22
vipulyea so the only thing that's make replicas differnt now is flavor04:23
vipuldoes that warrant a different API to provision04:23
hub_cappossibly not, so then does master-slave become a service_type?04:23
vipulpossibly..04:24
hub_capor do we go back to service_type=mysql cluster_type=master-slave,master-master,galera,tungsten04:24
vipullol04:24
vipulugh04:24
vipuli feel like that's where we are04:24
vipuljust changing terminology04:24
hub_capya04:25
hub_caplets do this04:25
hub_capim tired04:25
hub_capive got some more autocad shit to do b4 i go to bed04:25
hub_caplets chat w/ dsal tomorrow04:25
hub_capimsplitbit: ^ ^04:25
vipulsure, it's a good start..thanks for putting something up04:26
vipuli think we need to think about all these use cases04:26
*** demorris has quit IRC04:27
hub_capyup04:28
hub_capcuz even cassandra, u need to know the ips i think04:29
hub_capto let the clients know (client software balances reads/writes)04:29
hub_cap.setSeeds("h1,h2,h3")04:31
vipulYea and i think regarding differences in 'actions per service type'.. we said that replicas would have diffeent supported actions vs clusters04:31
vipulbut if you think about it.. rebalanc ewon't be supporte by all clusters04:31
vipulso we'd probably have some cases where all operations won't be homogenous acros cluster types04:32
hub_capfo sure04:32
*** mmcdaris has joined #openstack-trove04:32
vipulanyway.. time to get some beers04:32
vipultalk tomororw04:32
hub_capyup04:32
hub_capcu04:32
*** mmcdaris has quit IRC04:48
*** mmcdaris has joined #openstack-trove04:56
*** showka has quit IRC05:06
*** ashestakov_ has joined #openstack-trove05:19
*** SushilKM has joined #openstack-trove05:29
*** mmcdaris has quit IRC05:38
*** mmcdaris has joined #openstack-trove05:46
*** ashestakov_ has quit IRC05:48
*** matsuhashi has quit IRC06:01
*** yogeshmehra has joined #openstack-trove06:07
*** nosnos has quit IRC06:08
*** metral has quit IRC06:08
*** dmakogon_ipod has joined #openstack-trove06:09
openstackgerritSteve Leon proposed a change to openstack/trove: Add tenant id to guest_info file  https://review.openstack.org/4967106:13
*** krow has joined #openstack-trove06:13
*** debasish has joined #openstack-trove06:15
*** rushiagr has joined #openstack-trove06:21
PradeepChandaniAnybody have idea how to mock  a variable in python06:23
*** Kapil has joined #openstack-trove06:24
*** matsuhashi has joined #openstack-trove06:24
*** SushilKM has quit IRC06:27
*** krow has quit IRC06:30
*** krow has joined #openstack-trove06:30
*** dmakogon_ has joined #openstack-trove06:36
*** dmakogon_ipod has quit IRC06:37
*** mmcdaris has quit IRC06:45
*** nosnos has joined #openstack-trove06:54
*** mmcdaris has joined #openstack-trove07:16
*** rushiagr has quit IRC07:18
*** mmcdaris has quit IRC07:20
*** metral has joined #openstack-trove07:25
*** rushiagr has joined #openstack-trove07:29
*** tvb has joined #openstack-trove07:32
*** rushiagr has quit IRC07:39
*** ashestakov_ has joined #openstack-trove07:41
*** dmakogon_ has quit IRC07:55
*** rushiagr has joined #openstack-trove08:02
*** kevinconway has quit IRC08:07
*** Nate2 has quit IRC08:07
*** Nate1 has joined #openstack-trove08:07
*** kevinconway has joined #openstack-trove08:07
*** kevinconway has quit IRC08:08
*** redthrux has quit IRC08:08
*** Nate1 has quit IRC08:08
*** ashestakov_ has quit IRC08:13
*** SushilKM has joined #openstack-trove08:25
*** ashestakov_ has joined #openstack-trove08:47
*** yogeshmehra has quit IRC08:57
*** yogeshmehra has joined #openstack-trove08:57
*** SushilKM has quit IRC09:00
*** Kapil has quit IRC09:02
*** yogeshmehra has quit IRC09:02
*** redthrux has joined #openstack-trove09:05
*** Nate1 has joined #openstack-trove09:14
*** rushiagr has quit IRC09:15
*** rushiagr has joined #openstack-trove09:16
*** kevinconway has joined #openstack-trove09:17
*** Kapil has joined #openstack-trove09:25
*** Kapil has quit IRC09:29
*** Kapil has joined #openstack-trove09:34
*** rushiagr has quit IRC09:35
*** rushiagr has joined #openstack-trove09:38
*** Kapil has quit IRC09:45
*** tvb has quit IRC09:52
*** tvb has joined #openstack-trove09:53
*** yogeshmehra has joined #openstack-trove09:58
*** yogeshmehra has quit IRC10:02
*** yogeshmehra has joined #openstack-trove10:08
*** yogeshmehra has quit IRC10:12
*** matsuhashi has quit IRC10:28
*** krow has quit IRC10:53
*** dmakogon has left #openstack-trove10:56
*** rushiagr has quit IRC10:57
*** rushiagr has joined #openstack-trove11:06
*** SushilKM has joined #openstack-trove11:21
openstackgerritDenis M. proposed a change to openstack/trove: Update statuses on GA timeout  https://review.openstack.org/4572311:21
openstackgerritDenis M. proposed a change to openstack/trove: Security groups workflow update  https://review.openstack.org/4570811:25
*** nosnos has quit IRC11:32
*** rushiagr has quit IRC12:00
*** pdmars has joined #openstack-trove12:02
*** rushiagr has joined #openstack-trove12:09
*** SushilKM has quit IRC12:21
*** dmakogon has joined #openstack-trove12:37
*** debasish has quit IRC12:40
*** showka has joined #openstack-trove12:49
*** Barker has joined #openstack-trove12:51
*** jrodom has joined #openstack-trove12:58
*** demorris has joined #openstack-trove13:10
*** yogeshmehra has joined #openstack-trove13:10
*** robertmyers has joined #openstack-trove13:15
*** jcru has joined #openstack-trove13:16
*** yogeshmehra has quit IRC13:17
*** robertmyers has quit IRC13:18
*** robertmyers has joined #openstack-trove13:18
*** amytron has joined #openstack-trove13:33
*** djohnstone has joined #openstack-trove13:41
*** djohnstone1 has joined #openstack-trove13:42
*** djohnstone has quit IRC13:45
*** russellb is now known as rustlebee14:11
*** rnirmal has joined #openstack-trove14:14
*** jcru has quit IRC14:17
*** jcru has joined #openstack-trove14:18
*** jmontemayor has joined #openstack-trove14:19
openstackgerritRobert Myers proposed a change to openstack/trove: Fix the fake nova server implementation  https://review.openstack.org/4975914:25
*** datsun180b has joined #openstack-trove14:47
*** haomaiwang has quit IRC14:49
*** tvb has quit IRC14:55
*** amytron has quit IRC14:56
*** SushilKM has joined #openstack-trove15:06
*** paul_lodronio has joined #openstack-trove15:15
*** adrian_otto has joined #openstack-trove15:19
*** djohnstone1 is now known as djohnstone15:19
openstackgerritEd Cranford proposed a change to openstack/trove: Trove-conductor.  https://review.openstack.org/4511615:21
*** SushilKM has quit IRC15:22
*** djohnstone has quit IRC15:23
*** djohnstone has joined #openstack-trove15:23
*** djohnstone has quit IRC15:25
*** djohnstone has joined #openstack-trove15:25
datsun180boh hubris! i keep not asking pep8 if it likes my code15:30
openstackgerritEd Cranford proposed a change to openstack/trove: Trove-conductor.  https://review.openstack.org/4511615:35
datsun180bgrumble grumble blank lines expected15:36
*** tanisdl has joined #openstack-trove15:41
*** showka has quit IRC15:43
*** demorris has quit IRC15:58
*** demorris has joined #openstack-trove15:59
hub_capdatsun180b: cant u set that up in vim to run flake8 automagically?16:01
datsun180bif i wanted an ide i'd use pycharm. i like to keep vim pretty minimal. besides, i this was happening because i was running tox -epy26 while making sure my unit tests passed16:03
datsun180bif i'd run tox without -e i would've seen it immediately, i just got cocky16:03
dmakogonhub_cap: datsun180b: what's up16:04
hub_caplol datsun180b, for the record ive put flake8 into emacs and i see a nice little underline if ive got an offensive line16:04
datsun180bemacs16:04
hub_capsince ive added it i havent had issues w tox16:04
*** amytron has joined #openstack-trove16:04
hub_capu can do teh same in vim, its not making it an "ide"16:04
hub_capor u can keep submitting 'tox'ic code LULZZZZZ16:05
dmakogonhttps://review.openstack.org/#/c/45723/ |||||| https://review.openstack.org/#/c/48305/16:05
datsun180boh i know, i'm being unnecessarily unfair16:05
hub_capdmakogon: hey man16:05
hub_capdid u see me and metral fixed the gate?16:05
dmakogoni've heard today is a big day for review))16:05
hub_capwell metral did, i just watched16:05
hub_capdmakogon: it is!!! no code for me today :(16:05
hub_capjust reviewing16:05
dmakogonnice16:05
datsun180bisn't today day 1 for icehouse16:05
hub_caphonestly it depends on the RC1's16:06
hub_capbut at least 1/2 of them, yes16:06
dmakogonalso, today we are going to discuss Heat templates in Trove)))16:06
*** mmcdaris has joined #openstack-trove16:06
hub_capyup dmakogon16:06
datsun180btab groups are the best thing for gerrit reviews16:06
datsun180bjust pull a review tab into its own group before you hit Side-by-Side16:06
hub_caphmm ill have to look @it16:07
hub_capare they a firefox thing? or does chrome have too?16:07
datsun180bbeats me16:07
datsun180bi only use chrome for flash16:07
hub_capim assumign yer using ff16:07
datsun180bin ff you hit command-shift-e16:07
metralteam effort between hub_cap and i :D16:08
hub_cappssh16:09
metrallol16:09
hub_capmetral: fixed it ;)16:09
hub_capdmakogon: i havent reviewed (just looked over quickly) the secgrps, it looks good i thin16:10
hub_capthink16:10
hub_capill review more in a bit16:10
dmakogonok16:10
dmakogonthanks16:10
dmakogoni've got a lot of review without +/-16:10
dmakogonit would be nice16:10
dmakogonbtw, Pradeep is taking to long with his update of service registry16:11
hub_capdmakogon: do u _need_ it done for some reason? are you planning on deploying a custom service?16:19
hub_capif not, give him time16:19
hub_capyou dont have to do everything ;)16:20
dmakogoni know16:20
dmakogoni've already helped him with some troubles16:20
hub_caphe might be on vacation for a day or two for all we know, and have the code done on his local git16:20
dmakogoni thought that it would not take a lot of time to do it =(16:20
hub_capits also possible hes not working on trove 100% of his time16:20
hub_capdatsun180b: id like you to look @ this review since u did the host work originally https://review.openstack.org/#/c/47543/16:23
datsun180bon it16:23
hub_capby the way datsun180b16:23
datsun180bi've been kind of heads-down with conductor16:23
hub_capyou have 2 emails on launchpad, as 2 users16:23
hub_capdo u mean to do that?16:23
hub_capu can join accts on launchpad, fwiw16:23
datsun180bi thought i had16:23
dmakogonguys, does trove has yaml lib as requirements ?16:23
*** jmontemayor has quit IRC16:23
hub_capoh intersting16:23
datsun180bi should be unified as ed--cranford16:23
hub_capdmakogon: no16:23
dmakogoni thinks he should16:23
hub_capis yaml not built in?16:23
dmakogonPyYAML ?16:23
*** jmontemayor has joined #openstack-trove16:23
hub_capoh duh nm16:24
hub_capsee if its in global requiremetns16:24
hub_caphttps://github.com/openstack/requirements/blob/master/global-requirements.txt#L7816:24
hub_capso we can add when yogesh does the heat work16:24
kevinconwaydmakogon: what do you need YAML for?16:25
dmakogonheat templates16:25
dmakogonstoring them in yaml files, not in code16:25
dmakogoneasy to change16:25
dmakogonwithout recompiling16:26
kevinconwayyou know… YAML is a superset of JSON16:26
kevinconwayI wonder how HEAT would react to being given a JSON template16:26
kevinconwaysince JSON is valid YAML16:26
datsun180bi don't really have a problem with that host code, but one of the tests is confusing empty for null16:27
kevinconwayalso, do we need to actually parse YAML or just generate it for HEAT?16:27
dmakogonkevinconway: no generation!16:27
kevinconwayyou just want static templates?16:28
dmakogonyes16:28
hub_capkevinconway: are you sure about that? yaml doesnt quote strings, for one16:28
dmakogonadmin could easily change it16:28
kevinconwayhub_cap: YAML usually can't be parsed into JSON, but JSON is valid YAML16:28
hub_capok ic what u mean16:28
hub_capwhat youre saying is to use json, and parse into yaml (maybe) when we send to heat16:29
isvirido`hi, everybody16:29
kevinconwayi was half joking. you should be able to send json to HEAT and it will just understand it as YAML16:29
kevinconwaybut if we're going to do YAML we should probably just do YAML16:29
hub_capdokr16:30
hub_capdork16:30
hub_cap;)16:30
isvirido`kevinconway, +1 it handles json as well16:30
kevinconwaybut if we don't actually need to load and parse the YAML then just a static file or a JINJA2 template would work16:30
kevinconwayand JINJA2 is already a dependency16:30
hub_capyes we shoudl use jinja16:30
hub_capfor the "pieces"16:31
hub_capif possible :)16:31
hub_capbut at the same time, giving pyyaml a dict will turn it into a nice yaml doc for you... so im sure there will be pro's/con's16:32
hub_capbut there is static junk that a jinja template would be great for16:32
datsun180bso for my conductor work to really get through i need that devstack change to work and pass review. what's the etiquette for rechecking devstack reviews if i don't feel like i caused the test failures16:32
isvirido`actually we shouldn't do any parsing, template is just a string16:33
hub_capdatsun180b: send me the link yer talking about16:33
hub_capisvirido`: its a collection of strings, that will have to be built (think clustering, or think, adding a node to an existing cluster)16:33
datsun180bhub_cap: https://review.openstack.org/#/c/49237/16:34
isvirido`hub_cap, are you going to generate that template? What is use case?16:34
datsun180bjinja is great, let's use it for everything16:35
isvirido`To add the node, we can update instance number parameter for InstanceGroup resource16:35
datsun180bhungry? eat a jinja template. cold? wrap yourself up with jinja templates.16:35
*** tvb has joined #openstack-trove16:35
*** demorris has quit IRC16:35
hub_capisvirido`: fair point. we may be able to externalize enough of the variables such that we dont need to16:36
hub_caphow bout turning a single instance into a master/slave16:36
datsun180bgenerating output from structured data? use jinja templates to keep the presentation work in one place and keep from making assumptions about use cases16:36
datsun180bmaybe not everyone needs the ips of their instances or cares about how user and host are different fields of a user16:36
hub_capdatsun180b: just try a recheck no bug, i mean, its been 3 days16:36
datsun180bhub_cap: i don't want to invoke any ire from devs i don't know16:37
hub_capthen ill do it16:37
hub_capjust say teh word16:37
datsun180bi did it16:37
datsun180beggs and omelettes16:37
*** SnowDust has joined #openstack-trove16:37
hub_capLOL16:37
SnowDustTGIF !16:37
isvirido`hub_cap, yes parametrize the template with all need parameters and keep set of templates for different service types16:38
SnowDustis it heat templates .. still continuing ? :)16:38
hub_caplol SnowDust we havent technically started yet, we are waiting for yogesh16:39
dmakogonyes16:39
hub_capbut yes we started generalizing ideas16:39
SnowDustisvirido +1 from my side ..16:39
SnowDusti said that yesterday :) .. the gist is rock solid from dmakogon16:40
isvirido`For booting several instance with different configuration we can just declare ec2instance resource for master f.e. and instancegrouo for replicas. Each will have own userdata16:40
SnowDustbut .. we should keep the template as easy .. for any admin ..who knows heat but never spent any time digging trove16:40
hub_capisvirido`: awesome, SnowDust agreed16:41
hub_capwe should spend some time creating these templates16:41
isvirido`SnowDust, +1 yes, and propably in future give the possibility to tune it by admin16:41
kevinconwayjinja supports template that let you set defaults but then another dev can import them and extend them with custom block content16:41
hub_capdmakogon: that woudl be a good way to share the work, maybe one of yall does the template work, then another does the code work16:41
hub_capkevinconway: im not sure we will _need_ jinja, since heat is its own templating system16:41
isvirido`kevinconway, with any template generation we are loosing flexibility of template tunning16:42
dmakogonhub_cap: lets wait for yogesh to split tasks16:42
hub_capyup dmakogon16:42
kevinconwaytwo different uses of the same term are they not?16:42
hub_capyes but we will have to use a jinja template to turn a static template into a static string16:43
isvirido`oh... i mean only HEAT templates16:43
kevinconwaya HEAT template is a kind of configuration script. a JINJA template is a parametrized block of text.16:43
hub_capand then give heat the template variables16:43
kevinconwayeeeew. does HEAT take it's own variables in a template?16:43
hub_capyes16:43
dmakogonhub_cap: i just want to keep templates in one folder, each should be named as si_service-template.yaml and cl_service-template.yaml16:43
kevinconwayboo16:43
kevinconwaynever mind16:43
hub_capdefinitely dmakogon16:43
hub_caplol kevinconway16:44
hub_capkevinconway: https://github.com/openstack/heat-templates/blob/master/cfn/deb/MultiNode_DevStack.yaml16:44
isvirido`hub_cap,  what do you mean "to turn"? won't we just load it and handle as a string?16:44
hub_capsee Parameters section16:44
hub_capcorrest isvirido` we will, i was just point out to kevinconway why we didnt need jinja16:44
hub_capwhen is the heat chat today?16:45
isvirido`yeap, it is already powerfull enogh16:45
hub_cap1900 dmakogon?16:45
hub_capor 1800?16:45
dmakogon11 AM16:45
SnowDustlets keep the templates independent .. in their atomic state16:45
hub_cap11am pacific?16:45
hub_capSnowDust: ++16:45
isvirido`SnowDust, +116:45
dmakogonprobably yes16:46
datsun180bi'm seeing two senses of 'template' if i'm following this discussion properly16:46
SnowDusta admin doesnot have to fool with the class /object  cruft16:46
SnowDustan*16:46
hub_capdatsun180b: you are, but we dont care about one of them (jinja)16:46
hub_caplets pretend we never said jinja today16:46
dmakogonhub_cap: 11 AM PDT16:46
datsun180bjust making sure i'm keeping up16:46
hub_capdmakogon: <316:46
hub_capdatsun180b: yes ur :)16:46
SnowDustwhat ever we do .. on HEAT calls  .. thats internal .. we can make it better no objections :)16:46
hub_capHAHAHAHAH yes SnowDust16:46
hub_capits impossible to make it worse at this point16:47
SnowDusthehehe .. understand hub_cap ;)16:47
SnowDusttemplates can be organized in directory .. for each service_type16:48
dmakogonSnowDust: really ?))))16:48
isvirido`SnowDust, +116:48
SnowDustyeah why not .. its only os.path to fiddle it with :)16:48
SnowDustand get a dictionary .. for service type .. and actions .. on that service type16:49
SnowDusteach action is the template itself ..16:49
hub_capi dont think we will have > 1 template for service_type, right?16:49
hub_capservice_type.heat.template16:49
SnowDustnope .. multiple template ... 1 node .. n node cluster ..  loadbalanced  n node .. and so on !16:50
isvirido`whould be easier to manage16:50
*** shivamshukla_ has joined #openstack-trove16:51
dmakogonhub_cap: one template per single instance, one for cluster16:51
* isvirido` swithching locatoin but will join soon16:51
dmakogontemplates would be stored in etc/heat_templates16:52
SnowDustyeah .. demakogon .. so for a lamer .. its an action name .. ..  1_node(.yaml) .. n_node(.yaml) ... n_node_lb(.yaml)16:52
SnowDustdmakogon .. why not /var ?16:52
dmakogonwhy it should be there ?16:52
SnowDustjust asking :)16:53
SnowDustit should be a configurable path i meant ... hehe16:53
dmakogontrove already hat etc dir16:53
dmakogonyes, in config16:53
hub_capdmakogon: SnowDust ok im going AFK for ~1 hr16:56
hub_capill bbl16:56
*** yidclare has joined #openstack-trove17:13
*** SushilKM has joined #openstack-trove17:14
*** amytron has quit IRC17:21
*** yogesh has joined #openstack-trove17:21
SnowDustyogesh: welcome !17:22
yogeshhey, whaaup17:23
yogeshwhassup...17:23
yogeshwe are meeting at 11:00 PT for templates...right...17:24
SnowDustmeeting is continuing ..17:24
SnowDustdmakogon is here too17:24
yogeshawesome17:24
yogeshgreat...17:24
SnowDusthub_cap will be .. here in 40 mins17:24
yogeshi need to go in 6 mins for a meeting...17:24
yogeshwill be back in around half an hour17:26
yogeshbut i'll be participative.. :-)17:28
yogeshdmakagon: whassup....any new direction you guys thought of...17:29
esmutevipul, hub_cap, SlickNik: When your busy time allows. Pretty easy review. https://review.openstack.org/#/c/49671/17:30
*** SushilKM has quit IRC17:32
datsun180bwoohoo, https://review.openstack.org/#/c/49237/ passed17:37
*** SushilKM__ has joined #openstack-trove17:41
*** Seal has joined #openstack-trove17:41
*** yogesh has quit IRC17:41
*** yogesh has joined #openstack-trove17:42
*** yogesh has quit IRC17:43
*** yogesh_ has joined #openstack-trove17:43
*** krow has joined #openstack-trove17:47
*** yogesh_ has quit IRC17:51
*** yogesh has joined #openstack-trove17:52
*** dmakogon_ has joined #openstack-trove17:52
dmakogon_i'm here17:54
*** isviridov has joined #openstack-trove17:54
dmakogon_whaazzaaaaaaaaaaaa17:54
*** yogesh_ has joined #openstack-trove17:54
*** yogesh has quit IRC17:54
SnowDustzzzzzzzzzzz17:55
SnowDustyawning .. !17:55
dmakogon_today my cat is selebrating17:55
dmakogon_animal-care day17:56
dmakogon_he's drunk and want to fight17:56
isviridovyou are caring about him in a strange way17:57
dmakogon_yogesh, ping17:57
dmakogon_isviridov17:57
dmakogon_ping17:57
dmakogon_it's his day, let it be17:58
isviridovdmakogon_, pong17:58
dmakogon_hub_cap: pijg17:58
dmakogon_ping17:58
isviridovdmakogon_, trace17:58
SnowDustisviridov: u insulted  the cat :)17:58
SnowDustu called it an animal on caring day  LOL17:58
isviridov))17:59
dmakogon_current agenda https://blueprints.launchpad.net/trove/+spec/provide-custom-heat-templates-for-each-service-in-trove17:59
dmakogon_isviridov, i would like you 2 examine it17:59
isviridovk17:59
*** SushilKM__ has quit IRC18:00
*** VinodGupta has joined #openstack-trove18:00
*** yogesh_ has quit IRC18:00
*** yogesh has joined #openstack-trove18:01
*** yogesh has quit IRC18:01
*** yogesh has joined #openstack-trove18:02
yogeshhello...18:02
SnowDusthello again ..18:02
isviridovhi, yogesh18:02
hub_capok im back! let me scroll up in history18:03
dmakogon_what's up, yogesh18:03
dmakogon_i've updated BP a bit18:03
yogeshdmakagon: hub_cap: hi18:03
dmakogon_yogesh: typo in my nick18:03
yogeshoops...18:04
hub_capdmakogon_ we can never tell what version of dmakogon* you are ;)18:04
dmakogon_yogesh, my name is Denis Makogon18:04
dmakogon_lol18:04
*** jasonb365 has joined #openstack-trove18:04
hub_capits dmakogon dmakogon_ dmakogon-ipod18:04
yogeshdmakogon: got it....thanks18:04
dmakogon_hub_cap: my nicks are flexible18:04
yogeshyeah...18:04
yogeshper device...18:04
hub_caplol dmakogon_ my nick is rigid, u always know who i am18:04
hub_capi use a irssi proxy so i can connect to my user from anywhere18:04
hub_capand see my history18:04
dmakogon_nice18:05
dmakogon_so, shell we ?18:05
yogeshthats great...18:05
yogeshyup...18:05
hub_capya u should look into it, cuz then u can connect to your user from > 1 device (i use it on my phone too and youd never know)18:05
dmakogon_current state: keeping templates at etc/heat_templates18:05
yogeshabsolutely...i was having tuf time with multi dvices..18:05
dmakogon_Let's keep templates in etc/heat_templates18:05
yogeshlemme share my gist...18:05
dmakogon_1 per service single instance, 1 per service cluster.18:06
dmakogon_that is what we have discussed yesterday18:06
yogeshhttps://gist.github.com/mehrayogesh/682284918:06
yogeshthis is generic and i wanted this meeting to happen before we go further...18:06
dmakogon_Cluster-Flavor Management - is a part of Cluster API18:07
yogeshit will be the existing flavor api...18:07
dmakogon_flavor API ?18:08
yogeshto get the flavors per sevice types...18:08
yogeshthe service type / flavor mapping work...18:09
dmakogon_yogesh: do we really need them ?18:09
yogeshyes...it helps in establishing the CONTEXT for the template..18:09
dmakogon_when user passing flavor, he suppose to know if it is enough for service type18:09
yogesh.i am assuming multiple templates per service type18:09
dmakogon_what the difference ?18:10
yogeshthe template factory needs to know what template needs to be instantiated..18:10
yogeshthe CONTEXT part in the gist..18:10
dmakogon_i see18:11
isviridovIn https://wiki.openstack.org/wiki/Trove-Replication-And-Clustering-API we have clustertype, whay we need cluster flavor?18:11
hub_capwe dont need cluster flavor18:11
hub_capsorry just got back into the conversation18:11
yogeshsure...18:11
dmakogon_hub_cap: i think no18:11
hub_caplets _not_ talk clusters for now, the cluster API will be changed in the next few weeks18:11
yogeshfor deciding the epcific template...18:11
hub_capyogesh: its entirely possible we might just make a cluster a new service_type (thats an idea i had)18:12
dmakogon_yogesh: what is the difference between templates for flavors ?18:12
isviridovhub_cap, +118:12
hub_capso galera-cluster will be its own service_type, making configurations and guest impl simple18:12
yogeshservice_type for cluster?18:12
hub_capand a flavor wil ljust apply to all "nodes" in a cluster18:12
dmakogon_hub_cap +118:13
hub_capsee why i didnt want to talk clusters yogesh :)18:13
yogeshthats fine...18:13
hub_capits an idea i had yesterday18:13
hub_capit makes the code simpler18:13
yogeshso do we limit this discussion to single template...per service type18:13
dmakogon_+118:13
dmakogon_yes18:13
hub_capyes lets figure out, for now18:13
hub_caphow to make existing heat templates better18:13
SnowDusthub_cap .. single template means .. callable in the code .. but templates can be nested .. as allowed by heat18:13
dmakogon_i've described to to make it pluggable18:14
yogeshthen just externalizing and making it cconfigurable...18:14
dmakogon_*how to18:14
yogeshthe point 1 in gist, where the CONTEXT is configurable per service type18:14
dmakogon_yogesh +118:14
hub_captrue SnowDust18:14
yogeshthe point 1 in the gist..18:14
dmakogon_SnowDust +118:14
yogeshallows for mnexsted templates..18:14
yogeshnested*18:15
hub_capso lets try to figure out how to make templates external/extensible, right?18:15
dmakogon_we just loading template, no matter what would be in it18:15
yogeshyes...18:15
yogeshdmakogon: +118:15
dmakogon_hub_cap +118:15
dmakogon_so, could i suggest a algorithm of loading ?18:16
hub_capdmakogon_ sure18:16
yogeshit'll be a straight template load op, rigth?18:16
yogeshread the template content and pass it on to HEAT...18:16
dmakogon_as yogesh said we need factory which would load template according to service type18:17
SnowDustjust make service-type a folder .. and actions .. a template_name(.yaml) inside them18:17
dmakogon_templates in etc/heat_templates18:17
dmakogon_SnowDust : yes18:17
yogeshsnowdust: +118:17
yogeshactually now we don't need a factory...18:17
dmakogon_hub_cap: should we keep templates in YAML ?18:17
hub_capid prefer a factory-factory. lets rewrite in java!!!!! ;)18:18
yogeshi mean, it is loading a single flavor of the template18:18
yogesh:-)18:18
dmakogon_we need HeatTemplateLoader18:18
isviridovLet it be at least /etc/trove/heat_templates18:18
hub_capyes to yaml18:18
yogeshisviridov: +118:18
dmakogon_and templates in *.yaml18:19
hub_capitll be just like the way configuration code is18:19
hub_capbut w/o the need for jinja18:19
yogeshmakes sense18:19
SnowDustlets .. make flavor a parameter for the template .. it can .. load any type defined as its distinguished options ..18:19
hub_capbut u know... if we use jinja, it will make the loader easier18:19
dmakogon_hub_cap +118:19
hub_caplet me expand18:19
hub_caphttps://github.com/openstack/trove/blob/master/trove/common/template.py#L1718:19
hub_capyou can use a external directory, or a package18:20
isviridovFactory whould better, and used by different service types18:20
dmakogon_hap_cap: we could load yaml.load(..)18:20
dmakogon_isviridov +118:20
isviridovSnowDust, disctinary of parameters?18:20
hub_capdoes load work on files dmakogon_?18:20
dmakogon_yes18:20
SnowDustin heat .. u can define a list of values as an acceptable parameter ..18:20
yogeshwhat parameters do we envision?18:21
dmakogon_http://pyyaml.org/wiki/PyYAMLDocumentation18:21
dmakogon_stream = file('document.yaml', 'r')18:21
isviridovSnowDust, yes. Let us leave it flexible to define any parameter in template and pass them via dictionary to heat18:21
dmakogon_yaml.load(stream)18:21
SnowDustbut why we are loading the YAML  ?18:21
isviridov?18:21
SnowDustif heat -client  api has that function ?18:22
hub_capi dont think we need to use yaml if we arent manipulating it right?18:22
hub_capits just a string that u pass to heat18:22
SnowDusttrue : hub_cap !18:22
hub_capSnowDust: can u find docs for that? or the code?18:22
dmakogon_hub_cap: yes18:22
dmakogon_we can18:22
hub_capso i say we say _no_ to pyyaml unless we need to manipulate yaml18:22
isviridovyes, actually of can be json as well if admin prefers18:22
hub_capagreed?18:22
dmakogon_hub_cap: agreed18:22
hub_capid guess u can pass a file to heat client18:23
SnowDustdidnt understand .. why manupulate ..18:23
yogeshhub_cap: +118:23
isviridovstrongly agreed18:23
hub_capSnowDust: we arent manipulating18:23
SnowDustan admin authors the template .. / changes it !18:23
hub_capcorrect18:23
dmakogon_+118:23
SnowDustwe just pass the parameters .. flexibly to it :)18:23
SnowDustfor each .. service type18:23
SnowDustthats the discussion ..18:23
hub_capwe should cfg.template_dir the directory too18:23
hub_capso a operator can change it18:23
yogeshyes...18:23
hub_capwith the default to etc/trove/heat_templates18:24
SnowDusthub_cap : agreed18:24
hub_capand then we will have to add them to the distribution too (manifest.mf stuff)18:24
hub_cap*manifest.in18:24
isviridovyeap18:24
yogeshwhere do the parameter get loaded...18:24
yogeshparameters...18:25
dmakogon_hub_cap: template dir in taskmanager.conf18:25
hub_capyes dmakogon_18:25
yogeshright now they get done from within taskmanager..18:25
hub_capteh applicaiton will define most of the parameters18:25
yogeshare we saying that same set of params will be required for all service types..18:25
hub_capthe things tha can be external are user_data (like dmakogon_'s gist)18:25
isviridovprepared by service_type implementation and passed as dictionary18:25
SnowDustcan be .. the same "parameter=value;parameter2=value2;" approach ..18:25
dmakogon_hub_cap: <318:25
SnowDustover command line ..18:25
hub_capyogesh: lets figure that out when we have > 1 service lol18:25
yogeshtechnically, we have... :-)18:26
dmakogon_for now we are blocking parameters list18:26
hub_capok lets figure that out when we have > 1 service that needs different parameters ;)18:26
dmakogon_just using fixed set of parameters18:26
hub_capwe can refactor later :)18:26
SnowDusthub_cap : thats why i said ..18:26
SnowDustkeep it over command line18:26
SnowDustso that every service type .. receives a parameter string ..18:26
yogeshhub_cap: it is about peer-peer clustering and master-slave clustering...18:27
SnowDustwhich will be parsed ..18:27
dmakogon_hub_cap: i thinks we only need custom userdata, nothing else18:27
*** krow has quit IRC18:27
dmakogon_should be easy18:27
yogeshdoes it align with the cluster implementationt hat is being worked upon...18:27
dmakogon_yes18:27
isviridovwhy not dictinary?18:27
isviridovwhy we have to parse anything?18:27
dmakogon_yogesh: perfectly18:28
SnowDustisviridov .. after parsing .. it can be a dict ..18:28
dmakogon_sorry, i missed something, dict for what ?18:28
yogeshdict for params18:28
isviridovSnowDust, where we getting that parameters for parsing?18:28
SnowDustisviridov .. if u dont parse .. u are limited to a service type .. by your dict .. in config ..18:28
dmakogon_wowowoow18:28
kevinconwaySnowDust: what's the desire to parse?18:28
dmakogon_let's stop a bit18:28
dmakogon_for now we are blocking parameters18:29
SnowDust;) yes  ! NP18:30
yogeshwhat will be the params?18:30
isviridovparameters for heat template18:30
isviridovthe data we are sending to heat together with template as a string18:30
dmakogon_for the next sprint we could store or get parameters from somewhere18:30
yogeshthats fine...18:30
yogeshbut we are talking about a single param list for all service types...would it be same as defined int he heat template in template.py18:31
SnowDustyogesh: it cant be same for all service type ..18:31
isviridovIt can be really different18:31
yogeshyup...so we need to have different params...18:31
isviridovdisctionary?18:31
SnowDustisviridov : parsed to dictionary :)18:32
dmakogon_isviridov: could we look at template and than define what parameters we need and how much ?18:32
SnowDustso that .. during the client request .. its parsed into key/values18:32
yogeshso template manipulation is required....to add the param values per service type..18:32
hub_capwhen its different we will talk about this i think18:32
hub_capfor now it should be same right?18:32
hub_capfor percona/mysql18:33
dmakogon_yes18:33
dmakogon_hub_cap +18:33
dmakogon_118:33
yogeshtrue18:33
hub_capits just for provisioning an instance for now18:33
hub_caponce we do cool stuff w/ clusters, lets focus on fixing it18:33
hub_capfor now, lets make heat for instances not suck lol18:33
isviridovOk. Let us just resume to it when we will be closer18:33
dmakogon_let's discuss about tasks ?18:33
SnowDusthub_cap : its same cuz .. we never thought this week .. redis will hit us :)18:33
hub_capim not sure it will18:34
yogeshsnowdust: elaborate18:34
hub_capitll still need an instance of a certain flavor/disk18:34
hub_capall the _different_ things will be in the redis.heat.template18:34
SnowDusti saw redis .. is getting into trove ..18:34
hub_capyes it is18:34
hub_capbut remember the differences, like what packages to install, will be in the tempalte itslef18:34
hub_cap*itself18:34
dmakogon_hub_cap: who we delegate task for configuration if template loading ?18:34
openstackgerritA change was merged to openstack/trove: Add tenant id to guest_info file  https://review.openstack.org/4967118:34
hub_capi think yogesh was the one who brought this up, so ill let him decide that dmakogon_18:35
isviridovhub_cap, with cloud init you can define list of packages to install18:35
hub_capisviridov correct18:35
hub_capso the redis.heat.template will define that18:35
yogeshi can get it done..18:35
hub_capand the mysql.heat.template will define something else18:35
hub_capas well as the percona.heat.template18:35
yogeshthe params still stay same, right?18:35
hub_capwhat im sayin is that the _differences_ will be defined in the template18:36
hub_capcorrect yogesh18:36
hub_capthe userdata shoudl be loaded externally though18:36
hub_capas in, you can do custom goofy things if u want18:36
dmakogon_true18:36
yogeshwhy can't it be a part of the template itself...18:36
hub_capi guess thats a good point18:36
isviridovuserdata can be also paramatrizeds18:36
hub_capif u want to do goofy stuff18:36
hub_capput it in the custom template in your installation18:36
hub_capright?18:36
SnowDustwe are mixing it :)18:37
hub_capif you want to use pypy instead of stock python18:37
hub_capthen put that in your template. we wont support it18:37
hub_caplike the example dmakogon_ had was to add custom java stuff18:37
hub_capbut we migth need that for cassandra anyway, so it might be in the stock cassandra.heat.template18:37
*** krow has joined #openstack-trove18:37
hub_capand if its not, u can edit your template in /etc/trove/heat_templates in your installation, right?18:38
dmakogon_hub_cap: true18:38
hub_capinstead of putting a bunch of commands in the configuration file for taskmgr18:38
dmakogon_hub_cap +118:38
dmakogon_so, what about custom userdata ?18:38
yogeshi am slightly confused... :-)18:38
yogeshlet me recollect myself...18:38
hub_capdmakogon_ custom userdata = edit the template in your installation18:38
SnowDusthub_cap : understood .. but that means .. an admin has to fix the code by navigating to redis.heat.template ?18:39
hub_capno need for "custom" userdata18:39
isviridovhub_cap, true18:39
dmakogon_ok18:39
*** jasonb365 has quit IRC18:39
SnowDustif he needs .. some more params .. on the same template ?18:39
hub_caplets deal with that when someone needs it SnowDust18:39
hub_capthat will complicate things cuz ther is no way to inject parameters18:39
hub_capin the first version of what we are doing18:39
isviridovhub_cap, true18:40
SnowDustok .. makes sense !18:40
hub_capi do see that we could need it in the future tho SnowDust18:40
yogeshso, UserData stays in the template and if there is a modification required the template gets updated by the admin...18:40
hub_capcorrect yogesh18:40
dmakogon_true18:40
dmakogon_yogesh +118:40
yogeshgreat...18:40
yogeshall the instances in the instancegroup :-) get created of the flavor provided in the instance creation api...18:41
dmakogon_yogesh: any estimates for this task?18:41
dmakogon_yogesh: yes18:41
yogeshi think we have taken the complications away18:42
dmakogon_agreed18:42
yogeshin the wake of clustering api...18:42
hub_capyes we have some people working on that today18:42
yogeshso it lies in SIMPLE category now18:42
yogeshthe task breakup is:18:43
SnowDustservice_type = string = many possibilities .. GR8 !18:43
yogesh1. finalize the directory structure18:43
dmakogon_hub_cap: https://review.openstack.org/#/c/48305/18:43
dmakogon_2. template loader18:44
yogesh2. load the template18:45
dmakogon_i'm faster, yogesh ;)18:45
yogesh3. define what configurations...18:45
yogeshyou absolutely are... :-)18:45
*** krow has quit IRC18:45
hub_capdmakogon_ there is outstanding comments on https://review.openstack.org/#/c/48305/5/trove/taskmanager/models.py18:45
dmakogon_what do you mean about #3 ?18:45
hub_capare you going to fix?18:45
dmakogon_i think a suppose18:46
dmakogon_yes18:46
yogeshdmakogon: we may have to think more on what gets picked from conf...18:46
yogeshwill share more..18:47
dmakogon_we need only template dir18:47
SnowDusti guess the config = template directory18:47
dmakogon_nothing else18:47
dmakogon_SnowDust +18:47
SnowDustyeah .. thats it !18:47
yogeshdefining in conf, what is going to be the template directiry18:47
dmakogon_yogesh: what do you mean ?18:48
yogeshwhat if template needs to be laoded from an external package..18:48
dmakogon_really template dir is enough18:48
SnowDust/etc/trove/heat_templates/  should be the base dir18:48
dmakogon_that mean default = /etc/trove/heat_templates/18:49
SnowDust/etc/trove/ already takes trove.conf / trove-taskmanager.conf ..18:49
hub_capSnowDust: not entirely true18:49
SnowDustthen .. we will append service  type to it18:49
dmakogon_taskmanager conf should store teml.dir18:49
hub_capif u want it to work with the app, u need it to be pybasedir/etc/trove/templates18:49
yogeshit will be driven teh same way as service_registry18:49
dmakogon_yogesh why ?18:50
yogeshjust a sec18:50
yogeshfor an external service, the templates will not be in trove, right?18:51
dmakogon_we have dir, than we os.exists(CONF.template_dir + '%(service_type).heat.template' % {'service_type': self.service_type})18:51
dmakogon_yogesh: we have base dir18:52
dmakogon_yogesh: templates should be there18:52
yogesheven for a service which is external?18:52
dmakogon_and what do you mean 'external service' ?18:52
yogeshthey should be loaded from an external source, right?18:52
* isviridov went to away18:52
SnowDustinside the basedir ... we are creating a subfolder for each service type18:52
SnowDustno ..distinction of service ttype .. internal / external18:53
hub_capyogesh: u can define where the templates live18:53
yogeshyup18:53
yogeshthat was the point18:53
yogeshso, tehre could be some conf settings like this , per point 318:53
dmakogon_we should keep templates in one dir, without anyh subfolders18:53
yogeshwhy?18:53
dmakogon_why should we ?18:54
SnowDustdmakogon .. why will u do this ?18:54
dmakogon_why do we need to make comlicated tree18:54
yogeshwell...how would the templates be placed in the base dir for an external service, manually?18:54
yogeshcoz they won't be a part of repo...the service being external18:55
*** tanisdl has quit IRC18:55
SnowDustanyone can untar their folders .. with their template package  .. if we give a service type directory ...18:55
dmakogon_yogesh: https://github.com/openstack/heat-templates/tree/master/cfn/F1718:55
dmakogon_i would like to store them like that18:55
yogeshthese are heat samples...18:55
yogeshwe are talking about template implementations in trov..18:56
SnowDustyeah .. dmakogon ..  thats also a tree :)18:56
dmakogon_i'm showing you order18:56
yogeshfor the defined service types18:56
dmakogon_simle order18:56
*** dmakogon__ has joined #openstack-trove18:57
dmakogon__lol18:57
dmakogon__reconnect18:57
SnowDustdmakogon__ is long tail now !18:57
dmakogon__yep18:57
yogeshits from a device with an antenna18:57
yogesh;-)18:57
SnowDustin celebration ..to animal care day ..18:57
SnowDustfeelings for  a pet cat .. :D18:57
dmakogon__so, i'm insisting  at storing templates in one di18:58
dmakogon__r18:58
dmakogon__one repo18:58
yogeshdmakogon__: which repo?18:58
SnowDustone base dir ! :)18:58
dmakogon__yes18:58
yogeshwould theey be in openstack/heat-templates?18:58
yogeshor within trove?18:59
dmakogon__trove/etc/trove/heat-templates18:59
SnowDustjust keep that base dir +  service type  +  template_file   path logic ..18:59
SnowDustfrom where ever u are bringing it :)18:59
yogeshwhy should we have templates of a service in trove, which is not a part of the trove...18:59
dmakogon__if you want external repo please do support of git/svn/ppa/rhel repo19:00
yogeshas yet...i mean19:00
*** dmakogon_ has quit IRC19:00
yogeshwhen it becomes a part then its templates move into the base dir...and the config path changes accordingly...19:00
dmakogon__yogesh: have ypu heard about symlinks ?19:00
yogeshyes19:00
yogesh:-)19:00
*** jodom has joined #openstack-trove19:01
dmakogon__if external dir - make simlink of each template19:01
dmakogon__only one base dir is acceptable19:01
SnowDustdmakogon__  :  just keep that base dir +  service type  +  template_file   path logic ..19:01
yogeshyou know what, i was even thinking of making the template laoder specific to the service...19:01
SnowDustbring it from anywhere .. :)19:02
yogesh:-)19:02
dmakogon__yogesh: yes19:02
SnowDustthat can be a good option !19:02
yogeshlike our guest agent can be from anywhere and works with trove, completely conf driven now, with your gust..19:02
SnowDustyogesh: +119:02
dmakogon__here you go, the path to template CONF.template_dir + '%(service_type).heat.template' % {'service_type': self.service_type}19:02
dmakogon__guestagent works with trove via rpc19:03
SnowDustCONF.template_dir + '%(service_type)' % {'service_type': self.service_type} why not ?19:03
dmakogon__SnowDust %(name)s19:04
SnowDustooops !19:04
SnowDustcopy paste .. as they are !19:04
yogeshexample...19:04
yogeshplz19:04
dmakogon__each template has next name structure:              service_type.heat.template19:05
dmakogon__mysql.heat.template19:05
*** jrodom has quit IRC19:05
yogeshin case of multiple templates, eg. nesting...19:05
SnowDustit wont trouble .. in nesting ..19:05
yogeshnamingwise?19:06
dmakogon__yes19:06
dmakogon__just time correct path to nested one19:06
dmakogon__i see no problem in it19:06
dmakogon__it is really easy to manipulate with only one dir19:06
dmakogon__as it done with each trove conf19:07
yogeshmysql.heat.template, mysql.constituent1.template, mysql.constituent2.template...19:07
dmakogon__wowwowow19:07
dmakogon__no19:07
yogesh:-)19:07
dmakogon__1 template per service type19:07
dmakogon__in single instance19:07
dmakogon__i thought you got it19:08
openstackgerritA change was merged to openstack/trove: Correct the fake implementation of UsageVerifier  https://review.openstack.org/4954619:08
yogeshsnowdust: can two templates be defined in a single yaml file?19:08
SnowDustnope ..19:08
yogeshanyways, i think it is leading to somewhere nowhere... :-)19:08
SnowDusttwo templates can work .. together :)19:08
openstackgerritA change was merged to openstack/trove: Adding location attribute to Fake Backup object.  https://review.openstack.org/4960119:08
yogeshso, we for nesting, we will need to have more than one template..19:09
dmakogon__why can't you write template without nesting ?19:10
SnowDustwe can define .. base.yaml( as convention) as entry point .. and other templates to be called from it19:10
dmakogon__so, we are ok with it ?19:10
yogeshthis is imposing restrictions on the template crestor..19:10
dmakogon__SnowDust: to comlicated19:10
SnowDustthis is making decipline .. :)19:10
yogeshenforcing discipline... :-) no never...19:11
SnowDustdiscipline*19:11
SnowDustok .. !19:11
yogeshbasically, what doesn't fit well for me is, having templates in trove for the services which don't exist in trove by default..19:12
SnowDustwho wants that ?19:12
openstackgerritA change was merged to openstack/python-troveclient: Remove trove as default value for Service Name  https://review.openstack.org/4765319:13
yogeshit thoguht that was the point of debate..19:13
SnowDustbasedir + service_type + template_file (base.yaml)19:13
yogeshwhich leads to a template /etc/<service_not_in_trove>template_file19:14
hub_capyogesh: time out19:14
yogeshok19:14
hub_capif you change the directory in the config19:14
hub_capu can put them anywhere19:14
hub_capand u can add services that dont exist in trove19:14
yogeshcool...19:14
yogeshyup19:14
hub_capdone and done19:14
SnowDustyep ..  basedir is also configurable :)19:15
yogeshall configurable..19:15
yogeshawesome..19:15
yogeshwe are good19:15
hub_capyup SnowDust i think just the config_dir in general should be configurable19:15
dmakogon__how much paramters do we suppose to have in conf file ?19:15
hub_capand the default will be _in_ the application19:15
yogeshyou mean, heat_teplate_parameters or generally...19:16
dmakogon__heat templates19:16
*** paul_lodronio has quit IRC19:16
SnowDustsummerize ?19:16
SnowDustsleepy ...19:17
dmakogon__yogesh: how much parameters are you going to bring ?19:17
yogeshvery minimal..19:17
dmakogon__how much ?19:17
yogeshthe directory settings in conf19:17
yogeshand the number of nodes...19:18
dmakogon__?19:18
yogeshthats pretty much it...19:18
dmakogon__number of nodes ?19:18
*** paul_lodronio has joined #openstack-trove19:18
hub_cap!!!im free for reviewing code!!!19:18
openstackhub_cap: Error: "!!im" is not a valid command.19:18
hub_capanyone need anythign reviewed19:18
hub_capI HATE U openstack19:18
hub_cap^ ^ the bot, not the foundation LOL19:18
yogeshwell....that can also be skipped i guess...19:19
yogeshcan be configured within template...19:19
SnowDusthub_cap ..expect a checkin around pluggable extention19:19
yogeshdirectory setting only..19:19
yogeshsnowdust: can u share the bp around it..19:20
dmakogon__yogesh: what do you mean under number of nodes ??19:20
yogeshi was thinking if the template contains more than one nodes...then can it brought from conf..19:20
yogeshbut thats a template property, depends ont he person who is defining a template..19:21
hub_capSnowDust: sweet19:21
yogeshso only one setting for the directoty19:21
yogeshthats it19:21
dmakogon__finally19:21
dmakogon__yogesh: <319:21
dmakogon__<3 <3 <319:21
yogeshme too... :-)19:21
yogeshso first level done...19:22
yogeshi am onto it19:22
SnowDusthehe19:22
dmakogon__waiting for review19:22
yogeshcool19:23
yogeshthanks much guys...19:23
dmakogon__thanks to you 219:24
openstackgerritDmitriy Ukhlov proposed a change to openstack/trove: Task manager refactoring done  https://review.openstack.org/4523819:24
dmakogon__good conversation19:24
yogeshyup...relaxing conversation...lol19:24
dmakogon__to relaxed, my friend19:25
yogeshcheckin out...19:25
yogeshthanks much guys...19:25
yogeshappreciate the patience... :-)19:26
dmakogon__we are all interested in making trove better19:29
SnowDustits good .. btw !19:30
dmakogon__btw, who wants review with tone of lines ?19:30
SnowDustmany things are snap !19:30
dmakogon__SnowDust: what about fixing PEP8 rule with alphabetic import ?19:30
dmakogon__it is about 300-400 lines of code19:31
SnowDustdidnt understand ..19:34
SnowDusti will autopep :D !19:34
openstackgerritA change was merged to openstack/trove-integration: Importing poll_until method from correct place  https://review.openstack.org/4800819:35
openstackgerritA change was merged to openstack/trove: Remove Duplicate trove_auth_url Property  https://review.openstack.org/4960919:40
*** SnowDust has quit IRC19:54
*** VinodGupta has quit IRC20:02
*** jodom has quit IRC20:05
*** jrodom has joined #openstack-trove20:05
*** shivamshukla_ has quit IRC20:06
*** arborism has joined #openstack-trove20:21
*** jodom has joined #openstack-trove20:23
*** jrodom has quit IRC20:25
*** tvb has quit IRC20:29
*** jodom has quit IRC20:30
*** tanisdl has joined #openstack-trove20:33
*** yidclare has quit IRC20:34
*** paul_lodronio has left #openstack-trove20:38
*** vipul is now known as vipul-away20:51
*** vipul-away is now known as vipul20:52
*** esp_ has joined #openstack-trove20:57
*** esp_ has quit IRC20:59
*** tvb has joined #openstack-trove21:01
*** yidclare has joined #openstack-trove21:03
*** krow has joined #openstack-trove21:06
*** tanisdl has quit IRC21:07
*** tanisdl has joined #openstack-trove21:08
*** tanisdl has quit IRC21:10
*** tanisdl has joined #openstack-trove21:11
*** krow has quit IRC21:11
*** pdmars has quit IRC21:13
*** rnirmal has quit IRC21:16
openstackgerritA change was merged to openstack/trove: Fix the fake nova server implementation  https://review.openstack.org/4975921:29
*** dmakogon__ has quit IRC21:29
*** djohnstone has quit IRC21:32
*** ashestakov_ has quit IRC21:41
*** krow has joined #openstack-trove21:41
*** Barker has quit IRC21:43
*** yogesh has quit IRC21:43
*** yogesh has joined #openstack-trove21:44
*** jcru has quit IRC21:46
*** isviridov has quit IRC21:46
*** yogesh has quit IRC21:48
openstackgerritSteve Leon proposed a change to openstack/trove: Fix quota issue where usages can drop to negative value  https://review.openstack.org/4930121:54
*** krow has quit IRC21:54
*** metral has quit IRC21:58
*** robertmyers has quit IRC22:02
openstackgerritSteve Leon proposed a change to openstack/trove: Fix quota issue where usages can drop to negative value  https://review.openstack.org/4930122:04
*** krow has joined #openstack-trove22:25
*** krow has quit IRC22:34
*** jmontemayor has quit IRC23:07
*** krow has joined #openstack-trove23:12
openstackgerritDmitriy Ukhlov proposed a change to openstack/trove: Task manager refactoring done  https://review.openstack.org/4523823:18
*** tanisdl has quit IRC23:20
*** datsun180b has quit IRC23:22
*** dukhlov_ has joined #openstack-trove23:26
*** dukhlov_ has quit IRC23:28
*** krow has quit IRC23:32
*** krow has joined #openstack-trove23:43
openstackgerritJustin Hopper proposed a change to openstack/trove: Allow service_id per service_type for Usage Events  https://review.openstack.org/4530923:46
*** adrian_otto has quit IRC23:47
openstackgerritA change was merged to openstack/trove: Allow early host % on validate  https://review.openstack.org/4754323:51
juicehub_cap: vipul: this was approved and needed to be rebased and merged. https://review.openstack.org/#/c/45309/7  Please reapprove23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!