Thursday, 2014-11-13

*** Viswanath has joined #magnetodb00:17
*** Viswanath has quit IRC00:20
*** charlesw has joined #magnetodb01:23
*** jeromatron has joined #magnetodb02:58
*** jeromatron has quit IRC02:59
*** jeromatron has joined #magnetodb03:04
*** jeromatron has quit IRC03:12
*** jeromatr_ has joined #magnetodb03:12
*** jeromatr_ has quit IRC03:17
*** igormarnat_ has joined #magnetodb04:11
*** igormarnat has quit IRC04:12
*** igormarnat_ is now known as igormarnat04:12
*** rushiagr_away is now known as rushiagr04:27
*** vivekd has joined #magnetodb04:39
*** jeromatron has joined #magnetodb04:43
*** charlesw has quit IRC04:48
openstackgerritMerged stackforge/magnetodb: Include correct URL in JSON response for "list tables" call
*** jeromatron has quit IRC04:59
*** jeromatron has joined #magnetodb05:01
*** jeromatron has quit IRC05:13
*** jeromatron has joined #magnetodb05:14
*** vivekd has quit IRC05:36
*** jeromatr_ has joined #magnetodb05:38
*** jeromatron has quit IRC05:38
*** jeromatron has joined #magnetodb05:41
*** jeromatr_ has quit IRC05:41
*** vivekd has joined #magnetodb05:45
*** jeromatron has quit IRC05:52
*** jeromatron has joined #magnetodb05:52
*** ajayaa has joined #magnetodb05:58
*** vivekd has quit IRC06:05
*** jeromatron has quit IRC06:08
*** k4n0 has joined #magnetodb06:09
*** vivekd has joined #magnetodb06:09
*** romainh has joined #magnetodb08:09
*** ajayaa has quit IRC08:36
*** ajayaa has joined #magnetodb09:31
*** vivekd has quit IRC12:23
*** ajayaa has quit IRC12:35
*** ajayaa has joined #magnetodb12:48
ajayaaHi guys!12:59
ajayaaAre we having a meeting today?12:59
ajayaadukhlov  ikhudoshyn isviridov ^^13:00
ajayaaHI rushiagr.13:01
rushiagrDST strikes again? :)13:01
ikhudoshynhi guys13:04
ikhudoshynyes we are13:04
ikhudoshynisviridov will join us in a couple minutes13:05
isviridov_awayajayaa : rushiagr ikhudoshyn hello everybody13:10
*** isviridov_away is now known as isviridov13:10
isviridovDo you have time shift in your time zone?13:11
isviridovI expected that we have meeting in 1 hour and our coleagues from Symantec as well, I believe13:11
rushiagrnot in India13:12
isviridovI see. This week we have equal time difference with US, so I would keep it at 4 PM EET and 9 AM EST.13:16
isviridovI've updated wiki actually
isviridovrushiagr : ajayaa sorry for inconvenience13:17
ajayaaisviridov, np13:17
isviridovrushiagr : ajayaa are you able to join us in 43 mins?13:17
rushiagrisviridov: no probls13:18
rushiagrisviridov: I am doubtful. Can't say for sure13:18
isviridovrushiagr : anyhow you are very welcome13:18
rushiagrisviridov: thanks :)13:18
*** miqui has joined #magnetodb13:31
*** achuprin_ has joined #magnetodb13:51
*** rushiagr is now known as rushiagr_away13:51
*** charlesw has joined #magnetodb13:53
*** nunosantos has joined #magnetodb13:54
*** dukhlov_ has joined #magnetodb13:55
isviridovHello everybody14:00
isviridovIt is meeting time14:00
charleswHi guys14:01
isviridovikhudoshyn : dukhlov nunosantos ajayaa charlesw rushiagr_away hello14:01
isviridov#startmeeting isviridov14:01
openstackMeeting started Thu Nov 13 14:01:31 2014 UTC and is due to finish in 60 minutes.  The chair is isviridov. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:01
openstackThe meeting name has been set to 'isviridov'14:01
ajayaaHi all!14:01
isviridovromainh: ?14:02
romainhisviridov: yep14:03
isviridovOk, let us start14:03
isviridov#topic action items14:03
isviridov#1 dukhlov data encryption support blueprint14:03
dukhlov_a'm working on it now and plan to finish spec this week14:04
dukhlov_also draft for managment API is under review now14:05
dukhlov_it is dependency for encryption bp14:05
isviridovI see #link
dukhlov_but it has -1 from jenkisn14:05
ikhudoshynguys, sry, coult u remind me address of our meeting page14:05
dukhlov_sorry I will fix it14:06
isviridovikhudoshyn : here it is
ikhudoshynisviridov, tnx14:06
isviridovOk, dukhlov leaving AI on you14:06
isviridov#action dukhlov data encryption support blueprint14:06
isviridov#2 ikhudoshyn file a bug about dynamodb version support documentation14:07
*** keith_newstadt has joined #magnetodb14:07
isviridovikhudoshyn : any success?14:07
ikhudoshynmy bad, forgot bout it14:07
ajayaa#help what is the idea behind storage before create_table?14:07
isviridovkeith_newstadt : GM14:07
isviridovajayaa : what do you mean?\14:08
keith_newstadtmorning all14:08
isviridovikhudoshyn : leaving on you as well14:08
isviridov#action ikhudoshyn file a bug about dynamodb version support documentation14:09
ajayaaThe high level meaning of storage!14:09
ajayaaIs storage a high level container for tables?14:10
*** k4n0 has quit IRC14:10
ajayaaDoes it deal with amount of space being used by tables?14:10
isviridovajayaa : dukhlov seems there we need a bit more context14:10
dukhlov_storage is like keyspace for cassandra14:10
dukhlov_it seems ajayaa is talking about managment API bp14:11
ajayaaokay. But we have a high level container called tenant for tables.14:11
openstackgerritIlya Sviridov proposed stackforge/magnetodb-specs: Add specification for part of Management API
ajayaaI am looking at
dukhlov_yes, for no storage=tenant14:12
dukhlov_but tenant is openstack-wide container14:12
ikhudoshyndukhlov_, any plans for them to diverge?14:12
dukhlov_not for now14:12
ajayaaIf we add storage entity, then different users could have same table name in the same tenant.14:13
ajayaaAm I right?14:13
dukhlov_but I want to consider this usecase for future14:13
dukhlov_and make implementation with possibility to ve extended14:13
ajayaaCan you please tell me another usecase?14:14
ikhudoshynhm.. in our current API we have 1 tenant =  1 keyspace14:14
ajayaaikhudoshyn, +114:14
ikhudoshynbut for mgmt you want it different, why?14:14
keith_newstadtso far, the use case is simply that we want to provide encrypted tenants14:14
dukhlov_when 1 tenant can have a few storages14:14
ikhudoshynlooks like YAGNI for me14:14
dukhlov_1 more nested level14:14
keith_newstadti don't think we have a use case for different encryption settings for tables within a tenant14:15
keith_newstadti'd be concerned about adding unnecessary complexity...14:15
ajayaakeith_newstadt +114:15
dukhlov_but also storage can have another settings14:15
dukhlov_lilke consystency level for example14:16
ikhudoshynat least i'd like to see data API and mgmt API consistent to each other14:16
dukhlov_I don't sugget add comlexity now14:16
ikhudoshyndukhlov_, how could this affect existing data API?14:16
dukhlov_I only sugget to leave posibility to improve it in fute without changins ofxisting feature14:17
dukhlov_ikhudoshyn: for now - it don't affect API14:18
keith_newstadti'd think consistency would be an attribute of a table, rather than a tenant or storage14:18
dukhlov_but it is not possible for Cassandra14:18
keith_newstadtstorage settings would be specifically for... storage attributes. rather than for API related attributes.14:18
dukhlov_it is attribute of keyspace14:18
keith_newstadtwe can't set quorum settings (required number of reads, e.g.) on a particular table or read/write operation?14:19
dukhlov_we can14:20
charleswyes each query can set its tunable CL14:20
dukhlov_but we can't set how many replicas we have14:20
dukhlov_per table14:20
keith_newstadti think number of replicas would be fine as a per tenant setting14:21
keith_newstadtmakes setting quotas and showback easier as well14:21
charleswCan you change the config after initial setting?14:22
dukhlov_yes agree, drafted bp use only one storage for tenant14:22
dukhlov_only remove all storage and then initialize it again14:23
keith_newstadtshould probably reject requests to POST to an existing storage14:24
keith_newstadtrequire explicit DELETE before a rePOST14:24
isviridovdukhlov keith_newstadt ajayaa we are at the very beginning of BP discussion, so feel free to comment the spec14:26
* isviridov just remind14:26
dukhlov_yes it would be great14:26
keith_newstadtwill do14:27
isviridovajayaa : next topic?14:27
ikhudoshynlet's move on14:28
isviridov#3 isviridov ikhudoshyn clarify roadmap item14:28
ikhudoshynwe kinda did14:28
isviridovSo, before summit there was an item about DymanoDB support here
isviridovikhudoshyn : yeap, just sharing with team14:29
isviridovAs you know Amazon has released the new version of API with GlobalIndex support, map data type support and expression in querie...14:30
isviridovSo, we have defined the scope as AWS DynamoDB API 2011-12-05 version.14:31
isviridovThe documenntation is still available for downloading in PDF format14:31
isviridovajayaa : rushiagr_away any thoughts?14:31
isviridovOk, just let us move on.14:33
isviridovajayaa : rushiagr_away would be nice to hear from you later14:33
isviridov#topic  Open discussion14:33
ikhudoshyni'm working on backup/restore 4 mdb14:34
ikhudoshyn -- it's a draft for API14:34
ikhudoshynpls review and share yr thoughts14:34
keith_newstadtwe had an interesting discussion with the trove folks at the conference14:35
ikhudoshynpls note, the abocve doc is API spec only, it's not about impl14:35
keith_newstadtit occurred to us all that there will be some overlap in what we are designing14:35
ikhudoshynabout to delegate lo level maintenance to trove?14:35
keith_newstadtmagnetodb is a service on the data path, where trove is a db provisioning api14:35
keith_newstadtthe backup api's overlap between the two14:36
keith_newstadtwondering if this is an opportunity for us to collaborate - at least to have consistent apis, possibly to use trove's cassandra support for backup/restore under magnetodb14:36
keith_newstadtit would require some additional functionality from the trove folks - e.g. per tenant/keyspace backup14:37
keith_newstadtikhudoshyn: thoughts on that?14:37
ikhudoshynwell, trove was the 1st thing I looked at14:37
ikhudoshynwhen working on API14:37
ikhudoshyntried to make it alike but not follow it exactly14:38
keith_newstadtdo you think there are reasons to diverge from the trove api?14:38
keith_newstadtor could we consolidate on a single interface?14:38
ikhudoshynas for collaborating, 1st thing is, when will they be ready for prod use?14:38
keith_newstadtit's a good question.  i don't think they have been thinking about per keyspace backup until now. so i wouldn't expect them to be moving as quickly as we are in this direction14:39
keith_newstadtbut we could start working with them on the api design, with plans to either move the functinoality into trove, or for us to develop it directly into trove14:40
ikhudoshynI think I should recheck their API once again just to see if there are real blokers for us to have a single api14:41
isviridovWhat about import/export fucntionality?14:41
ikhudoshynthat was my #214:41
ajayaaHi guys. Sorry had a meeting.14:41
keith_newstadtwhat are the difference between the use cases for import/export and backup/restore?14:41
keith_newstadti'm picturing import/export to be more magnetodb specific14:42
ikhudoshynthe API I worked on supports backup in DB agnistic format14:42
isviridovIt looks for me that we can use the same API but with somethink like a type ir strategy: backend_database_native or magentodb14:42
ikhudoshynso user coudl have all his data in json format and could download it14:42
charleswbackup/restore should be operational/system, import/export is user data, logical level14:43
ikhudoshyni proposed 'strategy' param for 'create backup' call14:43
keith_newstadtgenerally i would use import/export when i want to get the data in a generalized format so that i could bring it into another application or db14:43
keith_newstadtcharlesw: +114:43
keith_newstadtso import/export would be magnetodb specific, where backup/restore could be via trove14:44
isviridovcharlesw : do you think we have to keep two APIs for that? export/import and backup/restore?14:44
isviridovkeith_newstadt: ?14:44
ajayaaWhen we use trove for backup, do we have to provision cassandra through trove only?14:44
charleswyes, we should have admin API14:45
keith_newstadti don't think so14:45
ikhudoshynajayaa, seems like yes14:45
keith_newstadthm.  why?14:45
ikhudoshynfrom what i remember, trove stores meta info about its deployments14:46
ajayaaBut all deployers might not want to go with trove reasons being cassandra performance on vms with shared disk.14:46
ikhudoshynas well as we do for our tables14:46
ikhudoshynso for trove to be able to backup our C* we should pass all meta they need14:46
ikhudoshynin short, this would require a deeper integration, than just calling trove api14:47
isviridovTrove relies on agent working on each Cassandra node, so you can backup only provisioned by Trove database14:47
keith_newstadtif we were to implement backup/restore without trove, would we need an agent on each node as well?14:48
ikhudoshynisviridov, +1, this is for the case of DB-aware backups14:48
ajayaaI feel that we could have two implementation on with trove and one without trove.14:48
keith_newstadtseems wasteful to implement it twice...14:48
ikhudoshynthe 'pro' of db agnostic backup -- we don't need any additional code on DB nodes14:48
ikhudoshynajayaa, keith_newstadt, in fact this could be chosen via strategy14:49
keith_newstadtthe 'con' would be performance and backup size14:49
ajayaakeith_newstadt, In trove case we wouldn't have to do much, because trove will take care of it.14:49
ikhudoshynkeith_newstadt, sure14:49
keith_newstadtwe would need an agent on the cassandra nodes as well to do backup/restore, right?14:50
keith_newstadt(for the more performant version)14:50
ikhudoshynanyway, in order to use Trove's backup we shoul deploy our C* via trove14:50
keith_newstadthow would we provision that agent, if we assume we weren't using trove?14:50
ikhudoshynthe same way we provision C*14:51
ajayaaikhudoshyn +114:51
keith_newstadtso we would provide instructions to the operator on how to deploy cassandra, and then our agent on top14:52
keith_newstadtand they would use heat or puppet or whatever to implement it.14:52
keith_newstadtis that right?14:52
isviridovkeith_newstadt ikhudoshyn I would suggest defining own backup/restore API and implement it to operate JSONs only. Also leave a stub for future DB-aware implementation. Just we have it in Trove we can employ it and ahve another backup/restore mechanis14:52
keith_newstadtcouldn't we still use the same agent as trove, or a subset of it14:52
keith_newstadtand then it is a question of whether trove deploys that agent, or if the operator does through heat/puppet/etc14:53
openstackgerritMerged stackforge/magnetodb: Adds Cassandra implemetation of monitoring API
isviridovkeith_newstadt : they deploys an agent via cloud-init and prebuild images14:53
ikhudoshynonce again trove-guestagent is tightly relies on the whole trove codebase as well as on trove's db with meta. its mysql for now14:53
charleswfyi, Netflix Priam has jar files inside C*, runs along with Cassandra. Could be another option14:54
keith_newstadtwe should consider meeting with the trove folks to discuss14:54
ajayaakeith_newstadt, +114:55
keith_newstadtas for db-agnostic backup/restore, we should talk about the functionality. sounds more like import/export, which is slightly different functionality-wise14:55
isviridovI believe that Trove weekly meeting is the best option14:55
keith_newstadte.g. import is additive rather than overwrite, doesn't support incrementals, etc14:55
*** denis_makogon has joined #magnetodb14:56
keith_newstadtk.  i also know the tesora folks, and could reach out to them if that would be useful14:56
isviridovdenis_makogon : hi14:56
isviridovkeith_newstadt : export/import sounds not quite right, but the reason was not divede it from backup/restore.14:57
denis_makogonisviridov, hi14:57
ajayaadenis_makogon, In trove do we have an option of deploying cassandra nodes on bare metal?14:58
ajayaaIn other words does it use ironic?14:58
keith_newstadtisviridov: ok. we can discuss the details in the context of the blueprint14:58
denis_makogonajayaa, no14:58
denis_makogonajayaa,  it's up to nova14:58
isviridovkeith_newstadt : but with defining type or strategy for backup, we can remove it14:58
denis_makogonajayaa, fyi bare metal is not main steam any more - docker/lxc rules the world =)14:59
isviridov* deprecate14:59
keith_newstadtdenis_makogon: also not yet supported :)14:59
denis_makogonkeith_newstadt, by whom ?14:59
ajayaadenis_makogon, that is for the cool kids. :)15:00
keith_newstadttrove.  or am i wrong?15:00
ajayaaNot prod-ready I believe.15:00
denis_makogonkeith_newstadt, trove would speak only to nova, but nova can run over tons of drivers including lxc/docker/ironic15:00
keith_newstadtprod ready?15:00
*** jeromatron has joined #magnetodb15:00
ajayaakeith_newstadr, Would you really run cassandra or any other prod application on docker, as it stands today?15:01
ajayaakeith_newstadt ^^15:01
* isviridov meeting time is over. But let us finish discussion if you don't mind15:02
ikhudoshynajayaa, +1 )15:02
ajayaabtw, is up for review. :)15:03
isviridovOk, let us talk to Trove guys15:03
keith_newstadtand i think we should hammer out our use cases for backup/restore15:03
isviridov#idea we should hammer out our use cases for backup/restore15:04
isviridov#idea deprecate export/import API and use backup/restore instead15:04
keith_newstadtgoals for talking to trove would be (1) can we avoid having two different apis for db backup/restore inside of openstack and (2) can we avoid having two different implementations of that api15:04
keith_newstadtisviridov: +115:05
isviridov#action keith_newstadt isviridov ikhudoshyn clarify if we can avoid having two different apis for db backup/restore inside of openstack15:06
ikhudoshyndenis_makogon, btw, does Trove have any publised spec for backup/restore API?15:06
denis_makogonikhudoshyn, sure it has15:06
isviridov#action keith_newstadt isviridov ikhudoshyn clarify if we can avoid having two different implementations of that api15:07
ikhudoshyncould you point me?15:07
isviridovSorry guys, return back to you later15:07
openstackMeeting ended Thu Nov 13 15:07:33 2014 UTC.  Information about MeetBot at . (v 0.1.4)15:07
openstackMinutes (text):
*** isviridov is now known as isviridov_away15:07
ikhudoshynkeith_newstadt, a quick look at the trove's spec shows it is per instance15:09
ikhudoshynwhich is really inappropriate for C*15:09
keith_newstadtya. we also need it to be per tenant.15:10
keith_newstadtso definitely need some discussion around it15:10
keith_newstadtit's possible that it won't be a good fit, but i think the your point about per node backup will affect the trove guys as much as us. need a solution15:11
ikhudoshynok, we should definitely visit trovians15:11
denis_makogonikhudoshyn, it's a gap for Trove which is pretty valid in general, i've spoke with couple trove cores and they are ok with such thing, but we need to have spec for it15:11
ikhudoshyndenis_makogon, could you also pls remind, when your weekly meeting usually happens?15:12
denis_makogonyesterday =)15:12
ikhudoshyndenis_makogon, great))15:12
ikhudoshynwas it the really last?15:12
*** jeromatron has quit IRC15:12
*** jeromatron has joined #magnetodb15:13
ikhudoshyni mean just point us to the schedule pls))15:13
*** ajayaa has quit IRC15:13
*** jeromatr_ has joined #magnetodb15:17
*** jeromatron has quit IRC15:17
*** rushiagr_away is now known as rushiagr15:38
*** jeromatr_ has quit IRC15:46
*** jeromatron has joined #magnetodb15:46
*** jeromatr_ has joined #magnetodb16:20
*** jeromatron has quit IRC16:22
*** dukhlov_ has quit IRC16:27
*** keith_newstadt has quit IRC16:28
*** keith_newstadt has joined #magnetodb16:30
*** jeromatr_ has quit IRC16:32
*** jeromatron has joined #magnetodb16:32
*** jeromatron has quit IRC16:35
*** jeromatron has joined #magnetodb16:35
*** romainh has left #magnetodb16:41
*** ajayaa has joined #magnetodb16:42
*** jeromatron has quit IRC16:55
*** jeromatron has joined #magnetodb16:58
*** ajayaa has quit IRC17:02
*** rushiagr is now known as rushiagr_away17:07
*** miqui has quit IRC17:08
*** miqui has joined #magnetodb17:11
*** openstackgerrit has quit IRC17:34
*** openstackgerrit has joined #magnetodb17:34
*** vnaboychenko has joined #magnetodb17:39
openstackgerritNuno Santos proposed stackforge/magnetodb: Get correct username from HTTP header
*** openstackgerrit has quit IRC18:03
*** openstackgerrit has joined #magnetodb18:04
*** openstackgerrit has quit IRC18:49
*** openstackgerrit has joined #magnetodb18:49
*** openstackgerrit has quit IRC19:18
*** openstackgerrit has joined #magnetodb19:19
*** romainh has joined #magnetodb20:04
*** keith_newstadt has quit IRC20:04
*** openstackgerrit has quit IRC20:04
*** openstackgerrit has joined #magnetodb20:05
*** nunosantos has left #magnetodb20:08
*** nunosantos has joined #magnetodb20:23
*** charlesw has quit IRC21:06
*** openstackgerrit has quit IRC21:19
*** openstackgerrit has joined #magnetodb21:19
*** romainh has left #magnetodb21:42
*** nunosantos has quit IRC21:54
*** denis_makogon has quit IRC23:47
*** denis_makogon has joined #magnetodb23:47

Generated by 2.14.0 by Marius Gedminas - find it at!