14:00:52 <isviridov> #startmeeting magentodb
14:00:53 <openstack> Meeting started Thu Oct 30 14:00:52 2014 UTC and is due to finish in 60 minutes.  The chair is isviridov. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:54 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:57 <openstack> The meeting name has been set to 'magentodb'
14:00:58 <rushiagr> wow, entry at just about the right time :)
14:01:08 <isviridov> Feel free to wave :)
14:01:13 <dukhlov> o\/_\/
14:01:17 <nunosantos> o/
14:01:20 <isviridov> rushiagr : it is becayse of time shift
14:01:29 <charlesw> o/
14:01:32 <rushiagr> isviridov: okay
14:01:37 <rushiagr> o\
14:01:49 <isviridov> rushiagr : works only for today meeting
14:02:03 <isviridov> Let us start gentlmen
14:02:11 <isviridov> Helo jeromatron
14:02:49 <jeromatron> Hi
14:02:53 <isviridov> Today agenda https://wiki.openstack.org/wiki/MagnetoDB/WeeklyMeetingAgenda#Oct_30.2C_2014.2C_14:00_UTC
14:03:02 <isviridov> jeromatron : welcome to mdb meeting
14:03:20 <jeromatron> thanks :)
14:03:36 <ominakov> hello, guys
14:03:42 <isviridov> #topic action items
14:03:46 <isviridov> ominakov : hello
14:03:55 <isviridov> You know last time meetbot didn't work, not sure that I remember all of them
14:04:03 <ajayaa> Hi All.
14:04:09 <isviridov> ajayaa : hello
14:04:51 <isviridov> I could say that we have discussed celiomete integration with ajayaa
14:05:02 <isviridov> ajayaa : any success with spec?
14:05:31 <ajayaa> isviridov, I was trying to create an example sample.
14:05:49 <isviridov> #link https://review.openstack.org/#/c/126335/
14:06:18 <ajayaa> When I am done with that I will update the spec. One of the comment by reviewers asks about example sample.
14:07:03 <isviridov> ok, I want to have it lookng good for summit. Ping me if you can't finish today or tomorrow
14:07:13 <isviridov> I would back you up
14:07:40 <ajayaa> isviridov, okay. Thanks
14:08:10 <isviridov> dukhlov : any update with cassandra backend v2 blueprint?
14:09:02 <isviridov> BTW we have 3 specs approved for kilo aleady #link http://magnetodb-specs.readthedocs.org/en/latest/
14:09:17 <dukhlov> I've started to write spec. As you know I created CUSTOM secondary index implementation for Cassandra
14:09:33 <dukhlov> and there a lot of staff to be described
14:09:38 * isviridov thanks to ominakov idegtiarov achudnovets and others
14:10:45 <isviridov> dukhlov : any draft available?
14:11:27 <dukhlov> in progress
14:11:32 <isviridov> :)
14:11:42 <dukhlov> no draft for now
14:11:46 <isviridov> Any other action items to discuss?
14:12:19 <ikhudoshyn> let's go forward
14:12:24 * isviridov next topic is on the way
14:12:29 <isviridov> #topic Kilo roadmap https://etherpad.openstack.org/p/kilo-crossproject-summit-topics isviridov
14:12:37 <isviridov> Yeap, we have it
14:13:11 <ikhudoshyn> https://etherpad.openstack.org/p/magnetodb-kilo-roadmap
14:13:15 <isviridov> #link https://etherpad.openstack.org/p/magnetodb-kilo-roadmap
14:13:20 <isviridov> ikhudoshyn : thank you
14:13:57 <isviridov> There are comments
14:14:34 <isviridov> ikhudoshyn : export/import api is about data backup/restore but database agnostic
14:14:46 <ikhudoshyn> isviridov: tnx
14:15:12 <charlesw> is it per tenant/table?
14:15:36 <isviridov> charlesw : I see it as per table
14:16:32 <ikhudoshyn> isviridov: for me export/import sounds like a have an artifact that represents my data
14:17:08 <ikhudoshyn> but backup/restore sounds more like a fact that this artifact is being stored (more like a service)
14:17:16 <isviridov> Yeap, artifact stored somewhere in swift
14:17:42 <ikhudoshyn> so we do mean import/export to/from swift?
14:17:58 <charlesw> so the exported data should include all my data row, and my table schema, but not index?
14:18:32 <ikhudoshyn> charlesw: if it is DB agnistic than yes
14:18:43 <ikhudoshyn> s/than/then
14:19:29 <isviridov> ikhudoshyn : probably wording is not the best, just didn't want to use backup/restore as it sounds really connected to cassandra. We can say that it will be big jsons in swift
14:20:21 <isviridov> charlesw : data, schema the magentodb one, so index are build during import
14:20:32 <isviridov> * indexes
14:20:51 <ikhudoshyn> i didn't mean it is connected to C*, just wanted behavior like 'backup my data' -> 'your data has been backed up'
14:21:06 <ikhudoshyn> on some external storage
14:21:18 <isviridov> ikhudoshyn : yeap
14:21:31 <charlesw> sounds good. Is swift just an option, or the only way
14:23:03 <isviridov> charlesw : just an option, I believe small ammounts of data could be imported/exported from client directly
14:24:07 <charlesw> +1
14:25:18 <isviridov> charlesw : ikhudoshyn let us move on
14:25:25 <ikhudoshyn> isviridov: ok
14:25:38 <isviridov> dukhlov : data encryption support?
14:26:38 <dukhlov> hm
14:26:52 <dukhlov> actually it is only idea
14:28:07 <dukhlov> I think that it is interesting  feature to store data securely on disk
14:28:29 <dukhlov> I saw this feature in Datastax Enterprise
14:29:17 <dukhlov> and I believe that we need to think about how we can implement it
14:29:48 <isviridov> dukhlov : will you blueprint it? I can be good topic to discuss on summit
14:29:54 <isviridov> * it
14:30:01 <dukhlov> so I'm going to investigate alternatives and prepare plueprint specification soon
14:30:20 <dukhlov> *blueprint
14:30:21 <isviridov> !m dukhlov
14:30:21 <[o__o]> You're doing good work, dukhlov!
14:30:22 <openstack> isviridov: Error: "m" is not a valid command.
14:31:06 <isviridov> #action dukhlov data encryption support blueprint
14:31:24 <isviridov> ikhudoshyn: what do you mean     Catch up with the latest DynamoDB API?
14:32:02 <ikhudoshyn> as far as i know they updated their DynamoDB API
14:32:19 <isviridov> As I know they have added GlobalIndexes and regexps on conditions at least
14:32:42 <ikhudoshyn> since we claim we support Amazon DynamoDB as well we should consider supporting those new features as wel
14:33:15 <charlesw> hbase has the Transparent table/CF encryption implemented: https://issues.apache.org/jira/browse/HBASE-7544
14:33:19 <ikhudoshyn> at least we should review changes and figure out can we support them
14:33:34 <isviridov> charlesw : good to know
14:34:09 <isviridov> ikhudoshyn : yeap, we have to clear stand what version of dynamo api we do support
14:34:37 <isviridov> Will you file a bug for documentation of it?
14:34:40 <isviridov> ikhudoshyn: ?
14:34:52 <ikhudoshyn> цшдд вщ
14:34:55 <ikhudoshyn> will do
14:35:25 <isviridov> #action ikhudoshyn file a bug about dynamodb version support documentation
14:35:46 <romainh> about data encryption: what about client-to-node and node-to-node encryption?
14:35:54 <isviridov> ikhudoshyn : let us rephrase your statement in etherpad, not it sounds really huge
14:35:56 <charlesw> also this is relatively new to dynamo
14:35:58 <charlesw> ConditionalOperator
14:35:59 <charlesw> Important
14:36:01 <charlesw> There is a newer parameter available. Use ConditionExpression instead. Note that if you use ConditionalOperator and ConditionExpression at the same time, DynamoDB will return a ValidationException exception.
14:36:02 <charlesw> This parameter does not support lists or maps.
14:36:17 <charlesw> we don't support it right now
14:37:22 <isviridov> ikhudoshyn : what about like cover gap with dynamo db api 2011-12-05 version
14:37:44 <ikhudoshyn> isviridov: sounds good
14:37:47 <dukhlov> romainh: client-to-node means encrypt data on client side and sent to cassandra already encrypted?
14:38:01 <isviridov> charlesw : yeap, there should be clear line we don't have now
14:38:20 <dukhlov> romainh: what does node-to-node mean?
14:38:27 <isviridov> ikhudoshyn : done
14:38:31 <romainh> dukhlov: yes
14:39:30 <romainh> dukhlov: when data are sent from one node to another, data could be encrypted by C*
14:39:31 <dukhlov> romainh: client-to-node encryption is possible but in this case we will lose possibility to perform range query
14:40:10 <dukhlov> because ordering will be lost
14:40:48 <isviridov> romainh : the client-cassandra connection is done in secure part of network, I think we can rely on general security policies for infrastructure
14:40:49 <romainh> dukhlov: no, the communication is made over SSL
14:41:00 <romainh> isviridov: ok
14:41:46 <charlesw> encryption for data at rest, over the wire is doen by ssl?
14:41:53 <dukhlov> as far as I know C* has possibility to communicate with other nodes using ssl encryption but it only save us from traffic sniffing
14:42:04 <isviridov> mdb client ---- HTTPS --->{ mdb api ---- TCP--> Cassandra}
14:42:21 <dukhlov> but data will be stored not encrypted to sstables
14:42:23 <isviridov> {} - secure network not reachable for mdb user
14:42:42 <romainh> dukhlov: exactly
14:43:17 <romainh> it prevents from traffic sniffing
14:44:02 <dukhlov> so this blueprint is aimed to prevent getting data from disk
14:44:21 <romainh> so, your unique concern is about disk encryption, right?
14:44:23 <isviridov> ikhudoshyn : how do you see 'Provide operability of the existing DynamoDB API with new data types (lists, dicts)'?
14:44:26 <romainh> ok
14:44:28 <dukhlov> yes
14:45:46 <ikhudoshyn> isviridov: i don't see it yet actually. But one will definitely get in trouble if he tries to access an existind table with maps and lists via AWS API
14:46:29 <ikhudoshyn> so my point for now is, we should provide some workaround or data transformation for that
14:46:44 <isviridov> ikhudoshyn : I think we have to hide tabels created via DynamoDB API from those what were created via mdb API and vice versa
14:47:33 <isviridov> ikhudoshyn : exactly the same approach C* has with CQL tables and column families. What do you think?
14:48:00 <ikhudoshyn> isviridov: this could be done easily, but first I'd like to look for some other ways
14:48:51 <ikhudoshyn> or we'll end up with two separate products
14:49:52 <charlesw> +1 for keeping data separate
14:50:27 <isviridov> It is a big topic, let us rephrase it in etherpad somehow. Now it sounds like a big challenge
14:51:20 <isviridov> Let us move on and do it offline
14:51:28 <isviridov> ikhudoshyn : sounds ok?
14:51:29 <ikhudoshyn> isviridov: +1
14:51:48 <isviridov> #action isviridov ikhudoshyn clarify roadmap item
14:51:51 <isviridov> #topic Design session topics https://etherpad.openstack.org/p/magnetodb-kilo-design-summit isviridov
14:52:37 <isviridov> Here is a current work-in-progress design session schedule, feel free to suggest topics
14:52:47 <isviridov> This summit we have 90 mins
14:53:18 <isviridov> #topic Next meeting isviridov
14:53:44 <isviridov> #info the next week meeting is canceled due to summit
14:54:00 <isviridov> #Open discussion isviridov
14:54:26 <aostapenko> I suggest to extend healthcheck request so it will check rpc and additionally return name of host, that responses this request. So it can be used multiple times thru the load balancer and all magnetodb nodes can be checked by multiple healthcheck requests
14:54:27 <isviridov> aostapenko: ?
14:55:54 <isviridov> aostapenko : how can you manage what node to send to?
14:56:01 <charlesw> how do you make sure all MDB nodes will be pinged by load balancer?
14:56:12 <isviridov> charlesw : +1
14:56:40 <dukhlov> charlesw:+1
14:56:42 <isviridov> aostapenko : why do we need it?
14:56:56 <dukhlov> isviridov: +1
14:57:05 <isviridov> dukhlov : +1
14:57:23 <dukhlov> handshake
14:57:28 <aostapenko> charlesw: only 1 node will be pinged by load balancer. But we will send many requests, and each response will contain info about the node that responded
14:58:03 <isviridov> aostapenko : healthcheck request is not designed to be called out of balancer by user.
14:58:27 <isviridov> It should be called by LB and it will call each node
15:00:25 <aostapenko> isviridov: sounds reasonable, I think this diagnostics should be implemented another way
15:00:29 * isviridov it is time for next meeting
15:00:48 <isviridov> I believe we are done for today
15:00:53 <charlesw> LB should provide status of wheather an MDB node is up/down
15:00:55 <isviridov> Thank you everybody for coming
15:01:04 <isviridov> aostapenko : thank you for your ideas
15:01:11 <isviridov> See you after summit
15:01:11 <charlesw> thanks for organizing
15:01:15 <isviridov> #endmeeting