| *** openstack has joined #magnetodb | 13:49 | |
| jeromatron | dukhlov: with your email - the query with an index would require allow filtering without the complete key - just as a matter of syntax. so I believe it would do a scan within the partition at this point. however, I'm more interested in whether it would do the whole scatter gather thing. I would look at a trace for the query to see if it just hits the one node. | 14:14 |
|---|---|---|
| jeromatron | also, what performance are you seeing with your data model | 14:25 |
| dukhlov | jeromatron: I measured only request processing time. Querying data form 10 records dataset needs ~100ms, querying from 5000 records dataset needs ~ 2000s on my laptop and single node cassandra cluster | 14:38 |
| dukhlov | dynamodb data model allows up to 10gb data stored per partition key | 14:40 |
| jeromatron | how wide are the rows? and you're trying to emulate the dynamodb limits right - so people can seamlessly move over? | 14:40 |
| dukhlov | so it could be a great amount of data and filtering is not a option | 14:40 |
| jeromatron | perhaps a different data modelling route might be good. Have you spoken with Patrick McFadin about the modelling that you're doing? he works for datastax on the community side and has done a whole lot of data models with customers. he might have some tips. | 14:41 |
| dukhlov | no, I haven't | 14:42 |
| jeromatron | I'll forward the email to him and CC you and see if he has any suggestions. Feel free to add to the email. | 14:43 |
| dukhlov | but problem is that DynamoDB has a lot of features. And if we change data model usually it makes implementing another feature impossible or too complicated | 14:47 |
| dukhlov | like condition operations | 14:47 |
| dukhlov | ok, thank you I will talk with McFadin | 14:49 |
| jeromatron | ah okay. are you able to make the indexed column a clustering key? or does that not work for you generally? | 15:01 |
| jeromatron | anyway, I'll send off an email. | 15:04 |
| *** [o__o] has quit IRC | 15:51 | |
| *** [o__o] has joined #magnetodb | 15:53 | |
| *** [o__o] has quit IRC | 15:55 | |
| *** [o__o] has joined #magnetodb | 15:57 | |
| dukhlov | jeromatron: DynamoDB supports up to 5 LSI. We can use clustering key to organize only one of them | 16:03 |
| dukhlov | hm, actually now we are using composite clustering key to organize DynamoDB LSIs | 16:04 |
| dukhlov | but in this case we have a few CQL records in table (for original row and for one per LSI row) for DynamoDB row | 16:05 |
| dukhlov | and must take care about index integrity and consistency ourself | 16:06 |
| *** [o__o] has quit IRC | 16:13 | |
| *** [o__o] has joined #magnetodb | 16:17 | |
| jeromatron | hmm, are they keys based indexes with ddb? or bitmap indexes where you can do inequalities and such? | 16:20 |
| *** jeromatron has quit IRC | 16:23 | |
| dukhlov | DynamoDB has primary key which consists of 2 parts HASH key (it is partition key in C*, you can specify only equality condition on it to choose partition) and RANGE key (it is clustering key in cassandra, you can get slice using it, means it allows inequalities like greater or less conditions) | 16:28 |
| dukhlov | LSI is alternative RANGE key on the table | 16:29 |
| dukhlov | so, yes you can do inequalities with LSI as well as with RANGE key as well as Cassandra clustering key | 16:30 |
| dukhlov | http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LSI.html | 16:30 |
| dukhlov | CREATE TABLE <table_name> (hash_key_attr <text/decimal/blob>, index_name <text>,indexed_attr_str_value text,indexed_attr_number_value decimal,indexed_attr_blob_value blob,range_key_att <text/decimal/blob>, PRIMARY KEY (hash_key_attr, index_name, indexed_attr_value, range_key_att)) | 16:35 |
| dukhlov | here is our table structure | 16:36 |
| dukhlov | hash_key_attr is partition key | 16:36 |
| dukhlov | other 4 fields is composit clustering key | 16:37 |
| dukhlov | our table contains duplicated CQL rows for each LSI | 16:38 |
| dukhlov | and original row | 16:38 |
| dukhlov | and those rows are combined in sorted groups by index value | 16:40 |
| openstackgerrit | Longgeek proposed a change to stackforge/magnetodb: Update requirements file matching global requ https://review.openstack.org/103589 | 16:53 |
| openstackgerrit | Longgeek proposed a change to stackforge/magnetodb: Update requirements file matching global requ and add the six module https://review.openstack.org/103589 | 16:56 |
| *** vnaboychenko has joined #magnetodb | 17:24 | |
| *** vnaboychenko has quit IRC | 17:33 | |
| *** jeromatron has joined #magnetodb | 19:02 | |
| *** vnaboychenko has joined #magnetodb | 19:09 | |
| *** vnaboychenko has quit IRC | 19:22 | |
| *** jeromatron has quit IRC | 19:22 | |
| *** keith_newstadt has quit IRC | 19:25 | |
| *** jeromatron has joined #magnetodb | 19:33 | |
| *** jeromatron has quit IRC | 19:37 | |
| *** jeromatron has joined #magnetodb | 21:55 | |
| *** charlesw has quit IRC | 22:34 | |
| *** jeromatron has quit IRC | 22:43 | |
| *** openstackgerrit has quit IRC | 23:19 | |
| *** openstackgerrit has joined #magnetodb | 23:21 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!