| *** vnaboychenko has quit IRC | 00:21 | |
| *** charlesw has joined #magnetodb | 00:53 | |
| *** openstackgerrit has quit IRC | 01:06 | |
| *** openstackgerrit has joined #magnetodb | 01:08 | |
| *** vnaboychenko has joined #magnetodb | 02:49 | |
| *** vnaboychenko has quit IRC | 02:50 | |
| *** charlesw has quit IRC | 03:05 | |
| *** vnaboychenko has joined #magnetodb | 04:32 | |
| *** idegtiarov has quit IRC | 08:07 | |
| *** idegtiarov has joined #magnetodb | 08:09 | |
| *** ominakov has joined #magnetodb | 08:14 | |
| *** jeromatron has joined #magnetodb | 09:31 | |
| *** denis_makogon has joined #magnetodb | 09:58 | |
| *** istegantsev has joined #magnetodb | 10:06 | |
| *** vnaboychenko has quit IRC | 10:08 | |
| *** istegantsev has quit IRC | 10:12 | |
| openstackgerrit | Ilya Sviridov proposed a change to stackforge/python-magnetodbclient: Fixed git repo name in .gitreview https://review.openstack.org/103080 | 10:33 |
|---|---|---|
| openstackgerrit | Illia Khudoshyn proposed a change to stackforge/magnetodb: Add notifications to streaming API https://review.openstack.org/102786 | 12:37 |
| *** keith_newstadt has quit IRC | 12:42 | |
| *** ashestakov has quit IRC | 12:42 | |
| *** keith_newstadt has joined #magnetodb | 12:59 | |
| *** charlesw has joined #magnetodb | 13:31 | |
| openstackgerrit | Oleksandr Minakov proposed a change to stackforge/magnetodb: Adds documentation from docstrings https://review.openstack.org/103132 | 13:53 |
| openstackgerrit | Oleksandr Minakov proposed a change to stackforge/magnetodb: Adds documentation from docstrings https://review.openstack.org/103132 | 13:54 |
| dukhlov | jeromatron: Hello | 13:55 |
| jeromatron | hi | 13:56 |
| dukhlov | Regarding Global Index story. Now MagnetoDB has only Local Secondary Indexes implemented | 14:03 |
| dukhlov | and for this feature we dont use Cassandra Native secondary indexes | 14:04 |
| dukhlov | because we need DynamoDB like LSI logic | 14:04 |
| dukhlov | so we need to determine partition where we want to lookup data | 14:05 |
| dukhlov | and then in scope of this partition get search data by indexed value | 14:06 |
| jeromatron | right. so it sounds like you have your own magneto system in place to be able to do that. okay. | 14:06 |
| dukhlov | yes | 14:06 |
| jeromatron | I would comment on that ticket if you're interested in seeing the global indexes doing something similar or if you have preferences there. they're still in the research phase too so it's really the best time to talk about things like that. | 14:07 |
| jeromatron | it would save you from having to maintain that code in the future I would think. | 14:07 |
| dukhlov | I want to understand for now do you plan to implement or improve Cassandra Native feature to help us? | 14:07 |
| jeromatron | currently it's that they would like to add a built-in way to do non-local indexes and it's initially similar to the way dynamodb does global indexes. that's my understanding. so now if you want to do an index like that, you have to denormalize to two different column families. | 14:09 |
| jeromatron | I'm not involved in the decision making for the details so I would comment on the ticket directly if you have feedback or preferences on how it's implemented. | 14:11 |
| dukhlov | Ok. I will comment that issue. We need GSI in future too. But for now it would be nice to know how we can organize LSI using Cassandra native features. Is it possible improve current Cassandra LSI? Maybe we can use C* GSI for MagnetoDB LSI, or maybe we have no choice and have to implement it manually | 14:17 |
| dukhlov | I would be great have vision what is the most perspective direction | 14:17 |
| jeromatron | what additions would you need to LSI? right now they're very limited and not performant. they're looking at improving that, but what specifically do you need? | 14:21 |
| dukhlov | As far I understood the problem is that C* need to query a lot of custer's nodes (in worst case) | 14:23 |
| dukhlov | but our LSI define patrition where to find | 14:24 |
| dukhlov | we need ability to specify Partition key | 14:24 |
| jeromatron | if you specify a partition key, then you don't need LSI... | 14:25 |
| dukhlov | and then execute '>','<','=' condition on indexed fields | 14:25 |
| dukhlov | I am talking about composite primary key | 14:26 |
| dukhlov | if we have primary key which consist of partition key and clustering key | 14:26 |
| dukhlov | we can execute query by primary key like | 14:27 |
| dukhlov | SELECT WHERE PK='pk1' AND CK>'cl1' | 14:27 |
| dukhlov | but we need ability to define alternative clustering key | 14:28 |
| dukhlov | this alternative clustering key is called MagnetoDB LSI | 14:30 |
| jeromatron | I'm just checking on that because it seems like it should be doable because if you have a clustering key, that should be a lookup away from the partition key. | 14:30 |
| dukhlov | yes, right | 14:31 |
| dukhlov | we can execute query by partition key and clustering key (query by primary index) | 14:32 |
| dukhlov | but, for example now we have additional field 'user_age' | 14:33 |
| ikhudoshyn | dukhlov, I guess I'm done with the https://review.openstack.org/#/c/102786/ | 14:34 |
| ikhudoshyn | could you pls take a look | 14:34 |
| dukhlov | if we define MagnetoDB LSI we should have ability to pefrom next query | 14:34 |
| ikhudoshyn | so we could decide which one yrs or mine will go 1st | 14:34 |
| dukhlov | SELECT WHERE PK='pk' and user_age>16 | 14:35 |
| dukhlov | but if we define C* LSI on user_age | 14:35 |
| dukhlov | we can't execute such query because somehow we can't mix primary key and LSI fields in the same query | 14:36 |
| dukhlov | maybe it make sens | 14:36 |
| dukhlov | but if C* LSI performance problem is connected with involving whole cluster, maybe we can specify partition for looking up in query on client side to improve performance | 14:38 |
| dukhlov | and allow to add partition key condition to query on LSI | 14:39 |
| dukhlov | ikhudoshyn: ok, thanks | 14:39 |
| jeromatron | yeah - seems like it should be taking advantage of the knowledge of the partition key instead of doing a scatter/gather query, then do the lookup on the clustering key. | 14:41 |
| openstackgerrit | Oleksandr Minakov proposed a change to stackforge/magnetodb: Adds documentation from docstrings https://review.openstack.org/103132 | 14:43 |
| jeromatron | dukhlov: so did you create a native C* secondary index for anything on that column family? | 15:11 |
| dukhlov | no now we don't use C* LSI | 15:14 |
| dukhlov | we implemented it manually and have performance issues | 15:15 |
| *** idegtiarov has quit IRC | 15:15 | |
| dukhlov | now I think how to improve them and all approaches to to this looks very complicated. And if we have choice to use some C* native support it can help our life a lot | 15:18 |
| jeromatron | right - if you specify the partition key, it shouldn't do the full secondary index lookup even if there is a 2i defined. | 15:19 |
| jeromatron | it will go right to the parition and look up the rest of the query. | 15:19 |
| *** openstackgerrit has quit IRC | 15:21 | |
| dukhlov | but as far as I understand it go to right to the partition and perform filtering on fields which are not part of clustering key | 15:23 |
| jeromatron | ok, maybe I've misunderstood then, what would be the preferred thing for it to do specifically. you're saying that it currently doesn't take into account the clustering key? | 15:24 |
| dukhlov | ок, I will explain on examples. let me 5 min to prepare queries which I want to execute | 15:34 |
| jeromatron | dukhlov: I need to take off but you can email me the details to jeremy at datastax and I can take a look and do some research. | 15:41 |
| *** jeromatron has quit IRC | 15:42 | |
| *** openstackgerrit has joined #magnetodb | 15:45 | |
| *** ominakov_ has joined #magnetodb | 16:11 | |
| *** ominakov has quit IRC | 16:13 | |
| *** vnaboychenko has joined #magnetodb | 16:47 | |
| *** vnaboychenko has quit IRC | 17:04 | |
| *** jeromatron has joined #magnetodb | 17:05 | |
| *** jeromatron has quit IRC | 17:05 | |
| *** jeromatron has joined #magnetodb | 17:07 | |
| *** jeromatron has quit IRC | 17:19 | |
| *** vnaboychenko has joined #magnetodb | 17:28 | |
| *** vnaboychenko has quit IRC | 17:32 | |
| *** ominakov_ has quit IRC | 17:33 | |
| *** ominakov has joined #magnetodb | 17:36 | |
| openstackgerrit | Charles Wang proposed a change to stackforge/magnetodb: Disable data item level operation events for message queueing. https://review.openstack.org/100736 | 18:03 |
| *** denis_makogon has quit IRC | 18:05 | |
| *** jeromatron has joined #magnetodb | 18:28 | |
| *** ominakov_ has joined #magnetodb | 18:39 | |
| *** ominakov has quit IRC | 18:40 | |
| *** jeromatron has quit IRC | 18:40 | |
| *** ominakov_ has quit IRC | 18:48 | |
| *** jeromatron has joined #magnetodb | 18:48 | |
| *** openstackgerrit has quit IRC | 19:20 | |
| *** openstackgerrit has joined #magnetodb | 19:22 | |
| *** jeromatron has quit IRC | 19:36 | |
| *** ikhudoshyn has quit IRC | 19:43 | |
| *** ikhudoshyn has joined #magnetodb | 19:43 | |
| *** jeromatron has joined #magnetodb | 19:47 | |
| openstackgerrit | Dmitriy Ukhlov proposed a change to stackforge/magnetodb: Fix storage tests after notification patch merging https://review.openstack.org/100874 | 20:35 |
| *** charlesw_ has joined #magnetodb | 21:25 | |
| *** ikhudoshyn_ has joined #magnetodb | 21:25 | |
| *** ikhudoshyn has quit IRC | 21:31 | |
| *** charlesw has quit IRC | 21:31 | |
| *** charlesw_ is now known as charlesw | 21:31 | |
| *** jeromatron has quit IRC | 21:40 | |
| *** keith_newstadt has quit IRC | 21:56 | |
| *** charlesw has quit IRC | 22:07 | |
| *** ominakov has joined #magnetodb | 23:13 | |
| *** openstackgerrit has quit IRC | 23:37 | |
| *** vnaboychenko has joined #magnetodb | 23:52 | |
| *** vnaboychenko has quit IRC | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!