14:00:52 #startmeeting magnetodb 14:00:53 Meeting started Thu Jan 29 14:00:52 2015 UTC and is due to finish in 60 minutes. The chair is isviridov. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:54 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:56 The meeting name has been set to 'magnetodb' 14:01:19 achudnovets_ miqui o/ 14:01:56 Not too many today 14:01:58 o/ 14:02:05 Hello ikhudoshyn 14:02:13 o/ 14:02:21 Ok, let us start with action items as usually 14:02:25 o/ 14:02:31 #topic Go through action items isviridov 14:02:38 ominakov hi 14:02:43 aostapenko charlesw isviridov brainstorm the scenario 14:03:09 Ok, it is about testing of concurrent writes 14:03:22 * isviridov looking for BP 14:03:44 #link https://blueprints.launchpad.net/magnetodb/+spec/test-concurrent-writes 14:04:18 I consider it as done. The BP is in approved state now 14:04:22 Dmitriy Ukhlov proposed stackforge/magnetodb: Move setting up default encoding into setup_global_env https://review.openstack.org/150810 14:04:49 During testing we have discovered a bug https://bugs.launchpad.net/magnetodb/+bug/1415478 14:04:55 there's a patch from aostapenko 14:05:01 I believe it is not the last one 14:05:19 i just mean it's not merged yet 14:05:28 why u say its done? 14:05:31 o/ 14:05:43 ikhudoshyn yeap. I'm not sure that it will be only one patch 14:05:58 hello 14:06:17 ikhudoshyn because action was to 'brainstorm the scenario'. You can read two patterns in BP description 14:06:25 ikhudoshyn: this patch does not fix bug that isviridov mentions 14:06:35 a, got it, its bout AI not BP 14:06:48 ikhudoshyn yeap 14:07:03 aostapenko exactly 14:07:14 aostapenko anything to add? 14:07:55 Looks like not 14:08:14 move on? 14:08:19 Next AI is dukhlov_ charlesw aostapenko isviridov review https://review.openstack.org/#/c/146534/ 14:08:50 It is merged, my congrats! 14:08:50 merged 14:09:27 We are done with action items 14:09:40 #topic Open discussion 14:10:10 ikhudoshyn I've seen reoly from charlesw in Notification refactoring BP, please take a look 14:10:25 sure 14:10:42 isviridov: my feedback is am still learning/ramping up on the project... 14:10:58 ..its been slow lately ... 14:11:07 miqui great to hear from you 14:11:27 What are you looking at right now? 14:11:45 Hi guys. I wanted to talk about ttl feature. 14:12:04 isviridov: how to test it and understand the code overall.. 14:12:21 ajayaa yes, we have started discussion internally but no outcome in mail list. Sorry 14:12:23 I sent a mail in the mailing list last to last week. Sadly nobody gave a look. 14:13:02 isviridov, okay. Let me know your thoughts on mailing list. 14:13:04 miqui are you online after meeting? I would love to unswer your Q then. 14:13:27 * isviridov serching for his notes 14:13:33 dukhlov_ please join 14:14:06 This blog post would help to understand the problems associated with ttl. 14:14:14 http://ajayaa.github.io/cassandra-difference-between-insert-update/ 14:14:15 #topic TTL feature for a row 14:14:15 isviridov: , thanks i'll be online, but i have to focus on something else... 14:14:21 #link http://lists.openstack.org/pipermail/openstack-dev/2015-January/054628.html 14:14:26 isviridov: is tomorrow good? 14:14:32 please read it when you are free. 14:14:36 miqui don't hesistate to contact me any question 14:14:47 oh k, great... thanks... 14:15:10 ajayaa will do, but let us use this time to share oppinions we have already 14:15:11 miqui, You can ping me also. I will be happy to help. 14:15:25 isviridov, yes. 14:15:40 +1 14:15:49 ..thanks... 14:16:15 So, the idea you are suggesting (please correct me) is to allow TTL only for inserting 14:16:22 yes. 14:16:46 ajayaa: that actually does not sound really good for me, sry 14:16:50 But with next update we will loose original TTL 14:17:20 exactly. Cassandra provides column level ttl. 14:17:44 CQL what illustrates it #link http://paste.openstack.org/show/163676/ 14:17:55 instead if row level. It is one of those features which is being demanded by the community for a long time. 14:18:34 there's two other approaches we could also consider: 1) provide per-column TTL and use C* native one 2) DIY per row ttl 14:18:55 DIY? 14:19:04 do-it-yourself )) 14:19:09 ahh 14:19:12 Andrei V. Ostapenko proposed stackforge/magnetodb: Adds tempest concurrent tests https://review.openstack.org/150943 14:19:45 we can easily implement like per-row TTL for inserts only 14:19:49 just introduce servie attribute with per-row ttl and handle it on our own 14:20:29 ikhudoshyn noy sure we can implement 1 with current data structure as far as we have a one column with set of attributes in C* 14:20:29 dukhlov_: and claim that TTL on updates is unavailable? 14:20:47 yes 14:21:03 or implemet updates using inserts 14:21:12 dukhlov_: that would sound weird from user perspective 14:21:16 read whole row and insert it again 14:21:19 ikhudoshyn, We would have to update all the columns for a row incase of an update. 14:21:37 Otherwise some columns would dissappear and some would remain. 14:22:01 For that you would need to fetch the row before updating. 14:22:03 our update_item request now works in this way 14:22:06 ajayaa: i actually meant not using C* native TTL, but intro separate attribute and check/update it manually 14:22:26 it reads whole row then peform modification and store it 14:23:07 ohh...We need to run a process or thread which would clean up the data regularly. 14:23:26 ajayaa: sure we'll need it 14:23:41 it is inefficient but now it works in this way 14:24:03 dukhlov_ any possibilities to improve it? 14:24:39 * isviridov evaluating if we have to rely on current implemetation or it will be changed soon 14:25:14 dukhlov, If you have specified return values as ALL_OLD then it is read, I think. 14:25:27 ajayaa with own TTL we have to take care about cleaning 14:25:31 we should choose Dynamo DB API support or efficiency 14:25:34 You have written it, I might have missed something. 14:25:45 dukhlov_ ^^ 14:25:55 isviridov, agreed. 14:27:01 no not only for return ald item 14:27:27 dukhlov_ +1. I that is done in put_item. You are right. 14:27:41 fort example when you want increase number value by +1 14:28:02 https://github.com/stackforge/magnetodb/blob/master/magnetodb/storage/driver/cassandra/cassandra_with_custom_lsi_impl.py#L863 14:28:36 cassandra cann't do this 14:28:41 got it. :) 14:29:00 There is a counter type in cassandra. 14:29:19 ajayaa: we evaluated that one, it's pretty limited 14:30:02 yes, but you can't create counter field with other types 14:30:02 ikhudoshyn, okay. I would love to have a discussion sometime later around that. Let's not hijack ttl thread for now. 14:30:17 agree 14:30:23 you need to create separate table with counters only 14:31:16 Do we have a consensus on how to go about ttl? 14:31:16 Ok, let me summarize 14:31:59 The only way for now looks like implementation ow raw TTL using custom field 14:32:21 The performance is not expected the best 14:32:44 isviridov: +1 14:32:55 TTL could be implemented to put and update 14:33:16 dukhlov_ ikhudoshyn ajayaa? 14:34:09 +1 for custom which would provide consistency. 14:34:29 I'm worried about sense of such efforts 14:34:53 If Cassandra add row-level ttl in future 14:35:00 should we check with C* folks to see if they have any plan for that? 14:35:15 I don't think that it is so complicated 14:35:20 charlesw +1 14:35:31 charlesw: +1 14:35:38 +1 14:35:40 charlesw: +1 14:35:41 +1 14:36:05 If we can get a timeline for that then it would be awesome. 14:37:00 ajayaa charlesw what are you excpectation for availability of this functionality if there are any? 14:37:22 * isviridov actually it is question to Symatec, Relience :) 14:37:28 Andrei V. Ostapenko proposed stackforge/magnetodb: Adds tempest concurrent tests https://review.openstack.org/150943 14:38:04 We don't have an immediate requirement but something like nice to have. 14:38:07 On symantec side, we are not in urgent need for that 14:38:48 Clear. charlesw will to check if C* is going to ass row level TTL? 14:39:02 * will you 14:39:12 sure, I'll dig into that 14:40:02 #action charlesw clarify what are plans for row TTL in C* community 14:40:29 Dmitriy Ukhlov proposed stackforge/magnetodb: Move setting up default encoding into setup_global_env https://review.openstack.org/150810 14:40:30 #agreed try leverage C* nativre TTL 14:40:38 Andrei V. Ostapenko proposed stackforge/magnetodb: Adds tempest concurrent tests https://review.openstack.org/150943 14:40:50 I've moved https://blueprints.launchpad.net/magnetodb/+spec/row-expiration to kilo-3 for now 14:41:14 okay. 14:41:18 Great discussion! 14:41:55 Next topic? Any suggestions? 14:42:24 ominakov how it is going with migration solution? 14:43:04 we should have a formal/standard way of upgrade/migration 14:43:48 now, I'am testing it on real env and have some issues - some tables failed with CREATE_FAILED status 14:44:17 you should remove those tables 14:44:45 are they existing CREATE_FAILED tables? 14:44:52 ominakov any BP for it? 14:45:05 isviridov, not yet ( 14:45:48 charlesw, nope, it first time creating 14:45:59 ominakov please create it with description of approach you are using, now it is not clear for community 14:46:19 sure, i'll do it 14:46:34 Also please submit your patch for review 14:47:15 #action ominakov file blueprint about migration 14:47:32 #topic Open discussion 14:48:31 https://review.openstack.org/146909 14:49:54 ikhudoshyn will check 14:50:00 tnx 14:50:08 Anythink else? 14:50:18 * anything 14:50:30 :) 14:50:38 nope. Thanks guy! 14:50:41 guys* 14:51:04 Finshed for today? 3..2..1.. 14:51:27 Thank you for comming 14:51:44 Thanks for organinzing 14:51:44 cya guys 14:52:18 vivekd miqui ygbo there is a PyCharm Pro license for MagnetoDB contributors if you need it, just ping me 14:52:38 #endmeeting