14:00:52 <isviridov> #startmeeting magnetodb
14:00:53 <openstack> Meeting started Thu Jan 29 14:00:52 2015 UTC and is due to finish in 60 minutes.  The chair is isviridov. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:54 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:56 <openstack> The meeting name has been set to 'magnetodb'
14:01:19 <isviridov> achudnovets_ miqui o/
14:01:56 <isviridov> Not too many today
14:01:58 <ikhudoshyn> o/
14:02:05 <isviridov> Hello ikhudoshyn
14:02:13 <ajayaa> o/
14:02:21 <isviridov> Ok, let us start with action items as usually
14:02:25 <ominakov> o/
14:02:31 <isviridov> #topic Go through action items isviridov
14:02:38 <isviridov> ominakov hi
14:02:43 <isviridov> aostapenko charlesw isviridov brainstorm the scenario
14:03:09 <isviridov> Ok, it is about testing of concurrent writes
14:03:22 * isviridov looking for BP
14:03:44 <isviridov> #link https://blueprints.launchpad.net/magnetodb/+spec/test-concurrent-writes
14:04:18 <isviridov> I consider it as done. The BP is in approved state now
14:04:22 <openstackgerrit> Dmitriy Ukhlov proposed stackforge/magnetodb: Move setting up default encoding into setup_global_env  https://review.openstack.org/150810
14:04:49 <isviridov> During testing we have discovered a bug https://bugs.launchpad.net/magnetodb/+bug/1415478
14:04:55 <ikhudoshyn> there's a patch from aostapenko
14:05:01 <isviridov> I believe it is not the last one
14:05:19 <ikhudoshyn> i just mean it's not merged yet
14:05:28 <ikhudoshyn> why u say its done?
14:05:31 <aostapenko> o/
14:05:43 <isviridov> ikhudoshyn yeap. I'm not sure that it will be only one patch
14:05:58 <dukhlov_> hello
14:06:17 <isviridov> ikhudoshyn because action was to 'brainstorm the scenario'. You can read two patterns in BP description
14:06:25 <aostapenko> ikhudoshyn: this patch does not fix bug that isviridov mentions
14:06:35 <ikhudoshyn> a, got it, its bout AI not BP
14:06:48 <isviridov> ikhudoshyn yeap
14:07:03 <isviridov> aostapenko exactly
14:07:14 <isviridov> aostapenko anything to add?
14:07:55 <isviridov> Looks like not
14:08:14 <ikhudoshyn> move on?
14:08:19 <isviridov> Next AI is dukhlov_ charlesw aostapenko isviridov review https://review.openstack.org/#/c/146534/
14:08:50 <isviridov> It is merged, my congrats!
14:08:50 <dukhlov_> merged
14:09:27 <isviridov> We are done with action items
14:09:40 <isviridov> #topic Open discussion
14:10:10 <isviridov> ikhudoshyn I've seen reoly from charlesw in Notification refactoring BP, please take a look
14:10:25 <ikhudoshyn> sure
14:10:42 <miqui> isviridov: my feedback is am still learning/ramping up on the project...
14:10:58 <miqui> ..its been slow lately ...
14:11:07 <isviridov> miqui great to hear from you
14:11:27 <isviridov> What are you looking at right now?
14:11:45 <ajayaa> Hi guys. I wanted to talk about ttl feature.
14:12:04 <miqui> isviridov: how to test it and understand the code overall..
14:12:21 <isviridov> ajayaa yes, we have started discussion internally but no outcome in mail list. Sorry
14:12:23 <ajayaa> I sent a mail in the mailing list last to last week. Sadly nobody gave a look.
14:13:02 <ajayaa> isviridov, okay. Let me know your thoughts on mailing list.
14:13:04 <isviridov> miqui are you online after meeting? I would love to unswer your Q then.
14:13:27 * isviridov serching for his notes
14:13:33 <isviridov> dukhlov_ please join
14:14:06 <ajayaa> This blog post would help to understand the problems associated with ttl.
14:14:14 <ajayaa> http://ajayaa.github.io/cassandra-difference-between-insert-update/
14:14:15 <isviridov> #topic TTL feature for a row
14:14:15 <miqui> isviridov: , thanks i'll be online, but i have to  focus on something else...
14:14:21 <isviridov> #link http://lists.openstack.org/pipermail/openstack-dev/2015-January/054628.html
14:14:26 <miqui> isviridov: is tomorrow good?
14:14:32 <ajayaa> please read it when you are free.
14:14:36 <isviridov> miqui don't hesistate to contact me any question
14:14:47 <miqui> oh k, great... thanks...
14:15:10 <isviridov> ajayaa will do, but let us use this time to share oppinions we have already
14:15:11 <ajayaa> miqui, You can ping me also. I will be happy to help.
14:15:25 <ajayaa> isviridov, yes.
14:15:40 <ajayaa> +1
14:15:49 <miqui> ..thanks...
14:16:15 <isviridov> So, the idea you are suggesting (please correct me) is to allow TTL only for inserting
14:16:22 <ajayaa> yes.
14:16:46 <ikhudoshyn> ajayaa: that actually does not sound really good for me, sry
14:16:50 <isviridov> But with next update we will loose original TTL
14:17:20 <ajayaa> exactly. Cassandra provides column level ttl.
14:17:44 <isviridov> CQL what illustrates it #link http://paste.openstack.org/show/163676/
14:17:55 <ajayaa> instead if row level. It is one of those features which is being demanded by the community for a long time.
14:18:34 <ikhudoshyn> there's two other approaches we could also consider: 1) provide per-column TTL and use C* native one 2) DIY per row ttl
14:18:55 <ajayaa> DIY?
14:19:04 <ikhudoshyn> do-it-yourself ))
14:19:09 <ajayaa> ahh
14:19:12 <openstackgerrit> Andrei V. Ostapenko proposed stackforge/magnetodb: Adds tempest concurrent tests  https://review.openstack.org/150943
14:19:45 <dukhlov_> we can easily implement like per-row TTL for inserts only
14:19:49 <ikhudoshyn> just introduce servie attribute with per-row ttl and handle it on our own
14:20:29 <isviridov> ikhudoshyn noy sure we can implement 1 with current data structure as far as we have a one column with set of attributes in C*
14:20:29 <ikhudoshyn> dukhlov_: and claim that TTL on updates is unavailable?
14:20:47 <dukhlov_> yes
14:21:03 <dukhlov_> or implemet updates using inserts
14:21:12 <ikhudoshyn> dukhlov_: that would sound weird from user perspective
14:21:16 <dukhlov_> read whole row and insert it again
14:21:19 <ajayaa> ikhudoshyn, We would have to update all the columns for a row incase of an update.
14:21:37 <ajayaa> Otherwise some columns would dissappear and some would remain.
14:22:01 <ajayaa> For that you would need to fetch the row before updating.
14:22:03 <dukhlov_> our update_item request now works in this way
14:22:06 <ikhudoshyn> ajayaa: i actually meant not using C* native TTL, but intro separate attribute and check/update it manually
14:22:26 <dukhlov_> it reads whole row then peform modification and store it
14:23:07 <ajayaa> ohh...We need to run a process or thread which would clean up the data regularly.
14:23:26 <ikhudoshyn> ajayaa: sure we'll need it
14:23:41 <dukhlov_> it is inefficient but now it works in this way
14:24:03 <isviridov> dukhlov_ any possibilities to improve it?
14:24:39 * isviridov evaluating if we have to rely on current implemetation or it will be changed soon
14:25:14 <ajayaa> dukhlov, If you have specified return values as ALL_OLD then it is read, I think.
14:25:27 <isviridov> ajayaa with own TTL we have to take care about cleaning
14:25:31 <dukhlov_> we should choose Dynamo DB API support or efficiency
14:25:34 <ajayaa> You have written it, I might have missed something.
14:25:45 <ajayaa> dukhlov_ ^^
14:25:55 <ajayaa> isviridov, agreed.
14:27:01 <dukhlov_> no not only for return ald item
14:27:27 <ajayaa> dukhlov_ +1. I that is done in put_item. You are right.
14:27:41 <dukhlov_> fort example when you want increase number value by +1
14:28:02 <ajayaa> https://github.com/stackforge/magnetodb/blob/master/magnetodb/storage/driver/cassandra/cassandra_with_custom_lsi_impl.py#L863
14:28:36 <dukhlov_> cassandra cann't do this
14:28:41 <ajayaa> got it. :)
14:29:00 <ajayaa> There is a counter type in cassandra.
14:29:19 <ikhudoshyn> ajayaa: we evaluated that one, it's pretty limited
14:30:02 <dukhlov_> yes, but you can't create counter field with other types
14:30:02 <ajayaa> ikhudoshyn, okay. I would love to have a discussion sometime later around that. Let's not hijack ttl thread for now.
14:30:17 <ikhudoshyn> agree
14:30:23 <dukhlov_> you need to create separate table with counters only
14:31:16 <ajayaa> Do we have a consensus on how to go about ttl?
14:31:16 <isviridov> Ok, let me summarize
14:31:59 <isviridov> The only way for now looks like implementation ow raw TTL using custom field
14:32:21 <isviridov> The performance is not expected the best
14:32:44 <ikhudoshyn> isviridov: +1
14:32:55 <isviridov> TTL could be implemented to put and update
14:33:16 <isviridov> dukhlov_ ikhudoshyn ajayaa?
14:34:09 <ajayaa> +1 for custom which would provide consistency.
14:34:29 <dukhlov_> I'm worried about sense of such efforts
14:34:53 <dukhlov_> If Cassandra add row-level ttl in future
14:35:00 <charlesw> should we check with C* folks to see if they have any plan for that?
14:35:15 <dukhlov_> I don't think that it is so complicated
14:35:20 <isviridov> charlesw +1
14:35:31 <ikhudoshyn> charlesw: +1
14:35:38 <ajayaa> +1
14:35:40 <dukhlov_> charlesw: +1
14:35:41 <aostapenko> +1
14:36:05 <ajayaa> If we can get a timeline for that then it would be awesome.
14:37:00 <isviridov> ajayaa charlesw what are you excpectation for availability of this functionality if there are any?
14:37:22 * isviridov actually it is question to Symatec, Relience :)
14:37:28 <openstackgerrit> Andrei V. Ostapenko proposed stackforge/magnetodb: Adds tempest concurrent tests  https://review.openstack.org/150943
14:38:04 <ajayaa> We don't have an immediate requirement but something like nice to have.
14:38:07 <charlesw> On symantec side, we are not in urgent need for that
14:38:48 <isviridov> Clear. charlesw will to check if C* is going to ass row level TTL?
14:39:02 <isviridov> * will you
14:39:12 <charlesw> sure, I'll dig into that
14:40:02 <isviridov> #action charlesw clarify what are plans for row TTL in C* community
14:40:29 <openstackgerrit> Dmitriy Ukhlov proposed stackforge/magnetodb: Move setting up default encoding into setup_global_env  https://review.openstack.org/150810
14:40:30 <isviridov> #agreed try leverage C* nativre TTL
14:40:38 <openstackgerrit> Andrei V. Ostapenko proposed stackforge/magnetodb: Adds tempest concurrent tests  https://review.openstack.org/150943
14:40:50 <isviridov> I've moved https://blueprints.launchpad.net/magnetodb/+spec/row-expiration to kilo-3 for now
14:41:14 <ajayaa> okay.
14:41:18 <isviridov> Great discussion!
14:41:55 <isviridov> Next topic? Any suggestions?
14:42:24 <isviridov> ominakov how it is going with migration solution?
14:43:04 <charlesw> we should have a formal/standard way of upgrade/migration
14:43:48 <ominakov> now, I'am testing it on real env and have some issues - some tables failed with CREATE_FAILED status
14:44:17 <charlesw> you should remove those tables
14:44:45 <charlesw> are they existing CREATE_FAILED tables?
14:44:52 <isviridov> ominakov any BP for it?
14:45:05 <ominakov> isviridov, not yet (
14:45:48 <ominakov> charlesw, nope, it first time creating
14:45:59 <isviridov> ominakov please create it with description of approach you are using, now it is not clear for community
14:46:19 <ominakov> sure, i'll do it
14:46:34 <charlesw> Also please submit your patch for review
14:47:15 <isviridov> #action ominakov file blueprint about migration
14:47:32 <isviridov> #topic Open discussion
14:48:31 <ikhudoshyn> https://review.openstack.org/146909
14:49:54 <isviridov> ikhudoshyn will check
14:50:00 <ikhudoshyn> tnx
14:50:08 <isviridov> Anythink else?
14:50:18 <isviridov> * anything
14:50:30 <aostapenko> :)
14:50:38 <ajayaa> nope. Thanks guy!
14:50:41 <ajayaa> guys*
14:51:04 <isviridov> Finshed for today? 3..2..1..
14:51:27 <isviridov> Thank you for comming
14:51:44 <charlesw> Thanks for organinzing
14:51:44 <ikhudoshyn> cya guys
14:52:18 <isviridov> vivekd miqui ygbo there is a PyCharm Pro license for MagnetoDB contributors if you need it, just ping me
14:52:38 <isviridov> #endmeeting