18:00:09 <SlickNik> #startmeeting trove-bp-review
18:00:10 <openstack> Meeting started Mon Oct 20 18:00:09 2014 UTC and is due to finish in 60 minutes.  The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:12 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:14 <openstack> The meeting name has been set to 'trove_bp_review'
18:00:50 <SlickNik> Waiting a couple of minutes for folks to trickle in.
18:01:10 <SlickNik> Agenda at
18:01:12 <SlickNik> #link https://wiki.openstack.org/wiki/Meetings/TroveBPMeeting
18:01:24 <amrith> ./
18:01:25 <dougshelley66> o/
18:01:29 <denis_makogon> o/
18:01:35 <vgnbkr> trickle, trickle
18:01:54 <amcrn> percolate percolate
18:02:12 <schang> o/
18:03:06 <SlickNik> Okay, let's get started.
18:03:12 <SlickNik> #topic Handle oslo-incubator module graduation
18:03:22 <amrith> hello
18:03:24 <amrith> that's me
18:03:29 <amrith> here comes a paste-bomb
18:03:31 <SlickNik> #link https://review.openstack.org/#/c/123571/
18:03:46 <amrith> the proposal in this blueprint initially came from the observation that oslo.concurrency has graduated and now we need to get code from there (not oslo-incubator).
18:03:46 <amrith> subsequent to writing this, a new proposal from oslo team indicates that they are going to more aggressively delete code from oslo-incubator and therefore I expect that in addition there will have to be a wider set of projects to address this change.
18:03:46 <amrith> http://markmail.org/message/3w7mcgfdwc4cll7f
18:03:46 <amrith> Robert's comment re: nova led to some email exchange where we (Robert and I) concluded that we have no dependency on lockutils so I'll update the bp to reflect that.
18:03:49 <amrith> I have submitted some sample code in the form of four reviews.
18:03:51 <amrith> https://review.openstack.org/#/c/129292/
18:03:53 <amrith> https://review.openstack.org/#/c/129294/
18:03:55 <amrith> https://review.openstack.org/#/c/129295/
18:03:57 <amrith> https://review.openstack.org/#/c/129378/
18:03:59 <amrith> One thing to keep in mind with these merges is that it is ok to change code in trove/openstack/common as far as it relates to repointing libraries to a different location for a graduated oslo library.
18:04:02 <amrith> A similar thing can be seen in this review:
18:04:06 <amrith> https://review.openstack.org/#/c/128644/
18:04:08 <amrith> I've got most of the changes ready to go, right now I'm trying to figure out why rdjenkins is failing for three of them; the tests work for me locally, not sure why they're failing on gate. Hypothesis is that the guest image doesn't have the dependencies installed.
18:04:12 <amrith> <end paste bomb>
18:04:26 <amrith> so, I've pushed a bunch of sample code modeled on other projects
18:04:31 <amrith> I'm just wrapping up i18n
18:04:35 <amrith> gettextutils
18:04:46 <amrith> feedback on the bp
18:04:52 <amrith> is primarily the purpose of this meeting
18:04:58 <georgelorch> o/
18:05:08 <amrith> but I felt that feedback on the bp may also depend on some idea of what the implementation looks like
18:05:12 <amrith> hence the sample code
18:05:17 <amrith> now I'll shut up and listen
18:06:06 <SlickNik> Awesome, thanks for getting started on this Amrith.
18:06:42 <SlickNik> We've got quite a bit of code that is being retired from oslo-incubator in favor of graduating oslo projects.
18:06:54 <SlickNik> So it will be good to move to them.
18:07:04 <denis_makogon> correct, but we can't get rid of wsgi stuff in terms of this BP
18:08:34 <SlickNik> So one question I have is whether these are BPs and bugs.
18:08:38 <SlickNik> or*
18:08:38 <peterstac> o/
18:08:54 <grapex> o/
18:08:55 <amrith> SlickNik, happy to go either way.
18:09:08 <amrith> I'd started with a BP for o-i -> o.c
18:09:11 <amrith> so I stuck with that
18:09:16 <amrith> other projects opted for bugs
18:09:19 <SlickNik> Probably best to figure out how the other projects are handling them, and stay consistent.
18:10:05 <amrith> If you follow the thread: http://markmail.org/message/3w7mcgfdwc4cll7f
18:10:15 <amrith> many projects have replied with a bug number and some patch sets
18:10:25 <amrith> I was going to reply with the patch sets once the BP's are discussed here
18:10:57 <amrith> one thing that I will point out
18:11:05 <amrith> is that we (trove) appear to have a lot more of this
18:11:07 <amrith> than other projects
18:11:21 <amrith> so, one way or another, we're going to have a lot more files changing
18:11:31 <amrith> not a lot of lines of change for the ones I'm dealing with
18:11:41 <amrith> but I know that sgotliv is working on rpc and that's a beast
18:12:13 <amrith> there are a couple of specific things I'd like people to consider
18:12:27 <amrith> 1. note that in this process, we're going to be updating files in trove/openstack/common
18:12:33 <amrith> things that were in o-i.
18:12:39 <amrith> this is to reflect the new dependencies.
18:12:55 <SlickNik> Yup, so I think it's probably best to split up the work into bugs (one per each module).
18:12:58 <amrith> 2. i've opted to do this as multiple chkins, one per graduated module
18:13:10 <amrith> and that way reviewers get to see commonality
18:13:16 <amrith> I've also made them a long dependency trail
18:13:28 <amrith> 3. finally, I'm having rdjenkins failures
18:13:33 <SlickNik> That way it's fairly easy to split up the work into different patchsets (from independent contributors) if need be.
18:13:36 <amrith> and I'm thinking this is because of the guest agent
18:13:47 <amrith> not having the required dependencies.
18:14:13 <amrith> SlickNik, re: multiple contributors; for the work in this BP, I've got most of the changes submitted at this point
18:14:16 <amrith> there are two more
18:14:26 <amrith> that I'm still tweaking a bit
18:14:33 <amrith> but I'll have those up in the next couple of days.
18:14:42 <amrith> then I get to wait for oslo.concurrency to graduate
18:14:49 <amrith> and then I can push that change set up as well.
18:15:12 <amrith> I'd appreciate thoughts on the BP, and especially concurrency on #1 above, and any thoughts on #3 above.
18:15:26 <amrith> s/concurrency/agreement/
18:16:41 <SlickNik> 3 is likely the missing dependencies. So we'll have to work on getting them in the cached image.
18:16:55 <amrith> ok. just to be clear, int-tests pass locally.
18:17:21 <SlickNik> amrith: Yes, but we need to make sure the gate isn't borked.
18:17:52 <amrith> yup
18:18:02 <amrith> also, the chain of chkins makes it easy to review
18:18:05 <SlickNik> So for #1 - if I read correctly, we will move from spec to bug?
18:18:09 <amrith> but for commit, do you want me to squash?
18:18:17 <amrith> #1: yes, I will enter a bug
18:18:32 <amrith> and sorry
18:18:35 <amrith> you entered a bug
18:18:36 <amrith> https://bugs.launchpad.net/trove/+bug/1380789
18:18:42 <amrith> I've been using that in commits
18:18:49 <amrith> as partial-bug: #1380789
18:18:53 <amrith> I'll stick with that.
18:20:15 <amrith> #1: ok with that bug?
18:20:26 <SlickNik> Sure thing. If any of them (like RPC) is a beast that warrants a separate bug, feel free to open a new one for it.
18:20:58 <SlickNik> But I'm okay with using a common bug to tackle a bunch of the smaller dependencies.
18:21:04 <amrith> RPC (I believe) is already being tracked by bp rpc-versioning
18:21:10 <amrith> see https://review.openstack.org/#/c/94484
18:21:22 <amrith> https://blueprints.launchpad.net/trove/+spec/rpc-versioning
18:22:29 <SlickNik> yes, I guess that's slightly different because moving to oslo.messaging also gives us RPC versioning.
18:22:52 <SlickNik> whereas for these other ones, we're just going for equivalence
18:23:24 <amrith> ok, with https://bugs.launchpad.net/trove/+bug/1380789 we have equivalence
18:23:35 <amrith> We can punt this bp if that is preferable
18:24:39 <SlickNik> Okay sounds good.
18:24:59 <SlickNik> Will work offline with you on #3.
18:25:06 <amrith> undersood.
18:25:34 <amrith> I think I have what I need to move forward. If people want to start reviewing the code, a list of change sets is available in the etherpad https://etherpad.openstack.org/p/trove-kilo-do-not-use-obsolete-oslo-modules
18:25:51 <SlickNik> Sounds good — thanks amrith!
18:25:59 <amrith> thanks.
18:26:34 <SlickNik> #topic Cassandra clustering
18:26:43 <denis_makogon> mine
18:27:53 <denis_makogon> #link https://review.openstack.org/#/c/122736/5/specs/kilo/cassandra-cluster.rst,cm
18:28:34 <vgnbkr> It seems that there are 2 components here: changes to APIs, and a Cassandra implementation of those APIs.  I would like to see separate blueprints for API changes so that they can be discussed with thought to how they affect all datastores.
18:29:14 <vgnbkr> (I mean that genearally, not just in this specific instance)
18:29:36 <denis_makogon> vgnbkr, those API already exists
18:29:38 <amcrn> i wouldn't mind if they stayed the same blueprint, but having a dedicated section that succinctly explains the api changes would be helpful.
18:29:43 <SlickNik> Are there really changes to the API that are needed for this?
18:29:45 <amcrn> ex: isn't returning shard_Id
18:30:01 <denis_makogon> SlickNik, no
18:30:01 <amcrn> well, not "changes", but differences i suppose is a better term
18:30:03 <SlickNik> Or is this an oversight with the BP, and the BP needs to be updated?
18:30:58 <vgnbkr> There is at small section at the end about making changes to APIs.  I thought API changes deserved more than a footnote.
18:32:23 <SlickNik> vgnbkr / amcrn: I haven't gotten there yet. +1 about calling those out upfront, since we need to make sure the API changes are backwards compatible.
18:32:24 <denis_makogon> those API changes are specific to C*, there's enough info about why those changes were made
18:33:16 <denis_makogon> SlickNik, API changes - is nothing else than another View, but specific for C*, same as amcrn did for Mongo
18:33:18 <denis_makogon> nothing else
18:33:20 <amcrn> as far as i can tell, the only noteable differences from the mongo spec are: not returning shard_id, using add_node vs. add_shard, returning all node's ip's vs. just query routers as seen in mongodb's case, naming of the instances as "-node-" vs. what's seen in mongodb.
18:33:45 <denis_makogon> amcrn, correct
18:33:55 <dougshelley66> wouldn't it make sense for that detail to be in the spec?
18:34:03 <amcrn> ^^
18:34:16 <denis_makogon> dougshelley66, what kind of details ?
18:34:45 <dougshelley66> denis_makogon, the details that you indicated about + what amcrn just stated
18:34:51 <dougshelley66> s/about/above
18:34:56 <SlickNik> denis_makogon: Details calling out what the _exact_ API differences are.
18:35:20 <amcrn> vs. having folks extrapolate them.
18:35:33 <amcrn> overall looks pretty good
18:35:36 <denis_makogon> SlickNik, we don't have generic API for clustering, we datastore-specific API
18:36:14 <denis_makogon> and don't see a value-add on comparing C* clustering API changes against Mongo clustering API
18:36:46 <amcrn> denis_makogon: the only point here is that it takes too long to answer the question "what changes in behavior will i see as a user compared to the mongodb clustering implementation?". all of the information is in there, it just has to be teased out.
18:37:33 <SlickNik> denis_makogon: We spent a lot of time making sure that the API we're using for clustering is similar for across datastores.
18:37:38 <amrith> one question which I have about this spec is this. In contrast to mysql and mongodb where we have end-users who can validate the implementation, do we have confidence in this implementation/design path? I'd hate to see us go down this path to find that the cassandra community prefers or recommends some other approach.
18:37:54 <SlickNik> So I'm not sure I understand your comment about "datastore-specific" API
18:38:47 <denis_makogon> SlickNik, current clustering framework allows to build datastore-specific response views, that's all
18:39:59 <amrith> the design choices we made around replication and clustering were intended to provide a common framework that would work for multiple datastores. So I'm a little concerned that we may be diverging from the framework we started with on MongoDB and if we are, I'd like to make sure we are all in agreement that the divergence is worthwhile, the best way to do things etc.,
18:40:10 <denis_makogon> amrith, i have lots of DBA of C* in production, they know best practices on how to do C* clustering, also there's enough docs with HOWTO (taking into account major DataStax docs)
18:40:42 <denis_makogon> there's only one way to do clustering for C*
18:40:57 <SlickNik> denis_makogon: Awesome, can we add one of these DBAs as a co-owner in the docs, and get him to the BP meeting to talk about the design tradeoffs?
18:41:21 <denis_makogon> SlickNik, not quite sure, but will ask
18:41:33 <SlickNik> Clustering is a critical area for Trove, so want to make sure we're doing the right thing here.
18:41:54 <denis_makogon> there's no way to make it worth than it is =)
18:42:19 <amrith> also in mongo and mysql we had people who used it in production and had experience operating in production. I'd like to make sure that this implementation will meet a real need completely.
18:42:23 <denis_makogon> please leave your comments/concerns on spec review, i will take them into account
18:42:54 <vgnbkr> My reading about C* suggested there are different implementation choices, though there are some best practices.
18:43:36 <denis_makogon> vgnbkr, let's talk about it offline
18:43:40 <SlickNik> denis_makogon: I'm also a bit concerned that you're the only primary for both https://review.openstack.org/#/c/122736/ and https://review.openstack.org/#/c/122767/. What's the priority here? Both of these seem like they would take a team of folks some time to implement, so what are the time-frames?
18:44:25 <vgnbkr> For example, I don't know that I agree that topology strategy, replication factor etc won't need to be configurable by the user.  Your spec suggests it would be pre-configured by the provider, unless I missed something.
18:44:33 <denis_makogon> SlickNik, C* clustering K-2, Oracle K-3
18:45:30 <denis_makogon> vgnbkr, replication factor is controlled by keyspace owner, topology strategy by a deployer
18:45:59 <denis_makogon> vgnbkr, in general, correct, it's up to provider
18:46:51 <vgnbkr> If you could clarify in your spec how those would be configured that would be a great help.
18:47:18 <denis_makogon> vgnbkr, ok, could you please leave a comment on the review
18:47:27 <vgnbkr> Sure.
18:47:33 <denis_makogon> vgnbkr, thanks
18:48:02 <denis_makogon> any other thoughts/suggestions?
18:49:22 <amrith> several but I'll put them in review comments.
18:49:27 <amrith> they were smaller
18:49:33 <amrith> and not worth taking everyones time ehre.
18:49:37 <amrith> s/ehre/here/
18:49:40 <denis_makogon> amrith, thanks, a lot =)
18:50:31 <SlickNik> Okay, sounds good.
18:50:45 <denis_makogon> one more thing
18:50:54 <denis_makogon> it's about all spec reviews
18:51:55 <denis_makogon> It's pretty hard to gather feedbacks during meeting. Why can't we do spec reviewing offline and once it's ready ask PTL and core to take a look at it and say if it's ready to go public
18:52:26 <SlickNik> denis_makogon: That's the point of doing the spec reviews as .rst files. When they're ready to go public, they will be +2 and approved.
18:52:53 <SlickNik> go public = ready to be implemented.
18:53:15 <denis_makogon> then we need to spend _lot_ more time doing reviews
18:53:30 <denis_makogon> not just 1 hour per 2-3 weeks
18:53:49 <denis_makogon> that's pretty much all that i wanted to say
18:53:56 <denis_makogon> thanks folks for feedback
18:54:31 <SlickNik> denis_makogon: I'm sure that all of us spend a _LOT_ more time than that doing reviews.
18:55:39 <vgnbkr> I understand what Denis is saying, though I still think the meeting can be useful.  Maybe we need to say the specs must be up by Friday morning so that people have a chance to review before the meeting, then we can concentrate on high level discussion rather than implementation details?
18:56:55 <denis_makogon> let's officially finish the meeting and then we can discuss it
18:57:01 <SlickNik> vgnbkr: The way I see it, it's for getting face-face feedback and for resolving contentious issues.
18:57:09 <SlickNik> #endmeeting