18:00:09 #startmeeting trove-bp-review 18:00:10 Meeting started Mon Oct 20 18:00:09 2014 UTC and is due to finish in 60 minutes. The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:12 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:14 The meeting name has been set to 'trove_bp_review' 18:00:50 Waiting a couple of minutes for folks to trickle in. 18:01:10 Agenda at 18:01:12 #link https://wiki.openstack.org/wiki/Meetings/TroveBPMeeting 18:01:24 ./ 18:01:25 o/ 18:01:29 o/ 18:01:35 trickle, trickle 18:01:54 percolate percolate 18:02:12 o/ 18:03:06 Okay, let's get started. 18:03:12 #topic Handle oslo-incubator module graduation 18:03:22 hello 18:03:24 that's me 18:03:29 here comes a paste-bomb 18:03:31 #link https://review.openstack.org/#/c/123571/ 18:03:46 the proposal in this blueprint initially came from the observation that oslo.concurrency has graduated and now we need to get code from there (not oslo-incubator). 18:03:46 subsequent to writing this, a new proposal from oslo team indicates that they are going to more aggressively delete code from oslo-incubator and therefore I expect that in addition there will have to be a wider set of projects to address this change. 18:03:46 http://markmail.org/message/3w7mcgfdwc4cll7f 18:03:46 Robert's comment re: nova led to some email exchange where we (Robert and I) concluded that we have no dependency on lockutils so I'll update the bp to reflect that. 18:03:49 I have submitted some sample code in the form of four reviews. 18:03:51 https://review.openstack.org/#/c/129292/ 18:03:53 https://review.openstack.org/#/c/129294/ 18:03:55 https://review.openstack.org/#/c/129295/ 18:03:57 https://review.openstack.org/#/c/129378/ 18:03:59 One thing to keep in mind with these merges is that it is ok to change code in trove/openstack/common as far as it relates to repointing libraries to a different location for a graduated oslo library. 18:04:02 A similar thing can be seen in this review: 18:04:06 https://review.openstack.org/#/c/128644/ 18:04:08 I've got most of the changes ready to go, right now I'm trying to figure out why rdjenkins is failing for three of them; the tests work for me locally, not sure why they're failing on gate. Hypothesis is that the guest image doesn't have the dependencies installed. 18:04:12 18:04:26 so, I've pushed a bunch of sample code modeled on other projects 18:04:31 I'm just wrapping up i18n 18:04:35 gettextutils 18:04:46 feedback on the bp 18:04:52 is primarily the purpose of this meeting 18:04:58 o/ 18:05:08 but I felt that feedback on the bp may also depend on some idea of what the implementation looks like 18:05:12 hence the sample code 18:05:17 now I'll shut up and listen 18:06:06 Awesome, thanks for getting started on this Amrith. 18:06:42 We've got quite a bit of code that is being retired from oslo-incubator in favor of graduating oslo projects. 18:06:54 So it will be good to move to them. 18:07:04 correct, but we can't get rid of wsgi stuff in terms of this BP 18:08:34 So one question I have is whether these are BPs and bugs. 18:08:38 or* 18:08:38 o/ 18:08:54 o/ 18:08:55 SlickNik, happy to go either way. 18:09:08 I'd started with a BP for o-i -> o.c 18:09:11 so I stuck with that 18:09:16 other projects opted for bugs 18:09:19 Probably best to figure out how the other projects are handling them, and stay consistent. 18:10:05 If you follow the thread: http://markmail.org/message/3w7mcgfdwc4cll7f 18:10:15 many projects have replied with a bug number and some patch sets 18:10:25 I was going to reply with the patch sets once the BP's are discussed here 18:10:57 one thing that I will point out 18:11:05 is that we (trove) appear to have a lot more of this 18:11:07 than other projects 18:11:21 so, one way or another, we're going to have a lot more files changing 18:11:31 not a lot of lines of change for the ones I'm dealing with 18:11:41 but I know that sgotliv is working on rpc and that's a beast 18:12:13 there are a couple of specific things I'd like people to consider 18:12:27 1. note that in this process, we're going to be updating files in trove/openstack/common 18:12:33 things that were in o-i. 18:12:39 this is to reflect the new dependencies. 18:12:55 Yup, so I think it's probably best to split up the work into bugs (one per each module). 18:12:58 2. i've opted to do this as multiple chkins, one per graduated module 18:13:10 and that way reviewers get to see commonality 18:13:16 I've also made them a long dependency trail 18:13:28 3. finally, I'm having rdjenkins failures 18:13:33 That way it's fairly easy to split up the work into different patchsets (from independent contributors) if need be. 18:13:36 and I'm thinking this is because of the guest agent 18:13:47 not having the required dependencies. 18:14:13 SlickNik, re: multiple contributors; for the work in this BP, I've got most of the changes submitted at this point 18:14:16 there are two more 18:14:26 that I'm still tweaking a bit 18:14:33 but I'll have those up in the next couple of days. 18:14:42 then I get to wait for oslo.concurrency to graduate 18:14:49 and then I can push that change set up as well. 18:15:12 I'd appreciate thoughts on the BP, and especially concurrency on #1 above, and any thoughts on #3 above. 18:15:26 s/concurrency/agreement/ 18:16:41 3 is likely the missing dependencies. So we'll have to work on getting them in the cached image. 18:16:55 ok. just to be clear, int-tests pass locally. 18:17:21 amrith: Yes, but we need to make sure the gate isn't borked. 18:17:52 yup 18:18:02 also, the chain of chkins makes it easy to review 18:18:05 So for #1 - if I read correctly, we will move from spec to bug? 18:18:09 but for commit, do you want me to squash? 18:18:17 #1: yes, I will enter a bug 18:18:32 and sorry 18:18:35 you entered a bug 18:18:36 https://bugs.launchpad.net/trove/+bug/1380789 18:18:42 I've been using that in commits 18:18:49 as partial-bug: #1380789 18:18:53 I'll stick with that. 18:20:15 #1: ok with that bug? 18:20:26 Sure thing. If any of them (like RPC) is a beast that warrants a separate bug, feel free to open a new one for it. 18:20:58 But I'm okay with using a common bug to tackle a bunch of the smaller dependencies. 18:21:04 RPC (I believe) is already being tracked by bp rpc-versioning 18:21:10 see https://review.openstack.org/#/c/94484 18:21:22 https://blueprints.launchpad.net/trove/+spec/rpc-versioning 18:22:29 yes, I guess that's slightly different because moving to oslo.messaging also gives us RPC versioning. 18:22:52 whereas for these other ones, we're just going for equivalence 18:23:24 ok, with https://bugs.launchpad.net/trove/+bug/1380789 we have equivalence 18:23:35 We can punt this bp if that is preferable 18:24:39 Okay sounds good. 18:24:59 Will work offline with you on #3. 18:25:06 undersood. 18:25:34 I think I have what I need to move forward. If people want to start reviewing the code, a list of change sets is available in the etherpad https://etherpad.openstack.org/p/trove-kilo-do-not-use-obsolete-oslo-modules 18:25:51 Sounds good — thanks amrith! 18:25:59 thanks. 18:26:34 #topic Cassandra clustering 18:26:43 mine 18:27:53 #link https://review.openstack.org/#/c/122736/5/specs/kilo/cassandra-cluster.rst,cm 18:28:34 It seems that there are 2 components here: changes to APIs, and a Cassandra implementation of those APIs. I would like to see separate blueprints for API changes so that they can be discussed with thought to how they affect all datastores. 18:29:14 (I mean that genearally, not just in this specific instance) 18:29:36 vgnbkr, those API already exists 18:29:38 i wouldn't mind if they stayed the same blueprint, but having a dedicated section that succinctly explains the api changes would be helpful. 18:29:43 Are there really changes to the API that are needed for this? 18:29:45 ex: isn't returning shard_Id 18:30:01 SlickNik, no 18:30:01 well, not "changes", but differences i suppose is a better term 18:30:03 Or is this an oversight with the BP, and the BP needs to be updated? 18:30:58 There is at small section at the end about making changes to APIs. I thought API changes deserved more than a footnote. 18:32:23 vgnbkr / amcrn: I haven't gotten there yet. +1 about calling those out upfront, since we need to make sure the API changes are backwards compatible. 18:32:24 those API changes are specific to C*, there's enough info about why those changes were made 18:33:16 SlickNik, API changes - is nothing else than another View, but specific for C*, same as amcrn did for Mongo 18:33:18 nothing else 18:33:20 as far as i can tell, the only noteable differences from the mongo spec are: not returning shard_id, using add_node vs. add_shard, returning all node's ip's vs. just query routers as seen in mongodb's case, naming of the instances as "-node-" vs. what's seen in mongodb. 18:33:45 amcrn, correct 18:33:55 wouldn't it make sense for that detail to be in the spec? 18:34:03 ^^ 18:34:16 dougshelley66, what kind of details ? 18:34:45 denis_makogon, the details that you indicated about + what amcrn just stated 18:34:51 s/about/above 18:34:56 denis_makogon: Details calling out what the _exact_ API differences are. 18:35:20 vs. having folks extrapolate them. 18:35:33 overall looks pretty good 18:35:36 SlickNik, we don't have generic API for clustering, we datastore-specific API 18:36:14 and don't see a value-add on comparing C* clustering API changes against Mongo clustering API 18:36:46 denis_makogon: the only point here is that it takes too long to answer the question "what changes in behavior will i see as a user compared to the mongodb clustering implementation?". all of the information is in there, it just has to be teased out. 18:37:33 denis_makogon: We spent a lot of time making sure that the API we're using for clustering is similar for across datastores. 18:37:38 one question which I have about this spec is this. In contrast to mysql and mongodb where we have end-users who can validate the implementation, do we have confidence in this implementation/design path? I'd hate to see us go down this path to find that the cassandra community prefers or recommends some other approach. 18:37:54 So I'm not sure I understand your comment about "datastore-specific" API 18:38:47 SlickNik, current clustering framework allows to build datastore-specific response views, that's all 18:39:59 the design choices we made around replication and clustering were intended to provide a common framework that would work for multiple datastores. So I'm a little concerned that we may be diverging from the framework we started with on MongoDB and if we are, I'd like to make sure we are all in agreement that the divergence is worthwhile, the best way to do things etc., 18:40:10 amrith, i have lots of DBA of C* in production, they know best practices on how to do C* clustering, also there's enough docs with HOWTO (taking into account major DataStax docs) 18:40:42 there's only one way to do clustering for C* 18:40:57 denis_makogon: Awesome, can we add one of these DBAs as a co-owner in the docs, and get him to the BP meeting to talk about the design tradeoffs? 18:41:21 SlickNik, not quite sure, but will ask 18:41:33 Clustering is a critical area for Trove, so want to make sure we're doing the right thing here. 18:41:54 there's no way to make it worth than it is =) 18:42:19 also in mongo and mysql we had people who used it in production and had experience operating in production. I'd like to make sure that this implementation will meet a real need completely. 18:42:23 please leave your comments/concerns on spec review, i will take them into account 18:42:54 My reading about C* suggested there are different implementation choices, though there are some best practices. 18:43:36 vgnbkr, let's talk about it offline 18:43:40 denis_makogon: I'm also a bit concerned that you're the only primary for both https://review.openstack.org/#/c/122736/ and https://review.openstack.org/#/c/122767/. What's the priority here? Both of these seem like they would take a team of folks some time to implement, so what are the time-frames? 18:44:25 For example, I don't know that I agree that topology strategy, replication factor etc won't need to be configurable by the user. Your spec suggests it would be pre-configured by the provider, unless I missed something. 18:44:33 SlickNik, C* clustering K-2, Oracle K-3 18:45:30 vgnbkr, replication factor is controlled by keyspace owner, topology strategy by a deployer 18:45:59 vgnbkr, in general, correct, it's up to provider 18:46:51 If you could clarify in your spec how those would be configured that would be a great help. 18:47:18 vgnbkr, ok, could you please leave a comment on the review 18:47:27 Sure. 18:47:33 vgnbkr, thanks 18:48:02 any other thoughts/suggestions? 18:49:22 several but I'll put them in review comments. 18:49:27 they were smaller 18:49:33 and not worth taking everyones time ehre. 18:49:37 s/ehre/here/ 18:49:40 amrith, thanks, a lot =) 18:50:31 Okay, sounds good. 18:50:45 one more thing 18:50:54 it's about all spec reviews 18:51:55 It's pretty hard to gather feedbacks during meeting. Why can't we do spec reviewing offline and once it's ready ask PTL and core to take a look at it and say if it's ready to go public 18:52:26 denis_makogon: That's the point of doing the spec reviews as .rst files. When they're ready to go public, they will be +2 and approved. 18:52:53 go public = ready to be implemented. 18:53:15 then we need to spend _lot_ more time doing reviews 18:53:30 not just 1 hour per 2-3 weeks 18:53:49 that's pretty much all that i wanted to say 18:53:56 thanks folks for feedback 18:54:31 denis_makogon: I'm sure that all of us spend a _LOT_ more time than that doing reviews. 18:55:39 I understand what Denis is saying, though I still think the meeting can be useful. Maybe we need to say the specs must be up by Friday morning so that people have a chance to review before the meeting, then we can concentrate on high level discussion rather than implementation details? 18:56:55 let's officially finish the meeting and then we can discuss it 18:57:01 vgnbkr: The way I see it, it's for getting face-face feedback and for resolving contentious issues. 18:57:09 #endmeeting