18:01:04 <cp16net> #startmeeting trove
18:01:04 <openstack> Meeting started Wed Jan 13 18:01:04 2016 UTC and is due to finish in 60 minutes.  The chair is cp16net. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:07 <openstack> The meeting name has been set to 'trove'
18:01:09 <vkmc> o/
18:01:15 <cp16net> howdy everyone
18:01:30 <johnma> o/
18:01:58 <imandhan> o/
18:02:00 <amrith> ./
18:02:05 <vkmc> howdy :)
18:02:17 <tosky> hi
18:02:23 <peterstac> o/
18:02:30 <cp16net> #topic Trove pulse update
18:02:38 <cp16net> #link http://bit.ly/1VQyg00
18:03:02 <atomic77> o//~~~
18:03:24 <cp16net> we've had a huge up tick in reviews this past week
18:03:36 <cp16net> and many of the small patches were merged
18:03:56 <cp16net> cleared out the queue quite a bit
18:05:19 <SlickNik> o/
18:05:25 <cp16net> great work all around
18:05:36 <vgnbkr> o/
18:06:12 <cp16net> i'd like to official welcome johnma to the core team!
18:07:10 <johnma> thank you cp16net
18:07:25 <johnma> look forward to working with all of you :)
18:07:31 <cp16net> w00t
18:07:31 <amrith> welcome johnma
18:07:42 <johnma> thanks Amrith :)
18:08:13 <cp16net> i dont have anything else releated to the pulse
18:08:42 <cp16net> moving on
18:08:51 <cp16net> #topic Use of TODO in the source code
18:09:06 <cp16net> i believe peterstac put this on the meeting
18:09:16 <peterstac> yep
18:09:23 <peterstac> I noticed that there are a lot of TODO messages in the source code
18:09:35 <cp16net> true
18:09:35 <peterstac> (quick count of 109)
18:09:51 <dougshelley66> o/
18:09:52 <peterstac> Most of them are old and maybe not relevant anymore
18:10:09 <cp16net> thats very possible
18:10:23 <peterstac> My thought is that there probably aren't too many good reasons to use TODO
18:10:36 <peterstac> when most of them would be better suited to launchpad bugs
18:11:05 <peterstac> (I'm not suggesting we get rid of all the ones that exist at once, but maybe that we try to not add any more)
18:11:05 <atomic77> This is related to the discussion we had about formalizing the tech debt at last midcycle, i'd think
18:11:34 <peterstac> sure (and as such, if we decide to remove them it would be at the start of a cycle)
18:11:54 <peterstac> Just wanted to know what everyone else thought
18:11:57 <cp16net> so my take is that a TODO in a commit would be ok as long as its refering to a bug
18:12:14 <peterstac> (especially about not adding more if not absolutely necessary)
18:12:30 <atomic77> cp16net, what about a 'tech debt' kind of todo?
18:13:07 <cp16net> i'd think most of them are reletated to something reguarding tech debt
18:13:08 <peterstac> The issue I have with 'TODO' is that it's not tracked, and there's no mechanism to discuss anything
18:13:31 <atomic77> i'd think the last thing we'd want is to discourage a dev , at the time of writing, to leave a note in there saying something like "uh, this doesn't work for this case" if we add too much overhead
18:13:33 <cp16net> peterstac: right but if there was a bug related to the todo it would make more sense would it not?
18:14:31 <cp16net> atomic77: i'd agree because then that comment or thought will be forever lost
18:14:34 <peterstac> sure (although even then you could argue that it's not necessary)
18:14:51 <atomic77> the py27 checkers enforce that todos have a user name associated with them i think
18:15:06 <atomic77> i'm sure someone has some scripts which can extract all of these into a nice report that we can pulll once in a while
18:15:07 <cp16net> i'm not sure about that
18:15:18 <amrith> peterstac, are you asking the question about whether to have 'TODO's' in the code or enter bugs, or what we do about fixing the TODO's already in the code?
18:15:31 <cp16net> but i always append my nick to the todo i write
18:16:03 <atomic77> peterstac, I think that they are tracked, in a sense, as long as the username is there
18:16:19 <amrith> ... or something else.
18:16:32 <pmackinn_> o/
18:16:40 <dougshelley66> i believe he is saying that if there are bugs/improvements noted they should be in launchpad
18:16:50 <dougshelley66> orphaning that info in the code is a waste of time
18:17:16 <dougshelley66> some of the TODOs, even though they have userids on them are totally meaningless
18:17:21 <dougshelley66> and now the people are gone
18:17:25 <dougshelley66> we have no idea what they meant
18:17:40 <SlickNik> I personally think TODO's in code are useful — sometimes when I'm reading code, I get context as to why a certain piece of code might not work in a certain case. I suppose you could make the argument that a comment would suffice, but then a TODO is a comment, so that's just semantics.
18:18:15 <pmackinn_> ide like pycharm parse TODO, no?
18:18:28 <cp16net> pmackinn_: yea it does
18:18:35 <cp16net> (i dont use it)
18:18:48 <amrith> thx dougshelley66, I agree. If this were a vote, I'd vote LP. #TODO is just a comment. Requiring a specific IDE seems like a bad idea. I use emacs (real people's IDE).
18:19:02 <pmackinn_> oh man
18:19:29 <cp16net> i think if there is a TODO that relates to a bug it should be docuemnted
18:19:44 <cp16net> (in lp bugs)
18:19:48 <atomic77> TODO is not just a comment if it is consistently applied -- and i think the pep8 rule is enforced
18:20:12 <atomic77> i'm finding 92 TODOs in our code base that have usernames
18:20:27 <atomic77> trying my awk-fu to get a summary count :)
18:20:44 <cp16net> if its used a doc tag for someone to reference who wrote it then it should be used as such
18:20:51 <SlickNik> atomic77: All TODOs are comments, but all comments are not TODOs :)
18:21:15 <atomic77> SlickNik, we're moving into higher philosophy here :)
18:22:31 <cp16net> #vote should TODOs be allowed in new code? Yes, No, Depends
18:22:42 <cp16net> doh...
18:22:54 <SlickNik> So I think I agree with peterstac that we have (quite) a few TODOs in our code that should ideally have representation in LP
18:22:55 * amrith votes doh
18:23:04 <cp16net> #startvote should TODOs be allowed in new code? Yes, No, Depends
18:23:04 <openstack> Begin voting on: should TODOs be allowed in new code? Valid vote options are Yes, No, Depends.
18:23:05 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
18:23:21 <cp16net> #vote Depends
18:23:21 <pmackinn_> #vote Depends
18:23:24 <atomic77> #vote yes
18:23:30 <mvandijk> #vote no
18:23:30 <johnma> depends
18:23:32 <dougshelley66> #vote No
18:23:37 <peterstac> #vote no
18:23:39 <amrith> #vote no
18:23:43 <imandhan> #vote no
18:23:49 * peterstac would rather vote: sparingly
18:23:51 <SlickNik> But instead of discouraging using TODOs, I'd probably encourage creating LP bugs for the case where a TODO represents a bug.
18:23:52 <cp16net> note the cap*
18:24:15 <peterstac> SlickNik +1
18:24:17 <cp16net> peterstac: thats the idea of Depends
18:24:25 <vgnbkr> Is it a valid vote with an "abstain"?
18:24:42 <amrith> yes, just don't #vote
18:24:42 <cp16net> #endvote
18:24:43 <openstack> Voted on "should TODOs be allowed in new code?" Results are
18:24:44 <SlickNik> #vote yes
18:24:44 <openstack> Depends (2): pmackinn_, cp16net
18:24:45 <openstack> Yes (1): atomic77
18:24:46 <openstack> No (5): dougshelley66, amrith, mvandijk, imandhan, peterstac
18:24:52 <vgnbkr> Not voting is not the same as "abstain".
18:24:53 <SlickNik> drat just missed it.
18:25:48 <cp16net> so with a majority voting to not allow todos in new code
18:25:50 <atomic77> well to provide some evidence for one of the arguments, some of the top TODOers:
18:26:00 <atomic77> 13 tim.simpson, 7 amcreynolds :)
18:26:18 <cp16net> heh
18:26:55 <pmackinn_> sorry, unless they have an LP bug? or doesn't matter?
18:26:57 <cp16net> if there is a todo in code i think it should be questioned then on the validity of why its there
18:27:24 <cp16net> i think that summarizes this topic
18:27:55 <cp16net> make sense? agree?
18:28:03 <vkmc> if the TODO represents a RFE, then it's ok as long as there is a LP bug related
18:28:05 <vkmc> IMO
18:28:06 <vgnbkr> Are you proposing an action to examine all existing TODOs?
18:28:14 <vkmc> but certainly that's something to ask on a review
18:28:23 <pmackinn_> so review -1 if it appears?
18:28:44 <vkmc> I'd say 0 if it appears
18:28:59 <cp16net> not saying its a -1 but question why
18:29:24 <vgnbkr> But didn't you just vote to not allow TODOs?
18:29:39 <cp16net> i we will figure this out more as we do it
18:30:01 <atomic77> i think it's reasonable for a reviewer to say "what is this TODO? is there a LP bug associated with it?"
18:30:20 <atomic77> that seems to be the consensus from the vote if i understood correctly
18:30:33 <vkmc> atomic77++
18:30:39 <pmackinn_> that sounds like -1 if LP missing in comment imo
18:31:44 <cp16net> i think we have a concensous on this and we will learn as we go
18:31:58 <cp16net> i'm not saying we need to go through all the existing TODOs now
18:32:07 <cp16net> but if you feel like you have some spare time
18:32:25 <cp16net> that might be useful to figure out if the todo still applies
18:32:42 <cp16net> i'd like to move on due to time and 2 other topics on the agenda
18:32:53 <cp16net> #topic Recent "Spam of Patches"
18:32:57 <mvandijk> The consensus was that no new TODOs should be allowed
18:33:00 <cp16net> amrith:
18:33:07 <amrith> There have been a rash of 'spam patches' of late. Wanted to know what we want to do about them. I have a couple of proposals.
18:33:14 <amrith> But would like to hear what others have to say first.
18:33:24 <amrith> the meeting announcement has details.
18:33:56 <johnma> https://wiki.openstack.org/wiki/Meetings/TroveMeeting
18:34:35 <amrith> hearing nothing
18:34:37 <atomic77> it's a bigger problem than Trove
18:34:49 <cp16net> seems like there have been alot of small patches that we went through that would be concidered SPAM
18:34:49 <amrith> go ahead atomic77
18:35:10 <atomic77> if people are going to abuse the community in order to get ATC passes and such, there's little we can do about it - the os foundation has to do something
18:35:19 <johnma> one thing to consider while we discuss this topic is also how we encourage those first time contributors who might have small/trivial fixes like typo fixing etc
18:35:56 <atomic77> just my opinion though - at best i guess we can be more willing to outright abandon/delete patchsets we can all agree are abusive
18:35:59 <peterstac> atomic77: well, you don't need to push up 30+ changesets to get an ATC pass
18:36:09 <peterstac> so that's probably not the motivation
18:36:32 <atomic77> peterstac, that's probably stats padding and maybe a misunderstanding of what 'benefit' the abuser gets from doing that
18:36:51 <amrith> ok, let me drop my 2c into the mix.
18:36:52 <amrith> As we review these kinds of changes between now and mitaka, could we take a moment to consider whether they add some feature or real value to the product.
18:36:52 <amrith> if they don't, could we deprioritize them for review and merging in favor of features, bug fixes and other things that materially improve Trove from an end user perspective?
18:36:52 <amrith> For example, one of the "small patches" that cp16net mentions is  https://bugs.launchpad.net/bugs/1512207.
18:36:54 <openstack> Launchpad bug 1512207 in tuskar "Fix usage of assertions" [Undecided,In progress] - Assigned to Swapnil Kulkarni (coolsvap)
18:36:54 <amrith> Now there is a feeling that this may have to be considered for reversal because the change isn't all that benign.
18:36:59 <amrith> I don't believe that all patches are like this.
18:37:16 <amrith> but, could we take a hard look at patches before accepting them at this stage.
18:37:28 <amrith> and focus our limited review bandwidth on things we really want in Mitaka.
18:37:48 <amrith> that's all I have to say.
18:37:55 <amrith> it is a proposal for all to consider.
18:38:03 <amrith> I don't believe there is any intended 'call to action'.
18:38:39 <pmackinn_> amrith, so this was cross-project spam from a few (one?) individuals?
18:38:41 <cp16net> amrith: i think bringing this up to the broader community as you have and atomic77 mentions is good step for the OS community as a whole to understand
18:38:46 <pmackinn_> the so-called stylistic change?
18:38:59 <amrith> cp16net, I did bring it up to the broader community
18:39:09 <amrith> and you all probably saw the response(s).
18:39:20 <amrith> and one such change got -2'ed on Trove and for that matter in almost all projects.
18:39:41 <johnma> is there a way of deprioritizing a review besides not looking at it at all or -2 it?
18:39:48 <cp16net> with Trove i think we could do better as well to help mitigate these "useless" patches
18:40:17 <SlickNik> johnma: not that I know of.
18:40:46 <cp16net> putting a -2 is a hard stop on patch and cant be removed unlike a -1
18:41:05 <amrith> johnma, SlickNik +1
18:41:07 <SlickNik> I mean we do de-prioritize trivial fixes once we're close to release milestones.
18:41:18 <amrith> don't know of a way to prioritize reviews
18:41:24 <cp16net> another way of managing the bugs is to refute them on LP
18:41:44 <cp16net> but in LP it doesnt affect the review
18:42:08 <peterstac> cp16net, but refuting a bug on LP should justify a -1 on the changeset
18:42:32 <cp16net> right its just another way
18:42:43 <peterstac> that's a pretty darn good method of deprioritizing a review (in my experience)
18:42:57 <cp16net> LOL
18:43:00 <cp16net> true...
18:43:37 <cp16net> ok anything else regaurding this ?
18:43:48 <amrith> not from me cp16net
18:44:00 <cp16net> ok thanks amrith for bringing it up
18:44:08 <cp16net> #topic HBase (again)
18:44:13 <cp16net> amrith: again
18:44:14 <amrith> How do we want to proceed with HBase? I promised to mail the ML, I did that.  As expected, the feedback from several was that we should consider sahara for hbase full distributed mode
18:44:14 <amrith> Which we plan to do, anyway. But that's not for this phase. This phase is only standalone and pseudo-distributed, both of which are single node. I'm wondering what the feeling is about going ahead with the present phase of work.
18:45:01 <amrith> </paste-bomb>
18:45:13 <tosky> there have been no Sahara meetings so far since the end-of-the-year holidays, so maybe some Sahara people didn't notice it yet
18:45:18 <amrith> code is available for review, as is a spec.
18:45:27 <tosky> I guess it will raised during the next meeting, tomorrow
18:46:09 <cp16net> amrith: i need to catch up on these thread i saw on the ML still
18:47:04 <amrith> cp16net, ok
18:47:10 <cp16net> my first thought is that it makes sense to have some sort of hbase support in trove even if thats not a full fledged delpoyment
18:47:58 <cp16net> because this is getting close to sahara we should be mindful of their position as well
18:48:12 <cp16net> my 2 cents
18:48:36 <cp16net> i will catch up with the ML today though
18:48:53 <amrith> anyone else?
18:49:00 * pmackinn_ responded in ML and spec
18:49:09 <amrith> pmackinn_, yes. thx.
18:49:14 <johnma> so Amrith, I think the spec talks about doing full distributed at a later time. If we are not planning to do this in Trove, I think the spec needs to be updated to reflect that thought
18:49:25 <vkmc> johnma++
18:49:28 <amrith> johnma, it does say that.
18:49:36 <amrith> has always said that.
18:50:24 <amrith> see lines 465 to 468 of rst
18:50:41 <amrith> in conjunction with line 110
18:50:42 <johnma> so are we planning on implementing fully distributed in Trove? I thought that was what the Sahara team was mostly concerned about
18:50:52 <johnma> let me check
18:51:31 <amrith> johnma, your question is open to interpretation. "implementing fully distributed in Trove" could be either all code in Trove, or a callout to Sahara.
18:52:15 <amrith> I expect to support fully distributed in Trove. Whether that is with Trove's own code, or whether it is with Sahara, that's for the O-Release timeframe. And there's much to learn between now and then.
18:52:39 <pmackinn_> fully distributed is a big ask and part of that backend could arguably used for other big data workloads
18:52:58 <amrith> pmackinn_, ++
18:53:14 <pmackinn_> thus, the slippery slope of trove standing up something beyond a "datastore"
18:53:29 <amrith> but so is pxc, mongo, cassandra clustering
18:53:32 <amrith> and we do those today
18:53:34 <atomic77> amrith, part of the confusion may be due to the fact that integration with sahara is listed in the "alternatives" section
18:53:37 <amrith> in some fashion.
18:54:03 <atomic77> so just from my superficial read through, i can understand some of the confusion regarding what is intended
18:54:08 <amrith> atomic77, go on
18:55:54 <pmackinn_> amrith, the reference to flat file-based storage instead of HDFS...i'd be surprised if customers were using HBase that way in prod (imho)
18:55:56 <amrith> atomic77, are you suggesting maybe rewording the alternatives section?
18:56:09 <amrith> pmackinn_, in development a lot of people do just that.
18:56:16 <amrith> and others use single node HDFS
18:56:31 <pmackinn_> "prod"
18:56:37 <atomic77> amrith, well that, along with the Architecture for fully dist.. it's mentioned that hbase will resemble other cluster impls
18:56:38 <amrith> a lot of Trove deployments serve dev/test and non-production use-cases.
18:57:06 <johnma> so AMrith, I think if the spec just talks about standalone adn psuedo-distributed which is what you are targeting fo rthis release anyways, the spec will get more traction
18:57:10 <amrith> atomic77, thanks, will look at the whole spec for those kinds of references.
18:57:30 <atomic77> so I think a reasonable assumption is that the intention is to stand up all the hadoop stuff eventually
18:57:33 <johnma> I think the major confusion is around fully distributed, if I understand things right
18:57:36 <amrith> johnma, ok. I can remove all references to fully distributed.
18:57:53 <pmackinn_> :-)
18:57:56 <amrith> cp16net, time-check. we can continue this on #openstack-trove
18:57:58 <johnma> just my thought Amrith
18:58:06 <amrith> if you have other things.
18:58:17 <amrith> johnma, all good input, thanks.
18:58:26 <amrith> johnma, atomic77, pmackinn_, ... thx
18:58:26 <cp16net> i have a few announcements
18:58:50 <cp16net> #topic announcements
18:58:59 <cp16net> #link http://docs.openstack.org/releases/schedules/mitaka.html
18:59:02 <cp16net> we have mitaka-2 happening next week.
18:59:09 <cp16net> #link https://review.openstack.org/#/q/project:openstack/trove-specs+status:open
18:59:12 <cp16net> Lets look over the specs we have and get the code up.
18:59:21 <cp16net> please push code up for the specs as soon as possible
18:59:25 <cp16net> Thanks!
18:59:48 <cp16net> thats all i have
18:59:55 <amrith> good timing
18:59:57 <cp16net> #endmeeting