16:04:57 <kgriffs> #startmeeting marconi
16:04:58 <openstack> Meeting started Mon Aug 12 16:04:57 2013 UTC and is due to finish in 60 minutes.  The chair is kgriffs. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:04:59 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:05:01 <openstack> The meeting name has been set to 'marconi'
16:05:04 <flaper87> \o/
16:06:28 <flaper87> toc toc ?
16:06:29 <kgriffs> let me pull up the agenda
16:06:30 <flaper87> :D
16:06:37 <kgriffs> any actions from last time?
16:06:53 <flaper87> Yeah, there were a couple of actions w.r.t tests
16:06:54 <kgriffs> I didn't see a summary posted on the wiki...
16:06:57 <flaper87> IIRC
16:07:06 <malini> we had a heavy test discussion & I added it to the bp
16:07:23 <malini> https://blueprints.launchpad.net/marconi/+spec/refactor-system-tests
16:07:36 <cppcabrera> I updated the summary a few days ago. I also noticed it was missing, kgriffs.
16:07:46 <cppcabrera> #link http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-08-02-16.02.html
16:08:03 * kgriffs gives cppcabrera a pop-tart
16:08:10 <cppcabrera> :)
16:08:13 <flaper87> arrgh, that's my fault
16:08:16 <flaper87> sorry guys!
16:08:23 <flaper87> I completely forgot ot update it
16:08:29 <malini> one of the ACTION is ' kgriffs to buy pop-tarts' ;)
16:08:33 <kgriffs> heh
16:08:36 <cppcabrera> No worries. You've got backup, flaper87. ;)
16:08:43 <kgriffs> #topic review actions from last time
16:09:05 <kgriffs> ok, so any items we need to carry forward?
16:09:16 <cppcabrera> Both.
16:09:22 <flaper87> yeah, 1 and 2
16:09:27 <malini> Action # 3 & 4 too
16:09:47 <cppcabrera> Client dev got very little love this go around.
16:09:51 <kgriffs> you mean, everything under topic 3?
16:09:57 <kgriffs> (c,e,f,g)
16:10:06 <kgriffs> oic
16:10:23 <malini> f is complete
16:10:40 <malini> (its in the bp + etherpad)
16:11:01 <kgriffs> 2.b., 2.d,  3.c, 3.e
16:11:07 <kgriffs> all those need to be carried forward?
16:11:31 <flaper87> (actions items are enumerated bellow)
16:11:32 <flaper87> kgriffs: yup
16:11:42 <kgriffs> ok
16:12:01 <kgriffs> #action cppcabrera and flaper87 to turn apiclient-marconi etherpad into bps/bugs
16:12:05 <malini> 2.d is abadoned
16:12:22 <kgriffs> #action  malini to update the etherpad and start working on the re-factor
16:12:31 <kgriffs> #action malini to make functional tests run with tox
16:12:37 <kgriffs> does that look right?
16:12:45 <malini> yes
16:12:47 <flaper87> it does to me
16:12:49 <cppcabrera> yup
16:12:58 <kgriffs> ok, moving right along
16:13:02 <cppcabrera> +1
16:13:02 <kgriffs> #link https://wiki.openstack.org/wiki/Meetings/Marconi#Agenda
16:13:14 <kgriffs> #topic poposal to deprecate Trello
16:13:31 * flaper87 is ready to vote
16:13:42 <kgriffs> Any question/comments before we vote?
16:13:45 <zyuan_> which one we use?
16:14:06 <cppcabrera> Launchpad (LP)
16:14:10 <kgriffs> today we use both Trello and Launchpad, but it had become a real waste of time (IMHO) to keep them in sync
16:14:17 <Sriram> +1
16:14:25 <zyuan_> launchpad has card?
16:14:39 <flaper87> zyuan_: nope, just issues and blueprints
16:14:41 <kgriffs> Trello seems really helpful when planning the next release, but after that becomes hard to maintain since it has to sync with Launchpad.
16:15:14 <kgriffs> also, we are the only project i know of using Trello at the moment. I would rather focus efforts on improving Launchpad and/or helping infra migrate to something like phabricator.
16:15:27 <zyuan_> it's somewhat not obvious to see who is working on which bp/bug
16:15:51 <flaper87> zyuan_: if we use assignee fields correctly, it becomes obvious
16:16:00 <flaper87> we haven't done that because we were focusing on trello
16:16:15 <kgriffs> zyuan_: agreed, a lot of people are not satisfied with Launchpad, but aint nobody got time to keep Trello and LP in sync, and sometimes comments and stuff end up in one place and not the other.
16:16:15 <malini> I like the moving thru column part of trello
16:16:26 <zyuan_> when i say not obvious i mean... the bugs page does not give you such info
16:16:30 <zyuan_> you have to open each
16:16:31 <malini> helps me track all work has tests associated
16:16:40 <kgriffs> how about we try it for a few weeks and see?
16:16:46 <kgriffs> (try not using Trello)
16:16:57 <zyuan_> cards are different. you don't need to open it to know lots of things
16:17:38 <flaper87> We'll have to move to launchpad anyway, that's what OS uses and that's what TC and everyone will use to get status of the project
16:17:43 <kgriffs> an alternative would be to take a day or two and write a "facelift" interface on top of launchpad that makes it more grokkable at a glance
16:18:00 <flaper87> Moving forward with trello wont replace launchpad, anyway
16:18:01 <zyuan_> ha ha ha ha
16:18:13 <flaper87> kgriffs: lol
16:18:20 <Sriram> :P
16:18:33 <flaper87> zyuan_: I agree with you :/
16:18:38 <kgriffs> so, if anyone is looking for a totally sweet project for their next hackathon...
16:18:51 <amitgandhi> im surprised no one has done it yet
16:19:11 <zyuan_> lunchpad - trello integration?
16:19:19 <zyuan_> or cards for launchpad?
16:19:21 <flaper87> to do that, you'd have to mess with launchpad's OpenId and permissions, THAT is not fun :D
16:19:23 <kgriffs> no, just making launchpad prettier
16:19:41 <flaper87> ok, vote ?
16:19:53 <zyuan_> like bugs in many views?
16:20:16 <kgriffs> yeah, better views. And maybe a nicer font or something. :p
16:20:24 <Sriram> ok lets vote.
16:20:46 * cppcabrera ready to vote
16:21:35 <kgriffs> #startvote Should we try freezing Trello and start using Launchpad exclusively? Yes, No
16:21:36 <openstack> Begin voting on: Should we try freezing Trello and start using Launchpad exclusively? Valid vote options are Yes, No.
16:21:37 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
16:21:52 <cppcabrera> #vote Yes
16:21:56 <flaper87> #vote Yes
16:22:00 <amitgandhi> #vote Yes
16:22:01 <Sriram> #vote Yes
16:22:05 <kgriffs> #vote Yes
16:22:09 <megan_w> #vote yes
16:22:32 <zyuan_> no solution in mind; abstain
16:22:39 <kgriffs> noted
16:22:58 <kgriffs> I'll give it 10 seconds in case anyone wants to object
16:23:23 <kgriffs> #endvote
16:23:24 <openstack> Voted on "Should we try freezing Trello and start using Launchpad exclusively?" Results are
16:23:25 <openstack> Yes (6): megan_w, kgriffs, cppcabrera, amitgandhi, Sriram, flaper87
16:24:23 <zyuan_> one thing i like about launchpad is Link to CVE
16:24:24 <kgriffs> #action kgriffs,flaper87, et al. to stop using Trello and focus efforts on getting/keeping Launchpad up to date
16:24:31 <zyuan_> but we never used that yet :)
16:25:10 <zyuan_> hope we don't use that in the future LOOL
16:25:18 <kgriffs> #action cppcabrera to assist in getting cards on Trello ported to Launchpad for those that aren't in sync.
16:25:37 <kgriffs> #topic triage bugs
16:25:51 <kgriffs> #link http://goo.gl/RjZJCw
16:26:07 <cppcabrera> +1
16:26:19 <kgriffs> malini has identified these as the only two critical bugs at the moment. Actually, I think there is one more pending...
16:26:26 <cppcabrera> There is. It's...
16:26:29 * cppcabrera searches for link
16:26:31 <kgriffs> cppcabrera: retry on duplicate key error?
16:26:36 <cppcabrera> #link https://bugs.launchpad.net/marconi/+bug/1211386
16:26:38 <cppcabrera> Yes
16:26:46 <cppcabrera> The regex match fails.
16:26:53 <cppcabrera> Yielding a 503
16:27:09 <kgriffs> has this been independently verified?
16:27:47 <malini> alejandro verified it, rt?
16:27:49 <cppcabrera> It's been noted by oz_akan and malini while load testing. I'm attempting to repro locally.
16:28:04 <flaper87> mhh, I wonder wether the buffer or the s/queue_id/queue_name/ patches could have caused that
16:28:05 <amitgandhi> cppcabrera reported it
16:28:19 <zyuan_> the releasing claim one, i can't repo it with unit test.  i may got something wrong...
16:28:41 <kgriffs> you have to do parallel requests to the same queue
16:28:43 <malini> amitgandhi: cppcabrera reported what oz_akan & me saw in load tests.
16:28:48 <kgriffs> the error is with duplicate markers
16:29:04 <kgriffs> I'd like to mark this one as critical. Any objections?
16:29:07 <Sriram> i'll try to repro it as well.
16:29:14 <malini> amitgandhi: So its not really independent verification
16:29:15 <flaper87> kgriffs: go for it
16:29:56 <kgriffs> ok folks, as soon as we get a reliable repro on this, let's get it fixed
16:30:04 <kgriffs> (pls update the "status")
16:30:09 <flaper87> malini: what version of mongo are you guys using?
16:30:40 <kgriffs> #action malini to verify/repro https://bugs.launchpad.net/marconi/+bug/1211386
16:30:40 <malini> oz_akan_
16:30:40 <cppcabrera> I'm using v2.4.1 (latest is 2.4.5, I believe)
16:31:18 <malini> kgriffs: this bug we found in load test & we have a reliable repro with tsung
16:31:25 <kgriffs> malini: mostly I just want to make sure we can reproduce this on demand; otherwise how can we prove we "fixed" it?
16:31:30 <kgriffs> sweet
16:31:39 <kgriffs> can you update the status accordingly?
16:31:40 <malini> it shows up on all tsung runs
16:31:52 <oz_akan_> 2.4.5
16:32:18 <kgriffs> moving on
16:32:20 <malini> I changed it to confirmed
16:32:23 <kgriffs> is this one confirmed yet?
16:32:24 <kgriffs> https://bugs.launchpad.net/marconi/+bug/1210662
16:32:28 <kgriffs> #link https://bugs.launchpad.net/marconi/+bug/1210662
16:33:14 <amitgandhi> i reported that.  either the spec needs to be corrected, or the code needs to be fixed
16:33:50 <amitgandhi> zyuan is contesting that its a bug and prefers the current approach (which conflicts with the spec)
16:33:52 <kgriffs> I think it's a bug
16:34:00 <zyuan_> i don't think it's bug...
16:34:02 <kgriffs> wait
16:34:09 <kgriffs> how does it conflict with the spec?
16:34:13 <zyuan_> it's spec has a glitch
16:34:25 <zyuan_> spec says age need to be reset
16:34:42 <zyuan_> while current age always reflect the "real" age -- time lived
16:34:45 <kgriffs> isn't that the very definition of renewing a claim?
16:34:47 <flaper87> kgriffs: I tested the regex and it doesn't seem to match, I'll test it a bit further
16:35:01 <kgriffs> does the client need to know/care about the "real age"?
16:35:18 <kgriffs> flaper87: kk, maybe the message changed with a recent Mongo releasse
16:35:30 * kgriffs wishes he didn't have to do a hack that's so fragile
16:35:44 <amitgandhi> #link: https://wiki.openstack.org/wiki/Marconi/specs/api/v1#Update_Claim "The server will reset the age of the claim and apply the new TTL."
16:35:45 <kgriffs> flaper87: we may need to make the regex more flexible
16:35:58 <zyuan_> amitgandhi: yes, i saw that
16:36:04 * flaper87 complained directly with the right guys about those worthless error messages
16:36:36 <kgriffs> so, the simplest fix now is to code to the spec as it now stands
16:36:49 <zyuan_> wait
16:36:50 <zyuan_> wait
16:36:54 <kgriffs> (in my opinion)
16:37:14 <zyuan_> sqlite's age is same as spec
16:37:54 <zyuan_> mongo's is not -_-
16:38:22 <kgriffs> ok, I propose we modify the mongo driver to be consistent with sqlite and the spec
16:38:29 <kgriffs> any objections?
16:38:31 <cppcabrera> +1 kgriffs
16:38:34 <amitgandhi> kgriffs: +`
16:38:36 <amitgandhi> +1
16:38:45 <flaper87> +1
16:38:52 <zyuan_> wait
16:38:52 <Sriram> seems good.
16:39:03 <zyuan_> mongo has some impl issue
16:39:07 <zyuan_> when doing that
16:39:12 <flaper87> feel free to assign that to me! kgriffs
16:39:22 <kgriffs> kk
16:39:29 <zyuan_> the created time is embaded into ObjectId
16:39:46 <kgriffs> flaper87: I think we have an expires field as well, nicht?
16:39:55 <zyuan_> you can either change the ID (i don't you dont); or add an extra field?
16:40:16 <flaper87> kgriffs: right
16:40:23 <cppcabrera> c.e
16:40:28 <cppcabrera> ?
16:40:28 <zyuan_> then... age - expires
16:40:29 <flaper87> we've 2: e and c.e
16:40:29 <kgriffs> kk, I think we are OK then
16:40:34 <flaper87> e is message's expire
16:40:36 <flaper87> c.e is claim's
16:40:53 <zyuan_> ok. if it's possible, add a test
16:40:58 <kgriffs> malini, can you get a repro?
16:41:09 <kgriffs> (and a new test!)
16:41:20 <malini> I cannot add a system test for this https://bugs.launchpad.net/marconi/+bug/1210662
16:41:24 <malini> will be too complex
16:41:30 <malini> We need unit tests for this one
16:41:53 <zyuan_> yes, i mean unit test
16:42:18 <flaper87> kgriffs: ahhh, I knew it, I know what happened
16:42:30 <zyuan_> we don't have to wait for some real time; just post, update, compare age, done
16:42:38 <flaper87> that error was originated by the s/qid/qname/ patch
16:42:39 <kgriffs> ok, let's just make sure there is a test somewhere that first proves it is broken (and pls set to confirmed) then proves it is fixed (when patch is submitted)
16:42:43 <kgriffs> darn
16:42:45 <kgriffs> kk
16:42:52 <flaper87> because the error now has the project in the message
16:42:57 <flaper87> I already created a more general regex
16:43:02 <flaper87> I'll submit a patch soon
16:43:08 <zyuan_> grate
16:43:11 <zyuan_> great
16:43:13 <cppcabrera> +1 flaper87
16:43:14 <zyuan_> -_-
16:43:21 <kgriffs> kk
16:43:28 * cppcabrera watches bugs fixed in real time
16:43:37 <kgriffs> I'll assign those two bugs to Mr. Percoco
16:43:43 <zyuan_> my key stokes alwyas .... errrrahhh
16:43:55 <flaper87> w00000t
16:44:17 <flaper87> kgriffs: question, did this really happen? https://github.com/stackforge/marconi/blob/master/marconi/tests/storage/test_impl_mongodb.py#L52
16:44:25 <flaper87> or that's just a sanity check ?
16:45:55 <kgriffs> flaper87: it must have happened, but I can't remember now for sure - it's been too long
16:46:03 <kgriffs> (and indexes have changed around so much)
16:46:05 <flaper87> kk, no worries
16:46:14 <kgriffs> next bug
16:46:14 <kgriffs> https://bugs.launchpad.net/marconi/+bug/1210633
16:46:40 <kgriffs> I need someone to take this
16:46:42 <cppcabrera> Claim release isn't being honored.
16:46:47 <zyuan_> alej can repo it; i tried with unit test... failed to repo
16:46:50 <zyuan_> (so far)
16:47:01 <cppcabrera> I'll take it.
16:47:04 <kgriffs> sold!
16:47:53 <cppcabrera> I just assigned myself - I'll knock it out after the meeting.
16:47:57 <kgriffs> ok, anything else on the topic of bugs before we move on?
16:48:29 <zyuan_> try sqlite
16:48:35 <zyuan_> if possible
16:48:37 <zyuan_> next time
16:48:46 <cppcabrera> will do, zyuan.
16:48:53 <kgriffs> #topic Triage blueprints
16:48:53 <zyuan_> i guess i implemeted it in a very precise way
16:49:40 <kgriffs> #link https://blueprints.launchpad.net/marconi
16:50:17 <kgriffs> so, go down the line, sorted by series
16:50:23 <kgriffs> perf-testing
16:50:44 <kgriffs> anything we need to update on this (status?)
16:50:44 <flaper87> did you guys take a look at the ones I put here? https://etherpad.openstack.org/queuing-scratch
16:51:00 <kgriffs> ah, that's a good idea
16:51:02 <kgriffs> let's take those first
16:51:20 <kgriffs> https://blueprints.launchpad.net/marconi/+spec/multi-db-support (Should it be deleted as per Monday's call?)
16:51:59 <zyuan_> probably
16:52:02 <flaper87> We said that addressing multi-db wasn't a good plan right now, I think we should close this and re-create it if needed
16:52:07 <kgriffs> based on a recent conversation with 10gen, I think partitioning across several DBs is valuable and would like to see a patch
16:52:13 <cppcabrera> I like the idea of addressing it later.
16:52:28 <kgriffs> oz_akan: thoughts?
16:52:59 <amitgandhi> i kind of want to see the results of the perf testing first
16:53:03 <kgriffs> basically, my take on this is 1. it is valuable, but 2. not sure on the priority
16:53:27 <amitgandhi> i think storage placement takes a higher priority
16:53:40 <flaper87> I think that one is easy to implement
16:53:45 <flaper87> I know oz_akan_ was working on that
16:53:52 <kgriffs> ok, mind if I put it in H3 for now but low priority?
16:54:01 <amitgandhi> +1
16:54:03 <flaper87> +1
16:54:27 <kgriffs> next
16:54:29 <megan_w> seems like the right call
16:54:30 <kgriffs> https://blueprints.launchpad.net/marconi/+spec/use-webob (I think this can be deleted) +1
16:54:44 <kgriffs> and the next one
16:54:51 <kgriffs> I put those in to appease the gods
16:55:23 <zyuan_> ..
16:55:25 <kgriffs> for the greater good, it may be nice to just port marconi over as a science project and see what a difference it makes
16:55:25 <cppcabrera> +1 for deleting pecan/webob bps
16:55:45 <kgriffs> but I'm not super optimistic that it is something that would make it into mainline
16:56:16 <flaper87> lets just close them
16:56:28 <kgriffs> re webob, we can say "we'll take that seriously as soon as swift does"
16:56:39 <flaper87> LOL
16:56:43 <kgriffs> AFAIK we are the only two data APIs so far
16:56:51 <notmyname> :-)
16:56:55 <flaper87> :P
16:56:55 <notmyname> yay data APIs!
16:57:09 <kgriffs> +1
16:57:11 <kgriffs> ok
16:57:16 <Sriram> :D
16:57:26 * notmyname just saw "swift" in here. I don't really have any context
16:57:28 <kgriffs> let me just close those. I think Doug and I have agreed to disagree at this point anyway
16:58:32 <kgriffs> hmm. how do I close a bp?
16:58:39 <zyuan_> lol
16:58:58 <kgriffs> priority "not" ?
16:59:00 <kgriffs> :p
16:59:08 <flaper87> kgriffs: definition
16:59:13 <flaper87> set it obsolete or superseeded
16:59:35 <kgriffs> they need a more accurate term, but oh well
16:59:37 <kgriffs> moving on
16:59:50 <kgriffs> https://blueprints.launchpad.net/marconi/+spec/stream-messages (I think this one can be deleted) +1
16:59:51 <kgriffs> https://blueprints.launchpad.net/marconi/+spec/streaming-serialization (I think this one can be deleted) +1
17:00:03 <kgriffs> I added those in case we wanted to revisit the idea later
17:00:51 <kgriffs> but, if nobody thinks we are going to go back to streaming posted messages, we can forget about it.
17:01:05 <flaper87> I'd close them and re-open them later if needed
17:01:05 <zyuan_> i can remember them
17:01:12 <kgriffs> note that at the moment we don't stream messages when listing them - we serialize to JSON in RAM
17:01:12 <zyuan_> so you can delete them now
17:01:19 <kgriffs> ok
17:01:26 <kgriffs> going once
17:01:30 <kgriffs> going twice
17:01:44 <kgriffs> sold to the guy eating a pop-tart in the back!
17:02:02 <flaper87> LOOOL
17:02:13 <cppcabrera> :P
17:02:36 <kgriffs> https://blueprints.launchpad.net/marconi/+spec/placement-service was superseeded by https://blueprints.launchpad.net/marconi/+spec/storage-placement (Is this for H3 or I-1 ?) (They need to be merged.)
17:02:58 <cppcabrera> If the contents of those two BPs could be merged, it's perfect, IMO.
17:03:15 <kgriffs> ok, let's merge and change the name to "cell" or whatever you want to call it
17:03:16 <flaper87> (the oldest wins) :P
17:03:40 <kgriffs> #action cppcabrera to merge marconi cell bp's
17:04:07 <kgriffs> flaper87: if you think "cell" is going to give people the wrong idea, we are open to suggestions for naming
17:04:16 <kgriffs> moving on
17:04:17 <kgriffs> https://blueprints.launchpad.net/marconi/+spec/routing-api (H-3 or I-1?)
17:04:38 <kgriffs> I think that would be I-1
17:04:44 <kgriffs> what does everyone think?
17:04:57 <kgriffs> I just want to get a solid, baseline release out first
17:05:08 <cppcabrera> I-1, as well as v1.1 or v2.0 of the API
17:05:08 <flaper87> kgriffs: I think cell is going to confuse people, I'd keep storage palcement and leave cell for a higher "cell" concept
17:05:30 <flaper87> I prefer I-1 as well
17:05:33 <kgriffs> flaper87: we've actually been contemplating replacing storage placement with a higher-level cell
17:05:35 <amitgandhi> flaper87: +1 to not using "cell"
17:05:48 <kgriffs> so, routing to webheads + storage as a big black box
17:05:50 <flaper87> kgriffs: mmh, seems you've discussed that service a bit further
17:06:04 <cppcabrera> yup, a bunch flaper87.
17:06:08 <flaper87> when can we sync up on that? (G+?)
17:06:16 <kgriffs> flaper87: yeah, last week, I wanted to discuss as part of this meeting but we are out of time
17:06:28 <kgriffs> let's sync up on all that tomorrow?
17:06:32 <flaper87> kgriffs: +1
17:06:52 <kgriffs> ok, cppcabrera can you schedule somethin'
17:07:24 <kgriffs> #action flaper87, cbbcabrera to schedule another placement service and/or cell architecture discussion
17:07:31 <cppcabrera> Will do.
17:07:34 <flaper87> +1
17:07:38 * cppcabrera is action-packed
17:07:49 <Sriram> woot!:)
17:07:51 <flaper87> cppcabrera: Is the wiki page updated?
17:08:07 <flaper87> I'll read it before our next G+
17:08:19 <cppcabrera> Yeah, I made sure the latest is up there last I worked on it. It should have lots of haproxy and cell ideas on it.
17:08:31 <cppcabrera> *"cell"
17:08:40 <kgriffs> ok folks, we are out of time. we will take care of the placement service chat in breakout, and we can audit the API next Monday.
17:08:53 <kgriffs> #endmeeting