21:11:30 <kgriffs> #startmeeting zaqar
21:11:31 <openstack> Meeting started Mon Aug 18 21:11:30 2014 UTC and is due to finish in 60 minutes.  The chair is kgriffs. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:11:32 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:11:34 <openstack> The meeting name has been set to 'zaqar'
21:11:39 <kgriffs> who's here to chat?
21:11:40 <kgriffs> o/
21:12:49 <vkmc> o/
21:15:56 <kgriffs> beep
21:15:57 <kgriffs> boop
21:16:08 <flwang> o/
21:16:11 <flwang> sorry for the late
21:16:21 <flwang> since I can't control the bus speed :(
21:16:31 <kgriffs> oic
21:16:39 <kgriffs> nice story. :)
21:17:10 <flwang> pls assign me the magic, God, so that I can control the bus speed next time :)
21:17:45 <kgriffs> I think the meeting will be fairly quick today since there are only three of us here.
21:18:11 <flwang> oh
21:18:13 <flwang> ok
21:18:17 <kgriffs> #topic updates on blueprints/bugs
21:18:38 <kgriffs> vkmc: you go first. wanna talk about the work you have been doing this past week?
21:18:50 <vkmc> kgriffs, sure
21:19:18 <vkmc> well I've been working mainly in writting docs
21:19:27 <vkmc> I had some issues with docbook so it took longer than expected
21:20:02 <vkmc> I'm finishing some details on the config reference
21:20:31 <vkmc> I'm making sure that it follows exactly the format of the OpenStack Configuration Reference
21:20:52 <vkmc> so after graduation it can be ported without (many) changes
21:21:08 <kgriffs> ah, OK
21:21:49 <vkmc> I also started working on adding functional tests to shards/pools https://bugs.launchpad.net/marconi/+bug/1246757
21:21:57 <vkmc> but I'm giving extra priority to docs
21:23:01 <vkmc> that's all from me
21:23:20 <flwang> vkmc: sounds good
21:24:02 <kgriffs> ok, sounds good. thanks!
21:24:25 <kgriffs> ok, flwang: any updates from your end?
21:25:37 <flwang> kgriffs: yep, I'm working on the detailed health bp, because the rename make the code a little bit messive
21:26:01 <flwang> I assume I can upload a new patch set tomorrow
21:26:31 <flwang> btw, I just filed another bug, pls take a look if it makes sense https://review.openstack.org/#/c/114944/
21:28:00 <kgriffs> flwang: hmm, why do we need keystoneclient?
21:28:30 <flwang> since keystonemiddleware depends on it
21:28:52 <flwang> but keystonemiddleware won't install keystoneclient automatically
21:29:20 <kgriffs> WAH
21:29:31 <flwang> kgriffs: you will see the other projects are depending the two meanwhile, https://github.com/openstack/ceilometer/blob/master/requirements.txt
21:29:51 <kgriffs> why doesn't the middleware itself list the client as a dep?
21:29:56 <kgriffs> (inquiring minds want to know)
21:31:08 <kgriffs> flwang: looks like it does here: https://github.com/openstack/keystonemiddleware/blob/master/requirements.txt
21:31:41 <flwang> interesting
21:31:56 <flwang> but it won't install it at my pure machine :)
21:32:18 <flwang> I just got a new laptop and yesterday when I setup the dev env, I found the issue
21:32:47 <flwang> after run 'sudo pip install -e .', the keystoneclient won't be instaled
21:32:57 <flwang> maybe it should be a bug of the middleware
21:33:02 <flwang> I will dig it
21:33:21 <kgriffs> kk
21:33:42 <kgriffs> sounds good, thanks!
21:33:47 <kgriffs> anything else?
21:33:48 <vkmc> :o
21:34:22 <flwang> nothing from me
21:34:28 <kgriffs> ok
21:34:32 <kgriffs> so, from my end
21:34:36 <flwang> btw, I will be Paris this time :)
21:34:44 <kgriffs> cool!
21:35:21 <flwang> kgriffs: :) pls move on
21:35:31 <kgriffs> I may not be there (TBD, would be the first summit I'll miss for quite a while) but flaper87|afk will be there for sure
21:36:57 <flwang> kgriffs: oh, noooooooooooooooooo
21:37:21 <flwang> I don't want to see that guy :D
21:37:21 <vkmc> whaaaaaa :(
21:37:52 <kgriffs> well, nothing is set in stone yet... we shall see. :)
21:37:58 <kgriffs> moving on
21:38:05 <kgriffs> from my end, I have a doc patch with some comments that I will be addressing shortly
21:39:06 <kgriffs> I've also been working on a bare-metal benchmark setup. Using a hacked version of https://review.openstack.org/#/c/112236/
21:39:33 <kgriffs> In fact, I have been making a list of ways to further improve the tool, so I may be submitting some patches of my own
21:39:56 <kgriffs> one thing we need to do, btw, is release a version of the client with the new name to pypi
21:40:16 <kgriffs> anyway, the cluster is set up and I've done a few smoke tests
21:40:56 <flwang> how about the graduation?
21:41:24 <kgriffs> from a single load generator I can do about 630 message posts per second, average latency of 1.6 ms/req
21:41:52 <kgriffs> I've seen some "counter out-of-sync" errors in the log that shouldn't be there... need to investigate
21:42:15 <kgriffs> anyway, once redis driver lands I will do a final run and then we can have some numbers to publish. I'll probably need to spin up some more load gen boxes
21:42:27 <kgriffs> flwang: we will discuss that next
21:42:33 <flwang> kgriffs: is there any baseline for zaqar's performance?
21:43:16 <flwang> such as what's the max time to post 1000 messages?
21:44:16 <kgriffs> hmm, that's an interesting way to think about it. So far the benchmark tool just reports throughput and latency averaged over a period of time, during which the tool goes as fast as it can
21:45:01 <kgriffs> I suppose you can extrapolate from reqs/sec to get to 1000
21:45:47 <kgriffs> actually, it does give average ms/req too
21:46:23 <kgriffs> anyway, this brings up a deeper question
21:46:42 <kgriffs> what is our goal, or what do we need to reach to achieve "fast enough to be useful"
21:48:09 <kgriffs> anyway, something to think about. Let's decide on some concrete numbers before the next team mtg
21:48:52 <vkmc> can we get this data from our competitors? and use it as a baseline
21:50:51 <kgriffs> this shows 2500/sec
21:50:51 <kgriffs> http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/throughput.html
21:51:38 <kgriffs> http://colby.id.au/benchmarking-sqs/
21:51:42 <kgriffs> #link http://colby.id.au/benchmarking-sqs/
21:51:47 <kgriffs> #link  http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/throughput.html
21:53:01 <kgriffs> also, Rackspace did some benchmarking last summer
21:53:16 <vkmc> good stuff
21:53:23 <kgriffs> found that requests were rate-limited to 300/sec for each ip (server)
21:53:40 <kgriffs> I will note this on the performance etherpad
21:54:28 <kgriffs> ok
21:54:34 <kgriffs> #topic open discussion
21:55:52 <kgriffs> so, you all notice the monster [all] thread about future of integrated projects?
21:56:26 <vkmc> yeah
21:57:14 <vkmc> there are mixed positions about Zaqar
21:58:19 <kgriffs> there always have been. :p
21:58:42 <kgriffs> of course, you also saw people piling on ceilometer and heat
21:59:01 <kgriffs> i think people are using this as an excuse to try and get rid of projects they personally don't agree with
21:59:11 <kgriffs> but even so
21:59:35 <kgriffs> seems like there was rough agreement not to graduate anything unless it has proven it is "production ready" - either through deployments or extensive testing
21:59:39 <vkmc> yeah of course... but I read really bad, unjustified, comments
21:59:44 <kgriffs> yep
22:00:06 <kgriffs> it is a shame that people can get away with those kinds of comments
22:00:34 <kgriffs> anyway, we better bring this up with flaper87|afk when he is back
22:00:37 <vkmc> yeah
22:00:47 <vkmc> sure :)
22:00:48 <kgriffs> it presents a significant risk for graduating this cycle
22:01:11 <kgriffs> even if we ostensibly check every box in the grad requirements doc
22:01:21 <flwang> kgriffs: yes, it's a bad news
22:02:41 <kgriffs> anyway, let's regroup and chat with more of the team to decide our response.
22:02:51 <kgriffs> anything else before we close?
22:05:38 <flwang> i'm good
22:05:46 <kgriffs> for the record, I don't bear the TC any ill-will. While I sometimes do not agree with the positions of every member, I respect their experience and understand they have a super tough job. We should look for constructive ways to handle concerns without causing divides in the community.
22:06:23 <kgriffs> anyway, keep up the good work folks.
22:06:35 <kgriffs> #endmeeting