19:07:27 <kgriffs> #startmeeting marconi
19:07:28 <openstack> Meeting started Thu May  9 19:07:27 2013 UTC.  The chair is kgriffs. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:07:29 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:07:31 <openstack> The meeting name has been set to 'marconi'
19:07:42 <kgriffs> #topic system testing
19:07:58 <kgriffs> malini: can you give us an update?
19:08:00 <oz_akan> hi all
19:08:07 <malini> sure..
19:08:52 <malini> I am working on getting us set up with a comprehensive set of system tests
19:09:19 <kgriffs> how far along are you?
19:09:45 <malini> I am working on getting the patch updated to get through Jenkins
19:10:21 <malini> The initial set of tests will cover all our functionalities
19:10:22 <kgriffs> OK, I saw that you submitted a second patch - you may want to withdraw it and then rebase and squash that into the first one
19:10:35 <malini> ok
19:10:35 <kgriffs> do those patches cover all functionality already?
19:10:58 <malini> yes, but it can certainly use more enhancement
19:11:12 <kgriffs> OK, so sounds like we'll have a good baseline once that's merged
19:11:13 <malini> & cover a larger set of scenarios.
19:11:26 <malini> Right now it covers all the basic functionalities
19:11:39 <kgriffs> got it
19:12:03 <malini> We also need to have the bugs fixed to turn those tests green
19:12:08 <malini> So we are doing TDD ;)
19:12:16 <kgriffs> he
19:12:17 <kgriffs> heh
19:12:28 <kgriffs> OK, so you got those bugs created.
19:12:30 * kgriffs looks
19:12:33 <malini> yes
19:13:05 <malini> we also need the auth & non ASCII queue names etc. addressed
19:13:05 <kgriffs> are those just in launchpad? I'm not seeing Trello cards at first glance
19:13:18 <malini> oops..I missed the trello part
19:13:21 <malini> I'll add those
19:13:38 <kgriffs> OK. Just add directly to "To Do" with a RED label
19:13:58 <malini> abt the "auth & non ASCII queue names etc.", I havent added any yet.What's our plan on that ?
19:14:30 <malini> auth —> its currently turned off
19:15:01 <kgriffs> when you mean we need them addressed, you mean there are no tests for them?
19:15:27 <malini> we have tests..But Auth is currently turned off in Marconi
19:15:40 <kgriffs> oic
19:16:03 <kgriffs> well, is that blocked by setting up the QA environment?
19:16:34 <malini> probably yes
19:17:04 <malini> will add a card in Trllo to follow up on tht
19:17:36 <kgriffs> OK
19:17:41 <kgriffs> anything else on the topic?
19:17:47 <malini> no
19:18:46 <kgriffs> #action malini will follow up on enabling auth in the system tests
19:19:12 <kgriffs> #action malini will add cards corresponding to bugs
19:19:34 <kgriffs> oz_akan: questions/thoughts re the QA environment?
19:19:51 <kgriffs> #topic QA Environment
19:20:26 <oz_akan> I have been working on scripting the creation of QA environment
19:20:53 <kgriffs> excellent. how soon do you think we'll have that stood up?
19:21:00 <oz_akan> it has be easy to create environments in order to be able to benchmark different sethps
19:21:52 <oz_akan> creating the initial setup takes a bit of time, but I think I am done with that already. I am trying to deliver it by mid next week
19:22:17 <kgriffs> w00t!
19:22:19 <kgriffs> nice work
19:23:09 <kgriffs> OK, I'll mark that as "good progress"
19:23:10 <kgriffs> https://blueprints.launchpad.net/marconi/+spec/qa-cluster
19:23:41 <oz_akan> yes, blueprints wait my attention
19:24:06 <kgriffs> cool. anything else on that topic before we move on?
19:24:08 <oz_akan> I am going to update them along the initial system creation
19:24:31 <kgriffs> ok, so you'll spec out how everything is configured and such?
19:24:37 <oz_akan> in a week or so, we may publish configurations
19:24:42 <oz_akan> yes
19:24:46 <kgriffs> perfect
19:24:48 <kgriffs> sounds like a plan
19:25:33 <kgriffs> #action oz_akan to finish building out the QA environment and document it
19:25:38 <kgriffs> #topic performance test
19:25:58 <kgriffs> malini: are you far enough along to start getting an idea of how many load generators we will need?
19:26:39 <malini> I am still in the process of writing the test scripts
19:26:48 <kgriffs> OK
19:27:09 <malini> I'll have a better of idea of how many load generators we'll need once we can start pointing to the QA env
19:27:10 <kgriffs> are we still evaluating JMeter?
19:27:18 <malini> I hope to have that by next week
19:27:22 <kgriffs> also, what did you think about that python one?
19:27:38 <kgriffs> OK,
19:27:42 <kgriffs> let me ad a depend
19:27:50 <malini> I am not looking at Jmeter now
19:28:05 <malini> tsung s can generate more load per machine
19:28:20 <malini> So I am still sticking with tsung
19:28:40 <malini> I am alos looking at the tempest stress tests to see if we can use that
19:28:54 <malini> But so far, its tsung
19:29:04 <kgriffs> ok, groovy
19:29:38 <kgriffs> so, during this next week you will be fleshing out the perf tests?
19:30:07 <malini> yes
19:30:26 <malini> & also get the baseline system tests into our repo
19:30:37 <kgriffs> #action malini to get baseline system tests merged
19:30:58 <kgriffs> #action malini will complete baseline performance tests this week
19:31:11 <kgriffs> ok, anything else on that topic?
19:32:27 <kgriffs> #topic triage blueprints for H1, H2
19:32:52 <kgriffs> so, next I'd like to take a few minutes to do a rough prioritization of the existing blueprints and add any that are missing
19:33:11 <kgriffs> #info https://blueprints.launchpad.net/marconi
19:34:08 <kgriffs> So, let's go down the list. I've sorted my view by the current priority order
19:34:18 <kgriffs> #topic bp/config-module
19:34:52 <kgriffs> should be self-explanatory, and I think stays at essential.
19:35:00 <kgriffs> questions/comments?
19:35:37 <oz_akan> I don't
19:35:50 <kgriffs> #topic bp/input-validation
19:36:12 <kgriffs> since this is security 101, I think it is also essential
19:36:48 <kgriffs> my plan was to create a proxy storage driver that does this
19:37:13 <kgriffs> that way, no matter what transport or storage driver is used, you get the same validation
19:38:28 <kgriffs> I'll just keep going - stop me if you have a question/comment/concern
19:38:37 <kgriffs> #topic bp/storage-base
19:39:07 <kgriffs> this guy is pretty much done, we just need a final sanity check
19:39:46 <kgriffs> same with reference storage
19:40:05 <oz_akan> does encrypting queues fall under this?
19:40:06 <kgriffs> in my mind, I'm thinking that "Essential" means "the bare minimum to do a demo"
19:40:42 <kgriffs> oz_akan: I had that under isolate-tenant-data
19:41:01 <oz_akan> ok
19:41:10 <kgriffs> we'll get there in a minute
19:41:33 <kgriffs> so, for my "demo" criteria, I guess I should demote system-tests to "High"
19:41:44 <malini> ok
19:41:58 <kgriffs> High means we still plan to do it for our first release
19:42:59 <kgriffs> #topic bp/system-tests
19:43:05 <kgriffs> so, we touched on this earlier
19:43:19 <kgriffs> I am going to set priority to High
19:43:37 <kgriffs> malini: once it get's merged you can update the status on that bp
19:43:44 <malini> I will
19:44:11 <kgriffs> #topic bp/transport-*
19:44:28 <kgriffs> so, these guys are mostly done. Just need to fix a few bugs and handle some edge cases.
19:44:44 <kgriffs> they stay at high since they act as our "Reference" transport driver
19:44:51 <kgriffs> same goes for the API def
19:45:05 <kgriffs> speaking of which, we need to lock down the API soon
19:45:25 <kgriffs> so anyone with a concern please let me or flaper87 know ASAP!
19:46:10 <kgriffs> I'm sure there will be some tweaking over the next 6 weeks or so, but we need to stabilize the API.
19:46:25 <kgriffs> #topic message-pagination
19:46:42 <kgriffs> so, this one has a basic implementation
19:46:57 <kgriffs> but it has a few issues; race conditions not handled, FIFO not guaranteed for a single producer
19:47:07 <kgriffs> (let me update the status)
19:47:47 <kgriffs> Is everyone OK with the proposal to guarantee FIFO for a single producer to a given queue?
19:48:14 <oz_akan> #agree
19:48:31 <kgriffs> #agreed guarantee FIFO only for a single producer to a single queue
19:49:13 <kgriffs> the second part of this is making client libs a little smarter so they can detect a skipped message and ask for it before returning the final set to the caller.
19:49:45 <kgriffs> (this possibility is a result of the way FIFO will be implemented - has to use a side counter, which introduces a race condition for observer clients)
19:50:14 <kgriffs> Anyone thought of a red flag in doing this?
19:51:24 <kgriffs> #agreed deal with side-counter race condition in the client
19:51:29 <kgriffs> https://etherpad.openstack.org/marconi-fifo
19:52:20 <kgriffs> error codes can go to "high"
19:52:37 <kgriffs> #topic error-codes
19:52:38 <kgriffs> https://blueprints.launchpad.net/marconi/+spec/error-codes
19:52:43 <kgriffs> actually, maybe make those a medium
19:53:04 <kgriffs> so, these are internal codes, like "M742
19:53:05 <kgriffs> "
19:53:16 <kgriffs> language-independent, and transport-independent
19:53:37 <kgriffs> it was a suggestion brought up at the summit, and I thought it was a good idea
19:54:23 <kgriffs> but, not a huge priority right now, so I'd say medium, H2
19:54:27 <kgriffs> sound good?
19:54:53 <kgriffs> Moving along...
19:54:59 <kgriffs> #topic bp/storage-placement
19:55:41 <kgriffs> flaper87 suggested putting priority on this at medium. We definitely need to get it done for launch, though
19:55:44 <kgriffs> thoughts?
19:57:51 <oz_akan> I agree this needs to be done quick
19:58:16 <oz_akan> it is one of the building blocks for scaling
19:58:31 <kgriffs> should it be for H1 (may 30) or is it ok for H2?
19:59:53 <oz_akan> I think we need to have it day one, it might be harder to implement later on
20:00:05 <oz_akan> I am not sure where it falls though in terms of timing
20:00:55 <kgriffs> OK. We've got a ton of stuff happening the next couple of weeks, so let's put this on the frontend of H2
20:01:05 <kgriffs> BTW - looks like we are out of time
20:01:50 <kgriffs> I will take a stab at prioritizing the rest of the stuff. Look things over and email or post your comments in #openstack-marconi
20:01:57 <kgriffs> any last-minute items?
20:02:04 <oz_akan> thanks kgriffs
20:02:44 <kgriffs> likewise
20:02:50 <kgriffs> cheers everyone
20:03:13 <kgriffs> #action kgriffs to finish prioritizing blueprints, will discuss next time
20:03:20 <kgriffs> #endmeeting