15:00:27 #startmeeting marconi 15:00:28 Meeting started Tue Feb 4 15:00:27 2014 UTC and is due to finish in 60 minutes. The chair is kgriffs. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:29 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:30 #topic roll call 15:00:32 The meeting name has been set to 'marconi' 15:00:35 o/ 15:00:37 o/ 15:00:50 o/ 15:00:50 anyone else here for marconi awesomeness 15:01:02 o/ 15:01:05 w00t 15:01:09 o/ 15:01:19 o/ 15:01:28 excellent 15:01:29 hurray - looking lively! 15:01:36 * kgriffs hands out teh pop-tartz 15:01:51 * flaper87 eats his and steals cpallares's 15:01:53 #link https://wiki.openstack.org/wiki/Meetings/Marconi#Agenda 15:01:55 good thing, havent had breakfast yet 15:01:56 :o 15:02:04 lol 15:02:15 * cpallares pouts 15:02:15 * alcabrera clones his poptarts, stashes for later 15:02:17 #topic Marconi SqlAlchemy (alcabrera) 15:02:33 \o/ 15:02:40 :) 15:02:42 Cool - I'm surprised we're launching with the sqlalchemy topic. :P 15:02:43 #link https://etherpad.openstack.org/p/marconi-sqlalchemy 15:02:44 go alcabrera, go alcabrera, go alcabrera 15:03:03 yes, that link. :D 15:03:04 Anyway 15:03:34 Here's the situation - I'm hoping to drive our effort to get us sqlalchemy support by the end of this month (really, Feb. 20ish) 15:03:47 That link above describes a proposed strategy to meet that 15:03:48 * kgriffs wants that 15:04:14 alcabrera: is there anything specific you wanted feedback on? 15:04:19 It's a tight schedule, a bit aggressive, but yes - sqlalchemy support is *really* desirable for ice house. 15:04:21 hmmm 15:04:27 hi 15:04:32 o/ 15:04:38 two things I can think of off the top of my head - 15:04:45 1) general thoughts on timeline 15:04:49 2) testing strategy 15:04:51 * kgriffs hands teh oz_akan_ and teh malini teh pot-tarts 15:04:53 / 15:04:55 o/ 15:04:58 yayy 15:04:58 o/ 15:05:23 alcabrera: do u want to discuss test strategy now or after the meeting? 15:05:24 Hey, why didn't I get a pop-tart when I showed up? 15:05:30 I got some feedback fro Yeela, the queue controller is almost done, she sent it to me. I'll ask here to push it to gerrit and mark it as WIP 15:05:36 so we can all provide feedback 15:05:39 alcabrera: I have been thinking abt how to test it 15:05:43 ammets, flaper87 probably ate it 15:05:44 * kgriffs passes the box to ametts 15:05:44 flaper87: thanks! 15:05:49 * alcabrera listens to malini 15:06:01 the issue is that the tests for the queue controller depend on the message controller to 15:06:09 I *think* 15:06:20 so, some work on the message controller is needed too 15:06:39 I am thinking of adding a gate/check job that will be functional-tests-devstack-mongodb 15:06:40 * flaper87 STFU 15:06:42 hmmm, flaper87 - I think that's right. The implementation order might need to be swapped so that the msg ctrl comes before the queue ctrl - or something like that 15:06:58 I can work on that asap 15:07:10 and we can isolate the queue_controller tests from the message controller 15:07:16 +1 15:07:18 that will help with getting the queues patch in 15:07:22 Once we have that in place , we can use the same template to do a functional-tests-devstack-postgres, functional-tests-devstack-mysql or whatever backend we need to 15:07:27 and then work on the message controller 15:07:46 I think the due date for the queue controller should be next monday! 15:07:52 malini: +1 - could you add notes in that regard to the etherpad? 15:07:59 alcabrera: sure 15:08:11 This will be in addition & separate from the tempest tests 15:08:13 alcabrera: so you and flaper87 have worked out which parts you are implementing? 15:08:24 not yet, kgriffs. :) 15:08:36 so, now is a good a time as any 15:08:49 flaper87: I'll be tackling the catalogue and shards controller this week 15:08:52 I think it would be nice if alcabrera could thake the shard part 15:08:55 LOL 15:08:57 :D 15:09:01 alcabrera: Can you post the link again, plz? 15:09:06 sure thing, malini 15:09:08 https://etherpad.openstack.org/p/marconi-sqlalchemy 15:09:10 alcabrera: yeah, I'll make sure the rest of the work is complete and lands 15:09:15 malini: +1 for the gate thing 15:09:18 ok, let's rock this out 15:09:18 sweet 15:09:24 so, let's distribute #actions 15:09:26 we need everyone to keep a close eye on the review queue 15:09:33 (to keep things truckin' along) 15:09:51 anything else on that topic? 15:10:02 I'll be tracking the work there, I'm kind of burried in stuff this week 15:10:04 I'm pretty satisfied for now. I think we're on track. 15:10:20 if there are reviews to do, feel free to ping me. I'm not monitoring gerrit much this week 15:10:26 #action alcabrera to implement shards controller (sqlalchemy) 15:10:33 #action alcabrera to implement catalogue controller (sqlalchemy) 15:10:33 I will try to, though! 15:10:36 flaper87: will you be able to work on the SQL driver this week? 15:10:41 and sorry about that! 15:10:51 kgriffs: Yeela is working on the queue controller 15:10:55 flaper87: no worries. :) 15:11:08 kgriffs: I'll start working on the message controller asap 15:11:09 ok, it's just that I am getting *really* nervous 15:11:20 we are down to the wire 15:11:28 16 days to go 15:11:51 #action flaper87, alcabrera, and malini to save the world by implementing the SQL driver in record time 15:11:54 yeah, I'll make that happen 15:11:56 hurray 15:11:59 :D 15:12:02 ;) 15:12:06 me too... 15:12:12 of course! 15:12:19 #info sqlalchemy effort prioritized - targeting Feb. 20th 15:12:21 I will really really focus on graduation starting now 15:12:25 our code would totally suck without you keeping us honest. srsly. 15:12:55 #topic pending patches 15:13:19 just wanted to take a moment to give people an opportunity to highlight any of these 15:13:22 https://review.openstack.org/#/q/status:open+project:openstack/marconi,n,z 15:13:23 there's several close patches in the queue 15:13:39 yep 15:13:43 #link https://review.openstack.org/#/c/70202/ (sqlalchemy config ++) 15:13:53 #link https://review.openstack.org/#/c/68267/ (lots of bug fixes) 15:14:00 so, last week I bloodied myself kicking against the wsgiref stuff 15:14:09 oh yes - the evils of wsgiref 15:14:26 after the team bandaged my wounds, we came up with the idea for that first patch there 15:14:53 there are a couple housekeeping patches with a -1 15:14:57 minor stuff 15:14:59 I'm very +2 for the idea. 15:15:05 let's get those updated and merged ASAP 15:15:06 re: wsgiref changes 15:15:13 * kgriffs gives alcabrera a big hug 15:15:20 :D 15:15:23 * kgriffs wipes a tear from his eye 15:15:23 * alcabrera returns hug 15:15:38 * ametts is getting nauseated 15:15:44 * kgriffs was traumatized 15:16:01 ametts: LOL 15:16:10 lol 15:16:13 flaper87: can you take a look at my patch there? 15:16:24 so, quick question: 15:16:25 kgriffs: I'm doing that as we speak 15:16:27 :D 15:16:28 "The style guideline says if there's no code the file shouldn’t contain any license header nor comments. I double checked." 15:16:29 it will make everyone's lives soooo much better. I promise. 15:16:43 oic 15:16:44 tht is just a crazy guideline 15:16:48 that is kinda silly 15:16:50 but whatever 15:16:56 I can remove my -1 15:16:59 not a big deal 15:17:02 cool 15:17:08 that's two patches we can merge now 15:17:10 ok, anything else on the topic? 15:17:18 hmmm 15:17:37 nah, I'm good. We can get back to this in #marconi later 15:18:18 #topic i-3 15:18:40 bps and bugs~ 15:18:49 this is pretty much graduation stuff, so I am going to combine those agenda items 15:18:50 #link https://launchpad.net/marconi/+milestone/icehouse-3 15:19:18 so, I deprioritized a few things there 15:19:26 and made sure all bps are assigned 15:19:37 let's go down these in order 15:19:40 balajiiyer: docs 15:20:06 Im working with Catherine on docs. she is going to start with user docs this week. 15:20:16 excellent 15:20:20 sweet 15:20:32 did you end up deciding to put those in the marconi repo then? 15:20:44 I have to talk to oz_akan_ mpanetta and get a draft installation guide, and give it to Catherine to make it pretty 15:20:54 yes, it will go to marconi repo 15:21:04 balajiiyer: cool. Let them do a very rough draft and let Catherine work her magic 15:21:08 balajiiyer: kk 15:21:18 #info balajiiyer making good progress on docs initiative 15:21:31 flaper87: we are going to try to mirror the manuals repo so it will be easy to move over once we are integrated 15:21:43 there are a few things Catherine & I will have to figure out, but I will post those questions once we cross the bridge 15:21:46 that sounds good to me 15:21:48 balajiiyer: thanks! 15:21:56 balajiiyer: +1 for the effort there! 15:21:56 balajiiyer: pecan 15:21:59 we really need that 15:22:32 Been working on pecan the last couple of days, bloodied (going to borrow your word, Kurt) 15:22:56 * kgriffs hands balajiiyer some bandages and salve 15:23:05 * amitgandhi the walls in our office are red with the bloodied work of balajiiyer 15:23:06 pecan shud be called macadamia - hard to crack 15:23:15 balajiiyer: do you think we will have a POC by the end of the week? 15:23:16 haha 15:23:17 alcabrera been helping me out, so will be successful. Lots of architectural changes reqd to get marconi going with pecan 15:23:41 sounds like pecan doesn't like to play nice with others 15:23:45 global request/response objects are a bit strange 15:23:48 malini: ++ 15:23:49 i knew a kid like that once 15:23:52 :D 15:23:57 'from pecan import request, response' 15:24:00 e.g. 15:24:21 ok, how soon before we can run a significant benchmark? 15:24:25 kgriffs: I have been updating the notes page, so take a look at that 15:24:31 balajiiyer: kewl 15:24:34 which reminds me 15:24:44 Im targetting EOW, lets see 15:24:50 I need to go write some marconi docs so we have something to show on that line item. :p 15:25:19 #info balajiiyer aiming to support enough pecan in marconi to benchmark by end of week 15:25:28 tbh, lots of people say when they use falcon the README is almost good enough, since Falcon is so easy to learn 15:25:37 15:25:39 :P 15:25:51 balajiiyer: thanks man, I REAAAAALLLLLYYYYY appreciate this! 15:26:00 :D 15:26:03 yeah, thanks balajiiyer! 15:26:04 kgriffs: yw 15:26:12 * kgriffs gives balajiiyer a high-five 15:26:16 ok 15:26:17 next 15:26:24 SQL storage driver 15:26:30 haha 15:26:32 yes, that 15:26:46 we already talke about that 15:26:46 next 15:26:46 malini: tempest 15:26:59 I got one patch merged y'day 15:27:15 have another one I am working on now https://review.openstack.org/#/c/65140/ 15:27:24 sweet 15:27:33 I have been neglecting these patches the last couple of weeks 15:27:41 HAve been crazy busy with other stuff 15:27:44 kk 15:27:58 But I will get back on track & get the outstanding patches merged this week 15:28:00 malini: is that other stuff winding down? 15:28:07 kgriffs: yes 15:28:10 gtk 15:28:13 very yes 15:28:27 * alcabrera gives a sigh of relief 15:28:31 heat template merged 15:28:38 flaper87: can you sanity-check that template? 15:28:53 kgriffs: will do. I just clicked on it :D 15:28:57 kk 15:29:15 #action flaper87 to sanity-check heat template, see if we are all done there 15:29:32 alcabrera: Make Basic Marconi Docs Available 15:29:37 is this superseded? 15:29:48 yes, balajiiyer is tackling the proper effort 15:30:09 kk 15:30:13 status updated 15:30:26 thanks 15:30:27 kgriffs: api v1.1 15:30:40 i am going to start working on that today 15:31:08 first step is to create a v1.0 clone i guess, since we wanted to make the schema work run in parallel 15:31:29 flaper87: did you have any thoughts on that? 15:31:40 kgriffs: I actually wanted to talk about that 15:31:51 * kgriffs picks up the floor and hands it to flaper87 15:32:05 I guess our best shot now is to move the wsgi contents under a v1 package 15:32:20 and copy that one to a v11 or something like that 15:32:24 well, 11 is a major 15:32:27 :P 15:32:32 ok. Let's do the silly copy-paste for now 15:32:49 CP works for me 15:32:53 we will clean up later assuming the fancy API abstraction works out 15:32:55 v1_1, I suppose 15:33:01 re: naming 15:33:23 not sure what the best thing to do there is, TBH. But yeah, I'd go with v1_1 for now 15:33:40 I know other projects do copy-paste, but that doesn't always mean it's right (just because another project is doing it) -- more like, we can get away with it if we need to 15:33:42 ugly but sounds like it'll work 15:33:45 kk 15:34:08 so next week I'd like to review the 1.1 spec 15:34:19 please everyone do their homework and come prepared to discuss the finer points 15:34:29 #info next week - reviewing v1.1 spec 15:34:33 #link https://wiki.openstack.org/wiki/Marconi/specs/api/v1.1 15:35:19 fwiw, pretty much the entire delta between 1.0 and 1.1 was driven by user/sdk developer feedback 15:35:36 so if you don't like it, TOUGH 15:35:41 * kgriffs is kidding 15:35:52 cpallares: Cross Transport API Spec 15:35:54 maybe worth having a few of those folks in that meeting too 15:36:01 amitgandhi: good idea! 15:36:28 #action kgriffs to invite sdk devs to v1.1 api review 15:36:52 cpallares: status on that bp? 15:36:58 kgriffs: speaking of sdk, did you need a note about fog being broken due to a change lower case headers? 15:37:09 need /see 15:37:10 yeah, working on tests for the api/storage communication 15:37:30 is it still "slow progress" or "good progress"? 15:37:32 ...Still. I got a little stuck on the testing but flaper87 helped me out 15:37:37 slow progress 15:37:39 kk 15:38:06 #info cpallares reports slow progress on cross transport API spec 15:38:10 thanks for working on that. glad to see it making some progress 15:38:23 thanks, cpallares! :) 15:38:24 balajiiyer: mmm. Parson my ignorance, but what is "fog"? 15:38:29 s/parson/pardon 15:38:43 Ruby cloud sdk, kinda like libcloud, kgriffs 15:38:47 oic 15:38:49 https://github.com/fog/fog 15:38:51 errr 15:38:54 #link https://github.com/fog/fog 15:39:24 um, so, clients are supposed to not care about header case 15:39:38 * kgriffs hides behind HTTPbis 15:39:39 kgriffs: http://stackoverflow.com/questions/21489139/exception-claiming-messages-from-cloud-queues-in-fog/21489263#21489263 15:39:43 +1 15:40:07 oh, glad to know they fixed it 15:40:09 hmm 15:40:17 after discussing with the fog sdk devs, they traced the case-sense issue back to one of their deps - excon, which is kind of like the requests module of ruby 15:40:20 I didn't realize that change went out in Cloud Queues 15:40:23 Cool... people are asking about us on Stack Overflow... 15:40:32 we are famous!! 15:40:36 w00t 15:40:42 alcabrera: gtk 15:40:42 There is a tag for 'Rackspace' in SO 15:41:23 malini: Basic Security Tests 15:41:34 as much as I hate to say this, I think this will have to slip into juno 15:41:36 havent started yet 15:41:42 I think so too 15:41:49 but don't forget about it! 15:41:51 :) 15:42:02 I will be focussing on tempest +sqlalchemy till feb 20 15:42:07 agreed - we're too full to do more than what we're doing now 15:42:12 Will get to it after tht 15:42:16 wont forget :) 15:42:53 #info security testing effort re-targetted for Juno 15:42:57 flaper87: Make TTL optional 15:43:07 so, I think that is basically part of the v1.1 API 15:43:18 I haven't started working on that but I think it's important 15:43:21 kgriffs: yeah 15:43:23 agreed 15:43:31 "The ttl field is no longer required when posting messages and creating claims; if omitted, it will default to the max TTL allowed by your cloud operator." 15:43:42 so, maybe I can just nuke this extra bp? 15:43:48 +1 for nuking 15:43:58 well, I wouldn't 15:44:06 I mean, it's part of v1.1 but it's a different thing 15:44:18 ah wait 15:44:24 I understood what you mean, mmh 15:44:34 so your thinking about making all those changes in a single bp? 15:44:34 well, is it really big enough to be it's own bp? 15:44:40 flaper87: mmm 15:44:44 good point 15:44:49 I'd prefer breaking those down into several bps 15:44:53 we can break it down and make them dep 15:44:59 and with that we can split the job 15:45:04 kk 15:45:47 I'll break out the other things too 15:45:54 +1 for breaking down v1.1 into bps - parallel development is good 15:46:03 #kgriffs to break out bps for v1.1 work 15:46:33 LOL 15:46:33 here's another v1.1 thing 15:46:34 #action kgriffs to break out bps for v1.1 work 15:46:34 https://blueprints.launchpad.net/marconi/+spec/detailed-health 15:46:40 #kgriffs is not a thing 15:46:42 lol 15:46:43 lol 15:46:43 you're cheating 15:46:46 :D 15:46:46 * kgriffs facepalm 15:47:33 flwang: && 15:47:36 flwang: ^^ 15:47:37 I mean 15:47:42 kgriffs: I have collected the KPI, 1) CPU utilization 2) # of active shards 3) # of queues/shard 4) response time / reqs/sec 15:47:45 * kgriffs wonders what is wrong with his keyboard 15:47:59 does that make sense for you for the first stage? 15:48:16 kgriffs: I'm not sure if all of them can be collected easily though 15:48:43 flwang: sure. So let me create a v1_1 transport and then you can start plugging it in. 15:48:50 mmm 15:48:53 kgriffs: great 15:48:55 flwang: let's talk about it in #openstack-marconi. I'll be glad to discuss the difficulty of each of those KPIs 15:49:02 flwang: start with just one metric that is easy to collect 15:49:15 that way you get all the plumbing done 15:49:17 kgriffs: sure, that's what I want to do 15:49:23 kk 15:49:36 then we can add others and see how many we get in before i-3 closes. 15:49:42 +1 15:49:46 rock on 15:49:51 flwang: thanks! 15:49:53 kgriffs: yep 15:50:03 kgriffs: sure, glad to contribute 15:50:21 malini: benchmark 15:50:48 malini: this would be helpful to add to the operator manual so people can make an educated decision 15:50:48 but 15:50:54 I'm thinking Juno 15:51:03 +1 for juno :) 15:51:07 #link https://blueprints.launchpad.net/marconi/+spec/benchmark 15:51:22 I also want to try out the stress test framework in tempest for this 15:51:40 Adding tsung configs is really easy & has worked out well for RAX implementation 15:51:50 cool 15:52:04 But doing it in for gate/jenkins will require the infra team to add tsung etc 15:52:09 #info flwang to continue work on more detailed health info/KPIs 15:52:12 which will raise a lot of questions 15:52:18 #info benchmark effort retargetted for Juno 15:52:21 alcabrera: Implement Sharding Controllers for sqlalchemy 15:52:30 I can make that dep on SQL Storage Drive, nicht? 15:52:33 started 15:52:35 and yes. :) 15:52:51 I have one patch in that is important for getting the rest going 15:53:01 done 15:53:02 #link https://review.openstack.org/#/c/70202/ 15:53:18 ah yes, thanks for the reminder 15:53:26 I started reviewing that a few days ago and got sidetracked 15:53:31 flaper87: ^^^ 15:53:34 no worries. 15:53:36 let's get that reviewed ASAP 15:53:50 * flaper87 clicks 15:53:51 #info alcabrera started shards controller for sql driver 15:53:59 oh yeah, that one! 15:54:21 sorry for not having reviewed it yet! I'll get to it asap! 15:54:26 thanks! 15:55:02 ok 15:55:05 #topic bugs 15:55:11 holdup 15:55:15 horizon update? 15:55:25 jay-atl: has been making progress 15:55:33 oh 15:55:36 wat? 15:55:38 ohhh 15:55:45 I didn't realize that was being worked on 15:55:50 bonus points! 15:55:51 * flaper87 neither 15:55:53 yeah yesterday I got devstack with Marconi up... and pointed my horizon server to that. 15:55:56 thanks 15:56:00 sweet 15:56:03 :D 15:56:05 jay-atl:++ 15:56:14 this guy? 15:56:15 https://blueprints.launchpad.net/marconi/+spec/marconi-horizon-integration 15:56:15 #info jay-atl launching horizon effort for marconi - progress: started 15:56:24 It's slow going, but today I hope to actually be populating it with data... 15:56:26 * amitgandhi to those that dont know jay he is one of our UX devs in ATL 15:56:40 kgriffs: yes, thats the bp Jay's been working on 15:56:50 ah, cool 15:56:54 * jaypipes really must remove "jay" as an IRC notification pinger... 15:56:55 let me move that into i-3 then 15:57:07 lol 15:57:10 :P 15:57:16 jay jay jay 15:57:21 LOL 15:58:03 jay-atl do you have a launchpad account? 15:58:10 I can assign the bp to you if you do 15:58:15 * alcabrera issues 2 minute warning 15:58:17 Nope... 15:58:20 kk 15:58:31 balajiiyer can get you set up 15:58:40 Ok 15:58:40 for now, I'll leave it assigned to balajiiyer 15:58:55 jay-atl: https://wiki.openstack.org/wiki/Getting_Started 15:58:56 thanks man! 15:59:25 ok, re bugs, if we can get that wsgiref thing sorted, I will be unblocked from working on more bugs 15:59:43 cool 15:59:44 fwiw, I have the autoreconnect almost done - I imagine it will need a few review cycles 15:59:51 (stay tuned) 15:59:57 #topic open discussion 15:59:59 30 seconds 16:00:00 ....go! 16:00:16 I'm happy about the increasing help. :) 16:00:35 w00t 16:00:38 I'm happy about Marconi and our team 16:00:38 #endmeeting