17:02:14 <pballand> #startmeeting CongressTeamMeeting
17:02:15 <openstack> Meeting started Tue May  6 17:02:14 2014 UTC and is due to finish in 60 minutes.  The chair is pballand. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:18 <openstack> The meeting name has been set to 'congressteammeeting'
17:02:52 <pballand> Agenda today: recap on action items, then planning for Atlanta
17:03:16 <pballand> rajdeep: do you mind going first?
17:03:34 <rajdeep> sure
17:03:37 <kudva> Hi
17:03:45 <thinrichs> kudva: hi
17:03:55 <rajdeep> last two weeks spent adding more get data to nova driver
17:03:55 <pballand> kudva: just getting started on recap
17:04:03 <kudva> ok
17:04:12 <rajdeep> added details about hosts and floating_ips
17:04:42 <rajdeep> now our nova driver has four tuple types
17:04:59 <rajdeep> servers, flavours, floating_ips and hosts
17:05:07 <rajdeep> and their associated field elements
17:05:37 <thinrichs> That definitely gives a richer feel to the Nova integration.  Good work.
17:05:43 <rajdeep> thanks!
17:06:03 <rajdeep> now i am planning to add some test cases as suggested by thinrichs
17:06:14 <rajdeep> need some guidance on test case structure
17:06:47 <pballand> I would suggest starting with unit tests, and everyting mocked up
17:07:14 <rajdeep> ok, we use mock framework?
17:07:33 <pballand> up to you
17:08:11 <rajdeep> where should these test cases be added in the tree.
17:08:24 <pballand> ‘tests’ subtree of the module
17:08:54 <rajdeep> ok
17:09:03 <thinrichs> While on the topic of subtrees, I think we ought to pull out the service_pluggins from the server directory and make it its own dir.
17:09:11 <thinrichs> And maybe rename it to just "datasources".
17:09:36 <thinrichs> I've been importing those things, and the paths are cumbersome.
17:09:41 <rajdeep> thats a good idea
17:09:53 <pballand> great, other questions on datasources?
17:10:06 <rajdeep> so we will have a tests directory under service_pluggins/datasources
17:10:29 <pballand> just /congress/datasources/tests
17:10:49 <rajdeep> ok..
17:11:11 <pballand> kudva: do you mind sharing an update?
17:11:16 <rajdeep> i will take the AI of changing the location and name of the directory service_pluggin
17:11:35 <kudva> Yes. I sent some code to Tim to review, and he responded with some nice suggestions, specifically
17:11:46 <pballand> #action rajdeep will change location of the service_plugin directory
17:12:09 <kudva> I have a builtin.py program, which gets imported into runtime.py (which was Tim's recommendation). I added code to runtime.py
17:12:14 <kudva> Now, I am writing tests using mock
17:12:41 <kudva> I will put these test locally for now. Once the tests pass, I will move them to the tests directory (should be simple to move).
17:13:29 <pballand> any estimate on when you may have soemthing for code reveiw?
17:13:32 <kudva> We talked a bit if the the working code into git, but he wasn't sure whether some of the jenkins test need to be disabled.
17:13:57 <pballand> currently, jenkins isn’t running the tests because i
17:13:58 <kudva> Tim has looked through the code. For the team, I would say next tuesday.
17:14:03 <pballand> ‘make’ isn’t wired in
17:14:09 <pballand> that’s on my plate (and has been for too long)
17:14:24 <pballand> you can still run the tests locally
17:14:24 <kudva> that way, I have some tests passing
17:14:29 <kudva> okay
17:15:02 <pballand> #action kudva to submit first pass of builtins for review next week
17:15:28 <pballand> that’s very exciting - I’m looking forward to having this fuctionality
17:15:46 <pballand> other questions or comments on builtins?
17:16:26 <pballand> ok, thinrichs: do you have any updates to share?
17:16:39 <thinrichs> I've been working on integrating the Plexxi code.
17:16:57 <thinrichs> It's been slow going, but I'm finally starting to understand that code.
17:17:18 <thinrichs> I've got the first test case working.
17:17:52 <cloudtoad> Excellent!
17:17:59 <thinrichs> That test case is just getting 2 data drivers to talk over the amqp bus.
17:18:08 <pballand> thats great - this will be really importaint for tying everything together in a nice runnable package!
17:18:14 <thinrichs> Next test case will be to get a data source and the policy engine to talk.
17:18:20 <thinrichs> cloudtoad: good to see you here!
17:18:32 <thinrichs> everyone else: cloudtoad wrote the Plexxi code.
17:18:44 <cloudtoad> Thanks, I'm glad to be a part of this.
17:18:45 <rajdeep> hi cloudtoad
17:18:53 <thinrichs> And we should probably start calling it DSE (Data services engine), instead of Plexxi code.
17:19:13 <rajdeep> or integration engine
17:19:43 <pballand> since cloudtoad is the author, I think he should get naming rights :)
17:19:54 <pballand> (it’s currently called dse)
17:20:19 <pballand> do you think you will be able to push some tests/examples soon?
17:20:30 <thinrichs> Maybe--depends on how things go.
17:20:47 <thinrichs> cloudtoad has also been planning on making some tweaks here and there.
17:20:56 <thinrichs> So I don't want to step on those.
17:21:02 <pballand> ok, fair enough :)
17:21:09 <thinrichs> But I'll push for review as soon as I have something solid.
17:21:26 <pballand> cloudtoad: anything add?
17:21:36 <cloudtoad> Yep..
17:21:42 <pballand> #action thinrichs to push tests for dse
17:22:03 <pballand> s/anything add/anything to add/
17:22:05 <cloudtoad> DSE needs a bit of cleanup work, I believe we will convert from Threading model to eventlets.
17:22:37 <cloudtoad> Comments, and perhaps some readme.md files in the DSE directory to walk through how it works and how to write data source modules.
17:22:53 <pballand> cloudtoad: yes, I took a peek at that, but haven’t made any changes yet - I was hesitant to change code without tests :)
17:23:16 <thinrichs> Maybe I'll push my baby unit test, so you've got a starting point.
17:23:26 <pballand> the dox would be a great help too
17:23:49 <pballand> !action cloudtoad to add documentation to DSE modules
17:23:50 <openstack> pballand: Error: "action" is not a valid command.
17:23:57 <pballand> #action cloudtoad to add documentation to DSE modules
17:24:04 * pballand is !-happy today for some reason
17:24:19 <rajdeep> quick question
17:24:43 <rajdeep> will the data source drivers need some rework for dse integration?
17:25:02 <pballand> it shouldn’t be much - just some boilerplate if anything
17:25:17 <thinrichs> I was playing around with that for my tests.
17:25:31 <thinrichs> I think we'll just have DataSourceDriver inherit from dse.deepSix
17:25:45 <rajdeep> ok that is good news
17:25:47 <rajdeep> !
17:25:54 <thinrichs> Might need to tweak __init__ but otherwise I think they should be completely compatible.
17:26:04 <thinrichs> As long as we're not stepping on the function names deepsix uses.
17:26:45 <pballand> anything else on DSE?
17:27:20 <pballand> I have been working on the API design, and have a proposal to send out
17:27:21 <cloudtoad> Not from me (that I can remember/think of)
17:27:38 <pballand> I started as a google doc for ease of comments: http://goo.gl/1E5MeY
17:28:37 <pballand> basically, I propose modeling all data as a data-source (including data output from a policy)
17:29:26 <cloudtoad> Good plan..
17:29:29 <pballand> I also left placeholders in for multi-tenancy, but will stick to just one ‘default’ policy for the first cut
17:29:57 <cloudtoad> If you can model policies and rules as tables, then policy data can be published on the DSE bus.
17:31:01 <thinrichs> We can publish policies/data/whatever on the DSE bus, right?  There's nothing about tables built into DSE.
17:31:19 <cloudtoad> Sorry, that's correct...
17:31:21 <thinrichs> But right I think we're expecting the API calls to be sent via the bus.
17:31:33 <cloudtoad> You can structure the information however you wish
17:32:04 <thinrichs> cloudtoad: agreed
17:32:20 <pballand> thinrichs: raises a good point - I envisioned the entire API running over the bus (it is just a REST projection of the bus)
17:32:47 <cloudtoad> How will that interface with Horizon, if at all?
17:33:23 <pballand> at this point, we don’t have any resources to work on horizon intergration, but that is definitely something we will need to takel
17:33:28 <pballand> tackle
17:33:33 <rajdeep> i thought REST was more for HTTP rather than messaging style interaction
17:34:30 <pballand> rajdeep: yes, they are different styles - the point is that the fundamental data interchange style of the core of the system is the message bus
17:35:00 <pballand> this model will facilitate scale-out in the future
17:35:12 <rajdeep> so it is json in a message body
17:35:17 <pballand> (and not lock us in to inefficient REST calls when we need better performance)
17:35:29 <cloudtoad> Rest calls really are requests for data...  Perhaps the URL will translate to AMQP address and body of request will be passed to policy module (or even a data source module).
17:35:39 <cloudtoad> rajdeep: Yes.
17:36:14 <pballand> cloudtoad: exactly
17:36:58 <pballand> the scalability is a bit of overkill at this point, but the pattern is pretty nice to work in
17:37:29 <pballand> other comments or questions about the API?
17:37:37 <pballand> (we can continue the conversation in the google doc)
17:38:14 <pballand> ok - I wanted to talk a bit about the conference next week
17:38:48 <pballand> there is one session scheduled for Congress on Wed evening, and a developer track on Friday morning 9-12:20
17:39:39 <pballand> I’ll just be giving an overview on Wed, but it would be great to have everyone in attendance
17:40:14 <pballand> On Friday, it will be an interactive design session
17:40:34 <pballand> #topic Open Discussion
17:41:16 <pballand> any other topics to discuss?
17:41:21 <cloudtoad> What is the actual agenda for Friday or is that freeform?
17:41:56 <pballand> We’re still working on that
17:42:06 <cloudtoad> Let me rephrase that... should I prepare something re: DSE for Wed or Fri?
17:42:26 <pballand> that would be awesome - for Fri
17:42:45 <rajdeep> sorry got disconnected
17:42:55 <pballand> I’ll get in touch with you about specifics
17:43:02 <cloudtoad> ok
17:43:49 <pballand> anything else?
17:44:25 <pballand> Thanks everyone, and I’m looking forward to seeing you in Atlanta next week!
17:44:36 <thinrichs> See you there!
17:44:43 <pballand> #endmeeting