00:03:11 <thinrichs> #startmeeting CongressTeamMeeting
00:03:12 <openstack> Meeting started Thu Jan 14 00:03:11 2016 UTC and is due to finish in 60 minutes.  The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:03:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:03:15 <pballand> hi
00:03:16 <openstack> The meeting name has been set to 'congressteammeeting'
00:03:17 <ekcs> hi
00:03:24 <ramineni> hi
00:03:30 <thinrichs> pballand, ekcs, ramineni: hi!
00:03:54 <thinrichs> masahito: you here?
00:04:05 <masahito> yes
00:04:23 <thinrichs> Here's my agenda for today…
00:04:39 <thinrichs> 1. distributed arch
00:04:42 <thinrichs> 2. mid-cycle meet-up
00:04:49 <thinrichs> 3. Austin
00:04:57 <thinrichs> 4. mitaka-2 next week
00:05:04 <thinrichs> 5. push driver
00:05:09 <thinrichs> Anything else?
00:05:43 <thinrichs> 6. stable branches
00:06:15 <thinrichs> Let's get started then.
00:06:18 <thinrichs> #topic distributed arch
00:06:28 <thinrichs> pballand: I saw you pushing code to review.  How's it going?
00:06:55 <pballand> yeah - I abandoned the old set and pushed a new one: https://review.openstack.org/#/q/status:open+project:openstack/congress+branch:master+topic:dse2
00:07:03 <pballand> here’s: where it’s at...
00:07:36 <pballand> the code is completely standalone still, pending enough functionality (I’ll get to that) to merge existing services
00:07:57 <pballand> it supports: launching multiple DseNodes, that discover eachother and share status
00:08:17 <pballand> DseNodes can expose RPCs and invoke them on other nodes
00:08:38 <pballand> each node runs zero or more services, which also support exposing and invoking RPCs
00:09:14 <pballand> the big missing piece is the DseService base class support for table subscription - I’m working on that right now
00:09:37 <pballand> the pieces pushed so far should give a good indication of where I am going, however, and should be ready for review
00:10:01 <pballand> that’s the summary - any questions?
00:10:08 <thinrichs> That's tremendous progress!
00:11:40 <pballand> thanks - I’ll try to respond to review comments quickly, so we can iterate on the design
00:11:43 <thinrichs> I'm trying to figure out what we can do with the code that's already there.
00:12:08 <thinrichs> We could (i) hook up the API models to the datasources and policy engines…
00:13:03 <pballand> yes, and we could link the main driver to launch DseNode with a command-line option to run those services that are hooked up
00:13:06 <thinrichs> I guess the only other thing we use the bus for right now having the policy engine/datasources pub/sub tables, which we can't do.
00:13:26 <pballand> thinrichs: right - that is the big missing piece, sorry :-(
00:13:43 <pballand> on the plus side, all of that relies on RPC, which has quite a bit of test coverage
00:13:53 <thinrichs> pballand: not calling out problems—just trying to enumerate the main pieces of functionality that we use the bus for.
00:14:26 <thinrichs> Is it fair to say that we have 2 uses for the bus: (1) RPC from API and (2) table pub/sub for everything else?
00:14:45 <pballand> that, and policy delegation
00:14:55 <thinrichs> policy delegation is rpc, right?
00:15:04 <thinrichs> action execution is also rpc
00:15:25 <pballand> I guess so, that’s fair
00:15:37 <thinrichs> How's the unit test framework looking?
00:16:18 <pballand> if anyone is looking for something to do, integrating the dse2 unit tests with the framework would be very helpful - it’s not something I’ve put _any_ time in
00:16:23 <pballand> on
00:17:50 <thinrichs> pballand: how's the test coverage looking (independent if those tests are running in the current framework).
00:18:42 <pballand> I haven’t run any coverage checks, but there are tests for all of the main methods
00:19:00 <pballand> mostly functional tests (little/nothing mocked) rather than unit tests
00:19:20 <pballand> https://review.openstack.org/#/c/267228/2/congress/dse2/test_dse_node.py
00:19:36 <pballand> for example
00:20:31 <thinrichs> All right, well that's some good background I'll use when I go to review.
00:20:48 <thinrichs> Any other questions/comments on the distributed arch?
00:21:30 <pballand> that test is an example of simple RPC calls, and the DseControlBus extends DataService in non-trivial ways
00:23:02 <ekcs> thinrichs: none from me.
00:23:06 <thinrichs> I'll take a look and make a review pass either tonight or tomorrow hopefully.
00:23:45 <thinrichs> Let's all get pballand reviews as soon as possible, so we all have a chance to pull down the code before the mid-cycle meetup.
00:23:52 <thinrichs> #action Everyone will review the DseNode code
00:24:01 <thinrichs> #topic Push driver
00:24:07 <thinrichs> masahito: how is the push driver going?
00:25:19 <masahito> I push patches which receive data with existing API server.
00:25:35 <masahito> https://review.openstack.org/#/c/256303/
00:25:40 <masahito> https://review.openstack.org/#/c/266347/
00:26:33 <masahito> I'm wondering if I remove http_driver.py or not in next patch set.
00:27:22 <masahito> because if Congress don't have it there is not reference implementation for other developers.
00:27:35 <masahito> That's my update.
00:27:50 <thinrichs> The http_driver.py shows how to set up a datasource to accept http requests for pushing data.  Is that right?
00:28:55 <masahito> right. and shows what developers have to implement like get_datasource_info()
00:29:28 <thinrichs> Gerrit saves everything forever…
00:29:46 <thinrichs> so what I've done in the past is abandon the change that has the code that might be useful later but that we decided to not use immediately...
00:30:05 <thinrichs> and then create a new changeset that has just the code that gets merged.
00:30:31 <thinrichs> That way, we always have the alternate code available, but we don't pollute master with code that isn't currently being used.
00:30:46 <masahito> Just my idea is keeping http_driver.py there or add a new driver that works for my OPNFV use-case.
00:31:19 <masahito> I understand your concern.
00:31:37 <thinrichs> Does your OPNFV use case not work if the http push is routed through the API model?
00:32:34 <masahito> no. it works when http push is routed via the API.
00:32:52 <ekcs> What if the reference code is moved into an rst? Would that be a good solution?
00:33:36 <masahito> but the table the driver should have would be different among the user.
00:34:14 <thinrichs> masahito: not sure I understand.  Can you give us some more context?
00:36:17 <masahito> example are following.
00:38:33 <masahito> user A want to use a table which has schema, event_id, host and status, but user B want to use a table which has 3 schema, name, status and disk.∫
00:39:03 <masahito> In this case,
00:41:02 <masahito> I don't think it's not good to extend the driver everytime when the number of user increases and they use different schema.
00:42:40 <thinrichs> So you're saying that if we have 2 users who want to push different schema, we need 2 different datasources in congress that define the translators for that schema.
00:43:01 <thinrichs> And then if we have N different schema, we'd need N different datasources in congress.
00:43:11 <masahito> right.
00:43:32 <thinrichs> If we could tell people that they are only allowed to push tables, then we don't really need a translator.
00:43:45 <thinrichs> And 1 datasource would be enough for everyone.
00:44:08 <thinrichs> The only time we need translators is when people are pushing data that is not a list of tuples of strings/numbers.
00:44:59 <thinrichs> So maybe we would want a TablePushDriver that anyone can use who wants to push relational data in any schema.
00:45:48 <thinrichs> I suppose we could also create a JsonPushDriver that people can use who want to push json data, and recursively walk that data to turn it into tables.
00:46:19 <thinrichs> I guess the downside to that idea is that the schema isn't fixed in the code.  The schema is dynamically computed at runtime.
00:46:47 <thinrichs> I think the policy engine should be flexible enough to handle that now.
00:47:18 <ekcs> question: is this a problem that arises only with push not poll?
00:47:36 <thinrichs> masahito: does that make sense?  (We're running short on time, so maybe we should continue this discussion offline.)
00:47:49 <masahito> yes.
00:47:52 <masahito> my idea for it is we implement create table API for definition of the driver.
00:47:52 <thinrichs> ekcs: right now all the datasources have static schemas.
00:48:15 <thinrichs> masahito: that makes sense
00:48:16 <masahito> I check if it works in our use-case.
00:49:08 <thinrichs> masahito: if not, we can always compute the schema dynamically based on the data that has been pushed.  (The schema is being set dynamically in either case.)
00:49:21 <thinrichs> We're short on time, so let's move on.
00:49:43 <thinrichs> #topic stable branches
00:50:07 <thinrichs> ramineni: I saw your change for project-config finally got merged.  Right?
00:50:33 <ramineni> thinrichs: yes :) , so now stable branches run only congress tests without plugin mechanism
00:50:48 <ramineni> https://review.openstack.org/#/c/266801/
00:50:51 <thinrichs> ramineni: congrats!  That one took a while.
00:50:56 <ramineni> this patch should turn the gate green
00:51:22 <thinrichs> ramineni: great!  I'll take a look once we finish up the meeting.
00:51:26 <ramineni> thinrichs: yep :D
00:51:38 <thinrichs> Hopefully then we can get the backlog of stable/liberty fixes in.
00:52:04 <thinrichs> ramineni: Do you remember if there's some stable/kilo work to do?
00:52:04 <ramineni> thinrichs: tempet tests are still disabled though , i will enable in one patch after it merges
00:52:31 <thinrichs> ramineni: are you on the stable-maint team?  (Seems like you should be.)
00:52:32 <ramineni> thinrichs: no, stable kilo we have disabled this job i guess right?
00:53:01 <thinrichs> Maybe so…I'm losing track.
00:53:06 <ramineni> thinrichs: ya, think so
00:53:11 <thinrichs> I'll try to look into it.
00:53:38 <ramineni> thinrichs: i think its disabled , but do we want to support stable/kilo also?
00:54:09 <thinrichs> ramineni: I don't remember if it was working at all.  I seem to think not.  Let's focus on stable/liberty for now.
00:54:32 <thinrichs> ramineni: let us know if there's any help you need getting stable/liberty back on track.
00:54:47 <ramineni> thinrichs: sure
00:54:55 <thinrichs> If you're in stable/maint, you should be able to +2 things and merge them.
00:55:24 <ramineni> thinrichs: ok
00:55:27 <thinrichs> 5 minutes left.
00:55:37 <thinrichs> #topic open discussion
00:55:44 <thinrichs> ekcs: anything to report?
00:56:05 <ekcs> I'm working on this bp for pub-sub communication between policy engine and datasources: https://blueprints.launchpad.net/congress/+spec/dist-datasources-on-bus
00:56:05 <ekcs> Almost ready to push reviewable WIP.
00:56:10 <thinrichs> anyone else have anything?  (We can interleave the discussions to finish on time).
00:56:55 <thinrichs> ekcs: nice!  Having that in the pipeline will be good once we start getting pballand's changes integrated.
00:57:16 <thinrichs> Couple of reminders…
00:57:37 <thinrichs> 1. Calll for speakers at Austin closes Feb 3
00:57:56 <thinrichs> 2. Our mid-cycle meetup is Jan 26-28.
00:58:01 <thinrichs> 3. Mitaka-2 is next week
00:58:15 <thinrichs> Think that's it.
00:58:18 <ekcs> masahito: also, on this patch (https://review.openstack.org/#/c/263464/), I’m not sure I understand your comment. Could we talk about it either here or later?
00:58:36 <thinrichs> I'd suggest taking that to #congress, since we've got 2 min left.
00:59:09 <masahito> ekcs: in #congrss later :)
00:59:11 <ekcs> thinrichs: k
00:59:24 <thinrichs> That does it for this week.  Thanks all!
00:59:35 <thinrichs> Let's make sure to get those reviews in.
00:59:52 <thinrichs> #endmeeting