00:00:42 <thinrichs> #startmeeting CongressTeamMeeting
00:00:42 <openstack> Meeting started Thu Jan  7 00:00:42 2016 UTC and is due to finish in 60 minutes.  The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:00:43 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:00:45 <openstack> The meeting name has been set to 'congressteammeeting'
00:01:17 <thinrichs> Since this is the first time back since the holidays, let's give people a couple of minutes
00:01:25 <thinrichs> Who is here now?
00:02:20 <masahito> Hi
00:02:38 <thinrichs> masahito: hi
00:02:45 <masahito> happy new year
00:03:19 <thinrichs> Happy New Year to you too!
00:03:37 <thinrichs> While we wait for others, want to chat about your Push driver?
00:04:10 <masahito> yap
00:04:55 <thinrichs> I think you're right about PUT http://<ip>:<port>/data-sources/<ds-id>/tables/<table-id or name>/rows
00:05:00 <thinrichs> instead of to …/tables
00:06:03 <thinrichs> Could you expand on your comment about the oslo-messaging driver?
00:06:15 <masahito> ok
00:06:39 <thinrichs> ekcs: hi!
00:06:50 <thinrichs> ekcs: Happy New Year!
00:07:12 <thinrichs> I should set this up as a topic before we get too deep into it.
00:07:24 <thinrichs> #topic Push datasource driver
00:07:29 <ekcs> hny  guys!
00:07:42 <masahito> ekcs: hi, happy new year!
00:08:26 <thinrichs> ekcs: we are discussing masahito's new push datasource driver.
00:08:30 <thinrichs> #link https://review.openstack.org/#/c/256303/
00:08:51 <ekcs> yup
00:09:33 <thinrichs> We're discussing now where the web server that accepts the PUT requests should be running
00:09:52 <thinrichs> Option 1 is for it to be our existing web server.
00:10:05 <thinrichs> Option 2 is for the server to be running as part of the datasource itself
00:11:09 <thinrichs> In the new distributed architecture, option 1 would have the existing API row model receive the request, and send an RPC call to the appropriate datasource.
00:11:23 <masahito> IMO, using *http* protocol is one of implementation for push driver. So I think we'll implement a datasource which uses other protocol in the future.
00:12:11 <thinrichs> In option 2, the sender would send the data to the IP/port of the datasource directly.
00:12:28 <thinrichs> masahito: it makes sense to think of http as 1 implementation.
00:12:54 <thinrichs> And oslo-messaging would be another implementation.
00:13:53 <masahito> I think using oslo-message matchs to OpenStack services.
00:14:05 <thinrichs> masahito: For http, it sounds odd that the data sender would target a different IP/port than all the other http API calls.
00:14:53 <masahito> since they have a function named notification to send an event to somewhere.
00:14:55 <thinrichs> masahito: But how would oslo-messaging work?  How do we configure it?  Would we configure each datasource to listen to it?
00:15:35 <thinrichs> Does each datasource subscribe to certain topics on oslo-messaging?
00:17:11 <masahito> My idea is each datasource subscribes it to update table.
00:17:36 <masahito> of course, it depends on usecase.
00:17:45 <masahito> This is another idea.
00:17:53 <pballand> hello - sorry I’m late
00:18:05 <thinrichs> pballand: Happy New Year!
00:18:13 <pballand> thanks, you too!
00:18:23 <thinrichs> pballand: we're discussing Masahito's Push datasource driver
00:18:26 <thinrichs> #link https://review.openstack.org/#/c/256303/
00:18:37 <pballand> thinrichs: thanks, I caught up on eavesdrop
00:18:38 <masahito> Monasca or other monitor services collects notifications by oslo-message and sends summary to datasource by http.
00:18:50 <masahito> pballand: Happy New Year!
00:18:52 <thinrichs> pballand: thanks for stopping me from rehashing
00:18:57 <pballand> I think its fine for datasource drivers to accept updates through the API service, and/or directly
00:19:16 <pballand> when using HTTP, it’s probably easisest to leverage the existing API listener
00:19:35 <pballand> for non-http, clearly a different service would need to bind to the port
00:20:27 <pballand> that’s my two cents at least :)
00:20:35 <thinrichs> pballand: agreed
00:21:21 <thinrichs> masahito: what do you think?
00:21:57 <pballand> to be clear, I’m not opposed to a datasource exposing its own http interface, but it seems like redundant work
00:22:21 <masahito> pballand: agreed
00:22:25 <thinrichs> pballand: what would be the benefit of having a separate http interface?
00:22:34 <thinrichs> skipping the rpc?
00:23:11 <pballand> if the driver had specific requirements (e.g. long polling) that weren’t needed by the API service
00:23:35 <pballand> (that may be a bad example, since long polling is a feature that could be useful)
00:23:46 <ekcs> makes sense to me to have http go to central api process. so that every sender doesn’t need to track a bunch of ip’s which could change as processes go down and restart etc.
00:24:10 <pballand> but the point is that a datasource may want non-standard semantics, and should be able to do so without breaking the rules
00:24:47 <thinrichs> ekcs: I think that once we go to HA the issues with multiple IPs will be more obvious
00:24:54 <pballand> ekcs: I agree that that would be ideal, I just don’t think it needs to be a requirement
00:25:14 <thinrichs> pballand: in general that makes sense.  Do we see a need for doing that right now for the Push driver?
00:25:46 <thinrichs> where by "that" I mean having http servers running inside the datasources
00:25:59 <masahito> My concern is just it. If we implements other protocol the architecture looks strange.
00:26:43 <pballand> we could have a generic way to POST data to a service RPC
00:26:58 <thinrichs> masahito: the arch is strange because pushing data over http goes through the usual API but pushing data through oslo.messaging (or whatever) goes through the datasource?
00:27:09 <pballand> (that way the push driver doesn’t need to register additional API endpoints)
00:27:14 <masahito> thinrichs: right
00:28:06 <pballand> hmm, that doesn’t seem strange to me :-)
00:28:54 <masahito> pballand: API has a general way to update data sounds nice.
00:28:55 <thinrichs> masahito: I agree with pballand.  There are 2 very different mechanisms being used to push data into the system.  It seems natural that they are implemented differently.
00:29:20 <pballand> we could add a ‘message bus’ service that can push to multiple drivers just like the API service could
00:29:43 <thinrichs> pballand: are you suggesting generalizing the idea that a PUT http://<ip>:<port>/data-sources/<ds-id>/tables/<table-id or name>/rows  results in an RPC?
00:30:34 <pballand> thinrichs: I was actually suggesting an alternative to that, but I’m fine with your suggestion as well
00:31:04 <thinrichs> Another option, more aligned with your suggestion would be to hang another action off of the services.
00:31:20 <pballand> I was suggesting something like POST http://<ip>:<port>/data-sources/<ds-id>/rpc?method=push_data
00:31:41 <pballand> (where method could be anything, and it is up to the service to validate the rest of the call)
00:32:50 <thinrichs> I wonder if that's too close to building a generic messaging service for the Congress components, instead of exposing a single abstraction that gets implemented on top of a messaging service.
00:33:30 <pballand> thinrichs: that’s a good point - suggests that your proposal makes more sense
00:34:22 <ekcs> in terms of HA, it may be helpful to not have *all* push datasource updates be interrupted because the api process goes down or is out of reach.
00:35:11 <thinrichs> I'm in favor of just extending the API to enable PUT to a datasource's rows (for http) and have message-buses push data directly to datasources.
00:35:22 <pballand> thinrichs: sounds good to me
00:35:59 <pballand> ekcs: if you are concerned with that, you could launch multiple API services, or go so far as to launch an API service for every datasource...
00:36:57 <thinrichs> masahito: what do you think?
00:37:51 <ekcs> just seems that we’re making a functionality that is conceptually distinct from the api depend on the api service. I don’t have a strong opinion either way but pointing it out.
00:38:10 <masahito> I also think extending existing the API has much good points now.
00:39:16 <thinrichs> sounds good.  Let's move ahead with that.
00:39:21 <thinrichs> And let's move ahead with the meeting.
00:39:32 <thinrichs> Couple of quick things before the dist arch...
00:39:35 <masahito> for HA, choosing datasource or API is not matter. but for sender it's easy to send data for API.
00:39:44 <thinrichs> #topic Reminders
00:40:07 <thinrichs> We have a mid-cycle sprint later this month.
00:40:10 <thinrichs> Agenda is here..
00:40:12 <thinrichs> #link https://wiki.openstack.org/wiki/Sprints/CongressMitakaSprint
00:40:47 <thinrichs> Call for speakers for Austin is available.
00:40:59 <thinrichs> Talks (max of 3 per person) are due Feb 3
00:41:00 <thinrichs> https://www.openstack.org/summit-login/login?BackURL=/summit/austin-2016/call-for-speakers/
00:41:16 <thinrichs> I'm happy to talk with anyone who wants to submit something.
00:41:42 <thinrichs> Final reminder: we are now gating on python3, thanks in large part to ekcs
00:42:01 <thinrichs> So if your submission isn't python3 compliant, it won't pass tests
00:43:01 <thinrichs> #topic distributed architecture
00:43:16 <thinrichs> pballand: it's early in the new year; any progress yet?
00:44:02 <pballand> I was out for much of the hoiday, still working toward next week’s deadline
00:44:42 <thinrichs> sounds good.  Let us know if we can help
00:44:45 <pballand> I’m hitting some bumps where oslo.messaging is rejecting messages based on version (even though the version seems right) - will post to IRC and/or mailing list if I can’t get it working by tomorrow
00:45:29 <thinrichs> Sounds like the right thing.
00:47:20 <thinrichs> Thought that would take a bit longer.
00:47:40 <thinrichs> Let's open it up for discussion then.
00:47:44 <thinrichs> #topic open discussion
00:49:04 <thinrichs> If there's nothing else, one thing to bounce off everyone is the agenda for the mid-cycle sprint
00:49:15 <thinrichs> On the wiki page, we have 3 topics (one for each of the 3 days):
00:49:34 <thinrichs> That was supposed to be a )   :, not a sad face.
00:49:42 <thinrichs> (1) sprint to implement the first draft of the distributed architecture designed during the Liberty cycle,
00:49:50 <thinrichs> (2) design a high-availability, high-throughput deployment of the distributed architecture
00:49:57 <thinrichs> (3) discuss using Monasca as a robust way of injecting aggregated time-series data into Congress
00:50:14 <thinrichs> How does that sound?
00:50:36 <thinrichs> I haven't heard from fabiog whether (3) is happening.
00:50:44 <thinrichs> At some point he mentioned maybe moving (3) to the front.
00:50:48 <pballand> sounds good to me
00:51:32 <ekcs> great.
00:51:42 <masahito> sounds good.
00:51:53 <thinrichs> It still sounds good to me too.
00:52:23 <thinrichs> Is there any sort of preparation we/I should do?
00:53:02 <thinrichs> I should probably email out to the ML to make sure everyone who wants to attend knows about it.
00:53:26 <thinrichs> I think I did that already, but we had more attendees at the last one than were regular on IRC.
00:54:03 <thinrichs> Go ahead and fill out the eventbrite if you haven't.  In case fabiog needs a headcount for food/etc.
00:54:24 <thinrichs> If there's nothing else, let's call that the end of the meeting.
00:54:26 <ekcs> we’d hopefully be working on (1) already and have a good grip on the issues so that during the session we try to finalize.
00:54:44 <thinrichs> ekcs: Agreed.
00:55:32 <thinrichs> If there's nothing else, Thanks all!
00:56:05 <masahito> thanks, bye
00:56:10 <thinrichs> #endmeeting