00:00:42 #startmeeting CongressTeamMeeting 00:00:42 Meeting started Thu Jan 7 00:00:42 2016 UTC and is due to finish in 60 minutes. The chair is thinrichs. Information about MeetBot at http://wiki.debian.org/MeetBot. 00:00:43 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 00:00:45 The meeting name has been set to 'congressteammeeting' 00:01:17 Since this is the first time back since the holidays, let's give people a couple of minutes 00:01:25 Who is here now? 00:02:20 Hi 00:02:38 masahito: hi 00:02:45 happy new year 00:03:19 Happy New Year to you too! 00:03:37 While we wait for others, want to chat about your Push driver? 00:04:10 yap 00:04:55 I think you're right about PUT http://:/data-sources//tables//rows 00:05:00 instead of to …/tables 00:06:03 Could you expand on your comment about the oslo-messaging driver? 00:06:15 ok 00:06:39 ekcs: hi! 00:06:50 ekcs: Happy New Year! 00:07:12 I should set this up as a topic before we get too deep into it. 00:07:24 #topic Push datasource driver 00:07:29 hny guys! 00:07:42 ekcs: hi, happy new year! 00:08:26 ekcs: we are discussing masahito's new push datasource driver. 00:08:30 #link https://review.openstack.org/#/c/256303/ 00:08:51 yup 00:09:33 We're discussing now where the web server that accepts the PUT requests should be running 00:09:52 Option 1 is for it to be our existing web server. 00:10:05 Option 2 is for the server to be running as part of the datasource itself 00:11:09 In the new distributed architecture, option 1 would have the existing API row model receive the request, and send an RPC call to the appropriate datasource. 00:11:23 IMO, using *http* protocol is one of implementation for push driver. So I think we'll implement a datasource which uses other protocol in the future. 00:12:11 In option 2, the sender would send the data to the IP/port of the datasource directly. 00:12:28 masahito: it makes sense to think of http as 1 implementation. 00:12:54 And oslo-messaging would be another implementation. 00:13:53 I think using oslo-message matchs to OpenStack services. 00:14:05 masahito: For http, it sounds odd that the data sender would target a different IP/port than all the other http API calls. 00:14:53 since they have a function named notification to send an event to somewhere. 00:14:55 masahito: But how would oslo-messaging work? How do we configure it? Would we configure each datasource to listen to it? 00:15:35 Does each datasource subscribe to certain topics on oslo-messaging? 00:17:11 My idea is each datasource subscribes it to update table. 00:17:36 of course, it depends on usecase. 00:17:45 This is another idea. 00:17:53 hello - sorry I’m late 00:18:05 pballand: Happy New Year! 00:18:13 thanks, you too! 00:18:23 pballand: we're discussing Masahito's Push datasource driver 00:18:26 #link https://review.openstack.org/#/c/256303/ 00:18:37 thinrichs: thanks, I caught up on eavesdrop 00:18:38 Monasca or other monitor services collects notifications by oslo-message and sends summary to datasource by http. 00:18:50 pballand: Happy New Year! 00:18:52 pballand: thanks for stopping me from rehashing 00:18:57 I think its fine for datasource drivers to accept updates through the API service, and/or directly 00:19:16 when using HTTP, it’s probably easisest to leverage the existing API listener 00:19:35 for non-http, clearly a different service would need to bind to the port 00:20:27 that’s my two cents at least :) 00:20:35 pballand: agreed 00:21:21 masahito: what do you think? 00:21:57 to be clear, I’m not opposed to a datasource exposing its own http interface, but it seems like redundant work 00:22:21 pballand: agreed 00:22:25 pballand: what would be the benefit of having a separate http interface? 00:22:34 skipping the rpc? 00:23:11 if the driver had specific requirements (e.g. long polling) that weren’t needed by the API service 00:23:35 (that may be a bad example, since long polling is a feature that could be useful) 00:23:46 makes sense to me to have http go to central api process. so that every sender doesn’t need to track a bunch of ip’s which could change as processes go down and restart etc. 00:24:10 but the point is that a datasource may want non-standard semantics, and should be able to do so without breaking the rules 00:24:47 ekcs: I think that once we go to HA the issues with multiple IPs will be more obvious 00:24:54 ekcs: I agree that that would be ideal, I just don’t think it needs to be a requirement 00:25:14 pballand: in general that makes sense. Do we see a need for doing that right now for the Push driver? 00:25:46 where by "that" I mean having http servers running inside the datasources 00:25:59 My concern is just it. If we implements other protocol the architecture looks strange. 00:26:43 we could have a generic way to POST data to a service RPC 00:26:58 masahito: the arch is strange because pushing data over http goes through the usual API but pushing data through oslo.messaging (or whatever) goes through the datasource? 00:27:09 (that way the push driver doesn’t need to register additional API endpoints) 00:27:14 thinrichs: right 00:28:06 hmm, that doesn’t seem strange to me :-) 00:28:54 pballand: API has a general way to update data sounds nice. 00:28:55 masahito: I agree with pballand. There are 2 very different mechanisms being used to push data into the system. It seems natural that they are implemented differently. 00:29:20 we could add a ‘message bus’ service that can push to multiple drivers just like the API service could 00:29:43 pballand: are you suggesting generalizing the idea that a PUT http://:/data-sources//tables//rows results in an RPC? 00:30:34 thinrichs: I was actually suggesting an alternative to that, but I’m fine with your suggestion as well 00:31:04 Another option, more aligned with your suggestion would be to hang another action off of the services. 00:31:20 I was suggesting something like POST http://:/data-sources//rpc?method=push_data 00:31:41 (where method could be anything, and it is up to the service to validate the rest of the call) 00:32:50 I wonder if that's too close to building a generic messaging service for the Congress components, instead of exposing a single abstraction that gets implemented on top of a messaging service. 00:33:30 thinrichs: that’s a good point - suggests that your proposal makes more sense 00:34:22 in terms of HA, it may be helpful to not have *all* push datasource updates be interrupted because the api process goes down or is out of reach. 00:35:11 I'm in favor of just extending the API to enable PUT to a datasource's rows (for http) and have message-buses push data directly to datasources. 00:35:22 thinrichs: sounds good to me 00:35:59 ekcs: if you are concerned with that, you could launch multiple API services, or go so far as to launch an API service for every datasource... 00:36:57 masahito: what do you think? 00:37:51 just seems that we’re making a functionality that is conceptually distinct from the api depend on the api service. I don’t have a strong opinion either way but pointing it out. 00:38:10 I also think extending existing the API has much good points now. 00:39:16 sounds good. Let's move ahead with that. 00:39:21 And let's move ahead with the meeting. 00:39:32 Couple of quick things before the dist arch... 00:39:35 for HA, choosing datasource or API is not matter. but for sender it's easy to send data for API. 00:39:44 #topic Reminders 00:40:07 We have a mid-cycle sprint later this month. 00:40:10 Agenda is here.. 00:40:12 #link https://wiki.openstack.org/wiki/Sprints/CongressMitakaSprint 00:40:47 Call for speakers for Austin is available. 00:40:59 Talks (max of 3 per person) are due Feb 3 00:41:00 https://www.openstack.org/summit-login/login?BackURL=/summit/austin-2016/call-for-speakers/ 00:41:16 I'm happy to talk with anyone who wants to submit something. 00:41:42 Final reminder: we are now gating on python3, thanks in large part to ekcs 00:42:01 So if your submission isn't python3 compliant, it won't pass tests 00:43:01 #topic distributed architecture 00:43:16 pballand: it's early in the new year; any progress yet? 00:44:02 I was out for much of the hoiday, still working toward next week’s deadline 00:44:42 sounds good. Let us know if we can help 00:44:45 I’m hitting some bumps where oslo.messaging is rejecting messages based on version (even though the version seems right) - will post to IRC and/or mailing list if I can’t get it working by tomorrow 00:45:29 Sounds like the right thing. 00:47:20 Thought that would take a bit longer. 00:47:40 Let's open it up for discussion then. 00:47:44 #topic open discussion 00:49:04 If there's nothing else, one thing to bounce off everyone is the agenda for the mid-cycle sprint 00:49:15 On the wiki page, we have 3 topics (one for each of the 3 days): 00:49:34 That was supposed to be a ) :, not a sad face. 00:49:42 (1) sprint to implement the first draft of the distributed architecture designed during the Liberty cycle, 00:49:50 (2) design a high-availability, high-throughput deployment of the distributed architecture 00:49:57 (3) discuss using Monasca as a robust way of injecting aggregated time-series data into Congress 00:50:14 How does that sound? 00:50:36 I haven't heard from fabiog whether (3) is happening. 00:50:44 At some point he mentioned maybe moving (3) to the front. 00:50:48 sounds good to me 00:51:32 great. 00:51:42 sounds good. 00:51:53 It still sounds good to me too. 00:52:23 Is there any sort of preparation we/I should do? 00:53:02 I should probably email out to the ML to make sure everyone who wants to attend knows about it. 00:53:26 I think I did that already, but we had more attendees at the last one than were regular on IRC. 00:54:03 Go ahead and fill out the eventbrite if you haven't. In case fabiog needs a headcount for food/etc. 00:54:24 If there's nothing else, let's call that the end of the meeting. 00:54:26 we’d hopefully be working on (1) already and have a good grip on the issues so that during the session we try to finalize. 00:54:44 ekcs: Agreed. 00:55:32 If there's nothing else, Thanks all! 00:56:05 thanks, bye 00:56:10 #endmeeting