15:00:27 <ad_rien_> #startmeeting massively_distributed_clouds
15:00:27 <ad_rien_> #chair ad_rien_
15:00:27 <ad_rien_> #info agenda
15:00:27 <ad_rien_> #link https://etherpad.openstack.org/p/massively_distributed_ircmeetings_2017
15:00:27 <openstack> Meeting started Wed Mar  1 15:00:27 2017 UTC and is due to finish in 60 minutes.  The chair is ad_rien_. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:31 <openstack> The meeting name has been set to 'massively_distributed_clouds'
15:00:33 <openstack> Current chairs: ad_rien_
15:00:45 <jamemcc> Hi
15:00:47 <ad_rien_> HI
15:01:31 <ad_rien_> #link https://etherpad.openstack.org/p/massively_distributed_ircmeetings_2017 (line 251) - Agenda for the meeting
15:01:35 <ansmith> o/
15:01:47 <denaitre> hi o/
15:01:52 <Menthos> hi
15:01:54 <mabderrahim> Hi o/
15:01:57 <ad_rien_> As usual Guys, may I ask you to add your name in the pad. line 253
15:01:58 <ad_rien_> thanks
15:02:23 <ad_rien_> let's wait one more minutes before starting
15:02:57 <ad_rien_> #topic Announcement
15:03:10 <ad_rien_> So from my side, nothing special,
15:03:32 <ad_rien_> I just put a link toward some discussions the Scientific WG had on identity federation challenges.
15:04:05 <ad_rien_> so that's all. These last few days were completely crazy from my side ;)
15:04:21 <ad_rien_> Anything else from your side before diving into the agenda
15:04:26 <ad_rien_> ansmith: jamemcc ?
15:04:59 <ad_rien_> ok looks not….
15:05:12 <ad_rien_> BTW I did receive the scores for the presentations
15:05:27 <ad_rien_> does anyone know when this should be notified?
15:05:38 <ad_rien_> s/I did/I didn't
15:05:42 <ad_rien_> sorry
15:06:14 <ad_rien_> ok so todays for the agenda we planned to discuss two points:
15:06:20 <jamemcc> lol - I'm not following adrien - ask it again - are you talking about the Summit seelction process
15:06:39 <ad_rien_> jamemcc:  yes
15:06:39 <ad_rien_> ;)
15:07:01 <ad_rien_> sorry I should probably try to be less enthusiast
15:07:11 <jamemcc> I didnt' think the selection was over - just the initial public voting
15:07:18 <ad_rien_> and leave time for getting answers
15:07:20 <ad_rien_> ok
15:07:42 <ad_rien_> do you have an idea about the notification of acceptance date?
15:08:15 <jamemcc> Though I'm not on the selection committee this time.  Lets keep going and I'll look as I am interested - it's a common question.
15:08:28 <ad_rien_> jamemcc:  ack
15:08:29 <ad_rien_> ok
15:08:55 <ad_rien_> so as mentioned today we planned to discuss two points mainly: ecomp (ATT solution) and QPID related discussions.
15:09:10 <Menthos> "After track chairs make their decisions, speakers will be informed by mid March 2017"
15:09:12 <ad_rien_> so who wants to start ?
15:09:21 <ad_rien_> thanks Menthos
15:09:53 <ad_rien_> #info speakers will be informed by mid March 2017
15:10:06 <kgiusti> I can present (it should be quick)...
15:10:18 <ad_rien_> ok please go ahead kgiusti
15:10:18 <kgiusti> https://docs.google.com/presentation/d/1ghwinrArfoCw1qIsNGxWSrd8NTynYomThERGUpN4f0U/edit#slide=id.p
15:10:42 <kgiusti> sure - the slide deck provides a very high level
15:10:57 <kgiusti> introduction to the various messging backends
15:11:03 <kgiusti> supported by oslo.messaging
15:11:33 <kgiusti> We'll talk about zeromq, rabbit/qpid/ and qdrouterd
15:11:40 <kgiusti> So Page 2
15:12:00 <kgiusti> This is the traditional message bus deployment for OS
15:12:22 <ad_rien_> ok
15:12:24 <kgiusti> Well understood, much tribal knowledge, used by 99.9999999 deployments
15:12:45 <kgiusti> I think we all understand this - so... slide 3
15:13:10 <kgiusti> (the pro/con slides were just added ... certainly incomplete)
15:13:25 <ad_rien_> this is a valuable starting point at least ;)
15:13:30 <ad_rien_> Let's move forward please
15:13:46 <kgiusti> So the limitations and pros should be obvious - any q's
15:13:47 <kgiusti> ?
15:14:17 <kgiusti> On page 4 - this is the logical topology that can be acheived using zeromq for RPC
15:14:32 <kgiusti> The broker is still present, but used only for notifications
15:14:55 <kgiusti> This has two immediately obvious advantages - less load on the broker, and direct (non-queued) RPC
15:15:13 <ad_rien_> I though that zeroMQ was fully P2P based.
15:15:34 <ad_rien_> Why should we go through the router?
15:15:36 <kgiusti> Yes it is, but doesn't support notificiation traffic
15:15:56 <kgiusti> Slide 4 - there is no router used in this scenario
15:16:05 <ad_rien_> ok sorry
15:16:09 <ad_rien_> was on slide 6
15:16:09 <ad_rien_> thanks
15:16:27 <kgiusti> Just broker for Notifications, Zmq for RPC (dual backend - supported in ocata+)
15:16:52 <ad_rien_> "(dual backend - supported in ocata+)"  did someone test it ?
15:16:58 <kgiusti> Slide 5:  The biggest issue is that zeromq has very little adoption
15:17:04 <kgiusti> yes - ansmith!
15:17:14 <ad_rien_> good job I didn't know that
15:17:17 <ad_rien_> is there any link?
15:17:22 <kgiusti> He's adding it to Triple O at the moment....
15:17:34 <kgiusti> ansmith: ?
15:17:39 <ad_rien_> #topic AMQP bus discussion
15:18:03 <ansmith> dual backend in tripleo and across puppet modules
15:18:03 <kgiusti> So the biggest issue here is that zeromq is largely unproven.
15:18:35 <kgiusti> I'm not sure if anyone is actually deploying it in production.... may be wrong here.
15:18:35 <ad_rien_> #info zeroMQ support will be available in ocata+ versions (ansmith is working on that)
15:18:57 <ansmith> specifically working on "enabling" dual backends
15:19:04 <kgiusti> ansmith: I believe it will be qdrouter first....
15:19:15 <kgiusti> We'll get to that in a moment
15:19:21 <ansmith> primary target is hybrid with qdrouterd for rpc and rabbit for notify
15:20:01 <kgiusti> The zeromq driver is very complex when compared to the other driver codebases since the driver has all the intelligence for message topology control embedded
15:20:19 <kgiusti> IOW: complexity "at the edge"
15:20:36 <ad_rien_> ok
15:20:59 <kgiusti> And it is unknown how much of a issue the large # of TCP connections required will be
15:21:08 <kgiusti> (think TCP resource limits).
15:21:22 <kgiusti> So and Q's for 2a?
15:21:32 <ad_rien_> once available it will be rather simple to evaluate that point.
15:21:38 <kgiusti> +1
15:21:54 <ad_rien_> #action make a comparison with Enos of RabbitMQ, ZeroMQ and QPid for scenario 1
15:21:55 <kgiusti> The driver seems well supported (mostly mirantis folks)
15:22:22 <ad_rien_> rcherrueau:  do you know whether kolla provides support for using zmq instead of rabbit?
15:22:23 <kgiusti> So bugs should be addressed relatively quickly I would think
15:23:03 <rcherrueau> ad_rien_: let me check
15:23:04 <kgiusti> on to slide 6...
15:23:26 <kgiusti> This is a variant of the last slide - just replacing zeromq with qdrouterd
15:23:58 <kgiusti> So any "single broker for notification" downsides/upsides also apply here
15:23:59 <rcherrueau> ad_rien_: No, there is no support in kolla-ansible
15:24:10 <ad_rien_> rcherrueau:  ack
15:24:24 <ad_rien_> could you please check what will be mandatory to provide such a feature?
15:24:51 <kgiusti> However, there is much lower TCP resource usage - one connection to the local router per client.
15:25:02 <ansmith> ad_rien_, rcherrueau : the effort will be akin to what what we did with puppet and tripleo
15:25:04 <kgiusti> And the ability to add redunancy
15:25:10 <kgiusti> path redundancy
15:25:20 <kgiusti> for failover (availability)
15:25:32 <ad_rien_> ansmith:  ack
15:25:34 <ad_rien_> thanks
15:26:01 <ad_rien_> kgiusti:  could you please just clarify why/when you have to go through the dispatch router?
15:26:10 <kgiusti> There is a degree of parallelism in the RPC traffic flows, though less than zmq
15:26:34 <kgiusti> ad_rien_: for RPC traffic only.
15:26:59 <ad_rien_> not sure I correctly understood.
15:27:00 <kgiusti> each client would also need a connection to the single centralized broker for notifications
15:27:09 <ad_rien_> ok like zmq
15:27:22 <kgiusti> Each client connects to it's qdrouterd "onramp"
15:27:32 <ad_rien_> I'm trying to understand the != between the zmq rpc stuff and the qpid one?
15:28:17 <kgiusti> Fault recovery via redundant routers, the ability to cross unroutable IP subnets (firewall)
15:28:38 <kgiusti> All the routing intellegence and policy is controlled by the routers
15:28:47 <ad_rien_> ok thanks
15:28:49 <kgiusti> the driver is very simple
15:29:05 <kgiusti> so page 7
15:29:24 <kgiusti> I think we already touched on that stuff - q's?
15:29:49 <kgiusti> Slide 8
15:30:04 <kgiusti> This simply puts the broker "behind" the router
15:30:20 <kgiusti> But this allows broker redundancy
15:30:33 <kgiusti> e.g. have a "higher cost link" to the backup broker
15:31:02 <kgiusti> Switch on failover, but some message loss of course.... unless clustering/HA is used by the broker
15:31:32 <ad_rien_> msimonin is absent today unfortunately (and is the AMQP expert one :( )
15:31:36 <kgiusti> but this is moving towards more of a "messaging as a service" model where the message bus is simply a black box
15:31:40 <kgiusti> :(
15:32:11 <ad_rien_> so we have a couple of questions but maybe they are naive
15:32:14 <kgiusti> For example there is an Openshift based project (enmass) which we hope to integrate Oslo.messaging with.... futrue stuff
15:33:08 <ad_rien_> if you are considering a large number of site, let's say 1000 , each composed of 50 servers
15:33:30 <ad_rien_> if you have router/broker functions on Site 0, would it scale?
15:33:57 <ad_rien_> in the zeromq (please don't get me wrong I do not say zmq is the solution to select)
15:34:14 <ad_rien_> only the notification would be sent to Site0 is it correct?
15:34:44 <kgiusti> Be aware I don't have actual working experience with such a deployment - so take this as an educated guess :(
15:34:55 <ad_rien_> ok
15:35:04 <kgiusti> Yes w/zmq only notifications goto site 0
15:35:13 <ad_rien_> ok
15:35:44 <kgiusti> The biggest issue is how stable zmq will be given the number of TCP connections necessary to interconnect all servers
15:35:52 <kgiusti> that need to communicate via RPC
15:36:10 <ad_rien_> can be an issue indeed.
15:36:57 <kgiusti> In the same case, but with qdrouterd the issue will be how long will the topology take to stablize
15:37:04 <ad_rien_> can the router block be a distributed one?
15:37:18 <ad_rien_> (HA/scalability considerations?)
15:38:13 <kgiusti> you can deploy as many routers as you deem necessary, but each hop across a router adds some latency (depends on the load on the router) and
15:38:48 <kgiusti> more routers == longer time to recover the topology should a failure occur
15:39:17 <kgiusti> so each site could have multiple routers if there is a benefit
15:39:42 <ad_rien_> ok
15:39:59 <kgiusti> any other Q's?
15:40:23 <kgiusti> Feel free to followup later if you want - my email and nick are in the notes
15:40:23 <ad_rien_> ok
15:40:30 <ad_rien_> if we consider the scenario 1
15:40:36 <ad_rien_> (i.e. the one we selected
15:40:57 <ad_rien_> where we have all control services in Site 0 and only compute nodes on remote sites)
15:41:20 <ad_rien_> if we correctly understood, compute nodes will not communicate each other, right?
15:41:52 <kgiusti> I believe that is correct - it would be a centralized hub and spoke topology
15:42:07 <ad_rien_> so the use of zmq or qpid is valid only for scenarios 2/3+ where we will have several services geographically spread
15:42:13 <ad_rien_> ok
15:42:21 <ad_rien_> So it is 45.
15:42:37 <ad_rien_> May I propose to prepare some questions for the next meeting. Can all please agree on that point ?
15:42:44 <kgiusti> well, you _could_ try zeromq for hub and spoke if the load on the broker becomes critical
15:43:12 <ad_rien_> #link  https://docs.google.com/presentation/d/1ghwinrArfoCw1qIsNGxWSrd8NTynYomThERGUpN4f0U/edit#slide=id.g1b037c92e2_0_269 AMQP discussion
15:43:34 <ad_rien_> #action all - please see the amqp slides and prepare questions for the next meeting.
15:43:50 <ad_rien_> kgiusti:  do you want to add something else? ansmith?
15:44:16 <ansmith> specifically, update to kolla-ansible would be to use oslo.messaging transport url inplace of deprecated rabbit host, port, auth as configuration options
15:44:32 <ansmith> on the list to do
15:44:54 <ansmith> this will enable separating the messaging backends
15:45:00 <rcherrueau> ansmith: you are right
15:45:59 <ad_rien_> ok
15:46:10 <ad_rien_> may I ask to move to the following point in the agenda?
15:46:20 <ad_rien_> (we can come back later if we have time)
15:46:47 <kgiusti> +1
15:46:52 <ad_rien_> thanks
15:47:05 <ad_rien_> #topic ecomp
15:47:14 <ad_rien_> jamemcc?
15:47:15 <jamemcc> Thanks - I'm here still. I first want to status that I didn't get a commitment from one of the more estblished eComp architects at AT&T to join us.
15:47:32 <jamemcc> Just a quick check though: Anyone else joined to talk about eComp?
15:48:16 <ad_rien_> jamemcc:  if you want we can postpone the discussion next time?
15:48:47 <jamemcc> I have better leads on who might be appropriate - but perhaps we could instead check to see if anyone read the materials sent out and then talk about where the investigation/collaboration might go.
15:49:22 <ad_rien_> guys ?
15:49:39 <ad_rien_> What do you prefer? To talk now or to discuss next time?
15:50:15 <ad_rien_> (from my side, I would rather prefer the second option as I did not have time to give a deeper look in the different materials)
15:50:20 <jamemcc> Yeah - next time - though if there are general quesions - like what level of functionality or whats the strategy for the project - I think I could answer that.
15:50:29 <ansmith> +1
15:50:58 <rcherrueau> Btw, the slot ends in 10 min, so next time is maybe more appropriate ...
15:51:15 <ad_rien_> #action all - give a look to the content sent by jamemcc and prepare questions for next meeting.
15:51:39 <jamemcc> Answering to the ML ahead of time will be helpful as well:
15:51:40 <jamemcc> http://lists.openstack.org/pipermail/user-committee/2017-February/001722.html
15:52:11 <ad_rien_> #link http://lists.openstack.org/pipermail/user-committee/2017-February/001722.html ML message regarding ecomp
15:52:37 <ad_rien_> #topic scenario deployment.
15:53:04 <ad_rien_> ok right now we are just performing the first experiments with Scenario 0. we expect to gather preliminary results by next meeting
15:53:27 <ad_rien_> nothing more so unless you have question I proposed to move on the next point.
15:54:35 <ad_rien_> #topic AMQP stresser
15:54:44 <ad_rien_> I'm not sure Orange folks are there?
15:55:01 <ad_rien_> I saw mabderrahim but didn't see jfpeltier nor mrohon
15:55:01 <ad_rien_> ?
15:55:03 <ad_rien_> guys ?
15:55:06 <ad_rien_> are you there?
15:55:12 <mabderrahim> I am here
15:55:44 <ad_rien_> ok I proposed to keep that on the agenda
15:55:56 <ad_rien_> next point OpenDiscussion
15:56:01 <ad_rien_> #topic open discussion
15:56:24 <ad_rien_> apart what I put in the pad I have nothing more
15:56:33 <ad_rien_> anything you would like to discuss from your side Guys ?
15:56:58 <ad_rien_> three minutes before ending the meeting?
15:57:41 <ad_rien_> ansmith:  jamemcc kgiusti anyone else ?
15:57:51 <kgiusti> I'm good :)
15:58:00 <ansmith> good
15:58:06 <ad_rien_> So I propose to conclude the meeting. thanks for being there.
15:58:13 <jamemcc> No - but perhaps we can chat a little offline afterwards - cross working group
15:58:13 <ad_rien_> #end	meeting
15:58:23 <ad_rien_> #endmeeting