17:02:00 <ativelkov> #startmeeting Murano
17:02:01 <openstack> Meeting started Tue Feb 18 17:02:00 2014 UTC and is due to finish in 60 minutes.  The chair is ativelkov. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:05 <openstack> The meeting name has been set to 'murano'
17:02:14 <ativelkov> Hi folks, any muranoers here?
17:02:25 <xwizard_> Hi
17:02:27 <stanlagun_> hi folks
17:02:33 <dteselkin> Hi
17:02:39 <sergmelikyan> Hi!
17:02:47 <sergmelikyan> o/
17:03:21 <ativelkov> Let's start then
17:03:35 <ativelkov> As usual, AI review first
17:03:39 <ativelkov> #topic AI review
17:04:19 <ativelkov> ativelkov to update project roadmap to reflect the metadata repository req's
17:04:41 <gokrokve_> Hi
17:04:49 <ativelkov> This is in progress, we are waiting for final input from gokrokve_ for MVP document
17:05:01 <ativelkov> gokrokve_: hi, just about time :)
17:05:56 <gokrokve_> I will update MVP document ASAP.
17:05:58 <ativelkov> Do we have a finalized version of MVP requirements which we may present and modify the BPs?
17:06:03 <ativelkov> ah, I see
17:06:42 <ativelkov> #action gokrokve_ to finalize MVP document, ativelkov to complete roadmap planning according to it
17:07:02 <ativelkov> next one
17:07:13 <ativelkov> ativelkov, stanlagun to complete the short abstract for the new DSL and then publish a complete description of Wiki
17:07:40 <ativelkov> The document is ready, but not published at wiki yet, as we are gathering the feedback from the community yet
17:07:45 <stanlagun_> we decided to temporary postpone it until we get some feedback in ML
17:08:19 <ativelkov> So, folks present, if you have any feedback on the DSL proposal - this is just a perfect time for that
17:09:16 <igormarnat> Speaking of the document - we don't need to wait for anything to publish it. Can do it in etherpad or wiki, but the sooner is the better, right?
17:09:42 <ativelkov> igormarnat: well, the abstract is indeed ready for publishing
17:10:16 <ativelkov> The language spec itself may be put into etherpad, but it is not final unless we agree on that
17:11:02 <igormarnat> Sure thing, we'll be editing it along the road
17:11:21 <ativelkov> Ok
17:11:31 <ativelkov> #action ativelkov publish the abstract on wiki
17:12:31 <ativelkov> #action ativelkov, stanlagun move spec to wiki and reference in blueprints
17:12:58 <tsufiev_> hi there
17:13:06 <ativelkov> Actually, it would be hard to put the spec into the etherpad - it has some fancy formatiing, tables etc
17:14:13 <stanlagun_> we can put spec onto wiki and create etherpad for discussions and code snippets. URL of etherpad would also be on wiki
17:14:41 <ativelkov> That's a good idea
17:14:46 <ruhe> or google-doc shared to everyone?
17:15:09 <stanlagun_> is it possible to share google doc to the whole world?\
17:15:15 <ativelkov> yes, it is possible
17:15:16 <ruhe> stanlagun_: yes
17:15:22 <ativelkov> but it is a bad practice
17:15:27 <gokrokve_> ruhe: We had some issue with google-doc before.
17:15:30 <ativelkov> as the owner may always close it back )
17:15:37 <gokrokve_> Probably now it wors better.
17:15:41 <gokrokve_> Probably now it works better.
17:15:45 <ativelkov> So, it is not a community-approved approach
17:16:05 <ativelkov> Also, the corporate policies may forbid doing that as well
17:16:19 <ativelkov> I mean, if we use Mirantis's google-drive
17:16:42 <gokrokve_> Etherpad or wiki are preferred solutions as they are owned by openstack.
17:16:56 <ativelkov> gokrokve_: +1
17:17:07 <gokrokve_> Googledoc is an external tool which will work but it is not under openstack community control.
17:17:49 <ativelkov> Well, let's get some content first, then we may discuss where to put it )
17:18:06 <ativelkov> We didn't get much feedback yet
17:18:11 <tnurlygayanov___> +1 for Wiki
17:19:11 <ativelkov> Are we done on this?
17:19:23 <stanlagun_> lets move on
17:19:41 <ativelkov> I would really appreciate some discussion on the DSL, but..
17:20:18 <ativelkov> If nobody have any questions, then it is either too obvious or too complicated..)
17:20:36 <ativelkov> so, let's move on
17:20:50 <ativelkov> #topic Oslo Messaging
17:21:03 <ativelkov> sergmelikyan: could you please share an update on this?
17:21:27 <dteselkin> I believe it just need to be used, before asking any other questions
17:22:45 <sergmelikyan> ativelkov, nothing new from previous week, http://lists.openstack.org/pipermail/openstack-dev/2014-February/026843.html
17:22:48 <dteselkin> I'm about DSL of course :)
17:23:40 <stanlagun_> I suggest we use oslo.messaging for API-conductor communications in 0.5 but not use it for conductor-agent ones
17:23:42 <ativelkov> sergmelikyan, so, our primary concerns are message delivery and queue life management?
17:24:26 <ativelkov> stanlagun_: and how to communicate with the agent then?
17:24:42 <sergmelikyan> ativelkov, yes
17:24:50 <stanlagun_> the same way we do now. Only maybe replace puka with kombu
17:25:01 <ativelkov> sounds good
17:25:19 <ativelkov> BTW, I've got a strange idea abouth this
17:25:34 <ativelkov> What if we use Marconi for managing this queues?
17:25:48 <ativelkov> Not in 0.5, but.. generally speaking?
17:26:33 <ativelkov> VMs are in Openstack's userland, so, message-queue-as-a-service may be a good choise there
17:26:37 <sergmelikyan> I am not sure, but AFAIK Marconi is a messaging queue with WebAPI, about what queue management you are talking about
17:26:39 <sergmelikyan> ?
17:26:43 <ruhe> ativelkov: we're considering Marconi as transport layer for guest agent in Savanna too. But it's far from being ready
17:27:22 <ativelkov> sergmelikyan: I am talking about agent<->engine communications
17:27:23 <stanlagun_> depends on what Marconi can do. I suspect we would need a lot of low-level control over RabbitMQ for tenant/agent isolation - SSL certificate authorization, maybe create vhosts for different tenants in runtime, control message lifetime, use transactions etc
17:27:57 <ativelkov> WebAPI may be good as it solves the connectivity issues (VMs not always have connection to infra-networks)
17:28:20 <ativelkov> ruhe can you share some details on that?
17:29:37 <ruhe> ativelkov: marconi doesn't yet speak amqp https://wiki.openstack.org/wiki/Marconi#Will_Marconi_work_with_AMQP.3F
17:30:16 <ativelkov> Yeah, then it seems like a long way to go yet..
17:30:36 <ativelkov> But we may put this idea somewhere in our roadmap
17:30:49 <ativelkov> ruhe: what are their incubation status?
17:32:07 <ruhe> ativelkov: according to https://blueprints.launchpad.net/marconi/+spec/graduation it is "Good Progress" :)
17:33:51 <ativelkov> Well, this means that we have at least to think about integrating with them
17:34:18 <ruhe> but, anyway, i'd suggest to use oslo messaging for communication
17:34:20 <ativelkov> #action ativelkov research potential Marconi integration for Agent communications
17:34:36 <ativelkov> ruhe, we have issues with message persistence there
17:35:02 <ativelkov> If the agent is not running when the message is sent, then the message is likely to be lost
17:35:21 <SergeyLukjanov> ativelkov, you can find details about marconi graduation in the tc meeting logs - http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-01-21-20.03.html
17:35:45 <ativelkov> I.e. the RPC will fail, while the notification will simply be lost forever
17:35:48 <ruhe> ativelkov: would it be difficult to modify oslo.messaging to remove all the obstacles? is it a big effort?
17:35:51 <ativelkov> SergeyLukjanov: thanks
17:36:00 <sergmelikyan> ruhe, different concepts
17:37:09 <sergmelikyan> ruhe, RPC is RPC, messaging is messaging. There is no guaranteed delivery in RPC terms, since there is no message delivery :)
17:37:17 <ativelkov> I suggest to start migrating slowly. API<->Engine communication can be done easily, I believe. Engine<->Agent may be trickier
17:37:37 <SergeyLukjanov> sergmelikyan, in RPC you already have a delivery guarantee
17:38:00 <ativelkov> RPC requires server to exist
17:38:09 <ativelkov> at the time when the call is being made
17:38:17 <SergeyLukjanov> ativelkov, yup
17:38:19 <ruhe> ativelkov: sergmelikyan: i will ask Dmitry Mescheryakov to join this discussion. He is implementing guest agent for Savanna and afaik he uses oslo.messaging
17:38:23 <ativelkov> Our workflows do not have this assumption
17:38:25 <stanlagun_> this is also not good for us for security reasons
17:38:45 <sergmelikyan> SergeyLukjanov, nope, you have call, and it is may happen or not.
17:39:01 <sergmelikyan> ruhe, thx!
17:39:59 <SergeyLukjanov> ativelkov, is it possible to extract communication code and make it pluggable to be able to add limited oslo.messaging backed engine?
17:40:55 <ativelkov> Probably, I need to do some more investigation here
17:41:01 <stanlagun_> anyway this doesn't solve security issues
17:42:05 <ativelkov> This needs to be coordinate between different projects
17:42:20 <ativelkov> I am pretty sure that this is a topic of Unified Agent initiative
17:42:51 <SergeyLukjanov> ativelkov, I
17:43:14 <SergeyLukjanov> ativelkov, I'm afraid that common approach for Unified Agents will not fit your requirements
17:43:28 <ativelkov> stanlagun_: what do you mean by security?
17:43:43 <SergeyLukjanov> ativelkov, and you need to communicate between services too
17:43:45 <ativelkov> SergeyLukjanov: then it is not Unified, I guess )
17:44:56 <ruhe> ativelkov: SergeyLukjanov: it would be great to discuss this topic with Dmitry, I believe he will have something interesting to say
17:45:16 <stanlagun_> VMs are need do be restricted so that they can't access other queues in any way. This is possible only be controlling RabbitMQ. And thats cannot be done with general purpose framework like oslo.messaging
17:45:24 <sergmelikyan> ativelkov, we need a way to physically restrict access from VM to a data in messaging queue that not intended to be accessed by that VM
17:45:25 <SergeyLukjanov> ativelkov, AFAIK you have some uncommon (for OpenStack at least) message queue usage pattern, that's not needed for all other project, so, in case if agent will work on top on of MQ than it'll use oslo.messaging
17:45:55 <ativelkov> ruhe: let's schedule some meeting then
17:46:02 <sergmelikyan> ativelkov, Any arbitrary code running on VM may have access to oslo.messaging settings and intercept whole cloud communications
17:46:09 <ruhe> SergeyLukjanov: i believe Murano has the same requirement as Savanna - tenant isolation, and VM isolation
17:46:25 <sergmelikyan> ruhe, yep!
17:47:07 <SergeyLukjanov> oh, just remembered, we have a repo for guestagent poc for savanna - https://github.com/stackforge/savanna-guestagent it's currently empty, but Dmitry is planning to start  moving code to it soon
17:47:07 <stanlagun_> It is impossible to physically isolate VMs on RabbitMQ side without controlling RabbitMQ ACLs
17:47:10 <ativelkov> We just have quite a sophisticated system of VM-side commands (called 'execution plans'), but the transport looks pretty-much generic
17:47:12 <ruhe> and the current approach - is to deploy another rabbitmq for murano<->VM communication. Dmitry wants to have the same approach for Savanna agents
17:47:55 <stanlagun_> it would definitely be another RabbitMQ. But we need to isolate different VMs accessing this RabbitMQ
17:48:02 <ativelkov> So, let's schedule a meeting with Dmitry. Which timezone is he in?
17:48:06 <SergeyLukjanov> the problem is that RabbitMQ isn't the standard MQ in OpenStack, Qpid should be supported too at least
17:48:19 <SergeyLukjanov> ativelkov, UTC+0400
17:48:37 <ativelkov> Good
17:48:48 <stanlagun_> yep. QPID is also AMQP. We can use QPID instead of RabbitMQ. We just need low-level control over it
17:49:06 <sergmelikyan> SergeyLukjanov, Qpid may be supported since has almost same capabilities due to implementation of same protocol, but some extensions available only in RabbitMQ: queue ttl for example
17:49:37 <ativelkov> #action ativelkov to schedule a meeting with Dmitry Mescheryakov to discuss agent messaging
17:49:38 <stanlagun_> anyway this is private message broker to Murano. So i doesn't metter what exactly broker it is.
17:50:54 <SergeyLukjanov> stanlagun_, if you'd like to be a part of OpenStack you should support common OpenStack installation, I know at least two of them - RabbitMQ and Qpid (in RDO), don't know how are using 0mq in prod
17:51:37 <stanlagun_> If it is private separate instance of RabbitMQ (QPID, whatever) it is not part of OpenStack installation anyway
17:52:35 <SergeyLukjanov> stanlagun_, you should read initial discussions about Barbician incubation
17:53:00 <SergeyLukjanov> and as sergmelikyan said before, murano depends on some RabbitMQ features (ttl)
17:53:25 <ruhe> stanlagun_: i believe SergeyLukjanov wanted to say that your code should work on top of common MQ brokers - which are rabbit, zero and qpid. code shouldn't be broker-specific
17:54:02 <SergeyLukjanov> ruhe, yup
17:54:47 <SergeyLukjanov> and by "part of OpenStack" I mean being incubated, not installed near the OpenStack
17:54:53 <ativelkov> Then we may have adapters for different kinds of broker
17:56:20 <ruhe> or let oslo.messaging to handle broker-specific stuff :)
17:57:27 <SergeyLukjanov> ruhe, AFAIK it was an incubation requirement for Barbician
17:57:44 <ruhe> yes it was, and it will be for Murano
18:00:25 <ativelkov> We are runnig out of time guys
18:00:41 <ativelkov> let's continue this in private channels
18:01:13 <ativelkov> #endmeeting