Monday, 2014-07-07

*** reed has joined #openstack-marconi00:03
*** reed has quit IRC00:50
*** oz_akan_ has joined #openstack-marconi01:07
*** nosnos has joined #openstack-marconi01:37
*** flwang_ has joined #openstack-marconi02:15
*** flwang_ has quit IRC02:20
*** mwagner_lap has joined #openstack-marconi02:39
*** nosnos has quit IRC03:25
*** nosnos has joined #openstack-marconi04:02
*** ajc_ has joined #openstack-marconi04:52
*** prashanthr_ has joined #openstack-marconi05:23
*** chandan_kumar has joined #openstack-marconi05:34
*** prashanthr_ has quit IRC05:45
*** prashanthr_ has joined #openstack-marconi05:48
*** AAzza_afk is now known as AAzza06:41
*** haomaiwa_ has joined #openstack-marconi07:05
*** chandan_kumar is now known as chandankumar07:23
*** flaper87|afk is now known as flaper8707:24
*** AAzza is now known as AAzza_afk07:31
*** ykaplan has joined #openstack-marconi07:33
*** AAzza_afk is now known as AAzza07:44
*** ykaplan has quit IRC08:07
*** ykaplan has joined #openstack-marconi08:08
openstackgerritA change was merged to openstack/marconi: Fix broken wiki link
* flaper87 facepalm08:11
flaper87flwang: hey, good morning08:11
flaper87erm, well08:11
flaper87morning/afternoon/evening/enjoy whatever part of the day your in08:11
*** flwang_ has joined #openstack-marconi08:18
*** flwang_ has quit IRC08:22
*** flwang_ has joined #openstack-marconi09:11
*** ykaplan has quit IRC09:41
*** ykaplan has joined #openstack-marconi10:35
*** haomaiwa_ has quit IRC10:38
*** haomaiwa_ has joined #openstack-marconi10:38
*** mkoderer has joined #openstack-marconi10:54
*** haomaiwa_ has quit IRC10:57
*** haomai___ has joined #openstack-marconi10:57
*** vkmc has joined #openstack-marconi11:04
vkmcgood morning!11:07
*** prashanthr_ has quit IRC11:19
*** ykaplan has quit IRC11:27
flaper87vkmc: gooooooooood morning11:29
vkmcflaper87, :) how are you? how was that bd?11:30
flaper87vkmc: it was great... Spent time w/ friends and family and hell I had lot of food.11:31
flaper87I think I gained like 10k KGs11:31
vkmcflaper87, lol, that always happen11:32
vkmcand you will be eating cake for the rest of the week11:32
flaper87My bday was yday but the party started on Saturday at noon11:32
vkmc:D I love bds11:32
vkmcflaper87, how many?11:33
flaper87oh not that many, we were like 10ppl. It was for local, close, friends.11:34
vkmchaha years I meant ;)11:35
flaper87I'll need the whole week to recover11:35
vkmcyou need another weekend to recover from your weekend11:35
flaper87ahhh, fucking english, it's so ambiguous11:35
flaper87(2014-flaper87) ;)11:36
openstackvkmc: Error: "!!" is not a valid command.11:36
vkmcyeah I supposed that but wasn't sure11:36
vkmcopenstack, stfu11:36
* vkmc sends some ear pulls to flaper8711:38
flaper87LOL, what did I do????????11:39
vkmclol it's a spanish thing I guess11:39
vkmcpeople here pull your ears for your bd... 1 pull for year11:40
vkmcat our age that's annoying, so I'm done with 'some'11:40
*** AAzza is now known as AAzza_afk11:41
flaper87vkmc: yeah, we do that as well (in Vzla, that is)11:41
vkmcflaper87, not in Italy though?11:41
flaper87vkmc: nah, we're too busy eating here11:41
*** flwang_ has quit IRC11:43
*** ykaplan has joined #openstack-marconi11:44
*** tedross has joined #openstack-marconi11:50
vkmcflaper87, thanks for replying my doubts about AMQP transport11:55
flaper87vkmc: I hope all that makes sense11:55
flaper87vkmc: let me know if you want to have another chat about that11:56
vkmcflaper87, just one nit regarding the interaction approach... while I agree that it's a good idea to make it URI based11:56
vkmcflaper87, I'm a bit concerned about how are we going ot handle operations11:56
vkmcI mean11:56
vkmcthis is a valid AMQP address amqp://
vkmcbut this is not amqp://
vkmcmaybe in our context it will make sense... but for the AMQP clients this could be misleading11:58
flaper87vkmc: that's what I meant with the point that I don't believe this transport should support *all* Marconi operations12:02
flaper87vkmc: Do you think you'll be able to look into what you can do with Azure's HTTP API and AMQP's12:02
vkmcflaper87, sure, np!12:03
flaper87I looked into it a bit earlier today and it seems they don't support the same things in both transports/APIs/"whatever term they use to differentiate these things"12:03
flaper87vkmc: what do you think about that?12:03
vkmcflaper87, if this makes sense for Marconi, then it's ok to me12:03
flaper87vkmc: would you prefer having the full HTTP api being supported by the AMQP transport or something more amqp-ish ?12:04
flaper87vkmc: hehehe, you're deciding what makes sense for marconi12:04
vkmcflaper87, if we provide full support for the HTTP API then we will be supporting operations that are out from the AMQP semantics12:04
vkmcflaper87, we will have AMQP users knocking in our doors as soon as we release it12:05
flaper87vkmc: hahaha, indeed12:05
vkmcflaper87, so.. it will make more sense to stick with AMQP's use cases12:06
vkmcand leave the cool operations Marconi provides aside12:07
*** tedross has quit IRC12:07
flaper87vkmc: I'm leaning towards that, although I'm crurious to know what Azure Queues does12:08
*** tedross has joined #openstack-marconi12:08
vkmcflaper87, Azure just provides sending and receiving of messages12:08
vkmcflaper87, as far as I can see in their docs12:09
vkmcI just subscribed to a 1 month free trial to see it from the inside12:10
*** oz_akan_ has quit IRC12:12
flaper87vkmc: good idea, lemme know how it goes12:19
vkmcflaper87, so, yes... it follows a pub-sub approach, the Service Bus behaves as a broker12:27
vkmcclients can send messages to a queue and other clients would receive them iff they are subscribed to that queue12:28
*** megan_w has quit IRC12:28
*** jraim has quit IRC12:28
*** megan_w|afk has joined #openstack-marconi12:28
*** Alex_Gaynor has quit IRC12:28
*** megan_w|afk is now known as megan_w12:29
flaper87vkmc: so it's avery simple prod/cons w/o fancy locking, claim and stuff12:29
vkmcflaper87, exactly12:29
flaper87vkmc: what do you think about doing that as a first simple approach?12:29
*** malini has quit IRC12:29
flaper87vkmc: we could use oslo.messaging for a full-API implementation over RPC12:30
*** Ephur has quit IRC12:30
vkmcflaper87, let's do it... we will tune it up afterwards if Marconi users ask for that12:30
*** malini|afk has joined #openstack-marconi12:30
*** malini|afk is now known as malini12:30
flaper87vkmc: sounds good12:30
*** jraim has joined #openstack-marconi12:30
vkmcbut we have to be extra careful to make it clear that not all the Marconi API is supported12:30
flaper87vkmc: sounds like a good topic for a blog post, what do you think?12:31
*** Alex_Gaynor_ has joined #openstack-marconi12:31
*** jraim has quit IRC12:31
*** jraim has joined #openstack-marconi12:31
flaper87vkmc: do you want to bring this up to the m-l to get more feedback?12:31
*** Alex_Gaynor_ has quit IRC12:31
*** Alex_Gaynor_ has joined #openstack-marconi12:31
vkmcif we are after AMQP's performance, then that's the way to go12:31
vkmcflaper87, that would be awesome yeah12:31
flaper87vkmc: ok, make sure you share all your tests, readings, findings, experiments and conclusions12:32
vkmcflaper87, will do12:32
flaper87vkmc: awesome, thanks :D12:32
*** Ephur has joined #openstack-marconi12:33
*** ajc_ has quit IRC12:33
vkmcflaper87, thanks you for the guided brainstorming :)12:34
*** sriram has joined #openstack-marconi12:43
*** AAzza_afk is now known as AAzza12:51
vkmcflaper87, if you are curious, here are some screenshots of how Service Bus with AMQP is
* flaper87 is always curious12:54
*** mpanetta has joined #openstack-marconi12:56
flaper87vkmc: good resource, +112:56
flaper87vkmc: do they support both pub/sub and prod/cons through AMQP?12:57
* flaper87 hopes vkmc is not hating him for asking so many things instead of just RTFM12:57
vkmcflaper87, lol np12:59
vkmcflaper87, in fact... that is not clear in any docs12:59
vkmcflaper87, it looks like they don't support prod/cons12:59
flaper87vkmc: I guess you could work on a small script that does that?13:00
flaper87using proton13:00
* flaper87 is now sure that vkmc hates him13:00
vkmchahaha I just was looking into it13:00
*** oz_akan_ has joined #openstack-marconi13:00
vkmcthe thing is13:01
vkmchow we differenciate between pub/sub and prod/cons?13:01
vkmcI'm aware that Azure uses the concept of 'topic' to talk about pub/sub13:01
vkmcthey have topics not queues13:01
*** nosnos has quit IRC13:02
vkmcin other words, topics provide a one-to-many communication using a pub/sub pattern whereas queues provide a one-to-one communication using a prod/cons pattern13:06
*** amalagon has joined #openstack-marconi13:06
*** tedross has quit IRC13:08
*** amalagon_ has joined #openstack-marconi13:08
flaper87vkmc: we don't have a resource for topics in marconi yet13:10
*** keith_newstadt has joined #openstack-marconi13:10
flaper87vkmc: we'll probably need one13:10
*** chandan_kumar has joined #openstack-marconi13:10
flaper87vkmc: I guess Azure has queues and topics13:11
*** amalagon has quit IRC13:11
flaper87how do they differentiate?13:11
vkmcflaper87, they do yeah13:11
vkmcflaper87, we can simulate topics though13:11
*** mpanetta has quit IRC13:12
*** keith_newstadt has quit IRC13:15
*** chandankumar has quit IRC13:15
*** chandan_kumar is now known as chandankumar13:15
*** keith_newstadt has joined #openstack-marconi13:16
*** mpanetta has joined #openstack-marconi13:16
*** mwagner_lap has quit IRC13:22
*** tedross has joined #openstack-marconi13:23
vkmcflaper87, they won't allow you to create a topic and a queue with the same name13:23
vkmcit makes sense :)13:23
vkmcthey have the concept of 'namespace' though13:24
flaper87vkmc: does it? That sounds like they're using queues as topics13:24
flaper87aaaha! That's why13:24
vkmcbut it's exactly the same13:24
vkmcin the marconi namespace they won't allow me to have a 'poptart' queue and a 'poptart' topic13:24
vkmcin the storage backend they probably save namespace + entity name13:25
vkmcall together13:25
flaper87yeah, that really sounds like they're re-using queues as topics13:25
flaper87vkmc: exactly13:25
* flaper87 stops saying what vkmc says13:25
vkmcas I was saying, we don't have topics but we can simulate them13:26
openstackgerritOpenStack Proposal Bot proposed a change to openstack/marconi: Updated from global requirements
vkmcbut once again, we don't have anything to differenciate between them -.-13:28
flaper87vkmc: do we have to?13:29
flaper87I mean, when posting a message users normally have to choose whether they want to send it to a topic or a queue, right?13:30
flaper87this basically means they're programmatically separating queues from topics based on the messaging pattern they're using13:30
vkmcthey are doing that yeah13:31
flaper87What happens if we simply use queues for prod/cons and pub/sub and let users keep doing that on the client side?13:31
vkmcsince the pattern followed by queues and topics is different, we should be able to differenciate between them13:31
vkmcwe could do that yeah13:31
flaper87it'd be a bad idea to mix them together but that can be stated in the docs13:31
vkmcbut we still need to differenciate the pattern in use13:32
flaper87vkmc: right, we can do that using the API13:32
vkmcbecause if we follow a prod/cons pattern then when a client gets a message we have to remove it from the storage backend13:33
vkmcin the pub/sub paradigm we don't need that13:33
flaper87vkmc: right13:33
vkmcI guess we can have some header to distinguish between them13:36
vkmcor support just one13:36
sriramhey hey hey :D13:41
flaper87sriram: hey :D13:41
vkmchi sriram! :D13:41
sriramflaper87, vkmc : how is it going? :)13:42
sriramI'm almost done with setting up the benchmark cluster finally :P13:42
sriramafter dealing with some annoying config issues, firewalls :P13:42
vkmcsriram, that's awesome!13:43
flaper87sriram: awesome news, please, lemme know when those are done13:43
sriramflaper87: its usable now.. need some performance tweaks.13:45
flaper87sriram: awesome13:46
flaper87vkmc: thanks for the review here:
flaper87sriram: can you review this?
flaper87malini: ^ ?13:46
*** amitgandhi has joined #openstack-marconi13:48
*** amitgandhi has quit IRC13:48
*** amitgandhi has joined #openstack-marconi13:48
vkmcmalini, \o13:49
flaper87malini: o/13:55
maliniheyyyy vkmc & flaper87!14:05
maliniflaper87: need some clarification abt your review comment
*** amitgandhi has quit IRC14:06
*** amitgand_ has joined #openstack-marconi14:06
flaper87malini: sure, what I meant is instead of iterating N times right away, you could use a generator there14:06
flaper87malini: basically, using () instead of [] should be enough14:07
flaper87then remember to use itertools.ifilter instead of filter to make sure you don't consume that generator right away14:07
maliniflaper87: let me try tht..14:08
flaper87malini: that's just to avoid doing the same iteration 3 times. It'll do the query at the very end of the function when the final iteration happens14:08
maliniBut the query will return/update only one record at a time14:09
malinimaybe I am missing something?14:10
flaper87malini: mmh, probably you're missing the part where N find_and_modify will still be called14:11
flaper87let me put this in code14:11
*** tonytan4ever has joined #openstack-marconi14:15
flaper87malini: and some docs:
maliniperfect thanks!!14:16
*** tonytan4ever has quit IRC14:18
*** tonytan4ever has joined #openstack-marconi14:18
*** tonytan4ever has quit IRC14:23
*** cpallares has joined #openstack-marconi14:25
*** alcabrera|afk is now known as alcabrera14:31
*** prashanthr_ has joined #openstack-marconi14:35
*** tonytan4ever has joined #openstack-marconi14:40
*** prashanthr_ has quit IRC14:40
*** prashanthr_ has joined #openstack-marconi14:43
*** oz_akan_ has quit IRC14:47
*** oz_akan_ has joined #openstack-marconi14:47
*** chandankumar has quit IRC14:51
*** jay-atl has joined #openstack-marconi14:54
AAzzai have a question: if i understand right in general project could be None, but when in pooled setup - cannot be?15:01
*** chandan_kumar has joined #openstack-marconi15:06
*** tonytan4_ has joined #openstack-marconi15:08
*** tonytan4ever has quit IRC15:11
*** amitgand_ is now known as amitgandhi15:11
*** tedross has quit IRC15:12
AAzzaflaper87: ping15:21
flaper87AAzza: pong15:21
flaper87mmh, it could be None when you're using pools15:21
flaper87The project is used to enforce multi-tenancy15:21
flaper87if you don't care about multi-tenancy you can choose not to use a project15:21
AAzzahmm... yes, it is working with mongo, but in sqlalchemy, i'm not sure15:23
AAzzait is is nullable=False there15:24
AAzzaflaper87: let me check one more time15:24
AAzzayes, when i run unit tests for storage on sqlaclchemy in pooled setup, some tests, where the project is none fails on  "not null constraint failed"15:26
*** tedross has joined #openstack-marconi15:28
*** sinfex has joined #openstack-marconi15:29
*** sinfex has left #openstack-marconi15:29
flaper87AAzza: oh, mmh, that needs fixing then :D15:35
prashanthr_Good morning all :)15:35
*** reed has joined #openstack-marconi15:37
flaper87prashanthr_: gooooood morning15:40
prashanthr_flaper87: thank you :) I needed some help from you.15:41
prashanthr_r u free for a couple of minutes ?15:41
vkmcprashanthr_, hey you!15:45
*** flwang_ has joined #openstack-marconi15:45
prashanthr_vkmc: hello :)15:45
vkmcprashanthr_, try making a trail of gummybears15:45
vkmcand flaper87 will come15:46
flaper87got distracted15:46
flaper87prashanthr_: shoot15:46
prashanthr_ha ha vkmc :) there is flaper87 ! :)15:46
vkmcit worked :P15:47
prashanthr_perfect i recently discussed with kgriffs about integrating15:47
prashanthr_redis as a oslo cache15:47
*** rwsu has joined #openstack-marconi15:47
prashanthr_Then the oslo team pointed me to
prashanthr_this particular bluepring15:47
prashanthr_and asked me to create a spec15:48
*** flwang_ has quit IRC15:50
*** tonytan4_ has quit IRC15:52
prashanthr_flaper87: You there ?15:53
flaper87prashanthr_: I'm here15:54
flaper87prashanthr_: keep going15:54
* flaper87 gets distracted very easily15:54
prashanthr_he he15:54
prashanthr_so i created a spec15:54
flaper87mmh, what would the benefit of using oslo.cache instead of py-redis ?15:54
flaper87ah wait, you're just talking about the oslo.cache backend for redis15:55
flaper87you're not talking about marconi-redis15:55
prashanthr_it's a cache backend15:55
prashanthr_Right it's a cache backend15:55
flaper87keep going15:56
prashanthr_I have created a spec here
prashanthr_can you please check it when you are free ?15:56
prashanthr_I have marked the items where I had a question as #HELP15:57
flaper87prashanthr_: ooooooook, will do that for sure15:57
flaper87thanks for the heads up15:57
flaper87I'm glad you're working on that15:57
flaper87btw, speaking of redis, how's the redis storage coming along?15:57
prashanthr_The storage backend is coming along pretty well15:58
prashanthr_all the controllers15:58
prashanthr_are now being reviewed15:58
prashanthr_so just the unit tests left to be submitted for reviewing15:58
*** AAzza is now known as AAzza_afk16:00
*** prashanthr_ is now known as prashanthr_|afk16:06
*** mpanetta_ has joined #openstack-marconi16:07
*** mpanetta has quit IRC16:10
*** prashanthr_|afk has quit IRC16:14
vkmcalcabrera, around? :)16:35
*** reed has quit IRC16:35
vkmcor flaper87...?16:37
vkmcwell, if someone has a moment later16:37
*** reed has joined #openstack-marconi16:37
* flaper87 here16:38
vkmcI wanted to ask you guys how are we supporting the producer/consumer pattern in Marconi16:38
*** ykaplan has quit IRC16:38
vkmc:) yay, but I don't want to keep distracting you from your things16:39
flaper87vkmc: you mean in terms of transport?16:39
flaper87or API ?16:39
vkmcflaper87, in terms of API16:39
flaper87vkmc: oh please. Don't, ever, hesitate to ping16:39
flaper87vkmc: in terms of API, we do that by allowing clients to post messages at any time to some queue and then accessing those messages later on16:40
vkmcok, thanks :)16:40
flaper87vkmc: you know that already16:40
flaper87but I think you want something more specific16:40
vkmcflaper87, I do yeah16:40
flaper87so I probably didn't understand your question :P16:40
vkmcflaper87, yes... I'm thinking about the strict producer/consumer16:41
vkmcflaper87, we don't remove the message from the queue when the consumer gets the message16:41
vkmcso... our producer/consumer ends up being publisher/subscriber16:42
vkmcbecause later on another consumer can come and consume that message16:42
vkmcand that until someone deletes that message or it expires16:42
flaper87vkmc: ah I see where you're coming from with that16:43
flaper87so, the prod/cons pattern relies on the acknowledgement of the message being consumed16:44
flaper87so, prod-> cons -> ack16:44
vkmcflaper87, exactly16:44
flaper87vkmc: Without claiming/deleting the message, the user would be doing pub/sub as you said16:46
flaper87vkmc: so the answer to your question is: We support prod/cons through claims/acks16:46
flaper87Does that make sense?16:46
vkmcflaper87, it does yeah16:47
vkmcflaper87, one extra doubt... how do you do an ack using the HTTP API?16:47
alcabreravkmc: delete the message passing in the claim ID16:49
alcabreraalso, hello all16:49
alcabrerahttp delete /v1/queues/{queue}/messages/{msg_id}?claim_id={cid} -- is an ACK16:49
vkmcalcabrera, cool, so there is no prod/cons without claims :)16:50
vkmcflaper87, alcabrera, thanks guys16:50
vkmcwe should add this to the wiki... a Marconi for dummies section16:51
*** mpanetta has joined #openstack-marconi16:51
*** mpanetta_ has quit IRC16:55
alcabreravkmc: I can see a section being added to the wiki that serves as a codex for translating between amqp terminology/processes and Marconi/Naav's way of doing things. :)17:08
*** mpanetta_ has joined #openstack-marconi17:19
*** mpanetta has quit IRC17:23
*** mpanetta_ is now known as mpanetta17:23
srirammpanetta: hey!!17:34
mpanettasriram: Hi :)17:34
*** keith_newstadt has quit IRC17:36
*** keith_newstadt has joined #openstack-marconi17:36
*** Alex_Gaynor_ is now known as Alex_Gaynor17:37
*** Alex_Gaynor has quit IRC17:37
*** Alex_Gaynor has joined #openstack-marconi17:37
*** flwang_ has joined #openstack-marconi17:46
vkmcalcabrera, sounds really good :)17:48
*** flwang_ has quit IRC17:51
*** keith_newstadt has quit IRC17:58
*** ametts has joined #openstack-marconi18:03
*** mpanetta has quit IRC18:14
*** mpanetta has joined #openstack-marconi18:15
*** keith_newstadt has joined #openstack-marconi18:48
*** tedross has quit IRC18:58
*** oz_akan__ has joined #openstack-marconi19:05
*** chandan_kumar has quit IRC19:06
*** oz_akan_ has quit IRC19:09
*** oz_akan__ has quit IRC19:12
*** tedross has joined #openstack-marconi19:13
*** oz_akan_ has joined #openstack-marconi19:13
*** tedross has quit IRC19:31
*** tedross has joined #openstack-marconi19:46
*** flwang_ has joined #openstack-marconi19:47
*** keith_newstadt has quit IRC19:51
*** flwang_ has quit IRC19:52
*** keith_newstadt has joined #openstack-marconi19:52
*** amitgandhi has quit IRC19:53
*** malini is now known as malini|afk19:55
*** amitgandhi has joined #openstack-marconi20:03
*** amitgandhi has quit IRC20:03
*** amitgandhi has joined #openstack-marconi20:04
* vkmc lurks20:07
*** alcabrera is now known as alcabrera|afk20:25
*** sriram has quit IRC20:34
flwangflaper87: hey there20:36
flaper87flwang: yo20:36
flwangwhat's the time of this weekly meeting?20:36
vkmc15.00 utc?20:41
flaper87yeah, we need to share the times with other folks and we can start switching times next week20:42
flwangvkmc: thanks20:48
*** tedross has quit IRC20:51
vkmcflwang, np20:54
*** mpanetta has quit IRC21:05
*** flaper87 is now known as flaper87|afk21:17
*** amitgandhi has quit IRC22:02
*** oz_akan_ has quit IRC22:19
*** cpallares has quit IRC22:37
*** cpallares has joined #openstack-marconi22:40
*** vkmc has quit IRC22:59
*** cpallares has quit IRC23:08
*** Obulpathi has joined #openstack-marconi23:36
*** Obulpathi has quit IRC23:37
*** Obulpathi has joined #openstack-marconi23:37
*** Obulpathi has quit IRC23:45

Generated by 2.14.0 by Marius Gedminas - find it at!