| *** reed has quit IRC | 00:18 | |
| *** amitgandhi has quit IRC | 00:30 | |
| *** megan_w|afk is now known as megan_w | 00:32 | |
| *** megan_w is now known as megan_w|afk | 00:42 | |
| *** nosnos has joined #openstack-marconi | 01:37 | |
| *** flwang has quit IRC | 03:06 | |
| *** nosnos has quit IRC | 03:17 | |
| *** flwang has joined #openstack-marconi | 03:39 | |
| *** megan_w|afk is now known as megan_w | 03:52 | |
| *** nosnos has joined #openstack-marconi | 04:06 | |
| *** megan_w is now known as megan_w|afk | 04:23 | |
| *** chandankumar_ has joined #openstack-marconi | 04:23 | |
| *** chandankumar_ has quit IRC | 04:23 | |
| *** flwang has quit IRC | 04:30 | |
| *** flwang has joined #openstack-marconi | 04:31 | |
| openstackgerrit | Fei Long Wang proposed a change to openstack/marconi: Improve cleanup of test_insert_queue_header_asterisk https://review.openstack.org/83016 | 05:36 |
|---|---|---|
| openstackgerrit | Fei Long Wang proposed a change to openstack/marconi: Improve cleanup of test_insert_queue_header_asterisk https://review.openstack.org/83016 | 05:37 |
| chandan_kumar | flaper87|afk, heya! | 06:04 |
| *** flwang has quit IRC | 06:35 | |
| *** flwang has joined #openstack-marconi | 06:37 | |
| *** saju_m has joined #openstack-marconi | 06:38 | |
| *** flwang has quit IRC | 06:41 | |
| *** flwang has joined #openstack-marconi | 06:43 | |
| *** flwang has quit IRC | 07:02 | |
| *** AAzza has joined #openstack-marconi | 07:30 | |
| *** haomaiw__ has quit IRC | 07:33 | |
| *** jeremyh28 has joined #openstack-marconi | 08:01 | |
| *** flaper87|afk is now known as flaper87 | 08:05 | |
| *** jeremyh281 has joined #openstack-marconi | 08:05 | |
| *** jeremyh281 has left #openstack-marconi | 08:06 | |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Add a help text for auth_strategy https://review.openstack.org/83006 | 08:23 |
| flaper87 | chandan_kumar: yo yo | 08:24 |
| chandan_kumar | flaper87, how are you? | 08:26 |
| flaper87 | chandan_kumar: I'm doing fine, what about you? | 08:27 |
| chandan_kumar | flaper87, i am also good. . | 08:28 |
| chandan_kumar | flaper87, have a look this hilarious ad http://www.youtube.com/watch?v=eoCgjuAcvjg . | 08:28 |
| *** jeremyh28 has quit IRC | 08:32 | |
| *** saju_m has quit IRC | 09:05 | |
| *** saju_m has joined #openstack-marconi | 09:25 | |
| *** jamie_h_ has joined #openstack-marconi | 09:47 | |
| *** nosnos has quit IRC | 10:16 | |
| *** flaper87 is now known as flaper87|afk | 10:18 | |
| *** flaper87|afk is now known as flaper87 | 10:42 | |
| *** AAzza has left #openstack-marconi | 10:54 | |
| *** tedross has joined #openstack-marconi | 11:48 | |
| *** saju_m has quit IRC | 11:50 | |
| *** oz_akan_ has joined #openstack-marconi | 12:24 | |
| *** mwagner_bbl has quit IRC | 12:35 | |
| *** sriram has joined #openstack-marconi | 12:36 | |
| *** AAzza has joined #openstack-marconi | 12:50 | |
| *** chandan_kumar has quit IRC | 12:51 | |
| *** cyrichardson has joined #openstack-marconi | 13:11 | |
| *** cpallares has joined #openstack-marconi | 13:14 | |
| *** saju_m has joined #openstack-marconi | 13:15 | |
| *** amitgandhi has joined #openstack-marconi | 13:21 | |
| *** oz_akan_ has quit IRC | 13:22 | |
| *** amitgandhi has quit IRC | 13:22 | |
| *** oz_akan_ has joined #openstack-marconi | 13:22 | |
| *** amitgandhi has joined #openstack-marconi | 13:23 | |
| *** mpanetta has joined #openstack-marconi | 13:27 | |
| *** vkmc has joined #openstack-marconi | 13:33 | |
| *** vkmc has quit IRC | 13:33 | |
| *** vkmc has joined #openstack-marconi | 13:33 | |
| *** mpanetta_ has joined #openstack-marconi | 13:38 | |
| *** mpanetta has quit IRC | 13:38 | |
| *** flwang has joined #openstack-marconi | 13:42 | |
| *** malini_afk is now known as malini | 13:46 | |
| *** mpanetta has joined #openstack-marconi | 13:49 | |
| *** balajiiyer has joined #openstack-marconi | 13:51 | |
| *** Kuwagata has joined #openstack-marconi | 13:51 | |
| *** mpanetta_ has quit IRC | 13:52 | |
| *** mwagner_bbl has joined #openstack-marconi | 13:53 | |
| malini | Happy Friday!! | 14:00 |
| malini | I just noticed this in the ML https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg19447.html | 14:00 |
| *** jmckind has joined #openstack-marconi | 14:00 | |
| malini | Might be a great idea for Marconi to be represented | 14:07 |
| malini | We 'll get validation from users | 14:07 |
| alcabrera|zz | good morning, all. :) | 14:08 |
| *** alcabrera|zz is now known as alcabrera | 14:08 | |
| *** russellb is now known as rustlebee | 14:08 | |
| malini | hello alcabrera | 14:11 |
| alcabrera | malini: +1 about that | 14:11 |
| alcabrera | ops at every session makes sense, since a system like OpenStack is pretty much deployer-driven | 14:12 |
| alcabrera | or should be | 14:12 |
| cpallares | malini, alcabrera: o/ | 14:13 |
| malini | good morning cpallares! | 14:13 |
| alcabrera | cpallares: hey! :D | 14:14 |
| openstackgerrit | Oz Akan proposed a change to openstack/marconi: keystone middleware version change due to a bug https://review.openstack.org/83757 | 14:17 |
| *** openstack has joined #openstack-marconi | 14:21 | |
| alcabrera | oz_akan_: thanks | 14:24 |
| alcabrera | cpallares: how go things? | 14:24 |
| cpallares | alcabrera, malini: Good morning! | 14:24 |
| cpallares | alcabrera: Pretty good and you? I slept so well last night. I was so tired yesterday. | 14:25 |
| alcabrera | cpallares: I'm great! Glad to hear you slept well. What got you so tired in the first place? Too much work/travel? :) | 14:26 |
| cpallares | alcabrera: Well I didn't sleep very well in Boston since I would get up really early (plus the hour difference) and I would go back really late since there were a lot of events happening at night. That plus the 4-5+ hours walking everyday exploring the city was exhausting. | 14:28 |
| alcabrera | ah, makes perfect sense. | 14:28 |
| cpallares | alcabrera: These past couple of days I haven't been sleeping well because I guess I'm still on that schedule :P | 14:28 |
| alcabrera | travel usually kills me, since I get my sleep all out of whack and then it's even harder to sleep because of the excitement of the travel itself | 14:28 |
| cpallares | alcabrera: yeah! I love being in new cities :) | 14:29 |
| cpallares | alcabrera: Sleep can wait till I get back to boring old home. | 14:29 |
| alcabrera | hahaha | 14:29 |
| alcabrera | I'm roughly like that. :P | 14:30 |
| alcabrera | Too many people to see, too many foods to eat, too many Tweets to Tweet | 14:30 |
| alcabrera | no time for sleep | 14:30 |
| malini | Nothing can stop me from sleeping :D | 14:30 |
| malini | That makes me a good traveller | 14:31 |
| cpallares | haha malini. I'm that way at home :P | 14:31 |
| alcabrera | and thusly, the world was balanced. malini will sleep for all of us, and in microincrements, we shall all sleep enough to offset the sleep that flaper87 doesn not get | 14:31 |
| alcabrera | *does not | 14:31 |
| cpallares | hahaha | 14:31 |
| * cpallares wonders if flaper87 even sleeps | 14:31 | |
| flaper87 | alcabrera: :D :D :D | 14:31 |
| * flaper87 doesn't sleep | 14:31 | |
| malini | Somebody needs to bear the brunt for the team ..I volunteer to do that ;) | 14:31 |
| alcabrera | so generous, malini. :D | 14:32 |
| malini | I am always generous like that ;) | 14:33 |
| cpallares | haha | 14:33 |
| malini | flaper87: did you see my post above from the ML ? | 14:33 |
| alcabrera | cpallares: I've got an awesome math article for you: http://www.science4all.org/le-nguyen-hoang/type-theory/ (math refactored: set theory -> type theory) | 14:35 |
| flaper87 | malini: nope, what happened ? | 14:37 |
| flaper87 | oh mmh | 14:37 |
| flaper87 | interesting | 14:37 |
| cpallares | alcabrera: :D | 14:38 |
| *** rwsu has quit IRC | 14:38 | |
| malini | flaper87: I think it'll be a great opportunity for Marconi | 14:38 |
| malini | We can get feedback from users + convince the skeptics that we are solving problems with real use cases | 14:38 |
| sriram | indeed. | 14:39 |
| *** rwsu has joined #openstack-marconi | 14:41 | |
| *** megan_w|afk is now known as megan_w | 14:43 | |
| openstackgerrit | A change was merged to openstack/marconi: keystone middleware version change due to a bug https://review.openstack.org/83757 | 14:48 |
| flaper87 | people, we should really add new design session proposals asap | 14:57 |
| flaper87 | alcabrera: oz_akan_ amitgandhi ^^^^^ | 14:57 |
| flaper87 | I know you guys had some ideas | 14:57 |
| alcabrera | hmmm | 14:58 |
| alcabrera | link, flaper87? | 14:58 |
| flaper87 | http://summit.openstack.org/ | 14:58 |
| alcabrera | thanks! | 14:58 |
| alcabrera | I don't remember having any recent ideas, but perhaps scanning the current proposals will inspire me. ;) | 14:58 |
| flaper87 | there are 4 proposed sessions, we'll likely merge some of those | 15:00 |
| flaper87 | we've space for just 4 sessions | 15:00 |
| amitgandhi | i think my ideas were around showcasing customers, and production stats - it looks like its covered by malini's proposal http://summit.openstack.org/cfp/details/135 | 15:01 |
| *** jcru has joined #openstack-marconi | 15:01 | |
| amitgandhi | are design sessions 1 hr? | 15:02 |
| amitgandhi | i wonder if its too much info for one session and should be broken up | 15:02 |
| malini | feel free to edit mine (I dont know if you can) | 15:02 |
| alcabrera | 173 sessions in all proposed on that page so far | 15:04 |
| *** zyuan has joined #openstack-marconi | 15:04 | |
| alcabrera | fmap (length . lines) $ readFile "sessions.txt" | 15:04 |
| alcabrera | taking a look, I'm pretty happy with the proposed sessions for Marconi | 15:07 |
| alcabrera | I'm thinking... "What gap have we left uncovered...?" | 15:07 |
| alcabrera | I suppose an obvious one would be storage backends | 15:07 |
| alcabrera | and what the plans are for supporting partially-compatible backends | 15:08 |
| *** zyuan has quit IRC | 15:08 | |
| alcabrera | e.g., amqp being able to implement most of the API, but not all | 15:08 |
| alcabrera | Other gaps... | 15:08 |
| alcabrera | API Evolution | 15:08 |
| *** reed has joined #openstack-marconi | 15:08 | |
| alcabrera | What's coming for 2.0 and 1.1? | 15:08 |
| *** zyuan has joined #openstack-marconi | 15:09 | |
| *** zyuan has quit IRC | 15:09 | |
| alcabrera | I think devs/ops/users would have something to say on that, and we'd be able to cover a lot of feedback quickly /cc kgriffs|afk | 15:09 |
| *** zyuan has joined #openstack-marconi | 15:09 | |
| *** zyuan has quit IRC | 15:10 | |
| alcabrera | Benchmarking for Marconi might be another interesting one -- what would people consider a worthwhile benchmark of a queuing system? | 15:10 |
| alcabrera | e.g., 3DMark : grapgics engines :: QueueBench : Queue Systems | 15:10 |
| *** zyuan has joined #openstack-marconi | 15:10 | |
| alcabrera | *graphics | 15:12 |
| *** zyuan has quit IRC | 15:12 | |
| amitgandhi | alcabrera: rather than asking what might be worthwhile benchmarking, i think a better topic would be what kind of performance do users expect and why | 15:13 |
| amitgandhi | as that is why ppl benchmark - to see if it meets their needs | 15:14 |
| alcabrera | hmmm | 15:14 |
| malini | tj*** already asked for the benchmarks in ML | 15:14 |
| malini | We shpuld just run a benchmark of whatever we have now & start the conversation on what else ppl want to see | 15:15 |
| amitgandhi | yeh its something that should happen in the ML | 15:15 |
| alcabrera | amitgandhi: noted. However, given that these are design sessions, how would founding a design session on the question "what performance levels do you expect of your queuing system?" influence concrete design proposals? | 15:16 |
| mpanetta | One thing that may help, is to split the benchmarks up by operation... Queue Creation/Deletion for instance does not need to be as performant as posts/claims/deletes | 15:16 |
| mpanetta | Just a suggestion... | 15:16 |
| amitgandhi | how does the method of benchmarking help design of marconi? | 15:16 |
| malini | We can also get answers from the operators during this session https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg19447.html | 15:17 |
| malini | I added Marconi as an interested project in etherpad | 15:17 |
| malini | mpanetta: +1 | 15:17 |
| mpanetta | I keep seeing people talking about benchmarks, but not talking about what we are benchmarking... Saying that we are 'benchmarking marconi' is kind of nebulous... | 15:17 |
| amitgandhi | i almost feel like we need to present our benchmark findings (on the ML) and the design session should be "is this acceptab;e | 15:18 |
| malini | We should just point them to our deployment & benchmark configs along with the results we have | 15:18 |
| amitgandhi | what kind of creation time are you looking for | 15:18 |
| malini | So ppl can play with it & reach their own conclusions | 15:18 |
| amitgandhi | what kind of read perf are u looking for | 15:18 |
| amitgandhi | why do you need that perf | 15:18 |
| amitgandhi | how can we meet that need? | 15:18 |
| *** zyuan has joined #openstack-marconi | 15:19 | |
| *** zyuan has joined #openstack-marconi | 15:19 | |
| amitgandhi | and that will drive the importance of things like redis | 15:20 |
| alcabrera | hmmm | 15:20 |
| amitgandhi | we can discuss redis with them | 15:20 |
| amitgandhi | will that meet their need | 15:21 |
| amitgandhi | are there other issues with that | 15:21 |
| amitgandhi | that we didnt think of | 15:21 |
| amitgandhi | maybe cust perf needs vary a lot | 15:21 |
| amitgandhi | s/vary/varie ? | 15:21 |
| alcabrera | how do we frame that as something actionable? The problem with asking "How much performance do you need for these operations?" is that we'll either get "as fast as you can go" or "as fast as my existing system" | 15:23 |
| amitgandhi | the question then becomes what is your existing system, and does marconi meet that need | 15:23 |
| amitgandhi | why do you need that performace | 15:23 |
| amitgandhi | is it realistic for marconi to reach that performance | 15:23 |
| amitgandhi | what can marconi achieve | 15:23 |
| amitgandhi | the actionable items become what storage drivers do we need to address | 15:24 |
| amitgandhi | is amqp really important | 15:24 |
| *** ayoung-afk is now known as ayoung | 15:24 | |
| amitgandhi | or is redis important | 15:24 |
| amitgandhi | or something else | 15:24 |
| amitgandhi | what perf do our users expect us to achieve | 15:24 |
| amitgandhi | it gives us something to aim for | 15:25 |
| amitgandhi | it tells us what our users want | 15:25 |
| amitgandhi | (that marconi aims to solve for) | 15:25 |
| flaper87 | sorry, I keep getting pulled into other things but I wanted to say that we shouldn't think that we are "fast enough" | 15:25 |
| flaper87 | we should always try to be faster | 15:25 |
| amitgandhi | flaper87: +1 | 15:25 |
| flaper87 | regardless we've met the user's need or not | 15:25 |
| amitgandhi | but we should know if we are even in the ballpark | 15:25 |
| alcabrera | that's part of the reason I feel that the performance discussion won't be fruitful | 15:25 |
| amitgandhi | ie user expects usec, and we are msec | 15:26 |
| alcabrera | designing a means to measure it *is* | 15:26 |
| flaper87 | We can know that by comparing marconi's performance with other queuing services /technologies | 15:26 |
| malini | alcabrera: & We should make it easy for anybody who wants to benchmark | 15:26 |
| flaper87 | (note the difference services AND technologies) | 15:26 |
| flaper87 | SQS and qpid / rabbitmq, that is | 15:26 |
| alcabrera | definitely, flaper87 | 15:27 |
| flaper87 | whether Marconi will ever be as fast as rabbit or not is not under discussion | 15:27 |
| malini | Give them the deployment/benchmrk scripts | 15:27 |
| flaper87 | but it should *always* try to get there | 15:27 |
| flaper87 | So the closer we are to qpid / rabbit / zmq / SQS / SNS / IronMQ the better | 15:27 |
| alcabrera | maybe I'll write queue bench -- asynchronous/distributed, overload a typeclass to handle new service/techs, give it a config file, and take down a queueing system | 15:28 |
| flaper87 | It really depends on what storage Marconi is using and that makes it *really* important to have bench tables w/ different storage drivers and transprots | 15:28 |
| alcabrera | *queuing | 15:28 |
| flaper87 | transports* | 15:28 |
| flaper87 | :P | 15:29 |
| alcabrera | :) | 15:29 |
| alcabrera | queuebench marconi_bench.conf --(html, txt, json) > report.(html, txt, json) | 15:30 |
| alcabrera | queuebench sqs_bench.conf | 15:30 |
| alcabrera | queuebench ironio_bench.conf | 15:30 |
| flaper87 | alcabrera: that sounds really interesting | 15:30 |
| alcabrera | cabal install queuebench | 15:31 |
| flaper87 | alcabrera: TBH, we could start by just benching marconi and coparing it to whatever bench exists out there for those services | 15:31 |
| flaper87 | alcabrera: LOL | 15:31 |
| alcabrera | I'll see what I can do on the next rax:hackday | 15:31 |
| alcabrera | haha | 15:31 |
| alcabrera | flaper87: definitely. I know we've used tsung extensively for determining how rax:marconi behaves under load. I wonder what other benchmarking efforts have been done with queue systems? :) | 15:32 |
| *** zyuan has left #openstack-marconi | 15:34 | |
| *** saju_m has quit IRC | 15:35 | |
| flaper87 | alcabrera: that's a really good question. TBFH, I'm not sure if there are "standard" benchmarks for queuing systems | 15:36 |
| * flaper87 googles | 15:36 | |
| alcabrera | that'd be pretty cool if there were | 15:36 |
| alcabrera | as far as I've seen, it's a lot of ad hoc measurements | 15:37 |
| alcabrera | at various levels of system load | 15:37 |
| alcabrera | with latency expressed as averages with stddevs at various load points | 15:37 |
| *** reed has quit IRC | 15:54 | |
| *** AAzza has quit IRC | 16:18 | |
| *** reed has joined #openstack-marconi | 16:32 | |
| oz_akan_ | about benchmarking, marconi can be really fast if you give enough hardware and fast network | 16:33 |
| oz_akan_ | I think the challenge is reducing the cost | 16:33 |
| oz_akan_ | and "fast" is so arguable... fast for what type of application | 16:34 |
| oz_akan_ | fast for writes or reads.. | 16:34 |
| oz_akan_ | anyone in operations knows not to rely on any published benchmarks. people would like to do their own benchmarks | 16:34 |
| alcabrera | oz_akan_: +1 | 16:34 |
| oz_akan_ | so we may try to provide what affects performance how | 16:35 |
| oz_akan_ | more shards, number of replicas, web servers etc.. | 16:35 |
| oz_akan_ | number of uwsgi instances | 16:35 |
| oz_akan_ | kernel parameters | 16:35 |
| oz_akan_ | then anyone can run his benchmark with his workload | 16:35 |
| flaper87 | oz_akan_: that's exactly the issue and you detailed it very well | 16:39 |
| flaper87 | normally, queuing systems are benched under different scenarios | 16:39 |
| flaper87 | the common scenarios are (at least the ones I'm familiar with) | 16:39 |
| flaper87 | Tons of posts with small / big messages (no reads) | 16:39 |
| flaper87 | Tons of reads with small / big messages (no writeS) | 16:40 |
| flaper87 | and concurrent writes / reads of small / big messages | 16:40 |
| flaper87 | obviously, The hardware, network and environment has to be the same | 16:40 |
| flaper87 | We should always try to make it as fast as possible under the worst environment | 16:41 |
| flaper87 | That is not much ram nor CPU | 16:41 |
| oz_akan_ | that is for pub-sub test case I guess | 16:41 |
| flaper87 | well, no, producer and consumer too. | 16:41 |
| flaper87 | why would it be just for pub-sub ? | 16:42 |
| oz_akan_ | you wrote no writes for reads test | 16:42 |
| oz_akan_ | with claims you have to read and write | 16:42 |
| flaper87 | but you can also read without claiming | 16:42 |
| oz_akan_ | that is pub - sub | 16:42 |
| flaper87 | why? | 16:43 |
| flaper87 | you're still consuming the message | 16:43 |
| oz_akan_ | claim is for producer - consumer | 16:43 |
| oz_akan_ | so there is only one consumer | 16:43 |
| oz_akan_ | pub - sub, there may be many subscribers | 16:44 |
| *** rossk has joined #openstack-marconi | 16:46 | |
| flaper87 | I see what you mean, however that's not exactly what makes it pub-sub. We could have prod-consumer without using claims. What claims bring is the guarantee that a message will be delirvered once. | 16:49 |
| flaper87 | btw, I didn't mean to be pedantic, sorry if that sounded a bit jerkish | 16:49 |
| flaper87 | but yeah, we'd likely use claims for the prod / consumer test | 16:50 |
| flaper87 | and well, by writes I meant message post | 16:50 |
| *** rossk has quit IRC | 16:51 | |
| *** rossk has joined #openstack-marconi | 16:51 | |
| oz_akan_ | what makes it pub-sub? | 16:59 |
| oz_akan_ | since marconi doesn't really have subscription? | 17:00 |
| *** jmckind has quit IRC | 17:03 | |
| oz_akan_ | flaper87: ^ | 17:05 |
| *** megan_w is now known as megan_w|afk | 17:09 | |
| oz_akan_ | can someone tell me what is about summit.openstack.org ? | 17:13 |
| alcabrera | oz_akan_: it's the design summit. My understanding is that marconi will get four 1-hour slots | 17:14 |
| alcabrera | oz_akan_: used for the purpose of discussing future design decisions with the rest of the OS community | 17:14 |
| amitgandhi | subscriber semantics for marconi is i have access to the queue and will just subscribe (read) messages, not claim/delete them | 17:15 |
| amitgandhi | subscribers keep track of the last seen id, so they dont read ones they already have | 17:15 |
| amitgandhi | and eventually those messages expire and disappear | 17:15 |
| oz_akan_ | amitgandhi: check flaper87 s response | 17:16 |
| oz_akan_ | how producer - consumer work without claiming messages, I couldn't get tat | 17:16 |
| oz_akan_ | that | 17:16 |
| amitgandhi | hmmmm.... | 17:17 |
| amitgandhi | i always think of producer-consumer as claiming | 17:17 |
| oz_akan_ | alcabrera: so we have 4 proposals, should we have just one, or is it good to have many? | 17:17 |
| amitgandhi | if a consumer isnt claiming, arent they just a subscriber? | 17:17 |
| oz_akan_ | amitgandhi: I would say so | 17:18 |
| amitgandhi | flaper87: ^^ | 17:18 |
| malini | oz_akan_: we have 4 slots & would want to use all of those | 17:18 |
| alcabrera | amitgandhi: producer-consumer is not dependent on claiming. Claiming is only necessary for once-and-only-once delivery. | 17:18 |
| amitgandhi | flaper87: can you please elaborate on why a consumer who doesnt claim is still a consumer and not a subscriber? | 17:18 |
| oz_akan_ | alcabrera: ok so you don't claim what do you do | 17:18 |
| * flaper87 back | 17:18 | |
| amitgandhi | then arent they just a subscriber if they dont claim | 17:19 |
| flaper87 | so, yeah, it can get very confusing in Marconi because there's basically no theoretical separation of those patterns in the way marocni does things | 17:19 |
| *** saju_m has joined #openstack-marconi | 17:19 | |
| amitgandhi | to me, publisher = producer. consumer is one who claims, subscriber is one who just reads | 17:20 |
| alcabrera | you read, possibly repeating tasks off the front of the producer queue, oz_akan_ - you may have duplicate processing, which might be okay/necessary if your producer-consumer model utilizes redundant processing | 17:20 |
| flaper87 | however, theoretically speaking, a pub-sub is works in a way that messages are produced without worrying who'll consume them and then there are several subscribers streaming those messages out | 17:20 |
| oz_akan_ | alcabrera: that is pub - sub, you may read more than once, a message might be for many workers | 17:20 |
| flaper87 | the big difference is that in a pub-sub fashiong the subscriber needs to keep track of the messages it already processed and move forward | 17:20 |
| flaper87 | the consumer doesn't have to do that | 17:21 |
| flaper87 | a consumer would get the message, do something, delete the message and get another one | 17:21 |
| amitgandhi | correct. consumer doesnt have to do that since they are claiming messages, hence messages read go away | 17:21 |
| flaper87 | the subscriber gets a set of messages, keeps track of the last processed and get another set starting from last+1 | 17:21 |
| amitgandhi | so explain to me a consumer who doesnt claim | 17:21 |
| oz_akan_ | flaper87: could you tell me how producer-consumer pattern happens without claiming | 17:21 |
| flaper87 | but the consumer doesn't need claims, claims are just a way to guarantee once and only once | 17:22 |
| amitgandhi | so how is that diff to a subscriber | 17:22 |
| flaper87 | just like I explained, without claims, though, the once and only once delivery is not guaranteed | 17:22 |
| oz_akan_ | :D | 17:22 |
| flaper87 | the difference is that the subscriber doesn't delete the message from the queue | 17:22 |
| amitgandhi | so a consumer who doesnt claim and keeps track of id === a subscriber | 17:22 |
| flaper87 | it just process the message and moves on | 17:22 |
| amitgandhi | a consumer who doesnt claim, and doesnt track id != subscriber ?? | 17:22 |
| *** megan_w|afk is now known as megan_w | 17:23 | |
| flaper87 | the consumer doesn't need to keep track of the id | 17:23 |
| flaper87 | the consumer gets the message, deletes it and moves on | 17:23 |
| flaper87 | a subscriber gets a message and moves on | 17:23 |
| flaper87 | (by keeping track of the id and getting a new one with id+1) | 17:24 |
| alcabrera | the semantics of messaging patterns are pretty cool. They're subtle, too. :) | 17:24 |
| alcabrera | there's so many that aren't encoded in Marconi | 17:24 |
| amitgandhi | so if a consumer gets a message, deletes it and moves on (without claiming), then another consumer can also work on it, until consumer1 deletes it | 17:24 |
| amitgandhi | i cant imagine why anyone would do that | 17:25 |
| oz_akan_ | +1 | 17:25 |
| flaper87 | amitgandhi: I'm not saying it's the right thing to do | 17:25 |
| flaper87 | I'm saying that *claims* don't define what prod-consumer is | 17:25 |
| flaper87 | What defines producer and consumer is the way you produce and consume messages | 17:25 |
| oz_akan_ | flaper87: do you mean it for marconi? | 17:25 |
| oz_akan_ | are there different ways to produce messages? | 17:26 |
| oz_akan_ | I think I just understand these as much as they are implemented in marconi | 17:26 |
| oz_akan_ | that might be the problem | 17:26 |
| flaper87 | oz_akan_: no differences in how messages are produced. There are some that think that in a prod-consumer pattern the producer *may* know who will consume the message but that's more a req-resp pattern -which we don't support) | 17:27 |
| oz_akan_ | if there are multiple costumers of a message, it sounds like they are subscribers | 17:27 |
| oz_akan_ | anyway, let me read 2 books on messaging patterns before I say anything further :) | 17:27 |
| amitgandhi | ive always thought of consumer as claiming. if amit consumes a coke, then no one else can have that coke | 17:29 |
| amitgandhi | i guess you can have amit consumes a coke while everyone else watches? | 17:29 |
| amitgandhi | but then im still the only consumer and everyone else is a subscriber | 17:29 |
| * amitgandhi takes another sip of his coke | 17:30 | |
| * alcabrera duplicates the coke and passes it on the amitgandhi-2 | 17:30 | |
| *** saju_m has quit IRC | 17:35 | |
| *** saju_m has joined #openstack-marconi | 17:36 | |
| *** cpallares has quit IRC | 17:41 | |
| *** Kuwagata has quit IRC | 18:24 | |
| *** Kuwagata has joined #openstack-marconi | 18:31 | |
| *** jamie_h_ has quit IRC | 18:35 | |
| *** openstackgerrit has quit IRC | 18:48 | |
| *** openstackgerrit has joined #openstack-marconi | 18:48 | |
| *** jamie_h_ has joined #openstack-marconi | 19:06 | |
| *** malini has left #openstack-marconi | 19:11 | |
| *** prashanthr_ has quit IRC | 19:24 | |
| *** Kuwagata has quit IRC | 19:27 | |
| *** prashanthr_ has joined #openstack-marconi | 19:31 | |
| *** prashanthr_ has quit IRC | 19:39 | |
| *** megan_w is now known as megan_w|afk | 20:00 | |
| *** alcabrera is now known as alcabrera|zz | 20:08 | |
| *** jamie_h_ has quit IRC | 20:09 | |
| *** vkmc has quit IRC | 20:12 | |
| *** prashanthr_ has joined #openstack-marconi | 20:13 | |
| *** metral has quit IRC | 20:15 | |
| *** metral has joined #openstack-marconi | 20:16 | |
| *** metral has quit IRC | 20:22 | |
| *** metral has joined #openstack-marconi | 20:22 | |
| *** metral has quit IRC | 20:22 | |
| *** metral has joined #openstack-marconi | 20:23 | |
| *** oz_akan_ has quit IRC | 20:24 | |
| *** metral has quit IRC | 20:26 | |
| *** metral has joined #openstack-marconi | 20:26 | |
| *** metral has quit IRC | 20:28 | |
| *** prashanthr_ has quit IRC | 20:28 | |
| *** metral has joined #openstack-marconi | 20:28 | |
| *** sriram has quit IRC | 20:30 | |
| *** metral has quit IRC | 20:33 | |
| *** metral has joined #openstack-marconi | 20:34 | |
| *** prashanthr_ has joined #openstack-marconi | 20:35 | |
| *** metral has quit IRC | 20:37 | |
| *** mwagner_bbl has quit IRC | 20:37 | |
| *** prashanthr_ has quit IRC | 20:41 | |
| *** prashanthr_ has joined #openstack-marconi | 20:46 | |
| *** metral has joined #openstack-marconi | 20:48 | |
| *** balajiiyer has quit IRC | 20:49 | |
| *** metral has quit IRC | 20:54 | |
| *** metral has joined #openstack-marconi | 20:58 | |
| *** mpanetta has quit IRC | 20:59 | |
| *** amitgandhi has quit IRC | 21:01 | |
| *** metral has quit IRC | 21:02 | |
| *** metral has joined #openstack-marconi | 21:02 | |
| *** metral has quit IRC | 21:06 | |
| *** metral has joined #openstack-marconi | 21:07 | |
| *** jergerber has joined #openstack-marconi | 21:08 | |
| *** metral has quit IRC | 21:18 | |
| *** prashanthr_ has quit IRC | 21:18 | |
| *** cyrichardson has quit IRC | 21:20 | |
| *** tedross has quit IRC | 21:23 | |
| *** prashanthr_ has joined #openstack-marconi | 21:24 | |
| *** prashanthr_ has quit IRC | 21:32 | |
| *** prashanthr_ has joined #openstack-marconi | 21:37 | |
| *** prashanthr_ has quit IRC | 21:40 | |
| *** prashanthr_ has joined #openstack-marconi | 21:42 | |
| *** mwagner_bbl has joined #openstack-marconi | 22:11 | |
| *** vkmc has joined #openstack-marconi | 22:55 | |
| *** jergerber has quit IRC | 22:57 | |
| *** jcru has quit IRC | 23:13 | |
| *** rossk has quit IRC | 23:40 | |
| *** vkmc has quit IRC | 23:52 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!