| *** sriram has joined #openstack-marconi | 00:50 | |
| *** prashanthr_ has left #openstack-marconi | 01:13 | |
| *** oz_akan__ has joined #openstack-marconi | 01:38 | |
| *** nosnos has joined #openstack-marconi | 01:38 | |
| *** oz_akan_ has quit IRC | 01:40 | |
| *** sriram has quit IRC | 02:50 | |
| *** rwsu has quit IRC | 03:10 | |
| *** vkmc has quit IRC | 03:23 | |
| *** nosnos has quit IRC | 03:29 | |
| *** haomaiwang has joined #openstack-marconi | 03:40 | |
| *** haomaiwang has quit IRC | 03:48 | |
| *** haomaiwang has joined #openstack-marconi | 03:48 | |
| *** haomai___ has joined #openstack-marconi | 04:05 | |
| *** haomaiwang has quit IRC | 04:05 | |
| *** oz_akan__ has quit IRC | 04:10 | |
| *** haomaiwang has joined #openstack-marconi | 04:21 | |
| *** haomai___ has quit IRC | 04:23 | |
| *** haomaiwa_ has joined #openstack-marconi | 04:25 | |
| *** haomaiwang has quit IRC | 04:25 | |
| *** nosnos has joined #openstack-marconi | 04:26 | |
| *** haomai___ has joined #openstack-marconi | 04:34 | |
| *** haomaiwa_ has quit IRC | 04:36 | |
| *** oz_akan_ has joined #openstack-marconi | 04:41 | |
| *** oz_akan_ has quit IRC | 04:46 | |
| *** shakamunyi has joined #openstack-marconi | 05:04 | |
| *** AAzza_afk is now known as AAzza | 05:22 | |
| *** flwang has quit IRC | 05:40 | |
| *** oz_akan_ has joined #openstack-marconi | 05:42 | |
| *** oz_akan_ has quit IRC | 05:46 | |
| *** haomai___ has quit IRC | 05:49 | |
| *** haomaiwa_ has joined #openstack-marconi | 05:49 | |
| *** AAzza is now known as AAzza_afk | 05:59 | |
| *** AAzza_afk is now known as AAzza | 06:17 | |
| *** mkoderer has joined #openstack-marconi | 06:20 | |
| *** jamie_h has joined #openstack-marconi | 06:32 | |
| *** oz_akan_ has joined #openstack-marconi | 06:43 | |
| *** oz_akan_ has quit IRC | 06:47 | |
| *** AAzza is now known as AAzza_afk | 07:08 | |
| *** oz_akan_ has joined #openstack-marconi | 07:43 | |
| *** renlt has joined #openstack-marconi | 07:46 | |
| *** oz_akan_ has quit IRC | 07:48 | |
| *** shakamunyi has quit IRC | 08:10 | |
| *** lvh_ is now known as lvh | 08:19 | |
| *** flaper87|afk is now known as flaper87 | 08:35 | |
| *** jamie_h has quit IRC | 08:48 | |
| *** mkoderer has quit IRC | 08:48 | |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** mkoderer has joined #openstack-marconi | 08:48 | |
| *** jamie_h has joined #openstack-marconi | 08:49 | |
| *** openstackgerrit has joined #openstack-marconi | 08:50 | |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Rename shards to pool https://review.openstack.org/96463 | 09:30 |
|---|---|---|
| *** oz_akan_ has joined #openstack-marconi | 09:45 | |
| *** oz_akan__ has joined #openstack-marconi | 09:47 | |
| *** oz_akan_ has quit IRC | 09:47 | |
| *** oz_akan__ has quit IRC | 09:51 | |
| *** wirehead_ has quit IRC | 09:52 | |
| *** renlt has quit IRC | 09:59 | |
| *** haomaiwa_ has quit IRC | 10:05 | |
| *** haomaiwang has joined #openstack-marconi | 10:06 | |
| *** nosnos has quit IRC | 10:08 | |
| *** haomai___ has joined #openstack-marconi | 10:09 | |
| *** haomaiwang has quit IRC | 10:12 | |
| *** wirehead_ has joined #openstack-marconi | 10:13 | |
| *** haomai___ has quit IRC | 10:24 | |
| *** haomaiwang has joined #openstack-marconi | 10:25 | |
| *** shakamunyi has joined #openstack-marconi | 10:37 | |
| *** oz_akan_ has joined #openstack-marconi | 10:38 | |
| *** shakamunyi has quit IRC | 10:41 | |
| *** oz_akan_ has quit IRC | 10:43 | |
| *** torgomatic has quit IRC | 10:48 | |
| *** torgomatic has joined #openstack-marconi | 10:55 | |
| *** flaper87 is now known as flaper87|afk | 11:25 | |
| *** oz_akan_ has joined #openstack-marconi | 11:39 | |
| *** oz_akan_ has quit IRC | 11:44 | |
| *** vkmc has joined #openstack-marconi | 12:05 | |
| *** vkmc has quit IRC | 12:05 | |
| *** vkmc has joined #openstack-marconi | 12:05 | |
| *** haomaiwang has quit IRC | 12:40 | |
| *** haomaiwang has joined #openstack-marconi | 12:40 | |
| openstackgerrit | Alex Bettadapur proposed a change to openstack/marconi: Decoupled Unit Tests https://review.openstack.org/97534 | 12:40 |
| *** jchai has joined #openstack-marconi | 12:43 | |
| *** abettadapur has joined #openstack-marconi | 12:44 | |
| *** sriram has joined #openstack-marconi | 12:45 | |
| *** haomaiw__ has joined #openstack-marconi | 12:46 | |
| abettadapur | sriram: i submitted another patch if you want to take a look | 12:46 |
| sriram | abettadapur: awesome, let me check. | 12:46 |
| *** Obulpathi has joined #openstack-marconi | 12:48 | |
| *** haomaiwang has quit IRC | 12:49 | |
| *** abettadapur has quit IRC | 12:55 | |
| *** Obulpathi has quit IRC | 13:04 | |
| *** abettadapur has joined #openstack-marconi | 13:09 | |
| *** jmckind has joined #openstack-marconi | 13:11 | |
| *** shakamunyi has joined #openstack-marconi | 13:17 | |
| openstackgerrit | Alex Gaynor proposed a change to openstack/marconi: Removed now unnecesary workaround for PyPy https://review.openstack.org/97074 | 13:24 |
| *** oz_akan_ has joined #openstack-marconi | 13:24 | |
| *** Obulpathi has joined #openstack-marconi | 13:31 | |
| sriram | kgriffs|afk: ping | 13:31 |
| *** balajiiyer has joined #openstack-marconi | 13:53 | |
| *** mwagner_lap has joined #openstack-marconi | 13:53 | |
| *** flaper87|phone has joined #openstack-marconi | 13:55 | |
| sriram | flaper87|phone: Is there a way to check response for deleting/claiming messages in the client? | 14:05 |
| sriram | deserealized content just returns None, would it require another patch? | 14:05 |
| *** alcabrera|afk is now known as alcabrera | 14:06 | |
| flaper87|phone | You should be able to know the result based on the response code | 14:06 |
| flaper87|phone | I don't have a link handy but there should be some info in the wiki | 14:06 |
| flaper87|phone | (Sorry) :D | 14:07 |
| flaper87|phone | As you may have figured, I'm using my phone | 14:07 |
| sriram | hmm, msg.delete() returns None, there is no response object. | 14:07 |
| flaper87|phone | Hahahaha | 14:07 |
| sriram | flaper87|phone: haha, thats fine :) | 14:07 |
| flaper87|phone | Then everything went well. The client raises an exception when a non 200 status code is returned | 14:07 |
| flaper87|phone | Or at least it should | 14:08 |
| flaper87|phone | BTW, make sure to mention me when you write something :D | 14:08 |
| flaper87|phone | Otherwise I won't know you did | 14:08 |
| flaper87|phone | (Stupid irc client) | 14:08 |
| *** Obulpathi has quit IRC | 14:09 | |
| sriram | flaper87|phone: the exceptions right now are raised only in the case of 400, and 404. | 14:09 |
| *** Obulpathi has joined #openstack-marconi | 14:09 | |
| flaper87|phone | Mmh, not for 500? | 14:10 |
| flaper87|phone | I wouldn't be surprised. I've been adding exceptions based on our API spec | 14:10 |
| sriram | flaper87|phone: hmm, alright. do we want another patch in there for that? I'm just making sure we can do error handling for marconi-bench | 14:15 |
| sriram | since it uses the client for both the consumer and producer | 14:15 |
| flaper87|phone | Absolutely, if there are unhandled errors then yes | 14:15 |
| sriram | flaper87|phone: all right, I'll put out another patch for that once claim and delete is merged. | 14:17 |
| sriram | flaper87|phone: Thanks for your thoughts! :) | 14:17 |
| sriram | alcabrera: hey! | 14:17 |
| sriram | alcabrera: could you have a look at https://review.openstack.org/#/c/97246/ ? | 14:18 |
| flaper87|phone | Loool @ "once claim is merged.... alcabrera could you have a look at...." | 14:20 |
| flaper87|phone | Np | 14:20 |
| alcabrera | haha | 14:21 |
| alcabrera | good morning, all. :) | 14:21 |
| alcabrera | sriram: sure thing | 14:21 |
| sriram | Thanks alcabrera :) and good morning :P | 14:21 |
| *** prashanthr_ has joined #openstack-marconi | 14:23 | |
| alcabrera | sriram: small nit. +0. I'm ready to merge this, but I wanted you to see my comment. :) | 14:24 |
| alcabrera | sriram: https://review.openstack.org/#/c/97246/2/examples/claims.py | 14:25 |
| Alex_Gaynor | alcabrera: Can you W+ this: https://review.openstack.org/#/c/97074/ (it's already got +2s, but lost the W+ on rebase) | 14:25 |
| sriram | alcabrera: ooh nice, thanks for the comment! I'll make that change. | 14:26 |
| alcabrera | Alex_Gaynor: sure thing | 14:26 |
| Alex_Gaynor | thanks | 14:27 |
| alcabrera | np. :) | 14:27 |
| alcabrera | sriram: I like to minimize the use of local bindings as much as possible, since it's one less assignment (local state) to track. | 14:28 |
| sriram | alcabrera: makes sense :) | 14:28 |
| *** kgriffs|afk is now known as kgriffs | 14:28 | |
| alcabrera | let me know when it's ready and I'll +2 and try to bug flaper87|phone or kgriffs to get the patch moving. :) | 14:28 |
| sriram | alcabrera: hmm, if we remove claim1 we wont we able to prove that the claim1.id and msg.claim_id is the same. | 14:31 |
| sriram | I think that print statement is required to show that we are indeed claiming messages that have the same claimid, as claim1.id | 14:32 |
| sriram | alcabrera: ^ | 14:32 |
| alcabrera | sriram: oohh | 14:32 |
| sriram | what are your thoughts? | 14:32 |
| alcabrera | oh I see | 14:32 |
| alcabrera | print('{claim_id} =? {id}'.format(claim_id=claim_id, id=claim1.id)) | 14:33 |
| alcabrera | that | 14:33 |
| sriram | yes | 14:33 |
| alcabrera | haha, the (=?) operator reminded me of prolog-like syntax | 14:33 |
| sriram | :P | 14:33 |
| alcabrera | so | 14:33 |
| alcabrera | given the need for the local state | 14:33 |
| alcabrera | and that it is an example | 14:33 |
| alcabrera | I'm super cool with it | 14:33 |
| alcabrera | +2 | 14:34 |
| sriram | alcabrera: woohoo :) | 14:34 |
| alcabrera | sriram: approved, thanks! | 14:35 |
| sriram | thanks for the review alcabrera! | 14:36 |
| alcabrera | np. :) | 14:38 |
| *** rwsu has joined #openstack-marconi | 14:40 | |
| sriram | kgriffs: ping, I'll submit another patch to the client for error handling. and we should be able to use those for marconi-bench's producer and consumer. | 14:41 |
| *** reed has joined #openstack-marconi | 14:43 | |
| kgriffs | ok | 14:44 |
| kgriffs | sriram: do you think we can get something extremely basic for benchmarking done by next tuesday? | 14:45 |
| sriram | kgriffs: I think so. | 14:46 |
| sriram | we have the producer and consumer ready. | 14:46 |
| sriram | just need to do some error handling. | 14:46 |
| kgriffs | Obulpathi: can you confirm this is fixed? https://bugs.launchpad.net/marconi/+bug/1287490 | 14:48 |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Enable deleting of message with claim id https://review.openstack.org/97246 | 14:48 |
| Obulpathi | kgriffs: looking into it | 14:50 |
| kgriffs | sriram: I'd like to do a 0.0.2 release of the client - can you recruit a couple folks to smoke-test the latest code? | 14:50 |
| sriram | sure. Obulpathi,abettadapur : can you guys help? | 14:51 |
| kgriffs | in other news, I'd love to get some reviews on my patch: https://review.openstack.org/#/c/67978/ | 14:53 |
| Obulpathi | sriram kgriffs: Sure, I can test the code | 14:54 |
| sriram | kgriffs: I'l have a look. | 14:54 |
| kgriffs | everyone: I've got some training to go to today, so I'll be offline for a few hours. thanks for everyone's help! | 14:54 |
| kgriffs | sriram: thanks! | 14:54 |
| sriram | kgriffs: \m/ | 14:54 |
| prashanthr_ | alcabrera , sriram , kgriffs: Good morning :) | 14:55 |
| kgriffs | prashanthr_: o/ | 14:56 |
| vkmc | o/ | 14:56 |
| alcabrera | prashanthr_: o/ | 14:56 |
| sriram | prashanthr_: hey! Good Morning :) | 14:56 |
| sriram | vkmc: \o | 14:56 |
| Obulpathi | kgriffs: confirm means ... should I test and confirm or should I mark it as done? | 14:57 |
| vkmc | good morning/evening all! | 14:57 |
| kgriffs | prashanthr_, alcabrera: oh, before I forget, you should try and sync with flaper87 regarding whether redis driver should be in the main repo... I think it should given some people's AGPL concerns with deploying mongo. But I'd like to get everyone else's thoughts. | 14:57 |
| *** haomaiw__ has quit IRC | 14:58 | |
| kgriffs | Obulpathi: I think it is marked as "fixed" but I want to be sure there is no pending work on it | 14:58 |
| Obulpathi | kgriffs: nop .. its done | 14:58 |
| prashanthr_ | kgriffs: Sure. Will sync with flaper87. | 14:58 |
| kgriffs | excellent | 14:58 |
| Obulpathi | malini and me both tested it. thanks kgriffs | 14:58 |
| *** haomaiwang has joined #openstack-marconi | 14:58 | |
| alcabrera | kgriffs, prashanthr_: good thought. | 14:59 |
| alcabrera | I'm favorable towards redis driver being part of marconi proper, for the record. :) | 14:59 |
| alcabrera | the license point is a good one | 14:59 |
| alcabrera | but also | 14:59 |
| alcabrera | having a non-persistent driver adds avenues for different use cases -- e.g., different performance optimizations | 15:00 |
| kgriffs | prashanthr_: I think Redis can support the entire v1 API, so it would be a good alternate for mongo. If we can find something else that could also do that, then we would need to decide which one to include "out of the box". I haven't investigated riak or kafka enough to know if they can support our entire API... TBD | 15:00 |
| kgriffs | gotta run | 15:01 |
| kgriffs | ttfn | 15:01 |
| kgriffs | o/ | 15:01 |
| alcabrera | kgriffs: o/ | 15:01 |
| kgriffs | prashanthr_, alcabrera: maybe you can start a thread on the ML to discuss? | 15:02 |
| *** flaper87|phone has quit IRC | 15:02 | |
| prashanthr_ | alcabrera: True. I remember the priority based queue sharding discussion we had initially. | 15:02 |
| *** kgriffs is now known as kgriffs|afk | 15:03 | |
| *** abettadapur has quit IRC | 15:11 | |
| *** sriram has quit IRC | 15:16 | |
| *** sriram has joined #openstack-marconi | 15:17 | |
| *** balajiiyer has quit IRC | 15:17 | |
| *** AAzza_afk is now known as AAzza | 15:19 | |
| *** malini_afk is now known as malini | 15:23 | |
| *** jamie_h has quit IRC | 15:37 | |
| sriram | alcabrera,malini: we also need reviews on https://review.openstack.org/#/c/97534/ | 15:38 |
| *** jamie_h has joined #openstack-marconi | 15:39 | |
| alcabrera | oh my -- +834, -244 | 15:39 |
| alcabrera | that's quite the patch. | 15:39 |
| alcabrera | I'll punt on reviewing that one 'til the end of the week. I don't think I can give that patch a proper review at the moment. | 15:40 |
| sriram | yeah, its big :) we need our unit tests decoupled pretty badly :P | 15:40 |
| sriram | alcabrera: ok | 15:40 |
| *** balajiiyer has joined #openstack-marconi | 15:41 | |
| *** AAzza is now known as AAzza_afk | 15:45 | |
| *** balajiiyer has quit IRC | 15:58 | |
| *** abettadapur has joined #openstack-marconi | 15:59 | |
| *** mpanetta has joined #openstack-marconi | 16:01 | |
| *** haomaiwang has quit IRC | 16:10 | |
| sriram | abettadapur: ping | 16:10 |
| *** mpanetta has quit IRC | 16:12 | |
| *** mpanetta has joined #openstack-marconi | 16:12 | |
| *** ametts has quit IRC | 16:19 | |
| *** oz_akan_ has quit IRC | 16:19 | |
| *** oz_akan_ has joined #openstack-marconi | 16:19 | |
| *** abettadapur has quit IRC | 16:20 | |
| *** AAzza_afk is now known as AAzza | 16:21 | |
| *** prashanthr_ has left #openstack-marconi | 16:26 | |
| *** abettadapur has joined #openstack-marconi | 16:33 | |
| *** jchai is now known as jchai_afk | 16:39 | |
| *** balajiiyer has joined #openstack-marconi | 17:06 | |
| *** shakamunyi has quit IRC | 17:13 | |
| tjanczuk_ | hello | 17:16 |
| tjanczuk_ | vkmc: are you there? | 17:16 |
| tjanczuk_ | flapper87: are you there? | 17:17 |
| *** jchai_afk is now known as jchai | 17:21 | |
| *** jamie_h has quit IRC | 17:49 | |
| *** jamie_h has joined #openstack-marconi | 17:50 | |
| vkmc | tjanczuk_, hey there! | 17:52 |
| tjanczuk_ | hey | 17:52 |
| tjanczuk_ | I spent some time with AMQP yesterday | 17:52 |
| tjanczuk_ | I ran into an issue I think we need to discuss | 17:52 |
| tjanczuk_ | It has to do with consuming messages using claims. I don't think this can be accomplished with the current shape of the HTTP APIs. | 17:53 |
| vkmc | yes I ran into the same issue with AMQP 1.0 | 17:53 |
| vkmc | we should discuss this with alcabrera, flaper87 or/and kgriffs who has been working on Marconi since its beginning | 17:54 |
| tjanczuk_ | With the HTTP APIs, one first creates a claim (and obtains the messages). In a subsequent HTTP call, the message[es] can be permanently deleted to indicate the client has "consumed" them. | 17:54 |
| vkmc | exactly | 17:55 |
| tjanczuk_ | Similar mechanism exists in AMQP: you first get the messages, and subsequently send an ack to the server to indicate they have been "consumed" and can be permanently deleted. | 17:55 |
| tjanczuk_ | The problem is (and I believe it exists in both AMQP 0.9 and 1.0) that the acknowledgements must be sent over the same AMQP channel instance (bound to a TCP connection) that the messages were received over. | 17:56 |
| *** jamie_h has quit IRC | 17:57 | |
| alcabrera | hmmm | 17:57 |
| vkmc | yes tjanczuk_, it happens in AMQP 1.0 as well | 17:57 |
| alcabrera | that sounds tricky. :/ | 17:57 |
| tjanczuk_ | To implement this semantics over the HTTP-AMQP protocol bridge, one would need server affinity. | 17:57 |
| alcabrera | so you'd need to maintain a mapping of connections | 17:57 |
| tjanczuk_ | I think there are a few options: | 17:57 |
| tjanczuk_ | 1) WebSockets. This is a natural way to maintain affinity. | 17:58 |
| tjanczuk_ | 2) When an API server receives a request to acknowledge a message for which it does not hold a channel instance, it forwards it (reverse proxy) to the appropriate backend instance that does. | 17:59 |
| tjanczuk_ | 3) Redirects. This would require individual API servers in the cluster to be addressable. Also, redirecting HTTP POSTs is a bit of a cutting edge, | 17:59 |
| tjanczuk_ | 4). Do not support claim semantics with AMQP storage. Basically an HTTP request would permanently consume the message. This is rather pathetic as it does not event support at-least-once, but I suppose it is better than not being able to receive messages at all ;) | 18:01 |
| tjanczuk_ | Thoughts? | 18:01 |
| *** jamie_h has joined #openstack-marconi | 18:02 | |
| tjanczuk_ | Also, re: 4, I was trying to find the code that implements the "pop" semantics of deleting messages from a queue as per spec here: https://wiki.openstack.org/wiki/Marconi/specs/api/v1.1#Delete_Multiple_Messages. I could not find that code. Where is it? | 18:03 |
| * alcabrera catches up | 18:04 | |
| vkmc | tjanczuk_, the pop functionality is not merged yet https://review.openstack.org/#/c/90202/ | 18:04 |
| tjanczuk_ | How does that code review thing work? Can I help move it along somehow? | 18:06 |
| alcabrera | I really like (1), in the sense of pursuing an alternate transport, since that would map more cleanly to amqp | 18:06 |
| vkmc | tjanczuk_, regarding the options to support claims... I think that looking into adding websockets is good idea | 18:06 |
| alcabrera | tjanczuk_: code review -- if 2 or more core reviewers +2 a given patch, it is then approved and merged | 18:06 |
| tjanczuk_ | I also like the websockets approach most, I think it has a number of other advantages besides fixing this problem. | 18:07 |
| vkmc | tjanczuk_, everyone can review :) https://wiki.openstack.org/wiki/Your_First_Review_(Marconi) | 18:07 |
| alcabrera | good link, vkmc! | 18:08 |
| vkmc | the pop fix has already been reviewed and is waiting for the changes... so there is nothing we can do there | 18:08 |
| vkmc | tjanczuk_, which are the other advantages of adding websockets? | 18:09 |
| vkmc | alcabrera, yay to our wiki | 18:10 |
| vkmc | btw tjanczuk_, did you created a blueprint to track the rabbit driver? https://blueprints.launchpad.net/marconi | 18:11 |
| tjanczuk_ | In general many message broker protocols are optimized around persistent connections. AMQP, MQTT, even STOMP. Following from that broker implementations try to optimize around that persistent connection assumption (e.g. Kafka). If we can somehow expose that long lived connection concept all the way through the transport layer, we can have a much more optimal implementation than with that HTTP/TCP disparity. | 18:12 |
| vkmc | tjanczuk_, I see, thanks | 18:14 |
| tjanczuk_ | I did not create a blueprint. If I created it yesterday it would be out of date already. It is a discovery mission. I think once the prototype is running an issues understood and designed around, it makes sense to capture it in a blueprint. | 18:14 |
| tjanczuk_ | What is the best way to go about adding support for WebSockets? | 18:14 |
| tjanczuk_ | I understand WSGI has no notion of websockets. | 18:15 |
| tjanczuk_ | Would that be a new transport layer in Marconi? | 18:15 |
| *** AAzza is now known as AAzza_afk | 18:16 | |
| tjanczuk_ | Also, even if we have a new, connection-oriented transport implementation, I expect we would still have to model that concept in the ABC of the storage layer. So I would expect the storage layer abstractions to need some changes. | 18:17 |
| alcabrera | tjanczuk_: that would be a new transport layer in marconi | 18:18 |
| alcabrera | e.g., websockets | 18:18 |
| alcabrera | flaper87|afk did a little work into this in the past | 18:18 |
| tjanczuk_ | Yes, I think flapper87lafk was just bridging websockets to WSGI, which would not make the cut for the problem at hand. | 18:19 |
| alcabrera | agreed | 18:19 |
| tjanczuk_ | Does Marconi support running multiple transports side by side? | 18:20 |
| tjanczuk_ | Or is it an exclusive choice? | 18:20 |
| tjanczuk_ | We have to reconcile the fact that WebSockets are really HTTP. | 18:21 |
| alcabrera | it's currently an exclusive choice | 18:21 |
| alcabrera | you'd have to run two instances of marconi side by side | 18:22 |
| alcabrera | to achieve using two transports on one machine, for example | 18:22 |
| alcabrera | tjanczuk_: ^ | 18:23 |
| tjanczuk_ | I am new to Python. What are folks using mostly for WebSockets? I saw some positive mentions of tornado, I think it also comes up towards the top in benchmarks. | 18:25 |
| alcabrera | I'm not familiar with websockets, so I'm not entirely sure what's being used atm. | 18:26 |
| alcabrera | or recommended | 18:26 |
| vkmc | I heard about Autobahn, but I haven't check it out yet | 18:26 |
| tjanczuk_ | What did flapper87lafk use? | 18:26 |
| openstackgerrit | A change was merged to openstack/marconi: Removed now unnecesary workaround for PyPy https://review.openstack.org/97074 | 18:27 |
| vkmc | tjanczuk_, | 18:30 |
| vkmc | https://github.com/FlaPer87/marconi-websocket | 18:30 |
| alcabrera | seems to be | 18:30 |
| alcabrera | ws4py | 18:30 |
| alcabrera | tjanczuk_: ^^ | 18:30 |
| alcabrera | thanks, vkmc! | 18:30 |
| openstackgerrit | Sriram Madapusi Vasudevan proposed a change to openstack/python-marconiclient: Throw exceptions on erroneous status codes https://review.openstack.org/97875 | 18:30 |
| vkmc | yeah he used ws4py :) | 18:30 |
| tjanczuk_ | Is that an HTTP server itself, or does it compose with other servers? | 18:32 |
| alcabrera | not sure. I haven't looked into it. | 18:33 |
| tjanczuk_ | Let me look at that code... | 18:37 |
| vkmc | tjanczuk_, there is something I'm not so sure about... don't we need server affinity for HTTP too? | 18:46 |
| vkmc | sorry if it's a silly question... I'm giving my first steps in this area | 18:46 |
| *** stritz has quit IRC | 18:48 | |
| tjanczuk_ | The only two storage technologies implemented so far were DB-based: mongo and SQL. The mode of accessing DBs in vast majority of cases involves running RPC-style queries which map very well to short lived HTTP requests. That is why I suppose this issue has not come up before. | 18:48 |
| tjanczuk_ | So to answer your question: there was no pressing need for server affinity of HTTP transport when the storage is Mongo or SQL. | 18:49 |
| vkmc | it makes sense yeah | 18:50 |
| vkmc | thanks for clearing it up | 18:50 |
| tjanczuk_ | On the notion of my being new to Python: is my understanding correct that WSGI implies a blocking IO implementation, i.e. there is a thread held up for every active HTTP request in the system? | 18:56 |
| Alex_Gaynor | Notionally yes, except there are things like eventlet which will monkey patch all your IO to use greenthreads, so you can get some concurrency | 18:57 |
| tjanczuk_ | How does this work? Does it detect blocking IO and yields the thread under the hood? Does it operate a thread pool or is it single-threaded? | 18:58 |
| Alex_Gaynor | still single threaded | 18:59 |
| Alex_Gaynor | yeah, it'll put the file descriptor in something like epoll and then switch to another green thread | 18:59 |
| tjanczuk_ | Interesting. How does it compare to systems that are built on async io from the ground up like tornado? | 19:00 |
| Alex_Gaynor | Personally I find all the monkey patching distateful, and prefer something more explicit, like twisted; that said performance wise they're all pretty much in the same ballpark. | 19:01 |
| tjanczuk_ | Also, can you take pretty much any WSGI app and run it with eventlet, or are there some restrictions? | 19:01 |
| tjanczuk_ | Does Marconi run with eventlet? | 19:01 |
| Alex_Gaynor | More or less, except you often have other stuff that does IO, like a databsae library, and you need to make sure you use something that's monkeypatched or otherwise handles eventlet correctly. | 19:01 |
| Alex_Gaynor | Yes, marconi uses eventlet. | 19:01 |
| Alex_Gaynor | (If you care about performance, I strongly recommend using PyPy) | 19:02 |
| tjanczuk_ | What is involved in "monkeypatching"? Does a library need to do something, the app using the library, or does eventlet just take care of it all? | 19:03 |
| *** reed has quit IRC | 19:06 | |
| tjanczuk_ | vkmc: another AMQP issue I ran into has to do with message ID. That concept does not really exist in AMQP, so something needs to be done about all the HTTP APIs that accept or return message IDs. For example, publishing messages endpoint is supposed to return an array of message IDs. These are later used to delete the message (along with a claim id). | 19:09 |
| openstackgerrit | Sriram Madapusi Vasudevan proposed a change to openstack/python-marconiclient: Throw exceptions on erroneous status codes https://review.openstack.org/97875 | 19:10 |
| tjanczuk_ | vkmc: a model that would work better for AMQP is to have 1-1 relationship between claim ID and a message (rather than 1-n as it is the case right now), and use the claim IDs alone to delete messages. | 19:13 |
| vkmc | tjanczuk_, iirc the messages in amqp-proton have an id attribute http://qpid.apache.org/releases/qpid-proton-0.7/protocol-engine/python/api/proton.Message-class.html | 19:31 |
| vkmc | tjanczuk_, perhaps that is library related? | 19:32 |
| tjanczuk_ | I am not sure. Does that ID later act as input to any of the operations? | 19:32 |
| *** malini is now known as malini_afk | 19:32 | |
| *** jamie_h has quit IRC | 19:33 | |
| * vkmc checking | 19:34 | |
| tjanczuk_ | In the AMQP 0.9 model, a "delivery tag" is issued for a message being consumed by the server. That tag is unique within a given AMQP channel. The client later uses that tag to acknowledge the message (and have the server permanently delete it) *on the same channel*. Message IDs are not involved in the process. | 19:36 |
| tjanczuk_ | (This is by the way simialar to how SQS and Azure model consumption and acknowledgment with HTTP). | 19:38 |
| *** reed has joined #openstack-marconi | 19:38 | |
| vkmc | well.. as far as I could see | 19:42 |
| vkmc | the id is not used | 19:42 |
| vkmc | I guess it's an added attribute to ease de management in applications using qpid-proton | 19:42 |
| tjanczuk_ | right, message IDs are not really relevant in the AMQP model | 19:43 |
| vkmc | Messages in qpid-proton also have an user-id... so I think we could use them for claims instead of websockets | 19:44 |
| vkmc | you are right tjanczuk_, is not part of the AMQP model | 19:45 |
| vkmc | sorry for the delay but there are no manuals for proton yet | 19:45 |
| tjanczuk_ | I am not sure what you mean by "we could use them for claims instead of websockets"? | 19:45 |
| vkmc | I mean that we could implement claims as we do with the HTTP API, using a client-id to claim messages | 19:47 |
| vkmc | user-id is also an added attribute in proton | 19:47 |
| vkmc | we would need websockets for rabbit | 19:48 |
| tjanczuk_ | I am still not sure how user-id helps. How does consumption/acknowledgement work in AMQP 1.0? | 19:49 |
| *** malini_afk is now known as malini | 19:56 | |
| *** oz_akan_ has quit IRC | 19:57 | |
| *** fifieldt_ has joined #openstack-marconi | 20:00 | |
| vkmc | well in fact it's confusing because it's quite different to how AMQP < 1.0 works | 20:02 |
| vkmc | I'm following this thread http://qpid.2158936.n2.nabble.com/Proton-Messenger-and-the-Request-Response-pattern-td7586652.html | 20:02 |
| *** fifieldt-afk has quit IRC | 20:04 | |
| vkmc | give me a moment to see if I'm heading in the right direction | 20:04 |
| *** mwagner_lap has quit IRC | 20:20 | |
| *** ykaplan has joined #openstack-marconi | 20:23 | |
| tjanczuk_ | People in the know (Clemens Vesters) pointed me at http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-transport-v1.0-os.html#doc-resuming-deliveries. Apparently AMQP 1.0 has a concept of resuming deliveries which I think (not sure) would allow acknowledgment (disposition in AMQP 1.0 lingo) to be sent over a different TCP connection. I suspect the devil is in the details. You may want to read through the AMQP 1.0 spec looking for things like | 20:31 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/marconi: Prepare marconi for localization https://review.openstack.org/97914 | 20:33 |
| *** abettadapur has quit IRC | 20:35 | |
| *** sriram has quit IRC | 20:38 | |
| *** alcabrera is now known as alcabrera|afk | 20:51 | |
| *** balajiiyer has quit IRC | 20:52 | |
| *** oz_akan_ has joined #openstack-marconi | 21:12 | |
| *** Obulpathi has quit IRC | 21:22 | |
| *** jchai has quit IRC | 21:28 | |
| *** oz_akan_ has quit IRC | 21:33 | |
| *** oz_akan_ has joined #openstack-marconi | 21:34 | |
| *** vkmc has quit IRC | 21:55 | |
| *** malini has left #openstack-marconi | 22:06 | |
| *** mkoderer has quit IRC | 22:06 | |
| *** mkoderer has joined #openstack-marconi | 22:11 | |
| *** ykaplan has quit IRC | 22:38 | |
| *** oz_akan_ has quit IRC | 23:29 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!