| *** flwang has joined #openstack-marconi | 00:17 | |
| *** mwagner_lap has joined #openstack-marconi | 00:40 | |
| *** balajiiyer has joined #openstack-marconi | 00:49 | |
| *** balajiiyer has quit IRC | 00:50 | |
| *** rwsu has quit IRC | 00:51 | |
| *** oz_akan_ has joined #openstack-marconi | 01:29 | |
| *** oz_akan_ has quit IRC | 01:33 | |
| *** oz_akan_ has joined #openstack-marconi | 01:36 | |
| *** oz_akan_ has quit IRC | 01:40 | |
| *** vkmc has joined #openstack-marconi | 01:42 | |
| *** nosnos has joined #openstack-marconi | 01:46 | |
| *** balajiiyer has joined #openstack-marconi | 01:48 | |
| *** oz_akan_ has joined #openstack-marconi | 01:49 | |
| *** mpanetta has quit IRC | 01:56 | |
| *** jmckind has quit IRC | 02:05 | |
| *** balajiiyer has quit IRC | 02:18 | |
| *** vkmc has quit IRC | 03:26 | |
| *** nosnos has quit IRC | 03:43 | |
| *** prashanthr_1 has joined #openstack-marconi | 03:43 | |
| *** reed has quit IRC | 03:47 | |
| *** prashanthr_1 has quit IRC | 03:51 | |
| *** prashanthr_ has joined #openstack-marconi | 03:57 | |
| *** whenry has quit IRC | 03:57 | |
| *** oz_akan_ has quit IRC | 04:04 | |
| *** renlt has joined #openstack-marconi | 04:15 | |
| *** prashanthr_ has quit IRC | 04:26 | |
| *** nosnos has joined #openstack-marconi | 04:41 | |
| *** renlt has quit IRC | 04:46 | |
| *** prashanthr_ has joined #openstack-marconi | 05:15 | |
| *** prashanthr_ has quit IRC | 05:36 | |
| *** oz_akan_ has joined #openstack-marconi | 05:37 | |
| *** oz_akan_ has quit IRC | 05:41 | |
| *** prashanthr_ has joined #openstack-marconi | 05:50 | |
| *** jraim has quit IRC | 06:05 | |
| *** jraim has joined #openstack-marconi | 06:06 | |
| *** flaper87|afk is now known as flaper87 | 06:08 | |
| flaper87 | tjanczuk_: Alex_Gaynor Marconi doesn't use eventlet. It used to have an "indirect" dependency on it but it's now gone. | 06:11 |
|---|---|---|
| flwang | flaper87: wooooooooooooooooooow | 06:11 |
| flaper87 | flwang: what's up buddy? | 06:12 |
| flaper87 | how are you doing? | 06:12 |
| flaper87 | long time no see | 06:12 |
| flwang | flaper87: yep, i'm working from New Zealand | 06:12 |
| flwang | everything good until now :) | 06:12 |
| flwang | just wanna say hi to let you know I'm still alive :D | 06:12 |
| flwang | and not far away from OpenStack | 06:12 |
| flwang | so far so good | 06:15 |
| *** renlt has joined #openstack-marconi | 06:16 | |
| tjanczuk_ | flapper87: does it mean Marconi is doing blocking IO then (i.e. a separate thread is used for every active request)? | 06:23 |
| flaper87 | tjanczuk_: it depends on what wsgi container you use | 06:33 |
| flaper87 | tjanczuk_: Marconi just ships the wsgi app and users are free to choose whatever wsgi container they want | 06:34 |
| flaper87 | you could use uwsgi (written in C and supports non-blocking IO) or gunicorn + eventlet | 06:34 |
| *** prashanthr_1 has joined #openstack-marconi | 06:34 | |
| prashanthr_1 | hi flaper87 , flwang :) | 06:34 |
| flaper87 | If you run `marconi-server` you're using wsgiref which is a "toy" container shipped with python | 06:34 |
| flaper87 | and it's meant to be used just for development | 06:35 |
| flaper87 | prashanthr_1: yo! | 06:35 |
| flaper87 | flwang: man, so happy to hear that! Take good care of yourself and your family. | 06:35 |
| flaper87 | flwang: are you going to spend more time on Marconi now/ | 06:35 |
| flaper87 | ? | 06:35 |
| prashanthr_1 | flaper87: How u doing ? | 06:35 |
| flwang | flaper87: recently, now, but next month, I suppose yes | 06:35 |
| flaper87 | prashanthr_1: doing great, thanks :D | 06:35 |
| flaper87 | prashanthr_1: you? | 06:36 |
| *** prashanthr_ has quit IRC | 06:36 | |
| prashanthr_1 | flaper87: doing great as well :) thanks ! | 06:38 |
| *** oz_akan_ has joined #openstack-marconi | 06:38 | |
| prashanthr_1 | i wanted 2 have a brief chat with you about supporting redis for marconi. | 06:38 |
| prashanthr_1 | are u free for sometime now ? | 06:38 |
| flaper87 | prashanthr_1: shoot :) | 06:38 |
| prashanthr_1 | flaper87: Yesterday we were having a discussion about supporting redis on the main branch of marconi. kgriffs|afk felt that it would be could to get a collective opinion. | 06:40 |
| prashanthr_1 | *could-> good | 06:40 |
| prashanthr_1 | we also had a brief chat about the AGPL issue for mongodb. | 06:41 |
| *** oz_akan_ has quit IRC | 06:42 | |
| flaper87 | prashanthr_1: mmh, TBH, I thought that was decided already :P | 06:45 |
| prashanthr_1 | flaper87: ha ha :). | 06:45 |
| flaper87 | prashanthr_1: it sounds good to me, FWIW. We've been looking for a store driver that would help us with the whole AGPL issue. | 06:46 |
| flaper87 | those 2 drivers are redis' and amqp's | 06:46 |
| prashanthr_1 | flaper87: awesome :). The AGPL issue was pretty serious i just noticed that yesterday. | 06:47 |
| flaper87 | prashanthr_1: I still need to dig better into how serious it is. There are a whole bunch of different opinions around it and some seem to be very subjective. | 06:48 |
| prashanthr_1 | flaper87: Ohh okay. I read a couple of posts yesterday. But the one's i read made it look draconian. | 06:49 |
| flaper87 | prashanthr_1: did you read this one? https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg19948.html | 06:50 |
| flaper87 | wait, not that one | 06:50 |
| prashanthr_1 | This one's interesting as well :) | 06:51 |
| flaper87 | prashanthr_1: this one: http://lists.openstack.org/pipermail/legal-discuss/2014-March/000174.html | 06:51 |
| flaper87 | that's the "legal discussion" | 06:52 |
| prashanthr_1 | flaper87: I will go through it. | 06:56 |
| flaper87 | prashanthr_1: btw, thanks for the heads up w.r.t the discussions and the comments | 06:59 |
| *** jamie_h has joined #openstack-marconi | 07:01 | |
| prashanthr_1 | flaper87: No problems :). Phew these licenses i never understand them well enough. Reading them is never fun :(. | 07:01 |
| *** prashanthr_ has joined #openstack-marconi | 07:03 | |
| *** prashanthr_1 has quit IRC | 07:05 | |
| *** ykaplan has joined #openstack-marconi | 07:19 | |
| *** jamie_h has quit IRC | 07:36 | |
| *** jamie_h has joined #openstack-marconi | 07:37 | |
| prashanthr_ | flaper87: I just had a small doubt about the oslo cache we r using. | 07:38 |
| prashanthr_ | when we append a list to another one in the cache | 07:38 |
| *** oz_akan_ has joined #openstack-marconi | 07:38 | |
| prashanthr_ | the one in the cache must already by of the same list type. Am i right ? | 07:38 |
| flaper87 | yup | 07:39 |
| flaper87 | (brb) | 07:39 |
| *** flaper87 is now known as flaper87|afk | 07:39 | |
| prashanthr_ | flaper87: thanks. | 07:39 |
| *** oz_akan_ has quit IRC | 07:43 | |
| *** prashanthr_ has quit IRC | 08:23 | |
| *** goodes has joined #openstack-marconi | 08:30 | |
| *** mkoderer has quit IRC | 08:32 | |
| *** oz_akan_ has joined #openstack-marconi | 08:39 | |
| *** oz_akan_ has quit IRC | 08:43 | |
| *** jamie_h has quit IRC | 08:48 | |
| *** prashanthr_ has joined #openstack-marconi | 08:52 | |
| *** haomaiwa_ has joined #openstack-marconi | 09:01 | |
| *** prashanthr_ has quit IRC | 09:04 | |
| *** haomai___ has joined #openstack-marconi | 09:04 | |
| *** haomaiwa_ has quit IRC | 09:07 | |
| *** flaper87|afk is now known as flaper87 | 09:19 | |
| *** flaper87 is now known as flaper87|afk | 09:38 | |
| *** oz_akan_ has joined #openstack-marconi | 09:40 | |
| *** oz_akan_ has quit IRC | 09:44 | |
| *** mkoderer has joined #openstack-marconi | 09:53 | |
| *** ykaplan has quit IRC | 10:05 | |
| *** goodes has quit IRC | 10:22 | |
| *** renlt has quit IRC | 10:24 | |
| *** oz_akan_ has joined #openstack-marconi | 10:38 | |
| *** haomai___ has quit IRC | 10:40 | |
| *** haomaiwang has joined #openstack-marconi | 10:41 | |
| *** oz_akan_ has quit IRC | 10:42 | |
| *** haomaiw__ has joined #openstack-marconi | 10:47 | |
| *** haomaiwang has quit IRC | 10:50 | |
| *** vkmc has joined #openstack-marconi | 10:51 | |
| *** vkmc has quit IRC | 10:51 | |
| *** vkmc has joined #openstack-marconi | 10:51 | |
| *** jamie_h has joined #openstack-marconi | 10:56 | |
| *** goodes has joined #openstack-marconi | 11:15 | |
| *** nosnos has quit IRC | 11:20 | |
| *** jamie_h has quit IRC | 11:29 | |
| *** AAzza_afk is now known as AAzza | 11:30 | |
| *** oz_akan_ has joined #openstack-marconi | 11:39 | |
| *** oz_akan_ has quit IRC | 11:43 | |
| *** jamie_h has joined #openstack-marconi | 11:59 | |
| *** flaper87|afk is now known as flaper87 | 12:02 | |
| *** jamie_h has quit IRC | 12:10 | |
| *** jamie_h has joined #openstack-marconi | 12:29 | |
| *** jchai has joined #openstack-marconi | 12:39 | |
| *** oz_akan_ has joined #openstack-marconi | 12:40 | |
| *** oz_akan_ has quit IRC | 12:44 | |
| *** sriram has joined #openstack-marconi | 12:45 | |
| *** sriram has quit IRC | 12:46 | |
| *** sriram has joined #openstack-marconi | 12:46 | |
| *** jamie_h has quit IRC | 12:47 | |
| *** AAzza is now known as AAzza_afk | 12:48 | |
| *** ykaplan has joined #openstack-marconi | 12:49 | |
| flaper87 | sriram: kgriffs|afk alcabrera|afk malini vkmc et all! Good morning! Just a heads up. The gate seems to be uner heavy load so lets not recheck, push, approve anything unless it helps fixing the gate. | 12:49 |
| *** goodes has quit IRC | 12:50 | |
| sriram | flaper87: thanks for the headup, I just read the email :) | 12:50 |
| sriram | and good morning :P | 12:50 |
| flaper87 | sriram: awesome :) | 12:50 |
| flaper87 | OH: "If you do a query on a database without indexes it will take a lot of time" | 12:51 |
| flaper87 | (Captain obvious) | 12:51 |
| * flaper87 in jerk mode | 12:51 | |
| *** jamie_h has joined #openstack-marconi | 12:51 | |
| vkmc | morning flaper87, sriram :) | 12:52 |
| vkmc | thanks for the update flaper87 | 12:52 |
| vkmc | s/update/heads up | 12:52 |
| sriram | morning vkmc, I saw that there was heavy discussion on AMQP going on, I learnt quite a bit ;) | 12:53 |
| vkmc | sriram, oh yes... there are a lot to discuss in that area :) | 12:53 |
| vkmc | I'm learning a lot | 12:54 |
| sriram | awesome | 12:54 |
| vkmc | the downside is that there are not enough docs :( | 12:55 |
| sriram | Oh I see, I have been doing some reading on rabbitmq. | 12:56 |
| * sriram has never used rabbitmq before :/ | 12:56 | |
| vkmc | rabbit folks did a great job with docs, unfortunately I'm with another version of the protocol and hence another library | 12:58 |
| sriram | oic, this is amqp 1.0? | 12:58 |
| *** jchai has quit IRC | 12:59 | |
| vkmc | yeah | 12:59 |
| vkmc | precisely, Apache's Proton | 13:00 |
| *** jchai has joined #openstack-marconi | 13:00 | |
| *** flaper87 is now known as flaper87|afk | 13:02 | |
| *** jmckind has joined #openstack-marconi | 13:07 | |
| *** jergerber has joined #openstack-marconi | 13:07 | |
| *** prashanthr_ has joined #openstack-marconi | 13:10 | |
| *** oz_akan_ has joined #openstack-marconi | 13:11 | |
| *** sriram has left #openstack-marconi | 13:15 | |
| *** sriram has joined #openstack-marconi | 13:15 | |
| *** stannie has joined #openstack-marconi | 13:18 | |
| *** jergerber has quit IRC | 13:22 | |
| *** Obulpathi has joined #openstack-marconi | 13:31 | |
| *** balajiiyer has joined #openstack-marconi | 13:36 | |
| *** goodes has joined #openstack-marconi | 13:36 | |
| *** Obulpathi has quit IRC | 13:39 | |
| sriram | kgriffs|afk: the client was smoke tested yesterday, and it looks good. | 14:04 |
| *** alcabrera|afk is now known as alcabrera | 14:05 | |
| *** goodes has quit IRC | 14:06 | |
| *** goodes has joined #openstack-marconi | 14:07 | |
| alcabrera | good morning. :) | 14:07 |
| sriram | alcabrera: good morning! :D | 14:07 |
| sriram | how are you doing? | 14:07 |
| *** malini has joined #openstack-marconi | 14:07 | |
| alcabrera | hungry, happy, thoughtful. I've been reading and learning a lot lately. How about you, sriram? :) | 14:08 |
| *** balajiiyer has quit IRC | 14:08 | |
| *** jay-atl has quit IRC | 14:08 | |
| sriram | awesome!, I didnt get much sleep yesterday, a bit tired. | 14:08 |
| alcabrera | oh no. this requires a midday nap to rectify, sriram! :D | 14:08 |
| sriram | heh | 14:09 |
| vkmc | morning alcabrera! | 14:09 |
| vkmc | +1 alcabrera | 14:09 |
| vkmc | you totally should get a power nap sriram | 14:09 |
| sriram | haha, I'll try! :) | 14:10 |
| alcabrera | vkmc: good morning! :) | 14:10 |
| *** AAzza_afk is now known as AAzza | 14:12 | |
| *** balajiiyer has joined #openstack-marconi | 14:12 | |
| *** goodes has left #openstack-marconi | 14:14 | |
| *** Obulpathi has joined #openstack-marconi | 14:14 | |
| *** jamie_h has quit IRC | 14:17 | |
| *** haomaiw__ has quit IRC | 14:20 | |
| *** cpallares has joined #openstack-marconi | 14:26 | |
| cpallares | o/ | 14:26 |
| alcabrera | cpallares: good morning! | 14:27 |
| cpallares | alcabrera: hi! how are you? | 14:28 |
| alcabrera | hungry, happy, thoughtful. How about you, cpallares? :) | 14:28 |
| vkmc | cpallares, morning :) | 14:29 |
| prashanthr_ | hi good morning all ! :) | 14:32 |
| cpallares | alcabrera: Doing great :) | 14:35 |
| cpallares | hi vkmc! hey prashanthr_ :) how's it going? | 14:36 |
| vkmc | good evening prashanthr_ :) | 14:36 |
| *** prashanthr_ has left #openstack-marconi | 14:38 | |
| *** prashanthr_ has joined #openstack-marconi | 14:38 | |
| prashanthr_ | thank you vkmc ! | 14:38 |
| *** haomaiwang has joined #openstack-marconi | 14:40 | |
| prashanthr_ | alcabrera: Do we have a meeting today ? | 14:41 |
| *** goodes has joined #openstack-marconi | 14:42 | |
| alcabrera | yes, prashanthr_! and also, good evening to you. :) | 14:43 |
| alcabrera | we'll get started in about 15m. | 14:43 |
| prashanthr_ | awesome :) Thank you ! :) | 14:43 |
| prashanthr_ | sure | 14:43 |
| *** haomaiwang has quit IRC | 14:45 | |
| *** oz_akan_ has quit IRC | 14:50 | |
| *** oz_akan_ has joined #openstack-marconi | 14:51 | |
| *** Obulpathi has quit IRC | 14:51 | |
| *** Obulpathi has joined #openstack-marconi | 14:52 | |
| *** haomaiwang has joined #openstack-marconi | 14:52 | |
| *** goodes has quit IRC | 14:54 | |
| *** kgriffs|afk is now known as kgriffs | 14:55 | |
| alcabrera | alright! I managed to finish breakfast and make coffee in time for today's OPW/GSoC meeting. :D | 15:00 |
| alcabrera | AAzza, prashanthr_, vkmc: are you all ready to get started? :) | 15:00 |
| prashanthr_ | yes :) | 15:01 |
| vkmc | ready! | 15:01 |
| AAzza | yeap) | 15:01 |
| alcabrera | awesome! | 15:01 |
| *** goodes has joined #openstack-marconi | 15:01 | |
| alcabrera | so let's get going: AAzza, could you start us off? | 15:01 |
| alcabrera | how has progress been going since last week on py3-marconi? :) | 15:02 |
| AAzza | alcabrera: hey, the commits were merged, and i'm ready to put next one with small fixes | 15:02 |
| AAzza | alcabrera: but have some doubts) | 15:02 |
| alcabrera | sure thing, what doubts have you got, AAzza? | 15:03 |
| AAzza | alcabrera: first of all is it okey to have one commit with small fixes, or better to have several with relevant grouped? | 15:03 |
| alcabrera | I prefer several, small commits grouped by relevance/category | 15:04 |
| alcabrera | though | 15:04 |
| alcabrera | if the small fixes in mind are less than say, 20-ish lines, and they all relate to the broad category of py3-compat, I think that's okay, too | 15:04 |
| AAzza | alcabrera: okey, maybe split in max two commits to be smth in the middle) | 15:05 |
| alcabrera | that'll work for me. :) | 15:05 |
| AAzza | alcabrera: next one is a little bit specific, maybe will better discuss it in commit, but nevertheless | 15:05 |
| alcabrera | yes, what's coming up next? | 15:06 |
| alcabrera | could you summarize? | 15:06 |
| AAzza | alcabrera: it is ugly to add encode/decode some specific encoding) | 15:06 |
| *** jay-atl has joined #openstack-marconi | 15:07 | |
| AAzza | alcabrera: always hard to find what is particular expected encoding for this piece of text) is it utf-8 or plain ascii text) | 15:07 |
| alcabrera | that is a tricky problem. :/ | 15:07 |
| AAzza | alcabrera: after this commit, i plan to have another one, also with strings | 15:08 |
| *** jchai is now known as jchai_afk | 15:08 | |
| alcabrera | thanks for that - the string problem is particularly bad between py2 <-> py3 | 15:08 |
| AAzza | alcabrera: need some time to research the problem, since some parts of this fix is not working on py3 https://github.com/openstack/marconi/commit/fa487de15e264798022c96de3caad1b272cfaea8 | 15:08 |
| AAzza | alcabrera: you know py3 is even more explicit than py2 | 15:09 |
| alcabrera | yup, yup | 15:09 |
| AAzza | alcabrera: you should always know where is text, and where is bytes and explicitly convert between them) | 15:09 |
| AAzza | alcabrera: and when you have in py2 pure ascii string, then it is ugly piece | 15:10 |
| alcabrera | oh yeah. :/ | 15:10 |
| *** jamie_h has joined #openstack-marconi | 15:10 | |
| AAzza | alcabrera: because either to have it in utf-8, or do tricks and magic) | 15:10 |
| AAzza | alcabrera: it is summary for now) | 15:10 |
| alcabrera | I suspect regex's don't handle the case of unicode strings very well. it's definitely something to dig into. | 15:10 |
| alcabrera | but thanks, AAzza! | 15:11 |
| alcabrera | I had one thought on py3 | 15:11 |
| alcabrera | even though we don't use memcached in marconi any more | 15:11 |
| alcabrera | when we do bring it back, it may be worthwhile to use this: https://github.com/Pinterest/pymemcache | 15:11 |
| kgriffs | I believe we decided a while back to try standardizing on wide unicode strings inside, while converting to utf-8 at I/O boundaries. Correct? | 15:11 |
| alcabrera | instead of python-memcached | 15:11 |
| alcabrera | kgriffs: I believe that was the decision | 15:12 |
| kgriffs | I think this py3k effort is highlighting where we got lazy. :p | 15:12 |
| alcabrera | yup, yup. :P | 15:12 |
| alcabrera | AAzza: any other thoughts before we move on to prashanthr_? :) | 15:12 |
| AAzza | alcabrera: oh... just want an advice | 15:13 |
| alcabrera | oh, sure. :) | 15:13 |
| AAzza | alcabrera: i see this commit https://review.openstack.org/#/c/97534/ with decoupling tests | 15:13 |
| AAzza | alcabrera: and my commit also changes the tests | 15:14 |
| alcabrera | ah | 15:14 |
| alcabrera | yes | 15:14 |
| AAzza | alcabrera: not sure how to handle this, since first commit copying them | 15:14 |
| alcabrera | there's likely going to be some conflict | 15:14 |
| *** jchai_afk is now known as jchai | 15:14 | |
| alcabrera | my suggestion | 15:14 |
| alcabrera | is to rebase your commit on top of https://review.openstack.org/#/c/97534/ | 15:14 |
| alcabrera | what do you think, AAzza? | 15:14 |
| AAzza | alcabrera: yes, good idea, and i will sure, that my changes will be in both tests groups, since i add them to both by myself) thanks :) | 15:15 |
| AAzza | alcabrera: i' done then for now) | 15:16 |
| *** haomaiwang has quit IRC | 15:16 | |
| alcabrera | thanks, AAzza! | 15:16 |
| *** haomaiwang has joined #openstack-marconi | 15:17 | |
| alcabrera | prashanthr_: what news have you to share on redis? :) | 15:17 |
| prashanthr_ | alcabrera: Redis yes i had a chat with flaper87|afk today in the morning as kgriffs had suggested yest | 15:17 |
| prashanthr_ | he also was of the same opinion that redis must be a part of the main branch | 15:18 |
| prashanthr_ | and we had a discussion of the AGPL issue of mongodb | 15:18 |
| alcabrera | wonderful! | 15:18 |
| kgriffs | quick thought on the topic | 15:18 |
| kgriffs | I like the idea of developign it first outside the main repo just to verify our test structure and plugin stuff works, but we can immediately pull it in after that | 15:19 |
| kgriffs | how close are we to a basic driver? | 15:19 |
| kgriffs | https://blueprints.launchpad.net/marconi/+spec/redis-storage-driver | 15:19 |
| kgriffs | any chance we will have something by next tuesday? | 15:20 |
| prashanthr_ | kgriffs: we are very close on my private forked repo | 15:20 |
| prashanthr_ | i just need to fix a couple of stuffs | 15:20 |
| prashanthr_ | and write some tests | 15:20 |
| kgriffs | great! You are making quick progress. | 15:20 |
| prashanthr_ | next tuesday yes i must be able to finish it up | 15:20 |
| alcabrera | fast! | 15:21 |
| vkmc | kudos prashanthr_! | 15:21 |
| kgriffs | I think we can call this blueprint "completed" without having to merge the driver into the main repo. Just need to see tests passing and a demo. | 15:21 |
| prashanthr_ | kgriffs and alcabrera: the code of marconi is so well organized that working is seemless | 15:21 |
| prashanthr_ | vkmc: he he thanks | 15:22 |
| kgriffs | rock on | 15:22 |
| alcabrera | nice. I'm glad your experience working with it was a good one, prashanthr_. :) | 15:22 |
| prashanthr_ | alcabrera: loving it :) | 15:22 |
| alcabrera | so then, it sounds like your next steps are to finish it up and get all the tests passing. | 15:23 |
| alcabrera | any thoughts/concerns, prashanthr_? | 15:23 |
| prashanthr_ | alcabrera: yes. Just one doubt. | 15:23 |
| prashanthr_ | while deleting a shard from the catalogue | 15:24 |
| prashanthr_ | to clear all the queues is it acceptable to perform a flush of the db ? | 15:24 |
| AAzza | prashanthr_: by the way, could i afterwards help with code review? not promise to find smth special (probably just neats in formatting, etc), but more people reviewing - better code) | 15:25 |
| prashanthr_ | AAzza: That would be really awesome thanks | 15:25 |
| prashanthr_ | https://github.com/PrashanthRaghu/marconi-redis/ | 15:25 |
| prashanthr_ | this is my private fork of marconi | 15:25 |
| prashanthr_ | and my source code resides in marconi-redis / marconi / queues / storage / redis | 15:26 |
| alcabrera | prashanthr_: so if a request to delete a shard comes along, it seems like the right thing to do would be to remove all queues associated with that shard, hmm... | 15:26 |
| vkmc | prashanthr_, are you also working in riak? neat! | 15:26 |
| prashanthr_ | vkmc: ha ha just added support for queues and messages | 15:27 |
| prashanthr_ | in my free time. | 15:27 |
| alcabrera | super nice | 15:27 |
| prashanthr_ | vkmc: the rest of the implementation is just mock interfaces. | 15:28 |
| alcabrera | I think flushing the db will work well | 15:28 |
| prashanthr_ | alcabrera: Sure. I will do that. Thank you. That's all i had to share for today. | 15:28 |
| alcabrera | kk, thanks prashanthr_! | 15:28 |
| alcabrera | vkmc: what's the latest on amqp? :) | 15:29 |
| vkmc | alcabrera, nothing much to bring up on this meeting | 15:29 |
| vkmc | the mapping is done I presume, and I was hoping to start working on the code for queues and messages today | 15:29 |
| alcabrera | nice! | 15:30 |
| vkmc | the need for websockets for claims is still not clear yet | 15:30 |
| alcabrera | it sounded like that was related to the connection affinity problem with amqp < 1.0 | 15:30 |
| vkmc | yes it does | 15:30 |
| alcabrera | where one must issue an ACK on the same connection one started with | 15:30 |
| vkmc | current AMQP 1.0 docs are not enough | 15:31 |
| alcabrera | I saw that flaper87|afk is investigating this more thoroughly on our behalf. I super appreciate that he's checking that out for us! | 15:31 |
| vkmc | so... I'll poke some Proton devs today and ask them about the undocumented functions/attributes in the lib | 15:31 |
| alcabrera | sounds good. I hope we'll get some more answers there. :) | 15:32 |
| alcabrera | amqp support in marconi is turning out to be quite the collaborative effort, and a bit of a research project! | 15:32 |
| vkmc | I'm hoping that too | 15:32 |
| vkmc | of course! | 15:32 |
| alcabrera | :) | 15:32 |
| *** haomai___ has joined #openstack-marconi | 15:32 | |
| vkmc | it's a different way of managing storage | 15:33 |
| alcabrera | very much so | 15:33 |
| vkmc | so there is a lot to discuss and there will be a lot do polish too | 15:33 |
| vkmc | I was hoping that flaper87 could provide some feedback about the server affinity issue | 15:34 |
| alcabrera | me, too | 15:34 |
| vkmc | he knows Proton better and probably he thought about a workaround for that | 15:34 |
| alcabrera | the latest I saw from flaper87|afk is that | 15:34 |
| alcabrera | we might be able to work around it by putting claims on a separate amqp-topic | 15:34 |
| alcabrera | something like that | 15:34 |
| vkmc | sounds good | 15:34 |
| vkmc | in the mean time, it would be good to start seeing some progress and have the basic driver support | 15:35 |
| alcabrera | definitely! | 15:35 |
| *** haomaiwang has quit IRC | 15:35 | |
| alcabrera | do you think you'll be able to simulate basic messaging support (-claims) where you're at now, vkmc? | 15:35 |
| vkmc | yes I think so :) | 15:36 |
| vkmc | I'm hoping to have it done for Monday | 15:36 |
| alcabrera | yay! | 15:36 |
| vkmc | so if something comes up we can discuss about it on Tuesday's meeting | 15:36 |
| alcabrera | I'm happy to hear we've made it to a point where we can begin playing with the code. :D | 15:36 |
| alcabrera | oh yeah! | 15:36 |
| vkmc | yes me too :) | 15:37 |
| alcabrera | good idea. we can leverage Tuesday's meeting to help cooordinate your effort and tjanczuk_'s | 15:37 |
| alcabrera | though they are a bit different, given the 0.9 - 1.0 diff | 15:37 |
| vkmc | yes there are really different | 15:37 |
| vkmc | s/there/they | 15:37 |
| alcabrera | other thoughts/concerns, vkmc? :) | 15:37 |
| vkmc | but well, it's good that we are joining efforts and working on the support for both of them | 15:38 |
| alcabrera | agreed | 15:38 |
| vkmc | I have one tiny doubt about this bp https://blueprints.launchpad.net/marconi/+spec/support-amq | 15:39 |
| alcabrera | oh? | 15:39 |
| vkmc | afaik we still don't need big changes for AMQP 1.0 | 15:39 |
| vkmc | we can make it work without changing the API | 15:40 |
| vkmc | that will change if it turns that we need websockets | 15:40 |
| alcabrera | oh, yeah. :/ | 15:40 |
| alcabrera | we'll need to watch that space. | 15:40 |
| alcabrera | if we determine that API changes are needed to support amqp, that'll be quite the discussion | 15:40 |
| alcabrera | though | 15:40 |
| vkmc | yeah | 15:41 |
| alcabrera | it's already known that to support amqp+marconi would lead to partial API support | 15:41 |
| vkmc | so the thing is... I'll start working on this bp https://blueprints.launchpad.net/marconi/+spec/storage-amqp-1.0 | 15:41 |
| alcabrera | that some operations aren't possible as is | 15:41 |
| alcabrera | *as-is | 15:41 |
| vkmc | and if we discover that we have to change the API (and we all agree on that) I'll take the first one | 15:41 |
| vkmc | yeah | 15:42 |
| alcabrera | awesome! | 15:42 |
| kgriffs | speaking of partial API support... we should discuss sometime soon how to approach that | 15:42 |
| kgriffs | brb (meeting) | 15:42 |
| vkmc | +1 kgriffs | 15:42 |
| alcabrera | kgriffs: agreed. :) | 15:42 |
| alcabrera | did I address all your concerns, vkmc? | 15:43 |
| vkmc | you did alcabrera, thanks | 15:43 |
| alcabrera | cool. | 15:43 |
| kgriffs | anybody know the status of this? https://blueprints.launchpad.net/marconi/+spec/decoupling-unit-tests | 15:43 |
| alcabrera | kgriffs: in review, as far as I've seen | 15:43 |
| vkmc | nothing else to bring up, hope to have some code to peek for next meeting :) | 15:43 |
| alcabrera | sriram was working on it last I checked | 15:43 |
| alcabrera | AAzza, prashanthr_, vkmc: we're drawing to a close on the meeting for this week. Any parting thoughts/concerns? :) | 15:44 |
| vkmc | not sriram, alex :) https://review.openstack.org/#/c/97534/ | 15:44 |
| alcabrera | ah! | 15:44 |
| alcabrera | thanks for the correction, vkmc | 15:45 |
| vkmc | np! | 15:45 |
| *** jchai is now known as jchai_afk | 15:45 | |
| alcabrera | so that's it for the meeting this week | 15:45 |
| alcabrera | thanks all for joining! | 15:45 |
| alcabrera | feel free to reach me if you all need any help | 15:45 |
| vkmc | thanks alcabrera, prashanthr_, AAzza :) | 15:45 |
| *** jchai_afk is now known as jchai | 15:45 | |
| prashanthr_ | alcabrera: I will start writing the tests for redis specific drivers from tmrw. I have one more day b4 the weekend to ask q's :) thanks alcabrera, vkmc, AAzza :) | 15:46 |
| vkmc | +1 prashanthr_ | 15:47 |
| kgriffs | AAzza: you are famous. I just bragged about your py3k work to Thierry, the openstack release manager. :D | 15:53 |
| *** cpallares has quit IRC | 15:54 | |
| alcabrera | wooo! | 15:55 |
| AAzza | kgriffs: Oh... :-[ | 15:55 |
| kgriffs | keep up the good work. | 15:56 |
| prashanthr_ | Aazza: awesome :) | 15:58 |
| prashanthr_ | kgriffs: I discussed with oslo.cache team today morning regarding adding support for redis | 16:00 |
| malini | AAzza: woot !! | 16:00 |
| prashanthr_ | malini : Hi good morning :) | 16:00 |
| malini | hello prashanthr_! | 16:01 |
| *** oz_akan_ has quit IRC | 16:03 | |
| prashanthr_ | kgriffs: They have asked me to create a spec for it and have also given me some sample templates to follow. | 16:05 |
| prashanthr_ | I will focus on that after completing the work on redis support for marconi | 16:06 |
| *** goodes has quit IRC | 16:07 | |
| kgriffs | prashanthr_: let us know how the spec process goes. A lot of projects are experimenting with that this cycle. | 16:08 |
| prashanthr_ | kgriffs: Sure. the spec writing seems to be fairly straight forward. I will keep you updated on it. | 16:09 |
| kgriffs | vkmc, alcabrera: thanks for the update. I see why the status on the bp wasn't auto-updated; Alex misspelled the blueprint name in the commit message. :p | 16:10 |
| kgriffs | prashanthr_: is it just an RST template you fill out or something? | 16:11 |
| *** prashanthr_ has left #openstack-marconi | 16:12 | |
| *** cpallares has joined #openstack-marconi | 16:14 | |
| *** rossk has joined #openstack-marconi | 16:14 | |
| *** goodes has joined #openstack-marconi | 16:17 | |
| *** openstackgerrit has quit IRC | 16:20 | |
| *** openstackgerrit has joined #openstack-marconi | 16:21 | |
| *** balajiiyer has quit IRC | 16:22 | |
| openstackgerrit | Alex Bettadapur proposed a change to openstack/marconi: Decoupled Unit Tests https://review.openstack.org/97534 | 16:23 |
| *** jmckind has quit IRC | 16:26 | |
| *** ovorp has joined #openstack-marconi | 16:35 | |
| *** haomai___ has quit IRC | 16:58 | |
| *** ykaplan has quit IRC | 16:59 | |
| *** jamie_h has quit IRC | 17:10 | |
| *** reed has joined #openstack-marconi | 17:18 | |
| tjanczuk_ | vkmc, flapper87: what are your thoughts currently on implementing claims on top of AMQP 1.0? Did you find anything new since yesterday? | 17:31 |
| *** oz_akan_ has joined #openstack-marconi | 17:35 | |
| tjanczuk_ | vkmc, flapper87: in particular, does the AMQP 1.0 library and any of the actual AMQP 1.0 backends allow acking messages using a different connection these messages were received on? | 17:35 |
| tjanczuk_ | flapper87: could you expand on the idea of using AMQP topics to represent claims mentioned earlier? | 17:36 |
| *** oz_akan_ has quit IRC | 17:39 | |
| *** goodes has quit IRC | 17:53 | |
| *** jmckind has joined #openstack-marconi | 17:54 | |
| *** jmckind has quit IRC | 18:07 | |
| *** jmckind has joined #openstack-marconi | 18:08 | |
| kgriffs | tjanczuk_: usually folks are more responsive in IRC, but I think flaper87 was on the road for a conference or something. You might try the mailing list. | 18:11 |
| kgriffs | (FWIW) | 18:11 |
| kgriffs | gotta run for a bit (lunch) | 18:11 |
| tjanczuk_ | thanks | 18:11 |
| kgriffs | vkmc: was in here earlier... not sure where she went. :p | 18:12 |
| *** kgriffs is now known as kgriffs|afk | 18:12 | |
| *** reed has quit IRC | 18:15 | |
| *** reed has joined #openstack-marconi | 18:16 | |
| *** balajiiyer has joined #openstack-marconi | 18:26 | |
| *** oz_akan_ has joined #openstack-marconi | 18:35 | |
| *** oz_akan__ has joined #openstack-marconi | 18:38 | |
| *** flaper87|phone has joined #openstack-marconi | 18:39 | |
| *** oz_akan_ has quit IRC | 18:40 | |
| *** jmckind has quit IRC | 18:47 | |
| *** flaper87|phone has quit IRC | 18:58 | |
| *** Obulpathi has quit IRC | 18:59 | |
| openstackgerrit | Nataliia Uvarova proposed a change to openstack/marconi: Small fixes in storage drivers for Python 3 https://review.openstack.org/98214 | 19:01 |
| vkmc | here here! | 19:02 |
| vkmc | tjanczuk_, I started working on adding queues and messages for AMQP 1.0, nothing new on claims since yesterday | 19:03 |
| alcabrera | AAzza: why doesn't map(str, collection) work, as compared to [str(x) for x in collection]? | 19:03 |
| *** jchai is now known as jchai_afk | 19:04 | |
| AAzza | because map returns iterator) | 19:05 |
| tjanczuk_ | vkmc: so without claims, how do you propose to consume messages? I thought since the "get & consume" HTTP API is not in yet, there is really no way, is there? | 19:05 |
| AAzza | alcabrera: in py3 I mean, and functions that use this storage api, kinda not happy to be able to see the values only once) | 19:06 |
| alcabrera | ah | 19:06 |
| alcabrera | I see | 19:06 |
| alcabrera | good catch! | 19:06 |
| *** Obulpathi has joined #openstack-marconi | 19:07 | |
| alcabrera | AAzza: +2'd that patch | 19:07 |
| AAzza | alcabrera: you are too fast) I' m just writing comments will the doubts about the patch) but thanks anyway) | 19:09 |
| alcabrera | hehe | 19:11 |
| alcabrera | I think utf8 is a safe default over ascii, AAzza | 19:11 |
| vkmc | tjanczuk_, the only problem with claims is the ability to access the claimed messages, but the send and receive funcionalities don't have that problem since we can send a response on receive (is a feature on 1.0) | 19:12 |
| vkmc | s/ability/inability | 19:13 |
| vkmc | tjanczuk_, I heading on that direction for now... a basic driver with these functionalities | 19:13 |
| vkmc | tjanczuk_, after that we would need to discuss the claims operation | 19:14 |
| tjanczuk_ | vkmc: maybe I misunderstand something about current state of APIs. What is the HTTP API I call today to receive a mesaege? | 19:14 |
| alcabrera | AAzza: I think the .format solution for handling marker_encode is good enough. | 19:15 |
| alcabrera | it's consistent between versions and easy to understand | 19:15 |
| vkmc | tjanczuk_, https://github.com/openstack/marconi | 19:16 |
| AAzza | alcabrera: yes, then let it be as it is) | 19:16 |
| AAzza | good night everybody | 19:17 |
| alcabrera | AAzza: good night! | 19:17 |
| vkmc | AAzza, nite :) | 19:17 |
| tjanczuk_ | vkmc: so I was looking at the Mongo implementation of the Message.get, Message.first, and Message.bulk_get. None of them seem to *remove* messages from the database, just read them. It seems to me the only API that removes messages in Message.delete, but it requires both ClaimID and MessageID. What am I missing? | 19:19 |
| vkmc | tjanczuk_, claimed messages can be deleted only for the consumer who claimed it | 19:21 |
| vkmc | tjanczuk_, messages can be either active or claimed | 19:22 |
| vkmc | tjanczuk_, if they are active, any consumer can get a copy of it or delete it | 19:22 |
| vkmc | tjanczuk_, if it is claimed, just the consumer who previously claimed it can get a copy of it or delete it | 19:22 |
| *** AAzza is now known as AAzza_afk | 19:22 | |
| tjanczuk_ | vkmc, I understand the model. All I am saying is the the *only* way to physically delete a message is to call Messages.delete. Is that correct? | 19:27 |
| tjanczuk_ | vkmc, the Mongo implementation of Message.delete *requires* at least messageID to be provided. How do you plan to reconcile that with AMQP? | 19:28 |
| vkmc | tjanczuk_, oh cool, sorry I wasn't sure if we were on the same page regarding the claims functionality | 19:30 |
| vkmc | tjanczuk_, well Proton messages have an ID that can be used to filter them | 19:31 |
| tjanczuk_ | vkmc, let's talk about a workflow of permanently consuming a single message off of the queue. What HTTP APIs would be called and how to they map onto AMQP 1.0? | 19:34 |
| vkmc | tjanczuk_, for now we should focus on the simple get, this is without id, since we were talking about the remove that from the API https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg25385.html | 19:35 |
| vkmc | s/the remove/removing | 19:35 |
| tjanczuk_ | vkmc, in your proposal, are you assuming the removal of IDs from these APIs, or are you trying to build on what's in the tree right now? I am referring to what's in the code right now. | 19:39 |
| vkmc | tjanczuk_, I'm assuming the API as is, but I'm considering the possibility of those functionalities being removed | 19:40 |
| vkmc | tjanczuk_, it's just a side note | 19:40 |
| tjanczuk_ | vkmc, so I still don't understand how you plan to implement the "consume one message from a queue | 19:40 |
| tjanczuk_ | using today's storage contract mapped onto AMQP. Can you walk me through it? What APIs are called and in which order? How do they map to AMQP? | 19:41 |
| vkmc | tjanczuk_, I'm implementing those right now, would you like to discuss more about this tomorrow? Proton lib have many features that can be used to handle messages, I'm figuring out how to use them accordingly | 19:44 |
| tjanczuk_ | OK, let's talk more about it tomorrow. | 19:46 |
| vkmc | tjanczuk_, how are things going on your side with AMQP < 1.0? | 19:47 |
| tjanczuk_ | I think supporting AMQP 0.9 will require changes in both HTTP APIs and storage driver abstraction. (I also suspect the same will turn out to be true for AMQP 1.0). | 19:49 |
| tjanczuk_ | Some of the issues I encountered: | 19:49 |
| tjanczuk_ | 1) AMQP 0.9 requires message acknowledgement on the same instance of AMQP channel that the message was received on. This is a major constraint that I think can only be reasonably addressed by exposing a connection-oriented endpoint to the client (e.g. WebSockets) which will support server affinitity. | 19:52 |
| tjanczuk_ | 2) The concept of message ID does not exist in AMQP 0.9, so all operations that accept or return message ID would require changes. | 19:52 |
| tjanczuk_ | 3) The concept of a "claim" in Marconi is unlike the corresponding concept in AMQP (0.9, 1.0), SQS, or Azure Queues. In Marconi a claim maps to N messages, and (in the current implementation) must be used in conjunction with a message ID to delete the message. In AMQP (0.9 and 1.0), as well as SQS and Azure Queues, a "claim" is associated with a single message. Think about it as a transient message ID created by the server at the time message is | 19:57 |
| tjanczuk_ | 4) Of course all the APIs that list messages or get messages by ID cannot be supported by AMQP, but we knew this all along. | 19:58 |
| tjanczuk_ | 5) The Message.first API is strange in that it allows sorting the queue. No sorting can be done with AMQP. I am not quite sure I understand why that API is there (except sorting), since the same can be accomplished with Message.get | 19:59 |
| *** jchai_afk is now known as jchai | 19:59 | |
| tjanczuk_ | So........ | 19:59 |
| tjanczuk_ | It looks more and more likely to me there is no way AMQP can be accommodated with just cosmetic changes to V1.1 HTTP APIs or storage driver abstraction. | 20:00 |
| tjanczuk_ | I think I am going to spend some time prototyping how a "V2" HTTP API that accommodates well both Mongo and AMQP would look like, and validate that HTTP API with AMQP 0.9 implementation. Thoughts? | 20:02 |
| tjanczuk_ | This is what alcabrera referred yesterday as a "new transport" I think. | 20:03 |
| *** goodes has joined #openstack-marconi | 20:07 | |
| *** flwang has quit IRC | 20:07 | |
| *** malini is now known as malini_afk | 20:12 | |
| *** goodes has left #openstack-marconi | 20:16 | |
| *** flwang has joined #openstack-marconi | 20:17 | |
| *** sriram has quit IRC | 20:33 | |
| *** balajiiyer has quit IRC | 20:39 | |
| *** balajiiyer has joined #openstack-marconi | 20:48 | |
| *** alcabrera is now known as alcabrera|afk | 20:54 | |
| *** balajiiyer has quit IRC | 20:58 | |
| *** malini_afk is now known as malini | 21:06 | |
| *** Obulpathi has quit IRC | 21:45 | |
| *** oz_akan__ has quit IRC | 21:46 | |
| *** jchai has quit IRC | 21:50 | |
| *** malini is now known as malini_afk | 21:54 | |
| *** malini_afk is now known as malini | 22:00 | |
| *** cpallares has quit IRC | 22:01 | |
| *** oz_akan_ has joined #openstack-marconi | 22:48 | |
| *** oz_akan_ has quit IRC | 22:55 | |
| *** malini is now known as malini_afk | 23:15 | |
| *** sriram has joined #openstack-marconi | 23:36 | |
| *** sriram has quit IRC | 23:48 | |
| *** oz_akan_ has joined #openstack-marconi | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!