| *** shakamunyi has quit IRC | 00:05 | |
| *** kgriffs is now known as kgriffs|afk | 00:10 | |
| *** ayoung_exercise has quit IRC | 00:31 | |
| *** flwang has quit IRC | 00:54 | |
| *** balajiiyer has joined #openstack-marconi | 00:59 | |
| *** balajiiyer has quit IRC | 01:03 | |
| *** balajiiyer has joined #openstack-marconi | 01:04 | |
| *** balajiiyer has quit IRC | 01:06 | |
| *** balajiiyer has joined #openstack-marconi | 01:06 | |
| *** flwang has joined #openstack-marconi | 01:10 | |
| *** balajiiyer has quit IRC | 01:11 | |
| *** ayoung has joined #openstack-marconi | 01:12 | |
| *** nosnos has joined #openstack-marconi | 01:36 | |
| *** nosnos has quit IRC | 01:44 | |
| *** nosnos has joined #openstack-marconi | 01:44 | |
| *** openstackgerrit has quit IRC | 02:04 | |
| *** openstackgerrit has joined #openstack-marconi | 02:06 | |
| *** nosnos has quit IRC | 02:51 | |
| *** rossk_ has quit IRC | 03:02 | |
| *** ayoung is now known as ayoung_ZZzz__zz_ | 03:44 | |
| *** nosnos has joined #openstack-marconi | 04:05 | |
| *** whenry has quit IRC | 04:31 | |
| *** chandan_kumar has joined #openstack-marconi | 04:36 | |
| *** reed has quit IRC | 04:50 | |
| *** malini is now known as malini_afk | 05:03 | |
| *** chandan_kumar has quit IRC | 05:47 | |
| *** chandan_kumar has joined #openstack-marconi | 06:01 | |
| *** rwsu is now known as rwsu-away | 06:14 | |
| *** flaper87|afk is now known as flaper87 | 06:50 | |
| *** chandan_kumar has quit IRC | 06:51 | |
| *** openstackgerrit has quit IRC | 06:57 | |
| *** chandan_kumar has joined #openstack-marconi | 07:05 | |
| *** Manishanker has joined #openstack-marconi | 08:06 | |
| *** haomai___ has quit IRC | 08:10 | |
| *** haomaiwang has joined #openstack-marconi | 08:11 | |
| *** rektide_ has joined #openstack-marconi | 08:23 | |
| *** tmu_ has joined #openstack-marconi | 08:30 | |
| *** ayoung_ has joined #openstack-marconi | 08:32 | |
| *** torgomatic_ has joined #openstack-marconi | 08:34 | |
| *** tmu has quit IRC | 08:34 | |
| *** flaper87 has quit IRC | 08:34 | |
| *** alcabrera|afk has quit IRC | 08:34 | |
| *** seiflotfy has quit IRC | 08:34 | |
| *** torgomatic has quit IRC | 08:34 | |
| *** torgomatic_ is now known as torgomatic | 08:34 | |
| *** ayoung_ZZzz__zz_ has quit IRC | 08:34 | |
| *** rektide has quit IRC | 08:34 | |
| *** seiflotfy has joined #openstack-marconi | 08:38 | |
| *** chandan_kumar has quit IRC | 08:40 | |
| *** alcabrera|afk has joined #openstack-marconi | 08:48 | |
| *** flaper87 has joined #openstack-marconi | 08:53 | |
| *** dickson.freenode.net sets mode: +o flaper87 | 08:53 | |
| *** chandan_kumar has joined #openstack-marconi | 08:53 | |
| *** rwsu-away has quit IRC | 08:56 | |
| *** prashanthr_ has joined #openstack-marconi | 10:48 | |
| *** nosnos has quit IRC | 11:37 | |
| *** nosnos has joined #openstack-marconi | 11:53 | |
| flaper87 | kgriffs|afk: thoughts on this? https://wiki.openstack.org/wiki/Oslo/blueprints/service-sync | 11:59 |
|---|---|---|
| *** whenry has joined #openstack-marconi | 12:33 | |
| *** jchai has joined #openstack-marconi | 12:35 | |
| *** shakamunyi has joined #openstack-marconi | 12:36 | |
| *** nosnos has quit IRC | 12:39 | |
| *** shakamunyi has quit IRC | 12:41 | |
| *** jmckind has joined #openstack-marconi | 12:49 | |
| *** sriram has joined #openstack-marconi | 12:50 | |
| *** sriram has quit IRC | 12:54 | |
| *** vkmc has joined #openstack-marconi | 13:05 | |
| *** vkmc has quit IRC | 13:05 | |
| *** vkmc has joined #openstack-marconi | 13:05 | |
| *** sriram has joined #openstack-marconi | 13:11 | |
| *** sriram has joined #openstack-marconi | 13:11 | |
| *** ayoung_ is now known as ayoung | 13:12 | |
| *** amitgandhi has joined #openstack-marconi | 13:18 | |
| *** amitgandhi has quit IRC | 13:18 | |
| *** amitgandhi has joined #openstack-marconi | 13:18 | |
| *** alcabrera|afk is now known as alcabrera | 13:22 | |
| prashanthr_ | Good morning :) | 13:30 |
| alcabrera | good morning, all. :) | 13:30 |
| alcabrera | prashanthr_: heya! | 13:30 |
| prashanthr_ | weekend around the corner ?? :) | 13:31 |
| alcabrera | oh, yes | 13:31 |
| flaper87 | goooooooood morning | 13:31 |
| alcabrera | and I am looking forward to it~ :D | 13:31 |
| alcabrera | flaper87: hey! | 13:31 |
| flaper87 | alcabrera: hey hey :D | 13:32 |
| prashanthr_ | :D Me just 2 hrs away ;) | 13:32 |
| vkmc | Good morning all! :) | 13:32 |
| flaper87 | vkmc: goood morning :) | 13:32 |
| alcabrera | vkmc: hey! :) | 13:33 |
| vkmc | Happy labours day btw :) | 13:33 |
| vkmc | flaper87, alcabrera, prashanthr_ o/ | 13:34 |
| alcabrera | prashanthr_: hahaha, enjoy the head start on the weekend for me. ;P | 13:34 |
| prashanthr_ | alcabrera: I will I will :) | 13:34 |
| prashanthr_ | vkmc: Good morning :) | 13:34 |
| *** Obulpathi has joined #openstack-marconi | 13:36 | |
| *** Obulpathi has quit IRC | 13:36 | |
| *** Obulpathi has joined #openstack-marconi | 13:37 | |
| flaper87 | alcabrera: you mentioned you worked on the FAQ yday, didn't you? | 13:43 |
| alcabrera | flaper87: yup. I filled in the section on oslo.messaging | 13:45 |
| alcabrera | posted to the wiki and all. Let me know what you think, or if it needs more substance. :) | 13:45 |
| flaper87 | alcabrera: coold, did you do that on the etherpad or the wiki? | 13:45 |
| flaper87 | mmh, I checked on Marconi/FAQ and it wasn't there | 13:45 |
| alcabrera | flaper87: https://wiki.openstack.org/wiki/Marconi#How_does_Marconi_compare_to_oslo.messaging.3F | 13:45 |
| flaper87 | ah ok | 13:46 |
| alcabrera | I lost track of the etherpad URL and any where else we were keeping FAQ content. >.> | 13:47 |
| alcabrera | flaper87: ^^ | 13:47 |
| flaper87 | alcabrera: https://etherpad.openstack.org/p/draft-marconi-faq | 13:48 |
| flaper87 | :D | 13:48 |
| alcabrera | thank you, flaper87. :) | 13:48 |
| * alcabrera checks to see what is lelft | 13:49 | |
| alcabrera | *left | 13:49 |
| sriram | what version of mongo is the least that is required to marconi? 2.4? | 13:51 |
| alcabrera | I believe that's correct, sriram | 13:51 |
| flaper87 | sriram: 2.0 but we would like to make it 2.4 | 13:51 |
| alcabrera | 2.4 gives us... TTL collections? | 13:52 |
| sriram | phew.. just wrote a different query that is supported only in 2.4 for lazy create. | 13:52 |
| flaper87 | erm sorry, 2.2 | 13:52 |
| sriram | :( | 13:53 |
| flaper87 | alcabrera: 2.2 gives us TTL collections | 13:53 |
| sriram | update with upsert was causing issues with a stalled message counter. | 13:53 |
| sriram | so I changed it around to this. | 13:53 |
| sriram | self._collection.find_and_modify({'p_q': scoped_name},{'$setOnInsert':{'p_q': scoped_name, 'm': {},'c': counter}}, upsert=True) | 13:53 |
| sriram | and this works. | 13:53 |
| sriram | btw good morning :P | 13:54 |
| flaper87 | sriram: mmh, you don't need find_and_modify, you can do .update({}, {}, upsert=True) | 13:54 |
| sriram | tried that, causing message counter issues | 13:54 |
| flaper87 | (unless I'm missing something) | 13:55 |
| flaper87 | ah see | 13:55 |
| sriram | even with $set | 13:55 |
| alcabrera | sriram: good morning! | 13:55 |
| flaper87 | mhh, what am I missing | 13:55 |
| flaper87 | ? | 13:55 |
| flaper87 | sriram: good morning :) | 13:55 |
| sriram | heh | 13:55 |
| flaper87 | sriram: what issues? | 13:55 |
| sriram | stalled message counter, according to the logs | 13:56 |
| sriram | after which it is incremented | 13:56 |
| sriram | and we get a response, with message posted -> but it takes a lot of time.. | 13:56 |
| sriram | and we dont want a stalled counter :| | 13:56 |
| flaper87 | sriram: that's.... weird | 13:57 |
| flaper87 | :/ | 13:57 |
| sriram | "Detected a stalled message counter for queue "maafzda" under project 23453. The counter was incremented to 2" -> this is the message | 13:57 |
| sriram | we do a have an unique index on p_q correct? | 13:58 |
| flaper87 | sriram: what's the status of the queue at that point? I mean, is it created? | 13:58 |
| flaper87 | sriram: any code you can show ? | 13:58 |
| flaper87 | :D | 13:58 |
| sriram | yeah, it gets created and message gets posted | 13:58 |
| flaper87 | sriram: yeah | 13:58 |
| flaper87 | we've a unique index | 13:58 |
| sriram | sure. | 13:59 |
| sriram | upsert sample (Have tried most combinations here, just setting p_q as well)-> https://gist.github.com/anonymous/b5e7c80cd110096c3884 | 14:01 |
| sriram | findandmodify -> https://gist.github.com/anonymous/0f801780f2f43cdbb9b2 | 14:02 |
| sriram | self._collection.update({'p_q': scoped_name, 'm': {},'c': counter},{'$set':{'p_q': scoped_name, 'm': {},'c': counter}}, upsert=True) -> This one seems to work, but are we changing state with this? | 14:07 |
| *** openstackgerrit has joined #openstack-marconi | 14:09 | |
| sriram | flaper87: ^ | 14:10 |
| flaper87 | oh sorry, just noticed your messages | 14:10 |
| flaper87 | sriram: what do you mean with changing state? | 14:12 |
| flaper87 | sriram: also, you probably just want .update({'p_q': ..}, {'m': {}, 'c': counter}, upsert=True) | 14:15 |
| flaper87 | mongodb will create a new document mixing both, the query and the update value | 14:15 |
| sriram | oh | 14:15 |
| flaper87 | ahh wait | 14:16 |
| flaper87 | I know why you're getting the stalled counter | 14:16 |
| flaper87 | sriram: you need to have the counter in the query | 14:16 |
| flaper87 | that's the only way we have to make sure we're not changing an existing queue | 14:17 |
| flaper87 | .update({'p_q': scoped_name, 'c': counter},{'$set':{'m': {}}}, upsert=True) | 14:17 |
| sriram | oh, thats why the previous query worked. Yes, by state, I meant not overwriting anything | 14:17 |
| flaper87 | gotcha! | 14:18 |
| flaper87 | that one should work | 14:18 |
| sriram | thanks a lot flaper87 :) . I just wanted to make sure on the query, I didnt want it breaking anything :) | 14:18 |
| flaper87 | sriram: np | 14:19 |
| flaper87 | :) | 14:19 |
| * flaper87 is cheap, he just charges 500$ per answer | 14:19 | |
| sriram | heh | 14:19 |
| *** fifieldt has quit IRC | 14:21 | |
| *** prashanthr_ has left #openstack-marconi | 14:28 | |
| *** prashanthr_ has joined #openstack-marconi | 14:28 | |
| *** jdbarry_ has joined #openstack-marconi | 14:29 | |
| *** ametts has joined #openstack-marconi | 14:30 | |
| alcabrera | see y'all in a bit. Off to the dentist. o/ | 14:32 |
| *** alcabrera is now known as alcabrera|afk | 14:32 | |
| vkmc | \o | 14:32 |
| openstackgerrit | Sriram Madapusi Vasudevan proposed a change to openstack/marconi: Implement Lazy Create Queue in v1.1 API https://review.openstack.org/91804 | 14:38 |
| openstackgerrit | Sriram Madapusi Vasudevan proposed a change to openstack/marconi: Implement Lazy Create Queue in v1.1 API https://review.openstack.org/91804 | 14:40 |
| *** shakamunyi has joined #openstack-marconi | 14:54 | |
| *** shakamunyi has quit IRC | 14:56 | |
| *** kgriffs|afk is now known as kgriffs | 14:58 | |
| *** shakayumi has joined #openstack-marconi | 14:59 | |
| *** shakayumi has quit IRC | 15:03 | |
| *** shakamunyi has joined #openstack-marconi | 15:03 | |
| *** shakamunyi has quit IRC | 15:03 | |
| *** jchai is now known as jchai_afk | 15:10 | |
| *** jdbarry_ has quit IRC | 15:12 | |
| *** reed has joined #openstack-marconi | 15:13 | |
| *** shakamunyi has joined #openstack-marconi | 15:18 | |
| *** alcabrera|afk is now known as alcabrera | 15:19 | |
| *** jchai_afk is now known as jchai | 15:25 | |
| *** shakayumi has joined #openstack-marconi | 15:37 | |
| *** shakamunyi has quit IRC | 15:38 | |
| *** kgriffs is now known as kgriffs|afk | 15:41 | |
| *** jmckind has quit IRC | 15:43 | |
| *** jchai is now known as jchai_afk | 15:54 | |
| *** chandan_kumar has quit IRC | 15:55 | |
| *** kgriffs|afk is now known as kgriffs | 15:58 | |
| kgriffs | flaper87: re sevice-sync | 16:00 |
| kgriffs | etcd could be a could backend | 16:00 |
| kgriffs | s/could/good | 16:01 |
| kgriffs | (holy strange word substitution batman!) | 16:01 |
| alcabrera | I'm +1 for zookeeper on that front -- it sounds like synchronizing the control plane could benefit from paxos, flaper87, kgriffs | 16:05 |
| kgriffs | alcabrera: etcd uses Raft FWIW | 16:05 |
| alcabrera | interesting | 16:06 |
| alcabrera | thanks for sharing, kgriffs! | 16:06 |
| alcabrera | I didn't realize that | 16:07 |
| alcabrera | so yes | 16:07 |
| alcabrera | something along those lines. I couldn't compare them properly beyond knowing that Raft and Paxos provide equivalent guarantees, afaik. | 16:07 |
| * flaper87 back | 16:12 | |
| flaper87 | the service sounds nice. | 16:12 |
| *** Manishanker has quit IRC | 16:13 | |
| kgriffs | flaper87: would I get fired for adding an etcd section to that page? | 16:14 |
| kgriffs | :D | 16:14 |
| * flaper87 keeps getting distracted | 16:14 | |
| kgriffs | guess it should be brought up with the bp author first | 16:14 |
| kgriffs | flaper87: can you mention that to them? | 16:15 |
| flaper87 | so, i think the service is cool, it's funny because it's basically the 3 service that I've seem come up in the last couple of days which has a *similar* goal than MArconi's | 16:15 |
| kgriffs | how so? | 16:15 |
| flaper87 | I mean, similar in terms of "Api" -> backends that do all the thing | 16:15 |
| kgriffs | isn't this just an openstack API on top of etcd/zookeeper | 16:15 |
| kgriffs | oic | 16:15 |
| flaper87 | :) | 16:15 |
| * flaper87 puts a sock in his mouth to avoid ranting and being a jackass | 16:16 | |
| kgriffs | flaper87: this is sort of random, but I came across this section in the kafka docs: | 16:18 |
| kgriffs | http://kafka.apache.org/documentation.html#design | 16:19 |
| kgriffs | search for "consumer position" | 16:19 |
| kgriffs | here's my question | 16:19 |
| kgriffs | they talk about how they do the "equivalent of message acknowledgements" | 16:20 |
| kgriffs | but in the previous paragraph they say there is a problem with explicit ack: "First of all, if the consumer processes the message but fails before it can send an acknowledgement then the message will be consumed twice. " | 16:20 |
| kgriffs | I don't see how kafka's implementation avoids that. It seems to only make keeping track of client state really efficient. If my worker crashes, how will it know to roll back? It still has to be keeping some kind of out-of-band state on what message it was working on, so it can figure out whether it crashes after popping the message but before processing it, or after processing it but before getting the next | 16:22 |
| kgriffs | message. | 16:22 |
| kgriffs | I must be missing something | 16:23 |
| flaper87 | you are not | 16:23 |
| flaper87 | that was exactly my understanding | 16:23 |
| flaper87 | it's more like: If a message is in the queue, it is safe to assume it could've been partially processed or not processed at all | 16:23 |
| flaper87 | and the worker processing the message, if no worker-side state management is present, will have to know what to do | 16:24 |
| kgriffs | ok, so kafka doesn't magically solve the problem, they just make keeping state server-side really efficient | 16:24 |
| flaper87 | I went throw kafka's paper the other day and I also read kestrels docs. Both have things that seem good but others that I deffinitely don't like | 16:25 |
| kgriffs | flaper87: you should write a blog post about it. :D | 16:25 |
| flaper87 | kgriffs: ohh I will | 16:25 |
| * kgriffs will read that post | 16:25 | |
| flaper87 | as soon as I get some free time in my hands :P | 16:25 |
| kgriffs | flaper87: then you should write a book: "7 message brokers in 7 days" | 16:25 |
| flaper87 | I took several notes and I thought we could go through them during the summit | 16:26 |
| kgriffs | flaper87: good idea | 16:26 |
| flaper87 | LOL @ book | 16:26 |
| kgriffs | oops | 16:27 |
| kgriffs | s/days/weeks | 16:27 |
| kgriffs | There is a precedent: http://pragprog.com/book/rwdata/seven-databases-in-seven-weeks | 16:27 |
| * kgriffs would by flaper87's book | 16:27 | |
| kgriffs | s/by/buy | 16:27 |
| * kgriffs wonders what is wrong with his English today | 16:28 | |
| * flaper87 wonders what is wrong with kgriffs | 16:28 | |
| flaper87 | oppppsss | 16:28 |
| flaper87 | did I say that? | 16:28 |
| flaper87 | :P | 16:28 |
| kgriffs | ROFL | 16:28 |
| * kgriffs goes to replace his faulty language module | 16:28 | |
| flaper87 | and that's how kgriffs learned Russian | 16:29 |
| kgriffs | Я не знаю, що ви говорите. | 16:30 |
| alcabrera | oh goodness | 16:30 |
| alcabrera | flaper87 predicts the future | 16:30 |
| alcabrera | :P | 16:30 |
| flaper87 | I just knew he would pick the wrong language module to replace the faulty one | 16:35 |
| flaper87 | probelm is, we don't know what he is saying | 16:35 |
| flaper87 | :/ | 16:35 |
| flaper87 | brb, dinner | 16:37 |
| alcabrera | be back in a bit. o/ | 16:38 |
| *** alcabrera is now known as alcabrera|afk | 16:38 | |
| *** Obulpathi has quit IRC | 16:39 | |
| *** prashanthr_ has quit IRC | 16:41 | |
| *** jchai_afk is now known as jchai | 16:41 | |
| kgriffs | Ах немає. Всього лише за секунду. | 16:43 |
| * kgriffs Ставить в правильний модуль. | 16:43 | |
| kgriffs | Testing 1...2...3... Testing. | 16:44 |
| kgriffs | <phew> | 16:44 |
| * kgriffs is back in action | 16:44 | |
| *** rossk has joined #openstack-marconi | 16:51 | |
| *** jchai is now known as jchai_afk | 17:13 | |
| *** rossk_ has joined #openstack-marconi | 17:23 | |
| *** rossk has quit IRC | 17:24 | |
| *** alcabrera|afk is now known as alcabrera | 17:41 | |
| * alcabrera observes that kgriffs has JIT correction and self-healing capabilities! | 17:41 | |
| alcabrera | amazing. :D | 17:42 |
| *** Obulpathi has joined #openstack-marconi | 17:42 | |
| *** ayoung has quit IRC | 17:46 | |
| *** shakayumi has quit IRC | 17:52 | |
| kgriffs | Me and Data, we go waaaay back. | 17:52 |
| alcabrera | haha | 17:57 |
| alcabrera | Data is pretty amazing, kgriffs | 17:57 |
| alcabrera | ah, those Trek memories~ | 17:57 |
| alcabrera | I haven't experienced all of it, but just enough to know a little about a lot. :) | 17:57 |
| kgriffs | It's good stuff | 18:00 |
| *** jchai_afk is now known as jchai | 18:00 | |
| alcabrera | I'll be back in a bit. | 18:02 |
| *** alcabrera is now known as alcabrera|afk | 18:02 | |
| kgriffs | preview of the new Marconi home page: https://wiki.openstack.org/wiki/Marconi/home | 18:27 |
| *** Obulpathi has quit IRC | 18:37 | |
| *** Obulpathi has joined #openstack-marconi | 18:38 | |
| *** vkmc has quit IRC | 18:42 | |
| kgriffs | flaper87: when you get back, we need to publish the latest python-client to pypi | 18:51 |
| *** alcabrera|afk is now known as alcabrera | 19:02 | |
| *** whenry has quit IRC | 19:15 | |
| *** sriram has quit IRC | 19:15 | |
| alcabrera | have a good weekend, all! :) | 20:18 |
| *** alcabrera is now known as alcabrera|afk | 20:18 | |
| *** vkmc has joined #openstack-marconi | 20:20 | |
| *** whenry has joined #openstack-marconi | 20:40 | |
| *** Obulpathi has quit IRC | 21:00 | |
| *** reed has quit IRC | 21:00 | |
| *** ametts has quit IRC | 21:10 | |
| *** whenry has quit IRC | 21:11 | |
| flaper87 | kgriffs: back | 21:11 |
| flaper87 | kgriffs: yeah, I don't have the powers to do that :( | 21:11 |
| flaper87 | kgriffs: it's either you or ttx | 21:12 |
| flaper87 | kgriffs: I'm trying to remember what the process is | 21:12 |
| flaper87 | I think you just need to push a release commit | 21:13 |
| flaper87 | kgriffs: I think the process is. Tag the client, push the tag to gerrit and gerrit does the magic | 21:17 |
| flaper87 | kgriffs: I can try to do it, lets see if it works | 21:18 |
| *** jchai has quit IRC | 21:20 | |
| kgriffs | ok. that sounds familiar. | 21:29 |
| flaper87 | kgriffs: I tried *but* I still can't access gerrit | 21:30 |
| flaper87 | I've contacted my ISP | 21:30 |
| flaper87 | we | 21:30 |
| flaper87 | we'll see what happens | 21:31 |
| *** flaper87 is now known as flaper87|afk | 21:34 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/python-marconiclient: chore: Classify as Beta (was pre-alpha) https://review.openstack.org/91909 | 21:43 |
| kgriffs | flaper87|afk: ok, I can tag it on monday. | 21:44 |
| kgriffs | flaper87|afk: please review my patch. ^^^ | 21:44 |
| *** amitgandhi has quit IRC | 21:52 | |
| *** amitgandhi has joined #openstack-marconi | 21:52 | |
| *** amitgandhi has quit IRC | 21:57 | |
| -openstackstatus- NOTICE: Zuul is being restarted with some dependency upgrades and configuration changes; ETA 2215 | 21:59 | |
| *** kgriffs is now known as kgriffs|afk | 22:28 | |
| *** vkmc has quit IRC | 23:01 | |
| *** vkmc has joined #openstack-marconi | 23:05 | |
| *** vkmc has quit IRC | 23:05 | |
| *** vkmc has joined #openstack-marconi | 23:05 | |
| *** rupsky has joined #openstack-marconi | 23:12 | |
| *** rupsky has left #openstack-marconi | 23:12 | |
| -openstackstatus- NOTICE: paste.openstack.org is going down for a short database upgrade | 23:26 | |
| *** amitgandhi has joined #openstack-marconi | 23:47 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!