Friday, 2014-05-02

*** shakamunyi has quit IRC00:05
*** kgriffs is now known as kgriffs|afk00:10
*** ayoung_exercise has quit IRC00:31
*** flwang has quit IRC00:54
*** balajiiyer has joined #openstack-marconi00:59
*** balajiiyer has quit IRC01:03
*** balajiiyer has joined #openstack-marconi01:04
*** balajiiyer has quit IRC01:06
*** balajiiyer has joined #openstack-marconi01:06
*** flwang has joined #openstack-marconi01:10
*** balajiiyer has quit IRC01:11
*** ayoung has joined #openstack-marconi01:12
*** nosnos has joined #openstack-marconi01:36
*** nosnos has quit IRC01:44
*** nosnos has joined #openstack-marconi01:44
*** openstackgerrit has quit IRC02:04
*** openstackgerrit has joined #openstack-marconi02:06
*** nosnos has quit IRC02:51
*** rossk_ has quit IRC03:02
*** ayoung is now known as ayoung_ZZzz__zz_03:44
*** nosnos has joined #openstack-marconi04:05
*** whenry has quit IRC04:31
*** chandan_kumar has joined #openstack-marconi04:36
*** reed has quit IRC04:50
*** malini is now known as malini_afk05:03
*** chandan_kumar has quit IRC05:47
*** chandan_kumar has joined #openstack-marconi06:01
*** rwsu is now known as rwsu-away06:14
*** flaper87|afk is now known as flaper8706:50
*** chandan_kumar has quit IRC06:51
*** openstackgerrit has quit IRC06:57
*** chandan_kumar has joined #openstack-marconi07:05
*** Manishanker has joined #openstack-marconi08:06
*** haomai___ has quit IRC08:10
*** haomaiwang has joined #openstack-marconi08:11
*** rektide_ has joined #openstack-marconi08:23
*** tmu_ has joined #openstack-marconi08:30
*** ayoung_ has joined #openstack-marconi08:32
*** torgomatic_ has joined #openstack-marconi08:34
*** tmu has quit IRC08:34
*** flaper87 has quit IRC08:34
*** alcabrera|afk has quit IRC08:34
*** seiflotfy has quit IRC08:34
*** torgomatic has quit IRC08:34
*** torgomatic_ is now known as torgomatic08:34
*** ayoung_ZZzz__zz_ has quit IRC08:34
*** rektide has quit IRC08:34
*** seiflotfy has joined #openstack-marconi08:38
*** chandan_kumar has quit IRC08:40
*** alcabrera|afk has joined #openstack-marconi08:48
*** flaper87 has joined #openstack-marconi08:53
*** sets mode: +o flaper8708:53
*** chandan_kumar has joined #openstack-marconi08:53
*** rwsu-away has quit IRC08:56
*** prashanthr_ has joined #openstack-marconi10:48
*** nosnos has quit IRC11:37
*** nosnos has joined #openstack-marconi11:53
flaper87kgriffs|afk: thoughts on this?
*** whenry has joined #openstack-marconi12:33
*** jchai has joined #openstack-marconi12:35
*** shakamunyi has joined #openstack-marconi12:36
*** nosnos has quit IRC12:39
*** shakamunyi has quit IRC12:41
*** jmckind has joined #openstack-marconi12:49
*** sriram has joined #openstack-marconi12:50
*** sriram has quit IRC12:54
*** vkmc has joined #openstack-marconi13:05
*** vkmc has quit IRC13:05
*** vkmc has joined #openstack-marconi13:05
*** sriram has joined #openstack-marconi13:11
*** sriram has joined #openstack-marconi13:11
*** ayoung_ is now known as ayoung13:12
*** amitgandhi has joined #openstack-marconi13:18
*** amitgandhi has quit IRC13:18
*** amitgandhi has joined #openstack-marconi13:18
*** alcabrera|afk is now known as alcabrera13:22
prashanthr_Good morning :)13:30
alcabreragood morning, all. :)13:30
alcabreraprashanthr_: heya!13:30
prashanthr_weekend around the corner ?? :)13:31
alcabreraoh, yes13:31
flaper87goooooooood morning13:31
alcabreraand I am looking forward to it~ :D13:31
alcabreraflaper87: hey!13:31
flaper87alcabrera: hey hey :D13:32
prashanthr_:D Me just 2 hrs away ;)13:32
vkmcGood morning all! :)13:32
flaper87vkmc: goood morning :)13:32
alcabreravkmc: hey! :)13:33
vkmcHappy labours day btw :)13:33
vkmcflaper87, alcabrera, prashanthr_ o/13:34
alcabreraprashanthr_: hahaha, enjoy the head start on the weekend for me. ;P13:34
prashanthr_alcabrera: I will I will :)13:34
prashanthr_vkmc: Good morning :)13:34
*** Obulpathi has joined #openstack-marconi13:36
*** Obulpathi has quit IRC13:36
*** Obulpathi has joined #openstack-marconi13:37
flaper87alcabrera: you mentioned you worked on the FAQ yday, didn't you?13:43
alcabreraflaper87: yup. I filled in the section on oslo.messaging13:45
alcabreraposted to the wiki and all. Let me know what you think, or if it needs more substance. :)13:45
flaper87alcabrera: coold, did you do that on the etherpad or the wiki?13:45
flaper87mmh, I checked on Marconi/FAQ and it wasn't there13:45
flaper87ah ok13:46
alcabreraI lost track of the etherpad URL and any where else we were keeping FAQ content. >.>13:47
alcabreraflaper87: ^^13:47
alcabrerathank you, flaper87. :)13:48
* alcabrera checks to see what is lelft13:49
sriramwhat version of mongo is the least that is required to marconi? 2.4?13:51
alcabreraI believe that's correct, sriram13:51
flaper87sriram: 2.0 but we would like to make it 2.413:51
alcabrera2.4 gives us... TTL collections?13:52
sriramphew.. just wrote a different query that is supported only in 2.4 for lazy create.13:52
flaper87erm sorry, 2.213:52
flaper87alcabrera: 2.2 gives us TTL collections13:53
sriramupdate with upsert was causing issues with a stalled message counter.13:53
sriramso I changed it around to this.13:53
sriramself._collection.find_and_modify({'p_q': scoped_name},{'$setOnInsert':{'p_q': scoped_name, 'm': {},'c': counter}}, upsert=True)13:53
sriramand this works.13:53
srirambtw good morning :P13:54
flaper87sriram: mmh, you don't need find_and_modify, you can do .update({}, {}, upsert=True)13:54
sriramtried that, causing message counter issues13:54
flaper87(unless I'm missing something)13:55
flaper87ah see13:55
srirameven with $set13:55
alcabrerasriram: good morning!13:55
flaper87mhh, what am I missing13:55
flaper87sriram: good morning :)13:55
flaper87sriram: what issues?13:55
sriramstalled message counter, according to the logs13:56
sriramafter which it is incremented13:56
sriramand we get a response, with message posted -> but it takes a lot of time..13:56
sriramand we dont want a stalled counter :|13:56
flaper87sriram: that's.... weird13:57
sriram"Detected a stalled message counter for queue "maafzda" under project 23453. The counter was incremented to 2" -> this is the message13:57
sriramwe do a have an unique index on p_q correct?13:58
flaper87sriram: what's the status of the queue at that point? I mean, is it created?13:58
flaper87sriram: any code you can show ?13:58
sriramyeah, it gets created and message gets posted13:58
flaper87sriram: yeah13:58
flaper87we've a unique index13:58
sriramupsert sample (Have tried most combinations here, just setting p_q as well)->
sriramfindandmodify ->
sriramself._collection.update({'p_q': scoped_name, 'm': {},'c': counter},{'$set':{'p_q': scoped_name, 'm': {},'c': counter}}, upsert=True) -> This one seems to work, but are we changing state with this?14:07
*** openstackgerrit has joined #openstack-marconi14:09
sriramflaper87: ^14:10
flaper87oh sorry, just noticed your messages14:10
flaper87sriram: what do you mean with changing state?14:12
flaper87sriram: also, you probably just want .update({'p_q': ..}, {'m': {}, 'c': counter}, upsert=True)14:15
flaper87mongodb will create a new document mixing both, the query and the update value14:15
flaper87ahh wait14:16
flaper87I know why you're getting the stalled counter14:16
flaper87sriram: you need to have the counter in the query14:16
flaper87that's the only way we have to make sure we're not changing an existing queue14:17
flaper87.update({'p_q': scoped_name, 'c': counter},{'$set':{'m': {}}}, upsert=True)14:17
sriramoh, thats why the previous query worked. Yes, by state, I meant not overwriting anything14:17
flaper87that one should work14:18
sriramthanks a lot flaper87 :) . I just wanted to make sure on the query, I didnt want it breaking anything :)14:18
flaper87sriram: np14:19
* flaper87 is cheap, he just charges 500$ per answer14:19
*** fifieldt has quit IRC14:21
*** prashanthr_ has left #openstack-marconi14:28
*** prashanthr_ has joined #openstack-marconi14:28
*** jdbarry_ has joined #openstack-marconi14:29
*** ametts has joined #openstack-marconi14:30
alcabrerasee y'all in a bit. Off to the dentist. o/14:32
*** alcabrera is now known as alcabrera|afk14:32
openstackgerritSriram Madapusi Vasudevan proposed a change to openstack/marconi: Implement Lazy Create Queue in v1.1 API
openstackgerritSriram Madapusi Vasudevan proposed a change to openstack/marconi: Implement Lazy Create Queue in v1.1 API
*** shakamunyi has joined #openstack-marconi14:54
*** shakamunyi has quit IRC14:56
*** kgriffs|afk is now known as kgriffs14:58
*** shakayumi has joined #openstack-marconi14:59
*** shakayumi has quit IRC15:03
*** shakamunyi has joined #openstack-marconi15:03
*** shakamunyi has quit IRC15:03
*** jchai is now known as jchai_afk15:10
*** jdbarry_ has quit IRC15:12
*** reed has joined #openstack-marconi15:13
*** shakamunyi has joined #openstack-marconi15:18
*** alcabrera|afk is now known as alcabrera15:19
*** jchai_afk is now known as jchai15:25
*** shakayumi has joined #openstack-marconi15:37
*** shakamunyi has quit IRC15:38
*** kgriffs is now known as kgriffs|afk15:41
*** jmckind has quit IRC15:43
*** jchai is now known as jchai_afk15:54
*** chandan_kumar has quit IRC15:55
*** kgriffs|afk is now known as kgriffs15:58
kgriffsflaper87: re sevice-sync16:00
kgriffsetcd could be a could backend16:00
kgriffs(holy strange word substitution batman!)16:01
alcabreraI'm +1 for zookeeper on that front -- it sounds like synchronizing the control plane could benefit from paxos, flaper87, kgriffs16:05
kgriffsalcabrera: etcd uses Raft FWIW16:05
alcabrerathanks for sharing, kgriffs!16:06
alcabreraI didn't realize that16:07
alcabreraso yes16:07
alcabrerasomething along those lines. I couldn't compare them properly beyond knowing that Raft and Paxos provide equivalent guarantees, afaik.16:07
* flaper87 back16:12
flaper87the service sounds nice.16:12
*** Manishanker has quit IRC16:13
kgriffsflaper87: would I get fired for adding an etcd section to that page?16:14
* flaper87 keeps getting distracted16:14
kgriffsguess it should be brought up with the bp author first16:14
kgriffsflaper87: can you mention that to them?16:15
flaper87so, i think the service is cool, it's funny because it's basically the 3 service that I've seem come up in the last couple of days which has a *similar* goal than MArconi's16:15
kgriffshow so?16:15
flaper87I mean, similar in terms of "Api" -> backends that do all the thing16:15
kgriffsisn't this just an openstack API on top of etcd/zookeeper16:15
* flaper87 puts a sock in his mouth to avoid ranting and being a jackass16:16
kgriffsflaper87: this is sort of random, but I came across this section in the kafka docs:16:18
kgriffssearch for "consumer position"16:19
kgriffshere's my question16:19
kgriffsthey talk about how they do the "equivalent of message acknowledgements"16:20
kgriffsbut in the previous paragraph they say there is a problem with explicit ack: "First of all, if the consumer processes the message but fails before it can send an acknowledgement then the message will be consumed twice. "16:20
kgriffsI don't see how kafka's implementation avoids that. It seems to only make keeping track of client state really efficient. If my worker crashes, how will it know to roll back? It still has to be keeping some kind of out-of-band state on what message it was working on, so it can figure out whether it crashes after popping the message but before processing it, or after processing it but before getting the next16:22
kgriffsI must be missing something16:23
flaper87you are not16:23
flaper87that was exactly my understanding16:23
flaper87it's more like: If a message is in the queue, it is safe to assume it could've been partially processed or not processed at all16:23
flaper87and the worker processing the message, if no worker-side state management is present, will have to know what to do16:24
kgriffsok, so kafka doesn't magically solve the problem, they just make keeping state server-side really efficient16:24
flaper87I went throw kafka's paper the other day and I also read kestrels docs. Both have things that seem good but others that I deffinitely don't like16:25
kgriffsflaper87: you should write a blog post about it. :D16:25
flaper87kgriffs: ohh I will16:25
* kgriffs will read that post16:25
flaper87as soon as I get some free time in my hands :P16:25
kgriffsflaper87: then you should write a book: "7 message brokers in 7 days"16:25
flaper87I took several notes and I thought we could go through them during the summit16:26
kgriffsflaper87: good idea16:26
flaper87LOL @ book16:26
kgriffsThere is a precedent:
* kgriffs would by flaper87's book16:27
* kgriffs wonders what is wrong with his English today16:28
* flaper87 wonders what is wrong with kgriffs16:28
flaper87did I say that?16:28
* kgriffs goes to replace his faulty language module16:28
flaper87and that's how kgriffs learned Russian16:29
kgriffsЯ не знаю, що ви говорите.16:30
alcabreraoh goodness16:30
alcabreraflaper87 predicts the future16:30
flaper87I just knew he would pick the wrong language module to replace the faulty one16:35
flaper87probelm is, we don't know what he is saying16:35
flaper87brb, dinner16:37
alcabrerabe back in a bit. o/16:38
*** alcabrera is now known as alcabrera|afk16:38
*** Obulpathi has quit IRC16:39
*** prashanthr_ has quit IRC16:41
*** jchai_afk is now known as jchai16:41
kgriffsАх немає. Всього лише за секунду.16:43
* kgriffs Ставить в правильний модуль.16:43
kgriffsTesting 1...2...3... Testing.16:44
* kgriffs is back in action16:44
*** rossk has joined #openstack-marconi16:51
*** jchai is now known as jchai_afk17:13
*** rossk_ has joined #openstack-marconi17:23
*** rossk has quit IRC17:24
*** alcabrera|afk is now known as alcabrera17:41
* alcabrera observes that kgriffs has JIT correction and self-healing capabilities!17:41
alcabreraamazing. :D17:42
*** Obulpathi has joined #openstack-marconi17:42
*** ayoung has quit IRC17:46
*** shakayumi has quit IRC17:52
kgriffsMe and Data, we go waaaay back.17:52
alcabreraData is pretty amazing, kgriffs17:57
alcabreraah, those Trek memories~17:57
alcabreraI haven't experienced all of it, but just enough to know a little about a lot. :)17:57
kgriffsIt's good stuff18:00
*** jchai_afk is now known as jchai18:00
alcabreraI'll be back in a bit.18:02
*** alcabrera is now known as alcabrera|afk18:02
kgriffspreview of the new Marconi home page:
*** Obulpathi has quit IRC18:37
*** Obulpathi has joined #openstack-marconi18:38
*** vkmc has quit IRC18:42
kgriffsflaper87: when you get back, we need to publish the latest python-client to pypi18:51
*** alcabrera|afk is now known as alcabrera19:02
*** whenry has quit IRC19:15
*** sriram has quit IRC19:15
alcabrerahave a good weekend, all! :)20:18
*** alcabrera is now known as alcabrera|afk20:18
*** vkmc has joined #openstack-marconi20:20
*** whenry has joined #openstack-marconi20:40
*** Obulpathi has quit IRC21:00
*** reed has quit IRC21:00
*** ametts has quit IRC21:10
*** whenry has quit IRC21:11
flaper87kgriffs: back21:11
flaper87kgriffs: yeah, I don't have the powers to do that :(21:11
flaper87kgriffs: it's either you or ttx21:12
flaper87kgriffs: I'm trying to remember what the process is21:12
flaper87I think you just need to push a release commit21:13
flaper87kgriffs: I think the process is. Tag the client, push the tag to gerrit and gerrit does the magic21:17
flaper87kgriffs: I can try to do it, lets see if it works21:18
*** jchai has quit IRC21:20
kgriffsok. that sounds familiar.21:29
flaper87kgriffs: I tried *but* I still can't access gerrit21:30
flaper87I've contacted my ISP21:30
flaper87we'll see what happens21:31
*** flaper87 is now known as flaper87|afk21:34
openstackgerritKurt Griffiths proposed a change to openstack/python-marconiclient: chore: Classify as Beta (was pre-alpha)
kgriffsflaper87|afk: ok, I can tag it on monday.21:44
kgriffsflaper87|afk: please review my patch. ^^^21:44
*** amitgandhi has quit IRC21:52
*** amitgandhi has joined #openstack-marconi21:52
*** amitgandhi has quit IRC21:57
-openstackstatus- NOTICE: Zuul is being restarted with some dependency upgrades and configuration changes; ETA 221521:59
*** kgriffs is now known as kgriffs|afk22:28
*** vkmc has quit IRC23:01
*** vkmc has joined #openstack-marconi23:05
*** vkmc has quit IRC23:05
*** vkmc has joined #openstack-marconi23:05
*** rupsky has joined #openstack-marconi23:12
*** rupsky has left #openstack-marconi23:12
-openstackstatus- NOTICE: is going down for a short database upgrade23:26
*** amitgandhi has joined #openstack-marconi23:47

Generated by 2.14.0 by Marius Gedminas - find it at!