Tuesday, 2014-10-07

*** alcabrera is now known as alcabrera|afk00:37
*** kgriffs|afk is now known as kgriffs00:50
*** kgriffs is now known as kgriffs|afk00:59
*** kgriffs|afk is now known as kgriffs01:43
*** reed has quit IRC01:48
*** X019 has quit IRC02:06
*** kgriffs is now known as kgriffs|afk02:07
*** kgriffs|afk is now known as kgriffs02:15
*** kgriffs is now known as kgriffs|afk03:06
*** alcabrera|afk is now known as alcabrera03:14
*** kgriffs|afk is now known as kgriffs03:33
exploreshaifalifrom where I can learn about storage plane of zaqar?03:52
*** kgriffs is now known as kgriffs|afk04:34
*** kgriffs|afk is now known as kgriffs04:38
*** fifieldt_ has quit IRC04:42
*** sgotliv has joined #openstack-zaqar04:43
exploreshaifalihi sgotliv04:45
sgotlivexploreshaifali, hi04:45
exploreshaifalisgotliv, I want to learn more about storage layer of zaqar, please can u help04:46
sgotlivexploreshaifali, it depends on what you need, I am not a regular zaqar follower04:46
sgotlivexploreshaifali, I can try04:47
exploreshaifalisgotliv,please :)04:48
exploreshaifalisgotliv, u are an zaqar developer?04:48
sgotlivexploreshaifali, no I am not04:48
exploreshaifalioh, ok np :)04:49
sgotlivexploreshaifali, but shoot ask your questions ;-)04:49
sgotlivexploreshaifali, I can bring someone who knows answers in the worst case04:50
exploreshaifalisgotliv, question is same as above, how to start learning for storage layer of zaqar04:50
exploreshaifaliI am only aware that it has two main planes, one is control and another is data plane04:51
sgotlivcorrect04:54
sgotlivSo now go and start reading code04:54
sgotlivControlDriverBase04:54
sgotlivand DataDriverBase04:55
exploreshaifalisgotliv, okay so I should understand everything via code only04:55
exploreshaifaliabove two are classes?04:55
exploreshaifaliright04:55
sgotlivOpen them please04:55
sgotlivI do the same :-)04:56
exploreshaifaliI am refering https://github.com/openstack/zaqar/tree/master/zaqar/queues/storage04:56
sgotlivit takes me time to rebase a code04:58
sgotlivI am reading it and I am not sure I understand why they need two driver types :-)05:03
exploreshaifalisgotliv, which file u are reading?05:05
sgotlivstorage/base.py05:05
sgotlivthe explanation for the base drivers05:06
exploreshaifalifrom https://github.com/openstack/zaqar where I need to move :P05:07
sgotlivzaqar/queues/storage05:08
sgotlivthen open base.py05:09
exploreshaifalisgotliv, yes :)05:09
sgotlivopend this https://review.openstack.org/#/c/50721/05:10
exploreshaifaliby "why they need two driver types" u mean for Control and Data drivers?05:10
sgotlivthis patch introduced those driver types and explain their purposes and differences05:10
sgotlivyes05:11
sgotlivI don't see an immediate usage for control drivers!05:12
exploreshaifalisgotliv, as per I know, these two planes have diff purposes05:12
exploreshaifalicontrol plane is for all operate parts like all administrative tasks and all05:12
exploreshaifaliand data plane contain real data that need to move05:13
sgotlivI understand that, this is obvious :-)05:13
sgotlivhowever there is no any example for such administrative tasks05:14
sgotlivControl driver seems to me like a theory05:14
exploreshaifali:P05:14
exploreshaifalimy task is to separate these two planes, currently they share many things05:16
exploreshaifalithey both are kind a mix together05:16
exploreshaifalilike just one dbconfig for both05:17
sgotlivwho gave you this task?05:18
exploreshaifaliI am applying for OPW05:19
exploreshaifalithere a project for this task is mentioned :P05:19
sgotlivoh congratulations!05:19
exploreshaifalinot yet, I am not selected right now05:19
sgotlivLet's talk with flaper87|afk later this morning to clarify things. He is a great guy and zaqar expert05:20
exploreshaifaliyeah, he is the mentor05:20
sgotlivreally?05:20
exploreshaifaliyes, he is mentor for this project05:21
exploreshaifalibtw, sgotliv on which components of openstack u work?05:21
sgotlivI am too lazy to work :-)05:21
*** alcabrera is now known as alcabrera|afk05:49
*** exploreshaifali is now known as expshaifali|afk06:26
*** prashanthr_ has joined #openstack-zaqar06:27
*** expshaifali|afk is now known as exploreshaifali06:37
*** flaper87|afk is now known as flaper8706:49
flaper87o/06:50
flaper87exploreshaifali: hey, what's up?06:50
flaper87:)06:50
exploreshaifalihi flaper87 :)06:50
flaper87exploreshaifali: how are you doing?06:51
exploreshaifaliI am just trying to understand storage layer06:51
exploreshaifaliand its two planes06:51
flaper87exploreshaifali: any specific question so far?06:51
*** prashanthr_ has quit IRC06:51
exploreshaifaliI should start with understanding the code06:51
exploreshaifalias mentioned by sgotliv06:51
exploreshaifaliso reading base.py file06:52
exploreshaifaliflapper87, am I on correct track?06:52
flaper87exploreshaifali: just 1 p :P06:53
flaper87flaper87 not flapper8706:53
flaper87otherwise I won't see your mentions :D06:53
flaper87exploreshaifali: yup, you're on the right track06:53
exploreshaifali:D yes06:53
flaper87reading the code is the way to go, you need to see how they are separated and how they interact with each other06:53
flaper87the other day I pointed you to the mongodb driver06:53
flaper87that's another good reference where you can see things working for real06:54
exploreshaifalithis file for mongodb driver https://github.com/openstack/zaqar/blob/master/zaqar/queues/storage/mongodb/driver.py06:54
flaper87correct06:55
exploreshaifaliok :)06:55
exploreshaifaliflaper87, just one question in mind, marconi is old name of zaqar?06:56
flaper87exploreshaifali: yes06:58
exploreshaifaliand does it work with Python3.306:59
exploreshaifalias here https://review.openstack.org/#/c/50721/ Jenkins failed its tests07:00
*** exploreshaifali is now known as expshaifali|afk07:01
sgotlivflaper87, I have a question07:01
flaper87sgotliv: let me think If I want to hear it :P07:02
flaper87sgotliv: kidding, go ahead07:02
flaper87expshaifali|afk: it does work with py3.307:02
flaper87expshaifali|afk: I mean, it's suppose to work with py3.307:02
flaper87ah no, it used to work with pypy until the gate broke. The client library does work with py3.3 though07:03
flaper87Zaqar works with py3.3 but some dependencies don't, AFAICS07:03
sgotlivflaper87, I disagree with the term driver07:03
sgotlivData or Control07:03
flaper87sgotliv: that's not a question07:03
flaper87:P07:03
sgotlivwhy you named them that way?07:03
sgotlivis that a question :-)07:04
sgotlivdriver is a computer program that operates or controls a particular type of device07:04
sgotlivin zaqar case they both operate the same "device" type - Redis, Mongo etc...07:05
flaper87sgotliv: because naming things is harder than writing code :D Right, just replace device with storage in this case07:05
flaper87I mean, each driver controls a specific storage07:05
sgotlivwhat do you mean, replace?07:05
sgotlivright07:06
flaper87that's why it's called driver07:06
sgotlivI probably don't understand you07:06
flaper87I mean07:06
sgotlivgo to redis/driver.py07:06
sgotlivwhy I see 2 different drivers inside?07:07
sgotlivI expect to see here only one driver - Redis Driver07:07
flaper87Because there are 2 diferent Drivers. 1 is for the "data" plane and the other one is for the "control" plane07:07
flaper87no because you could use 2 different storage for each driver07:07
flaper87(which is the task expshaifali|afk will work on)07:07
sgotlivI think its wrong approach :-)07:07
flaper87As of now, that's not clear but after they're separate you'll be able to do so07:08
flaper87why?07:08
sgotlivdriver is the piece of software wich is communicates with the Redis in this case07:08
sgotlivcontrol or data is the way you use it, right?07:09
flaper87No, it's just the typo of driver it is07:10
sgotlivI agree this is a typo :-)07:10
flaper87type of*07:10
flaper87:D07:10
sgotlivof we agree to disagree07:11
sgotliv:-) now it my typo07:11
sgotlivwe agree to disagree07:11
*** fifieldt has joined #openstack-zaqar07:22
*** sgotliv has quit IRC07:51
*** openstackgerrit has quit IRC08:11
*** expshaifali|afk is now known as exploreshaifali09:15
*** njirap has joined #openstack-zaqar09:54
*** sgotliv has joined #openstack-zaqar09:55
*** malini1 has joined #openstack-zaqar10:48
*** malini1 has quit IRC10:48
*** malini1 has joined #openstack-zaqar10:49
*** sgotliv has quit IRC10:54
*** sgotliv has joined #openstack-zaqar11:08
*** sgotliv has quit IRC11:19
*** sgotliv has joined #openstack-zaqar11:21
*** prashanthr_ has joined #openstack-zaqar11:50
*** X019 has joined #openstack-zaqar12:18
*** prashanthr_1 has joined #openstack-zaqar12:18
*** njirap has quit IRC12:20
prashanthr_1flaper87: Hello :)12:23
flaper87prashanthr_1: ahhh there you are12:23
flaper87finally12:23
flaper87prashanthr_1: how are you doing?12:23
flaper87prashanthr_1: any news on the sentinel thing?12:23
flaper87we *need* to get that done *before* Thursday12:24
flaper87:P12:24
prashanthr_1Ha ha yeah :) I am working on the bug we discussed about yesterday12:24
prashanthr_1kgriffs has commented that he would like to go ahead with the patch and update it12:25
prashanthr_1so that sentinel support can be added12:25
prashanthr_1flaper87: can you please go through the comments chain for the sentinel patch once ?12:27
* flaper87 gets the link12:27
flaper87prashanthr_1: what's up with that?12:28
prashanthr_1flaper87: I just want to know if any further work is required on it ?12:28
prashanthr_1link->https://review.openstack.org/12147412:29
flaper87I went through it yesterday and I think Kurt pretty much addressed my concerns too12:30
flaper87so, no other comments from me12:30
prashanthr_1flaper87: Thanks. I will rebase the changes and submit it so that it can be landed.12:31
flaper87prashanthr_1: awesome, thanks a lot12:31
flaper87sorry for the preasure12:31
flaper87but we need to get rc2 out asap12:31
*** njirap has joined #openstack-zaqar12:32
prashanthr_1flaper87: No probs at all. It's totally my mistake here. After shifting to bangalore I have just not been able to focus. Really sorry about it.12:32
flaper87prashanthr_1: no need to be sorry, you've done an amazing work and when life happens, we just need to surf it12:33
*** jchai has joined #openstack-zaqar12:33
prashanthr_1flaper87: Thanks :)12:34
*** openstackgerrit has joined #openstack-zaqar12:48
*** sgotliv has quit IRC12:56
*** jchai has quit IRC12:57
*** sgotliv has joined #openstack-zaqar13:11
*** sriram has joined #openstack-zaqar13:12
*** mpanetta has joined #openstack-zaqar13:14
*** bradjones has quit IRC13:15
*** mpanetta_ has joined #openstack-zaqar13:16
*** mpanetta has quit IRC13:16
*** mpanetta_ is now known as mpanetta13:16
*** jchai has joined #openstack-zaqar13:25
*** kgriffs is now known as kgriffs|afk13:40
*** prashanthr_1 has quit IRC13:45
*** prashanthr_ has quit IRC13:48
*** amitgandhinz has joined #openstack-zaqar14:07
*** bradjones has joined #openstack-zaqar14:20
*** kgriffs|afk is now known as kgriffs14:26
*** cpallares has joined #openstack-zaqar14:29
kgriffsflaper87: ping14:35
flaper87kgriffs: pong14:52
*** reed has joined #openstack-zaqar14:56
kgriffsflaper87: hey15:03
kgriffsa couple things15:03
kgriffsfirst, I discovered that redis-py does something interesting and wanted to share15:03
kgriffsafter a connection error, it first does one retry on it's own before re-raising the error15:03
kgriffssecondly, it will rest the connection when that happens, and there is some logic in the sentinel code to go find the new master in the case of a failover15:04
kgriffsBUT15:04
kgriffsif you tell sentinel to wait for a few seconds, say, before deciding a master is indeed "down", redis-py will raise MasterNotFound which is a subclass of ConnectionError15:05
kgriffsso the decorator in the driver for retrying those is still necessary15:05
kgriffseventually it will succeed after a few retries once the sentinel elects the new master15:05
kgriffsso, I think we are good there15:05
kgriffsthe one thing I do need to add though before submitting the updated patch15:06
kgriffsis to specify a list of sentinels in the connection URI, vs. just one15:06
kgriffsyou are supposed to run >1 sentinel for HA15:06
kgriffsdoes all that make sense?15:06
* kgriffs has been reading lots of source code lately15:06
*** alcabrera|afk is now known as alcabrera15:21
flaper87kgriffs: sorry, got distracted15:26
flaper87kgriffs: it does make sense15:26
kgriffsok15:26
flaper87glad to know redis-py does a bit of this already15:26
kgriffssecond question15:26
* flaper87 reads15:26
* flaper87 won't get distracted15:27
flaper87oh look, a fly15:27
kgriffswhich do you think looks better15:27
kgriffsredis://s1.example.com,s2.example.com,s3.example.com?master=dumbledore15:27
kgriffsredis://s1.example.com,s2.example.com,s3.example.com?sentinel=dumbledore15:27
kgriffscontext:15:28
kgriffshttp://redis.io/topics/sentinel15:28
kgriffs(look for the "Configuring Sentinel" section)15:28
kgriffsand15:28
kgriffshttps://github.com/andymccurdy/redis-py/blob/master/redis/sentinel.py#L14515:28
flaper87mmh, what does redis-cli outputs as help for the `sentinel` command ?15:29
flaper87what about sentinel_master=dumbledore ?15:29
flaper87longer but clearer15:29
flaper87it makes it clear we are talking about the master node the sentinel needs to monitor15:30
*** malini has quit IRC15:30
flaper87or `sm=dumbledore`15:30
flaper87and RTFM15:30
flaper87:P15:30
*** malini|afk has joined #openstack-zaqar15:30
flaper87we can document it in the redis URI config option15:30
*** malini|afk is now known as malini_znc15:30
flaper87bikeshedding in 3... 2... 1... GOOOOOOOOOOOOOOOOOOOOOO15:30
kgriffslol15:32
kgriffsmy bike shed burned down a few years ago along with my parent's house so I don't do that anymore.15:32
flaper87oh crap... :S15:33
kgriffsjokes aside, I think sentinel_master is OK15:33
flaper87cool15:33
kgriffsflaper87: yeah, well, they ended up moving into town out of the snow so it was probably a blessing in disguise. :)15:33
kgriffsand actually the bike shed didn't burn down - it was up on the hill behind the house.15:34
flaper87kgriffs: youuuuuuuuuu liar!!!! :P15:34
* kgriffs repents15:34
* kgriffs gives flaper87 gummy bears as penance15:36
flaper87oh look, gummy bears15:36
* flaper87 eats them15:36
*** echevemaster has joined #openstack-zaqar15:42
njiraphi everyone i need help with git review -s...i believe its ssh issues..the error is http://paste.openstack.org/show/119412/15:43
njirapand then the video with the useful tips on gerrit is private i cant view it please https://www.youtube.com/watch?v=mT2yC6ll5Qk&feature=youtu.be15:45
*** prashanthr_ has joined #openstack-zaqar15:51
flaper87kgriffs: btw, I replied to your comments on the marker index patch15:52
flaper87njirap: mmh, that link doesn't seem to work15:53
flaper87I think paste.openstack.org is down15:53
kgriffsflaper87: kk15:53
flaper87mind finding another paste service?15:53
flaper87like pastebin15:53
flaper87:D15:53
kgriffsnjirap: you can use github gists15:53
flaper87or that ^15:53
njiraphttp://paste.openstack.org/show/119412/15:53
kgriffsnjirap: I'm getting a server error on that as well15:54
kgriffsnjirap: try this https://gist.github.com/15:54
njirapflaper87....I've hacked the issue by clearing my known_hosts filw15:56
flaper87njirap: did you add your key here? https://review.openstack.org/#/settings/ssh-keys15:56
njirapfile15:56
flaper87ok15:56
njirapbut the youtube video is un-view-able15:57
njirapkgriffs..the keystone error for yesterday it was the /opt/stack one that was to be deleted not the dist_packages one....just for future reference :)16:01
openstackgerritPrashanth Raghu proposed a change to openstack/zaqar: Add support for redis-sentinel  https://review.openstack.org/12147416:05
kgriffsnjirap: ah, gtk16:07
openstackgerritKurt Griffiths proposed a change to openstack/zaqar: Fix the typo in the description of the write concern error  https://review.openstack.org/12661016:13
*** njirap has quit IRC16:18
*** exploreshaifali has quit IRC16:22
*** echevemaster has quit IRC16:31
*** sgotliv has quit IRC16:36
kgriffsflaper87: just thought of something16:51
flaper87oh god16:51
kgriffs"sentinel_master" could be interpreted to mean" the sentinel's master, not the redis master that is being managed by the sentinel16:51
flaper87:P16:51
kgriffsyeah, watch out when I think of things. ;)16:51
kgriffsbut I think that shouldn't be a problem16:52
flaper87mmh, then I'd lead to just master16:52
*** sgotliv has joined #openstack-zaqar16:52
flaper87like redis-py does16:52
kgriffsi mean, it may be a problem for people not super familiar with redis16:52
kgriffsbut whatever16:52
kgriffsmaster may be a little cleaner16:52
flaper87ok, as long as we document it thoroughly in the URI help text, I think we're fine16:54
kgriffskk16:54
*** alcabrera is now known as alcabrera|afk16:55
*** exploreshaifali has joined #openstack-zaqar17:04
*** sgotliv has quit IRC17:21
*** prashanthr_ has quit IRC17:41
*** echevemaster has joined #openstack-zaqar17:47
*** cpallares has quit IRC17:49
*** vkmc has joined #openstack-zaqar17:58
*** alcabrera|afk is now known as alcabrera17:59
* vkmc lurks18:01
*** amitgandhinz has quit IRC18:06
*** sgotliv has joined #openstack-zaqar18:14
*** sphoorti has joined #openstack-zaqar18:32
*** vkmc_ has joined #openstack-zaqar18:33
*** vkmc has quit IRC18:33
*** vkmc_ is now known as vkmc18:33
*** amitgandhinz has joined #openstack-zaqar18:43
*** sphoorti has quit IRC18:44
*** ametts has quit IRC18:56
vkmcok, gotta go19:01
vkmcbabai19:01
vkmco/19:01
*** vkmc has quit IRC19:01
*** jchai is now known as jchai_afk19:05
*** jchai_afk is now known as jchai19:09
*** kgriffs is now known as kgriffs|afk19:11
openstackgerritOpenStack Proposal Bot proposed a change to openstack/python-zaqarclient: Updated from global requirements  https://review.openstack.org/12668519:17
openstackgerritOpenStack Proposal Bot proposed a change to openstack/zaqar: Updated from global requirements  https://review.openstack.org/12668819:18
*** jchai has quit IRC19:34
*** bradjones has quit IRC19:44
*** bradjones has joined #openstack-zaqar19:46
flaper87kgriffs|afk: ping, I saw  Prashanth posted a new patch. Any chance we can review that today?20:08
flaper87kgriffs|afk: I'm confused, I thought you were going to update the patch20:08
openstackgerritA change was merged to openstack/python-zaqarclient: Updated from global requirements  https://review.openstack.org/12668520:11
openstackgerritA change was merged to openstack/zaqar: Fix the typo in the description of the write concern error  https://review.openstack.org/12661020:40
*** bradjones has quit IRC20:52
*** bradjones has joined #openstack-zaqar20:55
exploreshaifalihi flaper8720:59
exploreshaifaliflaper87, I have looked at mongo driver file21:00
exploreshaifaliflaper87 ping :P21:01
*** sriram has quit IRC21:13
*** alcabrera is now known as alcabrera|afk21:27
flaper87exploreshaifali: pong?21:44
flaper87:D21:44
exploreshaifalihi flaper21:45
exploreshaifali*flaper8721:45
exploreshaifaliI have gone through mongo driver file21:46
exploreshaifaliand got a sense of what need to do in the project21:46
flaper87exploreshaifali: awesome21:46
exploreshaifaliMy task is something like putting the common code(like importing i18n._ and def connection() method) in a separate file and putting specific code(like import storage is only for storage plane and import controllers is only for control plane in different files and importing common code in both files21:46
*** mpanetta has quit IRC21:47
flaper87kinda, that will likely happen but you'll need to do more than just that21:47
exploreshaifalilikeI have to split each file into 3 parts21:47
flaper87For example, both the Control and the Data plane are using the same config option for the store config21:48
exploreshaifaliyes, I will be happy by doing all that :P21:48
flaper87we need to change that and add a config section for each plane21:48
flaper87among other things like having more detailed tests21:48
flaper87and make sure we don't have cycles in the dependencies between those two21:49
exploreshaifaliok, probably u are saying to have diff config section for both21:49
exploreshaifaliwhile I thought, that since it is common in both, it will be written once and get referred or imported by both21:49
exploreshaifalias configs are same for both21:50
flaper87nope, that's exactly one of the things we need to change21:51
flaper87exploreshaifali: are you going to be around tomorrow my morning?21:51
flaper87I'd like to explain this a bit better to you21:51
exploreshaifaliya sure21:51
exploreshaifaliI was about to write a e-mail to u21:52
flaper87ok, then lets make sure we meet here tomorrow and I'll walk you through why we need to split these to planes21:52
flaper87emails are good but lets try to discuss things here as much as possible so that others can chime in as well21:52
exploreshaifaliokay, what will be the timings in UTC?21:52
exploreshaifaliahh.... increasing my competition :P21:53
exploreshaifalino worries, fine :)21:53
*** malini1 has quit IRC21:53
flaper87ah crap, I forgot I've something to do tomorrow morning. what about 12 UTC ?21:53
flaper87is that time ok for you?21:54
flaper87nah, don't feel like this is a competition, there's place for everyone and enough things to do21:54
exploreshaifaliOok about competition21:55
exploreshaifaliand also about timings21:55
flaper87ok21:55
flaper87exploreshaifali: please, do keep digging. The more questions you ask, the clearer it will be for you :)21:55
exploreshaifaliflaper87, sure :)21:56
exploreshaifaliThanks!!21:56
flaper87exploreshaifali: thank you ;)21:58
exploreshaifaliflaper87, thanking me for what?21:59
flaper87exploreshaifali: for your time, interest and perseverance. I appreciate you willing to work on this and I thank you for that. I look forward to see you around long enough to do way more.22:00
exploreshaifaliyes! I am enjoying it alot, talking and asking for help and getting work done...22:01
exploreshaifalisuperb!!22:01
*** amitgandhinz has quit IRC22:26
*** flaper87 is now known as flaper87|afk22:39
*** X019 has quit IRC22:44
openstackgerritKurt Griffiths proposed a change to openstack/zaqar: Add support for redis-sentinel  https://review.openstack.org/12147423:31

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!