Friday, 2013-08-30

*** vkmc has joined #openstack-marconi00:41
*** amitgandhi has quit IRC02:40
*** vkmc has quit IRC03:02
*** fifieldt has joined #openstack-marconi05:34
*** fifieldt has quit IRC06:06
*** flaper87|afk is now known as flaper8707:04
*** fifieldt has joined #openstack-marconi07:25
*** flaper87 is now known as flaper87|afk07:33
*** flaper87|afk is now known as flaper8707:38
*** flaper87 is now known as flaper87|afk09:24
*** flaper87|afk is now known as flaper8709:25
*** _alexr_ has joined #openstack-marconi09:36
flaper87!help11:19
openstackflaper87: (help [<plugin>] [<command>]) -- This command gives a useful description of what <command> does. <plugin> is only necessary if the command is in more than one plugin.11:19
*** JRow has joined #openstack-marconi11:23
*** vkmc has joined #openstack-marconi11:58
*** cppcabrera has joined #openstack-marconi12:22
cppcabreraMorning~12:23
*** amitgandhi has joined #openstack-marconi12:37
*** amitgandhi has quit IRC12:38
*** amitgandhi has joined #openstack-marconi12:38
*** zyuan has quit IRC12:39
*** flaper87 is now known as flaper87|afk12:54
*** flaper87|afk is now known as flaper8712:54
flaper87cppcabrera: Morning12:55
flaper87:D12:55
cppcabreraHey hey, flaper87. :)12:56
cppcabreraAnd it begins: https://github.com/cabrera/marconi-redis12:57
flaper87cppcabrera: awesome, looking forward to see it working12:58
*** malini_afk is now known as malini13:20
*** ayoung has joined #openstack-marconi13:34
* flaper87 is working on functional tests config refactor13:46
cppcabrerawoot :)13:47
flaper87cppcabrera: :D I've some patches ready but I'll submit them all once I finish this one13:47
cppcabreraAs they come in, and as I notice them, I'll be glad to review. :)13:51
* cppcabrera wrestles for the first time with the basics of openstack-common.conf13:51
cppcabreraIt's actually not too bad. :P13:51
flaper87hehehe, can't wait13:54
*** fifieldt has quit IRC14:02
*** kgriffs_afk is now known as kgriffs14:12
*** meganw has joined #openstack-marconi14:20
flaper87w00000000t, test_claims ported to oslo.config14:43
flaper872 more suites to go14:43
flaper87then tox, then embedded marconi-sever execution14:43
amettsflaper87:  Rock on!14:44
flaper87mmh, I guess first embedded execution14:44
flaper87ametts: :)14:44
* flaper87 wonders what kgriffs and malini hacking-day projects are14:45
kgriffsheh14:45
kgriffsnot sure if I am participating since I'm not in Atlanta, but if I do it will likely be Falcon 0.1.714:46
cppcabreraawesome, flaper87!14:46
amettsmalini is still stirring up marconi trouble.  I'm hearing the phrases "Random 404s and 400s" from her in the hallway....14:46
amitgandhishe was saying that yesterday too - i wonder if its from kgriffs patch14:47
flaper87LOOOOOL14:47
kgriffshmmm14:47
amitgandhieither that, or the patch has exposed a new issue prev unknown14:47
flaper87if there are no chairs flying, it means there's still hope14:48
kgriffsmalini: do you get the same errors w/o the patch?14:48
malinikgriffs: I 'think' it might be a pre-patch thing..14:49
*** oz_akan_ has joined #openstack-marconi14:49
maliniwhen I run the test_claims.py functional test, I get random 404s on post_claim, patch_claim etc.14:49
oz_akan_I forgot to start my irc client14:49
maliniSo I tried running the test in debug with pdb & the 404s no longer show up14:50
amitgandhiis marconi too fast now?14:50
maliniin any case ,it shud not be 404 (as long as the queue exists & it does)14:51
amitgandhiare we running into propogation issues?14:51
amitgandhireading from secondaries14:51
kgriffsthat was what I was wondering as well14:51
oz_akan_404 is different14:51
flaper87w0000t test_messages ported14:51
amitgandhipdb mode runs slower giving it enough time to propogate14:51
maliniflaper87 gets a cherry poptart14:52
flaper87w000t14:52
amitgandhii need to go make another smoothie - brb14:52
maliniamitgandhi: just because its healthy doesnt men, you can keep gorging ;)14:52
malinimean*14:52
oz_akan_hehe14:53
* ametts thinks amitgandhi is mixing the smoothie ingredients with the margarita ingredients. Stay out of that cabinet, Amit!14:53
oz_akan_what a smell14:54
malinioz_akan_ : chk if you still have your salt ;)14:54
oz_akan_:D14:55
oz_akan_I had stacked it, so I must have some more (saltstack)14:55
* cppcabrera awaits flaper87's refactoring patch14:56
malinikgriffs, cppcabrera: can one of you look at the 404 issue? Ihavent opened a bug yet, since we dont know if its the code14:57
kgriffsmalini: can you add a delay between creating the queue and testing whether it exists?14:58
amitgandhiametts: i already raided the tequila cabinet (but only to get the agave nectar)14:58
kgriffsamitgandhi: suuuuuure. ;)14:59
amitgandhirum would go well with this though - we need rum ametts - put it in the next stationary order please =)14:59
amettsI'm not sure they have that in the Staples catalog...14:59
malinikgriffs: http://paste.openstack.org/show/45472/15:00
malinisince the post messages get a 201, I dont see any reason the queue wud vanish before the claims15:02
cppcabrerathat's pretty weird, malini. AFAICS, it's like the claim|queue ceases to exist after posting enough messages to it.15:02
kgriffsit's because we don't check for queue existence anymore15:02
kgriffs(before posting a message)15:02
kgriffsif you try forcing a delay before attempting to claim some messages, ti should avoid the 404 if my hypothesis is correct15:04
malinikgriffs: hmmm, in tht case if the queue doesnt exist, dont we return a 404 ?15:05
maliniI am trying out the sleep now15:06
* ametts wishes malini wouldn't sleep on the job15:06
malini:D15:07
cppcabreratime.nap(10)  # a more human-friendly variant on time.sleep15:11
openstackgerritKurt Griffiths proposed a change to stackforge/marconi: chore: Track the up-and-coming oslo.cache module  https://review.openstack.org/4413115:11
malinia 'nap' between posting & claiming messages removed the 40415:13
amitgandhia ' nap15:13
amitgandhi ugh15:13
malinicppcabrera wanted a more human friendly term ;)15:13
amitgandhia 'nap' indicates that the nap can be interrupted.  'sleep' indicates DND until time is up15:13
malinibut I get a 404 between post & patch claims (with no sleep)15:13
malinitrying to add sleep between those two15:14
amitgandhido you get the 404's with just one worker also?15:14
* amitgandhi wonders if hitting some sort of race condition15:15
maliniI have just one worker15:16
malinikgriffs: The sleep seems to make things better. Please run test_claims.py against test env for a repro15:20
maliniYou'll see a few random 404s15:20
maliniI am back to figuring out whats wrong with the tsung tests15:21
maliniPing me if you need anything15:21
kgriffsmalini: if the sleep makes things better, then we have to decide whether we just document this behavior or try to work around it. Eventual consistency is kind of something you have to live with in order to scale out15:22
kgriffs(although there are some recent attempts to be fully consistent and yet still crazy fast with writes)15:23
maliniI think this is the same issue, kevin ran into in his test app15:23
malinikgriffs:But I still dont understand why its a 404 and not 204, for post claims15:25
maliniwhy does the server think there is no queue :-S15:25
*** meganw has quit IRC15:29
*** meganw has joined #openstack-marconi15:30
kgriffsmalni: implementation detail. post claim checks whether the queue exists before doing anything else, while post messages just assumes it is there (for performance)15:31
kgriffsmakes me wonder why we check when claiming messages?15:31
flaper87malini: some tests in test_queue are not passing :(15:31
maliniI know two of them15:32
kgriffsflaper87: thoughts on my last comment?15:32
malini1) tests invalid auth - wont work locally , but I am open to getting rid of tht if we need to15:32
flaper87kgriffs: mmh, I remember discussing that15:32
flaper87let me try to remember15:33
flaper87kgriffs: ah, nevermind, we discussed that for deletion15:34
malini2) non ASCII metadata - I think there is something wrong in tht comparison..I missed to fix it :-$15:34
*** meganw has quit IRC15:34
flaper87not creation15:34
flaper87malini: kk, I'll get rid of the auth tests, meaning disable if auth_on is false15:34
malini3) https://bugs.launchpad.net/marconi/+bug/1208873 is still failing15:34
maliniflaper87: tht is 3 failing tests, I remember15:35
flaper87malini: kk, I thought it was related to the migration15:35
flaper87thanks for the heads up15:35
flaper87kgriffs: FWIW, I'd prefer Marconi's API to be as lazy as possible15:35
flaper87meaning, less checks15:36
kgriffsso, just remove that check?15:36
flaper87kgriffs: yup, but, if we remove it we'll have to change the returned status code15:38
flaper87right?15:38
flaper87mmh, wait a sec15:38
kgriffsit would just return no messages claimed15:38
kgriffs20415:39
kgriffsalso, I was mistaken about posting messages15:39
kgriffsit also checks for existence15:39
flaper87kgriffs: but we do verify it for message posting https://github.com/stackforge/marconi/blob/master/marconi/storage/mongodb/messages.py#L48215:40
flaper87ah that15:40
flaper87:P15:40
flaper87sorry, I didn't read15:40
kgriffsheh, we need to stop doing this. :p15:40
flaper87yeah :D15:40
flaper87so, I t makes sense for mmessages' post15:41
flaper87to check, that is15:41
kgriffsI was just thinking about that15:42
kgriffsif you don't check, and a queue really doesn't exist, then you have orphaned messages that may never get cleaned up15:42
kgriffsI guess we could add that to the garbage collector15:42
flaper87exactly15:42
kgriffsbut then it gets weird with the stats call15:43
kgriffsso...15:43
kgriffswe could remove it for claims I guess15:43
flaper87I agree15:43
kgriffsand cache the call in the message post15:44
kgriffshttps://review.openstack.org/#/c/44131/15:44
kgriffs(hint, hint)15:44
kgriffsp.s. - https://review.openstack.org/#/c/44363/15:44
* kgriffs feels like a dope for forgetting to update the unit test the first timie15:45
* kgriffs is listening to http://www.radioparadise.com15:45
kgriffsmalini: is there a bug for that 404 thing?15:49
kgriffsif not, I'll just submit a patch w/o a reference15:49
* kgriffs loves quick turnaround on feedback15:49
* kgriffs q.e.d. loves IRC15:49
*** meganw has joined #openstack-marconi15:49
kgriffssweet!15:50
kgriffsit's the meganw-a-nator15:50
malinikgriffs: no ..let me open one15:50
malinihttps://bugs.launchpad.net/marconi/+bug/121899015:52
kgriffssweet. I'm on it15:56
openstackgerritA change was merged to stackforge/marconi: chore: Track the up-and-coming oslo.cache module  https://review.openstack.org/4413115:57
cppcabrerawoot15:58
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Pull actual tests out of marconi/tests  https://review.openstack.org/4447515:59
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Remove util packages  https://review.openstack.org/4447615:59
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Use oslo.config for functional tests  https://review.openstack.org/4447715:59
kgriffshooray for oslo.config!15:59
cppcabreraohh, flaper87 has test refactorings available for review.16:01
flaper87cppcabrera: :)16:01
* kgriffs wishes he could just click on an output for tox and jump to the line of code16:02
*** meganw has quit IRC16:03
* flaper87 wishes what kgriffs whishes16:03
kgriffsanybody handy with bash scripting who could write that  for me?16:03
kgriffs(hint hint)16:03
kgriffsflaper87: do you thing an image pastebin would best be integrated with paste directly, or standalone?16:04
* kgriffs is going to make that for hackday next friday)16:04
kgriffshttp://paste.openstack.org/16:04
kgriffsI think I need to find a way to run tox from sublime. If I'm likely it already is smart enough to understand tracebacks16:05
flaper87integrated means we'll have a single url for both text and images16:05
flaper87which sounds good16:05
flaper87:)16:06
flaper87I think you can extend logedit16:06
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Remove util packages  https://review.openstack.org/4447616:07
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Use oslo.config for functional tests  https://review.openstack.org/4447716:07
kgriffscool16:08
kgriffshttp://paste.openstack.org/about/16:08
kgriffshttp://www.pocoo.org/projects/lodgeit/16:09
flaper87cppcabrera: ha, you catch that import :P16:12
flaper87so, that's a temporary hack16:12
flaper87thing is that we need to import wsgi to register wsgi configs (and validation configs)16:13
cppcabreraahhh :)16:13
flaper87I should've added a comment16:13
* cppcabrera reviews 80% thoroughly16:13
flaper87if we let those patches land, I'll start working on the integration with tox right away16:14
flaper87kgriffs: can you review those patches ?16:15
kgriffshttps://groups.google.com/forum/#!topic/pocoo-libs/0WvXXwAjnpc16:17
kgriffsflaper87: will do, but it may take me an hour or two to get to that16:17
flaper87kgriffs: kk, thanks!!16:17
flaper87kgriffs: +1 for the email16:17
flaper87malini: patches are up there16:18
flaper87:D16:18
flaper87malini: you must review them16:18
maliniflaper87: wooot!! will get to it now16:18
* flaper87 moved his hand like a Jedi while writing his last message16:18
flaper87brb guys16:19
cppcabreralol16:20
* kgriffs wants pianobar for radio paradise16:22
* kgriffs would call it paradisefound16:23
kgriffsoooh16:27
kgriffshttp://wesbos.com/sublime-text-build-scripts/16:27
kgriffsmy wish has been granted. This will be easy16:27
cppcabrerawoot16:28
kgriffshttp://docs.sublimetext.info/en/latest/file_processing/build_systems.html16:30
* kgriffs is sooo off-topic today16:30
* kgriffs needs more sleep and less caffeine16:30
cppcabreraMe, too, kgriffs. I'm running on 2 hours worth of sleep today. :P16:31
kgriffsyay, got a +2 on https://review.openstack.org/#/c/44363/17:20
cppcabrerasweet - you'll have that merged in no time.time()!17:21
malinihttps://bugs.launchpad.net/marconi/+bug/121901917:24
cppcabreraFun Python fact of the day - expanding generators works like expanding lists, e.g.17:26
cppcabrerax = [i for i in range(100)]; print(*x)17:26
maliniflaper87: will oslo.cfg use the local functional-tests.conf if its there auto-magically ?17:27
cppcabrerax =([i for i in range(100)); print(*x)17:27
cppcabrerax = (i for i in range(100)); print(*x)17:27
cppcabreraBoth work17:27
cppcabreratested on py2.7 and py3.317:27
*** key4 has quit IRC17:33
*** key4 has joined #openstack-marconi17:33
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Remove util packages  https://review.openstack.org/4447618:15
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Use oslo.config for functional tests  https://review.openstack.org/4447718:15
flaper87cppcabrera: updated18:15
flaper87:)18:15
flaper87malini: correct18:15
flaper87malini: well, not magically, we're actually telling it where it is18:15
flaper87the user is expected to specify where the configs for tests are18:16
flaper87I'll document that better when the integration with tox is completed18:16
flaper87malini: commented here https://review.openstack.org/#/c/44475/18:18
*** ametts has quit IRC18:18
* flaper87 doesn't understand what weird formatting gerrit uses for comments18:19
flaper87brb.... again!18:19
maliniflaper87: tht sounds good to me..18:28
maliniJust a couple of minor stuff https://review.openstack.org/#/c/44475/18:28
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Pull actual tests out of marconi/tests  https://review.openstack.org/4447518:33
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Remove util packages  https://review.openstack.org/4447618:33
openstackgerritFlavio Percoco proposed a change to stackforge/marconi: Use oslo.config for functional tests  https://review.openstack.org/4447718:33
flaper87malini: addressed them, thanks  a lot18:34
cppcabrera+1 malini - good reviews :)18:36
cppcabreraI'm too sleep-deprived today to give solid reviews. :P18:36
maliniflaper87: review both..Thanks a lot!!!!!18:37
maliniI would give you a real poptart if you were here :)18:38
flaper87malini: you're missing this one https://review.openstack.org/#/c/44476/18:38
flaper87:D18:38
malinidone!18:39
flaper87malini: AWESOME! Thanks18:39
flaper87so, next step is making everything run under tox18:40
maliniYES!!18:40
cppcabrerayaaaay18:40
flaper87but, before that, we need to execute marconi from within our functional tests18:40
cppcabreraflaper87: setUpModule?18:41
flaper87cppcabrera: mmh, I think it has to be either setUpClass or setUp, most likely a mix of both18:42
cppcabrerasetUpClass sounds appropriate to me.18:42
maliniI was looking at the glance tests you pointed to a while back18:42
malinithose were pretty cool :)18:43
flaper87yeah :D18:43
maliniyou is flaper87 :D18:43
flaper87those are pretty cool, I'd like to do something along those lines18:43
flaper87but there are a couple of things I'd like to change to keep it fast and re-use resources when possible18:43
cppcabreraJust one more component left to the redis storage driver - claims. :)18:56
flaper87cppcabrera: w00000000t19:06
*** JRow has quit IRC19:18
oz_akan_kgriffs: has your patch been merged to master/19:37
kgriffsI think so19:38
oz_akan_https://github.com/stackforge/marconi/commits/master19:38
oz_akan_I can't see19:38
kgriffshmm19:38
kgriffsnevermind, need to fix some stuff19:39
kgriffshttps://review.openstack.org/#/c/44340/19:39
oz_akan_ok19:39
*** openstack has joined #openstack-marconi19:43
kgriffsflaper87: responded to you comments on the perf patch19:46
openstackgerritKurt Griffiths proposed a change to stackforge/marconi: fix: Claim can return 404  https://review.openstack.org/4453619:53
openstackgerritKurt Griffiths proposed a change to stackforge/marconi: fix: Claim can return 404  https://review.openstack.org/4453620:33
*** malini is now known as malini_afk20:37
kgriffsno meeting on Monday20:38
flaper87kgriffs: kk, I'll take a look in a bit20:47
flaper87kgriffs: mmh, when can we discuss the endpoints thing?20:47
flaper87we need to do that before the TC meeting20:48
kgriffsflaper87: yes, we need to do that20:48
kgriffstuesday earlier in the day?20:48
flaper87kgriffs: +120:49
kgriffsmost people won't be around monday (it's a holiday in the US)20:49
flaper87ah, didn't know that20:49
flaper87yeah, sure20:49
flaper87Tuesday earlier in the day sounds good20:49
kgriffskk, just ping me tues20:49
*** fifieldt has joined #openstack-marconi21:12
*** amitgandhi has quit IRC21:16
*** openstackgerrit has quit IRC21:25
*** openstackgerrit has joined #openstack-marconi21:25
*** oz_akan_ has quit IRC21:28
*** cppcabrera has quit IRC21:29
*** ayoung has quit IRC22:08
openstackgerritKurt Griffiths proposed a change to stackforge/marconi: fix: find_one() running slow (MongoDB)  https://review.openstack.org/4456022:14
*** _alexr_ has quit IRC22:42
*** kgriffs is now known as kgriffs_afk22:45
*** kgriffs_afk is now known as kgriffs23:14
*** flaper87 is now known as flaper87|afk23:22
*** kgriffs is now known as kgriffs_afk23:28

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!