Friday, 2013-11-15

*** sudorandom has joined #openstack-marconi00:14
*** reed has quit IRC00:23
*** reed_ has joined #openstack-marconi00:24
*** reed_ has quit IRC00:25
*** lakspace_ has joined #openstack-marconi00:31
*** amitgandhi has joined #openstack-marconi00:32
*** amitgandhi has quit IRC00:39
*** ayoung has quit IRC00:53
*** ayoung has joined #openstack-marconi00:59
*** nosnos has joined #openstack-marconi01:00
*** malini_afk is now known as malini01:19
*** amitgandhi has joined #openstack-marconi01:35
*** flwang has quit IRC01:37
*** amitgandhi has quit IRC01:40
*** amitgandhi has joined #openstack-marconi02:36
*** jcru has joined #openstack-marconi02:39
*** jcru has quit IRC02:40
*** amitgandhi has quit IRC02:40
*** oz_akan_ has quit IRC02:40
*** oz_akan_ has joined #openstack-marconi02:41
*** oz_akan_ has quit IRC02:44
*** oz_akan_ has joined #openstack-marconi02:45
*** amitgandhi has joined #openstack-marconi02:46
*** amitgandhi has quit IRC02:50
*** malini is now known as malini_afk03:00
*** amitgandhi has joined #openstack-marconi03:36
*** amitgandhi has quit IRC03:41
*** jburkhar1 has joined #openstack-marconi03:50
*** ekarlso has quit IRC03:52
*** jburkhart has quit IRC03:52
*** ekarlso has joined #openstack-marconi03:54
*** fandikurnia01 has joined #openstack-marconi03:58
*** flwang has joined #openstack-marconi04:13
*** amitgandhi has joined #openstack-marconi04:47
*** amitgandhi has quit IRC04:52
*** amitgandhi has joined #openstack-marconi05:37
*** amitgandhi has quit IRC05:42
*** amitgandhi has joined #openstack-marconi06:38
*** amitgandhi has quit IRC06:42
*** openstackgerrit has quit IRC06:46
*** openstackgerrit has joined #openstack-marconi06:46
*** cpallares has quit IRC07:01
*** amitgandhi has joined #openstack-marconi07:39
*** amitgandhi has quit IRC07:43
*** fandikurnia01 has quit IRC08:02
*** fandikurnia01 has joined #openstack-marconi08:03
*** flaper87|afk is now known as flaper8708:08
*** amitgandhi has joined #openstack-marconi08:39
*** amitgandhi has quit IRC08:44
*** flwang has quit IRC09:01
*** yassine has joined #openstack-marconi09:05
*** nosnos has quit IRC09:26
*** nosnos has joined #openstack-marconi09:27
*** nosnos has quit IRC09:31
*** amitgandhi has joined #openstack-marconi09:40
*** amitgandhi has quit IRC09:45
*** amitgandhi has joined #openstack-marconi10:41
*** amitgandhi has quit IRC10:45
*** haomaiwang has quit IRC10:54
*** haomaiwang has joined #openstack-marconi10:55
*** haomaiwang has quit IRC10:59
*** fandikurnia01 has quit IRC11:12
*** jamieh__ has joined #openstack-marconi11:18
*** amitgandhi has joined #openstack-marconi11:41
*** amitgandhi has quit IRC11:46
*** haomaiwang has joined #openstack-marconi11:54
*** vkmc has joined #openstack-marconi11:56
*** tedross has joined #openstack-marconi12:39
*** amitgandhi has joined #openstack-marconi12:42
*** amitgandhi has quit IRC12:47
*** amitgandhi has joined #openstack-marconi12:52
*** aniuskad has joined #openstack-marconi12:53
*** aniuskad has left #openstack-marconi12:54
*** amitgandhi has quit IRC12:56
*** amitgandhi has joined #openstack-marconi13:42
*** amitgandhi has quit IRC13:47
* flaper87 wonders where Alejandro is13:49
*** openstackgerrit has quit IRC13:53
*** openstackgerrit has joined #openstack-marconi13:53
*** aniuskad has joined #openstack-marconi14:04
*** jcru has joined #openstack-marconi14:10
*** russellb is now known as rustlebee14:13
* flaper87 shakes marconi channel14:23
*** amitgandhi has joined #openstack-marconi14:24
*** amitgandhi has quit IRC14:26
*** amitgandhi has joined #openstack-marconi14:27
*** malini_afk is now known as malini14:38
*** kgriffs_afk is now known as kgriffs14:42
flaper87kgriffs: malini morning :)14:42
* flaper87 is reviewing partition patches14:43
*** flwang has joined #openstack-marconi14:56
kgriffsflaper87: Comments here, just minor stuff -
flaper87kgriffs: awesome, thanks for reviewing that15:07
kgriffsflaper87: u see this?
flaper87kgriffs: damn!15:22
flaper87I keep looking at that diagram and reading Marconi everywhere in it!15:25
kgriffsesp given a high-perf queue flavor + push streaming to workers15:27
kgriffsa couple observations15:27
kgriffsFirst, they invented a new protocol and everything. They didn't reference, e.g., Storm.15:28
kgriffsSecond, they missed an opportunity to create a generic "worker pool as a service"15:28
kgriffsI suppose you get some performance benefits from tightly defining what producers and workers can and can't do15:29
flaper87I don't expect AWS to use any existing technology / protocol15:29
flaper87because they can't control it15:29
flaper87brb, meeting15:29
kgriffsyep, this is par for the course wrt Amazon15:29
kgriffsAmazon is the new Microsoft15:30
* kgriffs opinions aren't necessarily those of his employer.15:37
*** alcabrera has joined #openstack-marconi15:42
alcabreraGood morning, everyone. :)15:43
alcabrerakgriffs: yo! :D15:46
*** rustlebee is now known as drumkilla15:50
*** drumkilla is now known as rustlebee15:50
*** rustlebee is now known as drumkilla15:51
kgriffsalcabrera. malini: when flaper87 is back, I'd like to spend some time triaging bugs15:57
* flaper87 back15:58
flaper87kgriffs: LOL15:58
flaper87alcabrera: goooooooood morning!!15:58
maliniwe can set all the proxy related bugs to Invalid, rt?15:58
flaper87malini: I'd say yes, alcabrera ?15:59
malinialcabrera is busy discussing requests module with amitgandhi15:59
*** tedross has quit IRC16:01
* alcabrera catche sup16:02
alcabrera*catches up16:02
alcabreraflaper87: heeey! :D16:03
alcabreramalini: yup - `del proxy`16:03
maliniI dont have the privilege to 'Won't Fix'16:03
malinisomebody else change thtose, plz16:04
kgriffsalcabrera: you should have the ACL16:04
kgriffsI am assuming it is tied to being a member of Marconi Core16:04
kgriffsprotip: is *not* the link shortner you are looking for16:05
flaper87kgriffs: alcabrera it's16:05
alcabreraI'll "Won't Fix"-ify in just a moment. :)16:06
* alcabrera has the ACL powerz16:06
kgriffscrap. stupid won't shorten my launchpad link16:06
* kgriffs tries goo.gl16:06
alcabrerakgriffs: "俺の話しを聞け!更新 ~Vol.104 俺の話しを聞け!~" - definitely not the url shortener. :P16:07
flaper87kgriffs: you shortened the link16:07
kgriffslet's first go down and update status on triaged bugs, then we can look at some news ones16:08
kgriffslet's see16:08
kgriffshomedoc fix16:08
kgriffsI don't see Mr. Vollero in the chanell16:08
flaper87doesn't seem to be around!16:09
kgriffsafaik this work is still pending?16:11
flaper87are we going to update the home doc at some point or replace it with the discoverable API ?16:11
kgriffsI thought I remembered seeing a patch16:11
maliniI dont see anything here,n,z16:13
kgriffsflaper87: mmm, good question. Let's discuss that at the meeting. My first thought is to see if there is a way we can extend the home doc rather than inventing our own one-off format16:13
flaper87kgriffs: kk, kinda update it dynamically16:14
kgriffshmm, I was just reading Sam Harwell's comment on that bug16:15
kgriffsyou guys see that, at the bottom?16:15
kgriffs"is flawed due to the simple fact that URI construction from the listed templates using that RFC will always fail when used with RBAC."16:15
kgriffsI don't follow his logic16:15
*** yassine has quit IRC16:16
flaper87ah, sorry, I do see his comment, I thought you sent one.16:16
kgriffsRBAC doesn't come into play here, perhaps he is referring to keystone/catalog auth16:16
kgriffs(for those following along at home:
kgriffseven then, the RFC rules still work fine, don't they?16:17
flaper87mmh, I don't follow that part either16:17
flaper87I think they do!16:18
kgriffsIf I query "" then get back a URI with an absolute path, I would construct the next uri like so16:19
kgriffsif my path is relative, then I would do16:19
*** tedross has joined #openstack-marconi16:19
kgriffsflaper87: btw, Rackspace keystone currently is hard-coded to put the project ID in the URI in the catalog16:20
kgriffsso, today we just have a rewrite rule in uwsgi to strip it16:20
flaper87I was a bit confused16:20
alcabreramiddleware solves everything (wrt to hard-coded Project Id). :P16:21
flaper87but yeah, the base url is the one that returns the API spec16:21
flaper87somehow I read middleware as middleman16:21
kgriffsand the bug is that we return an absolute path, but didn't include the "v1" part16:21
flaper87it's Friday after all16:21
kgriffsso, for a stock deployment16:21
kgriffsclient queries ""16:22
flaper87also, the home generation can be pulled out of the version and be completely dynamically generated once we get the API layer done16:22
kgriffsget's back, e.g., "queues/{queue-name}"16:22
kgriffsand would construct a bogus URI;16:22
flaper87'of the version package'16:22
kgriffscan somebody sanity-check me on this? I want to make sure I am understanding RFC 3986 correctly16:23
kgriffscould be as simple as testing with urllib16:23
flaper87FWIW, I follow your thinking and I think it's correct16:24
kgriffsoops, not urllib16:24
alcabreraI mix those two up, as well, since urlparse and urllib methods were decoupled a bit between py2 and py316:25
malinikgriffs: anytime you mention RFC, a few of us are going to get lost :(16:25
alcabreraRequest For Cookies 3986. :D16:26
malinialcabrera: tht's a lot better :D16:26
flaper87and 3986 is the number of cookies being requested16:26
maliniRFC is a sure way to kill the triaging session..16:28
maliniI am trying to read the Section 5.2.2 & my poor brain cells are confused16:29
flaper87I started marking other bugs as Won't fix16:30
malini" A non-strict parser may ignore a scheme in the reference16:30
malini      -- if it is identical to the base URI's scheme.16:30
flaper87I mean, proxy bugs16:30
maliniwth does that mean?16:30
maliniMy vote is to come to this bug after all else16:30
flaper87alcabrera: I assume all those proxy bugs can be marked as such16:30
alcabreraThe parser can ignore "https://", for example, if it matches with the scheme for the Host, ""16:31
alcabreraflaper87: yup - WF16:31
kgriffsok, let's come back to this bug16:31
kgriffsLet me write up a counter-point to Sam's argument16:32
kgriffsand test it with urlparse16:32
kgriffsnext up16:32
kgriffs"Autoreconnect not handled in all cases"16:32
kgriffsSo, this one has stalled. I need to get back to that16:33
kgriffsI've got a patch halfway-done16:33
flaper87you should pick that bug16:34
flaper87assign it to you16:34
kgriffsit's already assigned16:34
kgriffsthis one is unassigned16:34
kgriffsLayer 3 errors in MongoDB driver not handled gracefully16:34
flaper87wait, I'm looking at the wrong bug16:34
flaper87ah, nevermind16:34
flaper87I was indeed looking at the wrong bug16:34
flaper87It's friday after all!16:34
kgriffsso, I think we should wait on that until the autoreconnect is done.16:35
kgriffsbetween Mondays and Fridays, it's a wonder anything productive gets done!16:35
kgriffsok, next16:35
kgriffs"Message body size validation is slow"16:35
kgriffsrelated: "total size for all message bodies is not validated"16:36
kgriffszyuan starting looking at these and thinking about what to do there16:36
alcabreraThe slowness bug - that's because we reserialize?16:36
kgriffsalcabrera: yes16:37
kgriffsideally, json.load would just count things as it went16:37
zyuankgriffs: i did some hack to simplejson16:38
kgriffseven if we say, we will include whitespace in the count16:38
zyuanso, it's possible16:38
kgriffswe still have a problem because we have to size individual objects independently16:38
kgriffslike, the size of each message in the array of posted messages16:38
zyuanjust, it's a question whether openstack requirements.txt can accept a hacked simplejson16:38
kgriffsso, I am starting to wonder if this is all worth it after all16:38
kgriffszyuan: exactly. More likely we would have to get the patch merged upstream into simplejson and then it becomes "standard" anyway16:39
kgriffsalternatively, we fork simplejson and include it directly in marconi16:39
kgriffsbut I am not exactly jumping up and down with joy to do that16:40
zyuanand, besides, currently our integer overflow checking slowed simplejson down16:40
kgriffszyuan: that is another good point. I suspect getting a patch into simplejson for just that wouldn't be too difficult16:40
kgriffsI guess that is a different bug16:41
zyuan-- so... if it's not openstack, based on "my ways" i'll fork simplejson and get it solved. :)16:41
kgriffshas one been registered for the integer overflow thing?16:41
zyuankgriffs: i'm not sure whether they care16:41
zyuanas same as json in python std lib, they both populate bignum16:42
kgriffszyuan: alternate ways to check overflow? Could we just do it after parsing the JSON?16:42
zyuanit's just database, msgpack, etc uses fixed size integers16:42
zyuankgriffs: if i do, i'll just get this checked within simplejson C code...16:43
zyuankgriffs: no. if so, we have have to walk down through the json tree16:43
zyuanthis is costly16:43
zyuanpypy's famous quote: O(1) is not doing nothing16:44
kgriffsremind be the original problem that overflow check was trying to solve?16:44
kgriffszyuan: LOL. So true.16:45
kgriffsbtw, created a bug for this just so we can track what happens with it16:45
zyuaninteger generated from simplejson/json != that used by backend16:45
zyuanwhile marconi defines data type based on what backend have16:45
kgriffsthat rings a bell16:46
kgriffsbackend was raising an error when trying to persist big ints16:46
kgriffsbut, some backends may support bignum16:47
zyuankgriffs: this is almost impossible16:47
kgriffsis it not possible to catch raised errors from the backend after the fact?16:47
zyuan(<del>if they do, i'll use msgpack intead</del>)16:48
zyuankgriffs: then you're going to run into teansaction problem16:48
zyuanmsg insertion must success as a whole16:49
kgriffsgood point16:49
kgriffswhat is the performance hit from that hook?16:49
kgriffs(the validation hook)16:49
zyuanno sure. basically, it requires an C-> Python -> C context switch16:50
zyuanwhile all other types of parsing are done within C16:50
zyuanan intersting thing is, if we just pure python json in pypy, it's not an issue16:51
zyuanbecause pypy optimize pure python very well16:51
zyuanit's just cpython is slow if you use C and python at the same time...16:51
zyuananyway, even with the hook, simplejson is much faster than pure python json under cpython16:52
zyuanso this "bug" is merely, i think, we can do better16:52
alcabrerazyuan: IIRC, 2.7 and 3.x python use simplejson as the implementation in the stdlib, so they're equivalent in perf.16:53
alcabrerain 2.6, it *is* really slow16:54
zyuanalcabrera: no16:54
zyuan2.7 is still pure python16:54
alcabreraah, I see.16:54
alcabrerafound it - you're right. :)16:54
zyuanas well as  py33?16:55
zyuannot a loadable module16:56
kgriffs"Note that CPython by default uses the optimized C extension"16:57
zyuanalready a part of the intepreter16:57
kgriffsso why is simplejson faster?16:57
zyuannow they are the same thing i belive16:57
zyuanbring py27 to py26?16:58
kgriffsthey do behave a little differently with unicode16:58
kgriffsif I set ensure_ascii=False then simplejson always returns unicode16:58
zyuani see16:58
kgriffswhereas the stdlib only returns unicode if some of the strings it serializes are unicode16:58
kgriffsit's annoying, actually16:59
kgriffsI prefer simplejson16:59
kgriffsregarding perf, I guess someone should benchmar it16:59
zyuan(i don't think so)16:59
alcabreradone ^^ (it's a lazy benchmark, admittedly)16:59
openstackgerritFlavio Percoco proposed a change to openstack/python-marconiclient: Bootstrap Messages support
kgriffsalcabrera: can you test py2.6 as well?17:04
kgriffsflaper87: It seems that simplejson isn't faster than the stdlib after all!17:05
alcabrerakgriffs: sure thing. I just happen to have py2.6 installed. :)17:05
flaper87I was just looking at alcabrera bench17:05
*** aniuskad has quit IRC17:05
flaper87that's interesting17:05
flaper87well, that removes a dependency from our requirements.txt17:05
kgriffsI do prefer it's unicode handling, but we can work around that I suppose17:05
zyuani'm in favor of the "sometimes not unicode" way17:06
zyuanit's just fine to use str as unicode in python2, they have the same interface17:07
zyuan-- duck typing17:07
*** reed has joined #openstack-marconi17:07
zyuanstr also supports encode, decode, etc17:07
zyuanunlike that in python 317:07
zyuananyway. if we support py26 as well, we may need to preserve simplejson for 26 as least17:08
zyuanif kurt insist to have "always unicode", then preserve it to 27 as well17:08
zyuanpy33, not needed17:09
kgriffsnah, it isn't a big deal.17:09
kgriffsI mean, always having unicode returned in py217:10
flaper87plus, I expect 26 to disapear soon17:10
kgriffsalcabrera: be sure to bench loads as well17:10
kgriffsflaper87: srsly?17:10
kgriffsWhat's the RHEL story there?17:10
zyuan^^ a fact that i don't like, always unicode17:10
* flaper87 hides17:11
alcabreraI'll be adding loads to the benchmark now.17:11
alcabreraHere's the dumps:
kgriffsflaper87: maybe docker will save us. :p17:12
alcabreraIIRC, Fedora (next) ships with py3 by default. I wonder if that's related? ;)17:12
alcabrerapy3 as default, I should say.17:12
kgriffsalcabrera: I just did a quick test and simplejson is faster for loads17:12
kgriffswhile stdlib is fater for dumps17:12
flaper87well, yeah, there're still some envs to support but I've this feeling17:13
kgriffssrsly, if people can deploy openstack with docker, then doesn't it make it less important to support py2.6 ?17:13
flaper87alcabrera: that's certainly related17:13
flaper87anyway, I'll have to step out in a bit17:14
flaper87lets move on!17:14
flaper87next bug17:14
alcabrera+1 - let's move away from this one. :D17:14
kgriffsactually, let's stop here and pick up again next week17:15
alcabrera( kgriffs: on py 3.3.2, json.loads is faster than simplejson.loads )17:15
kgriffsbtw, don't forget about the mtg on Monday17:15
alcabreraw00t - looking forward to the meeting.17:15
kgriffsAdd stuff to the agenda and tag with ur name if there is other stuff to discus17:15
kgriffsIt was recently suggested that all the projects stop spamming openstack-dev with meeting reminders17:16
kgriffsPart of the discussion to reduce noise on the list in general17:17
kgriffsso, let's get closure on the size issue and the simplejson issue17:17
kgriffsfirst, I feel myself being won over to just specifying a max size for the message posting overall, including whitespace17:18
kgriffsand removing the individual message size limit17:18
kgriffsthis is basically what we are already doing17:18
kgriffsjust formalize that17:19
kgriffsflaper87, alcabrera, zyuan, megan_w, malini: thoughts?17:19
alcabrerait's simple, it's fast, and it's easy to reason about17:19
alcabreralen(body_content) < max_size17:19
kgriffswe just say, hey clients, you can't post a serialized array of messages where the resulting text is greater than, e.g., 256 K17:20
kgriffsby extension, that means no individual message could be > 256 K17:20
alcabreraand most clients will already be using a quality JSON module (regardless of language) that'll mini-fy the JSON output.17:20
flaper87you mean, have a max_size for the whole post operation (bulk and single) instead of max_size per message?17:20
flaper87ok, yeah, you meant that17:21
kgriffsflaper87: that's correct17:21
kgriffsit also has the nice side-effect of encouraging clients to not pretty-print by default17:21
kgriffsreducing overall traffic going to a marconi deployment17:21
flaper87mhh, but what's the benefit there? When the request gets to Marconi, it's already in memory17:22
flaper87why shouldn't we process it if we already have it ?17:22
kgriffscouple things17:22
kgriffsfirst, you could also restrict this in the web server17:22
kgriffsin fact, just check content-length17:22
flaper87(genuine question, btw)17:22
kgriffsthen it isn't in memory17:22
kgriffswe only read content-length bytes anyway17:22
kgriffssecond benefit is we don't end up with arbitrarily large documents in the backend storage17:23
flaper87but for 1) it won't be marconi to enforce the limit but the web server, right?17:23
kgriffskeeps those sane17:23
kgriffsflaper87: good question; the web server could enforce the limit17:23
flaper87but we will end up with large documents17:23
kgriffsand we could also check it in the transport17:24
flaper87I can send 1 document of 50kbs and another one of 20617:24
kgriffsflaper87: yes, but no single document could exceed the overall max17:24
kgriffsso, assuming the overall max is something sane, then we are ok17:24
flaper87got your point - again, it's Friday in Italy, I don't know about the US but we become very dumb on Fridays -17:25
kgriffsso, idk whether the web server should be expected to enforce the limit or we add a second-tier check in the transport as wel17:25
*** cindy_ has joined #openstack-marconi17:25
flaper87I like the idea17:25
kgriffsflaper87: LOL17:25
flaper87so, +1 from me17:25
kgriffsflaper87: In the US we get very dump on Mondays17:25
flaper87I think it all comes down to food17:26
flaper87We have pasta like everyday 10 times per day17:26
kgriffsfolks, any advantages to doing the sanity-check in the marconi app itself?17:26
flaper87then on saturdays we're kinda dead17:26
kgriffsseems like it may make life a little easier for ops17:26
flaper87and then we have a 10 hours lunch on sundays17:26
alcabreraflaper87: soo much foooood. :)17:27
kgriffsi mean, they don't have to make a rule that only applies to POSTing messages17:27
kgriffs(in their web server config)17:27
flaper87alcabrera: isn't that what life is about? I mean, eating17:27
flaper87+1 for having that in Marconi17:27
kgriffsflaper87: WAH?!17:27
flaper87that's part of Marconi's API17:27
kgriffsflaper87: You need to invite me over for lunch17:27
kgriffsflaper87: I agree.17:27
kgriffsok, so the proposal is:17:27
alcabreraI favor having the check in marconi17:28
flaper87kgriffs: you're more than welcome! I'll make sure you'll have the best pasta ever!17:28
kgriffs1. Redefine max message size in the spec (v1.1 I guess)17:28
flaper87alcabrera: you too, so, stop crying!17:28
kgriffs2. Do a quick content-length check in the WSGI transport driver17:28
flaper87zyuan: and you too!17:28
* kgriffs is drooling all over his keyboard17:28
maliniI have no friends :'(17:29
flaper87malini: YOU TOOOO!17:29
maliniwhat a relief!!17:29
flaper87kgriffs: LOOOL17:29
flaper87kgriffs: +1 for the proposal17:29
alcabreraflaper87: hahaha, awesome. :D17:29
flaper87it sounds good!17:29
kgriffszyuan: sound good? ^^^17:29
kgriffsok, so we can basically do the implementation now17:30
kgriffsand formalize it in the docs for API v1.117:30
kgriffsin other words, we won't "fix" those bugs until 1.117:30
kgriffssomething like that17:30
kgriffsI always expected 1.0 to be a bit messy17:31
kgriffs(it always is)17:31
kgriffsok, let's run with that17:31
zyuansry, did not follow17:31
zyuani see. np17:33
kgriffszyuan: basically we are just formalizing what you are doing already17:33
kgriffswe remove the per-message size option from config17:34
zyuanno problem. i read conversation17:34
kgriffsand just check content-length header for max overall size17:34
zyuanon this
flaper87kk, gtg guys17:34
kgriffsthat will essentially "fix" those two bugs, no?17:34
kgriffsflaper87: cheers17:34
zyuani think we need a tiny patch to choose from json and simplejson17:34
kgriffsflaper87: one thing to ponder - in the client doing a seamless integration with swift17:35
kgriffs(for large payloads)17:35
alcabreraflaper87: take care! :)17:36
*** flaper87 is now known as flaper87|afk17:36
kgriffszyuan: gochya. I think we will be fine getting rid of simplejson17:36
kgriffsit should balance out performance-wise, since one is faster serializing while the other is faster deserializing17:36
kgriffsand, like you said, we get the side benefit of pypy goodness17:37
kgriffsalcabrera: any objections?17:37
zyuani mean, py26 still need simplejson17:37
zyuanand, it's just good for now, always good for pypy17:38
alcabreraI'm fine with simplejson being eliminated, all the moreso since python 3.3+ and pypy exhibit faster json behavior in all cases.17:38
zyuan...then why we still spport py26?17:38
alcabrerazyuan: legacy systems. D: (also, openstack still supports 2.6)17:39
zyuanthen why decrease its performance intentionally?17:39
kgriffszyuan: why does py26 need it?17:39
zyuanbecause pure python json is slow in py2617:39
zyuan10x slow17:40
kgriffsoh well17:40
kgriffswe should just tell people to use python 2.717:40
kgriffsit is faster in other ways besides json17:40
zyuanyou know, YaoMing face17:40
kgriffsif people complain it is slow, we just say "deploy on py27)17:40
alcabrerakgriffs: I was thinking that, too, and felt kind of mean, but kind of practical, too. :P17:40
alcabreraMoving Python forward. ;)17:41
kgriffsI think Rackspace is currently deployed on py26?17:41
zyuanno comment. agree with silence17:41
zyuankgriffs: i hope they don't?17:41
*** cindy_ is now known as cpallares17:41
alcabrerakgriffs: I think that's correct. I'll check with mpanetta and oz_akan_.17:41
kgriffsI don't mind being 2.6 *compatible* but saying that it won't give you the best experience17:41
kgriffsalcabrera: kk. We really need to upgrade. If it is because of RHEL, they should try docker17:42
kgriffswho wants to remove simplejson then?17:42
kgriffs(also from requirements.txt)17:42
kgriffsmalini: we will need to watch out for performance regressions - not a big deal with 2.6 but we shouldn't see any for 2.717:43
kgriffsand indeed, things should be faster with pypy17:43
alcabreraI'm fine with dropping simplejson17:44
alcabreraIt would require two changes in the code, namely, 'import simplejson as json' -> 'import json'17:44
kgriffsok, so you or zyuan want to do it?17:45
kgriffs(do the work)17:45
alcabreraI'll do it.17:46
alcabreraPatch incoming in 10 min. :D17:46
alcabrerakgriffs: do we have a bug open for this?17:48
alcabrerajust double-checking, since I've been multiplexing eom:bastion, book shopping, and benchmarking while keeping up with the simplejson discussion. ;)17:48
openstackgerritAlejandro Cabrera proposed a change to openstack/marconi: refactor: drop simplejson requirement
kgriffsalcabrera: mmm, there isn't a bug for just the json change17:51
alcabreraThe patch is now available.17:52
*** flwang has quit IRC18:01
*** whenry has quit IRC18:12
*** whenry has joined #openstack-marconi18:46
*** reed has quit IRC18:55
*** whenry has quit IRC19:02
*** jergerber has joined #openstack-marconi19:17
*** jburkhar1 is now known as jburkhart19:26
*** oz_akan_ has joined #openstack-marconi19:34
*** oz_akan_ has quit IRC19:34
*** tedross has quit IRC19:35
*** oz_akan_ has joined #openstack-marconi19:35
*** tedross has joined #openstack-marconi19:54
*** kgriffs is now known as kgriffs_afk20:19
*** malini is now known as malini_afk20:27
russell_hare there any docs/examples on how to use python-marconiclient?20:40
*** mpanetta has joined #openstack-marconi20:41
*** mpanetta has quit IRC20:42
*** mpanetta has joined #openstack-marconi20:43
alcabrerarussell_h: not yet. flaper87|afk has been hard at work making the implementation work. :)20:44
alcabreraWe've got a vision doc of sorts.20:44
alcabrera (might be a little dated)20:44
russell_hyeah, I found that, seems pretty reasonable, I was mostly just trying to figure out how to properly construct a client20:45
russell_hthe API doesn't appear to match that wiki page exactly right now, but looks pretty workable20:45
alcabreraI'm giving it a look. It's been awhile since I've had a chance to play with the client.20:49
alcabrerarussell_h: based on what I'm reading, you bring in a 'marconiclient.queues.v1.client.Client', pass it an instance of oslo.config.cfg.ConfigOpts (or oslo.config.cfg.CONF), and then...20:55
*** jamieh__ has quit IRC20:56
alcabreraThat config either indicates which config file to use, and contains 'os_queues_url' under [DEFAULT], or pass in the url to the Client(url=localhost:8000)20:56
alcabreraThat should do it.20:56
alcabreraFrom there, you can do 'c = Client(...)', 'c.queue(<queue_name>)' to inspect the functionality of the client.20:57
russell_hah, I didn't realize I could pass a url directly20:59
russell_hactually, unless there's some serious magic, that isn't implemented21:00
russell_hits set, just never used21:01
alcabreragood point21:02
alcabreraI'm out. Have a good weekend, everyone!21:08
*** alcabrera has quit IRC21:09
*** jamieh__ has joined #openstack-marconi21:11
*** jamieh__ has quit IRC21:35
*** jamieh__ has joined #openstack-marconi21:43
*** reed has joined #openstack-marconi21:53
*** whenry has joined #openstack-marconi21:54
*** drumkilla has quit IRC22:19
*** russellb has joined #openstack-marconi22:20
*** kgriffs_afk is now known as kgriffs22:21
*** amitgandhi has quit IRC22:23
*** amitgandhi has joined #openstack-marconi22:24
*** cpallares has quit IRC22:26
*** amitgandhi has quit IRC22:29
*** tedross has quit IRC22:39
openstackgerritA change was merged to openstack/marconi: Update openstack/common/lockutils
*** jcru has quit IRC23:20
*** amitgandhi has joined #openstack-marconi23:24
*** kgriffs is now known as kgriffs_afk23:27
*** amitgandhi has quit IRC23:31
*** amitgandhi has joined #openstack-marconi23:34
*** malini_afk is now known as malini23:37
*** amitgandhi has quit IRC23:39
*** malini is now known as malini_afk23:57
*** vkmc has quit IRC23:58

Generated by 2.14.0 by Marius Gedminas - find it at!