| *** malini_afk is now known as malini | 00:28 | |
| *** oz_akan_ has joined #openstack-marconi | 00:45 | |
| *** oz_akan_ has quit IRC | 00:46 | |
| *** oz_akan_ has joined #openstack-marconi | 00:47 | |
| *** nosnos has joined #openstack-marconi | 00:49 | |
| *** amitgandhi has joined #openstack-marconi | 01:29 | |
| *** malini is now known as malini_afk | 01:34 | |
| *** amitgandhi has quit IRC | 02:00 | |
| *** whenry has quit IRC | 03:26 | |
| *** oz_akan_ has quit IRC | 03:43 | |
| *** oz_akan_ has joined #openstack-marconi | 04:54 | |
| *** oz_akan_ has quit IRC | 04:59 | |
| *** tvb|afk has joined #openstack-marconi | 05:25 | |
| *** tvb|afk has quit IRC | 05:30 | |
| *** tvb|afk has joined #openstack-marconi | 05:54 | |
| *** tvb|afk_ has joined #openstack-marconi | 06:16 | |
| *** tvb|afk has quit IRC | 06:20 | |
| *** tvb|afk_ is now known as tvb | 06:31 | |
| *** tvb has quit IRC | 06:31 | |
| *** tvb has joined #openstack-marconi | 06:31 | |
| *** flaper87|afk is now known as flaper87 | 07:20 | |
| *** flaper87 is now known as flaper87|afk | 07:43 | |
| *** yassine has joined #openstack-marconi | 07:48 | |
| *** flaper87|afk is now known as flaper87 | 07:59 | |
| *** flaper87 is now known as flaper87|afk | 08:09 | |
| *** flaper87|afk is now known as flaper87 | 08:12 | |
| *** tvb has quit IRC | 08:26 | |
| *** flaper87 is now known as flaper87|afk | 08:28 | |
| *** flaper87|afk is now known as flaper87 | 08:39 | |
| *** ykaplan has joined #openstack-marconi | 08:54 | |
| *** ykaplan has quit IRC | 09:17 | |
| *** tvb has joined #openstack-marconi | 09:27 | |
| *** flaper87 is now known as flaper87|afk | 09:31 | |
| *** tvb has quit IRC | 09:31 | |
| *** flaper87|afk is now known as flaper87 | 09:42 | |
| *** ykaplan has joined #openstack-marconi | 09:50 | |
| *** tvb has joined #openstack-marconi | 10:28 | |
| *** tvb has quit IRC | 10:32 | |
| *** ykaplan has quit IRC | 10:33 | |
| *** ykaplan has joined #openstack-marconi | 10:48 | |
| *** ykaplan has quit IRC | 11:11 | |
| *** nosnos has quit IRC | 11:16 | |
| *** oz_akan_ has joined #openstack-marconi | 11:20 | |
| *** oz_akan_ has quit IRC | 11:22 | |
| *** tvb has joined #openstack-marconi | 11:23 | |
| *** ykaplan has joined #openstack-marconi | 11:32 | |
| *** flaper87 is now known as flaper87|afk | 11:53 | |
| *** tedross has joined #openstack-marconi | 11:59 | |
| *** fvollero-pto is now known as fvollero | 12:08 | |
| *** flaper87|afk is now known as flaper87 | 12:17 | |
| *** kgriffs_afk is now known as kgriffs | 12:27 | |
| kgriffs | flaper87: did u see my comments on this? https://review.openstack.org/#/c/48669/3/marconiclient/transport/request.py | 12:29 |
|---|---|---|
| kgriffs | oh, and good afternoon! | 12:29 |
| flaper87 | kgriffs: gooood morning :) | 12:30 |
| flaper87 | kgriffs: I saw them today but I've been tackling down some internal stuff today, I'll get to them in a bit | 12:30 |
| flaper87 | I think the previous patch is ready, though | 12:30 |
| kgriffs | np, just checking | 12:31 |
| flaper87 | kgriffs: also, Gerrit is way to stupid and hides -1s under +2s | 12:31 |
| kgriffs | yeah | 12:33 |
| kgriffs | totally lame | 12:33 |
| *** kgriffs is now known as kgriffs_afk | 12:33 | |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Add a base transport class https://review.openstack.org/48287 | 12:34 |
| *** kgriffs_afk is now known as kgriffs | 12:51 | |
| *** acabrera has joined #openstack-marconi | 12:53 | |
| kgriffs | flaper87: also, it would be great if you could review Alej's latest proxy patches today. I am planning to do so as well | 12:53 |
| flaper87 | kgriffs: will do, before working on other patches | 12:54 |
| kgriffs | rock on | 12:54 |
| acabrera | Morning! :) | 12:55 |
| acabrera | I get here and proxies are being mentioned. My timing is fantastic. ;) | 12:55 |
| acabrera | flaper87, kgriffs: Thanks! | 12:55 |
| flaper87 | acabrera: LOOOOOL | 12:55 |
| *** mpanetta has joined #openstack-marconi | 13:00 | |
| acabrera | mpanetta: o/ | 13:01 |
| mpanetta | acabrera: morning :) | 13:02 |
| mpanetta | acabrera: How goes? | 13:03 |
| mpanetta | I did not get up as early as I wanted | 13:03 |
| acabrera | mpanetta: I didn't get up as early as I intended, either. :P | 13:04 |
| acabrera | Things go well, though. :D | 13:04 |
| mpanetta | Good :) | 13:04 |
| mpanetta | 6 days 22hrs 54mins and 40 secs | 13:04 |
| mpanetta | says the countdown timer over here | 13:04 |
| acabrera | lol that timer... | 13:05 |
| acabrera | It's fun to watch, perhaps bordering on hypnotic. | 13:05 |
| acabrera | I'd be distracted by it if I were on that side of the office. | 13:05 |
| mpanetta | It needs to be spiced up a bit | 13:05 |
| acabrera | :P | 13:05 |
| mpanetta | It is mildly distracting, but I'll get over it ;) | 13:05 |
| flaper87 | timer? | 13:06 |
| mpanetta | flaper87: countdown until GA | 13:06 |
| acabrera | Google: "Spice Up Your Timer" -> "Do you want to spice up your games and even spice up your daily life with **random timers**..." | 13:07 |
| mpanetta | haha | 13:07 |
| acabrera | Google knows. :P | 13:07 |
| acabrera | My email inbox is awesomely clear this morning, so I'll take a little bit to catch up on accumulated, unsorted bookmarks. | 13:08 |
| mpanetta | acabrera: I could send you more email if you would like :P | 13:10 |
| acabrera | mpanetta: lol if it contains links to awesome videos/reading material, it's super welcome. :D | 13:11 |
| acabrera | mpanetta: but yeah, I'm sure I'll have an email within 10 minutes` | 13:11 |
| acabrera | from something | 13:11 |
| acabrera | someone | 13:11 |
| acabrera | some alerting system | 13:11 |
| acabrera | some subscription | 13:11 |
| mpanetta | Ah yes | 13:11 |
| mpanetta | I just wish every day was fruit tuesday | 13:13 |
| acabrera | mpanetta: We could make it happen as part of the Racklanta Health initiative. I'm sure it wouldn't be too difficult to get weekly volunteers to fetch apples, bananas, and more from the nearby Kroger's. | 13:14 |
| mpanetta | acabrera: That would be nice. | 13:15 |
| flaper87 | oh man! | 13:15 |
| *** ayoung has joined #openstack-marconi | 13:16 | |
| *** amitgandhi has joined #openstack-marconi | 13:22 | |
| acabrera | brb | 13:25 |
| *** acabrera has quit IRC | 13:25 | |
| *** cppcabrera has joined #openstack-marconi | 13:27 | |
| *** cppcabrera has quit IRC | 13:27 | |
| *** alcabrera has joined #openstack-marconi | 13:28 | |
| mpanetta | wb | 13:29 |
| alcabrera | mpanetta: Thanks! I just decided to swap back to using weechat after giving irssi a try for a week. | 13:30 |
| mpanetta | Ah irssi | 13:31 |
| alcabrera | flaper87: Thanks for the reviews! :) | 13:31 |
| flaper87 | alcabrera: weechat rocks. Are you using any fancy plugin that I should be using as well? | 13:31 |
| flaper87 | alcabrera: thank you for the hard work | 13:31 |
| mpanetta | alcabrera: Thanks for submitting that bug last minute last night. | 13:31 |
| alcabrera | flaper87: no fancy plugins here - just straight weechat. | 13:31 |
| flaper87 | I couldn't help smiling when I realized the last patch for the proxy is 260 LOC | 13:32 |
| alcabrera | mpanetta: no problem. I ran into that bug a little earlier so it was worthwhile to triage it's status higher now that it affects more people. | 13:32 |
| flaper87 | it was like: Ohhh, YES! We're getting there! | 13:32 |
| flaper87 | :P | 13:32 |
| alcabrera | flaper87: hell yeah! hahaha. I'm having the same feeling. :D | 13:33 |
| mpanetta | Think I will have something to test soon? | 13:33 |
| * mpanetta is itchy to test ;) | 13:33 | |
| alcabrera | lol | 13:33 |
| alcabrera | flaper87: it was also fun to work with jsonschema. I highly recommend it for validating transport-layer input moving forward. | 13:34 |
| alcabrera | mpanetta: I'll start working on that bug in about 25 minutes. I hope to send you a direct, early, pre-review patch by 11am EDT. | 13:34 |
| *** yassine has quit IRC | 13:34 | |
| *** yassine has joined #openstack-marconi | 13:35 | |
| mpanetta | alcabrera: Sweet, that actually fits in with my schedule pretty well today. We have a new interviewee coming in. | 13:35 |
| *** malini_afk is now known as malini | 13:37 | |
| flaper87 | alcabrera: I'm glad to hear that, really! | 13:37 |
| alcabrera | flaper87: I took your thoughts on versioned APIs in the mirror_queues patch and turned it into a BP. :) | 13:39 |
| flaper87 | alcabrera: awesome! | 13:40 |
| flaper87 | alcabrera: thanks | 13:40 |
| flaper87 | alcabrera: it's not clear to me why lookup is a function here: https://review.openstack.org/#/c/49262/2/marconi/proxy/transport/wsgi/partitions.py | 13:43 |
| alcabrera | flaper87: oooh, you found some of my cruft - nice! | 13:43 |
| alcabrera | It was *a lot* more useful when I had a WeightResource and a HostsResource implementing PUT on partitions in an earlier version of the patch. It was used to keep things DRY, since I used that idiom in three places. | 13:44 |
| alcabrera | Now it just gets in the way. | 13:44 |
| alcabrera | I'll remove that shortly. | 13:44 |
| flaper87 | kk, I'll drop a comment there | 13:45 |
| alcabrera | flaper87: Thanks! It's also likely that on_patch will benefit from the presence of a docstring, so I'll that add in for the coming patchset. | 13:47 |
| alcabrera | flaper87: nevermind, it's already there, just - a little weak. | 13:47 |
| * alcabrera doesn't remember what he wrote yestetday | 13:48 | |
| alcabrera | flaper87: thanks for the reviews! | 13:50 |
| flaper87 | alcabrera: LOL, I don't remember what I had for lunch anymore | 13:53 |
| flaper87 | :D | 13:53 |
| flaper87 | alcabrera: np, my pleasure | 13:53 |
| *** yassine has quit IRC | 14:00 | |
| *** yassine has joined #openstack-marconi | 14:01 | |
| flaper87 | kk, reviewed everythin | 14:08 |
| flaper87 | kk, reviewed everything | 14:08 |
| *** jcru has joined #openstack-marconi | 14:08 | |
| alcabrera | flaper87: super appreciated! | 14:09 |
| alcabrera | I'm almost done applying the suggested changes to the last proxy patch. | 14:10 |
| flaper87 | WHAT??? | 14:21 |
| * flaper87 runs away | 14:21 | |
| flaper87 | :P | 14:21 |
| flaper87 | alcabrera: man, take your time! | 14:21 |
| *** tvb has quit IRC | 14:22 | |
| alcabrera | lol | 14:24 |
| alcabrera | I also found some bugs in the unit test suite for test_partitions, so I'm clearing those up. tests went running. D: | 14:24 |
| openstackgerrit | Zhihao Yuan proposed a change to openstack/marconi: feat(api): Client-ID is now a real UUID https://review.openstack.org/49378 | 14:37 |
| zyuan | review wanted ^^ | 14:38 |
| *** oz_akan_ has joined #openstack-marconi | 14:39 | |
| flaper87 | zyuan: lol @ review wanted :D | 14:44 |
| openstackgerrit | Zhihao Yuan proposed a change to openstack/marconi: feat(api): Client-ID is now a real UUID https://review.openstack.org/49378 | 14:45 |
| zyuan | flaper87: an interesting fact is, to make this change work, mongo driver needs to do *nothing* | 14:46 |
| kgriffs | cool, I am looking forward to that patch | 14:51 |
| kgriffs | oz_akan_, mpanetta: are you currently including the Client-ID header in the uwsgi logs? | 14:53 |
| mpanetta | I do not believe so. | 14:54 |
| kgriffs | Can we add that to the "todo" list? | 14:54 |
| kgriffs | it would be really helpful for data mining and diagnostics | 14:54 |
| mpanetta | Yeah, only have client addr and port. | 14:55 |
| kgriffs | iirc, uwsgi does not allow for logging arbitrary headers | 14:55 |
| kgriffs | if it does, then GREAT | 14:56 |
| mpanetta | I do not know. | 14:56 |
| kgriffs | take a look and if there doesn't seem a way to do it, I've got an EOM branch that can help | 14:56 |
| kgriffs | https://github.com/racker/eom/blob/uwsgi/etc/uwsgi.json-sample | 14:56 |
| kgriffs | https://github.com/racker/eom/blob/uwsgi/eom/uwsgi_magic.py | 14:56 |
| kgriffs | If you need it, then ping me and i will get that code merged into the mainline | 14:57 |
| mpanetta | kgriffs: Do you mean client ID or customer ID? | 14:59 |
| kgriffs | Client-ID | 14:59 |
| kgriffs | it will be a UUID string | 14:59 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat (proxy/admin): allow partition modifications https://review.openstack.org/49262 | 14:59 |
| kgriffs | this is not X-Project-ID | 14:59 |
| mpanetta | ok | 14:59 |
| kgriffs | thanks! | 14:59 |
| mpanetta | I will take a look | 15:00 |
| * kgriffs gives mpanetta a high-five | 15:00 | |
| alcabrera | kgriffs: flaper87 reviewed all the proxy patches this morning. I've addressed all comments including those made on the most recent patch. Things are looking pretty awesome proxy-side (except for a certain memcached-keys bug that I'm going to destroy now). :) | 15:03 |
| mpanetta | kgriffs: I do not see it as an immediate option. | 15:03 |
| kgriffs | ok. I will add it to my list to get that EOM branch cleaned up and merged | 15:04 |
| mpanetta | kgriffs: check other channel please | 15:05 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Add Tests for Queue Stats https://review.openstack.org/48012 | 15:05 |
| zyuan | flaper87: the transport code used str(ex) everywhere. another thing is, tis ex is generated from Python std lib, it's not very helpful to make it unicode... | 15:11 |
| zyuan | flaper87: one more thing, python3 has no identifier "unicode" | 15:11 |
| flaper87 | zyuan: you can use six there, there's a bp for the unicode thing, i'll be changing the transport part soon | 15:22 |
| *** jdprax has quit IRC | 15:23 | |
| *** tvb has joined #openstack-marconi | 15:23 | |
| *** tvb has quit IRC | 15:23 | |
| *** tvb has joined #openstack-marconi | 15:23 | |
| zyuan | flaper87: i don't it's the case. i think for error message what we want is the default string, no explicitly unicode. python2 string has enough interfaces to make this part work, even some falcon part assume the message is unicode | 15:24 |
| zyuan | duck typing | 15:24 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: fix: encode keys before caching https://review.openstack.org/49388 | 15:26 |
| zyuan | flaper87: for example, if falcon want to encode the error message, just do s.encode('utf-8'), it works | 15:26 |
| zyuan | in python3, it also works. | 15:27 |
| *** tvb has quit IRC | 15:30 | |
| *** flaper87 is now known as flaper87|afk | 15:33 | |
| zyuan | alcabrera: are you working on this? https://bugs.launchpad.net/marconi/+bug/1233277 | 15:37 |
| alcabrera | zyuan: no | 15:38 |
| alcabrera | zyuan: I'm tackling the caching bug atm | 15:38 |
| *** flaper87|afk is now known as flaper87 | 15:41 | |
| flaper87 | zyuan: sorry, got disconnected | 15:41 |
| flaper87 | #$%@!$%#!@%# connection | 15:41 |
| flaper87 | dunno if you got my last messages | 15:41 |
| flaper87 | so | 15:41 |
| flaper87 | sure, but that's falcon's issue, not marconi's. Right? | 15:41 |
| flaper87 | so, I think we should be using unicode throughout marconi | 15:41 |
| zyuan | i don't think so. i tend to say use unicode's interface | 15:42 |
| flaper87 | and we should also be using unicode all around because is safer and doesn't make any assumptions | 15:43 |
| *** jdprax has joined #openstack-marconi | 15:43 | |
| zyuan | let's say, what do you want to do with unicode ther than encode? | 15:46 |
| zyuan | when the content is an error message comes out of Python standard library? | 15:48 |
| flaper87 | I just want to keep everything in unicode while it's in marconi-land because it's safer and doesn't make any assumption. I want to keep it consistent with the rest of the code and with what we're targeting. | 15:48 |
| zyuan | you can't. at least we have bytes in marconi | 15:48 |
| flaper87 | we want to use unicode throughout marconi and encode at the very end | 15:48 |
| zyuan | we have bytes, generic string, unicde string | 15:49 |
| zyuan | use them appropiately. | 15:49 |
| zyuan | unicode is good, but not good for everything. | 15:52 |
| flaper87 | where are those not being used appropriately? Plus, I don't think we should be trusting blindly what comes out from python's std | 15:52 |
| zyuan | i don't understand why people what to use it everywhere. | 15:52 |
| *** yassine has quit IRC | 15:52 | |
| zyuan | i don't want to "trust" unicode "blindly" too. | 15:53 |
| zyuan | unicode is good for very specific things. when you know your use case, why don't use a generic string? | 15:53 |
| *** kgriffs is now known as kgriffs_afk | 15:53 | |
| zyuan | early python3 is slower then python2, because all strings are unicode. now they made the string literal char string when possible. so here it is: do you really care about the string representation? no, the important thing is the interface. | 15:56 |
| zyuan | the bad part about python2 is that it has no bytes type, it's not "it's string is not unicode". | 15:57 |
| zyuan | unicode the type does not save you. | 15:57 |
| *** ykaplan has quit IRC | 16:01 | |
| flaper87 | zyuan: I agree with you, I also agree that unicode it's not a hammer, however, it's a practice we want to have in Marconi and in this case it won't affect performance since it's all about calling unicode on an exception that shouldn't be ocurring | 16:08 |
| flaper87 | frequently | 16:08 |
| zyuan | yes, i don't care the performance of unhappy path | 16:09 |
| zyuan | if you want you can use six on them later. | 16:10 |
| zyuan | (but i still don't see benefits here... :( | 16:10 |
| flaper87 | zyuan: I don't think we should keep accumulating technical dept :/ you would help me if you do it in this patch, pleeeeeeeeeeassseee :D | 16:11 |
| flaper87 | zyuan: there's no *real* benefit besides sticking to what we decided for marconi (re unicode / str ) | 16:11 |
| flaper87 | zyuan: like I said, I totally get your point and agree with it :) | 16:11 |
| alcabrera | flaper87: can I get a review (and maybe a +2) on this: https://review.openstack.org/#/c/49262/ :D | 16:20 |
| alcabrera | it's that patch from earlier - I got the fixes in + more tests | 16:21 |
| openstackgerrit | Zhihao Yuan proposed a change to openstack/marconi: fix: force exception content to be text_type https://review.openstack.org/49399 | 16:23 |
| openstackgerrit | Zhihao Yuan proposed a change to openstack/marconi: feat(api): Client-ID is now a real UUID https://review.openstack.org/49378 | 16:25 |
| alcabrera | oz_akan_, kgriffs_afk, flaper87: for whatever reason, python-memcached (cache.set) doesn't seem to work with redis (using Redis 2.6.16). | 16:27 |
| alcabrera | however, python-memcached *does* work with memcached 1.4.15 (thankfully) | 16:27 |
| alcabrera | This is all in relation to oslo.cache | 16:27 |
| alcabrera | Just learned a few things while making sure marconi hierarchical caching is up to par | 16:27 |
| flaper87 | alcabrera: mmh, does it throw any error? | 16:28 |
| flaper87 | alcabrera: btw, I'll take a look at those patches | 16:28 |
| flaper87 | zyuan: uuu, you did them all. THANKS! | 16:29 |
| alcabrera | flaper87: cache.set(name, thing) => False (while observing redis-cli monitor) | 16:30 |
| alcabrera | I've also tinkered and restarted things to make sure it wasn't a flop on my side. | 16:30 |
| alcabrera | It's a *silent* failure. :x | 16:30 |
| flaper87 | mmhh, looks like something related to redis-py | 16:31 |
| alcabrera | flaper87: Whenever I get a chance (if kgriffs_afk doesn't beat me to it), I'll have fun submitting a proper python-redis backend for oslo.cache. :D | 16:33 |
| alcabrera | (Probably not in the next week, but it's on my TODO). | 16:33 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Add prepare_request function to requests.py https://review.openstack.org/48669 | 16:33 |
| * alcabrera enjoys working with oslo.cache | 16:33 | |
| alcabrera | Now with a bug out of the way, I'm going to take a moment to have a desk lunch and review patches. | 16:33 |
| flaper87 | alcabrera: AWESOME | 16:34 |
| flaper87 | start with my last patch | 16:34 |
| flaper87 | :D | 16:34 |
| flaper87 | muahauhauhauhauah | 16:34 |
| alcabrera | lol | 16:34 |
| flaper87 | alcabrera: no code changed, just docstrings | 16:34 |
| alcabrera | sure thing. I'll then proceed to zyuan's patches, then see what else is pending | 16:34 |
| alcabrera | flaper87: also, beyond the proxy patch, tell me what you think of this patch (+4, -2 LOC): https://review.openstack.org/#/c/49388/ (bug fix) | 16:35 |
| alcabrera | I plan on adding unit tests to that later today, and I've tested it locally. | 16:35 |
| alcabrera | brb | 16:36 |
| flaper87 | alcabrera: perhaps we should fix that in oslo.cache directly | 16:37 |
| flaper87 | alcabrera: I remember seeing something similar but not sure if it was the same issue | 16:37 |
| alcabrera | flaper87: sounds good to me. I like the idea of making the fix in oslo.cache (ensuring type(key) == bytes) since it makes it more transparent for everyone else to use. | 16:40 |
| alcabrera | I'll do that. | 16:40 |
| * alcabrera grabs his magic six hammer | 16:40 | |
| flaper87 | alcabrera: :D | 16:40 |
| alcabrera | given the existence of self._prepare_key, this seems straightforward | 16:41 |
| *** tvb has joined #openstack-marconi | 16:41 | |
| *** whenry has joined #openstack-marconi | 16:42 | |
| flaper87 | no comments, stupid ISP | 16:45 |
| flaper87 | ok, links to BPs added: https://wiki.openstack.org/wiki/Marconi/Incubation#Incubation_Period_Notes | 16:45 |
| flaper87 | kgriffs_afk: ^ | 16:45 |
| *** tvb has quit IRC | 16:46 | |
| alcabrera | flaper87: sweet BP linking | 16:46 |
| alcabrera | thanks! | 16:46 |
| flaper87 | malini: ping | 16:48 |
| flaper87 | malini: are you already digging into marconi + tempest integration ? | 16:49 |
| *** vkmc has joined #openstack-marconi | 16:55 | |
| *** vkmc has quit IRC | 16:55 | |
| *** vkmc has joined #openstack-marconi | 16:55 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: fix: encode keys before caching https://review.openstack.org/49388 | 16:55 |
| alcabrera | flaper87: updated to make the change in oslo.cache. ^^ | 16:56 |
| flaper87 | alcabrera: awesome, I'm curious to know what keys failed and why memecached tests didn't raise that | 16:56 |
| alcabrera | cache.set(u'q._/taco', 'weee') | 16:58 |
| alcabrera | That started throwing... | 16:58 |
| flaper87 | mmmhhh, sounds like it doesn't like the '.' | 16:59 |
| flaper87 | I guess | 16:59 |
| flaper87 | not sure | 16:59 |
| flaper87 | just random-stupid guess | 16:59 |
| alcabrera | "MemcachedStringEncodingError: Keys must be str()'s, not unicode. Convert your unicode strings using mystring.encode(charset)!" | 17:02 |
| alcabrera | flaper87: ^^ | 17:02 |
| alcabrera | The '.' is treated just fine. :) | 17:03 |
| flaper87 | alcabrera: I said it was a random-stupid-guess!!! :D | 17:03 |
| alcabrera | Though it wouldn't hurt to have some form of regex that forces keys to conform to a particular character set in oslo.cache. :) | 17:03 |
| alcabrera | lol | 17:03 |
| flaper87 | Have I ever told you how much I hate memcached ? | 17:04 |
| alcabrera | You have not. ;) | 17:04 |
| * alcabrera listens | 17:04 | |
| mpanetta | flaper87: No, but I am starting to hate it by proxy now heh | 17:04 |
| mpanetta | No pun intended | 17:04 |
| flaper87 | alcabrera: ok, I hate every single bit of its very stupid and silly API | 17:04 |
| flaper87 | plus, its poor documentation and ridiculous protocol | 17:05 |
| flaper87 | Wanna know what a good documentation looks like, take a look here: http://redis.io/commands | 17:05 |
| flaper87 | no way you can't understand something from there! | 17:05 |
| alcabrera | flaper87: I *love* that page. I've been there over a hundred times since I discovered Redis. | 17:05 |
| flaper87 | https://code.google.com/p/memcached/wiki/NewCommands < You f1#@$@!$#@ing kidding me ? | 17:06 |
| flaper87 | https://code.google.com/p/memcached/wiki/NewCommands#stats <- WTF is this? | 17:06 |
| alcabrera | flaper87: rofl @ stats | 17:07 |
| alcabrera | That's just bad. | 17:07 |
| flaper87 | "An expiration time, in seconds. Can be up to 30 days. After 30 days, is treated as a unix timestamp of an exact date." <- Yeah right, why don't we apply the same thing to strings? After 30 chars your string will be treated as a BLOB and multiplied by 5 | 17:08 |
| flaper87 | ^ you know, for "consistency" ^ | 17:09 |
| alcabrera | flaper87: lol! | 17:09 |
| alcabrera | I noticed that in the oslo.cache:memcached docs | 17:09 |
| alcabrera | It's pretty terrible., | 17:09 |
| flaper87 | so yeah, I hate it! :D | 17:09 |
| * alcabrera takes note to avoid memcached when possible | 17:10 | |
| flaper87 | alcabrera: you mean, always, right? | 17:10 |
| flaper87 | :D | 17:10 |
| alcabrera | I sure wish. :P | 17:10 |
| torgomatic | the one saving grace is that it's 30 * 86400 seconds, not "30 days from now, including timezones and stuff" | 17:10 |
| alcabrera | torgomatic: +1 - though still quite the quirk | 17:11 |
| torgomatic | it's also a good thing memcached didn't exist in January 1970, because if your system time is near 0, there are expiration times you just can't express in that wonky format :) | 17:11 |
| flaper87 | torgomatic: lol | 17:12 |
| torgomatic | alcabrera: yeah, Swift has a workaround for that quirk: https://github.com/openstack/swift/blob/master/swift/common/memcached.py#L78 | 17:12 |
| torgomatic | it is not my favorite piece of code | 17:12 |
| alcabrera | torgomatic: looks a lot like the workaround in the upcoming oslo.cache. >.> | 17:15 |
| torgomatic | alcabrera: and every other project that touches memcached :| | 17:16 |
| alcabrera | seems like that workaround should be submitted to the python-memcached implementation. | 17:16 |
| alcabrera | Or better yet, in the core of memcached (if that's even possible at this point) | 17:16 |
| flaper87 | brb | 17:16 |
| openstackgerrit | Zhihao Yuan proposed a change to openstack/marconi: feat(api): Client-ID is now a real UUID https://review.openstack.org/49378 | 17:21 |
| alcabrera | brb | 17:30 |
| *** tvb has joined #openstack-marconi | 17:30 | |
| alcabrera | back | 17:36 |
| zyuan | alcabrera: ^^ | 17:41 |
| alcabrera | zyuan: I checked it out. I'll have to think on it a bit. Thanks for providing the rationale for the change in the commit message! | 17:45 |
| *** flaper87 is now known as flaper87|afk | 17:48 | |
| zyuan | alcabrera: can you take a look at this change first? https://review.openstack.org/#/c/49399/ | 17:49 |
| alcabrera | zyuan: +2 - works for me | 17:51 |
| alcabrera | zyuan: I'd prefer if that patch didn't depend on the UUID one. Why does it depend on the uuid one, btw? | 17:52 |
| *** flaper87|afk is now known as flaper87 | 18:01 | |
| zyuan | ChanServ: the uuid one depends on the text_type one | 18:02 |
| zyuan | alcabrera: ^^ | 18:02 |
| zyuan | (so, you see, a habit is very hard to change) | 18:02 |
| alcabrera | lol chanserv | 18:03 |
| flaper87 | ChanServ: feels loved !!!! | 18:03 |
| flaper87 | :D | 18:03 |
| zyuan | T_T | 18:03 |
| alcabrera | flaper87: I appreciate you taking the time to check on marconi late into the evening. I was just thinking, "It's at least 6pm over there!". :) | 18:06 |
| flaper87 | alcabrera: no worries! It's my pleasure! | 18:07 |
| flaper87 | ... and passion :D | 18:07 |
| alcabrera | I understand - totally! Still, very much appreciated. :D | 18:07 |
| alcabrera | flaper87: could you help me get this patch through soon (modifiable partitions): https://review.openstack.org/#/c/49262/ . If you'd prefer to tackle that one (and the oslo.cache fix) tomorrow morning, that works for me, too. | 18:09 |
| flaper87 | alcabrera: no worries, I'll take a look at it while my rust code compiles :D | 18:09 |
| flaper87 | damn, one can't use the compilation excuse nowadays anymore | 18:10 |
| flaper87 | :P | 18:10 |
| alcabrera | lol | 18:11 |
| alcabrera | yup - not in go, not in rust, and just barely in haskell | 18:11 |
| alcabrera | however - in c++, it's as valid as ever | 18:11 |
| mpanetta | alcabrera: "Sorry boss, I have this complex template library I need to compile, I will be off for a day." :P | 18:14 |
| zyuan | :) | 18:14 |
| alcabrera | lol so true... | 18:14 |
| flaper87 | alcabrera: are we sure these tests are running? https://review.openstack.org/#/c/49262/3/tests/unit/proxy/test_partitions.py | 18:17 |
| flaper87 | Why didn't that fail in the previous patch? | 18:17 |
| flaper87 | I mean, because of the borked super | 18:17 |
| alcabrera | flaper87: They're running. I checked with jenkins just now. | 18:18 |
| alcabrera | http://logs.openstack.org/62/49262/3/check/gate-marconi-python27/b7657a3/console.html (grep 'ExistingPartitionTest') | 18:18 |
| flaper87 | alcabrera: oh ok, that's weird, though. | 18:18 |
| flaper87 | alcabrera: LGTM | 18:18 |
| alcabrera | flaper87: I learned the default behavior of a borked super is to skip a test | 18:18 |
| flaper87 | o.0 | 18:18 |
| alcabrera | silent failure - yup... | 18:18 |
| zyuan | flaper87: https://review.openstack.org/#/c/49399/ ? | 18:19 |
| alcabrera | flaper87: thanks! | 18:21 |
| flaper87 | zyuan: +2'd | 18:29 |
| *** amitgandhi has quit IRC | 18:32 | |
| openstackgerrit | A change was merged to openstack/marconi: fix: force exception content to be text_type https://review.openstack.org/49399 | 18:34 |
| *** amitgandhi has joined #openstack-marconi | 18:35 | |
| alcabrera | zyuan: awesome! | 18:36 |
| zyuan | thanks | 18:37 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: add logging to proxy https://review.openstack.org/49418 | 19:03 |
| *** kgriffs_afk is now known as kgriffs | 19:11 | |
| alcabrera | kgriffs: o/ | 19:13 |
| kgriffs | hi | 19:15 |
| kgriffs | finally got my passport application submitted | 19:15 |
| * kgriffs is very relieved to have that done | 19:16 | |
| alcabrera | kgriffs: awesome. I did mine a few years ago. Was the process smooth for you? | 19:16 |
| * alcabrera remembers kgriffs was aiming to go to HKG | 19:16 | |
| malini | kgriffs: right in the middle of govt shutdown :( | 19:16 |
| kgriffs | yeah, except i forgot to bring a photocopy of my license, so I had to run over to FedEx copy center real quick | 19:16 |
| kgriffs | malini: apparently, passports are an "essential" service | 19:17 |
| kgriffs | :D | 19:17 |
| malini | yayyy!! | 19:17 |
| alcabrera | awesome! | 19:17 |
| alcabrera | close call there | 19:17 |
| zyuan | kgriffs: why the queue name checking need to be applied to all requests? | 19:17 |
| kgriffs | because all input must be validated. it's one of the most common way crackers break apps. | 19:19 |
| zyuan | how it can break apps since queue is jue not there? | 19:20 |
| kgriffs | it is included in the query string, is it not? | 19:20 |
| kgriffs | to the DB? | 19:20 |
| zyuan | queue name? | 19:20 |
| kgriffs | right | 19:20 |
| zyuan | the queue name is in uri | 19:20 |
| kgriffs | yes, but we pull it out and then pass it directly into the storage driver | 19:21 |
| kgriffs | which then uses it to query the DB | 19:21 |
| zyuan | and? | 19:21 |
| zyuan | all the queries will fail if there is no such queue | 19:21 |
| kgriffs | and there is a long, sordid history of security breaches caused by bugs and whatnot in databases | 19:21 |
| * ametts hates it when I get whatnot in my databases | 19:22 | |
| kgriffs | given that (a) there is a risk in NOT validating, and (b) the validation is cheap, it seems like an obvious choice | 19:22 |
| alcabrera | ametts: lol | 19:22 |
| kgriffs | ametts: you should try it on toast. It's delicious. | 19:22 |
| zyuan | sorry, i can't see any risk here... | 19:22 |
| zyuan | ...and i don't like things just because they are cheap. i like things which are correct. | 19:23 |
| kgriffs | can you guarantee, 100%, that every storage driver we write, every DB library that those drivers use, and every backend system they talk to is bug free and there is no way to trigger an exploit via a well-crafted string? | 19:24 |
| zyuan | no matter how well-crafted they are, they have to be encoded and decoded by python... | 19:25 |
| zyuan | before goes to DB | 19:25 |
| kgriffs | how do you define "correct" application security? | 19:26 |
| amitgandhi | what validation are we doing on the queue_name - character limitations? | 19:28 |
| kgriffs | character limitations and length | 19:28 |
| zyuan | as same as its logic meaning | 19:28 |
| amitgandhi | [a-zA-Z0-9] | 19:28 |
| kgriffs | zyuan: security is all about balancing risk with performance and usability | 19:28 |
| kgriffs | there is no such thing as a perfectly secure system | 19:28 |
| kgriffs | unless it is one that doesn't do anything | 19:29 |
| zyuan | of course you can't prove a system to be secure, but i also don't want to assume some part is not secure | 19:29 |
| kgriffs | sometimes you do have to assume things are not secure, and assign a risk level to it | 19:29 |
| kgriffs | particularly, any time you cross a trust boundary | 19:30 |
| zyuan | you need a reason to argue whether it's secure or not at least | 19:30 |
| kgriffs | and I, for one, do not trust databases | 19:30 |
| zyuan | i trust python. it's not reached to db yet. | 19:31 |
| * flaper87 doesn't trust databases either | 19:31 | |
| flaper87 | brb | 19:31 |
| kgriffs | zyuan: the reason is the history of DB security breaches and programmer bugs in general. If you want to spend 6 months rigorously evaluating the security of the current version of MongoDB and MySQL, the python runtime, etc., be my guest. But it simply isn't worth trying to save a few nanoseconds. | 19:32 |
| kgriffs | this is why it is an industry best practice to ALWAYS validate ALL input | 19:32 |
| kgriffs | Many people also advocate validating your output, although I'm not convinced that is necessary in all cases. | 19:32 |
| alcabrera | kgriffs: +1 | 19:33 |
| zyuan | it's equivient to being validated... because we done decoding | 19:33 |
| zyuan | the input is not sent directly to db | 19:33 |
| zyuan | and db can accept an unicode | 19:34 |
| zyuan | any* | 19:34 |
| kgriffs | are you confident enough in that hypothesis to risk even one security breach? | 19:35 |
| zyuan | i don't. but i think this is something we can't control, and performing input validation at this layer does not help, IMHO. | 19:36 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Add prepare_request function to requests.py https://review.openstack.org/48669 | 19:37 |
| kgriffs | btw, if we ever want to be PCI compliant (something users have asked for) we must rigorously validate all input. | 19:37 |
| kgriffs | given that always-on validation doesn't hurt anything, and it *may* help (TBD), it seems only logical to put it in place. | 19:39 |
| kgriffs | anyway, that is my $0.02 | 19:43 |
| * ametts thinks kgriffs writes a lot for two cents. Too bad we don't pay him by the word. | 19:44 | |
| alcabrera | kgriffs: whenever you get a free moment, there are three proxy patches ready to merge and 3 pending review (PATCH partitions, oslo.cache bug fix, add logging). | 19:45 |
| kgriffs | ok | 19:46 |
| kgriffs | I am typing this in between eating noodles. should be able to get into review mode shortly | 19:46 |
| alcabrera | kgriffs: enjoy the noodles! :D | 19:46 |
| kgriffs | thanks! | 19:47 |
| kgriffs | never had vietnamese before | 19:47 |
| kgriffs | so far so good | 19:47 |
| alcabrera | sounds delicious | 19:47 |
| zyuan | what? github down again? | 19:53 |
| *** kgriffs is now known as kgriffs_afk | 19:53 | |
| alcabrera | zyuan: likely - status.hithub.com says issues are being investigated | 19:55 |
| alcabrera | *github | 19:55 |
| zyuan | alcabrera: i found that proxy storage already used list; just the restful resource did something different -_- | 19:57 |
| alcabrera | zyuan: hehe, that sounds about right. | 19:57 |
| *** kgriffs_afk is now known as kgriffs | 20:01 | |
| *** malini has left #openstack-marconi | 20:04 | |
| *** tvb has quit IRC | 20:06 | |
| flaper87 | LOL http://s.f87.me/1aLXkdw | 20:06 |
| alcabrera | flaper87: lol... poor, valiant souls. Also, total_hours_wasted - nice metadata. ;) | 20:07 |
| zyuan | alcabrera: i just noticed we had no tech talk today | 20:08 |
| alcabrera | zyuan: yup. I haven't had any free time to gather together thoughts on tech cinemas or tech talks. :( | 20:09 |
| alcabrera | zyuan: I need a backup coordinator. :P | 20:10 |
| zyuan | O_O | 20:10 |
| zyuan | is there any plan for next Wed? | 20:10 |
| flaper87 | kgriffs: https://review.openstack.org/#/c/48669/ | 20:17 |
| flaper87 | kgriffs: addressed your comments there | 20:17 |
| kgriffs | thanks | 20:18 |
| kgriffs | so, re. operation | 20:18 |
| kgriffs | enum vs. strings? | 20:18 |
| kgriffs | (what are your thoughts) | 20:18 |
| alcabrera | kgriffs: enums are easier to use, IMO, for defining a set of allowed operations. They're also more lightweight, AFAIK, than strings. | 20:20 |
| alcabrera | kgriffs: they also have type guarantees in py3 (enum == int -> TypeError) | 20:21 |
| kgriffs | my other question was, should we follow the REST architectural style there? HTTP isn't required | 20:21 |
| flaper87 | kgriffs: that's the idea so far. Re string / enum, that's a bit unclear right now, I'd prefer enums TBH but, it's hard to say right now since that code is not being used yet but required to be there already | 20:22 |
| kgriffs | if so, then we may want to have two params: verb and resource; but I think that may make the code a little messier | 20:22 |
| flaper87 | dunno if that makes sense | 20:22 |
| flaper87 | The general idea is to have all that unpacked and let the transport pack the request | 20:22 |
| kgriffs | flaper87: I'm fine merging this patch as-is, just want to get everyone thinking about ways to refine it later | 20:23 |
| flaper87 | like: Put the headers where they should go, params where params belong to, etc | 20:23 |
| kgriffs | I kinda like the enum idea | 20:23 |
| kgriffs | flaper87: gotchya | 20:23 |
| flaper87 | kgriffs: indeed, that's a alcabrera suggestion (enums, I mean) | 20:23 |
| flaper87 | HTTP is the most fragmented protocol I can think of right now | 20:24 |
| flaper87 | zmq will pack everything in 1 msg | 20:24 |
| flaper87 | (most likely it'll keep things separated anyway) | 20:24 |
| kgriffs | have you checked out nanomsg? | 20:24 |
| flaper87 | kgriffs: yup | 20:24 |
| flaper87 | kgriffs: It sounds really amazing | 20:24 |
| flaper87 | already tested on a embedded device | 20:24 |
| flaper87 | :D | 20:24 |
| kgriffs | yeah. something to keep an eye on | 20:24 |
| alcabrera | flaper87, kgriffs: do we have a client issues page lying around, or do we want to start filing bug reports for ideas? | 20:26 |
| flaper87 | alcabrera: we have bps and bugs for the client | 20:27 |
| alcabrera | Namely, the two that you just brought up, kgriffs, as well as the client auth session caching idea. | 20:27 |
| flaper87 | but I think I misunderstood your question | 20:27 |
| alcabrera | flaper87: perfect, that's what I was asking about. :) | 20:27 |
| flaper87 | alcabrera: https://bugs.launchpad.net/python-marconiclient | 20:27 |
| flaper87 | alcabrera: https://blueprints.launchpad.net/python-marconiclient | 20:27 |
| alcabrera | flaper87: could you create bps/bug reports for the ideas brought up in the client discussions just now? | 20:27 |
| flaper87 | alcabrera: yes, I'll do that | 20:28 |
| alcabrera | flaper87: awesome! | 20:28 |
| alcabrera | We can close them later if find better ways of doing things. | 20:28 |
| alcabrera | **if we find | 20:28 |
| *** jdaggett has joined #openstack-marconi | 20:32 | |
| kgriffs | flaper87, alcabrera: ^^^ Only change it from using iteritems to items per alcabrera's feedback | 20:49 |
| kgriffs | s/it/is | 20:49 |
| kgriffs | oops | 20:49 |
| kgriffs | just noticed that my future self snuck in hear again | 20:49 |
| alcabrera | lol | 20:49 |
| kgriffs | it is soooo annoying to be a quantum being sometimes. | 20:50 |
| kgriffs | and why can't they ever speel things rite? | 20:50 |
| alcabrera | I'm headed out for the night, guys. | 20:50 |
| kgriffs | kk | 20:51 |
| kgriffs | take care | 20:51 |
| kgriffs | or luck | 20:51 |
| kgriffs | or | 20:51 |
| flaper87 | alcabrera: take care | 20:51 |
| kgriffs | whatever you want to take | 20:51 |
| alcabrera | lol | 20:51 |
| alcabrera | night~ :D | 20:51 |
| *** alcabrera has left #openstack-marconi | 20:51 | |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Add prepare_request function to requests.py https://review.openstack.org/48669 | 20:57 |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: fix(wsgi): Non-ASCII characters in the queue name trigger 500 status https://review.openstack.org/48776 | 21:04 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: async forwarding https://review.openstack.org/49441 | 21:04 |
| *** malini has joined #openstack-marconi | 21:12 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: fix(wsgi): Non-ASCII characters in the queue name trigger 500 status https://review.openstack.org/48776 | 21:22 |
| openstackgerrit | A change was merged to openstack/marconi: proxy: mirror structure of marconi queues + bootstrap https://review.openstack.org/47642 | 21:26 |
| openstackgerrit | A change was merged to openstack/marconi: feat: separate config for queues and proxy https://review.openstack.org/48333 | 21:29 |
| openstackgerrit | A change was merged to openstack/marconi: feat: split proxy API into admin and public apps https://review.openstack.org/48495 | 21:31 |
| flaper87 | Alej will be very happy tomorrow :D | 21:32 |
| kgriffs | yep | 21:35 |
| kgriffs | :D | 21:35 |
| kgriffs | hmm | 21:36 |
| kgriffs | https://review.openstack.org/#/c/49262/3/requirements.txt | 21:36 |
| kgriffs | jsonschema will have to go into global reqs then | 21:36 |
| flaper87 | kgriffs: it should be already | 21:37 |
| flaper87 | it's being used in glance | 21:37 |
| kgriffs | oh, that would be sweet | 21:37 |
| * kgriffs checks | 21:37 | |
| flaper87 | plus, pep8 checks would've failed otherwise | 21:38 |
| kgriffs | yep, it's there | 21:38 |
| flaper87 | or gate tests, whatever they're called | 21:38 |
| kgriffs | oh, right | 21:38 |
| * kgriffs smacks forhead | 21:38 | |
| * kgriffs goes to take his meds | 21:38 | |
| * flaper87 hands some extra meds to kgriffs | 21:38 | |
| openstackgerrit | A change was merged to openstack/marconi: feat (proxy/admin): allow partition modifications https://review.openstack.org/49262 | 21:47 |
| openstackgerrit | A change was merged to openstack/marconi: fix: encode keys before caching https://review.openstack.org/49388 | 21:50 |
| *** jcru has quit IRC | 22:03 | |
| *** jdaggett1 has joined #openstack-marconi | 22:03 | |
| *** jdaggett has quit IRC | 22:06 | |
| *** flaper87 is now known as flaper87|afk | 22:24 | |
| *** tedross has quit IRC | 22:30 | |
| *** malini is now known as malini_afk | 22:31 | |
| *** amitgandhi has quit IRC | 22:34 | |
| *** mpanetta has quit IRC | 22:35 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: chore: Rename MONGODB_TEST_LIVE to MARCONI_TEST_MONGODB https://review.openstack.org/49451 | 22:35 |
| *** oz_akan_ has quit IRC | 22:37 | |
| *** kgriffs is now known as kgriffs_afk | 22:49 | |
| *** jergerber has quit IRC | 23:31 | |
| *** oz_akan_ has joined #openstack-marconi | 23:48 | |
| *** oz_akan_ has quit IRC | 23:53 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!