| *** oz_akan_ has quit IRC | 00:00 | |
| openstackgerrit | Zhihao Yuan proposed a change to stackforge/marconi: feat(mongo): use UNIX timestamp instead of datetime https://review.openstack.org/45760 | 00:08 |
|---|---|---|
| *** oz_akan_ has joined #openstack-marconi | 00:12 | |
| *** oz_akan_ has quit IRC | 00:16 | |
| *** nosnos has joined #openstack-marconi | 00:40 | |
| *** oz_akan_ has joined #openstack-marconi | 01:15 | |
| *** malini_afk is now known as malini | 02:02 | |
| *** malini is now known as malini_afk | 02:16 | |
| *** malini_afk is now known as malini | 02:17 | |
| *** oz_akan_ has quit IRC | 02:54 | |
| *** malini is now known as malini_afk | 02:54 | |
| *** oz_akan_ has joined #openstack-marconi | 02:54 | |
| *** oz_akan_ has quit IRC | 02:59 | |
| *** ayoung has quit IRC | 03:57 | |
| *** oz_akan_ has joined #openstack-marconi | 04:05 | |
| *** oz_akan_ has quit IRC | 04:10 | |
| *** nosnos has quit IRC | 05:35 | |
| *** nosnos has joined #openstack-marconi | 05:36 | |
| *** notmyname has quit IRC | 06:05 | |
| *** notmyname has joined #openstack-marconi | 06:08 | |
| *** russellb has quit IRC | 06:12 | |
| *** russellb has joined #openstack-marconi | 06:21 | |
| *** flaper87|afk is now known as flaper87 | 07:14 | |
| *** ykaplan has joined #openstack-marconi | 07:23 | |
| *** nosnos has quit IRC | 07:52 | |
| flaper87 | knock knock | 08:11 |
| *** russell_h has quit IRC | 09:11 | |
| *** torgomatic has quit IRC | 09:11 | |
| *** gleicon__ has quit IRC | 09:11 | |
| *** key4 has quit IRC | 09:11 | |
| *** whenry has quit IRC | 09:11 | |
| *** briancline has quit IRC | 09:11 | |
| *** ykaplan has quit IRC | 09:14 | |
| *** whenry has joined #openstack-marconi | 09:17 | |
| *** briancline has joined #openstack-marconi | 09:17 | |
| *** russell_h has joined #openstack-marconi | 09:20 | |
| *** torgomatic has joined #openstack-marconi | 09:20 | |
| *** gleicon__ has joined #openstack-marconi | 09:20 | |
| *** key4 has joined #openstack-marconi | 09:20 | |
| *** ykaplan has joined #openstack-marconi | 09:27 | |
| openstackgerrit | Flavio Percoco proposed a change to stackforge/marconi: Move functional tests into wsgi/v1 https://review.openstack.org/45423 | 09:48 |
| openstackgerrit | Flavio Percoco proposed a change to stackforge/marconi: Implement small http client for tests https://review.openstack.org/45267 | 09:48 |
| openstackgerrit | Flavio Percoco proposed a change to stackforge/marconi: Implement embedded marconi-server execution https://review.openstack.org/44749 | 09:48 |
| openstackgerrit | Flavio Percoco proposed a change to stackforge/marconi: Run functional tests under tox https://review.openstack.org/44723 | 09:48 |
| *** oz_akan_ has joined #openstack-marconi | 11:17 | |
| * flaper87 shakes #openstack-marconi and wakes up Atlanta guys | 11:19 | |
| flaper87 | http://openstackreactions.enovance.com/2013/09/when-your-project-enters-incubation/ | 11:36 |
| *** tedross has joined #openstack-marconi | 11:40 | |
| EmilienM | flaper87: excellent one ^ | 11:46 |
| flaper87 | :D | 11:47 |
| flaper87 | I recently passed through that | 11:48 |
| *** oz_akan_ has quit IRC | 11:53 | |
| flaper87 | toc toc toc | 13:09 |
| al-maisan | hello!? anybody home? | 13:09 |
| al-maisan | :) | 13:09 |
| flaper87 | ding dong | 13:10 |
| *** zyuan_ has joined #openstack-marconi | 13:13 | |
| *** amitgandhi has joined #openstack-marconi | 13:13 | |
| amitgandhi | morning | 13:15 |
| flaper87 | goooood morning | 13:17 |
| *** malini_afk is now known as malini | 13:19 | |
| amitgandhi | flaper87: do you recall why we decided to use "X-Project-ID" instead of tenantid semantics? iirc it was the new openstack way but all i can find is stuff about dropping it from the URL, and nothing about projectid vs tenantid other than projectid was the old term and they are interchangeable | 13:20 |
| amitgandhi | eg: http://docs.openstack.org/trunk/openstack-compute/admin/content/users-and-projects.html | 13:21 |
| flaper87 | amitgandhi: yeah, it's because keystone is now using project and it will be removed from the url | 13:25 |
| flaper87 | tenantid is the old name | 13:26 |
| flaper87 | Keystone now knows about projects | 13:26 |
| *** ayoung has joined #openstack-marconi | 13:30 | |
| amitgandhi | is that in v3 of keystone | 13:32 |
| amitgandhi | ugh why is that github private? | 13:32 |
| flaper87 | amitgandhi: GH private? | 13:32 |
| flaper87 | amitgandhi: I think is v2 and 3 | 13:33 |
| flaper87 | ayoung: ping | 13:33 |
| amitgandhi | found it: https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3.md | 13:34 |
| flaper87 | amitgandhi: ah, then it's version 3 | 13:35 |
| flaper87 | :D | 13:35 |
| amitgandhi | yeh v2 is still tenant based | 13:36 |
| openstackgerrit | Flavio Percoco proposed a change to stackforge/marconi: Use format instead of % for string formatting https://review.openstack.org/45877 | 13:37 |
| flaper87 | I remember we decided to support Project right away | 13:37 |
| flaper87 | and to not have it in the URL as other projects | 13:37 |
| flaper87 | plus, IIRC, other projects will move away from there | 13:37 |
| flaper87 | malini: good morning | 13:38 |
| flaper87 | :) | 13:38 |
| flaper87 | malini: how are you doing? | 13:38 |
| amitgandhi | flaper87: yeh i found lots of docs about dropping it from the url. Was just confused about why it was now referred to as project instead of tenant and was being asked about it. i think that v3 spec documents it for me. thanks | 13:39 |
| flaper87 | amitgandhi: no worries | 13:40 |
| flaper87 | :) | 13:40 |
| flaper87 | malini: as promissed https://review.openstack.org/#/c/45877/ | 13:41 |
| flaper87 | will cppcabrera be around today? | 13:41 |
| ayoung | flaper87, yeah whats up? | 13:42 |
| flaper87 | malini: I also addressed your comment in the other patch: http://s.f87.me/154ODdT | 13:42 |
| flaper87 | ayoung: good morning, amitgandhi and I had a keystone doubt but I guess it's clear now | 13:43 |
| malini | flaper87: thanks!! will lookat it now | 13:43 |
| flaper87 | ayoung: we were wondering wether project was a v2 thing or v3 | 13:43 |
| malini | cppcabrera is in, shud be online soon | 13:43 |
| ayoung | the term was origianlly projects back in Nova. Someone got ambitious and decided to try for multitenancy without really implementing it, and renamed them to tenants in Keystone. We made the decision to switch back to using projects instead of misleading our users. Painful transition. | 13:44 |
| *** oz_akan_ has joined #openstack-marconi | 13:44 | |
| ayoung | I think all of the APIs in V2 take tenant and V3 take project. I'd like to say that you can replace tenant with project everywhere, but that I am not so certain of | 13:45 |
| flaper87 | ayoung: lol at "try multitenancy w/o implementing it" | 13:45 |
| malini | flaper87: do we need to update the hacking.rst for https://review.openstack.org/#/c/45877/ ? | 13:45 |
| flaper87 | ayoung: we're using project everywhere | 13:45 |
| ayoung | flaper87, yeah, multitenancy is hard, and requires a degree of information hiding that we didn't realy get until we had domains | 13:45 |
| flaper87 | malini: not until we replace all % in Marconi, I guess | 13:46 |
| amitgandhi | ayoung: cool thanks. Thats what i needed to know =) | 13:46 |
| flaper87 | malini: ah no, well, it makes sense | 13:46 |
| flaper87 | I'l update it | 13:46 |
| *** cppcabrera has joined #openstack-marconi | 13:46 | |
| flaper87 | cppcabrera: THERE YOU AREEEEEEEEEEEEEEEEEE | 13:46 |
| flaper87 | cppcabrera: GOOOD MORNING! | 13:46 |
| *** oz_akan_ has quit IRC | 13:47 | |
| *** oz_akan_ has joined #openstack-marconi | 13:48 | |
| flaper87 | malini: done | 13:49 |
| malini | thanks flaper87!! | 13:50 |
| openstackgerrit | Flavio Percoco proposed a change to stackforge/marconi: Use format instead of % for string formatting https://review.openstack.org/45877 | 13:50 |
| flaper87 | I mean, done ^ | 13:50 |
| flaper87 | :) | 13:50 |
| * flaper87 loves openstackgerrit | 13:50 | |
| flaper87 | cppcabrera: http://s.f87.me/154ODdT :) | 13:51 |
| cppcabrera | Morning, guys! :D | 13:53 |
| cppcabrera | There's a lot of reviews to get to. That'll ve one of the first things I'll tackle today. :) | 13:54 |
| cppcabrera | *be | 13:54 |
| cppcabrera | morning, flaper87! | 13:55 |
| cppcabrera | Thankfully, I was able to make it through my email backlog yesterday (~200) :P | 13:55 |
| flaper87 | cppcabrera: ahh, good news then :D | 13:56 |
| * flaper87 sits and waits for openstackgerrit to announce merged patches | 13:56 | |
| cppcabrera | What prompted the move from %-style formating to '.format()', out of curiosity? I've been on the fence about this one for some time. | 13:59 |
| cppcabrera | (flaper87 ^^) | 13:59 |
| flaper87 | cppcabrera: actually, you! :D | 14:00 |
| flaper87 | I refactored some tests and malini pointed out that you mentioned about using format instead of % | 14:00 |
| flaper87 | or something like that | 14:01 |
| malini | uh oh..I thought cppcabrera LOVED .format | 14:01 |
| cppcabrera | Haha, I do, malini. :P | 14:01 |
| flaper87 | it makes sense after all since it's the new syntax and forward compatible. | 14:01 |
| flaper87 | so, why not starting now? | 14:02 |
| flaper87 | :D | 14:02 |
| malini | cppcabrera: thanks for falling over the fence! | 14:02 |
| cppcabrera | +1 flaper87 | 14:02 |
| cppcabrera | malini, lol | 14:02 |
| flaper87 | that's the first patch, there will be others changing Marconi's code base | 14:02 |
| cppcabrera | % syntax is a bit faster than .format, but that shouldn't be a big deal since we're not .format-ting in the critical path. | 14:02 |
| flaper87 | cppcabrera: yeah | 14:03 |
| cppcabrera | % syntax also doesn'tt seem like it's going anywhere, so I'm not sure compatibility is a reason to prefer one over the other. | 14:03 |
| cppcabrera | .format can be much more readable, and it has certain maintainability benefits (reorder args, etc.) | 14:03 |
| cppcabrera | I'm still happy about it, I was just curious if there was another reason beyond my thoughts, hehe. :) | 14:04 |
| flaper87 | cppcabrera: mmh, isn't it going anywhere ? mmh, I thought that was the plan | 14:05 |
| flaper87 | well, I'm sure it's recomended to use .format instead of % | 14:05 |
| cppcabrera | I thought so, too, that .format was meant to replace %. | 14:06 |
| cppcabrera | I wasn't able to find any references to indicate that, though. | 14:06 |
| cppcabrera | All I've found is PEP 3101 that introduces .format | 14:07 |
| cppcabrera | "intended as a replacement for the existing '%' string formatting operator..." - PEP 3101 | 14:07 |
| flaper87 | cppcabrera: lol, it's right there in the abstract | 14:09 |
| flaper87 | :D | 14:09 |
| cppcabrera | Hehe, .format is the way of the future indeed. | 14:09 |
| cppcabrera | Seeing the tox patch for the tests, that reminds me that... | 14:11 |
| cppcabrera | Requests 2.0 is being actively developed: http://todd328.wordpress.com/2013/09/09/the-changelog-requests-2-0-coming-soon-with-proxy-support-and-bug-fixes/ | 14:11 |
| flaper87 | cppcabrera: +1 | 14:13 |
| cppcabrera | flaper87 - thanks for all your work on refactoring the test suite! | 14:14 |
| cppcabrera | Being able to tox my way through functional tests makes me very happy. | 14:14 |
| flaper87 | cppcabrera: :D | 14:14 |
| cppcabrera | Oooh, +1 for http client. | 14:15 |
| zyuan | ".format can be more maintainable than %-style, though it is slightly slower. " | 14:17 |
| zyuan | all replacements in python are slower than the old ones | 14:17 |
| cppcabrera | Hey, hey, zyuan. :) | 14:19 |
| flaper87 | cppcabrera: you know we just cheated here, don't you? https://review.openstack.org/#/c/37140/ | 14:22 |
| flaper87 | :D | 14:22 |
| flaper87 | but hey, sssh, don't say anything to key4 | 14:23 |
| flaper87 | kgriffs_afk: | 14:23 |
| flaper87 | cppcabrera: btw, that one is blocked by this one https://review.openstack.org/#/c/45439/ | 14:23 |
| cppcabrera | Haha, we did cheat on that one. :P | 14:24 |
| flaper87 | cppcabrera: we need to talk a bit about the client | 14:27 |
| flaper87 | next steps, the API etc | 14:27 |
| cppcabrera | Ahh, alright. | 14:28 |
| cppcabrera | Yeah! | 14:28 |
| cppcabrera | So we can get that moving. | 14:28 |
| * cppcabrera is having a good time reviewing all the pending patches | 14:29 | |
| * flaper87 won't bother cppcabrera until he finishes reviewing | 14:29 | |
| cppcabrera | I think I'll take another 15 minutes. How much longer will you be around today, flaper87? :) | 14:29 |
| flaper87 | I still got time, at least 2 more hours before dinner | 14:30 |
| cppcabrera | cool, cool. I don't want to miss out on client discussions since I can always review a little later. | 14:30 |
| cppcabrera | Oooohh, the author of six merged a patch to add py2/3-compatible metaclass handling: http://pythonhosted.org/six/#six.add_metaclass | 14:31 |
| cppcabrera | That came to mind while looking at the marconi-testing hosted server patch. | 14:32 |
| cppcabrera | @add_metaclass(abc.ABCMeta) | 14:32 |
| malini | flaper87: does oslo config create the environment variable MARCONI_TESTS_CONFIGS_DIR ? | 14:33 |
| malini | I am trying to run the new tests & seeing some weird stuff | 14:33 |
| malini | I have two MARCONI_TESTS_CONFIGS_DIR | 14:34 |
| flaper87 | malini: that variable is created in tests/__init__.py | 14:34 |
| flaper87 | malini: how are you running tests? | 14:34 |
| malini | nosetests -s -v | 14:34 |
| flaper87 | why not using tox? | 14:35 |
| flaper87 | well, it won't change anythiong | 14:35 |
| flaper87 | anything :P | 14:35 |
| flaper87 | tox uses nosetest | 14:35 |
| flaper87 | :D | 14:35 |
| flaper87 | what patch are you using? | 14:35 |
| malini | '"Use oslo.config for functional tests"' | 14:36 |
| malini | commit 5a12dbf5fcd94343c63ba3afb50b94b1c4a4eaf0 | 14:36 |
| malini | The tests point to 'MARCONI_TESTS_CONFIGS_DIR': '/Users/malini.kamalambal/marconi/tests/etc' | 14:37 |
| malini | Which is not the 'MARCONI_TESTS_CONFIGS_DIR', I created | 14:37 |
| malini | This is how my env variable look like,when running the test | 14:38 |
| malini | http://paste.openstack.org/show/46472/ | 14:38 |
| *** ykaplan has quit IRC | 14:38 | |
| *** ykaplan has joined #openstack-marconi | 14:39 | |
| flaper87 | cppcabrera: replied :) | 14:41 |
| malini | flaper87: got it now :) | 14:42 |
| flaper87 | malini: what was it? | 14:42 |
| flaper87 | (sorry, I was looking at it now) | 14:43 |
| malini | we shud just set the env variable , if we dont find it, rt? | 14:43 |
| malini | https://github.com/stackforge/marconi/blob/master/tests/__init__.py | 14:43 |
| flaper87 | malini: we do set it | 14:43 |
| malini | matbe this 'os.environ.setdefault("MARCONI_TESTS_DIR", tests_dir)' needs to go inside the if loop ? | 14:43 |
| flaper87 | malini: this sets the variable https://github.com/stackforge/marconi/blob/master/tests/__init__.py#L22 | 14:43 |
| malini | problem with tht is it always looks inside the tests/etc directory & never uses my specified path | 14:44 |
| flaper87 | malini: mmh, how are you setting your path ? | 14:44 |
| malini | I want to keep my default config in ~/.marconi or wherever, & not have it overwritten with each git checkout | 14:44 |
| flaper87 | MARCONI_TESTS_CONFIGS_DIR=/path/to/test nosetest -s -v | 14:44 |
| flaper87 | or export MARCONI_TESTS_CONFIGS_DIR=~/.marconi && nosetests -s -v | 14:45 |
| malini | yes, but I end up getting two MARCONI_TESTS_CONFIGS_DIR as in http://paste.openstack.org/raw/46472/ | 14:45 |
| malini | since it is being set in tests/__init__.py as well | 14:45 |
| flaper87 | MARCONI_TESTS_CONFIG_DIR < CONFIGS not CONFIG | 14:45 |
| flaper87 | MARCONI_TESTS_CONFIGS_DIR <- this is the right one | 14:46 |
| flaper87 | https://github.com/stackforge/marconi/blob/master/tests/__init__.py#L21 | 14:46 |
| malini | grrrr..my bad | 14:46 |
| malini | maybe I shud stop trying to get stuff working, after the lights are out | 14:47 |
| malini | I tried it y'day night & was pulling my hair out | 14:47 |
| flaper87 | malini: hehehe | 14:47 |
| cppcabrera | flaper87: +2'd (http patch) | 14:47 |
| flaper87 | cppcabrera: cooooool | 14:49 |
| cppcabrera | With that, I've reviewed all test patches. | 14:50 |
| cppcabrera | Want to talk client-side, flaper87? :) | 14:50 |
| flaper87 | cppcabrera: suuure | 14:51 |
| flaper87 | so, I know you had some thoughts about it | 14:52 |
| flaper87 | whether to use verbs, etc | 14:52 |
| cppcabrera | Yup | 14:52 |
| flaper87 | I started working on the auth part (keystone with support for other auths) | 14:52 |
| cppcabrera | +1 | 14:52 |
| cppcabrera | That auth part is the main blocker to getting a reliable connection class going. | 14:53 |
| flaper87 | it's not complete yet but it will be soon (I've got to write tests, yet again :P) | 14:53 |
| flaper87 | cppcabrera: want to take a look at the work so far? | 14:53 |
| cppcabrera | Tests - haha. marconi-proxy will get those todat. :P | 14:53 |
| flaper87 | I can submit it as WIP | 14:53 |
| cppcabrera | *today | 14:53 |
| cppcabrera | Yeah, that'd be cool. | 14:53 |
| * flaper87 does that | 14:53 | |
| cppcabrera | I'll review what you have and see if I catch anything that stands out. | 14:53 |
| cppcabrera | Now, re: verbs | 14:54 |
| cppcabrera | It goes against the openstack client designs I've seen up to this point. | 14:54 |
| cppcabrera | Err, hmm... | 14:54 |
| cppcabrera | Wait, verbs is the Openstack way - nouns are the way that's more intuitive to me. >.> | 14:55 |
| cppcabrera | connection.queues(...) (nouns) vs. connection.list(...) | 14:55 |
| flaper87 | cppcabrera: added you here: https://review.openstack.org/#/c/45887/ | 14:55 |
| cppcabrera | queue.messages() vs. queue.list() | 14:55 |
| cppcabrera | woot | 14:55 |
| flaper87 | it's a Draft, I didn't want to trigger jenkins for this patch | 14:55 |
| cppcabrera | queue.claim() vs queue.post_claim() (verb( | 14:55 |
| cppcabrera | )) | 14:55 |
| cppcabrera | np - draft is cool. :) | 14:56 |
| cppcabrera | woot - six.add_metaclass made it's way in there. _1 | 14:56 |
| cppcabrera | +1 | 14:56 |
| flaper87 | I prefer nouns, TBH. First question is: Would that make it difficult to support API changes? | 14:57 |
| flaper87 | cppcabrera: :) | 14:57 |
| flaper87 | I want to keep the client Py3K compliant | 14:57 |
| flaper87 | so, no breaking patches will get in | 14:57 |
| flaper87 | re API changes: Changes from v1 to v2 or v1 to v1.1 | 14:58 |
| cppcabrera | Nouns should be okay, as long as the meaning of the nouns and the interactions between them don't change (too much). | 14:59 |
| cppcabrera | For example, v2.0 has routing/exchanges planned, IIRC | 14:59 |
| cppcabrera | That would allow posting a message to multiple queues. | 14:59 |
| malini | But verbs are more explicit | 14:59 |
| flaper87 | another question: What would queues creation, for example, look like? | 15:00 |
| flaper87 | connection.queues() (lists them, I presume) | 15:00 |
| cppcabrera | connection.create_queue -> queue | 15:01 |
| malini | connection.list_queues() makes more sense to me than connection.queues() | 15:01 |
| * flaper87 prefers .queues | 15:01 | |
| flaper87 | :P | 15:01 |
| malini | I went thru https://github.com/boto/boto/blob/develop/docs/BotoCheatSheet.pdf & loved the simplcity | 15:01 |
| flaper87 | malini: that's a good reference | 15:02 |
| *** ykaplan has quit IRC | 15:02 | |
| malini | everything was obvious as to what it does | 15:02 |
| flaper87 | thanks for bringing it up | 15:02 |
| oz_akan_ | I like boto as well | 15:02 |
| *** pycabrera has joined #openstack-marconi | 15:02 | |
| pycabrera | disconnected. :( | 15:02 |
| malini | when we were initially brainstorming the client, thts what I used as a reference to define our methods | 15:02 |
| * pycabrera goes to find eavesdrop | 15:03 | |
| malini | that = boto | 15:03 |
| pycabrera | There, I'm caught up. :) | 15:03 |
| flaper87 | yeah, it looks simple, well, TBH, .queues() and .queue() are simple as well | 15:04 |
| flaper87 | get_all_queues is way too verbose | 15:04 |
| flaper87 | IMHO | 15:04 |
| flaper87 | and queue.write() <- erm, no :P | 15:04 |
| malini | simple as in obvious on what it does | 15:04 |
| flaper87 | malini: yup, that's what I mean | 15:04 |
| malini | sorry for the wrong use of words | 15:04 |
| pycabrera | agreed - I love short method names as long as it's obvious what they do, or that the intuition can be quickly gained. | 15:04 |
| malini | get_all_queues is verbose. −1 for that | 15:04 |
| malini | but get_queues ? | 15:05 |
| zyuan | iirc, pycabrera want to make an generator interface for listing queues? | 15:05 |
| malini | or list_queues() ? | 15:05 |
| zyuan | if so, i think | 15:05 |
| zyuan | for q in proj.queues(): | 15:05 |
| pycabrera | I'm fond of .queues() | 15:05 |
| zyuan | blah blah | 15:05 |
| zyuan | looks good to me | 15:06 |
| *** cppcabrera has quit IRC | 15:06 | |
| zyuan | or conn | 15:06 |
| pycabrera | zyuan: Yeah, I think a generator interface would be the most convenient. One could just filter through queues transparently while the client handled pagination. | 15:06 |
| malini | if we go with .queues() to list, how wud we have a delete queue ? | 15:06 |
| pycabrera | queue.delete() | 15:07 |
| *** pycabrera is now known as cppcabrera | 15:07 | |
| * cppcabrera got his name back | 15:07 | |
| flaper87 | connection.queue().delete() | 15:07 |
| zyuan | delete(q) | 15:07 |
| zyuan | ok, ignore me | 15:07 |
| flaper87 | :P | 15:07 |
| cppcabrera | haha, so many options. :P | 15:07 |
| malini | We want to keep it consistent across all APIs | 15:07 |
| malini | make it intuitive | 15:07 |
| flaper87 | that's also kinda how pymongo's con->db->col thing works | 15:08 |
| flaper87 | malini: yeah, that was my first question | 15:08 |
| cppcabrera | whenry mentioned he'd be happy to answer some client-dev questions for us several meeting-weeks ago. :D | 15:08 |
| flaper87 | whenry: yo yo, time for you to chime in | 15:08 |
| cppcabrera | I think we could use a little more perspective on the client API question, especially if we're going to branch from the current convention. | 15:09 |
| whenry | cppcabrera, flaper87 wassup? am I on the wrong call? | 15:09 |
| cppcabrera | IIRC, you said you might be able to help us with marconi-client API design questions. :) | 15:10 |
| cppcabrera | whenry ^ | 15:10 |
| cppcabrera | ooh, found it: "whenry cppcabrera, pls feel free to reachout to tedross or me re client apis" | 15:11 |
| cppcabrera | So... whenever you have some time, that'd be awesome. :D | 15:12 |
| whenry | cppcabrera, ok so I don't know if you mean I need to chime in here, or on a call that is currently happening or on some future call | 15:13 |
| flaper87 | looking at how other clients are organized: https://github.com/openstack/python-ceilometerclient/tree/master/ceilometerclient | 15:13 |
| malini | for queues we cud have q.create(), q.list() , q.delete(), q.stats() etc. ? | 15:13 |
| malini | for a consistent user experience | 15:13 |
| flaper87 | whenry: here | 15:13 |
| *** ykaplan has joined #openstack-marconi | 15:13 | |
| zyuan | q.list()? | 15:13 |
| cppcabrera | yeah, all IRC, sorry about not being clear. :P | 15:13 |
| zyuan | we don't have list on single queue | 15:13 |
| cppcabrera | Yeah, that's not clear, zyuan. :/ | 15:14 |
| zyuan | we don't have create on single queue either | 15:14 |
| cppcabrera | It maps to q.messages() | 15:14 |
| flaper87 | malini: that'd be kind of moving controller's API to the client | 15:14 |
| zyuan | then call it messages... | 15:14 |
| cppcabrera | Plus, that doc is a bit dated... :( | 15:14 |
| cppcabrera | I need to update that to match the current API. | 15:14 |
| malini | cppcabrera: which doc ? | 15:14 |
| cppcabrera | The API c lient doc: https://wiki.openstack.org/wiki/Marconi/PythonClient | 15:15 |
| cppcabrera | *client | 15:15 |
| cppcabrera | malini ^ | 15:15 |
| zyuan | again, i hoe we can have marconiclient.Connection | 15:16 |
| zyuan | hope* | 15:16 |
| flaper87 | zyuan: me too, that's my preference | 15:16 |
| zyuan | wait. Client or Connection | 15:16 |
| zyuan | ? | 15:16 |
| oz_akan_ | malini: I get this Illegal division by zero at /usr/lib/tsung/bin/tsung_stats.pl line 480 | 15:17 |
| flaper87 | Connection | 15:17 |
| oz_akan_ | when I run tsung_stats.pl | 15:17 |
| oz_akan_ | do you have an idea how we can fix it? | 15:17 |
| flaper87 | which would also make more sense on other transports as well, IMHO | 15:18 |
| cppcabrera | +1 for connection | 15:18 |
| malini | oz_akan_ : which xml ? | 15:18 |
| oz_akan_ | tsung-simple1.xml | 15:18 |
| malini | let me stop by | 15:18 |
| oz_akan_ | ok | 15:18 |
| flaper87 | another question | 15:19 |
| flaper87 | or at least, something we should keep in mind | 15:19 |
| whenry | sorry guys. /me catches up | 15:20 |
| flaper87 | Ideally, libraries implementation will try to be similar (Java, Python, C, whatever), so, we should keep that in mind as well | 15:20 |
| flaper87 | I'm not saying those libraries will be identical | 15:21 |
| flaper87 | but, it'd be good if some consistency could be maintained throughout them | 15:21 |
| * whenry is on a AMQP call and is trying to catch up | 15:21 | |
| cppcabrera | +1 flaper87 - at least as consistent as the zmq bindings across languages are. They work very similarly - create a context, set up some sockets, pump data through the nets. | 15:22 |
| cppcabrera | no worries, whenry. :) | 15:22 |
| cppcabrera | So the key goal of the client should be to facilitate operations on queues, messages, and claims after establishing a connection to a marconi endpoint. | 15:23 |
| flaper87 | in my head, connection is the starting point of every operation | 15:23 |
| flaper87 | you start from a connection, get a queue, post a message | 15:24 |
| flaper87 | start connection, get a queue, claim some messages | 15:24 |
| flaper87 | and so on | 15:24 |
| flaper87 | Kombu uses a similar approach | 15:24 |
| cppcabrera | It's the same on my end. | 15:24 |
| flaper87 | connection, publisher, etc | 15:24 |
| whenry | cppcabrera, so to be clear. we are talking about queues first (not notifications/topics) | 15:25 |
| zyuan | do we offer claim as an iterface | 15:26 |
| zyuan | or just offer pub-sub | 15:26 |
| zyuan | pattern | 15:26 |
| zyuan | (i'd like pub-sub pattern) | 15:27 |
| cppcabrera | +1 whenry | 15:27 |
| zyuan | ok... queue first | 15:28 |
| cppcabrera | the client establishes a connection to marconi, grabs a queue, and can then perform operations on that queue. | 15:28 |
| cppcabrera | topics/exchanges are still a ways off. | 15:28 |
| amitgandhi | who's doing notifications/topics currently? | 15:29 |
| amitgandhi | (i mean is there a team working on notifications?) | 15:29 |
| flaper87 | amitgandhi: no one, yet. al-maisan is putting some effort on that | 15:30 |
| flaper87 | amitgandhi: but we'll discuss that next week again | 15:30 |
| amitgandhi | is there a wiki | 15:30 |
| amitgandhi | oh i missed yesterdays meeting | 15:30 |
| amitgandhi | should eavesdrop it | 15:30 |
| flaper87 | amitgandhi: nope, it's actually a separate project for now https://github.com/rackerlabs/foghorn/ | 15:31 |
| flaper87 | amitgandhi: we didn't talk about that yday | 15:31 |
| whenry | ok so I'm going to weigh in | 15:32 |
| whenry | why do we need an explicit connection? | 15:32 |
| whenry | as oppsoed to a implicit one | 15:32 |
| cppcabrera | a connection can be used to handle authentication. | 15:32 |
| cppcabrera | if authentication is needed | 15:33 |
| whenry | i.e. in a SQS don't I basically go to some browerser service to get a address namespace for my queue? | 15:33 |
| whenry | and then when using it can't I just create a sender or receiver to that and all the security etc. happen under the covers? | 15:33 |
| cppcabrera | hmm... | 15:34 |
| whenry | ie.e. when I create my sender and receiver I provide security credentials that connect and authenitcate underneath | 15:34 |
| whenry | so that it truely is simple | 15:35 |
| Alex_Gaynor | flaper87: ah, thanks, I hadn't realized | 15:35 |
| cppcabrera | I'm not very familiar with AWS's SQS, TBH. It seems that they have the connection token as a part of the URL params. | 15:35 |
| amitgandhi | flaper87: oh yeah i know about foghorn. i thought you were talking about something else | 15:35 |
| cppcabrera | "&AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE" | 15:35 |
| whenry | tedross, ^^^ | 15:35 |
| cppcabrera | As far as senders and receivers, both of those sound like types of connections. | 15:35 |
| cppcabrera | That's the simplicity of it, though. A connection should be easy to establish and it's a set-it-and-forget-it deal. | 15:36 |
| whenry | cppcabrera, does a programmer q.create (I.e. setup a new SQS q dynamically? or is that something that is done as part of a web console to "register and provision" it etc. | 15:38 |
| flaper87 | Alex_Gaynor: np :) | 15:38 |
| whenry | i.e. the web console app calls q.create | 15:38 |
| *** kgriffs_afk is now known as kgriffs | 15:38 | |
| whenry | not the q user | 15:38 |
| flaper87 | how would a implicit connection approach look like for other transports? | 15:39 |
| flaper87 | say, zmq | 15:40 |
| flaper87 | ? | 15:40 |
| cppcabrera | Ahh, I think I understand the question... queues are not provisioned - they are dynamically created/deleted. | 15:40 |
| flaper87 | in my head, Connection just holds the info to communicate w/ the server | 15:40 |
| flaper87 | but it doesn't actually establishes it | 15:40 |
| whenry | cppcabrera, ah, ok that's different than what I was assuming this was. i.e. I assumed that there was some sort of service provided by the "cloud" to provision queues for you so that it could bill etc. | 15:41 |
| whenry | cppcabrera, but your saying they could dynamically provision queues in their apps. | 15:41 |
| tedross | whenry: cppcabrera: how are queues named? Is there a user-specific namespace? | 15:42 |
| cppcabrera | yes, whenry. For example, I would create a queue 'tasks' on the fly by communicating with a marconi endpoint (PUT /v1/queues/tasks') | 15:42 |
| tedross | what if I and someone else create a queue called "work" will they be different queues or the same? | 15:42 |
| cppcabrera | That PUT request would bringthe queue 'tasks' into existence. | 15:42 |
| flaper87 | tedross: queue names are unique per project | 15:43 |
| kgriffs | cppcabrera: Yesterday I had an idea about catalog caching/replication that I wanted to run by you | 15:43 |
| cppcabrera | kgriffs: o/ | 15:43 |
| kgriffs | hi! | 15:43 |
| whenry | cppcabrera, so all this is HTTP based? or is the RESTful HTTP semantics just for provisioning ? | 15:44 |
| whenry | cppcabrera, in above example would /v1/queues/tasks be under a namespace for my application or my user account? | 15:44 |
| flaper87 | cppcabrera: w.r.t using six.metaclass, I think it would be better to replace it all around Marconi in a separate patch, does that make sense ? | 15:44 |
| cppcabrera | It's an HTTP-based API that communicates with a storage backend. Marconi doesn't provision - it's queues + messages as a service. | 15:44 |
| flaper87 | kgriffs: https://review.openstack.org/#/q/status:open+project:stackforge/marconi+branch:master+topic:refactor-system-tests,n,z | 15:45 |
| flaper87 | kgriffs: pls | 15:45 |
| cppcabrera | +1 flaper87 | 15:45 |
| whenry | cppcabrera, how would my tasks be different to your tasks? how would we bill? | 15:45 |
| kgriffs | whenry: Queues are scoped according to project (AKA tenant). That is passed in X-Project-ID | 15:45 |
| kgriffs | (header) | 15:45 |
| cppcabrera | whenry: In that example, it would be in a namespace under your user account. That namespace is determined by X-Project-Id (tenant ID) | 15:45 |
| whenry | cppcabrera, is it assume that each user gets their own "storage backend" or is that a shared cloud resource? | 15:46 |
| flaper87 | whenry: yes | 15:47 |
| cppcabrera | It's a shared resource. :) | 15:47 |
| whenry | cppcabrera, in the case that this is a messaging "broker" for example would each get their own broker? | 15:47 |
| whenry | ok shared | 15:47 |
| flaper87 | whenry: not exactly broker, they can have their own project | 15:47 |
| whenry | hmm. ok so for something like AMQP then they'd have an address like amqp://<X-Project-ID>/vi/queues/tasks | 15:48 |
| flaper87 | cppcabrera: btw, can you change your -1 :D | 15:50 |
| flaper87 | kgriffs: if you get a chance, pls review test patches | 15:50 |
| whenry | cppcabrera, I want to get back to a another question that may not seem relevant but it is. | 15:50 |
| cppcabrera | Oh, yes, flaper87. :P | 15:50 |
| kgriffs | flaper87: yep, will do | 15:50 |
| kgriffs | cppcabrera: got a minute to discuss that idea? | 15:50 |
| cppcabrera | whenry: I can't speak to the AMQP side of things. I have little experience there. | 15:50 |
| whenry | cppcabrera, do we care how this is billed? | 15:50 |
| whenry | cppcabrera, ack. I was just recording that here. | 15:51 |
| whenry | ;-) | 15:51 |
| cppcabrera | :) | 15:51 |
| cppcabrera | kgriffs: Yeah, I have a moment. :) | 15:51 |
| kgriffs | whenry: Marconi doesn't care, but operators will, obviously | 15:51 |
| cppcabrera | re: cachinhg/replication. | 15:51 |
| flaper87 | re billing, that's a long topic, I think that will happen through ceilometer but again, that's definitely a bigger topic | 15:51 |
| flaper87 | kgriffs: we were discussing whether to sue nouns or verbs in the client. We ( cppcabrera, zyuan and me) were leaning towards using nouns, as in: connection.queues(); connection.queue().post() | 15:53 |
| flaper87 | etc | 15:53 |
| * whenry needs to take a look at AWS SQS to see what they do. I would assume that their might be two types of billing in our case: i.e. one where you have kinda static queues that you payfor that you get provisioned. and one type of charge for a project/app that wants to do lots of dynamica q creation | 15:53 | |
| kgriffs | whenry: they bill per-message, also bandwidth iirc | 15:53 |
| zyuan | *also bandwidth* | 15:54 |
| zyuan | hehehehehe | 15:54 |
| kgriffs | flaper87: i like it. | 15:54 |
| kgriffs | (re nouns) | 15:54 |
| zyuan | nouns -> non-stateful | 15:54 |
| kgriffs | nouns == resources I guess | 15:54 |
| kgriffs | cppcabrera: sooo | 15:54 |
| flaper87 | kgriffs: yeah | 15:54 |
| kgriffs | here is the gedank | 15:54 |
| cppcabrera | flaper87: done (-1 -> +2) | 15:55 |
| whenry | kgriffs, but I can create qs on the fly, millions of them if I want and they don't care? | 15:55 |
| flaper87 | cppcabrera: thanks | 15:55 |
| cppcabrera | kgriffs: ready | 15:55 |
| kgriffs | whenry: they cap you on max queues iirc | 15:55 |
| kgriffs | you have to talk to support if you want the cap raised | 15:55 |
| zyuan | kgriffs: we don't have that yet. but we need. | 15:55 |
| kgriffs | i know that is the case for SNS topics, pretty sure SQS is similar | 15:55 |
| flaper87 | cppcabrera: kgriffs zyuan malini can we say we agree on using an API as that example? | 15:55 |
| whenry | kgriffs, ack good to know. I'm trying to fill in my knowledge gap here as I think about this | 15:55 |
| zyuan | which example? | 15:55 |
| flaper87 | zyuan: connection.queues()......... | 15:56 |
| flaper87 | I mean, nouns, or however we want to call them | 15:56 |
| zyuan | this +1. details? | 15:56 |
| kgriffs | also, SQS will nuke idle queues after some time period (can't remember exactly what it is off the top of my head) | 15:56 |
| kgriffs | marconi queues are super light-weight, so we care a lot less about # of queues people create | 15:56 |
| flaper87 | dunno, it'll be along those lines. | 15:56 |
| cppcabrera | +1 for nouns on my end | 15:57 |
| kgriffs | cppcabrera: so, the idea is to use mongodb all the way down | 15:57 |
| kgriffs | you have a primary | 15:57 |
| flaper87 | kgriffs: whenry re usage, I think we'll have to emit "internal" notifications for every operation | 15:57 |
| kgriffs | and then a secondary that is colocated on each proxy host | 15:57 |
| flaper87 | and let the billing system take care of the rest | 15:57 |
| kgriffs | then you don't have to worry about an extra caching layer | 15:57 |
| kgriffs | and it's super fast | 15:57 |
| cppcabrera | I see - so we let mongo's clustering support take care of replication/caching for us? | 15:58 |
| kgriffs | yeah | 15:58 |
| kgriffs | also ensure super high redundancy/durability | 15:58 |
| flaper87 | what happens if mongodb is not being used? | 15:58 |
| flaper87 | would that block users from using -proxy ? | 15:59 |
| kgriffs | flaper87: you can do same thing with MySQL | 15:59 |
| kgriffs | master and many slaves, right? | 15:59 |
| flaper87 | kgriffs: wait, I think i missed the argument | 15:59 |
| whenry | cppcabrera, kgriffs is this all assuming a DB as the messaging service??? | 15:59 |
| flaper87 | you were talking about not using redis for the catalog, right? | 15:59 |
| flaper87 | I mean, cache | 15:59 |
| kgriffs | flaper87: correct | 15:59 |
| kgriffs | just use the secondaries as read-only | 15:59 |
| flaper87 | -1 from me | 15:59 |
| kgriffs | and stick them on the same host as the proxy | 15:59 |
| flaper87 | I wouldn't rely on the storage being used | 16:00 |
| flaper87 | if we consider using some broker as storage in the future | 16:00 |
| kgriffs | can you elaborate? | 16:00 |
| flaper87 | we'll make it impossible to use the -proxy | 16:00 |
| flaper87 | unless other backends are installed | 16:00 |
| kgriffs | I think it is fine to require more than one storage system | 16:00 |
| kgriffs | it would be separate from the messages backend anyway, right? | 16:00 |
| flaper87 | i think it is fine to support having more than one storage | 16:00 |
| flaper87 | (as in, running at the same time) | 16:01 |
| kgriffs | i mean, you don't want to share the same cluster - since the catalog goes across partitions | 16:01 |
| flaper87 | but I'm not sure about requiring them | 16:01 |
| zyuan | (proxy itself is already a storage system) | 16:01 |
| kgriffs | i mean, you could do this with redis for all I care | 16:01 |
| flaper87 | cache seems something common through all storages | 16:01 |
| kgriffs | the point is, just have one primary and replicate to secondaries (AKA master and slaves) that are colocated on the proxy nodes | 16:01 |
| * whenry thinks this is a different view of messaging than I was understanding | 16:01 | |
| kgriffs | whenry: we are talking about marconi-proxy which shards queues across multiple marconi clusters | 16:02 |
| kgriffs | imho, marconi-proxy stack should be independent from marconi-server as much as possible | 16:03 |
| zyuan | ^ +1 | 16:03 |
| kgriffs | catalogues are super read-heavy | 16:03 |
| flaper87 | kgriffs: agreed but it doesn't have the same requirements | 16:03 |
| flaper87 | mmh | 16:03 |
| kgriffs | in that case, it makes sense to coopt the slaves | 16:03 |
| cppcabrera | I like the idea of taking advantage of slave/master replication for handling proxy implementation. (+1 for marconi-proxy being decoupled) | 16:03 |
| cppcabrera | Regardless of the storage engine used. | 16:03 |
| flaper87 | but what if they are using mongo shards instead of repls ? | 16:04 |
| kgriffs | does the catalog need to be sharded? | 16:04 |
| whenry | kgriffs, right but this seems very synchronous (i.e. HTTP based) | 16:04 |
| cppcabrera | I don't think so, given that it is mostly read ops. | 16:04 |
| flaper87 | kgriffs: if it is supper heavy, why not? I mean, we shouldn't be telling people how to deploy their stuff | 16:04 |
| flaper87 | (meaning mongo, mysql) | 16:04 |
| flaper87 | IMGO | 16:04 |
| flaper87 | IMHO | 16:04 |
| flaper87 | we should just use them and apply different strategies based on the way they were deployed | 16:05 |
| kgriffs | flaper87: do you think people want non-redis options for marconi-proxy? | 16:05 |
| *** whenry is now known as whenry_afk | 16:05 | |
| zyuan | if there is a better redis | 16:05 |
| zyuan | we can just replace the impl | 16:05 |
| flaper87 | zyuan: LOL | 16:05 |
| cppcabrera | whenry_afk: yeah, Marconi v1.0 is synchronous. You post meesages via HTTP (over WSGI) | 16:06 |
| kgriffs | seems like we can just ship with redis and support stevedore drivers if people want something different | 16:06 |
| kgriffs | (they can develop it themselves) | 16:06 |
| flaper87 | kgriffs: they may want something different | 16:06 |
| cppcabrera | +1 for stevedore | 16:06 |
| zyuan | they have option to do not use proxy | 16:06 |
| cppcabrera | I avoided getting to caught up on the stevedore side of things while working on the proxy, but it makes sense to leave it up to people to decide how they want to deploy. | 16:06 |
| cppcabrera | *too | 16:07 |
| zyuan | cppcabrera: LOL stevedore | 16:07 |
| flaper87 | TBH, I'd try keeping the proxy decoupled but very simple for now | 16:08 |
| zyuan | we all hope so | 16:09 |
| flaper87 | zyuan: we all have to :) | 16:09 |
| *** pycabrera has joined #openstack-marconi | 16:09 | |
| pycabrera | dc'd~ | 16:09 |
| flaper87 | anyway, that's my $0.02 | 16:09 |
| * pycabrera looks up eavesdrop | 16:10 | |
| pycabrera | in a meeting, I'll be back in a bit. | 16:11 |
| zyuan | kgriffs: https://bugs.launchpad.net/marconi/+bug/1222933 | 16:12 |
| *** pycabrera is now known as cppcabrera_afk | 16:12 | |
| *** cppcabrera has quit IRC | 16:12 | |
| zyuan | we don't have it? | 16:12 |
| zyuan | oh, that's falcon... | 16:12 |
| zyuan | we have some... | 16:13 |
| flaper87 | where did everybody go? | 16:15 |
| zyuan | ?? | 16:15 |
| flaper87 | zyuan: marconi-proxy discussion stopped suddenly | 16:16 |
| flaper87 | :D | 16:16 |
| zyuan | flaper87: https://review.openstack.org/#/c/45760/ | 16:16 |
| cppcabrera_afk | flaper87: Meeting. :( | 16:16 |
| cppcabrera_afk | on a side note, flaper87 - I'll update the client API page before the end of the day to bring back the nouns. | 16:17 |
| cppcabrera_afk | marconi-proxy testing comes first, though. :) | 16:17 |
| flaper87 | cppcabrera_afk: kk :) | 16:20 |
| flaper87 | thanks | 16:20 |
| flaper87 | zyuan: looking | 16:20 |
| *** amitgandhi1 has joined #openstack-marconi | 16:23 | |
| *** amitgandhi has quit IRC | 16:25 | |
| *** cppcabrera has joined #openstack-marconi | 16:28 | |
| *** cppcabrera_afk has quit IRC | 16:28 | |
| *** cppcabrera is now known as cppcabrera_afk | 16:29 | |
| flaper87 | brb, dinner | 16:37 |
| *** amitgandhi has joined #openstack-marconi | 16:39 | |
| *** amitgandhi1 has quit IRC | 16:39 | |
| *** ykaplan has quit IRC | 16:40 | |
| *** kgriffs is now known as kgriffs_afk | 17:05 | |
| *** cppcabrera_afk is now known as cppcabrera | 17:13 | |
| *** kgriffs_afk is now known as kgriffs | 17:14 | |
| kgriffs | malini: I'd like to see +1's from you on all the test refactoring patches before approving them | 17:32 |
| kgriffs | malini, oz_akan_: has this been benchmarked? https://review.openstack.org/#/c/44340/ | 17:32 |
| openstackgerrit | A change was merged to stackforge/marconi: Run functional tests under tox https://review.openstack.org/44723 | 17:33 |
| oz_akan_ | kgriffs: I haven't | 17:45 |
| *** pycabrera has joined #openstack-marconi | 17:50 | |
| *** cppcabrera has quit IRC | 17:52 | |
| kgriffs | oz_akan_: OK, can you or malini do that and +1 for final approval? | 17:53 |
| kgriffs | I want to make sure it is good before it's merged | 17:53 |
| oz_akan_ | ok | 17:54 |
| zyuan | pycabrera: are your two nicknames mutual exclusive? | 17:54 |
| pycabrera | zyuan: errm, yes. I only acquire pycabrera when I notice that I've disconnected from the network thanks to flaky VPN. :/ | 17:55 |
| *** pycabrera is now known as cppcabrera | 17:55 | |
| kgriffs | pycabrera: u need ZNC. It has a "keep trying for my preferred nick" plugin | 17:57 |
| *** cppcabrera has quit IRC | 18:05 | |
| *** cppcabrera has joined #openstack-marconi | 18:06 | |
| *** cppcabrera has quit IRC | 18:10 | |
| *** cppcabrera has joined #openstack-marconi | 18:16 | |
| * flaper87 is back | 18:28 | |
| flaper87 | kgriffs: ping | 18:29 |
| flaper87 | kgriffs: very quick https://review.openstack.org/#/c/45439/ | 18:30 |
| flaper87 | malini: ping | 18:30 |
| flaper87 | knock knock | 18:47 |
| *** whenry_afk is now known as whenry | 18:58 | |
| flaper87 | knock knock | 19:01 |
| zyuan | the office is too cold T_T | 19:06 |
| openstackgerrit | Alejandro Cabrera proposed a change to stackforge/marconi: feat: marconi proxy https://review.openstack.org/43909 | 19:06 |
| cppcabrera | The office makes me sleepy. | 19:09 |
| cppcabrera | Especially with all of the meetings today. | 19:09 |
| zyuan | i already sleeped during the meeting... | 19:09 |
| cppcabrera | flaper87: kgriffs is attending even more meetings than I today. :P | 19:09 |
| cppcabrera | zyuan: Me, too. :P | 19:09 |
| zyuan | cppcabrera: how much code will be cut-off if proxy uses falcon's | 19:10 |
| zyuan | "new" apis? | 19:10 |
| cppcabrera | Lessee... | 19:11 |
| cppcabrera | I believe this entire file will be eliminated: https://review.openstack.org/#/c/44364/1/marconi/proxy/resources/forward.py | 19:12 |
| malini | flaper87: pong.. | 19:12 |
| kgriffs | speaking of which, can you guys reviewed these? https://github.com/racker/falcon/pulls | 19:12 |
| malini | Sorry..was in a meeting again | 19:12 |
| kgriffs | Once those are merged I can cut a 0.1.7 release | 19:12 |
| cppcabrera | As well as all the routes of the form '/v1/queues/{queue}/.*' | 19:12 |
| cppcabrera | now reviewing, kgriffs | 19:12 |
| kgriffs | thanks! | 19:12 |
| kgriffs | cppcabrera: re catalog storage | 19:13 |
| kgriffs | so, here is my proposal | 19:13 |
| * cppcabrera listens while reviewing | 19:13 | |
| kgriffs | assuming you have an interface defined for the catalog (interface-oriented design FTW!), you can let the catalog backend driver decide how to cache or whatever | 19:14 |
| kgriffs | 2. | 19:14 |
| kgriffs | the official driver can be redis where we read from slaves. we can recommend putting one slave on each proxy host (colocated), but it isn't required | 19:15 |
| kgriffs | that way we don't have people complaining about AGPL | 19:15 |
| cppcabrera | I need to define an interface for the storage driver. All I have atm is a schema. :P | 19:15 |
| kgriffs | and if people want to write something else, let them and we will link to them from our wiki | 19:15 |
| cppcabrera | +1 for linking | 19:16 |
| malini | kgriffs: all the refactor test patches have my +1 now | 19:17 |
| kgriffs | if another driver becomes popular, we can vote to pull it in to the project propper | 19:17 |
| kgriffs | malini: cool, thanks! | 19:17 |
| cppcabrera | malini: awesome! | 19:17 |
| cppcabrera | Ooohh, I like the voting idea. I hadn't made any considerations as to how to bring in drivers to core after the TC meeting a few weeks ago. | 19:18 |
| oz_akan_ | kgriffs: waiting for the result of the test then I will do +1 | 19:18 |
| cppcabrera | Anyway, that's a good TODO before the end of the day - I can sketch out an interface that the proxy caching driver must obey. | 19:19 |
| cppcabrera | re: Redis, I was reading about it's support for slaves. It seems it'll be fully baked by 2.8, but it's already production ready with redis --sentinel in 2.6 | 19:20 |
| cppcabrera | I'm +1 for offloading slave/master replication to the storage technology instead of reimplementing that ourselves. | 19:20 |
| cppcabrera | And since I haven't gotten to distributed caching with the proxy just yet, it doesn't change any of the current implementation at this step. | 19:21 |
| oz_akan_ | I am reading this | 19:21 |
| oz_akan_ | kgriffs: what is the proposal about redis? | 19:22 |
| oz_akan_ | how do you plan to use replication? | 19:22 |
| oz_akan_ | cppcabrera: which slave/master replication is that you want to offload? | 19:22 |
| oz_akan_ | kgriffs: found your proposal above, reading it | 19:23 |
| kgriffs | oz_akan_: ok, FYI I need to rebase that | 19:27 |
| kgriffs | (the large patch test) | 19:27 |
| oz_akan_ | kgriffs: again? | 19:27 |
| kgriffs | s/patch/queue | 19:27 |
| kgriffs | oz_akan_: yeah, people keep doing work. :p | 19:27 |
| oz_akan_ | test result is about to come in a few minutes | 19:27 |
| oz_akan_ | :) | 19:27 |
| kgriffs | but it will be same - just test refactoring | 19:27 |
| kgriffs | (so you don't have to retest) | 19:27 |
| oz_akan_ | good news :) | 19:27 |
| oz_akan_ | meanwhile test seems to run fast | 19:28 |
| kgriffs | cppcabrera: the only other thing you might consider is in-process LRU cache | 19:28 |
| kgriffs | but having localhost redis and using a domain socket will be really fast still | 19:29 |
| cppcabrera | For now, I feel very YAGNI towards in-process cache. | 19:32 |
| cppcabrera | Given localhost caching + domain socket Redis communication. :) | 19:32 |
| flaper87 | back, sorry guys, got disconnected | 19:35 |
| flaper87 | kgriffs: can I get +2 here? https://review.openstack.org/#/q/status:open+project:stackforge/marconi+branch:master+topic:refactor-system-tests,n,z | 19:35 |
| flaper87 | pls :D | 19:35 |
| flaper87 | I got +1 from malini and +2 from cppcabrera | 19:36 |
| malini | I can add new tests after this is merged…woohooo... | 19:37 |
| flaper87 | malini: +100000^ | 19:37 |
| cppcabrera | I just want to see openstack-gerrit merge 6-8 patches in less than 15 minutes. :P | 19:46 |
| cppcabrera | (also +1e9 for refactored tests - that's cool, too) | 19:46 |
| *** malini is now known as malini_afk | 19:49 | |
| flaper87 | w0000t | 19:51 |
| flaper87 | Marconi will be migrated under openstack/ this weekend | 19:51 |
| flaper87 | yoooooohhhhhhhhhhoooooooooooooooooooooooooooooooooooooo | 19:51 |
| flaper87 | cppcabrera: did you see this one? http://openstackreactions.enovance.com/2013/09/when-your-project-enters-incubation/ | 19:52 |
| cppcabrera | hahaha, flaper87. :P | 19:53 |
| cppcabrera | That looks about right. | 19:53 |
| cppcabrera | Kermit knows how to get incubated. :D | 19:54 |
| flaper87 | kgriffs: ping :) | 19:57 |
| amitgandhi | w0000t! | 19:58 |
| kgriffs | flaper87: https://review.openstack.org/#/c/44749/11/marconi/tests/functional/base.py | 20:00 |
| kgriffs | should self.server be cls.server? | 20:00 |
| kgriffs | (in setUp() | 20:00 |
| cppcabrera | kgriffs: oz_akan and I talked about the replication proposal and we found a few disadvantages to it. It we used slave/master style replication to implement -proxy, the local caches cease being caches. That'll lead to a lot of duplicated state. The other half of the problem with this approach is that there's no longer a notion of a "cache miss" for slaves - just stale data. | 20:06 |
| kgriffs | cppcabrera: re the latter concern | 20:09 |
| kgriffs | flaper87: pong | 20:13 |
| * flaper87 back | 20:14 | |
| * flaper87 just got a new server | 20:14 | |
| flaper87 | :D | 20:14 |
| flaper87 | kgriffs: nup | 20:15 |
| flaper87 | kgriffs: cls is when setting things up in setUpClass | 20:15 |
| oz_akan_ | kgriffs: are we close for index patch to be merged? | 20:15 |
| flaper87 | kgriffs: did you read that Marconi will be migrated under openstack/ this saturday ? | 20:16 |
| kgriffs | but how do you share the server instance across runs? | 20:16 |
| kgriffs | flaper87: can't wait! | 20:16 |
| kgriffs | i mean, why does tearDownClass refer to cls.server ? | 20:17 |
| flaper87 | kgriffs: because server is defined at class level | 20:17 |
| flaper87 | line 32 | 20:17 |
| kgriffs | oh crap | 20:17 |
| kgriffs | sorry | 20:17 |
| flaper87 | :D | 20:17 |
| flaper87 | no worries | 20:17 |
| kgriffs | stupid python syntax | 20:17 |
| flaper87 | you've been into many meetings today | 20:17 |
| flaper87 | (I've been told) | 20:17 |
| kgriffs | self.foo is ambiguous - access class AND instance vars | 20:18 |
| kgriffs | yes, many meetings | 20:18 |
| kgriffs | got a talk on Marconi in 45 minutes at the Rackspace Austin office | 20:18 |
| kgriffs | so, on the run all day | 20:18 |
| flaper87 | :0 | 20:18 |
| flaper87 | well talk about Marconi sounds good | 20:18 |
| flaper87 | :D | 20:18 |
| flaper87 | better than a meeting | 20:18 |
| flaper87 | :P | 20:18 |
| kgriffs | yeah | 20:18 |
| kgriffs | :D | 20:18 |
| flaper87 | anyway, can I get your bless on those patches? | 20:18 |
| flaper87 | I promise not to bother you anymore | 20:19 |
| cppcabrera | lol | 20:19 |
| flaper87 | .... for today | 20:19 |
| flaper87 | :D | 20:19 |
| kgriffs | oz_akan: will merge that patch very soon | 20:19 |
| kgriffs | flaper87: let me rebase that indexing patch on top of all the test refactors, then you are good to approve | 20:19 |
| kgriffs | (oz_akan tested perf and was good) | 20:19 |
| flaper87 | kgriffs: awesome, +! | 20:20 |
| flaper87 | +1 | 20:20 |
| kgriffs | flaper87: hooray for format() ! | 20:21 |
| flaper87 | kgriffs: :D | 20:21 |
| flaper87 | that's the first one, next one will touch marconi/* | 20:21 |
| kgriffs | hmm, was the format thing announced on the dev list? | 20:23 |
| flaper87 | kgriffs: nope | 20:23 |
| flaper87 | we can mention it | 20:23 |
| flaper87 | but I don't think it is a big deal | 20:24 |
| kgriffs | so are any other projects doing it? | 20:24 |
| kgriffs | i think it's fine, just curious | 20:24 |
| flaper87 | TBH, I didn't check | 20:24 |
| kgriffs | ok | 20:25 |
| flaper87 | so, it's being used in nova | 20:25 |
| flaper87 | in some places | 20:25 |
| flaper87 | well, actually, grep keeps going | 20:25 |
| flaper87 | I'd say it's fine | 20:25 |
| flaper87 | AFAICT, new code is using .format | 20:26 |
| flaper87 | old code hasn't ben touched | 20:26 |
| kgriffs | gtk | 20:26 |
| flaper87 | but that's my assumption based on what I just saw | 20:26 |
| flaper87 | I mean, grep is not that smart | 20:26 |
| flaper87 | neither am I | 20:26 |
| flaper87 | :D | 20:26 |
| flaper87 | kgriffs: 1 more to go :D | 20:26 |
| * flaper87 is refreshing that page | 20:26 | |
| flaper87 | :D | 20:27 |
| cppcabrera | I brought up .format awhile ago as part of the test refactoring effort. :) | 20:27 |
| flaper87 | yeah, cppcabrera it's your fault | 20:27 |
| flaper87 | :D | 20:27 |
| cppcabrera | I couldn't find any official justification for moving to .format from %-style either in OS docs or in the python community, other than pep3101. | 20:27 |
| kgriffs | flaper87: would you like to add links to your blog post(s) to the bottom of the wiki under "Articles"? https://wiki.openstack.org/wiki/Marconi | 20:28 |
| flaper87 | kgriffs: sure :D | 20:28 |
| cppcabrera | +1 kgriffs - let's keep adding to our awesome resources list. | 20:28 |
| flaper87 | I'll do that | 20:28 |
| cppcabrera | woot! | 20:28 |
| flaper87 | cppcabrera: did you read tests one ? | 20:28 |
| cppcabrera | I glanced over it yesterday. :) | 20:29 |
| flaper87 | kk, wanted to get your thoughts about it | 20:29 |
| cppcabrera | "Third-party modules can take advantage of Marconi's test suite. People writing their own modules will be able to import the test suite for the module they're working on" -> | 20:30 |
| cppcabrera | That's awesome. | 20:30 |
| cppcabrera | That's been one of the coolest testing strategies I've learned about while working on marconi. | 20:30 |
| kgriffs | crappies | 20:31 |
| kgriffs | jenkins barfed on those patches | 20:31 |
| flaper87 | :( | 20:31 |
| flaper87 | pkg_resources.VersionConflict: (six 1.4.1 (/home/jenkins/workspace/gate-marconi-pep8/.tox/pep8/lib/python2.7/site-packages), Requirement.parse('six<1.4.0')) | 20:31 |
| flaper87 | seriously ? | 20:32 |
| flaper87 | damn it! | 20:32 |
| flaper87 | they updated that on my back | 20:32 |
| cppcabrera | blahhh | 20:32 |
| flaper87 | let me propose a quick patch for that | 20:32 |
| cppcabrera | six 1.4.1 is too new. :P | 20:32 |
| flaper87 | wait, something else is pulling six 1.4.1 in | 20:33 |
| flaper87 | could it be falcon? | 20:33 |
| kgriffs | could be. it doesn't specify a six version | 20:34 |
| kgriffs | iirc | 20:34 |
| flaper87 | mmh, it that case it shouldn't be an issue | 20:34 |
| flaper87 | https://github.com/racker/falcon/blob/master/setup.py#L9 | 20:34 |
| flaper87 | it doesn't specify it | 20:34 |
| * flaper87 is now confused | 20:35 | |
| cppcabrera | some project is probably bringing in six. | 20:37 |
| cppcabrera | So my best guess is that this six update is causing some havoc across openstack right now. :P | 20:37 |
| cppcabrera | yup, it's being brought up in -infra right now. | 20:38 |
| flaper87 | yeah, just read that :P | 20:38 |
| flaper87 | "Is the error of the moment and it's being worked" | 20:39 |
| kgriffs | ok folks, bbl - gotta talk to do | 20:51 |
| cppcabrera | good luck! | 20:53 |
| kgriffs | thanks! | 20:56 |
| *** kgriffs is now known as kgriffs_afk | 20:56 | |
| openstackgerrit | Oz Akan proposed a change to stackforge/marconi: Adds support for multiple databases per mongodb replica set. https://review.openstack.org/45952 | 20:57 |
| oz_akan_ | https://review.openstack.org/45952 | 20:58 |
| oz_akan_ | oh, openstackgerrit is faster than I am | 20:59 |
| oz_akan_ | flaper87: zyuan flaper87 kgriffs_afk ^^ | 20:59 |
| oz_akan_ | as this is my first, I assume I violated several best practices so waiting to get comments | 20:59 |
| *** torgomatic has quit IRC | 21:00 | |
| cppcabrera | lol | 21:01 |
| cppcabrera | openstackgerrit is the fastest. | 21:02 |
| zyuan | news - FreeBSD-CURRENT now only build clang and libc++ by default. | 21:02 |
| *** russell_h has quit IRC | 21:02 | |
| cppcabrera | +1 | 21:03 |
| oz_akan_ | what do I do when jenkins give me -1 ? | 21:03 |
| *** torgomatic has joined #openstack-marconi | 21:03 | |
| zyuan | read error log and re-submit | 21:03 |
| oz_akan_ | hmm, if I find the error log :) | 21:04 |
| zyuan | actually i already saw some mis-aligned code | 21:04 |
| oz_akan_ | found it | 21:04 |
| zyuan | it's kinda stupid... hope clang-format someday can work for python | 21:05 |
| zyuan | i mean, to let human reformat the code... | 21:05 |
| *** ykaplan has joined #openstack-marconi | 21:06 | |
| *** russell_h has joined #openstack-marconi | 21:07 | |
| cppcabrera | python already has autopep8, zyuan. :) | 21:11 |
| cppcabrera | But +1 for leaving formatting to humans. | 21:11 |
| cppcabrera | Err... | 21:11 |
| cppcabrera | +1 against leaving formatting to humans. That's a job for computers. | 21:11 |
| zyuan | after listened to Chandler's talk, some people who developping development tools on iOS want to use clang-format in their editor... | 21:12 |
| *** malini_afk is now known as malini | 21:16 | |
| oz_akan_ | cppcabrera: when I reply to a messages in review, that draft things, is draft for the reply? | 21:21 |
| cppcabrera | oz_akan_: Yes. The "drafts" will only be published after you click the "Review" button and then hit "Publish". :) | 21:24 |
| oz_akan_ | yay! I did it | 21:25 |
| oz_akan_ | thanks cppcabrera | 21:26 |
| *** kgriffs_afk is now known as kgriffs | 21:26 | |
| oz_akan_ | zyuan: thanks for the comments, I will review them | 21:26 |
| oz_akan_ | ups, seems like i didn't really submit the latest version.. anyway, I will do it with lru thing and zyuan comments | 21:30 |
| *** oz_akan_ has quit IRC | 21:31 | |
| *** malini is now known as malini_afk | 21:32 | |
| *** flaper87 is now known as flaper87|afk | 21:37 | |
| zyuan | what's wrong with gerrit... version conflict | 21:37 |
| *** tedross has quit IRC | 21:38 | |
| *** kgriffs is now known as kgriffs_afk | 21:40 | |
| *** flaper87|afk is now known as flaper87 | 21:47 | |
| *** flaper87 is now known as flaper87|afk | 21:48 | |
| cppcabrera | Heading home for the night. Take care, guys. :) | 21:52 |
| *** cppcabrera has quit IRC | 21:52 | |
| *** ayoung has quit IRC | 22:03 | |
| *** kgriffs_afk is now known as kgriffs | 22:06 | |
| openstackgerrit | A change was merged to stackforge/marconi: Implement embedded marconi-server execution https://review.openstack.org/44749 | 22:14 |
| *** ykaplan has quit IRC | 22:14 | |
| openstackgerrit | A change was merged to stackforge/marconi: Implement small http client for tests https://review.openstack.org/45267 | 22:14 |
| openstackgerrit | A change was merged to stackforge/marconi: Move functional tests into wsgi/v1 https://review.openstack.org/45423 | 22:15 |
| openstackgerrit | A change was merged to stackforge/marconi: Use format instead of % for string formatting https://review.openstack.org/45877 | 22:17 |
| *** amitgandhi has quit IRC | 22:23 | |
| *** oz_akan_ has joined #openstack-marconi | 22:42 | |
| *** oz_akan_ has quit IRC | 22:47 | |
| openstackgerrit | Kurt Griffiths proposed a change to stackforge/marconi: perf: Partition messages collection by project https://review.openstack.org/45966 | 22:47 |
| openstackgerrit | Kurt Griffiths proposed a change to stackforge/marconi: fix: Requests get slower when queues have a lot of messages https://review.openstack.org/44340 | 22:47 |
| *** amitgandhi has joined #openstack-marconi | 23:46 | |
| *** amitgandhi has quit IRC | 23:47 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!