| *** malini_afk is now known as malini | 00:02 | |
| *** tedross has joined #openstack-marconi | 00:14 | |
| *** tedross has quit IRC | 00:21 | |
| *** nosnos has joined #openstack-marconi | 01:03 | |
| *** russellb has quit IRC | 01:43 | |
| *** russellb has joined #openstack-marconi | 01:44 | |
| *** malini is now known as malini_afk | 02:01 | |
| *** oz_akan_ has joined #openstack-marconi | 02:31 | |
| *** oz_akan_ has quit IRC | 03:46 | |
| *** oz_akan_ has joined #openstack-marconi | 03:47 | |
| *** oz_akan_ has quit IRC | 03:51 | |
| *** reed has quit IRC | 04:43 | |
| *** oz_akan_ has joined #openstack-marconi | 04:57 | |
| *** oz_akan_ has quit IRC | 05:02 | |
| *** flaper87|afk is now known as flaper87 | 06:58 | |
| *** ykaplan has joined #openstack-marconi | 07:08 | |
| flaper87 | al-maisan: ping | 07:19 |
|---|---|---|
| *** gordonsim has joined #openstack-marconi | 07:48 | |
| al-maisan | hello flaper87 ! | 07:50 |
| al-maisan | sitting at the airport -- can talk for 10 minutes :-P | 07:50 |
| flaper87 | al-maisan: ah, just wanted to know if you could update your patch | 07:51 |
| flaper87 | or if I can do it for you | 07:51 |
| flaper87 | kinda need that to move on | 07:51 |
| al-maisan | either is good -- what would you like changed? | 07:52 |
| flaper87 | if you don't mind and you agree with the comments I wrote | 07:52 |
| flaper87 | I reviewed it yday | 07:52 |
| al-maisan | I pushed a new patch set | 07:52 |
| al-maisan | https://review.openstack.org/47892 | 07:52 |
| flaper87 | mmh, the last one is PS number 4 | 07:52 |
| al-maisan | sorry, new review | 07:52 |
| al-maisan | I should have pushed a new patch set (#5) instead? | 07:53 |
| flaper87 | yeah | 07:53 |
| al-maisan | sorry -- learning the ropes still | 07:53 |
| flaper87 | let me do that for you, if you don't mind? | 07:53 |
| flaper87 | no worries at all | 07:53 |
| flaper87 | I'll push a new patch set in your old review | 07:54 |
| flaper87 | to keep track of history | 07:54 |
| al-maisan | flaper87: can we leave at least 'operation' as an explicit property of request.Spec? | 07:54 |
| al-maisan | flaper87: I can do that as well -- give a me 2 mins | 07:54 |
| flaper87 | al-maisan: awesome, thanks a lot! Thought you were jumping into the plain | 07:55 |
| flaper87 | we can keep that explicit for now, TBH, I don't think it should | 07:55 |
| al-maisan | 5 minutes left | 07:56 |
| flaper87 | we'll always need an operation | 07:56 |
| flaper87 | al-maisan: cool, thanks man!!! | 07:56 |
| al-maisan | let's discuss explicit vs. param later today | 07:56 |
| al-maisan | you are welcome | 07:56 |
| flaper87 | al-maisan: okey-dokey | 07:56 |
| openstackgerrit | Muharem Hrnjadovic proposed a change to openstack/python-marconiclient: This adds a transport layer request spec class https://review.openstack.org/47538 | 08:08 |
| al-maisan | flaper87: ^^ | 08:08 |
| * al-maisan runs to the airplane | 08:08 | |
| flaper87 | al-maisan: thanks | 08:08 |
| flaper87 | al-maisan: have a safe flight | 08:08 |
| al-maisan | flaper87: my flight is delayed :-P is patch set #5 any good? | 08:21 |
| flaper87 | al-maisan: it looks good, thanks a lot! We can discuss the required thing later if you want | 08:23 |
| flaper87 | we can let that land in a separate patch | 08:23 |
| al-maisan | OK | 08:24 |
| * al-maisan abandons the new/unneeded review (https://review.openstack.org/47892) | 08:26 | |
| al-maisan | flaper87: why did you suggest the ".copy()" bit for common.api.info() ? | 08:29 |
| flaper87 | al-maisan: just as a security thing, trying to avoid obscure bugs caused because that dict was modified in some place | 08:32 |
| flaper87 | that dict shouldn't be modified anywhere | 08:33 |
| flaper87 | but you know.... | 08:33 |
| al-maisan | hmm .. I see .. a bit of paranoia can be handy at times ;-) | 08:33 |
| flaper87 | don't want the whole think to break just because of a silly .pop() or something | 08:33 |
| flaper87 | yeah | 08:33 |
| al-maisan | thanks! | 08:33 |
| flaper87 | al-maisan: thank you :) | 08:34 |
| flaper87 | I'll submit my patch very soon | 08:35 |
| * flaper87 is signing the contract for his new ISP | 08:35 | |
| flaper87 | Optical fiber FTW | 08:36 |
| al-maisan | hmm .. that should result in many happy streaming bits ;) | 08:40 |
| *** ykaplan has quit IRC | 09:07 | |
| *** tvb has joined #openstack-marconi | 09:09 | |
| *** tvb has quit IRC | 09:35 | |
| *** Guest22854 has joined #openstack-marconi | 09:37 | |
| *** Guest22854 is now known as tvb | 09:37 | |
| *** tvb has joined #openstack-marconi | 09:37 | |
| *** ykaplan has joined #openstack-marconi | 10:19 | |
| *** malini_afk is now known as malini | 10:28 | |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Add Tests for Queue Stats https://review.openstack.org/48012 | 11:10 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Add Tests for Queue Stats https://review.openstack.org/48012 | 11:14 |
| flaper87 | malini: goood morning | 11:32 |
| flaper87 | :D | 11:32 |
| *** malini is now known as malini_afk | 11:42 | |
| *** cppcabrera has joined #openstack-marconi | 12:28 | |
| cppcabrera | Goood morning! | 12:28 |
| *** ayoung has quit IRC | 12:32 | |
| *** EmilienM has quit IRC | 12:35 | |
| *** EmilienM has joined #openstack-marconi | 12:36 | |
| flaper87 | cppcabrera: GOOOD MORNING | 12:45 |
| *** tvb has quit IRC | 12:46 | |
| *** ametts has quit IRC | 12:46 | |
| *** ametts has joined #openstack-marconi | 12:46 | |
| cppcabrera | how go things, flaper87? :) | 12:48 |
| flaper87 | cppcabrera: allllllll gooooood, you? | 12:49 |
| cppcabrera | pretty sweet. It's been a good morning so far. :D | 12:49 |
| *** vkmc has joined #openstack-marconi | 12:49 | |
| cppcabrera | I see we have a patch working to get marconi into devstack. | 12:51 |
| flaper87 | cppcabrera: Fuck Yeah!!! | 12:54 |
| flaper87 | and it works pretty nice | 12:54 |
| flaper87 | it sets up marconi+keystone | 12:54 |
| *** kgriffs_afk is now known as kgriffs | 12:54 | |
| cppcabrera | awesome. :D | 12:55 |
| cppcabrera | I'll be reviewing that patch shortly. | 12:55 |
| * cppcabrera is on the morning review train | 12:55 | |
| kgriffs | cppcabrera: is mpanetta in teh house? | 12:56 |
| cppcabrera | I believe not, kgriffs. I'll check. | 12:56 |
| kgriffs | flaper87: the internet works sooooo much better once you upgrade from 56k | 12:56 |
| cppcabrera | lol | 12:57 |
| cppcabrera | mpanetta's not here yet, kgriffs. | 12:57 |
| kgriffs | also, guys, need a final review on this: https://review.openstack.org/#/c/47272/ | 12:57 |
| kgriffs | see my last comment (and email) | 12:57 |
| kgriffs | ok | 12:57 |
| flaper87 | kgriffs: LOOOOOOL | 12:57 |
| kgriffs | cppcabrera: ok | 12:57 |
| flaper87 | kgriffs: will review that soon | 12:57 |
| kgriffs | just want to make sure he knows he is *invited* to join this channel | 12:57 |
| flaper87 | kgriffs: I read your email | 12:58 |
| kgriffs | kk | 12:58 |
| flaper87 | who is mpanetta ? | 12:58 |
| kgriffs | Mike Panetta | 12:58 |
| flaper87 | I saw him / her yesterday | 12:58 |
| flaper87 | ah ok | 12:58 |
| kgriffs | he is a new Rackspace d00d | 12:58 |
| kgriffs | works with Oz on devops stuff | 12:58 |
| flaper87 | coooool! | 12:58 |
| flaper87 | kgriffs: btw, was there a meeting yday? Didn't see any znc backlog | 12:59 |
| kgriffs | there was, but we got started really late | 12:59 |
| cppcabrera | yeah, we started at like... | 12:59 |
| kgriffs | mostly just discussed the proxy | 12:59 |
| flaper87 | oh, ok, will check the logs in the wiki then | 12:59 |
| cppcabrera | 4:30 UTC. :P | 12:59 |
| cppcabrera | flaper87: ^ | 12:59 |
| cppcabrera | kgriffs was sidetracked discussing requirements/deps in -infra. | 13:00 |
| flaper87 | sorry for not being around yday | 13:00 |
| cppcabrera | no worries, flaper87. sometimes other things come up. :) | 13:00 |
| flaper87 | cppcabrera: yeah, the Pecab / Falcon discussion was pretty intense. I had to leave and abandone kgriffs there | 13:00 |
| cppcabrera | the big discussion point, as kgriffs mentioned, was the proxy. By the end of the day, kgriffs worked with me to identify the remaining issues with the proxy based on the initial storage driver patches. | 13:02 |
| cppcabrera | Those issues continue to live here: https://etherpad.openstack.org/marconi-proxy-issues | 13:02 |
| *** oz_akan_ has joined #openstack-marconi | 13:02 | |
| cppcabrera | both the memory and mongodb proxy storage driver patches have been approved and merged (with notes) at this point. The next steps are finalizing the caching and transport-mirror patches while I work on fixing issues and adding tests. | 13:03 |
| *** oz_akan_ has quit IRC | 13:03 | |
| cppcabrera | ah, and we found a negative age bug. I'll fix that one up after reviewing patches this morning. | 13:03 |
| flaper87 | cppcabrera: awesome, I'd really appriciate it if you could takle the pending TODOs first | 13:04 |
| *** oz_akan_ has joined #openstack-marconi | 13:04 | |
| cppcabrera | will do, flaper87. | 13:04 |
| flaper87 | cppcabrera: awesome, thanks d00000000d | 13:05 |
| * flaper87 learnt d00d from kgriffs :D | 13:05 | |
| cppcabrera | haha | 13:05 |
| cppcabrera | everytime I hear d000d, I think of Prinnies. | 13:05 |
| *** mpanetta has joined #openstack-marconi | 13:05 | |
| cppcabrera | mpanetta: o/ | 13:05 |
| cppcabrera | kgriffs was looking for you. :P | 13:06 |
| mpanetta | Oh? | 13:06 |
| kgriffs | o/ | 13:06 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Add keystone backend https://review.openstack.org/48041 | 13:06 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Authentication API + NoAuth backend https://review.openstack.org/47334 | 13:06 |
| kgriffs | just making sure you saw my email | 13:06 |
| kgriffs | and would love to get that −1 removed if you agree | 13:06 |
| mpanetta | Oh, yes. Will do | 13:06 |
| kgriffs | brb | 13:07 |
| flaper87 | btw, this babies here need some love https://review.openstack.org/#/q/status:open+project:openstack/python-marconiclient,n,z | 13:07 |
| cppcabrera | flaper87: I'm happy to see marconiclient coming along awesomely. | 13:07 |
| cppcabrera | New patches everyday | 13:07 |
| cppcabrera | I'm already got most of the marconiclient patches open for review in my tabs. ;D | 13:07 |
| flaper87 | yeaaaah, I really want to have the lib ready sooooooon! | 13:07 |
| cppcabrera | I'm tackling kgriff's marker patch atm. | 13:08 |
| flaper87 | yeah, I'll do that as well | 13:08 |
| mpanetta | Ok, updated. Probably should have done that from home last night but didn't think about it. | 13:09 |
| cppcabrera | no worries, mpanetta. :) | 13:10 |
| mpanetta | I assume it is ok for me to move on to next patch now? | 13:11 |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Update requirements and pbr version https://review.openstack.org/46259 | 13:13 |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Remove mox from test-requirements.txt https://review.openstack.org/47486 | 13:13 |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Adding first oslo module and config https://review.openstack.org/46275 | 13:14 |
| *** amitgandhi has joined #openstack-marconi | 13:15 | |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Added a base class for tests https://review.openstack.org/46288 | 13:15 |
| cppcabrera | my, my - look at all those patches. | 13:16 |
| cppcabrera | busy day to be a reviewer. | 13:16 |
| * flaper87 is going to cry | 13:17 | |
| flaper87 | all those babies growing up so fast | 13:17 |
| flaper87 | they're all landing together | 13:17 |
| cppcabrera | kgriffs: marker patch reviewed - I'm -1 on it because of a testing decision, but I can be convinced that the random approach is okay. | 13:17 |
| cppcabrera | flaper87: I know, right? :') | 13:18 |
| kgriffs | well, my thinking there was that you run 3 envs | 13:18 |
| kgriffs | so chances are it will run at least once | 13:18 |
| kgriffs | i guess I guess i could change it to 33% | 13:18 |
| cppcabrera | Since it occurs only in the mongo tests, wouldn't it *never* be run in the jenkins gate jobs? | 13:19 |
| kgriffs | the alternatives are to always run, which is really annoying because it takes 1 minute | 13:19 |
| kgriffs | cppcabrera: true, but what about flaper87 and myself who always test against mongo? | 13:19 |
| kgriffs | :p | 13:19 |
| kgriffs | other alternative is to only run when some env variable is set or something, but I am worried that eventually everyone will forget to do that | 13:20 |
| cppcabrera | hmmm... | 13:20 |
| kgriffs | other ideas? | 13:20 |
| cppcabrera | we already have the case of MONGODB_TEST_LIVE | 13:20 |
| cppcabrera | As long as the env variables follow a convention of some sort... | 13:20 |
| cppcabrera | I'm fond of X_TEST_LIVE since I can usually grep for _TEST_LIVE to see what we have | 13:21 |
| cppcabrera | So... | 13:21 |
| cppcabrera | I'm sayuing I like the environment variable approach. :D | 13:21 |
| cppcabrera | MONGODB_SLOW_TEST=1 | 13:21 |
| cppcabrera | *saying | 13:21 |
| flaper87 | cppcabrera: I like cppcabrera idea about MONGODB_SLOW_TEST | 13:22 |
| *** cthulhup has joined #openstack-marconi | 13:22 | |
| flaper87 | dunno why I mentioned cppcabrera 2 times in my last message, I guess I just wanted to mention him again cppcabrera and again cppcabrera and again cppcabrera | 13:23 |
| flaper87 | anyway | 13:23 |
| cppcabrera | lol | 13:23 |
| cppcabrera | "person: Hey, there, person" gets me from time to time because I know I want to address {person} but I don't realize I'm going to refer to {person} again by the end of my statement. (flaper87) | 13:24 |
| flaper87 | the other thing could be not testing expired messages at all for mongod since they are managed by mongodb's internal which we *don't control* but between those options I prefer the MONGODB_SLOW_TEST | 13:24 |
| flaper87 | cppcabrera: indeed T_T | 13:24 |
| *** mpanetta has quit IRC | 13:24 | |
| *** mpanetta has joined #openstack-marconi | 13:24 | |
| kgriffs | ok, let me do MONGODB_SLOW_TEST | 13:26 |
| kgriffs | any other comments? | 13:26 |
| cppcabrera | One more - self.assertIsNotNone | 13:27 |
| flaper87 | kgriffs: I'm still reviewing it | 13:27 |
| cppcabrera | I'm +2 on the patch after MONGODB_SLOW_TEST, though. | 13:27 |
| kgriffs | should we just have a MARCONI_SLOW_TEST | 13:27 |
| kgriffs | make it more generic? | 13:27 |
| *** cthulhup has quit IRC | 13:27 | |
| kgriffs | so we can use it elsewhere later if we need to? | 13:27 |
| flaper87 | kgriffs: +1 | 13:28 |
| cppcabrera | +1 kgriffs | 13:28 |
| flaper87 | btw, why is the test_expired_messages a private method ? | 13:28 |
| kgriffs | so that we can pass different sleep times to it dep on the backend | 13:29 |
| kgriffs | sqlite doesn't need the sleep, for example | 13:29 |
| flaper87 | kgriffs: what about having a `gc_timeout` class attribute ? | 13:30 |
| kgriffs | hmm, that could work | 13:30 |
| kgriffs | re envs | 13:30 |
| kgriffs | how about | 13:30 |
| kgriffs | MARCONI_TEST_SLOW | 13:30 |
| flaper87 | I don't like having to explicitly call that method in the TestCase, sounds like we could forget about it for new implementations | 13:30 |
| kgriffs | MARCONI_TEST_MONGODB | 13:30 |
| kgriffs | (renaming the latter from MONGODB_TEST_LIVE) | 13:31 |
| flaper87 | kgriffs: +1 | 13:31 |
| flaper87 | sounds like something for a different patch, though | 13:31 |
| kgriffs | kk | 13:31 |
| flaper87 | at least the rename | 13:31 |
| cppcabrera | Renaming later and settling on a good convetion sounds good to me. | 13:32 |
| cppcabrera | I'm favoring MARCONI_TEST_X for a general approach. | 13:32 |
| cppcabrera | MARCONI_TEST is the namespace, and X is the aspect we want to activate. | 13:32 |
| cppcabrera | Lends itself well to scripting/automation (for aspect in desired_aspected: os.environ['MARCONI_TEST_{aspect}'] = 1) | 13:33 |
| *** malini_afk is now known as malini | 13:34 | |
| flaper87 | kgriffs: https://review.openstack.org/#/c/47272/7/marconi/queues/storage/mongodb/messages.py | 13:35 |
| flaper87 | line 565 | 13:35 |
| flaper87 | why would we usually endup in there? | 13:35 |
| flaper87 | end up | 13:35 |
| flaper87 | doesn't _inc_counter return the new counter ? | 13:35 |
| kgriffs | if window is specified, then _inc_counter will add it to the query which may result in the marker not being incremented | 13:37 |
| flaper87 | kgriffs: kk, makes sense | 13:37 |
| flaper87 | danke | 13:37 |
| flaper87 | btw, what about adding a test case for a "duplicated-counter" scenario? | 13:38 |
| flaper87 | I think we could simulate that by mocking the _get_counter method | 13:38 |
| flaper87 | and making it return an already used counter | 13:38 |
| kgriffs | hmm, it would be nice to be able to test that automatically | 13:40 |
| kgriffs | let me see what I can do there | 13:40 |
| kgriffs | alternatively I could just inject some messages into the collection without incrementing the counter | 13:41 |
| flaper87 | kgriffs: that should work too | 13:42 |
| flaper87 | awesome | 13:42 |
| flaper87 | patch LGTM btw | 13:42 |
| cppcabrera | cool, cool - we're keeping a good pace with reviews this morning! | 13:45 |
| kgriffs | ok, so let me fix up the tests and submit an update | 13:45 |
| malini | good morning!! | 13:46 |
| malini | cppcabrera: thanks for the review..that was an embarassing one :-$ | 13:46 |
| cppcabrera | malini: morning! No worries - it's what code review is for. To catch all the things that escape one person. :D | 13:47 |
| malini | between y'day evening & this morning, my circuit broke :D | 13:48 |
| malini | while we are all here, I also wanted some thought on testing proxy | 13:49 |
| cppcabrera | sure thing, malini. I'm open for discussion on that while I review all the patches. | 13:50 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/marconi: Add Tests for Queue Stats https://review.openstack.org/48012 | 13:50 |
| malini | cppcabrera: Do you have the proxy wiki url handy ? forgot to bookmark | 13:52 |
| *** nosnos has quit IRC | 13:54 | |
| cppcabrera | malini: https://wiki.openstack.org/wiki/Marconi/specs/proxy/v1 | 13:55 |
| *** nosnos has joined #openstack-marconi | 13:55 | |
| malini | thanks cppcabrera | 13:56 |
| *** jraim has quit IRC | 13:56 | |
| malini | w.r.t adding functional tests for proxy, it is easy to test each api independently | 13:56 |
| malini | but its going to be really hard to test this functionally | 13:57 |
| malini | as in add a new partition & verify that the requests are routed to the new partition | 13:57 |
| cppcabrera | hmm... | 13:57 |
| *** nosnos has quit IRC | 13:58 | |
| cppcabrera | I have an idea on that, malini | 13:58 |
| malini | bring it on | 13:58 |
| cppcabrera | Alright, so let's say we register a set of partitions | 13:58 |
| cppcabrera | Now we create a queue | 13:58 |
| cppcabrera | We want to verify that the queue was created | 13:59 |
| cppcabrera | So then... | 13:59 |
| cppcabrera | How about checking through each partition and verifying that exactly one partition contains that queue? | 13:59 |
| malini | let's take one step back, | 13:59 |
| kgriffs | question | 13:59 |
| kgriffs | os.environ.get | 13:59 |
| cppcabrera | k | 14:00 |
| kgriffs | that always returns None or a string, right? | 14:00 |
| cppcabrera | kgriffs: yes - verified it just now | 14:00 |
| cppcabrera | unless you do os.environ.get() => {str: str} | 14:00 |
| malini | registering a set of partitions assumes that we have a number of partitions available :( | 14:01 |
| cppcabrera | malini: yeah, that would depend on a test environment consisting of >1 partition. :/ | 14:01 |
| cppcabrera | partitions can be small, too, for the sake of the test | 14:01 |
| cppcabrera | IMO, it's enough to have a partition consist of 2 hosts and 1 storage node. | 14:01 |
| malini | I am thinking of just starting up another marconi server, thouhts ? | 14:02 |
| cppcabrera | they can all be 512MB (or smaller instances) | 14:02 |
| cppcabrera | sounds good to me, malini | 14:02 |
| cppcabrera | 2 partitions, 1 proxy | 14:02 |
| cppcabrera | I think that's good enough to cover many of the basic functional cases | 14:02 |
| malini | that will make our tests very complex (creating new 512 MB instances) | 14:02 |
| cppcabrera | malini: I'd just let the tests assume that such an environment exists, esp. for now. No need to have the tests spin up/tear down envs | 14:03 |
| *** EmilienM has quit IRC | 14:04 | |
| malini | cppcabrera: if the tests assume that the env exists, it'll break our tox compatibility:/ | 14:04 |
| cppcabrera | hmm... | 14:04 |
| cppcabrera | I have another idea! | 14:05 |
| cppcabrera | :D | 14:05 |
| cppcabrera | This one preserves tox | 14:05 |
| cppcabrera | (thanks for mentioning that!) | 14:05 |
| cppcabrera | So, given that we functionally test the proxy using the memory driver | 14:06 |
| cppcabrera | and that the marconi-servers themeselves use some kind of persistent storage (sqlite is fine) | 14:06 |
| cppcabrera | Then... | 14:06 |
| cppcabrera | It can all be done on localhost | 14:06 |
| cppcabrera | 1. Spin up two marconi servers on different ports | 14:06 |
| cppcabrera | 2. Have each of them use a different storage location | 14:06 |
| cppcabrera | 3. Spin up one proxy | 14:06 |
| cppcabrera | 4. Test away | 14:07 |
| cppcabrera | That's representative of 2 partitions and 1 proxy | 14:07 |
| cppcabrera | thoughts, malini? | 14:07 |
| *** EmilienM has joined #openstack-marconi | 14:07 | |
| cppcabrera | br | 14:08 |
| malini | 2. means just pointing to two different dbs, rt? | 14:08 |
| cppcabrera | b | 14:08 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Move tests/unit/storage/base under m/tests/storage https://review.openstack.org/48057 | 14:08 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Move tests/unit/storage/base under m/tests/storage https://review.openstack.org/48057 | 14:11 |
| *** mpanetta has quit IRC | 14:14 | |
| *** mpanetta_ has joined #openstack-marconi | 14:14 | |
| mpanetta_ | gah | 14:14 |
| *** mpanetta_ is now known as mpanetta | 14:14 | |
| flaper87 | guys, there are just 3 design sessions for Marconi, please, add there topics you would like to discuss during the summit | 14:15 |
| flaper87 | http://summit.openstack.org/ | 14:15 |
| flaper87 | we've got 4 slots | 14:15 |
| cppcabrera | back | 14:16 |
| cppcabrera | malini: That's correct | 14:16 |
| cppcabrera | flaper87: ooohh, I'll check that out! | 14:17 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Move test_default_limits to the new tests location https://review.openstack.org/48059 | 14:17 |
| cppcabrera | I'd love to see alternate transports addressed, all the moreso since there's much more opportunity for doing interesting things with zeromq, websockets, etc. than in HTTP | 14:17 |
| malini | cppcabrera: How easy is spinning up a proxy with the tests ? | 14:17 |
| cppcabrera | as of the latest patch, malini, it should be as easy as launching a proxy.transport.wsgi.app in a multiprocess call. | 14:18 |
| cppcabrera | zyuan did this in one of his test patches | 14:18 |
| cppcabrera | I'll look it up. | 14:18 |
| malini | cppcabrera: do I hear that we are already doing the stuff we just discusses? | 14:19 |
| malini | discussed* | 14:19 |
| cppcabrera | malini: I believe that's correct - I think zyuan is already doing some of this launching of proxy/queues_servers in the unit test suite. | 14:20 |
| cppcabrera | Check out this patch: https://review.openstack.org/#/c/46963/4/tests/unit/proxy/test_forwarding.py | 14:20 |
| cppcabrera | flaper87, kgriffs: We've got an operational request for modifying the proxy a little. | 14:22 |
| cppcabrera | The idea is to separate it into two wsgi apps - one that does almost strictly forwarding and the other that allows for partitions management. | 14:22 |
| cppcabrera | I've got on how to do this going off of the latest patch. | 14:22 |
| cppcabrera | I just wanted to check out what you guys think. | 14:22 |
| malini | cppcabrera: that is interesting..so it wouldn't make sense to repeat this in functional tests | 14:22 |
| cppcabrera | malini: marconi is a strange beast when it comes to testing - the lines between unit/functional tests are quite blurred. :) | 14:23 |
| malini | yeah :) | 14:24 |
| malini | its a good problem to have ;) | 14:24 |
| malini | cppcabrera: do you feel there areas in proxy that need to be in functional tests? | 14:25 |
| cppcabrera | hmm... | 14:25 |
| flaper87 | cppcabrera: mmh, not sure I follow | 14:26 |
| flaper87 | I mean, I understand what you want to do, I don't understand why that's needed | 14:26 |
| flaper87 | cppcabrera: malini re unit/functional I would prefer having both if possible and at least 1 of them, always | 14:27 |
| amitgandhi | flaper87: security purposes, the proxy admin calls are more sensitive which require bettter security around who can access them and how. the forwarding calls can be open to public | 14:27 |
| flaper87 | if there's a doubt whether a feature should be functionaly tested or not, it means it needs botht tests :D | 14:28 |
| flaper87 | mmmhhh | 14:28 |
| amitgandhi | if the partition management calls are in its own app, they can be deployed seperately to a secure server which is not accessible by public | 14:28 |
| cppcabrera | amitgandhi: thanks for explaining! | 14:28 |
| cppcabrera | That's the gist of it, flaper87. | 14:29 |
| flaper87 | I'd rather have an admin CLI tool for adding / removing partitions instead of 2 wsgi apps | 14:29 |
| cppcabrera | Lock down on /v1/partitions* | 14:29 |
| amitgandhi | we cant rely on just eom auth to provide the security as if that had a bug that allows someone to bypass the security, then they could add/delete partitions and create havoc | 14:29 |
| flaper87 | I see the need | 14:29 |
| flaper87 | mmh, ignore my last message | 14:30 |
| flaper87 | that would work but, it's definitely nice to have an admin API | 14:30 |
| amitgandhi | lol - damn i was about to reply to it haha | 14:30 |
| flaper87 | which we already have | 14:30 |
| flaper87 | and cppcabrera already worked on that | 14:30 |
| flaper87 | amitgandhi: HA! | 14:30 |
| flaper87 | :D | 14:30 |
| flaper87 | i mean, it'd be lot easier to have a CLI that talks directly to the database (as glance-cache work, nova db_sync, nova-manage etc) | 14:31 |
| flaper87 | but that requires the admin to ssh to the server and what not | 14:31 |
| cppcabrera | admin API (starts at L91 and ends at L101): https://review.openstack.org/#/c/47642/4/marconi/proxy/transport/wsgi/driver.py | 14:31 |
| flaper87 | if we have an API for the whole thing, it can be done remotely from a secure network | 14:31 |
| flaper87 | and it can be bound to private IPs | 14:31 |
| flaper87 | aaaaand whatnot | 14:31 |
| cppcabrera | +1 flaper87. :D | 14:31 |
| amitgandhi | yup thats our thought | 14:31 |
| amitgandhi | with the admin api | 14:32 |
| flaper87 | so, +1 from me to me and to cppcabrera and to amitgandhi and everyone in this channel for being sooo amazing | 14:32 |
| amitgandhi | lets celebrate with poptarts | 14:32 |
| cppcabrera | awesome | 14:32 |
| cppcabrera | my plan is to split that off and provide an admin_driver.py along with driver.py, an admin_app.py along with app.py, and an admin_bootstrap.py along with bootstrap.py | 14:33 |
| cppcabrera | I may end up using directories to make this feel more elegantly organized. :P | 14:33 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/marconi: Move queues' unittests under tests/unit/queues/ https://review.openstack.org/48065 | 14:33 |
| cppcabrera | yaaay, restructuring. :D | 14:34 |
| kgriffs | ok, so the idea here is to separate out the apis so you can run the admin one as a separate service and lock that sucker down | 14:34 |
| cppcabrera | kgriffs: yup! | 14:35 |
| kgriffs | is that the gist? | 14:36 |
| flaper87 | kgriffs: dude, we were already celebrating | 14:37 |
| flaper87 | T_T | 14:37 |
| kgriffs | rock on | 14:37 |
| flaper87 | or should I say, d000d | 14:37 |
| flaper87 | :D | 14:37 |
| cppcabrera | lol | 14:37 |
| * kgriffs goes to get some pop-tarts | 14:38 | |
| cppcabrera | yaaay | 14:38 |
| cppcabrera | kgriffs: you're heading over to ATL tomorrow, eh? | 14:38 |
| kgriffs | yep | 14:38 |
| flaper87 | kgriffs: you can also go and get some reviews, they are right next to pop-tarts | 14:38 |
| flaper87 | :D | 14:38 |
| kgriffs | lol | 14:38 |
| flaper87 | :D | 14:38 |
| cppcabrera | lol | 14:38 |
| cppcabrera | sweet - which reminds me... | 14:39 |
| cppcabrera | I'm considering hosting a py3k discussion tomorrow in the place of tech cinema and/or tech talks - want to join up for that? | 14:39 |
| flaper87 | kgriffs: btw, any news re Pecan / falcon discussion? I had to leave at some point | 14:40 |
| flaper87 | Can I? Can I? Can I??????? :D | 14:40 |
| flaper87 | Py3k FTW | 14:40 |
| cppcabrera | lol | 14:40 |
| cppcabrera | I'd love to have you join, flaper87. :D | 14:40 |
| flaper87 | btw, I was thinking to start running some `sed` commands on Marconi's codebase to improve PY3K compat | 14:41 |
| flaper87 | there are some really basic stuff we could start doing | 14:41 |
| cppcabrera | .encode/.decode, u'literal' | 14:41 |
| kgriffs | cppcabrera: i may not be there in time, but if i am, i'd love to join | 14:42 |
| cppcabrera | import future_builtins | 14:42 |
| cppcabrera | kgriffs: around what time do you expect to arrive? | 14:42 |
| kgriffs | flaper87: re wsgi framework, i just touched bases with mordred and assured him that the pecan driver was a "real thing" | 14:42 |
| kgriffs | he just didn't want us to say "it's too hard to switch" | 14:43 |
| kgriffs | so, the plan of record is to write up enough of a Pecan-based driver so we can benchmark and also gauge developer productivity and stuff | 14:43 |
| kgriffs | if Pecan is awesome and fast enough we will use it | 14:43 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Authentication API + NoAuth backend https://review.openstack.org/47334 | 14:44 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Add keystone backend https://review.openstack.org/48041 | 14:44 |
| kgriffs | otherwise, we will need to decide as a community whether to: | 14:44 |
| kgriffs | a. fix Pecan (if it is fixable) | 14:44 |
| kgriffs | b. allow three frameworks (Pecan, Falcon, swob) | 14:44 |
| kgriffs | (and by being fixable I mean not totally breaking backwards compatibility or something) | 14:45 |
| flaper87 | ok, that's exactly what I tried to explain to them, I guess I didn't explained it well :P | 14:45 |
| flaper87 | anyway +1 | 14:45 |
| flaper87 | I like the approach (jus as we mentioned in the TC meeting) | 14:45 |
| cppcabrera | +1 here, too. I like that there's a plan in place and that the talk has been talked. | 14:45 |
| flaper87 | btw, what about that 'requirements' patch? | 14:45 |
| flaper87 | will that get in soon ? | 14:45 |
| kgriffs | so | 14:45 |
| kgriffs | so, we have 3 patches there | 14:46 |
| kgriffs | I think the falcon one can go in if we want to get an exception | 14:46 |
| kgriffs | we could also get mmh3 in if we need it, but I'm thinking we could just use crc32 and call it a day | 14:46 |
| kgriffs | and I'm not convinced we need pylru at all | 14:46 |
| kgriffs | i can haz data? | 14:47 |
| kgriffs | (someone needs to benchmark) | 14:47 |
| cppcabrera | pylru is not needed unless we use pypy + mmh3. | 14:47 |
| cppcabrera | If we use crc32, we're good. | 14:48 |
| cppcabrera | (wanted to run ipypy) | 14:48 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: This adds a transport layer request spec class https://review.openstack.org/47538 | 14:49 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Authentication API + NoAuth backend https://review.openstack.org/47334 | 14:49 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Refactor client's structure to reflect Marconi's https://review.openstack.org/47169 | 14:49 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Add keystone backend https://review.openstack.org/48041 | 14:49 |
| kgriffs | cppcabrera: +1 | 14:49 |
| kgriffs | and we can use try_import to use zlib.crc32 and fallback to binascii | 14:49 |
| kgriffs | btw, I did a little test | 14:49 |
| kgriffs | assuming we mod over ~4 dbs | 14:49 |
| cppcabrera | ooohhh... | 14:49 |
| kgriffs | the distribution looks pretty uniform for crc32 | 14:49 |
| cppcabrera | under pypy, binascii.crc32 runs faster than zlib.crc32 | 14:50 |
| kgriffs | so, we don't really need a fancy hash in this case anyway | 14:50 |
| cppcabrera | +1 kgriffs - that was my concern with crc32 | 14:50 |
| kgriffs | cppcabrera: lol | 14:50 |
| cppcabrera | I'm happy to hear the distribution is satisfactory | 14:50 |
| kgriffs | yep | 14:50 |
| kgriffs | I think you only get in trouble when you have a lot of buckets to hash into | 14:50 |
| cppcabrera | -1 to mmh3, -1 to pylru | 14:50 |
| flaper87 | yeah, tried to point that in the review. Also, depending on more external things makes installing marconi more painful | 14:50 |
| flaper87 | if we don't need those patches anymore, lets make sure we comment there asap | 14:51 |
| kgriffs | i will say that mmh3 is still a little faster than crc32, but i am hoping that 200 nanoseconds won't matter | 14:51 |
| kgriffs | :p | 14:51 |
| flaper87 | so -infra guys don't approve them | 14:51 |
| mpanetta | 200ns is unacceptable! :P | 14:52 |
| flaper87 | huahuahuahua | 14:52 |
| kgriffs | http://paste.openstack.org/show/47386/ | 14:52 |
| kgriffs | and… crc32 screams on pypy, unlike mmh3 | 14:52 |
| cppcabrera | flaper87, kgriffs: I -1'd pylru and mmh3 patches | 14:52 |
| kgriffs | although, I am still tempted to add cffi support to mmh3 | 14:53 |
| kgriffs | just for fun | 14:53 |
| cppcabrera | over at openstack/requirements | 14:53 |
| flaper87 | cppcabrera: thanks | 14:53 |
| cppcabrera | kgriffs: go for it, and then put together a lightning talk about porting packages to cffi | 14:53 |
| cppcabrera | "cffi: For the Rest of Us" | 14:53 |
| cppcabrera | Regarding the openstack design summits... that page makes me feel like I should only submit a design session proposal if I'm attending. >.> | 14:54 |
| *** reed has joined #openstack-marconi | 14:54 | |
| cppcabrera | Is that the case, flaper87? ^ | 14:54 |
| * cppcabrera is not attending HKG | 14:55 | |
| kgriffs | cppcabrera: thanks for the comments! | 14:55 |
| flaper87 | cppcabrera: that's the case but, in our case, I think it is fair to submit topics that folks attending could discuss as well | 14:56 |
| cppcabrera | alright, flaper87, then I'll go through with submitting the alternate transports design session proposal. :) | 14:57 |
| flaper87 | cppcabrera: +1 :D | 14:57 |
| flaper87 | we could also prepare an etherpad as reference for your proposal | 14:57 |
| cppcabrera | cool, I'd like that | 14:58 |
| cppcabrera | I'll bring one up now, though I won't write too much atm. I'll work on that proposal over the duration of the day, while I work on those proxy fixes/updates. | 14:58 |
| cppcabrera | https://etherpad.openstack.org/marconi-alternate-transports | 14:58 |
| flaper87 | cppcabrera: erm, re breaking py3k | 15:12 |
| flaper87 | it's not my fault, keystoneclient breaks it | 15:12 |
| flaper87 | I was thinking about not depending directly on it | 15:12 |
| cppcabrera | I figured it was something like that, flaper87. :/ | 15:12 |
| flaper87 | so pissed off about that | 15:13 |
| flaper87 | arrggh | 15:13 |
| cppcabrera | we need keystoneclient for easy auth, but as I understand, keystoneclient isn't quite py3k-compliant yet. Is this presumption correct? | 15:13 |
| flaper87 | yeah! | 15:14 |
| flaper87 | Marconi's default behavior is no-auth | 15:14 |
| cppcabrera | I'd take working auth right now over py3k-compat, as long as marconiclient code itself doesn't break py3k compat. | 15:14 |
| flaper87 | but lets face it, noone will deploy marconi on production without auth | 15:14 |
| ametts | flaper87: Here's what's on my whiteboard for potential summit design session topics: | 15:14 |
| flaper87 | so, the default auth should be keystone | 15:14 |
| ametts | 1) Support for notification semantics (e.g., foghorn) | 15:15 |
| cppcabrera | hmmm... | 15:15 |
| flaper87 | cppcabrera: got ad porn idea :) | 15:15 |
| flaper87 | cppcabrera: what if we skip that test for py3k ? | 15:15 |
| cppcabrera | that'd be sweet, flaper87. :D | 15:15 |
| ametts | 2) Direction on alternate transports/drivers | 15:15 |
| flaper87 | ametts: al-maisan already proposed the foghorn one :) | 15:15 |
| flaper87 | ametts: +1 for the 2nd | 15:15 |
| al-maisan | yup :) | 15:15 |
| cppcabrera | ametts: I'm writing up the 2nd proposal. | 15:15 |
| flaper87 | al-maisan: weren't you on a plain? | 15:15 |
| al-maisan | on and off :) | 15:15 |
| flaper87 | cppcabrera: btw, https://review.openstack.org/#/c/48057/ | 15:16 |
| flaper87 | cppcabrera: the patch depending on that one does exactly what you asked | 15:16 |
| flaper87 | :) | 15:16 |
| ametts | 3) Routing and Exchange protocol (this is yours I think) | 15:16 |
| *** jraim has joined #openstack-marconi | 15:16 | |
| flaper87 | ametts: correct | 15:16 |
| al-maisan | flaper87: the flight was delayed for 2.5 hours because of fog in Heathrow :-P | 15:16 |
| cppcabrera | I'll brb. | 15:16 |
| ametts | 4) Message Tagging/filtering (a feature we were considering in the early days) | 15:16 |
| flaper87 | al-maisan: ah, that lovely weather of London | 15:17 |
| al-maisan | indeed :) | 15:17 |
| flaper87 | ametts: I also added this http://summit.openstack.org/cfp/details/49 and this http://summit.openstack.org/cfp/details/48 | 15:17 |
| * al-maisan needs to run | 15:17 | |
| al-maisan | bye for now! | 15:17 |
| flaper87 | I think we could re-order some of those | 15:17 |
| ametts | 5) Other "big" features we were considering as part of the roadmap (I haven't looked to see what those might be yet) | 15:17 |
| flaper87 | al-maisan: have a safe flight | 15:17 |
| flaper87 | or squash them | 15:17 |
| flaper87 | ametts: I think 4 and 3 can be squashed | 15:18 |
| flaper87 | I can see messages being routed based on their tags | 15:19 |
| kgriffs | +! | 15:19 |
| kgriffs | +1 | 15:19 |
| amitgandhi | is eom worth a talk? | 15:19 |
| kgriffs | I think we could just have a single session for "roadmap" or something | 15:19 |
| kgriffs | so we can have a forum to discuss what's next, get feedback on priorities, suggestions from the community | 15:20 |
| cppcabrera | back | 15:21 |
| flaper87 | kgriffs: that sounds like a good idea, I'd also have a backup plan for that session | 15:21 |
| cppcabrera | +1 for roadmap session | 15:21 |
| flaper87 | something like, if no one comes to that session we should talk about X | 15:22 |
| ametts | flaper87, kgriffs: So is it really ok to submit these design session proposals as one-liners like this? I haven't submitted anything, because I thought they needed to be well-thought-out with blueprint refrerences and such. | 15:22 |
| cppcabrera | I know getting non-GPL drivers in is a big deal to some deployments. | 15:22 |
| ametts | ... which seems like work. | 15:22 |
| ametts | ... and I haven't had time for work. :) | 15:22 |
| flaper87 | ametts: you can submit them, they'll then be approved /rejected by kgriffs | 15:22 |
| flaper87 | I'm pretty sure kgriffs will reject all your proposals | 15:22 |
| kgriffs | I don't think many of the proposals have elaborate descriptions | 15:23 |
| flaper87 | because you aint buy him nutella, which I'm doing | 15:23 |
| kgriffs | LOL | 15:23 |
| ametts | Yeah, but I do his performance reviews. :D | 15:23 |
| * kgriffs accepts bribes | 15:23 | |
| flaper87 | damnit | 15:23 |
| kgriffs | ametts: excellent point | 15:23 |
| * flaper87 just saw his sessions flying away | 15:23 | |
| cppcabrera | lol | 15:24 |
| *** vkmc has quit IRC | 15:25 | |
| flaper87 | ametts: btw, no long description is required. It's important that people understand what we'll discuss in the session so they can decided wether to attend or not | 15:25 |
| flaper87 | I think you can use real titles and then put "We'll give nutella, wine, beer and stickers away" in the description | 15:26 |
| * amitgandhi nutella and beer is a weird combo | 15:26 | |
| cppcabrera | that'll bring people to the design session. :P | 15:26 |
| ametts | flaper87: LOL You have everything submitted that you wanted to submit? Maybe when kgriffs is in Atlanta, we can hone/refine the four slots into decently-described session submissions that we really want to spend the time on. | 15:26 |
| cppcabrera | ametts: https://etherpad.openstack.org/marconi-alternate-transports | 15:26 |
| flaper87 | amitgandhi: hahaha, it depends. You can start with nutella, take some rest, and then start with beer / wine | 15:27 |
| flaper87 | ametts: I submitted 2, I haven't submitted the routing one | 15:27 |
| ametts | Does anyone but kgriffs review/approve them? | 15:27 |
| flaper87 | ametts: you're siriously considering to use the description I mentioned, aren't you? | 15:28 |
| flaper87 | :D | 15:28 |
| kgriffs | brainstorming page for that roadmap session thingy: https://etherpad.openstack.org/marconi-summit-roadmap | 15:28 |
| kgriffs | feel free to add/update/reference | 15:28 |
| flaper87 | ametts: mmh, as for the approval process, I think it's completely up to the PTL | 15:28 |
| ametts | flaper87: Whatever gets people in the door. :D | 15:29 |
| flaper87 | seriously* | 15:29 |
| ametts | And are we sure we have EXACTLY four sessions to fill? No one at OpenStack will decide to give us more or less after everything is submitted? | 15:29 |
| kgriffs | Thierry said he thought we should have 4 but if some other projects get incubated, we may need to give up a slot | 15:31 |
| kgriffs | so, I'd plan on 3 slots, maybe 4 | 15:31 |
| kgriffs | and there's always the unconference slots we can try to nab | 15:32 |
| cppcabrera | as well as the hallway track. ;) | 15:33 |
| ametts | kgriffs: Let's plan on 4,-5 with 1-2 being lower-priority. If it doesn't look like we have the slots filled with meaningful discussions, we're more likely to lose the time. | 15:33 |
| ametts | We can come up with session topics like "Achieve world peace", "End the middle east crisis", "solve world hunger" | 15:34 |
| ametts | Those would be WAY too important to eliminate. | 15:34 |
| cppcabrera | lol | 15:35 |
| cppcabrera | ametts: They'd just be delegated to another conference. :P | 15:35 |
| * cppcabrera sees the news reports - "OS does not support world peace!" | 15:35 | |
| ametts | ... but they solved World Hunger with gnutella for everyone. | 15:36 |
| cppcabrera | Nutella, pop tarts, and beer - let it be known that team marconi is decadent. :D | 15:36 |
| cppcabrera | (and has a sweet tooth) | 15:37 |
| flaper87 | ametts: http://en.wikipedia.org/wiki/Gnutella ? :D | 15:37 |
| flaper87 | the shipped nutella through gnutella | 15:37 |
| cppcabrera | A FOSS variant on Nutella? Copyleft sharing? | 15:37 |
| * cppcabrera clicks | 15:37 | |
| cppcabrera | Oh, right - the p2p sharing system. :P | 15:37 |
| cppcabrera | I'd forgotten about that package. | 15:38 |
| ametts | flaper87: I was hoping someone would notice how clever I was to work nutella into a legitimate discussion on open source | 15:38 |
| * cppcabrera imagines copyleft sharing for Nutella | 15:38 | |
| flaper87 | ametts: LOL | 15:38 |
| cppcabrera | lol | 15:38 |
| amitgandhi | github for recipes: nutella cookies: http://forkthecookbook.com/recipes/paleo-nutella-cookies-bcfa1 | 15:39 |
| cppcabrera | woah, paleo nutella - that's insane. | 15:41 |
| kgriffs | guys, I'm thinking that Pecan driver needs to happen before the summit | 15:45 |
| kgriffs | I need some data to bring with me | 15:45 |
| flaper87 | kgriffs: I can dedicate some time to that | 15:45 |
| amitgandhi | is that worth a dedicated talk: wsgi frameworks for openstack | 15:46 |
| ametts | kgriffs: Anyone on the Pecan team that might be interested in helping out? | 15:46 |
| *** ayoung has joined #openstack-marconi | 15:47 | |
| kgriffs | ametts: thing is, we need someone who has experience with Falcon to do it so we can compare their experience in writing both | 15:48 |
| kgriffs | I want to know, for example, which one is more productive | 15:49 |
| kgriffs | the one thing we can't measure i guess is how easy it is to debug production problems | 15:49 |
| kgriffs | i have heard from the swift guys that webob obscures some details that can make it hard to diagnose issues | 15:50 |
| kgriffs | but, I don't know of a great way to verify that. :p | 15:50 |
| flaper87 | mmh, not sure if I'm that *experienced* with falcon | 15:51 |
| flaper87 | I'd definitely have this in a separate repo, though | 15:51 |
| flaper87 | as a third-party transport | 15:51 |
| flaper87 | until we figure out the whole thing | 15:51 |
| flaper87 | HA, guess who just restored PY3K compat in marconiclient... | 15:52 |
| cppcabrera | flaper87: you? ;) | 15:52 |
| flaper87 | :D | 15:52 |
| cppcabrera | :D | 15:52 |
| kgriffs | so… I can do the pecan thing, but I'm not exactly a neutral party. :p | 15:54 |
| ametts | Is Marconi itself PY3 - compatible? | 15:54 |
| flaper87 | kgriffs: lets let the numbers speak and do our best | 15:54 |
| flaper87 | ametts: not yet | 15:54 |
| flaper87 | the client is though | 15:54 |
| flaper87 | and we're seriously meaning to keep it that way | 15:55 |
| flaper87 | cppcabrera: why does the pt.get_string thing break Py3K ? | 15:55 |
| flaper87 | I think I'm missing something there | 15:55 |
| * ametts just heard flaper87 say we have a client. Sounds like we're "done" and ready for launch. :D | 15:56 | |
| cppcabrera | flaper87: It's not the pt.get_string | 15:56 |
| cppcabrera | It's the print statement next to it. | 15:56 |
| flaper87 | now I'm confused, don't see any print | 15:57 |
| flaper87 | cppcabrera: https://review.openstack.org/#/c/47334/5/marconiclient/common/utils.py | 15:57 |
| flaper87 | line 53 | 15:57 |
| * cppcabrera double checks | 15:59 | |
| cppcabrera | "print pt.get_string(sortby=field_labels[sortby])" | 15:59 |
| cppcabrera | ^^^^^ | 15:59 |
| flaper87 | T_T | 15:59 |
| flaper87 | holy moly, I need a doctor | 15:59 |
| flaper87 | seriously | 15:59 |
| cppcabrera | haha, I missed it the first time around, too, heh. | 15:59 |
| flaper87 | somehow I read `return` instead of print | 16:00 |
| cppcabrera | I just read 'pt.get_string...'. My mind's eye did not want to acknowledge that print there, but I knew there had to be one because the function is named 'print_list'. | 16:00 |
| cppcabrera | So Ctrl+F to save the day | 16:01 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Authentication API + NoAuth backend https://review.openstack.org/47334 | 16:01 |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Add keystone backend https://review.openstack.org/48041 | 16:01 |
| flaper87 | there you go, Py3K happy patches | 16:01 |
| cppcabrera | request spec - I see, that ties in with al-maisan's patch. | 16:02 |
| cppcabrera | The notion of request spec continues to confuse me. | 16:02 |
| cppcabrera | Is that a pattern that's been used in other client impls.? | 16:02 |
| cppcabrera | It just seems like an unnecessary layer of indirection. :x | 16:02 |
| cppcabrera | flaper87: +2'd | 16:04 |
| flaper87 | no, that's something al-maisan and I came up with during our Geneva's brainstorm (I just mentioned Geneva to make kgriffs jealous) | 16:04 |
| flaper87 | cppcabrera: so, the whole idea is to be able to go through all the auth and request prepartion chain 'til we get to the transport without any transport specific object | 16:05 |
| flaper87 | that abstraction will allow us to have a more common API between transport and it'll make it easier to split the client API in 2 levels | 16:05 |
| flaper87 | a higher level API and a lower level one | 16:06 |
| flaper87 | in other words, when we get to the `authenticate` call, we still don't know what transport we will use | 16:06 |
| cppcabrera | cool, cool - that makes sense. | 16:06 |
| flaper87 | for instance, keystone could return a zmq://blah.blah.blha endpoint | 16:06 |
| flaper87 | which we'll then parse and load the right transport for it | 16:07 |
| cppcabrera | What bugs me is the "operation" aspect of it. | 16:07 |
| flaper87 | cppcabrera: ah yeah, that part is not 100% clear yet | 16:07 |
| cppcabrera | Where we seem to be duplicating the work of the Python interpreter | 16:07 |
| cppcabrera | optional vs. required args, arg names, etc. | 16:07 |
| flaper87 | ah, got it. Well, IMHO, endpoint and operation should always be required | 16:08 |
| flaper87 | we won't be able to succeed without knowing what to do and where to do it | 16:08 |
| cppcabrera | +1 flaper87 - some way to identify the endpoint, and some way to match up to an operation. | 16:08 |
| cppcabrera | I'm just concerned about the way we identify the operation atm. | 16:08 |
| flaper87 | yeah, TBH, I'm a bit concerned about that. My thinking is that we'll improve that code when we'll start using it | 16:09 |
| flaper87 | which means, the next couple of patches or so | 16:09 |
| cppcabrera | Ah, I can totally understand. :D | 16:09 |
| * cppcabrera went through that messy improve/revise process with the proxy | 16:10 | |
| cppcabrera | I'll remove my -1 on that request spec patch. | 16:10 |
| malini | cppcabrera: qn abt proxy - do we store the queue metadata in the catalogue? | 16:11 |
| cppcabrera | flaper87: I'll leave a few comments on the request spec patch along the lines of => I feel like using some combination of enums and/or interfaces and/or dicts + lambdas would be easier to use. | 16:12 |
| cppcabrera | malini: yes, for now | 16:12 |
| flaper87 | cppcabrera: +1 thanks for that feedback! | 16:12 |
| flaper87 | I just +2'd that patch | 16:12 |
| malini | cppcabrera: why ? (sorry, if its already dissussed) | 16:12 |
| cppcabrera | malini: It's currently needed to make the listing operation efficient (e.g. GET /v1/queues), because only the proxy knows where all the queues for PROJECT_ID live. | 16:13 |
| cppcabrera | malini: There's a cool idea in the works to eventually replace that by using a concurrent request/merge operation. | 16:13 |
| cppcabrera | That way, metadata won't have to be stored any more. | 16:13 |
| *** mpanetta is now known as mpanetta_afk | 16:14 | |
| cppcabrera | At the proxy/catalogue level, anyway. | 16:14 |
| *** mpanetta_afk is now known as mpanetta_lunch | 16:14 | |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Refactor client's structure to reflect Marconi's https://review.openstack.org/47169 | 16:14 |
| flaper87 | cppcabrera: after commenting on that patch, could you take a look here as well? https://review.openstack.org/#/c/48041/ | 16:14 |
| cppcabrera | will do, flaper87 | 16:16 |
| flaper87 | cppcabrera: thank you sir! | 16:17 |
| cppcabrera | flaper87: regarding enums, I learned this is the official backport - https://pypi.python.org/pypi/enum34/ | 16:17 |
| cppcabrera | :) | 16:17 |
| *** iampims has left #openstack-marconi | 16:18 | |
| flaper87 | cppcabrera: awesome, I think it'll be definitely more organized with enums | 16:18 |
| Alex_Gaynor | So it looks like marconiclient's tests currently pass on py33, do we want to make that voting now, to ensure it doesn't regress, or are we not sure we're ready to commit to that? | 16:20 |
| cppcabrera | Alex_Gaynor: I'm +2 on commiting to voting w/ gate-py33, esp. since flaper87 proposed skipping non-py33 compatible test cases atm using conditionals. | 16:21 |
| cppcabrera | flaper87: thoughts? | 16:21 |
| flaper87 | Alex_Gaynor: I was about to do that :D | 16:21 |
| flaper87 | Alex_Gaynor: go ahead | 16:21 |
| cppcabrera | woot | 16:21 |
| Alex_Gaynor | flaper87: Ah! Well, don't let me stop you :) | 16:21 |
| cppcabrera | let's do it! | 16:21 |
| openstackgerrit | A change was merged to openstack/python-marconiclient: This adds a transport layer request spec class https://review.openstack.org/47538 | 16:21 |
| flaper87 | kgriffs: could you take a look at this 2 patches https://review.openstack.org/#/q/status:open+project:openstack/python-marconiclient,n,z | 16:22 |
| cppcabrera | flaper87: I left a hefty thought-comment w/ the request spec patch. :D | 16:22 |
| flaper87 | cppcabrera: I'll copy that comment into the client etherpad, don't want it to get lost | 16:23 |
| cppcabrera | +1 flaper87 | 16:23 |
| cppcabrera | I lost the client etherpad. >.> | 16:23 |
| * cppcabrera bets it's linked on the wiki | 16:23 | |
| flaper87 | cppcabrera: https://etherpad.openstack.org/marconi-client-api | 16:23 |
| flaper87 | cppcabrera: and yes, it's linked on the wiki | 16:24 |
| flaper87 | under the Python client page | 16:24 |
| *** jcru has joined #openstack-marconi | 16:24 | |
| cppcabrera | which reminds me... | 16:24 |
| * cppcabrera links proxy issues pad to front page of wiki | 16:24 | |
| cppcabrera | thanks, flaper87. I cleaned up the comment a little to make it 10% more readable. | 16:26 |
| cppcabrera | flaper87: keystone auth patch reviewed - +2 | 16:29 |
| flaper87 | cppcabrera: w0000000t | 16:30 |
| flaper87 | thanks a lot | 16:30 |
| flaper87 | Alex_Gaynor: https://review.openstack.org/#/c/48089/ | 16:30 |
| flaper87 | cppcabrera: ^ | 16:30 |
| Alex_Gaynor | flaper87: boom! | 16:30 |
| flaper87 | Alex_Gaynor: danke, sir! :) | 16:31 |
| cppcabrera | awesome, Alex_Gaynor! | 16:34 |
| cppcabrera | flaper87: requesting review on this proxy patch where I add oslo.cache: https://review.openstack.org/#/c/47475/ | 16:34 |
| cppcabrera | I've already gone through and made note of the big things that are fixed in the patch that follows from it. | 16:35 |
| * flaper87 looking at it | 16:36 | |
| * cppcabrera is very thankful that the proxy is starting to stabilize and that the patches are getting smaller as a result | 16:37 | |
| flaper87 | cppcabrera: we may want to have a separate config file for the proxy | 16:40 |
| flaper87 | marconi-proxy.conf | 16:40 |
| cppcabrera | That thought crossed my mind. | 16:41 |
| cppcabrera | How would I go about making that happen? | 16:41 |
| flaper87 | not 100% sure about that but I'm 65% in favor of splitting those files | 16:41 |
| cppcabrera | The split feels even more important now that there's going to be an admin API and a public API for the proxy | 16:41 |
| cppcabrera | because they're both going to require separate bind:port pairs | 16:42 |
| cppcabrera | Another note - for small use cases, there's no reason a user should ever need to see the proxy configuration details | 16:43 |
| cppcabrera | That's my primary rationale for having >1 conf files | 16:43 |
| flaper87 | cppcabrera: yeah! | 16:44 |
| flaper87 | cppcabrera: so, 2 things about the config code | 16:44 |
| flaper87 | 1) I'd prefer not using marconi.common.config, TBH, I think we should get rid of that code. | 16:45 |
| *** ykaplan has quit IRC | 16:46 | |
| flaper87 | 2) Lets not use a global config instance. The proxy is just starting so we can work on that since the very beginning | 16:46 |
| cppcabrera | re: 2, hmm... | 16:46 |
| cppcabrera | How's this look? https://review.openstack.org/#/c/47642/4/marconi/proxy/transport/wsgi/driver.py | 16:46 |
| cppcabrera | That's the updated version. | 16:46 |
| cppcabrera | Ahh, I see. | 16:46 |
| cppcabrera | Yeah, that's still global | 16:47 |
| flaper87 | :D | 16:47 |
| *** reed has quit IRC | 16:47 | |
| malini | I am running the tests with tox locally &am getting errors with py26 on the tests I am trying to add - ' 'expected count = {}'.format(expectedCount) + | 16:47 |
| malini | ValueError: zero length field name in format' . Any pointers? | 16:47 |
| cppcabrera | re: 2) again - how do you propose we handle the configuration? Will it be similar to oslo.cache.get_cache() | 16:47 |
| flaper87 | and 3) You can tell oslo.config what file you want to load, if you don't it'll try to load sys.argv[0].conf | 16:47 |
| cppcabrera | malini: '{0}'.format | 16:47 |
| flaper87 | malini: {0} | 16:47 |
| cppcabrera | py26 doesn't support empty format specifiers | 16:48 |
| flaper87 | damnit | 16:48 |
| cppcabrera | lol | 16:48 |
| flaper87 | :D | 16:48 |
| malini | cppcabrera: so what do we do about it? | 16:48 |
| flaper87 | cppcabrera: hope that feedback about the config makes sense | 16:48 |
| flaper87 | malini: what tests are you running? | 16:48 |
| flaper87 | seems weird it passed through the gate w/o failing | 16:48 |
| cppcabrera | malini: 'expected count = {0}'.format(expectedCount) | 16:49 |
| cppcabrera | that should fix it | 16:49 |
| cppcabrera | where 0 is the first argument passed to .format | 16:49 |
| malini | flaper87: it didnt pass thru the gate :) | 16:49 |
| flaper87 | malini: lol | 16:49 |
| flaper87 | cppcabrera: how much does the proxy code depend on the global instance? | 16:50 |
| cppcabrera | flaper87: the driver portion of it should encapsulate all the config deps | 16:51 |
| cppcabrera | So lessee... | 16:51 |
| cppcabrera | storage controllers depend on configuration | 16:51 |
| cppcabrera | cache driver depends on configuration | 16:52 |
| cppcabrera | auth strategy | 16:52 |
| cppcabrera | ^^^^ ^^^^^^^^ which is only ref'd in the driver.py | 16:52 |
| cppcabrera | that should be it. So to make it non-global config, it'd just be a matter of adding a new param to the driver __init__ for the transport and the storage | 16:53 |
| cppcabrera | flaper87: ^ | 16:53 |
| flaper87 | is there an easy way to pass a config instance to all the instances needing one ? | 16:53 |
| *** EmilienM has quit IRC | 16:53 | |
| flaper87 | erm | 16:53 |
| flaper87 | I just missed your last message | 16:53 |
| flaper87 | I guess I'm just starving | 16:54 |
| cppcabrera | lol | 16:54 |
| flaper87 | and should stop writing :D | 16:54 |
| cppcabrera | It'd just take that and modifying the app.py/admin_app.py slightly. | 16:54 |
| flaper87 | cppcabrera: do you see that like something we could have now, as in current patch ? | 16:54 |
| flaper87 | I'd prefer so, if possible | 16:54 |
| *** EmilienM has joined #openstack-marconi | 16:54 | |
| cppcabrera | flaper87: I need to give priority to the admin separation patch atm - otherwise, I'd work on that now. | 16:55 |
| cppcabrera | So to answer your exact question... hmm... | 16:55 |
| flaper87 | cppcabrera: I'm happy to let it land as-is as long as newer patches don't depend on the global instance. I can help you with the to-non-global migration thing | 16:57 |
| cppcabrera | I'd like to tackle it as a separate patch, since the theme would be clear - rm global conf. | 16:57 |
| flaper87 | if you want | 16:57 |
| flaper87 | cppcabrera: kk | 16:57 |
| flaper87 | sounds good | 16:57 |
| cppcabrera | thanks, flaper87! | 16:57 |
| cppcabrera | Also requesting assistance on the whole notion of making separate conf files. | 16:57 |
| flaper87 | cppcabrera: could you create a bug for that? | 16:57 |
| flaper87 | just to make sure we don't forget about it, I'll pick it and help you out | 16:57 |
| cppcabrera | will do | 16:57 |
| flaper87 | awesome | 16:58 |
| cppcabrera | That's kind of cool - we can start registering proxy bugs since it's mostly in the repo now. :D | 16:58 |
| flaper87 | cppcabrera: yeaaahhh!!! | 16:58 |
| flaper87 | btw, re splitting confs, it's just a matter of telling ConfOpts what file it should load, that sounds like something I'd do after getting rid of the global instance | 16:59 |
| cppcabrera | cool, cool | 16:59 |
| * flaper87 puts his hand on his heart, looks at the sky and says with a very promissing voice: "For a better world without globals" | 16:59 | |
| cppcabrera | I'm starting to get familiar with the oslo.config system | 16:59 |
| cppcabrera | flaper87: Hahahaha | 16:59 |
| cppcabrera | Beautifully said. :') | 16:59 |
| cppcabrera | It brings a tear to my eyes, and resonates with my functional heart. :) | 17:00 |
| flaper87 | hahahaha! +1 | 17:00 |
| flaper87 | damn, gotta tweet that | 17:00 |
| cppcabrera | woooo | 17:01 |
| flaper87 | brb, dinner | 17:02 |
| cppcabrera | k | 17:02 |
| *** reed has joined #openstack-marconi | 17:08 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: perf(mongodb): Combine project and queue message fields https://review.openstack.org/47259 | 17:18 |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: fix(mongodb): Marker generation creates a bottleneck under heavy load https://review.openstack.org/47272 | 17:18 |
| cppcabrera | kgriffs: looks like patchset 8 on the markers adds a lot of mongo tests, | 17:23 |
| cppcabrera | @testing.slow_test - nice | 17:24 |
| cppcabrera | ! | 17:24 |
| kgriffs | :D | 17:26 |
| kgriffs | so, the changes are just in the tests | 17:26 |
| kgriffs | per our discussion earlier today | 17:27 |
| cppcabrera | I'm really digging the decorator approach | 17:27 |
| kgriffs | excellent | 17:28 |
| *** mpanetta_lunch is now known as mpanetta | 17:28 | |
| cppcabrera | I think using the lambda as a default_arg is both self-documenting and safe, since lambdas aren't modifiable. | 17:28 |
| cppcabrera | ref: slow_test(is_slow=lambda ...) | 17:28 |
| kgriffs | the one thing is that you always have to say slow_test() instead of just slow_test | 17:29 |
| kgriffs | (when you want to use the default) | 17:29 |
| kgriffs | I couldn't think of a way around that, tho | 17:30 |
| cppcabrera | that's fine - IMO | 17:30 |
| cppcabrera | kgriffs: +2 from me | 17:32 |
| kgriffs | w00t | 17:36 |
| cppcabrera | ametts, kgriffs, flaper87: https://etherpad.openstack.org/marconi-alternate-transports (proposal text added for summit design session, requesting review) | 17:36 |
| kgriffs | I gotta run for a bit - be back soon | 17:37 |
| cppcabrera | see ya, kgriffs. :) | 17:37 |
| ametts | kgriffs: Don't forget Queues meeting in 23 mins. | 17:37 |
| *** kgriffs is now known as kgriffs_afk | 17:37 | |
| * ametts is pretty sure he didn't see that | 17:38 | |
| cppcabrera | brb meeting | 18:03 |
| *** amitgandhi has quit IRC | 18:03 | |
| *** amitgandhi has joined #openstack-marconi | 18:04 | |
| *** amitgandhi1 has joined #openstack-marconi | 18:05 | |
| *** amitgandhi1 has quit IRC | 18:05 | |
| *** amitgandhi1 has joined #openstack-marconi | 18:05 | |
| *** acabrera has joined #openstack-marconi | 18:06 | |
| *** cppcabrera has quit IRC | 18:07 | |
| *** amitgandhi has quit IRC | 18:08 | |
| *** kgriffs_afk is now known as kgriffs | 18:14 | |
| *** kgriffs is now known as kgriffs_afk | 18:16 | |
| *** gordonsim has quit IRC | 18:45 | |
| *** kgriffs_afk is now known as kgriffs | 18:57 | |
| *** kgriffs is now known as kgriffs_afk | 19:02 | |
| *** kgriffs_afk is now known as kgriffs | 19:05 | |
| *** pycabrera has joined #openstack-marconi | 19:12 | |
| *** pycabrera is now known as cppcabrera | 19:13 | |
| cppcabrera | woot | 19:13 |
| cppcabrera | I'm out for the day. I've got some errands to take care of. | 19:13 |
| cppcabrera | I'll tackle the admin API changes tomorrow. | 19:14 |
| cppcabrera | Take care~ | 19:14 |
| *** cppcabrera has quit IRC | 19:14 | |
| *** acabrera has quit IRC | 19:15 | |
| *** malini is now known as malini_afk | 19:35 | |
| *** russell_h has quit IRC | 19:58 | |
| *** russell_h has joined #openstack-marconi | 20:00 | |
| kgriffs | flaper87: this one is ready for final review. I updated the tests per our discussion earlier. | 20:01 |
| kgriffs | https://review.openstack.org/#/c/47272/ | 20:01 |
| kgriffs | mpanetta: could you benchmark this patch for me (long test): https://review.openstack.org/#/c/47259/ | 20:02 |
| kgriffs | ? | 20:02 |
| *** cppcabrera has joined #openstack-marconi | 20:02 | |
| flaper87 | kgriffs: awesome, thanks. I'll take a look at it | 20:03 |
| kgriffs | kk | 20:05 |
| mpanetta | Yeah, in a bit | 20:05 |
| kgriffs | thanks! | 20:05 |
| mpanetta | I think I am going to have to give up on malini_afk's test env right now. Salt is abusing me. | 20:06 |
| *** ayoung has quit IRC | 20:09 | |
| *** ayoung has joined #openstack-marconi | 20:24 | |
| openstackgerrit | A change was merged to openstack/marconi: Move test_default_limits to the new tests location https://review.openstack.org/48059 | 20:34 |
| *** russell_h has left #openstack-marconi | 20:36 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack/marconi: Remove reference to oslo.config beta tarball https://review.openstack.org/48142 | 21:00 |
| flaper87 | kgriffs: hey I replied to your comments here: https://review.openstack.org/#/c/47334/ | 21:07 |
| flaper87 | kgriffs: thanks for reviewing those patches | 21:07 |
| *** kgriffs is now known as kgriffs_afk | 21:08 | |
| *** oz_akan_ has quit IRC | 21:09 | |
| *** kgriffs_afk is now known as kgriffs | 21:12 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack/marconi: Require oslo.config 1.2.0 final https://review.openstack.org/48142 | 21:15 |
| *** cppcabrera has quit IRC | 21:21 | |
| *** ayoung has quit IRC | 21:23 | |
| openstackgerrit | A change was merged to openstack/marconi: Require oslo.config 1.2.0 final https://review.openstack.org/48142 | 21:24 |
| mpanetta | kgriffs: Almost to your test | 21:31 |
| *** ayoung has joined #openstack-marconi | 21:36 | |
| kgriffs | mpanetta: cool, looking forward to the results | 21:39 |
| mpanetta | About 5 mins. | 21:45 |
| mpanetta | kgriffs: I posted the link on the internal channel | 21:55 |
| *** amitgandhi1 has quit IRC | 22:13 | |
| *** jcru has quit IRC | 22:18 | |
| *** oz_akan_ has joined #openstack-marconi | 22:20 | |
| *** mpanetta has quit IRC | 22:22 | |
| *** oz_akan_ has quit IRC | 22:25 | |
| *** openstackgerrit has quit IRC | 22:52 | |
| *** openstackgerrit has joined #openstack-marconi | 22:52 | |
| *** flaper87 is now known as flaper87|afk | 23:26 | |
| *** notmyname has quit IRC | 23:47 | |
| *** gleicon__ has quit IRC | 23:47 | |
| *** westmaas has quit IRC | 23:47 | |
| *** openstackgerrit has quit IRC | 23:47 | |
| *** briancline has quit IRC | 23:47 | |
| *** westmaas has joined #openstack-marconi | 23:47 | |
| *** gleicon___ has joined #openstack-marconi | 23:47 | |
| *** briancline has joined #openstack-marconi | 23:53 | |
| *** openstackgerrit has joined #openstack-marconi | 23:53 | |
| *** notmyname has joined #openstack-marconi | 23:53 | |
| *** malini_afk is now known as malini | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!