| *** amitgandhi has quit IRC | 00:10 | |
| *** amitgandhi has joined #openstack-marconi | 00:11 | |
| *** amitgandhi has quit IRC | 00:15 | |
| *** vkmc has quit IRC | 00:23 | |
| *** nosnos has joined #openstack-marconi | 00:49 | |
| *** amitgandhi has joined #openstack-marconi | 01:09 | |
| *** oz_akan_ has joined #openstack-marconi | 01:15 | |
| *** oz_akan_ has quit IRC | 01:19 | |
| *** oz_akan_ has joined #openstack-marconi | 01:20 | |
| *** etoews1 has joined #openstack-marconi | 01:20 | |
| *** kgriffs_afk is now known as kgriffs | 02:37 | |
| *** amitgandhi has quit IRC | 03:00 | |
| *** amitgandhi has joined #openstack-marconi | 03:00 | |
| *** amitgandhi has quit IRC | 03:04 | |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Implement HTTP Transport https://review.openstack.org/49788 | 03:15 |
|---|---|---|
| *** oz_akan_ has quit IRC | 03:17 | |
| *** kgriffs is now known as kgriffs_afk | 03:23 | |
| *** amitgandhi has joined #openstack-marconi | 03:30 | |
| *** amitgandhi has quit IRC | 03:48 | |
| *** amitgandhi has joined #openstack-marconi | 04:13 | |
| *** amit has joined #openstack-marconi | 04:15 | |
| *** amitgandhi has quit IRC | 04:15 | |
| *** amit is now known as Guest62508 | 04:16 | |
| *** cpallares has quit IRC | 04:23 | |
| *** Guest62508 has quit IRC | 04:32 | |
| *** amitgandhi has joined #openstack-marconi | 04:57 | |
| *** amitgandhi has quit IRC | 05:15 | |
| *** amitgandhi has joined #openstack-marconi | 05:41 | |
| *** amitgandhi has quit IRC | 06:00 | |
| *** amitgandhi has joined #openstack-marconi | 06:26 | |
| *** amit has joined #openstack-marconi | 06:28 | |
| *** amitgandhi has quit IRC | 06:28 | |
| *** amit is now known as Guest57573 | 06:29 | |
| *** flaper87|afk is now known as flaper87 | 06:37 | |
| *** Guest57573 has quit IRC | 06:42 | |
| *** amitgandhi has joined #openstack-marconi | 07:08 | |
| *** amitgandhi has quit IRC | 07:10 | |
| *** amitgandhi has joined #openstack-marconi | 07:11 | |
| *** flaper87 is now known as flaper87|afk | 07:27 | |
| *** amitgandhi has quit IRC | 07:28 | |
| *** amitgandhi has joined #openstack-marconi | 07:53 | |
| *** yassine has joined #openstack-marconi | 07:57 | |
| *** amit has joined #openstack-marconi | 08:25 | |
| *** amitgandhi has quit IRC | 08:25 | |
| *** amit is now known as Guest55100 | 08:25 | |
| *** Guest55100 has quit IRC | 08:34 | |
| *** amitgandhi has joined #openstack-marconi | 09:00 | |
| *** amitgandhi has quit IRC | 09:15 | |
| *** amitgandhi has joined #openstack-marconi | 09:40 | |
| *** amit has joined #openstack-marconi | 09:43 | |
| *** amitgandhi has quit IRC | 09:43 | |
| *** amit is now known as Guest91384 | 09:43 | |
| *** flaper87|afk is now known as flaper87 | 10:13 | |
| *** Guest91384 has quit IRC | 10:14 | |
| *** ekarlso has quit IRC | 10:17 | |
| *** ekarlso has joined #openstack-marconi | 10:17 | |
| *** nosnos has quit IRC | 10:31 | |
| *** amitgandhi has joined #openstack-marconi | 10:38 | |
| *** amit has joined #openstack-marconi | 10:41 | |
| *** amitgandhi has quit IRC | 10:41 | |
| *** amit is now known as Guest83514 | 10:41 | |
| *** flaper87 is now known as flaper87|afk | 10:48 | |
| *** Guest83514 has quit IRC | 10:54 | |
| *** tedross has joined #openstack-marconi | 11:19 | |
| *** amitgandhi has joined #openstack-marconi | 11:19 | |
| *** amitgandhi has quit IRC | 11:36 | |
| *** malini_afk is now known as malini | 11:39 | |
| *** amitgandhi has joined #openstack-marconi | 12:00 | |
| *** malini is now known as malini_afk | 12:02 | |
| *** amitgandhi has quit IRC | 12:05 | |
| *** flaper87|afk is now known as flaper87 | 12:09 | |
| *** vkmc has joined #openstack-marconi | 12:17 | |
| *** amitgandhi has joined #openstack-marconi | 12:37 | |
| *** amitgandhi has quit IRC | 12:42 | |
| *** oz_akan_ has joined #openstack-marconi | 13:04 | |
| *** oz_akan_ has quit IRC | 13:06 | |
| *** oz_akan_ has joined #openstack-marconi | 13:06 | |
| *** mpanetta has joined #openstack-marconi | 13:07 | |
| *** mpanetta has quit IRC | 13:09 | |
| *** mpanetta has joined #openstack-marconi | 13:09 | |
| *** mpanetta has joined #openstack-marconi | 13:10 | |
| *** russellb is now known as rustlebee | 13:12 | |
| *** amitgandhi has joined #openstack-marconi | 13:22 | |
| *** alcabrera has joined #openstack-marconi | 13:30 | |
| alcabrera | Morning! :D | 13:31 |
| flaper87 | alcabrera: goood morning! | 13:31 |
| flaper87 | alcabrera: did you read the tstream news? | 13:32 |
| alcabrera | Yep, just now! | 13:32 |
| alcabrera | "I'll get that implemented soon." :D | 13:32 |
| *** amitgandhi has quit IRC | 13:32 | |
| alcabrera | jtauber is awesome. :) | 13:33 |
| *** amitgandhi has joined #openstack-marconi | 13:33 | |
| alcabrera | wait wait wait... | 13:33 |
| alcabrera | It's already implemented! | 13:33 |
| alcabrera | Fast! | 13:33 |
| * alcabrera is catching up on email | 13:34 | |
| *** jcru has joined #openstack-marconi | 13:43 | |
| flaper87 | alcabrera: :D | 13:45 |
| flaper87 | it is and it works | 13:45 |
| flaper87 | now we've markdown and atom there | 13:45 |
| alcabrera | sweeeet | 13:46 |
| alcabrera | I'll be flaper87 - I've been called for office-type duties. :P | 13:46 |
| alcabrera | s/be/brb | 13:46 |
| *** alcabrera is now known as alcabrera|afk | 13:46 | |
| *** malini_afk is now known as malini | 13:47 | |
| *** kgriffs_afk is now known as kgriffs | 13:58 | |
| kgriffs | morning/afternoon! | 13:59 |
| flaper87 | kgriffs: hellooooo | 14:02 |
| openstackgerrit | A change was merged to openstack/python-marconiclient: change assertEquals to assertEqual https://review.openstack.org/53512 | 14:04 |
| kgriffs | flaper87: so... | 14:05 |
| kgriffs | I was doing some thinking about the client design | 14:05 |
| kgriffs | not sure yet if this makes sense, but here is my idea | 14:06 |
| flaper87 | kgriffs: go ahead | 14:06 |
| kgriffs | the goal is to reduce churn in the client when the api changes | 14:06 |
| kgriffs | so, what if the api schema was loaded from the api itself? | 14:07 |
| kgriffs | instead of it being hard-coded into the client | 14:07 |
| kgriffs | the client *would* have to hard-code operation names | 14:07 |
| flaper87 | that's the plan, I wanted to use the /home endpoint, but I don't think the current format is very helpful | 14:08 |
| kgriffs | so it can look them up still in the schema | 14:08 |
| flaper87 | and that's where a server side schema will come handy | 14:08 |
| flaper87 | :D | 14:08 |
| kgriffs | flaper87: cool, so that was my other point | 14:08 |
| kgriffs | JSON home is geared toward HTTP | 14:08 |
| kgriffs | i mean, that is the bias | 14:09 |
| *** zyuan_ has joined #openstack-marconi | 14:10 | |
| kgriffs | second thing is that HATEOAS isn't limited to HTTP | 14:10 |
| kgriffs | you could have a schema of entry points and then subsequent requests would return refs or whatever to follow for other operations | 14:10 |
| flaper87 | indeed, TBH, the fact that it's geared toward HTTP is not a huge issue here. If you check the API schema I'm currently using for the client, it's basically related to HTTP | 14:11 |
| flaper87 | since it's the most partitioned protocol we're supporting so far | 14:11 |
| flaper87 | and it's easier to translate from HTTP to zmq than the other way around | 14:11 |
| flaper87 | kgriffs: re refs returned, totally agreed | 14:12 |
| kgriffs | yeah. I have a feeling we will have to get another protocol in place before we will be able to really work out the best design | 14:12 |
| kgriffs | so, back to the home doc thing | 14:12 |
| flaper87 | I was thinking about adding support for that after queues + messages + claims land | 14:12 |
| flaper87 | since we're already returning refs | 14:12 |
| kgriffs | excellent | 14:12 |
| flaper87 | the missing part would be "Getting the base API schema" from the server | 14:13 |
| flaper87 | which we already agreed on | 14:13 |
| flaper87 | :D | 14:13 |
| flaper87 | kk, back to home document | 14:13 |
| kgriffs | re home doc, let's start thinking about what a more *useful* document format would look like | 14:13 |
| kgriffs | it would be cool to define a rich schema behind the scenes, that could then be exposed as various media types, home doc being one if it ends up being useful to developers | 14:14 |
| kgriffs | s/home doc/JSON home | 14:14 |
| zyuan_ | useful to machine or to user | 14:15 |
| kgriffs | i think this plays nicely into what you were talking about the other day re server-side schema | 14:15 |
| zyuan_ | ? | 14:15 |
| kgriffs | zyuan_: either, but it seems like home doc is mostly for humans or for simple documentation "click through my API" tools. | 14:15 |
| kgriffs | sorry, by home doc I mean json-home | 14:16 |
| kgriffs | what i am proposing is we have some other kind of x-home format that works better for flaper87's needs | 14:16 |
| flaper87 | kgriffs: indeed, that's exactly what I was talking about the other day. A server side schema will allow us to easily support discovering and extensions | 14:16 |
| zyuan_ | hmm, i don't think a human will find json home document is readable to "click through"... | 14:16 |
| kgriffs | zyuan_: i mean, you would have a web app or something that consumes it and turns it into HTML | 14:17 |
| zyuan_ | ok | 14:17 |
| kgriffs | flaper87: ok, looks like once again we have proved our brains are quantum-entangled. | 14:18 |
| flaper87 | kgriffs: btw, I checked that 'API extensions' talk you shared the other day | 14:18 |
| flaper87 | kgriffs: hahahahaha | 14:18 |
| flaper87 | kgriffs: Programmatically speaking, I was thinking about supporting API schema inheritance, so, newer API's would be able to inherit from older if needed | 14:19 |
| flaper87 | kgriffs: we would play with mixins for extensions | 14:20 |
| flaper87 | and transport will register everything found in an API spec dynamically | 14:20 |
| flaper87 | I'm still putting thoughts on that, I created a blueprint for it (you should've an email for that) and I'll start writing some stuff on a epad first | 14:21 |
| flaper87 | I don't think I'll have some code ready for our next meeting | 14:21 |
| flaper87 | but definitelly I'll have some specs written down and a more detailed idea | 14:21 |
| kgriffs | cool beans | 14:23 |
| kgriffs | you definitely want to be able to inherit for minor API version bumps | 14:23 |
| kgriffs | btw, re sharding additions to the API | 14:24 |
| * kgriffs wonders where alcabrera|afk got off to | 14:24 | |
| flaper87 | kgriffs: he said he had some office duties to attend | 14:28 |
| flaper87 | I'll get to sharding patches very soon | 14:28 |
| kgriffs | sounds fishy to me. ;) | 14:30 |
| kgriffs | anyway | 14:30 |
| kgriffs | flaper87: I was thinking that sharding could be implemented as a v1.0 extension | 14:30 |
| kgriffs | then promoted into v1.1 when we start work on that | 14:30 |
| kgriffs | that being said | 14:31 |
| kgriffs | it is a backwards-compatible update to the api, so we could sneak it into v1 | 14:31 |
| kgriffs | not sure which way to go | 14:31 |
| kgriffs | technically, v1.0 is frozen | 14:32 |
| kgriffs | so, maybe we let the patches land as-is, then circle back and move them into extension land | 14:33 |
| kgriffs | or | 14:33 |
| kgriffs | we just start work on v1.1 now | 14:33 |
| kgriffs | what do you think? | 14:34 |
| * flaper87 thinking | 14:35 | |
| flaper87 | I'm leaning towards sneaking it into v1, TBH. | 14:35 |
| zyuan_ | what are the changes you want now? | 14:36 |
| flaper87 | We should see how far we get with the API spec blueprint and perhaps decide based on that | 14:36 |
| kgriffs | ok | 14:37 |
| kgriffs | so, for now we will sneak it in, but not document it until v1.1 | 14:37 |
| kgriffs | and revisit as the API spec stuff evolves | 14:37 |
| kgriffs | man, I wish wiki.openstack.org had an option to submit markdown | 14:39 |
| zyuan_ | me too... | 14:39 |
| *** ametts has joined #openstack-marconi | 14:40 | |
| flaper87 | kgriffs: zyuan_ tell me about it! :/ | 14:40 |
| kgriffs | interesting article here, has lessons applicable beyond SolidFire | 14:45 |
| kgriffs | http://blog.mongodb.org/post/64962828969/performance-tuning-mongodb-on-solidfire | 14:45 |
| kgriffs | btw, the guy you started SolidFire is a former boss of mine | 14:45 |
| kgriffs | s/you/who | 14:45 |
| *** ayoung has joined #openstack-marconi | 14:45 | |
| flaper87 | ametts: it now has atom and markdown: https://thoughtstreams.io/combined/marconi-progress-and-updates/ | 14:47 |
| flaper87 | :) | 14:47 |
| flaper87 | our combined stream | 14:47 |
| ametts | Wow. I should send a letter to the customer service department of that software company... :) | 14:48 |
| flaper87 | ametts: you can just tweet them :D | 14:48 |
| flaper87 | ametts: this is how that feature got requested and implemented: https://twitter.com/cppcabrera/status/393732927009206272 | 14:49 |
| flaper87 | ametts: I mean, this: https://twitter.com/flaper87/status/393383932411916288 | 14:49 |
| ametts | Yeah, I saw that -- nice that they did it so fast. So I've added it to my newsreader, and now I'll know everything you guys are thinking.... | 14:50 |
| kgriffs | :o | 14:51 |
| zyuan_ | json is pypy: http://morepypy.blogspot.com/2011/10/speeding-up-json-encoding-in-pypy.html | 14:51 |
| zyuan_ | in* | 14:51 |
| *** alcabrera|afk is now known as alcabrera | 14:51 | |
| alcabrera | back~ | 14:51 |
| alcabrera | I missed all the things, didn't I? | 14:52 |
| flaper87 | alcabrera: you did, I'm not goint to lie | 14:53 |
| flaper87 | :D | 14:53 |
| alcabrera | lol | 14:54 |
| alcabrera | I've caughtup at this point. | 14:54 |
| alcabrera | x-home and such | 14:54 |
| flaper87 | LOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOL: https://thoughtstreams.io/combined/marconi-progress-and-updates/ (read the last 2 posts) | 14:54 |
| flaper87 | kgriffs: man, this is scaring me now. ^ | 14:54 |
| alcabrera | wow | 14:54 |
| kgriffs | :D | 14:54 |
| alcabrera | you guys are so sync'd. :P | 14:55 |
| flaper87 | I think the "We're now thinking about" is the scariest part of both posts | 14:55 |
| kgriffs | alcabrera: hurry up and find your own doppelgänger so we can have 4 core team members instead of just 3. | 14:56 |
| alcabrera | kgriffs: I'll have one ready by the end of the day. :P | 14:56 |
| flaper87 | kgriffs: btw, I love the sharding diagram | 14:57 |
| flaper87 | seriously :D | 14:57 |
| flaper87 | also, I added this: https://wiki.openstack.org/wiki/OutreachProgramForWomen/Ideas#Marconi_API_Spec | 14:57 |
| flaper87 | I'm not expecting the intern to do it all but to work together on that | 14:57 |
| kgriffs | cool | 14:58 |
| alcabrera | awesome! | 14:58 |
| kgriffs | I wasn't sure about the idea of putting Pecan POC on there, since it would be cool to give them something that would get them more into the guts of the project | 14:59 |
| flaper87 | kgriffs: agreed! | 15:00 |
| kgriffs | nobody likes to get that "here is this thing that we are procrastinating and may or may not turn out to be useful" project | 15:00 |
| kgriffs | btw, thanks a ton for working on the OPW | 15:01 |
| alcabrera | I'm favorable towards having people working on alternative storage drivers. It's quite at the heart of the project and gives volunteers a chance to work on/with cool technologies. :D | 15:03 |
| kgriffs | i mean, it will definitely be useful in one way or another, but may not be as satisfying as some other types of work | 15:03 |
| kgriffs | flaper87: ready to talk about the talk? | 15:04 |
| flaper87 | alcabrera: I thought about adding a storage driver there (like redis one) but I then remembered that we want to keep the number of storage drivers in the code base small | 15:04 |
| alcabrera | that's true | 15:04 |
| flaper87 | so, I'd like us all to discuss what drivers it makes sense to have in the codebase first | 15:04 |
| flaper87 | kgriffs: yup, gimme 2 secs. I'll find a quite place. | 15:05 |
| flaper87 | read ? | 15:08 |
| kgriffs | https://plus.google.com/hangouts/_/72cpjvbuiq4ajgfi9ri5m3ublk?hl=en | 15:10 |
| kgriffs | https://etherpad.openstack.org/p/icehouse-marconi-talk | 15:12 |
| *** jergerber has joined #openstack-marconi | 15:14 | |
| kgriffs | flaper87: can't hear you... | 15:16 |
| kgriffs | flaper87: let's just chat here | 15:22 |
| kgriffs | silly internets | 15:22 |
| *** flaper87 is now known as flaper87|afk | 15:24 | |
| * ametts is back and having a google hangout with nobody but himself | 15:29 | |
| kgriffs | heh | 15:30 |
| kgriffs | flaper87|afk: is still MIA | 15:30 |
| kgriffs | ametts: do you think 10 minutes would be better for demo, or just 5? | 15:31 |
| ametts | Depends on what we demo, I guess. Let's think first about what we want the content/messaging to be, and what we set aside as backup material for extra time and Q/A. | 15:33 |
| ametts | Then we can figure out the time allocations, I guess. | 15:33 |
| alcabrera | alright, now with all my office duties complete, let's see how things are looking in marconi-land~ | 15:34 |
| malini | the bagel were good, werent they alcabrera ? | 15:35 |
| ametts | We have bagels? | 15:35 |
| kgriffs | ametts: https://plus.google.com/hangouts/_/72cpjvbuiq4ajgfi9ri5m3ublk?hl=en | 15:35 |
| alcabrera | I had no idea we had bagels. :P | 15:36 |
| malini | ametts: I was just referring to alej's office duty ;) | 15:36 |
| alcabrera | lol | 15:36 |
| * ametts is hungry | 15:36 | |
| malini | me too... | 15:37 |
| alcabrera | I am, too. | 15:42 |
| alcabrera | I'll probably step out of the office for lunch today. Rare things happen sometimes. ;) | 15:42 |
| openstackgerrit | Chad Lung proposed a change to openstack/marconi: Update the Marconi readme with more detailed installation instructions https://review.openstack.org/53673 | 15:45 |
| mpanetta | We have bagles? | 15:45 |
| mpanetta | bagels too? | 15:46 |
| openstackgerrit | Chad Lung proposed a change to openstack/marconi: Update the Marconi readme with more detailed installation instructions https://review.openstack.org/53673 | 15:47 |
| alcabrera | mpanetta: maybe? If we did, they'd be in the break room. | 15:52 |
| alcabrera | I haven't checked, heh. | 15:52 |
| malini | I am sorry everybody..There are NO bagels :( | 15:54 |
| malini | I am so hungry, my mind is playing tricks on everybody | 15:54 |
| mpanetta | malini: Damn you! Stop being hungry! :P | 15:54 |
| *** cpallares has joined #openstack-marconi | 15:55 | |
| *** alcabrera is now known as alcabrera|lunch | 16:02 | |
| *** flaper87|afk is now known as flaper87 | 16:06 | |
| cpallares | hi flaper87 | 16:08 |
| flaper87 | damn it, got kicked out again | 16:08 |
| *** etoews1 has quit IRC | 16:08 | |
| flaper87 | cpallares: heeey :d | 16:09 |
| flaper87 | :D | 16:09 |
| flaper87 | cpallares: how are you doing? | 16:09 |
| cpallares | flaper87: Good I got marconi installed, but I'm a bit confused about the bug. Is it to rename exceptions.py? | 16:09 |
| cpallares | cpallares: Also I'm good and how are you? :) | 16:10 |
| flaper87 | cpallares: doing well, thanks :D | 16:10 |
| flaper87 | cpallares: that bug is about renaming exceptions.py into errors.py, replacing Exception with Error in classes' names and fixing the imports all around | 16:12 |
| *** etoews1 has joined #openstack-marconi | 16:12 | |
| flaper87 | cpallares: btw, I added a new project to the wiki page | 16:12 |
| flaper87 | in case you want to check that out | 16:12 |
| cpallares | flaper87: I will, there's two exceptions.py, but this one is just for the proxy dir, right? | 16:13 |
| *** etoews1 has quit IRC | 16:13 | |
| flaper87 | cpallares: feel fre to change queues/* as well, just to keep consistency | 16:14 |
| flaper87 | cpallares: also, proxy will probably go away | 16:14 |
| cpallares | So, all the exceptions.py that I find? | 16:15 |
| cpallares | There's one in /common | 16:16 |
| cpallares | I mean change them to keep consistency | 16:17 |
| flaper87 | cpallares: yup! | 16:18 |
| cpallares | ok! | 16:18 |
| flaper87 | cpallares: but not the ones under openstack/common | 16:19 |
| flaper87 | ok? | 16:19 |
| cpallares | ok | 16:19 |
| flaper87 | cpallares: quick intro to Oslo. All files under openstack/common come from oslo-incubator and we never modify them directly in Marconi, instead we do it in oslo-incubator and then backport the patches | 16:19 |
| flaper87 | cpallares: https://wiki.openstack.org/wiki/Oslo | 16:19 |
| cpallares | ok | 16:21 |
| cpallares | what about marconi/common? | 16:23 |
| flaper87 | cpallares: that's under our control :D | 16:25 |
| flaper87 | so, go ahead :D | 16:25 |
| *** alcabrera|lunch is now known as alcabrera | 16:29 | |
| alcabrera | cpallares, flaper87: o/ | 16:32 |
| alcabrera | I made it back with 100% more lunch available. I skipped breakfast so I was rather hungry. :x | 16:32 |
| * flaper87 will have this for dinner today: http://southamericanfood.about.com/od/holidayrecipes/r/Pan-De-Jamon-Venezuelan-Ham-And-Olive-Bread.htm | 16:34 | |
| * flaper87 is starving, so so so much | 16:34 | |
| alcabrera | mmmm... :x | 16:35 |
| alcabrera | sounds yummy! | 16:35 |
| alcabrera | I love olives. | 16:35 |
| flaper87 | alcabrera: this looks even better: http://i1.trekearth.com/photos/32025/pan_de_jamon__191203mod.jpg | 16:36 |
| alcabrera | flaper87: ohhh yes. :D | 16:37 |
| alcabrera | I am having sausages for lunch: these (http://www.johnsonville.com/products/beer-brats.html) and these (http://www.jimmydean.com/products/fresh-sausage/premium-original-pork-sausage-patties) | 16:38 |
| *** whenry has joined #openstack-marconi | 16:38 | |
| kgriffs | flaper87: you should totally come to montreal next april | 16:46 |
| kgriffs | https://us.pycon.org/2014/ | 16:46 |
| * flaper87 proposed a talk, fingers crossed | 16:47 | |
| flaper87 | :D | 16:47 |
| flaper87 | alcabrera: that looks GOOOOOOD | 16:47 |
| kgriffs | let me know if you end up going - we can meet up | 16:47 |
| flaper87 | now I'm even more hungry | 16:47 |
| kgriffs | the pizza truck is calling my name | 16:48 |
| alcabrera | lol | 16:48 |
| flaper87 | kgriffs: totally, If I do, I'm planning to spend some time overseas but, shhhhhhhhhh | 16:48 |
| alcabrera | We're all aiming to go to pycon Montreal, afaict. :) | 16:48 |
| zyuan_ | forgot to say, simplejson only .read() stream then do parsing; no json parser i found works on stream level. | 16:48 |
| flaper87 | I hope to be there even if I'm not talking but, you know! | 16:48 |
| zyuan_ | wow | 16:48 |
| kgriffs | zyuan_: gtk | 16:49 |
| *** cpallares has quit IRC | 16:50 | |
| *** jergerber has quit IRC | 16:50 | |
| *** cpallares has joined #openstack-marconi | 16:50 | |
| *** amitgandhi has quit IRC | 16:50 | |
| *** amitgandhi has joined #openstack-marconi | 16:50 | |
| *** jergerber has joined #openstack-marconi | 16:51 | |
| kgriffs | alcabrera: so, about sharding | 16:52 |
| kgriffs | any updates on your patches? I am getting ready to review them again | 16:54 |
| *** amitgandhi has quit IRC | 16:55 | |
| alcabrera | kgriffs: none yet. It's been one of those chaotic mornings. I'm almost done updating the catalogue patch and I need to work on removing the admin concept from the storage layer for the other patches. | 16:57 |
| flaper87 | kgriffs: in the meantime you can keep approving client patches :D | 16:57 |
| kgriffs | heh | 16:57 |
| alcabrera | I'm all for that. :D | 16:57 |
| kgriffs | alcabrera: re admin concept | 16:58 |
| kgriffs | can you elaborate? | 16:58 |
| *** oz_akan_ has quit IRC | 16:58 | |
| alcabrera | kgriffs: I mean, to stop using AdminDriver(catalogue_controller, shards_controller) and Driver(messages_, claims_, queues_) for storage, and just use Driver. | 16:59 |
| *** yassine has quit IRC | 17:01 | |
| kgriffs | so, in that case sharding.catalog will reference mongodb.Driver ? | 17:01 |
| alcabrera | yup | 17:01 |
| kgriffs | i mean, get the shard controller from mongodb.Driver | 17:01 |
| kgriffs | ok | 17:01 |
| kgriffs | hmm | 17:02 |
| alcabrera | adding in the AdminDriver thing for storage made things weird, which stood out to me all the more yesterday when flaper87 was talking about it. | 17:02 |
| kgriffs | conceptually, I still feel like it would be better to separate control plane and data plane drivers | 17:02 |
| *** cpallares has quit IRC | 17:02 | |
| kgriffs | http://grab.by/rsoI | 17:02 |
| *** cpallares has joined #openstack-marconi | 17:02 | |
| kgriffs | what is the weirdness you mentioned? | 17:04 |
| alcabrera | lemme look it up... | 17:04 |
| kgriffs | if we combine the two | 17:05 |
| kgriffs | when sharding is disabled, you use one subset of the driver | 17:05 |
| kgriffs | when it is enabled you use a different, non-overlapping set | 17:05 |
| alcabrera | so there's this: https://review.openstack.org/#/c/50998/7/marconi/queues/storage/sharding.py (how is a sharding AdminDriver handled?) and https://review.openstack.org/#/c/50998/7/marconi/queues/storage/pipeline.py (what does AdminDriver mean in a pipeline?) | 17:06 |
| * kgriffs reads that back and tries to determine whether it makes sense | 17:06 | |
| *** oz_akan_ has joined #openstack-marconi | 17:06 | |
| alcabrera | whether or not sharding is enabled, messages_, claims_, and queues_ come into the picture. shards_ and catalogue_ only come into play when sharding is enabled. | 17:07 |
| kgriffs | hmmm | 17:07 |
| alcabrera | whether or not they are in an admin context or not (since they're both needed for making performing lookups: (project, queue) => shard). | 17:07 |
| alcabrera | s/making// | 17:08 |
| kgriffs | The control API endpoints don't go through the pipeline | 17:08 |
| kgriffs | the get a reference to the control plane AKA admin driver's controllers directly | 17:08 |
| kgriffs | same with sharding.Catalog | 17:08 |
| alcabrera | hmm | 17:10 |
| kgriffs | and the admin driver doesn't have multiple backends, only one, so it is never sharded | 17:10 |
| kgriffs | (so no need for that meta-driver in storage.sharding) | 17:10 |
| alcabrera | alright, so I can remove the AdminDriver concept from pipeline and sharding | 17:11 |
| kgriffs | that's correct | 17:11 |
| alcabrera | I wonder if it helps to remove the admin notion from the storage layer entirely... | 17:11 |
| alcabrera | Since the transport layer determines that. | 17:12 |
| alcabrera | This is orthogonal thinking to the notion of control plane vs. data plane | 17:12 |
| alcabrera | I feel like having the admin notion duplicated in the storage layer is redundant. | 17:12 |
| kgriffs | a couple points | 17:13 |
| kgriffs | first, exposing shardcontroller through the data drivers just feels like we are violating the principle of cohesion in object-oriented design. | 17:14 |
| kgriffs | mainly, because we are putting N of those drivers behind sharding.Driver, and shardcontroller isn't sharded itself | 17:15 |
| kgriffs | makes things weird | 17:15 |
| kgriffs | also, there is the control vs. data | 17:15 |
| kgriffs | if that point is valid, then the second point holds | 17:15 |
| kgriffs | that we need to load a separate driver | 17:15 |
| alcabrera | I see what you're saying with regards - I see. | 17:16 |
| kgriffs | and, assuming that, you probably don't want to make people configure that control plane driver unless they are going to use it | 17:16 |
| kgriffs | which brings back the idea of an admin mode | 17:17 |
| alcabrera | Yeah, it's a bit silly to require sharding.Driver to also provide a sharding and a catalogue controller. | 17:17 |
| *** amitgandhi has joined #openstack-marconi | 17:17 | |
| alcabrera | So that takes care of my redundancy concern - it's not redundant. | 17:17 |
| kgriffs | so, if we follow this design: http://grab.by/rsoI | 17:18 |
| kgriffs | then the bootstrap would load the admin driver and pass it to the admin transport which grafts on to the data API | 17:18 |
| kgriffs | also, bootstrap would pass the same driver through so that eventually sharding.Catalog can reference it as well | 17:19 |
| kgriffs | (so it can dynamically create/delete mappings, as well as lookup mappings) | 17:19 |
| kgriffs | anyway, that is my $0.02 | 17:20 |
| alcabrera | Makes sense. | 17:20 |
| kgriffs | I think that if we do it that way, it ends up being less weird. It would be nice to get Flavio's blessing, but I have a hard time seeing how putting everything in a single driver is going to be a good design | 17:21 |
| alcabrera | That's pretty much what convinced me. | 17:21 |
| alcabrera | I think the last bit of terminology is the problem. | 17:21 |
| alcabrera | Driver -> DataDriver, AdminDriver -> ControlDriver | 17:22 |
| kgriffs | i suspect that he pushed back on the idea originally because it *was* looking weird | 17:22 |
| alcabrera | Because it's not about admin or public - it's about control and data. | 17:22 |
| kgriffs | yeah | 17:22 |
| alcabrera | I can update the names if you like the idea above. | 17:22 |
| kgriffs | kudos to amitgandhi for giving me that idea | 17:22 |
| kgriffs | (terminology) | 17:22 |
| alcabrera | It makes it more clear we're talking about data/control rather than permissions (admin/public). | 17:22 |
| alcabrera | amitgandhi: +1 | 17:22 |
| kgriffs | yeah | 17:23 |
| kgriffs | permissions are there too | 17:23 |
| kgriffs | but permissions and planes aren't the same thing | 17:23 |
| alcabrera | yup | 17:23 |
| kgriffs | helps to think about them as complimentary instead | 17:23 |
| kgriffs | aaaanyway | 17:23 |
| kgriffs | i know you are sick of refactoring... | 17:24 |
| kgriffs | what is the quickest way to get something in malini's hands to kick the tires? | 17:24 |
| kgriffs | and then what is the plan for getting to the new design we just discussed above? | 17:25 |
| alcabrera | get the admin API merged in. :/ | 17:25 |
| alcabrera | that's what'll give malini the ability to at least test the control side of things. | 17:25 |
| kgriffs | do we just merge everything in as-is and then refactor in subsequent patches, or do we just have her deploy from gerrit? | 17:26 |
| kgriffs | malini: ^^^ | 17:26 |
| alcabrera | hmmm, she's not at her desk/computer. | 17:26 |
| kgriffs | hackday? | 17:27 |
| alcabrera | nah - Glenn's visiting. :) | 17:27 |
| alcabrera | Hackday's next week. | 17:27 |
| malini | kgriffs: ping | 17:27 |
| kgriffs | ah | 17:28 |
| malini | Sorry missed you | 17:28 |
| kgriffs | no worries | 17:28 |
| kgriffs | so, let's come up with a plan to get you testing sharding ASAP | 17:28 |
| malini | tht wud be awesome!! | 17:28 |
| kgriffs | at this point, we know what the functionality will be and the data schema | 17:28 |
| * flaper87 lost a great convo | 17:28 | |
| flaper87 | damn it! | 17:28 |
| flaper87 | malini: quick question: Are you planning to work on tempest support for Marconi? | 17:29 |
| malini | I have a personal server tht runs marconi..I can start playing on tht, if we are not ready to deploy to an actual env | 17:29 |
| malini | flaper87: I am | 17:29 |
| kgriffs | malini: my idea was to get something in your hands right away and then we can circle back and refine the implementation | 17:29 |
| flaper87 | malini: cool beans! | 17:29 |
| malini | flaper87: I havent started yet, But I am starting to plan poking around it soon | 17:29 |
| kgriffs | flaper87: What do you think about this design? http://grab.by/rsoI | 17:30 |
| malini | flaper87: are you working on the devstack part ? | 17:30 |
| kgriffs | the original approach to splitting into an admin and non-admin storage drivers *was* weird | 17:30 |
| kgriffs | but I think we have a plan for making it *not* | 17:30 |
| flaper87 | kgriffs: ha, you missed my happy comments and many congrats about that design earlier today :D | 17:31 |
| kgriffs | and indeed, preserving cohesion | 17:31 |
| flaper87 | kgriffs: it looks really good! | 17:31 |
| kgriffs | ok, lets run with it and see how the implementation turns out | 17:31 |
| kgriffs | flaper87, alcabrera: thoughts on refactoring patches in-place, or letting them land and following up | 17:32 |
| flaper87 | malini: I am, I finally got some feedback about it and I need to re-work it a bit | 17:33 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: add catalogue storage driver for queues https://review.openstack.org/52160 | 17:33 |
| flaper87 | kgriffs: I'd prefer not letting those patches land and work on our final thoughts right away | 17:33 |
| * kgriffs just heard that the pizza truck is not coming by today and is very sad | 17:33 | |
| *** amit has joined #openstack-marconi | 17:33 | |
| alcabrera | kgriffs: pizza. :( | 17:33 |
| kgriffs | ok | 17:33 |
| alcabrera | I'm cool with refactoring in place. | 17:33 |
| kgriffs | malini: Can you just fetch from gerrit directly then and start kicking the tires? | 17:33 |
| *** amit is now known as Guest34983 | 17:34 | |
| alcabrera | I just finished taking care of the catalogue storage driver updates above. ^^ | 17:34 |
| malini | kgriffs: sure.. | 17:34 |
| kgriffs | alcabrera: can you assist malini in getting started? | 17:34 |
| kgriffs | ok guys | 17:34 |
| kgriffs | I need some food and then i've got some client patches screaming for attention. ;) | 17:35 |
| flaper87 | w0000000t | 17:35 |
| flaper87 | I've to step away for a bit! I'll hopefully be back later and review patches, otherwise I'll do it tomorrow | 17:36 |
| alcabrera | flaper87: thanks! | 17:36 |
| alcabrera | kgriffs: get the noms! | 17:36 |
| flaper87 | (yes tomorrow, Got to go to MongoTorino) | 17:36 |
| flaper87 | an talk about marconi | 17:36 |
| flaper87 | HOLY SHIT! | 17:36 |
| flaper87 | My slides! | 17:36 |
| flaper87 | O.O | 17:36 |
| *** amitgandhi has quit IRC | 17:36 | |
| kgriffs | flaper87: rock on | 17:37 |
| *** flaper87 is now known as flaper87|afk | 17:39 | |
| alcabrera | malini: https://wiki.openstack.org/wiki/Marconi/specs/sharding/v1 | 17:42 |
| *** oz_akan_ has quit IRC | 17:44 | |
| *** oz_akan_ has joined #openstack-marconi | 17:45 | |
| *** Guest34983 has quit IRC | 18:08 | |
| *** amitgandhi has joined #openstack-marconi | 18:08 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards storage controller interface https://review.openstack.org/50721 | 18:11 |
| alcabrera | kgriffs: refactored for Data/Control drivers - part I. ^^ | 18:11 |
| alcabrera | I'll be refactoring child patches of that now. | 18:11 |
| alcabrera | hmmm... I seem to have forgotten to fix some super().__init__'s. Going back for those. | 18:13 |
| alcabrera | nvm | 18:14 |
| alcabrera | git rebase does some crazy things sometimes. | 18:14 |
| alcabrera | Patch above is ready for review. ^^ | 18:14 |
| *** kgriffs is now known as kgriffs_afk | 18:17 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards storage controller interface https://review.openstack.org/50721 | 18:30 |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: shards mongodb driver + tests https://review.openstack.org/50815 | 18:31 |
| *** ayoung has quit IRC | 18:36 | |
| *** etoews1 has joined #openstack-marconi | 18:37 | |
| cpallares | if I'm ready to commit a fix for a bug, do I have to put anything as a commit message, like a bug number or something? | 18:42 |
| *** kgriffs_afk is now known as kgriffs | 18:42 | |
| malini | cpallares: https://wiki.openstack.org/wiki/GerritWorkflow#Committing_Changes | 18:43 |
| cpallares | thanks malini! | 18:44 |
| alcabrera | kgriffs: thought - what do you think of changing the name of the bootstrap storage attribute to data and exposing a control attribute? | 18:47 |
| *** amitgandhi has quit IRC | 18:47 | |
| alcabrera | rationale: make that data/control distinction more prominent and make it possible to initialize a control driver if needed. | 18:47 |
| *** amitgandhi has joined #openstack-marconi | 18:47 | |
| alcabrera | amitgandhi: hey, hey. :) | 18:48 |
| *** relaxdiego has joined #openstack-marconi | 18:49 | |
| *** ayoung has joined #openstack-marconi | 18:50 | |
| amitgandhi | alcabrera: yo | 18:51 |
| alcabrera | amitgandhi: you gonna join in on our other IRC channel? :P | 18:51 |
| *** amitgandhi has quit IRC | 18:52 | |
| *** amitgandhi has joined #openstack-marconi | 18:52 | |
| amitgandhi | kgriffs: ping | 19:02 |
| *** relaxdiego has quit IRC | 19:18 | |
| *** relaxdiego has joined #openstack-marconi | 19:22 | |
| *** alcabrera is now known as alcabrera|afk | 19:30 | |
| *** cpallares1 has joined #openstack-marconi | 19:31 | |
| *** cpallares has quit IRC | 19:31 | |
| kgriffs | w00t: http://icehousedesignsummit.sched.org/type/marconi | 19:33 |
| kgriffs | amitgandhi: pong | 19:33 |
| *** cpallares1 has quit IRC | 19:33 | |
| *** cpallares has joined #openstack-marconi | 19:33 | |
| *** malini is now known as malini_afk | 19:36 | |
| *** kgriffs has left #openstack-marconi | 19:38 | |
| *** kgriffs has joined #openstack-marconi | 19:38 | |
| *** ChanServ sets mode: +o kgriffs | 19:38 | |
| *** cpallares has quit IRC | 19:44 | |
| *** cpallares has joined #openstack-marconi | 19:44 | |
| *** cpallares has quit IRC | 19:47 | |
| *** cpallares has joined #openstack-marconi | 19:47 | |
| *** cpallares has quit IRC | 19:49 | |
| *** cpallares has joined #openstack-marconi | 19:50 | |
| *** amitgandhi has quit IRC | 19:58 | |
| *** amitgandhi has joined #openstack-marconi | 19:59 | |
| *** amitgandhi has quit IRC | 20:03 | |
| openstackgerrit | A change was merged to openstack/python-marconiclient: Pass keywords to the request initialization https://review.openstack.org/50567 | 20:04 |
| *** amitgandhi has joined #openstack-marconi | 20:04 | |
| *** amitgandhi has quit IRC | 20:07 | |
| *** alcabrera|afk is now known as alcabrera | 20:08 | |
| *** amitgandhi has joined #openstack-marconi | 20:08 | |
| openstackgerrit | Alejandro Cabrera proposed a change to openstack/marconi: feat: integrate shard storage with transport https://review.openstack.org/50998 | 20:11 |
| kgriffs | alcabrera: there you are! | 20:11 |
| alcabrera | yup. :D | 20:12 |
| alcabrera | All patches rebased. | 20:12 |
| alcabrera | The refactoring was epic, but not so bad. | 20:12 |
| alcabrera | DataDriver + ControlDriver - and everyone was happy. | 20:12 |
| alcabrera | kgriffs: ^ | 20:12 |
| kgriffs | ah, ok | 20:13 |
| kgriffs | let me finish reviewing this client library patch | 20:13 |
| alcabrera | I kept the name of the storage property the same, and added a control property. | 20:13 |
| alcabrera | the transport takes that control and uses it as needed. | 20:13 |
| alcabrera | kgriffs: sure thing | 20:13 |
| kgriffs | ah | 20:15 |
| kgriffs | I think renaming storage property could actually be good, but we can look at that in another patch | 20:15 |
| alcabrera | +1 | 20:16 |
| alcabrera | we can talk about it some more with flaper87|afk come Monday morning. | 20:16 |
| kgriffs | crazy idea | 20:27 |
| kgriffs | support msgpack media type | 20:27 |
| kgriffs | as in | 20:27 |
| kgriffs | application/msgpack | 20:27 |
| kgriffs | and make it the default in client libs | 20:27 |
| alcabrera | hmmm | 20:27 |
| kgriffs | faster, more compact, more predictable (size) | 20:27 |
| kgriffs | and | 20:28 |
| kgriffs | supports binary message bodies! | 20:28 |
| alcabrera | very true. | 20:28 |
| alcabrera | Refactoring towards supporting different protocols should be fun. :P | 20:28 |
| alcabrera | content-type -> handler -> handler.load() -> process -> respond | 20:29 |
| alcabrera | also: content-type -> handler -> handler_not_found -> respond | 20:29 |
| alcabrera | kgriffs: as time allows, can I get you to review these: https://review.openstack.org/#/c/50721/, https://review.openstack.org/#/c/50815/, https://review.openstack.org/#/c/50998/ ? | 20:30 |
| alcabrera | That's the rest of the admin API. | 20:30 |
| alcabrera | I'm going to rebase the catalogue to depend on the storage interface in the first of those, | 20:30 |
| kgriffs | https://blueprints.launchpad.net/marconi/+spec/msgpack | 20:33 |
| kgriffs | back to reviewing | 20:33 |
| alcabrera | kgriffs: nice! | 20:33 |
| alcabrera | kgriffs: when sharding is done, I'm going to triage those BPs in the style of 'sharding-merge-strategy'. | 20:34 |
| alcabrera | We have *a lot* of BPs that need some love. | 20:34 |
| kgriffs | yes we do! | 20:34 |
| alcabrera | back to coding thee things~ | 20:35 |
| alcabrera | *the | 20:35 |
| alcabrera | I'm heading out for the night, guys. :) | 20:43 |
| alcabrera | email me if I can help speed up the review process. I'm willing to help out during the weekend. | 20:44 |
| kgriffs | kk | 20:44 |
| kgriffs | thanks! | 20:44 |
| alcabrera | o/ | 20:44 |
| *** alcabrera has quit IRC | 20:44 | |
| *** tedross has quit IRC | 20:50 | |
| *** ayoung has quit IRC | 21:07 | |
| *** relaxdiego has quit IRC | 21:23 | |
| *** oz_akan_ has quit IRC | 21:52 | |
| *** oz_akan_ has joined #openstack-marconi | 21:52 | |
| *** oz_akan_ has quit IRC | 21:57 | |
| openstackgerrit | Cindy Pallares proposed a change to openstack/marconi: Closes-Bug: 1232074 https://review.openstack.org/53962 | 21:59 |
| *** amitgandhi has quit IRC | 22:07 | |
| *** amitgandhi has joined #openstack-marconi | 22:07 | |
| *** amitgandhi has quit IRC | 22:12 | |
| *** kgriffs is now known as kgriffs_afk | 22:16 | |
| *** mpanetta has quit IRC | 22:19 | |
| *** ametts has quit IRC | 22:24 | |
| openstackgerrit | Cindy Pallares proposed a change to openstack/marconi: Closes-Bug: 1232074 https://review.openstack.org/53967 | 22:28 |
| *** cpallares has quit IRC | 22:43 | |
| *** etoews1 has quit IRC | 22:46 | |
| *** jergerber has quit IRC | 22:51 | |
| *** amitgandhi has joined #openstack-marconi | 23:08 | |
| *** amitgandhi has quit IRC | 23:14 | |
| *** jcru has quit IRC | 23:17 | |
| *** amitgandhi has joined #openstack-marconi | 23:18 | |
| *** amitgandhi has quit IRC | 23:22 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
