| *** mpanetta has quit IRC | 00:35 | |
| *** balajiiyer has joined #openstack-marconi | 00:59 | |
| *** balajiiyer has quit IRC | 01:17 | |
| *** mwagner_lap has quit IRC | 01:20 | |
| *** mpanetta has joined #openstack-marconi | 01:31 | |
| *** nosnos has joined #openstack-marconi | 01:37 | |
| *** nosnos has quit IRC | 01:38 | |
| *** nosnos has joined #openstack-marconi | 01:40 | |
| *** haomaiwa_ has joined #openstack-marconi | 02:06 | |
| *** haomaiwang has quit IRC | 02:06 | |
| *** mpanetta has quit IRC | 02:16 | |
| *** haomaiwang has joined #openstack-marconi | 02:41 | |
| *** flwang has quit IRC | 02:45 | |
| *** haomaiwa_ has quit IRC | 02:45 | |
| *** flwang has joined #openstack-marconi | 03:28 | |
| *** anamalagon_ has joined #openstack-marconi | 03:41 | |
| *** anamalagon has quit IRC | 03:43 | |
| *** mwagner_lap has joined #openstack-marconi | 04:08 | |
| *** haomaiwa_ has joined #openstack-marconi | 04:33 | |
| *** haomaiwang has quit IRC | 04:34 | |
| *** chandan_kumar has joined #openstack-marconi | 04:43 | |
| *** chandan_kumar has quit IRC | 04:52 | |
| *** saju_m has joined #openstack-marconi | 04:59 | |
| *** chandan_kumar has joined #openstack-marconi | 05:01 | |
| *** prashanthr_ has quit IRC | 05:32 | |
| *** nosnos has quit IRC | 05:39 | |
| *** nosnos has joined #openstack-marconi | 05:39 | |
| *** flwang has quit IRC | 05:40 | |
| *** flwang has joined #openstack-marconi | 05:40 | |
| *** saju_m has quit IRC | 05:54 | |
| *** saju_m has joined #openstack-marconi | 05:55 | |
| *** saju_m has quit IRC | 06:00 | |
| *** saju_m has joined #openstack-marconi | 06:02 | |
| *** belief has quit IRC | 06:07 | |
| *** dimolya has joined #openstack-marconi | 06:28 | |
| *** nosnos_ has joined #openstack-marconi | 06:51 | |
| *** nosnos has quit IRC | 06:51 | |
| *** nosnos has joined #openstack-marconi | 06:56 | |
| *** nosnos_ has quit IRC | 06:58 | |
| *** dimolya has quit IRC | 07:04 | |
| *** nosnos has quit IRC | 07:05 | |
| *** nosnos has joined #openstack-marconi | 07:08 | |
| *** nosnos has quit IRC | 07:12 | |
| *** nosnos has joined #openstack-marconi | 07:12 | |
| *** davidhadas has quit IRC | 07:16 | |
| *** mkoderer has joined #openstack-marconi | 07:50 | |
| *** davidhadas has joined #openstack-marconi | 08:09 | |
| *** flaper87|afk is now known as flaper87 | 08:14 | |
| *** saju_m has quit IRC | 08:22 | |
| *** saju_m has joined #openstack-marconi | 08:28 | |
| *** davidhadas has quit IRC | 08:31 | |
| *** davidhadas has joined #openstack-marconi | 08:31 | |
| *** saju_m has quit IRC | 08:50 | |
| *** saju_m has joined #openstack-marconi | 09:04 | |
| *** saju_m has quit IRC | 09:10 | |
| *** saju_m has joined #openstack-marconi | 09:11 | |
| *** flwang has quit IRC | 09:24 | |
| *** fvollero|PTO is now known as fvollero | 09:31 | |
| *** davidhadas has quit IRC | 10:34 | |
| *** davidhadas has joined #openstack-marconi | 10:36 | |
| *** saju_m has quit IRC | 11:05 | |
| *** vkmc has joined #openstack-marconi | 11:34 | |
| *** nidhi_ has joined #openstack-marconi | 12:09 | |
| *** mwagner_lap has quit IRC | 12:28 | |
| *** flwang has joined #openstack-marconi | 12:42 | |
| *** ayoung-ZZZZZZZzz has quit IRC | 12:45 | |
| *** vkmc has quit IRC | 12:46 | |
| *** sriram has joined #openstack-marconi | 12:55 | |
| *** sriram has quit IRC | 12:56 | |
| *** sriram has joined #openstack-marconi | 12:56 | |
| *** oz_akan_ has joined #openstack-marconi | 13:05 | |
| *** tedross has joined #openstack-marconi | 13:06 | |
| *** jcru has joined #openstack-marconi | 13:14 | |
| *** rustlebee is now known as russellb | 13:26 | |
| *** alcabrera has joined #openstack-marconi | 13:26 | |
| *** oz_akan_ has quit IRC | 13:27 | |
| *** mpanetta has joined #openstack-marconi | 13:28 | |
| *** oz_akan_ has joined #openstack-marconi | 13:28 | |
| *** oz_akan_ has quit IRC | 13:32 | |
| *** amitgandhi has joined #openstack-marconi | 13:37 | |
| *** vkmc has joined #openstack-marconi | 13:37 | |
| *** nosnos has quit IRC | 13:38 | |
| *** amitgandhi has quit IRC | 13:39 | |
| *** nidhi_ has quit IRC | 13:39 | |
| *** amitgandhi has joined #openstack-marconi | 13:39 | |
| *** oz_akan_ has joined #openstack-marconi | 13:42 | |
| flaper87 | alcabrera: goood morning | 13:42 |
|---|---|---|
| alcabrera | flaper87: good morning! :) | 13:44 |
| flaper87 | alcabrera: how are you doing? | 13:44 |
| flaper87 | alcabrera: had a nice weekend? | 13:44 |
| alcabrera | flaper87: it was pretty good. Played some Path of Exile with Jess and house mate. You? :) | 13:45 |
| flaper87 | alcabrera: awesome! I... ate... literally. | 13:46 |
| flaper87 | Went to visit my grandma and she literally made me eat the whole day | 13:47 |
| flaper87 | Just like any Italian movie out there! :D | 13:47 |
| flaper87 | Those movies don't lie :P | 13:47 |
| alcabrera | haha | 13:48 |
| alcabrera | I remember seeing a Tweet about that, and was like, "I bet that's *a lot* of delicious food!", flaper87. :P | 13:48 |
| mpanetta | Mmmm food | 13:48 |
| *** cpallares has joined #openstack-marconi | 13:50 | |
| flaper87 | alcabrera: yeah, tons of delicious things | 13:50 |
| vkmc | Hi all! | 13:50 |
| vkmc | Oh... nonna's food :) | 13:51 |
| alcabrera | vkmc: hey! | 13:51 |
| vkmc | alcabrera, o/ | 13:51 |
| *** jmckind has joined #openstack-marconi | 13:51 | |
| flaper87 | vkmc: hey hey :D | 13:52 |
| flaper87 | alcabrera et all, feedback ? http://blog.flaper87.com/post/531cd585d987d24e83f082a5/ | 13:52 |
| flaper87 | (I need to fix the code samples, they got rendered very weird) | 13:52 |
| alcabrera | queues as topics | 13:52 |
| vkmc | flaper87, o/ :) | 13:53 |
| alcabrera | I like it | 13:53 |
| alcabrera | I'm favorable towards lazy queue creation in general | 13:53 |
| alcabrera | I'm *even* more curious about how thinking of queues as topics will change our design thinking | 13:54 |
| alcabrera | for example | 13:54 |
| alcabrera | multi-topic messages | 13:54 |
| alcabrera | ephemeral topics ("queues" that exist only for a short time) | 13:54 |
| alcabrera | private/public topics | 13:54 |
| alcabrera | flaper87: feedback rendered - I'd love to explore queues as topics. | 13:56 |
| flaper87 | alcabrera: yeah, I put some serious thoughts on that and I'm very convinced that we (1) don't need queues, (2) We can't preserve some features without making them first-citizen resources in Marconi | 13:57 |
| *** tedross has quit IRC | 13:59 | |
| *** tedross has joined #openstack-marconi | 14:00 | |
| *** mwagner_lap has joined #openstack-marconi | 14:03 | |
| alcabrera | I'll be back in a bit, all. | 14:03 |
| alcabrera | Done at the dentist! | 14:03 |
| alcabrera | See you in about 75ish. :D | 14:03 |
| *** alcabrera has quit IRC | 14:03 | |
| *** malini_afk is now known as malini | 14:06 | |
| *** balajiiyer has joined #openstack-marconi | 14:20 | |
| *** rwsu has joined #openstack-marconi | 14:36 | |
| *** ayoung has joined #openstack-marconi | 14:41 | |
| *** Kuwagata has joined #openstack-marconi | 14:44 | |
| malini | flaper87: ping | 14:49 |
| flaper87 | malini: pong | 14:49 |
| malini | flaper87: how are you? | 14:49 |
| flaper87 | malini: I'm doing well. How are you? | 14:49 |
| malini | good..I have my brain back today :) | 14:49 |
| flaper87 | malini: hehehe | 14:49 |
| malini | flaper87: I got some feedback on tempest patch suggesting to use class level setup/tearDown for tests | 14:49 |
| malini | because tht is how they do it for everything else | 14:49 |
| malini | https://review.openstack.org/#/c/63449/ | 14:49 |
| openstackgerrit | Jenkins proposed a change to openstack/marconi: Updated from global requirements https://review.openstack.org/79335 | 14:49 |
| malini | flaper87: you had some thoughts against it a while back..See https://review.openstack.org/#/c/63449/6/tempest/api/queuing/base.py | 14:49 |
| malini | do you still strongly feel against class level set up? | 14:49 |
| malini | I like how the test level set up /teardown in the current patch set | 14:49 |
| flaper87 | malini: I do but I'm not a tempest core so if that's how they do things then lets do it that way | 14:50 |
| malini | at this point, dont think its a battle worth fight | 14:50 |
| flaper87 | malini: indeed | 14:50 |
| malini | So I'll just update it to how everything else is | 14:50 |
| *** ayoung has quit IRC | 14:50 | |
| *** ayoung has joined #openstack-marconi | 14:50 | |
| flaper87 | and re tests-isolation, psss, nothing breaks tests isolation more than class level hacking | 14:50 |
| malini | flaper87: from the way other projects implement it, it certainly breaks test isolation | 14:51 |
| malini | But we have lot of other things to worry about :( | 14:51 |
| *** jay-atl has quit IRC | 14:52 | |
| flaper87 | malini: but please, make sure you speak up and say that you think Class methods are bad for test-isolation | 14:52 |
| *** alcabrera has joined #openstack-marconi | 14:54 | |
| alcabrera | I'm back, with 100% more desk in front of me. :D | 14:57 |
| alcabrera | flaper87: so, topics as first class citizens in marconi? | 14:58 |
| *** kgriffs_afk is now known as kgriffs | 14:59 | |
| sriram | I read the blog, sounds cool. Context is everything. | 15:00 |
| kgriffs | flaper87: re topics, it's funny you bring that up | 15:04 |
| kgriffs | in RSE, the precursor to Marconi, "queue" was actually just an attribute of a message, and didn't exist as a separate resource | 15:05 |
| malini | now that we have the whole house here ..I have good news & bad newa | 15:05 |
| flaper87 | alcabrera: kgriffs that's the way I think it should be but calling queue would just be confusing | 15:05 |
| * flaper87 in a call | 15:05 | |
| flaper87 | brb | 15:05 |
| flaper87 | calling it* | 15:06 |
| alcabrera | malini? | 15:06 |
| malini | they approved the devstack patch https://review.openstack.org/#/c/77832/ | 15:06 |
| alcabrera | that sounds good, but... | 15:06 |
| cpallares | malini: wooh finally | 15:06 |
| malini | BUT ! | 15:06 |
| malini | 'I still think there are some unanswered questions here that need to be solved before graduation is considered (with my TC hat on)' | 15:06 |
| malini | tht's from Sean | 15:06 |
| kgriffs | referring to what, specifically? | 15:07 |
| kgriffs | is he around? can we invite him in here to discuss? | 15:07 |
| malini | let me chk | 15:07 |
| malini | kgriffs: I think we shud discuss this in #openstack-qa | 15:08 |
| malini | dtroyer was also looking at this on friday | 15:08 |
| malini | so he might have some thoughts | 15:08 |
| kgriffs | ok | 15:10 |
| alcabrera | welp | 15:10 |
| flwang | kgriffs: how about scheduling a short chat with sdague? | 15:10 |
| alcabrera | that sounds like a call for discussion | 15:10 |
| flwang | alcabrera: yep | 15:10 |
| flwang | it will be good to get some thoughts from sdague about our graduation | 15:11 |
| *** jraim has quit IRC | 15:15 | |
| *** jraim has joined #openstack-marconi | 15:17 | |
| kgriffs | ok, he just wants us to figure out the root cause to opening an stderr fd. | 15:21 |
| kgriffs | completely reasonable request | 15:22 |
| malini | tht is reasonable..but why should tht impact our graduation .. | 15:22 |
| malini | anyways I would like us to get to the root of this | 15:22 |
| mpanetta | sudo su - root | 15:23 |
| mpanetta | :P | 15:23 |
| malini | mpanetta: you always make it easy ;) | 15:23 |
| mpanetta | I try :P | 15:24 |
| kgriffs | malini: idk why it should impact graduation specifically | 15:26 |
| malini | kgriffs: I am working on some review comments on my tempest patch..will continue with the stderr stuff, after this | 15:26 |
| kgriffs | They should be able to trust us that we'll fix this bug | 15:26 |
| malini | apparently, they don't | 15:27 |
| kgriffs | <sigh> | 15:27 |
| mpanetta | What is wrong with stderr being open? | 15:27 |
| mpanetta | Or printed to? | 15:27 |
| mpanetta | Can they tell us why that is bad? | 15:27 |
| kgriffs | mpanetta: heh, now *that* the root cause of the root cause, IMHO | 15:28 |
| kgriffs | we should bring that up once we have fixed it on our end | 15:28 |
| malini | exactly! | 15:28 |
| mpanetta | AFAIK It does not say anywhere in the devstack docs that it is illegal to do any printing to stderr... | 15:28 |
| mpanetta | Not that I have read them fully | 15:29 |
| flwang | kgriffs: maybe I missed something, but may I know what's the gap between now and the graduation? except this bug | 15:29 |
| mpanetta | I guess I kinda feel like they are pushing a bug in devstack off on us... | 15:29 |
| mpanetta | Feel free to correct me if I am wrong. | 15:29 |
| malini | mpanetta: The problem is we really need to know what the root cause is, to prove where the issue is | 15:30 |
| malini | Issue #1 : marconi opening stderr fd | 15:30 |
| malini | Issue #2: devstack killing the process | 15:30 |
| malini | We need to solve Issue #1, before we can say Issue #2 is a problem | 15:31 |
| mpanetta | Is issue #1 really an issue? That is what I am trying to figure out... | 15:31 |
| malini | it is an issue from their perspective, because 'nobody else does that' | 15:31 |
| mpanetta | I mean yes, it has been made an issue... Bleh, so confusing... | 15:31 |
| mpanetta | Hmm... | 15:32 |
| mpanetta | Are we so sure? heh | 15:32 |
| mpanetta | I guess we are going on their word on that, but we have found redirects in other projects. | 15:33 |
| kgriffs | unfortunately, 'nobody else does that' is an often-used crutch in OpenStack | 15:33 |
| mpanetta | That is kinda crappy :P | 15:33 |
| alcabrera | bleh | 15:33 |
| flwang | kgriffs: is there any v1.1 blueprint I help to balance your workload? | 15:40 |
| kgriffs | good question | 15:43 |
| kgriffs | flwang: since 1.1 isn't going into Icehouse, I was just thinking we have time to do two more things we've been discussing | 15:44 |
| kgriffs | #1 pop messages support (basically, claim and delete in a single call) | 15:44 |
| kgriffs | #2 make queues lazy per flaper87's idea | 15:45 |
| alcabrera | +1 to both | 15:45 |
| flwang | kgriffs: cool, anything I can help? | 15:45 |
| kgriffs | otherwise, there are several blueprints that already exist that I took my name off that are fair game | 15:45 |
| malini | kgriffs: do you want me to add a new bp to port over functional tests for v1.1 ? | 15:45 |
| kgriffs | that would be great, thanks! | 15:46 |
| kgriffs | I'd like to ship v1.1 in Juno-1 | 15:46 |
| flwang | kgriffs: I'd like to take a little bit more, because I'm shifting more time from Glance to Marconi ;) | 15:46 |
| kgriffs | flwang: ok, cool! | 15:46 |
| kgriffs | we are super happy to have the help! | 15:46 |
| kgriffs | coming up for Juno there will be lots of exciting things to work on | 15:47 |
| flwang | kgriffs: sure, my pleasure | 15:47 |
| kgriffs | queue flavors, notifications, security testing, etc. | 15:47 |
| flwang | yep, notification :D | 15:47 |
| * kgriffs wonders if anyone else thinks security is exciting. :) | 15:47 | |
| kgriffs | speaking of which, I think we should start getting in the habit of doing/updating a thread model for each milestone | 15:47 |
| kgriffs | s/thread/threat | 15:48 |
| kgriffs | flaper87: ^^^ | 15:48 |
| malini | I would love to work on security testing | 15:48 |
| kgriffs | I'd like to create a preflight checklist for every release we do | 15:48 |
| kgriffs | threat model would be on there | 15:48 |
| flwang | kgriffs: cool | 15:48 |
| malini | kgriffs: added new bp https://blueprints.launchpad.net/marconi/+spec/api-v1.1-functional-tests | 15:49 |
| kgriffs | malini: cool; I'd like to schedule a visit for you to come to SAT for some training on that | 15:49 |
| flwang | kgriffs: btw, as I asked above, what's the gap between now and the graduation? | 15:49 |
| kgriffs | malini: can you ask your supervisor if that would be OK with them? | 15:49 |
| malini | kgriffs: tht sounds awesome.. | 15:49 |
| malini | kgriffs: Sure.. | 15:49 |
| flwang | kgriffs: I want to see what i can do | 15:49 |
| kgriffs | malini: kk, follow up with me when you know | 15:50 |
| malini | kgriffs: sure | 15:50 |
| *** tedross has quit IRC | 15:50 | |
| kgriffs | flwang: we need to get the doc patches merged, so I'd appreciate your reviews on those | 15:51 |
| kgriffs | (there are two) | 15:51 |
| kgriffs | (two patches) | 15:51 |
| kgriffs | let's see, what else... | 15:51 |
| kgriffs | tempest integration | 15:51 |
| kgriffs | malini: anything there that you could use some help with? | 15:51 |
| flwang | kgriffs: I noticed them, and I will review them asap. actually, I have downloaded them and have built them successfully locally | 15:51 |
| kgriffs | awesome, thanks! | 15:51 |
| malini | kgriffs: I am working on some comments I got friday..I'll have a new patch set soon | 15:52 |
| *** tedross has joined #openstack-marconi | 15:52 | |
| malini | I am trying to get it merged irrespective of the devstack | 15:52 |
| kgriffs | otherwise, for graduation... | 15:52 |
| flwang | kgriffs: i assume graduation is our highest priority work item for now, right? | 15:52 |
| kgriffs | yeah | 15:53 |
| malini | it'll be good to have a few eyes on the stderr | 15:53 |
| kgriffs | but we are 99% done with the reqs the TC gave us originally | 15:53 |
| kgriffs | https://blueprints.launchpad.net/marconi/+spec/graduation | 15:53 |
| malini | we want different views on what could be going on there | 15:53 |
| flwang | so I got from you: doc, tempest | 15:53 |
| kgriffs | basically, we have docs and tempest left | 15:53 |
| kgriffs | and publishing balajiiyer's Pecan evaluation | 15:53 |
| flwang | kgriffs: fabulous, so will it be happened in Icehouse? | 15:54 |
| flwang | i mean the graduation | 15:54 |
| kgriffs | flwang: we have our graduation review a week from tomorrow during the regular TC meeting | 15:55 |
| kgriffs | so, we'll see! | 15:55 |
| flwang | kgriffs: ah, nice | 15:56 |
| flwang | so, I will review the doc patches asap | 15:56 |
| *** saikrishna_ has joined #openstack-marconi | 15:56 | |
| kgriffs | kk, thanks! | 15:56 |
| flwang | otherwise, any v1.1 bp or new stuff I can pick? | 15:56 |
| kgriffs | balajiiyer: let's try and get those doc patches merged by the end of the week | 15:57 |
| flwang | +1 | 15:57 |
| kgriffs | flwang: feel free to pick up any unassigned 1.1 bps - there are several | 15:57 |
| flwang | kgriffs: cool | 15:57 |
| kgriffs | I will also add a couple new ones soon | 15:57 |
| flwang | kgriffs: btw, seems the design session of summit is opening, is it? | 15:58 |
| kgriffs | flwang: after rc1 is cut we can start allowing v1.1 patches to merge again | 15:58 |
| flwang | i mean opening for submitting | 15:58 |
| kgriffs | flwang: yep, they are open | 15:58 |
| flwang | how many time slots we got? | 15:58 |
| kgriffs | everyone can submit ideas | 15:58 |
| mpanetta | kgriffs: May I have blueberry flavo(u)red queues? :P | 15:59 |
| kgriffs | flwang: good question... I haven't heard yet | 15:59 |
| kgriffs | mpanetta: mmmm | 15:59 |
| kgriffs | mpanetta: as long as you can write a driver that stores messages in blueberries, the answer is "yes" | 15:59 |
| kgriffs | flwang: last year we had 4 slots iirc | 16:00 |
| mpanetta | haha | 16:00 |
| flwang | kgriffs: ah, too less, then I'm afraid I can work out a good idea to compete with flaper87, alcabrera and you :D | 16:01 |
| flwang | O:-) .... | 16:01 |
| kgriffs | mpanetta, malini: https://thoughtstreams.io/kgriffs/technical-communities/#card-3853 | 16:02 |
| mpanetta | kgriffs++ | 16:03 |
| kgriffs | flwang: go ahead and suggest some ideas. My plan is to combine related topics into the same session if we don't have enough slots | 16:03 |
| kgriffs | flwang: also, I think we will have some unconference space in Atlanta, so we can use that for extra topics | 16:03 |
| flwang | kgriffs: nice | 16:03 |
| flwang | kgriffs: +1 | 16:04 |
| *** tedross has quit IRC | 16:04 | |
| kgriffs | flwang: TL;DR - I am a big fan of ideas, and don't like artificially restricting them | 16:04 |
| kgriffs | :) | 16:04 |
| flwang | kgriffs: haha, I'm the labor to implement those ideas if I can | 16:05 |
| flwang | 't work out them | 16:05 |
| kgriffs | w00t | 16:05 |
| * alcabrera catches up | 16:05 | |
| kgriffs | balajiiyer: you have some time to brainstorm ideas for the pecan evaluation report? | 16:06 |
| alcabrera | flwang: yes, compete and make great ideas! :D | 16:07 |
| kgriffs | The leaders I've always admired the most are great at farming ideas. | 16:07 |
| flwang | alcabrera: hehe, as a new comer, I will try, I think I can have more time from now on, I have delivered a lot in Glance | 16:08 |
| *** davidhadas has quit IRC | 16:08 | |
| flwang | alcabrera: so now, I will work on Marconi to find more fun :D | 16:08 |
| alcabrera | good to hear, flwang. You certainly contribute a lot! :D | 16:08 |
| balajiiyer | kgriffs: sure yes. Im about to head out for lunch now, anytime in the afternoon works. (except for 3-4 est) | 16:09 |
| alcabrera | kgriffs: +1 - ideate, ideate, ideate; can't let that muscle atrophy, not in oneself, not in the team, not in the community | 16:09 |
| mpanetta | Hmm, now we need a dalek poster that says ideate, ideate, ideate! | 16:10 |
| alcabrera | hahaha | 16:11 |
| alcabrera | no time lords in this room | 16:11 |
| alcabrera | nope, nope, nope | 16:11 |
| mpanetta | hehe | 16:12 |
| kgriffs | balajiiyer: ok, just ping me when you get back | 16:13 |
| kgriffs | alcabrera: man, this channel often makes me wax philosophical. https://thoughtstreams.io/kgriffs/technical-communities/#card-3854 | 16:14 |
| malini | kgriffs: working on devstack has opened up my mind :D | 16:14 |
| alcabrera | kgriffs: that's how you know you're part of a healthy channel. :) | 16:14 |
| kgriffs | alcabrera: d00d, that comment was so meta | 16:15 |
| kgriffs | :) | 16:15 |
| kgriffs | malini: ;) | 16:15 |
| *** balajiiyer is now known as balajiiyer_afk | 16:16 | |
| mpanetta | My latest (re) obsession has been stargate, not been watching much Doctor Who lately. | 16:16 |
| mpanetta | One can easily tell by my g+ page heh | 16:17 |
| *** amitgandhi has quit IRC | 16:17 | |
| flwang | kgriffs: about this bp https://blueprints.launchpad.net/marconi/+spec/api-v1.1-homedoc-changes | 16:17 |
| alcabrera | hehe | 16:17 |
| flwang | kgriffs: is there any known RFC violations? | 16:17 |
| flaper87 | 'sup people | 16:18 |
| flaper87 | I'm back | 16:18 |
| kgriffs | flwang: I tried to find an updated RFC but it doesn't look like Mark N. has posted one yet... | 16:18 |
| flwang | kgriffs: or just want to confirm | 16:18 |
| flaper87 | looong call | 16:18 |
| flaper87 | so, thoughts on s/queues/topics/ | 16:18 |
| flwang | flaper87: I like the idea | 16:18 |
| kgriffs | flwang: so, we should be still OK based on http://tools.ietf.org/html/draft-nottingham-json-home-03 | 16:19 |
| kgriffs | flwang: I will ping mnot again and see what is up | 16:19 |
| kgriffs | flaper87: I was thinking we could make progress toward that idea in v1.1 of the API | 16:19 |
| kgriffs | I'd like to ship v1.1 for Juno-1 | 16:19 |
| kgriffs | and that gives us some time to play with it | 16:20 |
| flwang | flaper87: just like a default/ transparent queue | 16:20 |
| alcabrera | following Mark N. on Twitter pretty much keeps me up to date on HTTP 2.0 developments. It's surprisingly effective, I've learned. | 16:20 |
| flwang | kgriffs: got it, thanks | 16:20 |
| flaper87 | kgriffs: yeah, althought that sounds more like a API v2 thing | 16:20 |
| flaper87 | :P | 16:20 |
| flaper87 | well, TBH it doesn't | 16:20 |
| flaper87 | I mean, the URL scheme won't change | 16:20 |
| flaper87 | probably the queue metadata endpoints would go away | 16:21 |
| flaper87 | but that's something we need to decide | 16:21 |
| flaper87 | I mean, whether we want to keep them or drop them | 16:21 |
| kgriffs | flaper87: I was thinking we could do some "minor" things in v1.1 | 16:21 |
| flaper87 | kgriffs: awww :( | 16:21 |
| kgriffs | like, not require a queue to exist before you post to it | 16:21 |
| flaper87 | just minors ? | 16:21 |
| flaper87 | :P | 16:21 |
| flaper87 | kgriffs: that sounds good | 16:22 |
| * flaper87 thinking how to implement that in the mongodb driver | 16:22 | |
| flaper87 | (without requiring 2 queries) | 16:23 |
| kgriffs | re queue metadata, I'm not sure how useful it is. It is easy to just store that out of band in a DB or something | 16:23 |
| flaper87 | kgriffs: exactly | 16:23 |
| kgriffs | if we removed that in 1.1 would people be up in arms? | 16:23 |
| kgriffs | I guess it is kind of a 2.0 thing | 16:24 |
| kgriffs | I was just thinking, the sooner we remove it, the sooner we can deprecate it, and the sooner we can remove it | 16:24 |
| kgriffs | hmm | 16:24 |
| kgriffs | It's probably not a big deal to leave it in for 1.1, and we probably should since that seems more like a "major" change | 16:25 |
| flaper87 | so, I wouldn't mind doing it in v1.1 but if we're going to do that then we better discuss the proposal further and do it all | 16:26 |
| flaper87 | well no | 16:27 |
| flaper87 | mmhh | 16:27 |
| flaper87 | we can still drop metadata and make queue lazy in v1.1 | 16:27 |
| flaper87 | and then rename queue with topic (or whatsoever) and drop the resource entirely | 16:27 |
| kgriffs | the rename happens in 2.0? | 16:28 |
| flaper87 | yeah | 16:28 |
| flaper87 | rename + drop queue as a resource in v2.0 | 16:28 |
| kgriffs | At that point I wonder if we just call it a tag and let people set a few tags per message | 16:28 |
| flwang | so will we be named as 'Queue as a Service' anymore? | 16:28 |
| flaper87 | flwang: LOOOOOOOOOOOOOL | 16:29 |
| flaper87 | I said the change was big | 16:29 |
| flaper87 | ;) | 16:29 |
| flaper87 | kgriffs: using just tags scares me a bit. I'm thinking about how we can isolate them completely per tenant and project | 16:30 |
| flaper87 | also, I would like to avoid making some drivers life's hard | 16:30 |
| kgriffs | flaper87: I was thinking to just have, say, 3 tags | 16:30 |
| kgriffs | you can set 1-3 tags or "topics" | 16:30 |
| kgriffs | then you define three columns in your table or whatever | 16:30 |
| kgriffs | queries should still be quite fast | 16:31 |
| *** mkoderer has quit IRC | 16:31 | |
| kgriffs | and null entries won't take up much (if any) space | 16:31 |
| flaper87 | mmh, but wouldn't that be like giving wings to a fairy and then forbid her to use them ? | 16:31 |
| *** saikrishna_ has quit IRC | 16:31 | |
| flaper87 | I mean, setting a limit on 3 tags | 16:32 |
| flaper87 | I like the tags idea | 16:32 |
| flaper87 | TBH, I'm thinking how that could be implemented in a amqp driver | 16:32 |
| kgriffs | ah, yeah | 16:32 |
| flaper87 | I'm pretty sure it's quite easy to do it in mongodb and sqla | 16:32 |
| flaper87 | it's not impossible, it could have 3 different exchanges | 16:32 |
| kgriffs | we will need to define a subset of functionality for drivers | 16:32 |
| flaper87 | or perhaps use 3 different topics | 16:33 |
| flaper87 | etc | 16:33 |
| kgriffs | ah | 16:33 |
| flaper87 | s/3/N/ | 16:33 |
| kgriffs | flaper87: would that be hard to do on the fly, tho? | 16:33 |
| kgriffs | i mean, if any message could have any number of topics? | 16:33 |
| flaper87 | kgriffs: I don't think so | 16:33 |
| flaper87 | I mean, it should be fair simple | 16:33 |
| kgriffs | kk | 16:34 |
| *** amitgandhi has joined #openstack-marconi | 16:34 | |
| kgriffs | in mongo, the query can just be over a list | 16:34 |
| flaper87 | and it would be more like: bulk post on topics | 16:34 |
| kgriffs | in SQL it gets harder | 16:34 |
| flwang | kgriffs: +1 | 16:34 |
| kgriffs | (to make it performant) | 16:34 |
| flwang | flaper87: i'm wondering how to do that in sql | 16:34 |
| *** amitgandhi has quit IRC | 16:34 | |
| flaper87 | what's sql ? | 16:34 |
| * flaper87 has no idea | 16:34 | |
| flaper87 | what's a relational database? | 16:34 |
| * flaper87 has no idea | 16:34 | |
| kgriffs | flaper87: suuuure you don't have any idea | 16:35 |
| flaper87 | :P | 16:35 |
| *** amitgandhi has joined #openstack-marconi | 16:35 | |
| cpallares | haha | 16:35 |
| cpallares | this irc channel is the best | 16:35 |
| flaper87 | cpallares: isn't it? kgriffs is soooooooooooooo funny, he keeps making sql jokes | 16:36 |
| kgriffs | flaper87: you started it! | 16:36 |
| flwang | kgriffs: flaper87: is there any requirement for a signed message? | 16:36 |
| kgriffs | flaper87: ooooh. That is something I want in Juno | 16:37 |
| kgriffs | did you find the bp? | 16:37 |
| alcabrera | select funny where has_sql from kgriffs, flaper87 | 16:37 |
| kgriffs | It could be an optional feature that if you have barbican, you can enable | 16:37 |
| kgriffs | barbican FTW! | 16:37 |
| *** Kuwagata has left #openstack-marconi | 16:37 | |
| kgriffs | we can lookup people's keys in barbican to verify authenticity | 16:38 |
| flwang | kgriffs: so it's not necessary to create a new wheel in Marconi, is it? | 16:38 |
| kgriffs | request header could have the key ID | 16:38 |
| flwang | kgriffs: got it | 16:38 |
| kgriffs | flwang: heck no. This is one of the rare cases where "reinventing the wheel" actually applies. ;) | 16:38 |
| flwang | kgriffs: so? should I post a design session proposal for that? | 16:39 |
| *** amitgandhi has quit IRC | 16:39 | |
| alcabrera | brb lunch. :D | 16:40 |
| *** alcabrera is now known as alcabrera|afk | 16:40 | |
| kgriffs | flwang: please do. It would be great to invite the barbican folks to that | 16:40 |
| flwang | kgriffs: cool | 16:40 |
| flaper87 | If we can rely on barbican for that, it would be really cool | 16:41 |
| flaper87 | anyway, re topics on sql | 16:41 |
| flaper87 | mmh, I guess the easiest way to do it is to insert the message several times | 16:41 |
| flaper87 | and just have 1 topic field | 16:41 |
| flwang | flaper87: MANY TIMES? | 16:42 |
| flaper87 | (which is basically the same thing that would happen in an amqp driver) | 16:42 |
| flaper87 | unless we have 1 text field that we can regex on | 16:42 |
| flaper87 | to get the message | 16:42 |
| flwang | is it possible to map one msg to many tags in another table? | 16:42 |
| flaper87 | flwang: yeah, I guess so | 16:42 |
| flwang | P/F key | 16:43 |
| flaper87 | but wait, there's another thing, we also need to consider deletes | 16:43 |
| flaper87 | deleting a message from a topic | 16:43 |
| flaper87 | in mongodb it would mean .pop on the tags field | 16:43 |
| flaper87 | sqla probably remove it from the msg_topic table | 16:44 |
| flaper87 | and we also need to clean up messages without topics | 16:44 |
| kgriffs | wait | 16:45 |
| kgriffs | I guess I was thinking about this a little differently | 16:45 |
| kgriffs | if you delete a message you just delete it | 16:45 |
| kgriffs | queries for any tags would simply not turn up that deleted message | 16:45 |
| kgriffs | think of a blog | 16:46 |
| kgriffs | where you tag entries | 16:46 |
| flaper87 | kgriffs: mmh, but then tags != topics | 16:46 |
| flaper87 | if you send a message to a topic, it is isolated from other topics | 16:46 |
| flaper87 | mmh, I think I get your point | 16:46 |
| kgriffs | I could send Message A with tag "Foo" | 16:47 |
| flaper87 | it's like, lets send it to these 3 topics and see which one consumes it first | 16:47 |
| kgriffs | then I send message be with tag "Bar" | 16:47 |
| kgriffs | when I query for "Foo" I only get those | 16:47 |
| kgriffs | if I query for "Bar" I only get the others | 16:47 |
| kgriffs | but now I could query "Foo or Bar" and get both messages | 16:47 |
| *** balajiiyer_afk has quit IRC | 16:47 | |
| kgriffs | flaper87: if my workers want to treat the tag sets as disjoint, then that is up to the way you define your tags and the way you publish messages | 16:48 |
| kgriffs | the semantics are emergent | 16:48 |
| flaper87 | yeah | 16:48 |
| flaper87 | I agree with that | 16:48 |
| kgriffs | I guess that was my point - "topic" is really a specialization of "tag" | 16:48 |
| kgriffs | but, this may be tricky to map onto amqp | 16:49 |
| flwang | good discussion :) | 16:49 |
| flaper87 | yeah, so, topic is actually a contextualization whereas tag is an attribute | 16:49 |
| kgriffs | flaper87: but, you can treat a tag as a topic | 16:50 |
| kgriffs | but not the other way around | 16:50 |
| flaper87 | kgriffs: you can, for sure! | 16:50 |
| kgriffs | ...or something like that. :p | 16:50 |
| * flaper87 locks kgriffs keyboard | 16:50 | |
| kgriffs | Wait a minu... | 16:51 |
| * kgriffs goes to get a different KB | 16:51 | |
| flaper87 | so, Message{topics:[a, b, c], id: 1}, Message{topics:[a], id:2}, Messages{topics:[b], id:3} | 16:51 |
| *** saikrishna_ has joined #openstack-marconi | 16:52 | |
| flaper87 | Or in english, a message sent to 3 topics will represent 1 message that lives across several topics | 16:52 |
| kgriffs | ok, I think I see what you mean | 16:52 |
| flaper87 | if the user wants to keep them separate, then the user will need to insert the message separately to 3 topics | 16:52 |
| kgriffs | ok, and now deleting from one topipc means popping from the list | 16:53 |
| kgriffs | gotchya | 16:53 |
| flaper87 | :) | 16:53 |
| * flaper87 thought kgriffs had the ability to read his mind. It is now clear that just flaper87 has the ability to read kgriffs mind | 16:53 | |
| kgriffs | flaper87: you keep thinking that. Gives me an advantage. ;) | 16:54 |
| kgriffs | flaper87: would a person be able to delete a message from all topics in one request? | 16:55 |
| kgriffs | conversely, can I request messages from multiple topics in one request? | 16:55 |
| flaper87 | I think the delete action would look something like: | 16:56 |
| flaper87 | >>> client.delete(topics=[a, b, c]) | 16:57 |
| flaper87 | >>> client.delete(id=1, topics=[a, b, c]) | 16:57 |
| flaper87 | >>> client.delete(id=2, topics=[a]) | 16:57 |
| flaper87 | etc | 16:57 |
| flaper87 | even | 16:57 |
| flaper87 | >>> client.delete(id=3, topics=[a, b, c]) | 16:57 |
| flaper87 | which would delete message 3 from topic b | 16:57 |
| flaper87 | the first example would 'empty' topics a, b and c | 16:57 |
| flaper87 | the second deletes message 1 from a, b and c | 16:58 |
| *** davidhadas has joined #openstack-marconi | 16:58 | |
| flaper87 | the third deletes 2 from a | 16:58 |
| flaper87 | and the fourth deletes 3 from b (because it wasn't created in a and c | 16:58 |
| *** saikrishna_ has quit IRC | 17:01 | |
| kgriffs | hmm | 17:02 |
| kgriffs | if the user is thinking in terms of "topics" then perhaps they would be OK with saying a message can be in max N topics | 17:03 |
| kgriffs | like, 5 | 17:03 |
| kgriffs | today we only let them be in 1 | 17:03 |
| flaper87 | While I agree with that, I'm thinking that the implementation of this stuff will probably enable them to have N topics automagically | 17:04 |
| flaper87 | the bit that is not clear yet is sqla | 17:05 |
| flaper87 | but flwang's idea is quite good | 17:05 |
| flwang | flaper87: what's the my idea? ???? | 17:05 |
| flwang | hah | 17:05 |
| kgriffs | flaper87: well, I think there must be some upper limit, even if it is like 100 | 17:05 |
| kgriffs | since, you don't want to gum up the DB with too many | 17:05 |
| flwang | kgriffs: we need a quota | 17:05 |
| flwang | for overall Marconi | 17:06 |
| kgriffs | +! | 17:06 |
| kgriffs | +1 | 17:06 |
| flaper87 | kgriffs: could that be up to the operator ? | 17:06 |
| kgriffs | and that can be configurable | 17:06 |
| kgriffs | d00d, stop *doing* that! | 17:06 |
| flwang | kgriffs: yes, it's configurable | 17:06 |
| flaper87 | kgriffs: :) | 17:07 |
| flwang | kgriffs: sorry, did you mean we should not do that? :D | 17:07 |
| kgriffs | heh | 17:08 |
| *** saikrishna_ has joined #openstack-marconi | 17:08 | |
| kgriffs | much confusion | 17:08 |
| kgriffs | let's do it | 17:08 |
| kgriffs | it should be configurable like the other limits we have | 17:08 |
| kgriffs | now, if it is not a fixed number, sqla is a bit tricksy | 17:08 |
| kgriffs | i mean, if that number can change after Marconi is deployed, you would have to do schema changes if you have a column for each topic | 17:09 |
| kgriffs | so, that seems like a bad idea | 17:09 |
| kgriffs | (having a separate column for each) | 17:09 |
| kgriffs | I have seen some people have one or two fixed columns then a text blob | 17:10 |
| kgriffs | (for any extra) | 17:10 |
| flwang | kgriffs: oh, it's bad | 17:10 |
| kgriffs | if the DB supports simple text search within a given column, that could work | 17:11 |
| kgriffs | anyway, we need to have a rough plan for sqla before we decide to do this | 17:11 |
| flwang | +1 | 17:11 |
| *** alcabrera|afk is now known as alcabrera | 17:12 | |
| kgriffs | flwang: maybe you can play around with some options? | 17:12 |
| kgriffs | and let us know what you find out? | 17:12 |
| flaper87 | flwang: go go go! | 17:12 |
| flaper87 | :P | 17:12 |
| flaper87 | I think a separate table that keeps the map between a message and a topic might work | 17:12 |
| * flwang will be the labor of flaper87 on this proposal :D | 17:12 | |
| flaper87 | message_id | topic | 17:13 |
| kgriffs | join tables could work | 17:13 |
| flwang | flaper87: that seems like we did in Glance | 17:13 |
| kgriffs | just need to see how performant that would be | 17:13 |
| flaper87 | kgriffs: stop using sqlish terms | 17:13 |
| flaper87 | :P | 17:13 |
| *** balajiiyer has joined #openstack-marconi | 17:13 | |
| kgriffs | flaper87: just trying to normalize the discussion | 17:13 |
| kgriffs | ;) | 17:13 |
| flwang | nosql nosql nosql | 17:14 |
| flaper87 | but that makes my mongoheart sad | 17:14 |
| flaper87 | :D | 17:14 |
| flaper87 | (just kidding) | 17:14 |
| flaper87 | ok, anyway | 17:14 |
| kgriffs | hehe | 17:14 |
| flaper87 | what's the plan then? | 17:14 |
| flaper87 | do we want to discuss this further in the next meeting? | 17:14 |
| flaper87 | so we can vote | 17:14 |
| kgriffs | sure | 17:14 |
| flaper87 | and raise some other concerns | 17:14 |
| flaper87 | and hear other folks thoughts | 17:15 |
| flaper87 | and come up with a better plan | 17:15 |
| flwang | flaper87: kgriffs: I will dig into more | 17:15 |
| kgriffs | feel free to add this to the agenda | 17:15 |
| flwang | to get something back | 17:15 |
| kgriffs | flwang: thanks! | 17:15 |
| flaper87 | flwang: awesome, thanks | 17:15 |
| *** amitgandhi has joined #openstack-marconi | 17:15 | |
| flwang | flaper87: kgriffs: should we raise it in the design session? | 17:15 |
| *** amitgandhi has quit IRC | 17:16 | |
| flwang | after we get an overall plan? | 17:16 |
| kgriffs | so, is this a proposal for v1.1 or 2.0 ? | 17:16 |
| flaper87 | flwang: it depends on whether we have a good plan and if we want it for v1.1 or v2 | 17:16 |
| flaper87 | LOOOOOOL | 17:16 |
| kgriffs | crap! | 17:16 |
| *** amitgandhi has joined #openstack-marconi | 17:16 | |
| flwang | kgriffs: I prefer 2.0 | 17:16 |
| * kgriffs looks for the bug surgically implanted in his head | 17:16 | |
| kgriffs | flwang: so, this is my suggestion | 17:17 |
| kgriffs | we lay the groundwork in 1.1 for this, but we only support 1 topic | 17:17 |
| flwang | since I'm not sure what's the service we will deliver then :D | 17:17 |
| kgriffs | basically, we want topics to be "not a separate resource" | 17:17 |
| kgriffs | in 2.0 we can allow multiple topics? | 17:18 |
| kgriffs | that would be how I would break it down if we don't want to go all the way in 1.1 | 17:18 |
| flaper87 | kgriffs: sounds like a plan | 17:18 |
| flaper87 | what about the renaming ? | 17:18 |
| flaper87 | s/queues/topics/ | 17:18 |
| flaper87 | looks like v2 | 17:18 |
| kgriffs | oh yeah, the renaming I guess would wait until 2.0 | 17:18 |
| flwang | kgriffs: +1, and I want to raise the stupid question again, what's the service we're delivering then? | 17:18 |
| flwang | not queue ? | 17:18 |
| kgriffs | hmm | 17:18 |
| kgriffs | messaging? | 17:18 |
| flaper87 | I think the most critical part is changing the backend | 17:18 |
| kgriffs | but then, people will conflate this with oslo messaging | 17:18 |
| flaper87 | once we've got the storage drivers sorted out, it'll be fairly simple to map the API | 17:19 |
| flaper87 | well, to be fair, Marconi's always been Messaging as a Service | 17:19 |
| flwang | flaper87: any comments for my question? | 17:19 |
| flaper87 | a place where people can post messages | 17:19 |
| flwang | flaper87: hmmmm. | 17:19 |
| flaper87 | Using queuing was just a convention | 17:19 |
| kgriffs | We actually do queue messages, but what we are doing is relaxing the definition of "queuing" | 17:19 |
| flaper87 | which made it simple to explain what the service's goal is | 17:20 |
| flwang | flaper87: I don't think all of the community are on the same page with us/you | 17:20 |
| flaper87 | I'm pretty sure I'm on a different planet | 17:20 |
| flaper87 | don't worry | 17:20 |
| flaper87 | :P | 17:20 |
| flaper87 | jokes apart | 17:20 |
| kgriffs | so, if oslo.messaging didn't exist | 17:20 |
| flaper87 | flwang: I agree, but that's not their / our fault. People is used to think about queues when they think about messages | 17:21 |
| kgriffs | then we could just say Marconi is "OpenStack Messaging Service" or "OpenStack Message Bus" | 17:21 |
| flwang | flaper87: i'm just raising my concern, I mean we should make sure avoid any confusion for the user | 17:21 |
| flaper87 | but then when they install the client the first think they ask is: "How do I post a message?" | 17:21 |
| kgriffs | but people already get Marconi confused with oslo messaging, so I don't know what the best terminology is | 17:21 |
| flaper87 | the disambiguation between oslo.messaging and marconi is the 'service' part in Marconi's description | 17:21 |
| kgriffs | oic | 17:22 |
| flaper87 | oslo.messaging is a messaging library | 17:22 |
| flaper87 | Marconi is a Messaging service | 17:22 |
| kgriffs | well, maybe we propose to rename the programming during Juno? | 17:22 |
| kgriffs | s/programming/program | 17:22 |
| flaper87 | actually, to be fair, oslo.messaging is more an rpc framework / library | 17:22 |
| flwang | kgriffs: maybe | 17:22 |
| * kgriffs stupid muscle memory | 17:22 | |
| flaper87 | but yeah, it has messaging in the name | 17:22 |
| kgriffs | flaper87: maybe oslo should be renamed to RPC? | 17:22 |
| flaper87 | kgriffs: +1 for renaming the program | 17:22 |
| kgriffs | seems like it used to be or something | 17:23 |
| flwang | kgriffs: +1 | 17:23 |
| kgriffs | (oslo.rpc) | 17:23 |
| flaper87 | kgriffs: but that will confuse people with the old oslo-rpc that still lives in the incubator | 17:23 |
| flaper87 | :D | 17:23 |
| kgriffs | AAAAAARRRRRGGGHHHHH | 17:23 |
| * alcabrera listens to the confusion | 17:23 | |
| * kgriffs can't win | 17:23 | |
| alcabrera | an unbeatable legacy | 17:23 |
| * flaper87 confuses alcabrera | 17:23 | |
| flwang | Marconi is a highly-available, modular, scalable message bus for the cloud. Marconi is a new OpenStack project to create a multi-tenant cloud queuing service. The project will define a clean, RESTful API, use a modular architecture, and will support both eventing and job-queuing semantics. Users will be able to customize Marconi to achieve a wide range of performance, durability, availability, and efficiency goals. | 17:23 |
| flaper87 | so, I'd say, lets wait 'til Juno and we come up with a meter description | 17:23 |
| * alcabrera decomplects, and feels better | 17:23 | |
| kgriffs | so oslo-messaging is really just oslo-rpc redux? | 17:23 |
| flwang | I copy-paste above desc from launchpad | 17:23 |
| flaper87 | kgriffs: right | 17:24 |
| kgriffs | flaper87: sounds good. We can also take into account that we want to do notifications under the same program | 17:24 |
| flwang | we're claiming that Marconi is a queuing service | 17:24 |
| flaper87 | flwang: see? '..., ..., ... message bus for the cloud" | 17:24 |
| flwang | so... | 17:24 |
| flwang | flaper87: yep, I saw that | 17:24 |
| kgriffs | Cloud Message Bus was my pick for Rackspace's product | 17:24 |
| flwang | flaper87: but we're also saying: Marconi is a new OpenStack project to create a multi-tenant cloud queuing service. | 17:24 |
| kgriffs | but the marketing wizards liked Cloud Queues better | 17:24 |
| * kgriffs sad panda | 17:25 | |
| kgriffs | flwang: yeah, we need to update the wiki | 17:25 |
| flaper87 | flwang: we can drop that line | 17:25 |
| flaper87 | :P | 17:25 |
| kgriffs | I've been meaning to revamp the wiki for a while now | 17:25 |
| flwang | kgriffs: so maybe we should named 'Message queuing as a Service'? | 17:25 |
| flaper87 | anyway, Messaging as a service covers notification as well | 17:25 |
| kgriffs | flaper87: +1 | 17:25 |
| flwang | highlight the Message, but keeping the 'queuing' | 17:26 |
| flaper87 | KILL THE QUEUE, KILL THE QUEUE | 17:26 |
| flaper87 | people don't like to queue up | 17:26 |
| flaper87 | I hate being queued up | 17:26 |
| flaper87 | :P | 17:26 |
| flaper87 | it's like a waste of time while you watch other people being prioritized | 17:26 |
| flaper87 | :P | 17:26 |
| kgriffs | mission updated | 17:26 |
| flaper87 | and that's a piece of my venezuelan roots | 17:27 |
| flaper87 | :D | 17:27 |
| flwang | I HATE THE TIEMZONE | 17:28 |
| flaper87 | flwang: tell me about it | 17:28 |
| kgriffs | flwang: you are with IBM, right? | 17:28 |
| * alcabrera watched marconi become a message bus | 17:29 | |
| alcabrera | and it was good | 17:29 |
| flwang | kgriffs: yep | 17:29 |
| flwang | flaper87: now it's 1:30AM | 17:29 |
| kgriffs | flwang: d00d, I'm sorry | 17:30 |
| kgriffs | that you are up so late | 17:30 |
| flwang | kgriffs: no worries :) I'm just blaming the timezone :D | 17:30 |
| flaper87 | flwang: don't you dare to go to sleep | 17:30 |
| flaper87 | we've a queuing service to fix here | 17:30 |
| flwang | kgriffs: it's really good to discuss with you guys :) | 17:31 |
| flwang | flaper87: yep, DARK :) | 17:31 |
| * flaper87 next mission. Convert flwang to something different than human | 17:31 | |
| *** tedross has joined #openstack-marconi | 17:31 | |
| balajiiyer | kgriffs: Im back. | 17:31 |
| flwang | flaper87: drop me in a queue | 17:32 |
| flwang | flaper87: oh, no queue now | 17:32 |
| flwang | flaper87: send me as a msg | 17:32 |
| flaper87 | flwang: we'll tag you and send you to some random contry on a pigeon | 17:32 |
| flwang | flaper87: pls make sure the time difference is no more than 6 hours :D | 17:33 |
| alcabrera | the topic is 'marconi-fied' - flwang will travel well | 17:33 |
| flwang | alcabrera: haha, pls :D | 17:33 |
| flwang | guys, I really need some sleep now | 17:34 |
| kgriffs | removed some references to queues on the wiki and added flwang to the core team | 17:34 |
| alcabrera | w00t | 17:34 |
| kgriffs | https://wiki.openstack.org/wiki/Marconi | 17:34 |
| alcabrera | flwang: go rest. :) | 17:34 |
| alcabrera | very RESTful rest | 17:34 |
| flwang | kgriffs: alcabrera: flaper87: have a nice day! | 17:34 |
| kgriffs | note that I also mentioned requirements for getting on the core team | 17:34 |
| kgriffs | flwang: thanks man, catch ya later! | 17:34 |
| flwang | 再见 | 17:35 |
| alcabrera | mentorship and good judgement - good qualities, kgriffs | 17:35 |
| flwang | it means bye :D | 17:35 |
| alcabrera | TIL: 再见 | 17:36 |
| kgriffs | 再见 | 17:36 |
| *** nidhi has joined #openstack-marconi | 17:37 | |
| kgriffs | flaper87: just thought of something | 17:40 |
| kgriffs | listing topics may still be useful | 17:40 |
| flaper87 | kgriffs: mmh, agreed | 17:41 |
| kgriffs | if so, then we will have to do some kind of grouping query if topics don't exist as n outside thing | 17:41 |
| kgriffs | but, maybe listing topics isn't useful | 17:41 |
| kgriffs | idk | 17:41 |
| flaper87 | well, more than listing, I'd say counting | 17:42 |
| kgriffs | that is something an app could always keep track of in their DB as ell | 17:42 |
| kgriffs | hmm | 17:43 |
| kgriffs | so for v1.1 maybe we keep "queue" but we lazy create and we don't let them have metadata | 17:44 |
| flaper87 | sounds good | 17:44 |
| kgriffs | and if everyone keeps using 1.0 because they want the metadata feature, we can add it back into 2.0 | 17:44 |
| flaper87 | and then we do the major API refactor for the API v1 | 17:44 |
| flaper87 | erm v2 | 17:44 |
| kgriffs | kk | 17:44 |
| flaper87 | brb | 17:45 |
| balajiiyer | kgriffs: do you have moment to talk about pecan eval? | 17:56 |
| *** mwagner_lap has quit IRC | 17:56 | |
| kgriffs | yep | 17:56 |
| kgriffs | one moment | 17:57 |
| *** saikrishna_ has quit IRC | 17:57 | |
| kgriffs | balajiiyer: actually give me 5 minutes. I need to go run and grab something | 17:58 |
| balajiiyer | kgriffs: np | 17:59 |
| kgriffs | one sec | 18:02 |
| kgriffs | give me another 2 mins | 18:02 |
| kgriffs | ok, ready | 18:07 |
| kgriffs | balajiiyer: ping | 18:07 |
| balajiiyer | kgriffs: pong | 18:07 |
| kgriffs | so, a few things | 18:07 |
| kgriffs | we need to come up with a basic outline | 18:08 |
| balajiiyer | *listening* | 18:08 |
| kgriffs | it would be cool to come up with a web page that lets people plug in their own weights for the criteria, but idk how much effort that would be | 18:09 |
| kgriffs | basically just a simply spreadsheet | 18:09 |
| *** oz_akan_ has quit IRC | 18:09 | |
| kgriffs | but let me back up | 18:09 |
| kgriffs | so, one of the most useful things I learned when I was in school was this notion of a "decision matrix" | 18:10 |
| kgriffs | you list out the criteria you care about | 18:10 |
| kgriffs | give each a weight based on how important each one is to you | 18:10 |
| kgriffs | and then you score each option | 18:10 |
| kgriffs | that helps you make an objective decision | 18:11 |
| kgriffs | balajiiyer: make sense? | 18:11 |
| balajiiyer | kgriffs: yup. and Agreed. something like this http://www.multi-channelmarketing.com/blog/wp-content/uploads/2010/05/decision-matrix.jpg | 18:11 |
| kgriffs | yep! | 18:11 |
| kgriffs | except you also need weights | 18:12 |
| kgriffs | since each criterion AKA "objective" isn't equal in importance to the others | 18:12 |
| balajiiyer | right, e.g git hub stars will have less weight than performance | 18:12 |
| balajiiyer | far less weight, infact. :) | 18:13 |
| kgriffs | yep | 18:13 |
| kgriffs | balajiiyer: how quickly could we make a dynamic decision matrix that would allow people to put in their own weights? | 18:14 |
| kgriffs | I wish there were a little service for making these | 18:14 |
| balajiiyer | kgriffs: lookng at couple of online tools | 18:15 |
| balajiiyer | http://www.weighteddecision.com/weighted-decision-making-matrix/ | 18:15 |
| balajiiyer | kgriffs: nvm, that is not the tool we are looking for. They sell predefined templates | 18:17 |
| kgriffs | balajiiyer: I bet someone could whip up a quick one with angular or knockout or something | 18:18 |
| kgriffs | just have a list of text fields people can type weights into | 18:18 |
| balajiiyer | kgriffs: yeah, I will write it and host it. Have a couple of meetings this afternoon, could have it ready by tonight | 18:19 |
| kgriffs | ok, | 18:20 |
| kgriffs | next topic | 18:20 |
| kgriffs | can you bring up that etherpad? | 18:20 |
| *** tedross has quit IRC | 18:20 | |
| kgriffs | balajiiyer: so, last think to decide is which criteria to include | 18:25 |
| kgriffs | balajiiyer: one last thing | 18:37 |
| balajiiyer | kgriffs: yup | 18:37 |
| kgriffs | I think it would be good to briefly comment on each weight | 18:37 |
| kgriffs | why this weight was chosen from Marconi's perspective | 18:38 |
| balajiiyer | kgriffs: sure thing, yes. | 18:38 |
| kgriffs | kk | 18:38 |
| kgriffs | so, I thinnk we have a good outline figured out | 18:38 |
| kgriffs | anything else we need to cover? | 18:39 |
| balajiiyer | I will also have to find a balance between this email not being too long and to the point. This outlines helps a lot. | 18:39 |
| kgriffs | kk | 18:39 |
| balajiiyer | kgriffs: so, weighted decision matrix and I will have a draft by tomorrow morning. | 18:39 |
| kgriffs | balajiiyer: ok. Let's get some people outside our team to review the draft ASAP and I'd like to send this out tomorrow before lunch if we can | 18:40 |
| kgriffs | balajiiyer: thanks! | 18:41 |
| balajiiyer | kgriffs: thank YOU | 18:41 |
| kgriffs | flaper87: ^^^ | 18:41 |
| kgriffs | balajiiyer: I was just looking over your benchmarks | 18:48 |
| kgriffs | looks like you used apache bench? | 18:49 |
| balajiiyer | I used tsung and AB | 18:50 |
| *** vkmc has quit IRC | 18:50 | |
| balajiiyer | kgriffs: do you have any other recommendation? | 18:50 |
| kgriffs | oh, let me look for the tsung numbers | 18:51 |
| balajiiyer | kgriffs: hang on. looks like that file has only AB | 18:51 |
| kgriffs | ok | 18:52 |
| kgriffs | while I wait to see that, a couple thoughts | 18:52 |
| kgriffs | first, when benchmarking system noise must be accounted for | 18:53 |
| kgriffs | one way to do that is to run the same benchmark multiple times, then take the BEST time | 18:53 |
| kgriffs | you don't want the everage | 18:53 |
| *** nidhi has quit IRC | 18:53 | |
| kgriffs | the reason things are slower in one run, faster in another is because something in the system went off and slowed down your process | 18:54 |
| balajiiyer | kgriffs: got it. | 18:55 |
| mpanetta | Yay! I am signed up for the openstack summit. | 18:56 |
| mpanetta | BTW if you got your code you have to use it by the 21st to get in free... | 18:56 |
| kgriffs | I actually prefer autobench (and httperf) myself | 18:56 |
| alcabrera | mpanetta: hurray! | 18:56 |
| mpanetta | Just in case you don't know... | 18:56 |
| kgriffs | it is more accurate, and let's you hit the target server from multiple boxes | 18:56 |
| kgriffs | I'm trying to remember if httperf only reports mean numbers | 18:57 |
| *** reed has joined #openstack-marconi | 18:57 | |
| kgriffs | balajiiyer: so, I would recommend re-running with autobench even if you only do it against localhost | 18:58 |
| flaper87 | kgriffs: balajiiyer what's the link to the bench / eval? | 18:58 |
| balajiiyer | kgriffs: ok. | 18:59 |
| kgriffs | balajiiyer: if you look at this blob post, it has some suggestions for autobench params | 19:00 |
| kgriffs | http://blog.kgriffs.com/2012/11/13/python-vs-node-vs-pypy-benchmarks.html | 19:00 |
| kgriffs | s/blob/blog | 19:01 |
| kgriffs | balajiiyer: also, did you just run marcon-server? | 19:01 |
| kgriffs | and did you use the mongodb driver? | 19:01 |
| balajiiyer | kgriffs: yeah, marconi-server with mongodb | 19:03 |
| kgriffs | ah, ok | 19:03 |
| kgriffs | so, I think it would be better to run with uwsgi and enable gevent | 19:03 |
| kgriffs | that will be more realistic | 19:03 |
| kgriffs | basically, it would be great to create a graph like one shown in this post: http://blog.kgriffs.com/2012/12/18/uwsgi-vs-gunicorn-vs-node-benchmarks.html | 19:03 |
| flaper87 | can pecan run under gevent ? | 19:03 |
| flaper87 | 'under' | 19:03 |
| kgriffs | afaik | 19:04 |
| balajiiyer | flaper87: https://drive.google.com/file/d/0B7_erUkmaWcKV3pSUVE1dFFTdm8/edit?usp=sharing | 19:04 |
| *** vkmc has joined #openstack-marconi | 19:04 | |
| *** vkmc has quit IRC | 19:04 | |
| *** vkmc has joined #openstack-marconi | 19:04 | |
| kgriffs | balajiiyer: also, it would be good if mongo was on a separate baox | 19:04 |
| kgriffs | box | 19:04 |
| kgriffs | balajiiyer: and might as well put the autobench client on it's own box as well; that way it isn't fighting for CPU | 19:05 |
| *** oz_akan_ has joined #openstack-marconi | 19:05 | |
| *** oz_akan_ has quit IRC | 19:05 | |
| *** oz_akan_ has joined #openstack-marconi | 19:06 | |
| kgriffs | balajiiyer: I apologize for not bringing all this up earlier | 19:06 |
| kgriffs | the reason this is important is that minor changes in latency can have a big impact when you have lots of concurrent requests, and I want to make sure that is captured | 19:07 |
| * mpanetta smacks kgriffs around with a piece of python code | 19:07 | |
| kgriffs | marconi-server is single-threaded, and is also pretty slow by itself, so it can mask other performance issues | 19:08 |
| balajiiyer | kgriffs: my thinking was all of these have been done before, 1000+ iterations, different drivers, differetn setup.. my idea was not to do an exhaustive benchamarking, but to gather enough metrics for the eval. | 19:08 |
| kgriffs | balajiiyer: so, that graph there | 19:08 |
| kgriffs | what I mean is | 19:08 |
| kgriffs | it should just have two lines | 19:08 |
| kgriffs | one for uwsgi+falcon+mongo | 19:09 |
| kgriffs | and one for uwsgi+pecan+mongo | 19:09 |
| * alcabrera garbage collects mpanetta's python code | 19:09 | |
| kgriffs | the reason I think the graph is important is it makes it obvious what happens when you crank up concurrency | 19:10 |
| mpanetta | haha | 19:10 |
| kgriffs | balajiiyer: make sense? I mean, I don't expect you to go and bench all possible combinations | 19:10 |
| mpanetta | So we don't have any sort of in memory only DB? | 19:10 |
| kgriffs | mpanetta: depends on how you define that | 19:10 |
| mpanetta | Something that won't hit disk | 19:11 |
| kgriffs | so, never hit the disk, we only have sqlite memory db | 19:11 |
| kgriffs | and a POC for redis | 19:11 |
| kgriffs | mongodb memory-maps it's files, so given enough RAM, it's almost as good | 19:12 |
| mpanetta | Hmm ok | 19:12 |
| kgriffs | assuming you don't require disk flushing for every insert | 19:12 |
| kgriffs | s/it's/its | 19:12 |
| mpanetta | Go go gadget english :P | 19:12 |
| balajiiyer | kgriffs: have to run for a meeting now… will be back in a few | 19:13 |
| kgriffs | balajiiyer: kk | 19:13 |
| *** balajiiyer is now known as balajiiyer_afk | 19:13 | |
| kgriffs | mpanetta: can you start looking into how to use autobench? | 19:13 |
| mpanetta | I have the same meeting :P | 19:13 |
| kgriffs | I think balajiiyer will need a partner in crime | 19:13 |
| kgriffs | curses! foiled again! | 19:13 |
| mpanetta | haha | 19:14 |
| mpanetta | bbiab | 19:14 |
| *** Kuwagata has joined #openstack-marconi | 19:34 | |
| openstackgerrit | Cindy Pallares proposed a change to openstack/marconi: Add queue and queue test to API https://review.openstack.org/70210 | 19:35 |
| *** malini is now known as malini_afk | 19:37 | |
| kgriffs | flaper87: when were you arriving in Atlanta? | 19:48 |
| * flaper87 back | 19:50 | |
| flaper87 | kgriffs: 10th ~17:30 | 19:50 |
| kgriffs | ok | 19:50 |
| flaper87 | kgriffs: you? | 19:50 |
| kgriffs | I am making my travel arrangements as we speak | 19:51 |
| flaper87 | I wanted to organize a Marconi team dinner | 19:51 |
| kgriffs | I was thinking about coming in on the 10th as well | 19:51 |
| flaper87 | or drink up | 19:51 |
| kgriffs | +1 | 19:51 |
| flaper87 | or whatever that makes us forget our queued problems | 19:51 |
| kgriffs | heh | 19:51 |
| kgriffs | flaper87: flying home on the 17th? | 19:51 |
| flaper87 | 16th late at night | 19:52 |
| kgriffs | ooh boy | 19:52 |
| flaper87 | yeah, I know | 19:52 |
| *** oz_akan_ has quit IRC | 19:52 | |
| *** oz_akan_ has joined #openstack-marconi | 19:55 | |
| *** oz_akan__ has joined #openstack-marconi | 19:56 | |
| *** oz_akan_ has quit IRC | 19:59 | |
| *** oz_akan__ has quit IRC | 20:09 | |
| kgriffs | mpanetta: done w/ meeting? | 20:09 |
| flaper87 | kgriffs: I think he's hiding from you | 20:14 |
| flaper87 | just sayin' | 20:14 |
| kgriffs | booh | 20:14 |
| kgriffs | I'm not *that* scary | 20:14 |
| kgriffs | well, I didn't shave this morning | 20:15 |
| *** balajiiyer_afk has quit IRC | 20:16 | |
| *** malini_afk is now known as malini | 20:25 | |
| alcabrera | Night, all. :) | 20:25 |
| cpallares | bye alcabrera | 20:26 |
| kgriffs | ciao | 20:27 |
| *** alcabrera has quit IRC | 20:27 | |
| kgriffs | jraim: FYI, there's been a delay in the pecan eval going out... we are trying to get it done now for tomorrow. Balaji has been swamped with some other things. | 20:28 |
| *** ayoung has quit IRC | 20:32 | |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: fix(cli): Print statement in global error handler https://review.openstack.org/78311 | 20:32 |
| jraim | kgriffs: no worries | 20:34 |
| jraim | kgriffs: as an FYI, we got our votes - we have enough to be incubated :) | 20:34 |
| kgriffs | sweet! | 20:34 |
| openstackgerrit | Kurt Griffiths proposed a change to openstack/marconi: Add missing logging trace queues collection https://review.openstack.org/78405 | 20:36 |
| *** oz_akan_ has joined #openstack-marconi | 20:40 | |
| *** sriram has quit IRC | 20:40 | |
| *** oz_akan__ has joined #openstack-marconi | 20:43 | |
| *** oz_akan_ has quit IRC | 20:44 | |
| openstackgerrit | John Heatherington proposed a change to openstack/python-marconiclient: Refactoring iterators https://review.openstack.org/77199 | 21:00 |
| *** oz_akan__ has quit IRC | 21:07 | |
| jraim | kgriffs: ttx just merged our patch - we're officially incubated :) | 21:19 |
| kgriffs | w00000000t | 21:20 |
| kgriffs | jraim: rock on | 21:21 |
| cpallares | yay *\o/* | 21:21 |
| kgriffs | jraim: Barbican is going to be super useful in so many ways | 21:21 |
| jraim | kgriffs: hopefully - now we just have to not screw it up :) | 21:22 |
| kgriffs | :D | 21:22 |
| *** ayoung has joined #openstack-marconi | 21:42 | |
| *** mpanetta has quit IRC | 21:53 | |
| *** kgriffs is now known as kgriffs_afk | 21:53 | |
| *** kgriffs_afk is now known as kgriffs | 21:54 | |
| malini | I am running into a weird out of space issue at the gate now | 22:00 |
| malini | http://logs.openstack.org/49/63449/21/experimental/check-tempest-dsvm-marconi/046a0a6/logs/screen-marconi-server.txt.gz | 22:01 |
| *** Kuwagata has quit IRC | 22:01 | |
| *** amitgandhi has quit IRC | 22:15 | |
| *** jmckind has quit IRC | 22:37 | |
| *** balajiiyer has joined #openstack-marconi | 22:40 | |
| balajiiyer | kgriffs: ping | 22:41 |
| kgriffs | balajiiyer: pong | 22:41 |
| *** mwagner_lap has joined #openstack-marconi | 22:43 | |
| balajiiyer | kgriffs: so, had to go pick up my kid immediately after the meeting. Had to leave in the middle of the conversation. Sorry about that. | 22:43 |
| kgriffs | no worries | 22:43 |
| balajiiyer | Im going to spend sometime tonight working with autobench | 22:43 |
| kgriffs | balajiiyer: I asked mike to work on setting up the boxes | 22:44 |
| kgriffs | 3 perf servers | 22:44 |
| kgriffs | 1 for mongo, 1 for web head with uwsgi, one for autobench | 22:44 |
| kgriffs | that way autobench isn't fighting for CPU | 22:44 |
| kgriffs | and we can really test concurrency | 22:44 |
| balajiiyer | kgriffs: one of the reasons why I stuck with a single box was I had network throughput issues from a cloud server | 22:44 |
| kgriffs | balajiiyer: perf servers should be less prone to that? | 22:45 |
| kgriffs | balajiiyer: hopefully we will be ok with some brand new perf boxes | 22:46 |
| balajiiyer | kgriffs: I hope so. Last Friday I was trying with 500 concurrent reqs for 10 secs, and only 20% of them succeeded | 22:46 |
| kgriffs | balajiiyer: some of that may be due to apache bench | 22:46 |
| kgriffs | some may be due to kernel params | 22:46 |
| balajiiyer | kgriffs: I see. ok | 22:46 |
| kgriffs | we can tune the kernel params (i have a gist) | 22:46 |
| kgriffs | balajiiyer: I mean, it may not have been network - could very well have been just using up local sockets | 22:47 |
| kgriffs | balajiiyer: sorry again for springing this on you last-minute. I should have brought up autobench earlier | 22:47 |
| balajiiyer | kgriffs: no worries. I should have talked about it more about it in this chanvel, instead of working on it in a silo | 22:48 |
| balajiiyer | *channel | 22:48 |
| kgriffs | balajiiyer: if you have a few minutes to play with autobench to get a feel for how it works, then we can hit the ground running in the morning | 22:48 |
| kgriffs | one gotchya that I remember is the way params are passed through to httperf | 22:49 |
| kgriffs | seems like httperf params are named differently or something when you specify them to autohost, but I can't remember for sure | 22:49 |
| kgriffs | we'll have to figure that out so we can pass the --hog option | 22:49 |
| kgriffs | balajiiyer: one of these days I've been meaning to write an HTTP API benchmarking tool in node | 22:50 |
| kgriffs | you know, just for the heck of it | 22:50 |
| kgriffs | I think I could make something that is easier to configure and run than [tsung, autobench, jmeter], esp. if I make the simplifying assumption that this is for APIs, not web pages. | 22:51 |
| balajiiyer | kgriffs: +1 and thats what is unique about you. I go about complaining things, rather than fixing it. :) | 22:52 |
| kgriffs | my grandfather was a carpenter. If he didn't have a tool he needed, he wasn't afraid to build it. | 22:52 |
| kgriffs | balajiiyer: lol | 22:52 |
| kgriffs | anyway, I gotta run. You should be able to glean some autobench info from my blog | 22:52 |
| balajiiyer | kgriffs: malini has been talking about 'Tsung as a Service' for a while now | 22:52 |
| kgriffs | I wish I had recorded the exact autobench config file and such, but I neglected to. | 22:53 |
| kgriffs | balajiiyer: there's something like that, iirc, but WAY TOO EXPENSIVE | 22:53 |
| kgriffs | anyway, ttfn | 22:53 |
| kgriffs | have a good night! | 22:53 |
| balajiiyer | kgriffs: take care | 22:53 |
| kgriffs | balajiiyer: p.s. - got my flight to Atlanta. I'll be there May 10th (sat). | 22:54 |
| *** balajiiyer has left #openstack-marconi | 22:54 | |
| kgriffs | If anyone wants to go for food and/or drinks, Flavio's paying | 22:54 |
| kgriffs | ;) | 22:54 |
| kgriffs | (he said that, didn't he?) | 22:54 |
| kgriffs | bye awesome Marconi peeps! | 22:55 |
| *** kgriffs is now known as kgriffs_afk | 23:04 | |
| *** flaper87 is now known as flaper87|afk | 23:22 | |
| *** vkmc has quit IRC | 23:43 | |
| *** jcru has quit IRC | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
