Monday, 2014-03-10

*** mpanetta has quit IRC00:35
*** balajiiyer has joined #openstack-marconi00:59
*** balajiiyer has quit IRC01:17
*** mwagner_lap has quit IRC01:20
*** mpanetta has joined #openstack-marconi01:31
*** nosnos has joined #openstack-marconi01:37
*** nosnos has quit IRC01:38
*** nosnos has joined #openstack-marconi01:40
*** haomaiwa_ has joined #openstack-marconi02:06
*** haomaiwang has quit IRC02:06
*** mpanetta has quit IRC02:16
*** haomaiwang has joined #openstack-marconi02:41
*** flwang has quit IRC02:45
*** haomaiwa_ has quit IRC02:45
*** flwang has joined #openstack-marconi03:28
*** anamalagon_ has joined #openstack-marconi03:41
*** anamalagon has quit IRC03:43
*** mwagner_lap has joined #openstack-marconi04:08
*** haomaiwa_ has joined #openstack-marconi04:33
*** haomaiwang has quit IRC04:34
*** chandan_kumar has joined #openstack-marconi04:43
*** chandan_kumar has quit IRC04:52
*** saju_m has joined #openstack-marconi04:59
*** chandan_kumar has joined #openstack-marconi05:01
*** prashanthr_ has quit IRC05:32
*** nosnos has quit IRC05:39
*** nosnos has joined #openstack-marconi05:39
*** flwang has quit IRC05:40
*** flwang has joined #openstack-marconi05:40
*** saju_m has quit IRC05:54
*** saju_m has joined #openstack-marconi05:55
*** saju_m has quit IRC06:00
*** saju_m has joined #openstack-marconi06:02
*** belief has quit IRC06:07
*** dimolya has joined #openstack-marconi06:28
*** nosnos_ has joined #openstack-marconi06:51
*** nosnos has quit IRC06:51
*** nosnos has joined #openstack-marconi06:56
*** nosnos_ has quit IRC06:58
*** dimolya has quit IRC07:04
*** nosnos has quit IRC07:05
*** nosnos has joined #openstack-marconi07:08
*** nosnos has quit IRC07:12
*** nosnos has joined #openstack-marconi07:12
*** davidhadas has quit IRC07:16
*** mkoderer has joined #openstack-marconi07:50
*** davidhadas has joined #openstack-marconi08:09
*** flaper87|afk is now known as flaper8708:14
*** saju_m has quit IRC08:22
*** saju_m has joined #openstack-marconi08:28
*** davidhadas has quit IRC08:31
*** davidhadas has joined #openstack-marconi08:31
*** saju_m has quit IRC08:50
*** saju_m has joined #openstack-marconi09:04
*** saju_m has quit IRC09:10
*** saju_m has joined #openstack-marconi09:11
*** flwang has quit IRC09:24
*** fvollero|PTO is now known as fvollero09:31
*** davidhadas has quit IRC10:34
*** davidhadas has joined #openstack-marconi10:36
*** saju_m has quit IRC11:05
*** vkmc has joined #openstack-marconi11:34
*** nidhi_ has joined #openstack-marconi12:09
*** mwagner_lap has quit IRC12:28
*** flwang has joined #openstack-marconi12:42
*** ayoung-ZZZZZZZzz has quit IRC12:45
*** vkmc has quit IRC12:46
*** sriram has joined #openstack-marconi12:55
*** sriram has quit IRC12:56
*** sriram has joined #openstack-marconi12:56
*** oz_akan_ has joined #openstack-marconi13:05
*** tedross has joined #openstack-marconi13:06
*** jcru has joined #openstack-marconi13:14
*** rustlebee is now known as russellb13:26
*** alcabrera has joined #openstack-marconi13:26
*** oz_akan_ has quit IRC13:27
*** mpanetta has joined #openstack-marconi13:28
*** oz_akan_ has joined #openstack-marconi13:28
*** oz_akan_ has quit IRC13:32
*** amitgandhi has joined #openstack-marconi13:37
*** vkmc has joined #openstack-marconi13:37
*** nosnos has quit IRC13:38
*** amitgandhi has quit IRC13:39
*** nidhi_ has quit IRC13:39
*** amitgandhi has joined #openstack-marconi13:39
*** oz_akan_ has joined #openstack-marconi13:42
flaper87alcabrera: goood morning13:42
alcabreraflaper87: good morning! :)13:44
flaper87alcabrera: how are you doing?13:44
flaper87alcabrera: had a nice weekend?13:44
alcabreraflaper87: it was pretty good. Played some Path of Exile with Jess and house mate. You? :)13:45
flaper87alcabrera: awesome! I... ate... literally.13:46
flaper87Went to visit my grandma and she literally made me eat the whole day13:47
flaper87Just like any Italian movie out there! :D13:47
flaper87Those movies don't lie :P13:47
alcabreraI remember seeing a Tweet about that, and was like, "I bet that's *a lot* of delicious food!", flaper87. :P13:48
mpanettaMmmm food13:48
*** cpallares has joined #openstack-marconi13:50
flaper87alcabrera: yeah, tons of delicious things13:50
vkmcHi all!13:50
vkmcOh... nonna's food :)13:51
alcabreravkmc: hey!13:51
vkmcalcabrera, o/13:51
*** jmckind has joined #openstack-marconi13:51
flaper87vkmc: hey hey :D13:52
flaper87alcabrera et all, feedback ?
flaper87(I need to fix the code samples, they got rendered very weird)13:52
alcabreraqueues as topics13:52
vkmcflaper87, o/ :)13:53
alcabreraI like it13:53
alcabreraI'm favorable towards lazy queue creation in general13:53
alcabreraI'm *even* more curious about how thinking of queues as topics will change our design thinking13:54
alcabrerafor example13:54
alcabreramulti-topic messages13:54
alcabreraephemeral topics ("queues" that exist only for a short time)13:54
alcabreraprivate/public topics13:54
alcabreraflaper87: feedback rendered - I'd love to explore queues as topics.13:56
flaper87alcabrera: yeah, I put some serious thoughts on that and I'm very convinced that we (1) don't need queues, (2) We can't preserve some features without making them first-citizen resources in Marconi13:57
*** tedross has quit IRC13:59
*** tedross has joined #openstack-marconi14:00
*** mwagner_lap has joined #openstack-marconi14:03
alcabreraI'll be back in a bit, all.14:03
alcabreraDone at the dentist!14:03
alcabreraSee you in about 75ish. :D14:03
*** alcabrera has quit IRC14:03
*** malini_afk is now known as malini14:06
*** balajiiyer has joined #openstack-marconi14:20
*** rwsu has joined #openstack-marconi14:36
*** ayoung has joined #openstack-marconi14:41
*** Kuwagata has joined #openstack-marconi14:44
maliniflaper87: ping14:49
flaper87malini: pong14:49
maliniflaper87: how are you?14:49
flaper87malini: I'm doing well. How are you?14:49
malinigood..I have my brain back today :)14:49
flaper87malini: hehehe14:49
maliniflaper87: I got some feedback on tempest patch suggesting to use class level setup/tearDown for tests14:49
malinibecause tht is how they do it for everything else14:49
openstackgerritJenkins proposed a change to openstack/marconi: Updated from global requirements
maliniflaper87: you had some thoughts against it a while back..See
malinido you still strongly feel against class level set up?14:49
maliniI like how the test level set up /teardown in the current patch set14:49
flaper87malini: I do but I'm not a tempest core so if that's how they do things then lets do it that way14:50
maliniat this point, dont think  its a battle worth fight14:50
flaper87malini: indeed14:50
maliniSo I'll just update it to how everything else is14:50
*** ayoung has quit IRC14:50
*** ayoung has joined #openstack-marconi14:50
flaper87and re tests-isolation, psss, nothing breaks tests isolation more than class level hacking14:50
maliniflaper87: from the way other projects implement it, it certainly breaks test isolation14:51
maliniBut we have lot of other things to worry about :(14:51
*** jay-atl has quit IRC14:52
flaper87malini: but please, make sure you speak up and say that you think Class methods are bad for test-isolation14:52
*** alcabrera has joined #openstack-marconi14:54
alcabreraI'm back, with 100% more desk in front of me. :D14:57
alcabreraflaper87: so, topics as first class citizens in marconi?14:58
*** kgriffs_afk is now known as kgriffs14:59
sriramI read the blog, sounds cool. Context is everything.15:00
kgriffsflaper87: re topics, it's funny you bring that up15:04
kgriffsin RSE, the precursor to Marconi, "queue" was actually just an attribute of a message, and didn't exist as a separate resource15:05
malininow that we have the whole house here ..I have good news & bad newa15:05
flaper87alcabrera: kgriffs that's the way I think it should be but calling queue would just be confusing15:05
* flaper87 in a call15:05
flaper87calling it*15:06
malinithey approved the devstack patch
alcabrerathat sounds good, but...15:06
cpallaresmalini: wooh finally15:06
maliniBUT !15:06
malini'I still think there are some unanswered questions here that need to be solved before graduation is considered (with my TC hat on)'15:06
malinitht's from Sean15:06
kgriffsreferring to what, specifically?15:07
kgriffsis he around? can we invite him in here to discuss?15:07
malinilet me chk15:07
malinikgriffs: I think we shud discuss this in #openstack-qa15:08
malinidtroyer was also looking at this on friday15:08
maliniso he might have some thoughts15:08
flwangkgriffs: how about scheduling a short chat with sdague?15:10
alcabrerathat sounds like a call for discussion15:10
flwangalcabrera: yep15:10
flwangit will be good to get some thoughts from sdague about our graduation15:11
*** jraim has quit IRC15:15
*** jraim has joined #openstack-marconi15:17
kgriffsok, he just wants us to figure out the root cause to opening an stderr fd.15:21
kgriffscompletely reasonable request15:22
malinitht is reasonable..but why should tht impact our graduation ..15:22
malinianyways I would like us to get to the root of this15:22
mpanettasudo su - root15:23
malinimpanetta: you always make it easy ;)15:23
mpanettaI try :P15:24
kgriffsmalini: idk why it should impact graduation specifically15:26
malinikgriffs: I am working on some review comments  on my tempest patch..will continue with the stderr stuff, after this15:26
kgriffsThey should be able to trust us that we'll fix this bug15:26
maliniapparently, they don't15:27
mpanettaWhat is wrong with stderr being open?15:27
mpanettaOr printed to?15:27
mpanettaCan they tell us why that is bad?15:27
kgriffsmpanetta: heh, now *that* the root cause of the root cause, IMHO15:28
kgriffswe should bring that up once we have fixed it on our end15:28
mpanettaAFAIK It does not say anywhere in the devstack docs that it is illegal to do any printing to stderr...15:28
mpanettaNot that I have read them fully15:29
flwangkgriffs: maybe I missed something, but may I know what's the gap between now and the graduation? except this bug15:29
mpanettaI guess I kinda feel like they are pushing a bug in devstack off on us...15:29
mpanettaFeel free to correct me if I am wrong.15:29
malinimpanetta: The problem is we really need to know what the root cause is, to prove where the issue is15:30
maliniIssue #1 : marconi opening stderr fd15:30
maliniIssue #2: devstack killing the process15:30
maliniWe need to solve Issue #1, before we can say Issue #2 is a problem15:31
mpanettaIs issue #1 really an issue?  That is what I am trying to figure out...15:31
maliniit is an issue from their perspective, because 'nobody else does that'15:31
mpanettaI mean yes, it has been made an issue...  Bleh, so confusing...15:31
mpanettaAre we so sure? heh15:32
mpanettaI guess we are going on their word on that, but we have found redirects in other projects.15:33
kgriffsunfortunately, 'nobody else does that' is an often-used crutch in OpenStack15:33
mpanettaThat is kinda crappy :P15:33
flwangkgriffs: is there any v1.1 blueprint I help to balance your workload?15:40
kgriffsgood question15:43
kgriffsflwang: since 1.1 isn't going into Icehouse, I was just thinking we have time to do two more things we've been discussing15:44
kgriffs#1 pop messages support (basically, claim and delete in a single call)15:44
kgriffs#2 make queues lazy per flaper87's idea15:45
alcabrera+1 to both15:45
flwangkgriffs: cool, anything I can help?15:45
kgriffsotherwise, there are several blueprints that already exist that I took my name off that are fair game15:45
malinikgriffs: do you want me to add a new bp to port over functional tests for v1.1 ?15:45
kgriffsthat would be great, thanks!15:46
kgriffsI'd like to ship v1.1 in Juno-115:46
flwangkgriffs: I'd like to take a little bit more, because I'm shifting more time from Glance to Marconi ;)15:46
kgriffsflwang: ok, cool!15:46
kgriffswe are super happy to have the help!15:46
kgriffscoming up for Juno there will be lots of exciting things to work on15:47
flwangkgriffs: sure, my pleasure15:47
kgriffsqueue flavors, notifications, security testing, etc.15:47
flwangyep, notification :D15:47
* kgriffs wonders if anyone else thinks security is exciting. :)15:47
kgriffsspeaking of which, I think we should start getting in the habit of doing/updating a thread model for each milestone15:47
kgriffsflaper87: ^^^15:48
maliniI would love to work on security testing15:48
kgriffsI'd like to create a preflight checklist for every release we do15:48
kgriffsthreat model would be on there15:48
flwangkgriffs: cool15:48
malinikgriffs: added new bp
kgriffsmalini: cool; I'd like to schedule a visit for you to come to SAT for some training on that15:49
flwangkgriffs: btw, as I asked above, what's the gap between now and the graduation?15:49
kgriffsmalini: can you ask your supervisor if that would be OK with them?15:49
malinikgriffs: tht sounds awesome..15:49
malinikgriffs: Sure..15:49
flwangkgriffs: I want to see what i can do15:49
kgriffsmalini: kk, follow up with me when you know15:50
malinikgriffs: sure15:50
*** tedross has quit IRC15:50
kgriffsflwang: we need to get the doc patches merged, so I'd appreciate your reviews on those15:51
kgriffs(there are two)15:51
kgriffs(two patches)15:51
kgriffslet's see, what else...15:51
kgriffstempest integration15:51
kgriffsmalini: anything there that you could use some help with?15:51
flwangkgriffs: I noticed them, and I will review them asap. actually, I have downloaded them and have built them successfully locally15:51
kgriffsawesome, thanks!15:51
malinikgriffs: I am working on some comments I got friday..I'll have a new patch set soon15:52
*** tedross has joined #openstack-marconi15:52
maliniI am trying to get it merged irrespective of the devstack15:52
kgriffsotherwise, for graduation...15:52
flwangkgriffs: i assume graduation is our highest priority work item for now, right?15:52
maliniit'll be good to have a few eyes on the stderr15:53
kgriffsbut we are 99% done with the reqs the TC gave us originally15:53
maliniwe want different views on what could be going on there15:53
flwangso I got from you: doc, tempest15:53
kgriffsbasically, we have docs and tempest left15:53
kgriffsand publishing balajiiyer's Pecan evaluation15:53
flwangkgriffs: fabulous, so will it be happened in Icehouse?15:54
flwangi mean the graduation15:54
kgriffsflwang: we have our graduation review a week from tomorrow during the regular TC meeting15:55
kgriffsso, we'll see!15:55
flwangkgriffs: ah, nice15:56
flwangso, I will review the doc patches asap15:56
*** saikrishna_ has joined #openstack-marconi15:56
kgriffskk, thanks!15:56
flwangotherwise, any v1.1 bp or new stuff I can pick?15:56
kgriffsbalajiiyer: let's try and get those doc patches merged by the end of the week15:57
kgriffsflwang: feel free to pick up any unassigned 1.1 bps - there are several15:57
flwangkgriffs: cool15:57
kgriffsI will also add a couple new ones soon15:57
flwangkgriffs: btw, seems the design session of summit is opening, is it?15:58
kgriffsflwang: after rc1 is cut we can start allowing v1.1 patches to merge again15:58
flwangi mean opening for submitting15:58
kgriffsflwang: yep, they are open15:58
flwanghow many time slots we  got?15:58
kgriffseveryone can submit ideas15:58
mpanettakgriffs: May I have blueberry flavo(u)red queues? :P15:59
kgriffsflwang: good question... I haven't heard yet15:59
kgriffsmpanetta: mmmm15:59
kgriffsmpanetta: as long as you can write a driver that stores messages in blueberries, the answer is "yes"15:59
kgriffsflwang: last year we had 4 slots iirc16:00
flwangkgriffs: ah, too less, then I'm afraid I can work out a good idea to compete with flaper87, alcabrera and you :D16:01
flwangO:-) ....16:01
kgriffsmpanetta, malini:
kgriffsflwang: go ahead and suggest some ideas. My plan is to combine related topics into the same session if we don't have enough slots16:03
kgriffsflwang: also, I think we will have some unconference space in Atlanta, so we can use that for extra topics16:03
flwangkgriffs: nice16:03
flwangkgriffs: +116:04
*** tedross has quit IRC16:04
kgriffsflwang: TL;DR - I am a big fan of ideas, and don't like artificially restricting them16:04
flwangkgriffs: haha, I'm the labor to implement those ideas if I can16:05
flwang't work out them16:05
* alcabrera catches up16:05
kgriffsbalajiiyer: you have some time to brainstorm ideas for the pecan evaluation report?16:06
alcabreraflwang: yes, compete and make great ideas! :D16:07
kgriffsThe leaders I've always admired the most are great at farming ideas.16:07
flwangalcabrera: hehe, as a new comer, I will try, I think I can have more time from now on, I have delivered a lot in Glance16:08
*** davidhadas has quit IRC16:08
flwangalcabrera: so now, I will work on Marconi to find more fun :D16:08
alcabreragood to hear, flwang. You certainly contribute a lot! :D16:08
balajiiyerkgriffs: sure yes. Im about to head out for lunch now, anytime in the afternoon works. (except for 3-4 est)16:09
alcabrerakgriffs: +1 - ideate, ideate, ideate; can't let that muscle atrophy, not in oneself, not in the team, not in the community16:09
mpanettaHmm, now we need a dalek poster that says ideate, ideate, ideate!16:10
alcabrerano time lords in this room16:11
alcabreranope, nope, nope16:11
kgriffsbalajiiyer: ok, just ping me when you get back16:13
kgriffsalcabrera: man, this channel often makes me wax philosophical.
malinikgriffs: working on devstack has opened up my mind :D16:14
alcabrerakgriffs: that's how you know you're part of a healthy channel. :)16:14
kgriffsalcabrera: d00d, that comment was so meta16:15
kgriffsmalini: ;)16:15
*** balajiiyer is now known as balajiiyer_afk16:16
mpanettaMy latest (re) obsession has been stargate, not been watching much Doctor Who lately.16:16
mpanettaOne can easily tell by my g+ page heh16:17
*** amitgandhi has quit IRC16:17
flwangkgriffs: about this bp
flwangkgriffs: is there any known RFC violations?16:17
flaper87'sup people16:18
flaper87I'm back16:18
kgriffsflwang: I tried to find an updated RFC but it doesn't look like Mark N. has posted one yet...16:18
flwangkgriffs: or just want to confirm16:18
flaper87looong call16:18
flaper87so, thoughts on s/queues/topics/16:18
flwangflaper87: I like the idea16:18
kgriffsflwang: so, we should be still OK based on
kgriffsflwang: I will ping mnot again and see what is up16:19
kgriffsflaper87: I was thinking we could make progress toward that idea in v1.1 of the API16:19
kgriffsI'd like to ship v1.1 for Juno-116:19
kgriffsand that gives us some time to play with it16:20
flwangflaper87: just like a default/ transparent queue16:20
alcabrerafollowing Mark N. on Twitter pretty much keeps me up to date on HTTP 2.0 developments. It's surprisingly effective, I've learned.16:20
flwangkgriffs: got it, thanks16:20
flaper87kgriffs: yeah, althought that sounds more like a API v2 thing16:20
flaper87well, TBH it doesn't16:20
flaper87I mean, the URL scheme won't change16:20
flaper87probably the queue metadata endpoints would go away16:21
flaper87but that's something we need to decide16:21
flaper87I mean, whether we want to keep them or drop them16:21
kgriffsflaper87: I was thinking we could do some "minor" things in v1.116:21
flaper87kgriffs: awww :(16:21
kgriffslike, not require a queue to exist before you post to it16:21
flaper87just minors ?16:21
flaper87kgriffs: that sounds good16:22
* flaper87 thinking how to implement that in the mongodb driver16:22
flaper87(without requiring 2 queries)16:23
kgriffsre queue metadata, I'm not sure how useful it is. It is easy to just store that out of band in a DB or something16:23
flaper87kgriffs: exactly16:23
kgriffsif we removed that in 1.1 would people be up in arms?16:23
kgriffsI guess it is kind of a 2.0 thing16:24
kgriffsI was just thinking, the sooner we remove it, the sooner we can deprecate it, and the sooner we can remove it16:24
kgriffsIt's probably not a big deal to leave it in for 1.1, and we probably should since that seems more like a "major" change16:25
flaper87so, I wouldn't mind doing it in v1.1 but if we're going to do that then we better discuss the proposal further and do it all16:26
flaper87well no16:27
flaper87we can still drop metadata and make queue lazy in v1.116:27
flaper87and then rename queue with topic (or whatsoever) and drop the resource entirely16:27
kgriffsthe rename happens in 2.0?16:28
flaper87rename + drop queue as a resource in v2.016:28
kgriffsAt that point I wonder if we just call it a tag and let people set a few tags per message16:28
flwangso will we be named as 'Queue as a Service' anymore?16:28
flaper87flwang: LOOOOOOOOOOOOOL16:29
flaper87I said the change was big16:29
flaper87kgriffs: using just tags scares me a bit. I'm thinking about how we can isolate them completely per tenant and project16:30
flaper87also, I would like to avoid making some drivers life's hard16:30
kgriffsflaper87: I was thinking to just have, say, 3 tags16:30
kgriffsyou can set 1-3 tags or "topics"16:30
kgriffsthen you define three columns in your table or whatever16:30
kgriffsqueries should still be quite fast16:31
*** mkoderer has quit IRC16:31
kgriffsand null entries won't take up much (if any) space16:31
flaper87mmh, but wouldn't that be like giving wings to a fairy and then forbid her to use them ?16:31
*** saikrishna_ has quit IRC16:31
flaper87I mean, setting a limit on 3 tags16:32
flaper87I like the tags idea16:32
flaper87TBH, I'm thinking how that could be implemented in a amqp driver16:32
kgriffsah, yeah16:32
flaper87I'm pretty sure it's quite easy to do it in mongodb and sqla16:32
flaper87it's not impossible, it could have 3 different exchanges16:32
kgriffswe will need to define a subset of functionality for drivers16:32
flaper87or perhaps use 3 different topics16:33
kgriffsflaper87: would that be hard to do on the fly, tho?16:33
kgriffsi mean, if any message could have any number of topics?16:33
flaper87kgriffs: I don't think so16:33
flaper87I mean, it should be fair simple16:33
*** amitgandhi has joined #openstack-marconi16:34
kgriffsin mongo, the query can just be over a list16:34
flaper87and it would be more like: bulk post on topics16:34
kgriffsin SQL it gets harder16:34
flwangkgriffs: +116:34
kgriffs(to make it performant)16:34
flwangflaper87: i'm wondering how to do that in sql16:34
*** amitgandhi has quit IRC16:34
flaper87what's sql ?16:34
* flaper87 has no idea16:34
flaper87what's a relational database?16:34
* flaper87 has no idea16:34
kgriffsflaper87: suuuure you don't have any idea16:35
*** amitgandhi has joined #openstack-marconi16:35
cpallaresthis irc channel is the best16:35
flaper87cpallares: isn't it? kgriffs is soooooooooooooo funny, he keeps making sql jokes16:36
kgriffsflaper87: you started it!16:36
flwangkgriffs: flaper87: is there any requirement for a signed message?16:36
kgriffsflaper87: ooooh. That is something I want in Juno16:37
kgriffsdid you find the bp?16:37
alcabreraselect funny where has_sql from kgriffs, flaper8716:37
kgriffsIt could be an optional feature that if you have barbican, you can enable16:37
kgriffsbarbican FTW!16:37
*** Kuwagata has left #openstack-marconi16:37
kgriffswe can lookup people's keys in barbican to verify authenticity16:38
flwangkgriffs: so it's not necessary to create a new wheel in Marconi, is it?16:38
kgriffsrequest header could have the key ID16:38
flwangkgriffs: got it16:38
kgriffsflwang: heck no. This is one of the rare cases where "reinventing the wheel" actually applies. ;)16:38
flwangkgriffs: so? should I post a design session proposal for that?16:39
*** amitgandhi has quit IRC16:39
alcabrerabrb lunch. :D16:40
*** alcabrera is now known as alcabrera|afk16:40
kgriffsflwang: please do. It would be great to invite the barbican folks to that16:40
flwangkgriffs: cool16:40
flaper87If we can rely on barbican for that, it would be really cool16:41
flaper87anyway, re topics on sql16:41
flaper87mmh, I guess the easiest way to do it is to insert the message several times16:41
flaper87and just have 1 topic field16:41
flwangflaper87:  MANY TIMES?16:42
flaper87(which is basically the same thing that would happen in an amqp driver)16:42
flaper87unless we have 1 text field that we can regex on16:42
flaper87to get the message16:42
flwangis it possible to map one msg to many tags in another table?16:42
flaper87flwang: yeah, I guess so16:42
flwangP/F key16:43
flaper87but wait, there's another thing, we also need to consider deletes16:43
flaper87deleting a message from a topic16:43
flaper87in mongodb it would mean .pop on the tags field16:43
flaper87sqla probably remove it from the msg_topic table16:44
flaper87and we also need to clean up messages without topics16:44
kgriffsI guess I was thinking about this a little differently16:45
kgriffsif you delete a message you just delete it16:45
kgriffsqueries for any tags would simply not turn up that deleted message16:45
kgriffsthink of a blog16:46
kgriffswhere you tag entries16:46
flaper87kgriffs: mmh, but then tags != topics16:46
flaper87if you send a message to a topic, it is isolated from other topics16:46
flaper87mmh, I think I get your point16:46
kgriffsI could send Message A with tag "Foo"16:47
flaper87it's like, lets send it to these 3 topics and see which one consumes it first16:47
kgriffsthen I send message be with tag "Bar"16:47
kgriffswhen I query for "Foo" I only get those16:47
kgriffsif I query for "Bar" I only get the others16:47
kgriffsbut now I could query "Foo or Bar" and get both messages16:47
*** balajiiyer_afk has quit IRC16:47
kgriffsflaper87: if my workers want to treat the tag sets as disjoint, then that is up to the way you define your tags and the way you publish messages16:48
kgriffsthe semantics are emergent16:48
flaper87I agree with that16:48
kgriffsI guess that was my point - "topic" is really a specialization of "tag"16:48
kgriffsbut, this may be tricky to map onto amqp16:49
flwanggood discussion :)16:49
flaper87yeah, so, topic is actually a contextualization whereas tag is an attribute16:49
kgriffsflaper87: but, you can treat a tag as a topic16:50
kgriffsbut not the other way around16:50
flaper87kgriffs: you can, for sure!16:50
kgriffs...or something like that. :p16:50
* flaper87 locks kgriffs keyboard16:50
kgriffsWait a minu...16:51
* kgriffs goes to get a different KB16:51
flaper87so, Message{topics:[a, b, c], id: 1}, Message{topics:[a], id:2}, Messages{topics:[b], id:3}16:51
*** saikrishna_ has joined #openstack-marconi16:52
flaper87Or in english, a message sent to 3 topics will represent 1 message that lives across several topics16:52
kgriffsok, I think I see what you mean16:52
flaper87if the user wants to keep them separate, then the user will need to insert the message separately to 3 topics16:52
kgriffsok, and now deleting from one topipc means popping from the list16:53
* flaper87 thought kgriffs had the ability to read his mind. It is now clear that just flaper87 has the ability to read kgriffs mind16:53
kgriffsflaper87: you keep thinking that. Gives me an advantage. ;)16:54
kgriffsflaper87: would a person be able to delete a message from all topics in one request?16:55
kgriffsconversely, can I request messages from multiple topics in one request?16:55
flaper87I think the delete action would look something like:16:56
flaper87>>> client.delete(topics=[a, b, c])16:57
flaper87>>> client.delete(id=1, topics=[a, b, c])16:57
flaper87>>> client.delete(id=2, topics=[a])16:57
flaper87>>> client.delete(id=3, topics=[a, b, c])16:57
flaper87which would delete message 3 from topic b16:57
flaper87the first example would 'empty' topics a, b and c16:57
flaper87the second deletes message 1 from a, b and c16:58
*** davidhadas has joined #openstack-marconi16:58
flaper87the third deletes 2 from a16:58
flaper87and the fourth deletes 3 from b (because it wasn't created in a and c16:58
*** saikrishna_ has quit IRC17:01
kgriffsif the user is thinking in terms of "topics" then perhaps they would be OK with saying a message can be in max N topics17:03
kgriffslike, 517:03
kgriffstoday we only let them be in 117:03
flaper87While I agree with that, I'm thinking that the implementation of this stuff will probably enable them to have N topics automagically17:04
flaper87the bit that is not clear yet is sqla17:05
flaper87but flwang's idea is quite good17:05
flwangflaper87: what's the my idea? ????17:05
kgriffsflaper87: well, I think there must be some upper limit, even if it is like 10017:05
kgriffssince, you don't want to gum up the DB with too many17:05
flwangkgriffs: we need a quota17:05
flwangfor overall Marconi17:06
flaper87kgriffs: could that be up to the operator ?17:06
kgriffsand that can be configurable17:06
kgriffsd00d, stop *doing* that!17:06
flwangkgriffs: yes, it's configurable17:06
flaper87kgriffs: :)17:07
flwangkgriffs: sorry, did you mean we should not do that? :D17:07
*** saikrishna_ has joined #openstack-marconi17:08
kgriffsmuch confusion17:08
kgriffslet's do it17:08
kgriffsit should be configurable like the other limits we have17:08
kgriffsnow, if it is not a fixed number, sqla is a bit tricksy17:08
kgriffsi mean, if that number can change after Marconi is deployed, you would have to do schema changes if you have a column for each topic17:09
kgriffsso, that seems like a bad idea17:09
kgriffs(having a separate column for each)17:09
kgriffsI have seen some people have one or two fixed columns then a text blob17:10
kgriffs(for any extra)17:10
flwangkgriffs: oh, it's bad17:10
kgriffsif the DB supports simple text search within a given column, that could work17:11
kgriffsanyway, we need to have a rough plan for sqla before we decide to do this17:11
*** alcabrera|afk is now known as alcabrera17:12
kgriffsflwang: maybe you can play around with some options?17:12
kgriffsand let us know what you find out?17:12
flaper87flwang: go go go!17:12
flaper87I think a separate table that keeps the map between a message and a topic might work17:12
* flwang will be the labor of flaper87 on this proposal :D17:12
flaper87message_id | topic17:13
kgriffsjoin tables could work17:13
flwangflaper87: that seems like we did in Glance17:13
kgriffsjust need to see how performant that would be17:13
flaper87kgriffs: stop using sqlish terms17:13
*** balajiiyer has joined #openstack-marconi17:13
kgriffsflaper87: just trying to normalize the discussion17:13
flwangnosql nosql nosql17:14
flaper87but that makes my mongoheart sad17:14
flaper87(just kidding)17:14
flaper87ok, anyway17:14
flaper87what's the plan then?17:14
flaper87do we want to discuss this further in the next meeting?17:14
flaper87so we can vote17:14
flaper87and raise some other concerns17:14
flaper87and hear other folks thoughts17:15
flaper87and come up with a better plan17:15
flwangflaper87: kgriffs: I will dig into more17:15
kgriffsfeel free to add this to the agenda17:15
flwangto get something back17:15
kgriffsflwang: thanks!17:15
flaper87flwang: awesome, thanks17:15
*** amitgandhi has joined #openstack-marconi17:15
flwangflaper87: kgriffs: should we raise it in the design session?17:15
*** amitgandhi has quit IRC17:16
flwangafter we get an overall plan?17:16
kgriffsso, is this a proposal for v1.1 or 2.0 ?17:16
flaper87flwang: it depends on whether we have a good plan and if we want it for v1.1 or v217:16
*** amitgandhi has joined #openstack-marconi17:16
flwangkgriffs: I prefer 2.017:16
* kgriffs looks for the bug surgically implanted in his head17:16
kgriffsflwang: so, this is my suggestion17:17
kgriffswe lay the groundwork in 1.1 for this, but we only support 1 topic17:17
flwangsince I'm not sure what's the service we will deliver then :D17:17
kgriffsbasically, we want topics to be "not a separate resource"17:17
kgriffsin 2.0 we can allow multiple topics?17:18
kgriffsthat would be how I would break it down if we don't want to go all the way in 1.117:18
flaper87kgriffs: sounds like a plan17:18
flaper87what about the renaming ?17:18
flaper87looks like v217:18
kgriffsoh yeah, the renaming I guess would wait until 2.017:18
flwangkgriffs: +1, and I want to raise the stupid question again, what's the service we're delivering then?17:18
flwangnot queue ?17:18
flaper87I think the most critical part is changing the backend17:18
kgriffsbut then, people will conflate this with oslo messaging17:18
flaper87once we've got the storage drivers sorted out, it'll be fairly simple to map the API17:19
flaper87well, to be fair, Marconi's always been Messaging as a Service17:19
flwangflaper87: any comments for my question?17:19
flaper87a place where people can post messages17:19
flwangflaper87: hmmmm.17:19
flaper87Using queuing was just a convention17:19
kgriffsWe actually do queue messages, but what we are doing is relaxing the definition of "queuing"17:19
flaper87which made it simple to explain what the service's goal is17:20
flwangflaper87: I don't think all of the community are on the same page with us/you17:20
flaper87I'm pretty sure I'm on a different planet17:20
flaper87don't worry17:20
flaper87jokes apart17:20
kgriffsso, if oslo.messaging didn't exist17:20
flaper87flwang: I agree, but that's not their / our fault. People is used to think about queues when they think about messages17:21
kgriffsthen we could just say Marconi is "OpenStack Messaging Service" or "OpenStack Message Bus"17:21
flwangflaper87: i'm just raising my concern, I mean we should make sure avoid any confusion for the user17:21
flaper87but then when they install the client the first think they ask is: "How do I post a message?"17:21
kgriffsbut people already get Marconi confused with oslo messaging, so I don't know what the best terminology is17:21
flaper87the disambiguation between oslo.messaging and marconi is the 'service' part in Marconi's description17:21
flaper87oslo.messaging is a messaging library17:22
flaper87Marconi is a Messaging service17:22
kgriffswell, maybe we propose to rename the programming during Juno?17:22
flaper87actually, to be fair, oslo.messaging is more an rpc framework / library17:22
flwangkgriffs: maybe17:22
* kgriffs stupid muscle memory17:22
flaper87but yeah, it has messaging in the name17:22
kgriffsflaper87: maybe oslo should be renamed to RPC?17:22
flaper87kgriffs: +1 for renaming the program17:22
kgriffsseems like it used to be or something17:23
flwangkgriffs: +117:23
flaper87kgriffs: but that will confuse people with the old oslo-rpc that still lives in the incubator17:23
* alcabrera listens to the confusion17:23
* kgriffs can't win17:23
alcabreraan unbeatable legacy17:23
* flaper87 confuses alcabrera17:23
flwangMarconi is a highly-available, modular, scalable message bus for the cloud.          Marconi is a new OpenStack project  to create a multi-tenant cloud queuing service. The project will define a  clean, RESTful API, use a modular architecture, and will support both  eventing and job-queuing semantics. Users will be able to customize  Marconi to achieve a wide range of performance, durability,  availability, and efficiency goals.17:23
flaper87so, I'd say, lets wait 'til Juno and we come up with a meter description17:23
* alcabrera decomplects, and feels better17:23
kgriffsso oslo-messaging is really just oslo-rpc redux?17:23
flwangI copy-paste above desc from launchpad17:23
flaper87kgriffs: right17:24
kgriffsflaper87: sounds good. We can also take into account that we want to do notifications under the same program17:24
flwangwe're claiming that Marconi is a queuing service17:24
flaper87flwang: see? '..., ..., ... message bus for the cloud"17:24
flwangflaper87: yep, I saw that17:24
kgriffsCloud Message Bus was my pick for Rackspace's product17:24
flwangflaper87: but we're also saying: Marconi is a new OpenStack project  to create a multi-tenant cloud queuing service.17:24
kgriffsbut the marketing wizards liked Cloud Queues better17:24
* kgriffs sad panda17:25
kgriffsflwang: yeah, we need to update the wiki17:25
flaper87flwang: we can drop that line17:25
kgriffsI've been meaning to revamp the wiki for a while now17:25
flwangkgriffs: so maybe we should named 'Message queuing as a Service'?17:25
flaper87anyway, Messaging as a service covers notification as well17:25
kgriffsflaper87: +117:25
flwanghighlight the Message, but keeping the 'queuing'17:26
flaper87people don't like to queue up17:26
flaper87I hate being queued up17:26
flaper87it's like a waste of time while you watch other people being prioritized17:26
kgriffsmission updated17:26
flaper87and that's a piece of my venezuelan roots17:27
flaper87flwang: tell me about it17:28
kgriffsflwang: you are with IBM, right?17:28
* alcabrera watched marconi become a message bus17:29
alcabreraand it was good17:29
flwangkgriffs: yep17:29
flwangflaper87: now it's 1:30AM17:29
kgriffsflwang: d00d, I'm sorry17:30
kgriffsthat you are up so late17:30
flwangkgriffs: no worries :) I'm just blaming the timezone :D17:30
flaper87flwang: don't you dare to go to sleep17:30
flaper87we've a queuing service to fix here17:30
flwangkgriffs: it's really good to discuss with you guys :)17:31
flwangflaper87: yep, DARK :)17:31
* flaper87 next mission. Convert flwang to something different than human17:31
*** tedross has joined #openstack-marconi17:31
balajiiyerkgriffs: Im back.17:31
flwangflaper87: drop me in a queue17:32
flwangflaper87: oh, no queue now17:32
flwangflaper87: send me as a msg17:32
flaper87flwang: we'll tag you and send you to some random contry on a pigeon17:32
flwangflaper87: pls make sure the time difference is no more than 6 hours :D17:33
alcabrerathe topic is 'marconi-fied' - flwang will travel well17:33
flwangalcabrera: haha, pls :D17:33
flwangguys, I really need some sleep now17:34
kgriffsremoved some references to queues on the wiki and added flwang to the core team17:34
alcabreraflwang: go rest. :)17:34
alcabreravery RESTful rest17:34
flwangkgriffs: alcabrera: flaper87: have a nice day!17:34
kgriffsnote that I also mentioned requirements for getting on the core team17:34
kgriffsflwang: thanks man, catch ya later!17:34
alcabreramentorship and good judgement - good qualities, kgriffs17:35
flwangit means bye :D17:35
alcabreraTIL: 再见17:36
*** nidhi has joined #openstack-marconi17:37
kgriffsflaper87: just thought of something17:40
kgriffslisting topics may still be useful17:40
flaper87kgriffs: mmh, agreed17:41
kgriffsif so, then we will have to do some kind of grouping query if topics don't exist as n outside thing17:41
kgriffsbut, maybe listing topics isn't useful17:41
flaper87well, more than listing, I'd say counting17:42
kgriffsthat is something an app could always keep track of in their DB as ell17:42
kgriffsso for v1.1 maybe we keep "queue" but we lazy create and we don't let them have metadata17:44
flaper87sounds good17:44
kgriffsand if everyone keeps using 1.0 because they want the metadata feature, we can add it back into 2.017:44
flaper87and then we do the major API refactor for the API v117:44
flaper87erm v217:44
balajiiyerkgriffs: do you have moment to talk about pecan eval?17:56
*** mwagner_lap has quit IRC17:56
kgriffsone moment17:57
*** saikrishna_ has quit IRC17:57
kgriffsbalajiiyer: actually give me 5 minutes. I need to go run and grab something17:58
balajiiyerkgriffs: np17:59
kgriffsone sec18:02
kgriffsgive me another 2 mins18:02
kgriffsok, ready18:07
kgriffsbalajiiyer: ping18:07
balajiiyerkgriffs: pong18:07
kgriffsso, a few things18:07
kgriffswe need to come up with a basic outline18:08
kgriffsit would be cool to come up with a web page that lets people plug in their own weights for the criteria, but idk how much effort that would be18:09
kgriffsbasically just a simply spreadsheet18:09
*** oz_akan_ has quit IRC18:09
kgriffsbut let me back up18:09
kgriffsso, one of the most useful things I learned when I was in school was this notion of a "decision matrix"18:10
kgriffsyou list out the criteria you care about18:10
kgriffsgive each a weight based on how important each one is to you18:10
kgriffsand then you score each option18:10
kgriffsthat helps you make an objective decision18:11
kgriffsbalajiiyer: make sense?18:11
balajiiyerkgriffs: yup. and Agreed. something like this
kgriffsexcept you also need weights18:12
kgriffssince each criterion AKA "objective" isn't equal in importance to the others18:12
balajiiyerright, e.g git hub stars will have less weight than performance18:12
balajiiyerfar less weight, infact. :)18:13
kgriffsbalajiiyer: how quickly could we make a dynamic decision matrix that would allow people to put in their own weights?18:14
kgriffsI wish there were a little service for making these18:14
balajiiyerkgriffs: lookng at couple of online tools18:15
balajiiyerkgriffs: nvm, that is not the tool we are looking for. They sell predefined templates18:17
kgriffsbalajiiyer: I bet someone could whip up a quick one with angular or knockout or something18:18
kgriffsjust have a list of text fields people can type weights into18:18
balajiiyerkgriffs: yeah, I will write it and host it. Have a couple of meetings this afternoon, could have it ready by tonight18:19
kgriffsnext topic18:20
kgriffscan you bring up that etherpad?18:20
*** tedross has quit IRC18:20
kgriffsbalajiiyer: so, last think to decide is which criteria to include18:25
kgriffsbalajiiyer: one last thing18:37
balajiiyerkgriffs: yup18:37
kgriffsI think it would be good to briefly comment on each weight18:37
kgriffswhy this weight was chosen from Marconi's perspective18:38
balajiiyerkgriffs: sure thing, yes.18:38
kgriffsso, I thinnk we have a good outline figured out18:38
kgriffsanything else we need to cover?18:39
balajiiyerI will also have to find a balance between this email not being too long and to the point. This outlines helps a lot.18:39
balajiiyerkgriffs: so, weighted decision matrix and I will have a draft by tomorrow morning.18:39
kgriffsbalajiiyer: ok. Let's get some people outside our team to review the draft ASAP and I'd like to send this out tomorrow before lunch if we can18:40
kgriffsbalajiiyer: thanks!18:41
balajiiyerkgriffs: thank YOU18:41
kgriffsflaper87: ^^^18:41
kgriffsbalajiiyer: I was just looking over your benchmarks18:48
kgriffslooks like you used apache bench?18:49
balajiiyerI used tsung and AB18:50
*** vkmc has quit IRC18:50
balajiiyerkgriffs: do you have any other recommendation?18:50
kgriffsoh, let me look for the tsung numbers18:51
balajiiyerkgriffs: hang on. looks like that file has only AB18:51
kgriffswhile I wait to see that, a couple thoughts18:52
kgriffsfirst, when benchmarking system noise must be accounted for18:53
kgriffsone way to do that is to run the same benchmark multiple times, then take the BEST time18:53
kgriffsyou don't want the everage18:53
*** nidhi has quit IRC18:53
kgriffsthe reason things are slower in one run, faster in another is because something in the system went off and slowed down your process18:54
balajiiyerkgriffs: got it.18:55
mpanettaYay!  I am signed up for the openstack summit.18:56
mpanettaBTW if you got your code you have to use it by the 21st to get in free...18:56
kgriffsI actually prefer autobench (and httperf) myself18:56
alcabrerampanetta: hurray!18:56
mpanettaJust in case you don't know...18:56
kgriffsit is more accurate, and let's you hit the target server from multiple boxes18:56
kgriffsI'm trying to remember if httperf only reports mean numbers18:57
*** reed has joined #openstack-marconi18:57
kgriffsbalajiiyer: so, I would recommend re-running with autobench even if you only do it against localhost18:58
flaper87kgriffs: balajiiyer what's the link to the bench / eval?18:58
balajiiyerkgriffs: ok.18:59
kgriffsbalajiiyer: if you look at this blob post, it has some suggestions for autobench params19:00
kgriffsbalajiiyer: also, did you just run marcon-server?19:01
kgriffsand did you use the mongodb driver?19:01
balajiiyerkgriffs: yeah, marconi-server with mongodb19:03
kgriffsah, ok19:03
kgriffsso, I think it would be better to run with uwsgi and enable gevent19:03
kgriffsthat will be more realistic19:03
kgriffsbasically, it would be great to create a graph like one shown in this post:
flaper87can pecan run under gevent ?19:03
*** vkmc has joined #openstack-marconi19:04
*** vkmc has quit IRC19:04
*** vkmc has joined #openstack-marconi19:04
kgriffsbalajiiyer: also, it would be good if mongo was on a separate baox19:04
kgriffsbalajiiyer: and might as well put the autobench client on it's own box as well; that way it isn't fighting for CPU19:05
*** oz_akan_ has joined #openstack-marconi19:05
*** oz_akan_ has quit IRC19:05
*** oz_akan_ has joined #openstack-marconi19:06
kgriffsbalajiiyer: I apologize for not bringing all this up earlier19:06
kgriffsthe reason this is important is that minor changes in latency can have a big impact when you have lots of concurrent requests, and I want to make sure that is captured19:07
* mpanetta smacks kgriffs around with a piece of python code19:07
kgriffsmarconi-server is single-threaded, and is also pretty slow by itself, so it can mask other performance issues19:08
balajiiyerkgriffs: my thinking was all of these have been done before, 1000+ iterations, different drivers, differetn setup.. my idea was not to do an exhaustive benchamarking, but to gather enough metrics for the eval.19:08
kgriffsbalajiiyer: so, that graph there19:08
kgriffswhat I mean is19:08
kgriffsit should just have two lines19:08
kgriffsone for uwsgi+falcon+mongo19:09
kgriffsand one for uwsgi+pecan+mongo19:09
* alcabrera garbage collects mpanetta's python code19:09
kgriffsthe reason I think the graph is important is it makes it obvious what happens when you crank up concurrency19:10
kgriffsbalajiiyer: make sense? I mean, I don't expect you to go and bench all possible combinations19:10
mpanettaSo we don't have any sort of in memory only DB?19:10
kgriffsmpanetta: depends on how you define that19:10
mpanettaSomething that won't hit disk19:11
kgriffsso, never hit the disk, we only have sqlite memory db19:11
kgriffsand a POC for redis19:11
kgriffsmongodb memory-maps it's files, so given enough RAM, it's almost as good19:12
mpanettaHmm ok19:12
kgriffsassuming you don't require disk flushing for every insert19:12
mpanettaGo go gadget english :P19:12
balajiiyerkgriffs: have to run for a meeting now… will be back in a few19:13
kgriffsbalajiiyer: kk19:13
*** balajiiyer is now known as balajiiyer_afk19:13
kgriffsmpanetta: can you start looking into how to use autobench?19:13
mpanettaI have the same meeting :P19:13
kgriffsI think balajiiyer will need a partner in crime19:13
kgriffscurses! foiled again!19:13
*** Kuwagata has joined #openstack-marconi19:34
openstackgerritCindy Pallares proposed a change to openstack/marconi: Add queue and queue test to API
*** malini is now known as malini_afk19:37
kgriffsflaper87: when were you arriving in Atlanta?19:48
* flaper87 back19:50
flaper87kgriffs: 10th ~17:3019:50
flaper87kgriffs: you?19:50
kgriffsI am making my travel arrangements as we speak19:51
flaper87I wanted to organize a Marconi team dinner19:51
kgriffsI was thinking about coming in on the 10th as well19:51
flaper87or drink up19:51
flaper87or whatever that makes us forget our queued problems19:51
kgriffsflaper87: flying home on the 17th?19:51
flaper8716th late at night19:52
kgriffsooh boy19:52
flaper87yeah, I know19:52
*** oz_akan_ has quit IRC19:52
*** oz_akan_ has joined #openstack-marconi19:55
*** oz_akan__ has joined #openstack-marconi19:56
*** oz_akan_ has quit IRC19:59
*** oz_akan__ has quit IRC20:09
kgriffsmpanetta: done w/ meeting?20:09
flaper87kgriffs: I think he's hiding from you20:14
flaper87just sayin'20:14
kgriffsI'm not *that* scary20:14
kgriffswell, I didn't shave this morning20:15
*** balajiiyer_afk has quit IRC20:16
*** malini_afk is now known as malini20:25
alcabreraNight, all. :)20:25
cpallaresbye alcabrera20:26
*** alcabrera has quit IRC20:27
kgriffsjraim: FYI, there's been a delay in the pecan eval going out... we are trying to get it done now for tomorrow. Balaji has been swamped with some other things.20:28
*** ayoung has quit IRC20:32
openstackgerritKurt Griffiths proposed a change to openstack/marconi: fix(cli): Print statement in global error handler
jraimkgriffs: no worries20:34
jraimkgriffs: as an FYI, we got our votes - we have enough to be incubated :)20:34
openstackgerritKurt Griffiths proposed a change to openstack/marconi: Add missing logging trace queues collection
*** oz_akan_ has joined #openstack-marconi20:40
*** sriram has quit IRC20:40
*** oz_akan__ has joined #openstack-marconi20:43
*** oz_akan_ has quit IRC20:44
openstackgerritJohn Heatherington proposed a change to openstack/python-marconiclient: Refactoring iterators
*** oz_akan__ has quit IRC21:07
jraimkgriffs: ttx just merged our patch - we're officially incubated :)21:19
kgriffsjraim: rock on21:21
cpallaresyay *\o/*21:21
kgriffsjraim: Barbican is going to be super useful in so many ways21:21
jraimkgriffs: hopefully - now we just have to not screw it up :)21:22
*** ayoung has joined #openstack-marconi21:42
*** mpanetta has quit IRC21:53
*** kgriffs is now known as kgriffs_afk21:53
*** kgriffs_afk is now known as kgriffs21:54
maliniI am running into a weird out of space issue at the gate now22:00
*** Kuwagata has quit IRC22:01
*** amitgandhi has quit IRC22:15
*** jmckind has quit IRC22:37
*** balajiiyer has joined #openstack-marconi22:40
balajiiyerkgriffs: ping22:41
kgriffsbalajiiyer: pong22:41
*** mwagner_lap has joined #openstack-marconi22:43
balajiiyerkgriffs: so, had to go pick up my kid immediately after the meeting. Had to leave in the middle of the conversation. Sorry about that.22:43
kgriffsno worries22:43
balajiiyerIm going to spend sometime tonight working with autobench22:43
kgriffsbalajiiyer: I asked mike to work on setting up the boxes22:44
kgriffs3 perf servers22:44
kgriffs1 for mongo, 1 for web head with uwsgi, one for autobench22:44
kgriffsthat way autobench isn't fighting for CPU22:44
kgriffsand we can really test concurrency22:44
balajiiyerkgriffs: one of the reasons why I stuck with a single box was I had network throughput issues from a cloud server22:44
kgriffsbalajiiyer: perf servers should be less prone to that?22:45
kgriffsbalajiiyer: hopefully we will be ok with some brand new perf boxes22:46
balajiiyerkgriffs: I hope so. Last Friday I was trying with 500 concurrent reqs for 10 secs, and only 20% of them succeeded22:46
kgriffsbalajiiyer: some of that may be due to apache bench22:46
kgriffssome may be due to kernel params22:46
balajiiyerkgriffs: I see. ok22:46
kgriffswe can tune the kernel params (i have a gist)22:46
kgriffsbalajiiyer: I mean, it may not have been network - could very well have been just using up local sockets22:47
kgriffsbalajiiyer: sorry again for springing this on you last-minute. I should have brought up autobench earlier22:47
balajiiyerkgriffs: no worries. I should have talked about it more about it in this chanvel, instead of working on it in a silo22:48
kgriffsbalajiiyer: if you have a few minutes to play with autobench to get a feel for how it works, then we can hit the ground running in the morning22:48
kgriffsone gotchya that I remember is the way params are passed through to httperf22:49
kgriffsseems like httperf params are named differently or something when you specify them to autohost, but I can't remember for sure22:49
kgriffswe'll have to figure that out so we can pass the --hog option22:49
kgriffsbalajiiyer: one of these days I've been meaning to write an HTTP API benchmarking tool in node22:50
kgriffsyou know, just for the heck of it22:50
kgriffsI think I could make something that is easier to configure and run than [tsung, autobench, jmeter], esp. if I make the simplifying assumption that this is for APIs, not web pages.22:51
balajiiyerkgriffs: +1 and thats what is unique about you. I go about complaining things, rather than fixing it. :)22:52
kgriffsmy grandfather was a carpenter. If he didn't have a tool he needed, he wasn't afraid to build it.22:52
kgriffsbalajiiyer: lol22:52
kgriffsanyway, I gotta run. You should be able to glean some autobench info from my blog22:52
balajiiyerkgriffs: malini has been talking about 'Tsung as a Service' for a while now22:52
kgriffsI wish I had recorded the exact autobench config file and such, but I neglected to.22:53
kgriffsbalajiiyer: there's something like that, iirc, but WAY TOO EXPENSIVE22:53
kgriffsanyway, ttfn22:53
kgriffshave a good night!22:53
balajiiyerkgriffs: take care22:53
kgriffsbalajiiyer: p.s. - got my flight to Atlanta. I'll be there May 10th (sat).22:54
*** balajiiyer has left #openstack-marconi22:54
kgriffsIf anyone wants to go for food and/or drinks, Flavio's paying22:54
kgriffs(he said that, didn't he?)22:54
kgriffsbye awesome Marconi peeps!22:55
*** kgriffs is now known as kgriffs_afk23:04
*** flaper87 is now known as flaper87|afk23:22
*** vkmc has quit IRC23:43
*** jcru has quit IRC23:54

Generated by 2.14.0 by Marius Gedminas - find it at!