| *** whenry has joined #openstack-marconi | 00:07 | |
| *** openstack has joined #openstack-marconi | 00:20 | |
| *** flaper87 is now known as flaper87|afk | 00:26 | |
| *** whenry has quit IRC | 00:37 | |
| *** mwagner_lap has joined #openstack-marconi | 01:23 | |
| *** rwsu has quit IRC | 01:26 | |
| *** nosnos has joined #openstack-marconi | 01:29 | |
| *** rwsu has joined #openstack-marconi | 01:42 | |
| *** malini_afk is now known as malini | 01:54 | |
| *** haomaiwang has joined #openstack-marconi | 02:31 | |
| *** rossk has quit IRC | 02:33 | |
| *** malini is now known as malini_afk | 03:00 | |
| openstackgerrit | A change was merged to openstack/marconi: Fix functional config file instructions https://review.openstack.org/79925 | 03:27 |
|---|---|---|
| *** haomaiw__ has joined #openstack-marconi | 03:28 | |
| *** nosnos has quit IRC | 03:29 | |
| *** haomaiwang has quit IRC | 03:32 | |
| *** chandankumar_ has joined #openstack-marconi | 03:33 | |
| *** chandankumar_ has quit IRC | 03:40 | |
| *** chandankumar_ has joined #openstack-marconi | 03:55 | |
| *** chandankumar_ has quit IRC | 04:23 | |
| *** nosnos has joined #openstack-marconi | 04:25 | |
| *** chandan_kumar has quit IRC | 05:48 | |
| *** prashanthr_ has quit IRC | 06:58 | |
| *** saju_m has joined #openstack-marconi | 07:30 | |
| *** prashanthr_ has joined #openstack-marconi | 07:47 | |
| prashanthr_ | flwang: Hi :) | 07:48 |
| *** flaper87|afk is now known as flaper87 | 07:51 | |
| *** saju_m has quit IRC | 08:50 | |
| *** saju_m has joined #openstack-marconi | 08:52 | |
| *** chandan_kumar has joined #openstack-marconi | 09:13 | |
| *** prashanthr_1 has joined #openstack-marconi | 09:20 | |
| *** mkoderer has joined #openstack-marconi | 09:21 | |
| *** ykaplan has joined #openstack-marconi | 09:25 | |
| *** saju_m has quit IRC | 09:38 | |
| *** nosnos has quit IRC | 09:39 | |
| *** saju_m has joined #openstack-marconi | 09:56 | |
| *** ykaplan has quit IRC | 10:42 | |
| *** saju_m has quit IRC | 11:07 | |
| *** saju_m has joined #openstack-marconi | 11:08 | |
| *** saju_m has quit IRC | 11:09 | |
| *** mwagner_lap has quit IRC | 11:12 | |
| *** saju_m has joined #openstack-marconi | 11:15 | |
| *** malini_afk is now known as malini | 11:17 | |
| *** ykaplan has joined #openstack-marconi | 11:36 | |
| *** saju_m has quit IRC | 11:38 | |
| *** saju_m has joined #openstack-marconi | 11:38 | |
| *** tedross has joined #openstack-marconi | 11:58 | |
| *** malini is now known as malini_afk | 12:09 | |
| *** oz_akan_ has joined #openstack-marconi | 12:21 | |
| flwang | flaper87: ping | 12:29 |
| *** sriram has joined #openstack-marconi | 12:34 | |
| *** sriram has quit IRC | 12:34 | |
| *** sriram has joined #openstack-marconi | 12:34 | |
| flaper87 | flwang: pong | 12:35 |
| *** saju_m has quit IRC | 12:41 | |
| *** oz_akan_ has quit IRC | 12:44 | |
| *** oz_akan_ has joined #openstack-marconi | 12:44 | |
| flwang | flaper87: i'm testing the client | 12:45 |
| flaper87 | flwang: awesome! Thanks a lot for that! | 12:46 |
| *** mwagner_lap has joined #openstack-marconi | 12:46 | |
| flwang | flaper87: i'm wondering if we should add a sample against the keystone auth strategy mode | 12:46 |
| flaper87 | flwang: sure, that would be amazing | 12:47 |
| flwang | flaper87: seems the client also supports a private conf file, is it? | 12:47 |
| flaper87 | flwang: mmh, what do you mean? | 12:48 |
| flaper87 | the client doesn't have support for config files | 12:48 |
| *** seiflotfy_ has joined #openstack-marconi | 12:51 | |
| flwang | flaper87: ok, i see. | 12:51 |
| *** seiflotfy has quit IRC | 12:52 | |
| *** saju_m has joined #openstack-marconi | 12:55 | |
| flwang | flaper87: so for keystone mode, I can just pass the param like auth_opts={'os_auth_token': 'xxxx'}, right? | 12:56 |
| flwang | flaper87: ah, no | 12:57 |
| *** saju_m has quit IRC | 12:59 | |
| flwang | flaper87: would you mind briefing how the client side to talk with marconi server with the keystone mode? | 13:00 |
| *** saju_m has joined #openstack-marconi | 13:00 | |
| *** saju_m has quit IRC | 13:04 | |
| *** saju_m has joined #openstack-marconi | 13:05 | |
| flaper87 | flwang: you're correct, to talk to keystone you need to pass the auth_strategy and the rest of the options in auth_opts | 13:08 |
| *** saju_m has quit IRC | 13:09 | |
| *** saju_m has joined #openstack-marconi | 13:10 | |
| flwang | flaper87: do you have a sample? | 13:10 |
| flwang | to construct the params | 13:10 |
| flaper87 | flwang: erm, mmh, yeah, sure | 13:10 |
| * flaper87 runs away | 13:10 | |
| * flaper87 hides from flwang | 13:10 | |
| * flaper87 is scared because he lied to flwang | 13:11 | |
| * flaper87 doesn't really have a sample | 13:11 | |
| flaper87 | flwang: https://github.com/openstack/python-marconiclient/blob/master/marconiclient/transport/request.py#L45 | 13:11 |
| flwang | flaper87: haha | 13:11 |
| flaper87 | flwang: we should have more tests about that | 13:12 |
| flaper87 | flwang: this is all we have so far: https://github.com/openstack/python-marconiclient/tree/master/tests/auth | 13:12 |
| flwang | flaper87: i have reviewed both of them actually :D | 13:13 |
| flaper87 | flwang: :( then my hands are empty :( | 13:13 |
| flaper87 | flwang: you could write some samples and tests ;) | 13:14 |
| flaper87 | flwang: https://review.openstack.org/#/c/82820/ | 13:14 |
| flwang | flaper87: haha, ok, i see. | 13:14 |
| flaper87 | flwang: I'm really sorry about that :/ | 13:14 |
| flwang | flaper87: then we can backport them to client /sample | 13:14 |
| flaper87 | flwang: yup | 13:15 |
| flwang | flaper87: good, let's rock on | 13:15 |
| *** mpanetta has joined #openstack-marconi | 13:19 | |
| *** ChanServ changes topic to "the gerrit event stream is currently hung, blocking all testing. troubleshooting is in progress (next update at 14:00 utc)" | 13:21 | |
| *** amitgandhi has joined #openstack-marconi | 13:23 | |
| openstackgerrit | Flavio Percoco proposed a change to openstack/python-marconiclient: Move auth tests under the unit package https://review.openstack.org/82820 | 13:26 |
| *** jcru has joined #openstack-marconi | 13:26 | |
| *** ChanServ changes topic to "OpenStack Queuing and Notification Service || Smile :D || Meetings every Tuesday @ 15:00 UTC || Wiki: https://wiki.openstack.org/wiki/Marconi || Paste: http://paste.openstack.org/ || Send messages and make some noise :D" | 13:29 | |
| *** ykaplan has quit IRC | 13:37 | |
| *** alcabrera has joined #openstack-marconi | 13:54 | |
| *** balajiiyer has joined #openstack-marconi | 13:54 | |
| alcabrera | Good morning! :D | 13:54 |
| *** ykaplan has joined #openstack-marconi | 13:55 | |
| flaper87 | alcabrera: GOOOOOOOOOOOOOOOOOOOOOOD morning | 13:55 |
| alcabrera | flaper87: hey! How goes? :) | 13:56 |
| flaper87 | alcabrera: good... You ? | 13:56 |
| flaper87 | more writing less coding :/ | 13:57 |
| flaper87 | btw, I forgot I have an internal presentation today and I won't be able to attend the meeting :( | 13:57 |
| alcabrera | I'm doing alright this morning. It may also be a day of more writing, less coding for me. :P | 13:57 |
| alcabrera | Oh, alright. | 13:57 |
| alcabrera | I'll be attending, so feel free to ping me after wards, or... | 13:58 |
| alcabrera | is there any topic you'd like to see come up in today's meeting? | 13:58 |
| flaper87 | I should learn that calendars are not just to create events but also to check when you have events | 13:58 |
| alcabrera | heh - write-only mediums? ;) | 13:58 |
| flaper87 | yeah! | 13:58 |
| flaper87 | fun part is that I know my memory sucks | 13:58 |
| flaper87 | There's a part of me that refuses to admit it and use calendars | 13:59 |
| flaper87 | :D | 13:59 |
| flaper87 | anyway, I think we were planning to discuss a lot about what happened last week | 13:59 |
| flaper87 | (again) | 13:59 |
| flaper87 | and prepare a small plan forward | 13:59 |
| flwang | flaper87: seems we need an 'auto-discovery' for https://github.com/openstack/python-marconiclient/blob/master/marconiclient/transport/request.py#L45 | 13:59 |
| flaper87 | I started writing Marocni's FAQ today | 13:59 |
| flaper87 | alcabrera: flwang kgriffs_afk malini_afk https://etherpad.openstack.org/p/draft-marconi-faq | 13:59 |
| flaper87 | Answering some of the questions that came up and adding a couple more that I've heard | 13:59 |
| flaper87 | I'm planning to give a very quick walkthrough that FAQ on Thursday's hangout | 14:00 |
| alcabrera | flaper87: thanks for kickstarting that! | 14:00 |
| flwang | flaper87: to read the correct url/api from keystone service catalog | 14:00 |
| flwang | flaper87: it's a good start | 14:01 |
| flwang | flaper87: the faq | 14:01 |
| alcabrera | re: calendaring | 14:02 |
| alcabrera | that reminds me that I need to add a thing or two to my own | 14:02 |
| flaper87 | alcabrera: YOU'RE WELCOME! >.> | 14:03 |
| flaper87 | :D | 14:03 |
| flaper87 | (that was re calendaring) | 14:03 |
| *** alcabrera is now known as alcabrera|afk | 14:03 | |
| *** ykaplan has quit IRC | 14:06 | |
| *** malini_afk is now known as malini | 14:06 | |
| *** AnastasiosAndron has joined #openstack-marconi | 14:12 | |
| *** whenry has joined #openstack-marconi | 14:18 | |
| *** ykaplan has joined #openstack-marconi | 14:19 | |
| *** alcabrera|afk is now known as alcabrera | 14:21 | |
| *** oz_akan_ has quit IRC | 14:24 | |
| *** oz_akan_ has joined #openstack-marconi | 14:26 | |
| *** Kuwagata has joined #openstack-marconi | 14:29 | |
| *** cpallares has joined #openstack-marconi | 14:33 | |
| *** jmckind has joined #openstack-marconi | 14:35 | |
| openstackgerrit | Sriram Madapusi Vasudevan proposed a change to openstack/marconi: fix() : introduce division import https://review.openstack.org/82838 | 14:45 |
| sriram | alcabrera,flaper87,flwang,malini: Could you guys look at the above patch? Thanks! | 14:47 |
| alcabrera | sriram: nice catch. | 14:47 |
| * alcabrera had already looked at it | 14:47 | |
| alcabrera | ;) | 14:47 |
| sriram | awesome! :) | 14:47 |
| flwang | sriram: will +2 after Mr. Jenkins says it's ok | 14:48 |
| alcabrera | I'll +2 after I finish writing an email. I hope jenkins beats me to the punch. :P | 14:48 |
| malini | I hope I can beat alcabrera ;) | 14:48 |
| sriram | :) | 14:48 |
| alcabrera | malini: oh yeah! +2 powers. :D | 14:48 |
| malini | just kidding | 14:48 |
| alcabrera | Let's all join together and give sriram +10 | 14:48 |
| alcabrera | (which amuses me, because +10 in binary -> +2) | 14:49 |
| * sriram binary puns | 14:49 | |
| alcabrera | 10**10 -> 100 -- only in binary | 14:50 |
| flwang | yep, in the developer world, there are 10 kinds of people, one know what's binary, one doesn't :) | 14:50 |
| sriram | That's the developer 101 | 14:50 |
| alcabrera | heh | 14:50 |
| alcabrera | sounds like a super hero group -- the developer *five* | 14:51 |
| sriram | heh. | 14:51 |
| *** reed has joined #openstack-marconi | 14:51 | |
| *** oz_akan_ has quit IRC | 14:57 | |
| *** oz_akan_ has joined #openstack-marconi | 14:58 | |
| alcabrera | marconi team meeting in 2 minutes | 14:58 |
| *** kgriffs_afk is now known as kgriffs | 14:59 | |
| alcabrera | kgriffs: o/ | 15:00 |
| flwang | meeting... | 15:00 |
| megan_w | o/ | 15:01 |
| malini | megan_w: #openstack-meeting-alt | 15:01 |
| kgriffs | flaper87: meeting? | 15:01 |
| alcabrera | kgriffs: flaper87 is out, for now. Internal presentation. | 15:02 |
| megan_w | malini: thanks :) | 15:02 |
| *** saju_m has quit IRC | 15:13 | |
| mpanetta | Wee binary | 15:18 |
| *** whenry has quit IRC | 15:20 | |
| *** vkmc has joined #openstack-marconi | 15:26 | |
| *** vkmc has quit IRC | 15:26 | |
| *** vkmc has joined #openstack-marconi | 15:26 | |
| *** ayoung has quit IRC | 15:37 | |
| *** ayoung has joined #openstack-marconi | 15:48 | |
| *** jergerber has joined #openstack-marconi | 15:54 | |
| kgriffs | megan_w: I guess we have to register for the marketing mtg? | 16:02 |
| kgriffs | http://openstack.enterthemeeting.com/m/JM4CKMMV | 16:02 |
| megan_w | kgriffs: i'll take care of it | 16:02 |
| kgriffs | thanks! | 16:03 |
| megan_w | oh wow, its on a phone bridge | 16:03 |
| kgriffs | yeah. I didn't realize. Expected it to be IRC. | 16:04 |
| kgriffs | megan_w: I think Niki Acosta @ Rackspace may be on that team | 16:04 |
| megan_w | oh good, i can talk to her beforehand | 16:04 |
| kgriffs | maybe you could reach out to here and see what we should do | 16:04 |
| megan_w | by the way, my registration is only good for one participant, apparentlhy | 16:05 |
| megan_w | so if you're going to join, you should register too | 16:05 |
| kgriffs | the graduation requirement is: "Project should have engaged with marketing team to check suitable official name" | 16:05 |
| *** alcabrera is now known as alcabrera|afk | 16:05 | |
| kgriffs | megan_w: yep, I registered | 16:05 |
| kgriffs | OK, wrt g+ hangouts... | 16:06 |
| *** alcabrera|afk is now known as alcabrera | 16:07 | |
| kgriffs | I think the things that most useful... | 16:07 |
| kgriffs | 1. voice | 16:07 |
| kgriffs | 2. show slides or virtual whiteboard | 16:07 |
| kgriffs | 3. know who is talking, when | 16:07 |
| kgriffs | 4. have minutes/notes | 16:08 |
| alcabrera | because we sacrifice logs for (4) w/ g+, we'll need a minutes-taker | 16:08 |
| kgriffs | g+ works well for smaller teams, but as we grow, I don't want to leave people out | 16:08 |
| kgriffs | alcabrera: yep! | 16:09 |
| alcabrera | I'd be happy to help with that. | 16:09 |
| kgriffs | rock on | 16:09 |
| flaper87 | alcabrera: malini kgriffs re hangout. It will be a proper presentation about Marconi | 16:10 |
| flaper87 | I'll just take that change to also go through the FAQ | 16:10 |
| flaper87 | at least the most common question | 16:10 |
| flaper87 | (not that it hasn't been done before) | 16:10 |
| alcabrera | flaper87: ¡gracias! | 16:11 |
| kgriffs | flaper87: ah, ok | 16:11 |
| kgriffs | flaper87: so, we were talking about doing a monthly checkpoint meeting with the team | 16:11 |
| kgriffs | we talk about how our community and development process is working, what we should continue and what we need to do better on | 16:12 |
| flaper87 | kgriffs: sounds A.M.A.Z.I.N.G | 16:12 |
| kgriffs | and in part 2 we track where we are wrt graduation | 16:12 |
| flaper87 | like really, we should do this monthly, invite people | 16:12 |
| flaper87 | and even if it's just us, we'll talk about stuff that have happened | 16:13 |
| flaper87 | and that will happen | 16:13 |
| kgriffs | flaper87: so, the question is, do we try to do this with video conferencing or stick to IRC | 16:13 |
| kgriffs | (video conferencing ala Hangouts, GoToMeetings, etc.) | 16:13 |
| kgriffs | malini: ^^^ | 16:14 |
| flaper87 | video | 16:14 |
| alcabrera | video | 16:14 |
| flaper87 | we need a different way for people to reach out to us | 16:14 |
| flaper87 | that's a good way community-wise | 16:14 |
| kgriffs | ok. So, with G+ we are limited to 10 participants methinks | 16:14 |
| flaper87 | PLUS, that will be recorded and people (or even us) can go back to it | 16:14 |
| flaper87 | are we? | 16:15 |
| flaper87 | mmh, didn't know that | 16:15 |
| kgriffs | flaper87: I think so. There may be a way for voice-only via telephone for more than 10 - there used to be, but last time I went looking I couldn't find that option. | 16:15 |
| alcabrera | "Hangouts are limited to 10 people at a time with video and audio and additional 10 users by phone." | 16:17 |
| alcabrera | https://productforums.google.com/forum/#!topic/google-plus-discuss/dP7Wg_IiQEs | 16:17 |
| alcabrera | kgriffs, flaper87: ^^ | 16:17 |
| flaper87 | eeeeeeeeeeeeeeeenteresting | 16:18 |
| flaper87 | I wonder how the Marconi hangout was setup :/ | 16:18 |
| oz_akan_ | kgriffs: flaper87 malini what is the vision statement for Marconi? | 16:18 |
| kgriffs | Commercial solutions allow higher numbers of users, but obviously aren't free | 16:19 |
| kgriffs | oz_akan_: I've tried to write that down in the first couple sections on our wiki, https://wiki.openstack.org/wiki/Marconi | 16:20 |
| kgriffs | but, I think we've some work to do to make our key value proposition / mission crystal-clear | 16:21 |
| kgriffs | megan_w: does Rackspace have any conferencing accounts we can leverage for Marconi meetings? | 16:22 |
| flaper87 | kgriffs: I'd say community-understandable :D | 16:22 |
| oz_akan_ | mission defines how to achieve a vision, I guess that first sentence under mission is vision | 16:22 |
| kgriffs | yeah | 16:22 |
| megan_w | kgriffs: i have a phone bridge | 16:22 |
| megan_w | we also have webex, which *might* have video | 16:23 |
| kgriffs | hmmm | 16:23 |
| oz_akan_ | and "variety of distributed application messaging patterns in an efficient, scalable and highly-available manner" has a very big scope | 16:23 |
| kgriffs | I've had bad experiences with webex in the past. :( | 16:23 |
| megan_w | ditto | 16:23 |
| kgriffs | has anyone tried GoToMeeting or another alternative? | 16:23 |
| * flaper87 has had really bad experiences with webex | 16:23 | |
| oz_akan_ | kgriffs: should we try to define the scope better? with better I mean narrowing it | 16:24 |
| kgriffs | oz_akan_: agreed. | 16:24 |
| mpanetta | webex is horrible in my experience. And it leaks like a sieve, not that it would matter for an OS project. | 16:25 |
| oz_akan_ | does web-friendly mean REST? | 16:25 |
| kgriffs | The mission statement is hard because it is supposed to be very brief. This was actually copied from the governance incubation YAML | 16:25 |
| kgriffs | (although I've now modified it slightly) | 16:26 |
| kgriffs | maybe rename mission -> vision and summary -> mission ? | 16:26 |
| kgriffs | oz_akan_: more or less. I guess we could just say that | 16:27 |
| oz_akan_ | vision is something like this: to provide web friendly api then mission statement is provide a web friendly api by using RESTful interface etc.. | 16:27 |
| oz_akan_ | so yes, mission statement describes how to achieve that desired vision | 16:27 |
| oz_akan_ | and I think we need to identify the audience | 16:28 |
| oz_akan_ | for marconi so we can decide easily what is in the scope of that vision, what is not | 16:28 |
| kgriffs | (made a couple tweaks if you want to refresh) | 16:28 |
| oz_akan_ | and vision statement may change after achieved] | 16:28 |
| oz_akan_ | that vision discards flavors | 16:29 |
| oz_akan_ | we won't write it now, I am just trying to gain some interest in defining the scope of the project as early as possible | 16:30 |
| oz_akan_ | I am scared of "backend" changes, rest vs amqp etc... | 16:31 |
| oz_akan_ | we invested a year in mongodb and now it is all for nothing | 16:32 |
| kgriffs | not all for nothing | 16:32 |
| kgriffs | there are lots of people who have no problem running mongodb | 16:32 |
| kgriffs | but there are also lots who would rather not, and need other options | 16:32 |
| oz_akan_ | now we know what we could use what not, that is a good start and if we define who is our audience we can understand what is more important to provide | 16:32 |
| kgriffs | +1 for defining the audience | 16:33 |
| oz_akan_ | kgriffs: we need to be able to move queues from one shard to another, queues on multiple shards etc. these will be different on different storage types | 16:33 |
| oz_akan_ | so if mongodb loses focus, that is going to effect the only in production marconi implementation on earth | 16:34 |
| oz_akan_ | if we keep mongodb as the primary, that seems to block us next time as well | 16:34 |
| *** AnastasiosAndron has quit IRC | 16:36 | |
| *** balajiiyer has quit IRC | 16:38 | |
| kgriffs | I started an etherpad. Let's brainstorm on there and discuss with flaper87 et al. | 16:43 |
| kgriffs | https://etherpad.openstack.org/p/marconi-mission | 16:43 |
| kgriffs | Before the summit, I want to overhall our wiki site and have all this incorporated | 16:43 |
| kgriffs | alcabrera: ping | 16:44 |
| kgriffs | amitgandhi: ping | 16:45 |
| kgriffs | bleh. My internets are clogged | 16:46 |
| kgriffs | guys, I'm gonna move locations. Be back online in a bit | 16:46 |
| *** ykaplan has quit IRC | 16:50 | |
| alcabrera | kgriffs: pong | 16:52 |
| * alcabrera catches up on mongodb discussion | 16:52 | |
| *** vkmc has quit IRC | 16:55 | |
| kgriffs | alcabrera: I was just thinking about the pop messages thing | 16:57 |
| oz_akan_ | kgriffs: I added who are not our users, I guess it helps too | 16:58 |
| kgriffs | http POST marconi.example.org/v1.1/queues/crunchy/claims?limit=10&pop=true | 16:59 |
| kgriffs | that's another idea | 16:59 |
| kgriffs | oz_akan_: definitely! | 16:59 |
| flwang | oz_akan_: ping | 17:00 |
| kgriffs | gotta run for a little while. be back l8r | 17:00 |
| kgriffs | o/ | 17:00 |
| flwang | oz_akan_: except the operation doc you contributed, is there any doc can be shared? now i'm working on setting up Marconi server with production starndard | 17:01 |
| alcabrera | kgriffs: I like that better, associating pop w/ claims | 17:01 |
| alcabrera | kgriffs: o/ | 17:02 |
| oz_akan_ | flwang: I don't have anything else at the moment | 17:04 |
| oz_akan_ | flwang: at some point we may have saltstack formulas | 17:04 |
| flwang | oz_akan_: so you're using saltstack to deploy Marconi? | 17:04 |
| flwang | oz_akan_: how to handle the scale issue? | 17:05 |
| oz_akan_ | yes we use saltstack | 17:05 |
| flwang | oz_akan_: so based on RAX policy, can the salkstack formulas be shared? | 17:06 |
| oz_akan_ | it could be shared if we didn't have anything specific to RAX which we have | 17:07 |
| *** rossk has joined #openstack-marconi | 17:07 | |
| oz_akan_ | that is why I was saying at some point we can open them and that point is based on the time we can invest to separate the formulas | 17:07 |
| flwang | oz_akan_: my question is how do you guys monitor the performance of Marconi and deploy more to handle the rquest spike? | 17:07 |
| oz_akan_ | at this point we deploy more than we need and won't have to act for spikes | 17:08 |
| flwang | oz_akan_: got it, thanks | 17:08 |
| oz_akan_ | scaling is not that granular | 17:08 |
| flwang | oz_akan_: so are you willing to monitor the performance/metering marconi? | 17:08 |
| oz_akan_ | sure | 17:09 |
| *** balajiiyer has joined #openstack-marconi | 17:09 | |
| oz_akan_ | we will probably make some changes in the way we deployed it | 17:09 |
| oz_akan_ | to be able to scale more granulary | 17:09 |
| *** kgriffs is now known as kgriffs_afk | 17:10 | |
| flwang | I'm going to contribute Ceilometer so as to leverage Heat to get better scale? | 17:10 |
| flwang | oz_akan_: any comments? | 17:10 |
| *** balajiiyer has quit IRC | 17:13 | |
| oz_akan_ | I am not sure if we need anything other than saltstack to scale automatically | 17:14 |
| flwang | oz_akan_: ok | 17:15 |
| flwang | oz_akan_: thanks | 17:15 |
| oz_akan_ | I guess you meant scaling automatically we could use heat right/ | 17:16 |
| oz_akan_ | ? | 17:16 |
| flwang | oz_akan_: exactly | 17:19 |
| *** balajiiyer has joined #openstack-marconi | 17:19 | |
| *** balajiiyer has left #openstack-marconi | 17:29 | |
| *** balajiiyer has joined #openstack-marconi | 17:29 | |
| amitgandhi | kgriffs_afk: you pinged? | 17:31 |
| *** flaper87 is now known as flaper87|afk | 17:41 | |
| alcabrera | Getting close: https://github.com/linsomniac/python-memcached/pull/26 (Python 3.3 support) | 17:42 |
| amitgandhi | +1 on using ceilometer and heat to scale automatically | 17:55 |
| amitgandhi | it gives us better openstack integration | 17:56 |
| * amitgandhi has caught up on thread | 17:56 | |
| amitgandhi | re google hangouts for the retro's why not do the "hangouts on air" so that everyone can at least watch if not participate directly | 17:57 |
| amitgandhi | kgriffs_afk: ^ | 17:57 |
| *** alcabrera is now known as alcabrera|afk | 18:00 | |
| *** balajiiyer has left #openstack-marconi | 18:01 | |
| *** kgriffs_afk is now known as kgriffs | 18:02 | |
| kgriffs | amitgandhi: that could work | 18:05 |
| kgriffs | re hangouts on air | 18:05 |
| kgriffs | flaper87|afk: ^^^ | 18:05 |
| kgriffs | amitgandhi: was wondering about your thoughts on adding a pop param to the claims resource instead of the messages resource | 18:06 |
| *** amitgandhi has quit IRC | 18:09 | |
| *** amitgandhi has joined #openstack-marconi | 18:09 | |
| *** oz_akan_ has quit IRC | 18:21 | |
| *** oz_akan__ has joined #openstack-marconi | 18:34 | |
| *** oz_akan__ has quit IRC | 18:37 | |
| *** amitgandhi has quit IRC | 18:39 | |
| *** oz_akan_ has joined #openstack-marconi | 18:40 | |
| kgriffs | folks, I am going to sign off for a bit so I can focus on teh codez. ttfn! | 18:49 |
| *** kgriffs is now known as kgriffs_afk | 18:50 | |
| *** malini is now known as malini_afk | 19:16 | |
| *** openstackgerrit has quit IRC | 19:18 | |
| *** openstackgerrit has joined #openstack-marconi | 19:18 | |
| *** alcabrera|afk is now known as alcabrera | 19:19 | |
| *** balajiiyer has joined #openstack-marconi | 19:19 | |
| *** balajiiyer has quit IRC | 19:19 | |
| *** balajiiyer has joined #openstack-marconi | 19:19 | |
| *** kgriffs_afk is now known as kgriffs | 19:21 | |
| *** amitgandhi has joined #openstack-marconi | 19:24 | |
| *** amitgandhi has quit IRC | 19:24 | |
| *** amitgandhi has joined #openstack-marconi | 19:25 | |
| *** balajiiyer has left #openstack-marconi | 19:25 | |
| *** oz_akan_ has quit IRC | 19:27 | |
| *** oz_akan_ has joined #openstack-marconi | 19:27 | |
| kgriffs | meeting | 19:32 |
| *** balajiiyer has joined #openstack-marconi | 19:33 | |
| *** tjanczuk has joined #openstack-marconi | 19:41 | |
| tjanczuk | Hello folks, anyone there? I was wondering if there have been any performance measurements of Marconi done, and if the results are available somewhere I can look? | 19:42 |
| kgriffs | tjanczuk: hi! | 19:43 |
| tjanczuk | Hello again. | 19:43 |
| kgriffs | tjanczuk: balajiiyer and malini_afk have done some tests and I've asked them to post some data on the wiki. I don't have an ETA on it yet, because the team has been in the midst of Summit and Juno planning, but feel free to ping them | 19:44 |
| tjanczuk | Thanks. Is there a good way to ping them other than through here? | 19:45 |
| kgriffs | tjanczuk: do you have specific requirements in mind? | 19:45 |
| kgriffs | tjanczuk: go ahead and send an email to [marconi] on the mailing list and I will get them to reply | 19:45 |
| kgriffs | :) | 19:45 |
| kgriffs | tjanczuk: it would be great if in that email you could mention a range of numbers you are after | 19:47 |
| kgriffs | for example, posting X number of messages per second or whatever | 19:47 |
| tjanczuk | I am generally trying to set my expectations in terms of latency and throughput as a function of the size of the deployment, number of queues, number of producers/consumers, type of backend, size of backend cluster etc.. Trying to understand how Marconi would do in a large multi-tenant deployment. | 19:47 |
| kgriffs | tjanczuk: the reason I ask is we want to have some concrete targets as we work on Marconi | 19:47 |
| kgriffs | tjanczuk: ok, that is some good background you could include in your email as well. | 19:48 |
| kgriffs | if you have certain variables in mind, please mention those as well | 19:48 |
| kgriffs | such as number of queues, size of cluster, etc. | 19:48 |
| tjanczuk | Sounds good. Let me move this to the mailing list. | 19:48 |
| kgriffs | cool, thanks! | 19:49 |
| tjanczuk | By the way, I just learned I am going to the summit in Atlanta, so I hope we will have a chance to meet in person. | 19:53 |
| alcabrera | tjanczuk: awesome! Hope to see you there. :) | 19:55 |
| sriram | alcabrera, kgriffs : Could you have a look at the patch, Jenkins seems to have +1'd it. https://review.openstack.org/#/c/82838/ | 20:02 |
| alcabrera | sriram: +2 | 20:03 |
| sriram | alcabrera: Thanks. | 20:04 |
| alcabrera | Good night, everyone. :) | 20:07 |
| *** alcabrera has quit IRC | 20:08 | |
| openstackgerrit | A change was merged to openstack/marconi: fix() : introduce division import https://review.openstack.org/82838 | 20:11 |
| *** tjanczuk has quit IRC | 20:15 | |
| *** sriram has quit IRC | 20:22 | |
| *** vkmc has joined #openstack-marconi | 20:24 | |
| *** vkmc has quit IRC | 20:24 | |
| *** vkmc has joined #openstack-marconi | 20:24 | |
| *** tjanczuk has joined #openstack-marconi | 20:38 | |
| kgriffs | balajiiyer: ping | 21:05 |
| amitgandhi | kgriffs: re pop parameter | 21:09 |
| amitgandhi | so you would claim a message with pop, which hides the message from all others, and deletes it from the queue? | 21:09 |
| amitgandhi | all in one step? | 21:09 |
| kgriffs | amitgandhi: correct. | 21:09 |
| amitgandhi | so claim semantic is maintained, and a subsequent delete call is now unecessary | 21:09 |
| kgriffs | you can compare that to adding a pop query param to messages resource | 21:10 |
| kgriffs | amitgandhi: right, although the only loose end | 21:10 |
| kgriffs | is you have a claim created that isn't really necessary | 21:10 |
| kgriffs | actually, i think claims aren't implemented as separate records in mongo, so that is ok | 21:10 |
| amitgandhi | if you pop the message, and crash before processing it, you lose the message right (since no claim ttl) | 21:10 |
| kgriffs | but they exist in SQL | 21:10 |
| amitgandhi | does pop infer FIFO? | 21:11 |
| kgriffs | amitgandhi: that is correct, you lose it. That is the tradeoff, but for some kinds of events that is OK (or so one user told me) | 21:11 |
| kgriffs | amitgandhi: pop is identical to claiming, then deleting all messages before the claim expires. messages are claimed in FIFO order | 21:11 |
| kgriffs | iirc | 21:12 |
| * kgriffs checks | 21:12 | |
| amitgandhi | kgriffs: i bought that up since we were considering dropping FIFO guarantee | 21:12 |
| amitgandhi | kgriffs: i dont think i understood the loose end about claims existing | 21:13 |
| *** tedross has quit IRC | 21:17 | |
| *** balajiiyer has quit IRC | 21:23 | |
| *** Alex_Gaynor has quit IRC | 21:39 | |
| *** Alex_Gaynor has joined #openstack-marconi | 21:41 | |
| *** oz_akan_ has quit IRC | 21:45 | |
| *** russellb has quit IRC | 21:47 | |
| *** metral has quit IRC | 21:47 | |
| *** russellb has joined #openstack-marconi | 21:48 | |
| *** mwagner_lap has quit IRC | 21:49 | |
| *** metral has joined #openstack-marconi | 21:49 | |
| kgriffs | amitgandhi: oh, just that conceptually you are creating a claim, deleting it's messages, but leaving the claim around | 21:50 |
| kgriffs | (if you add the pop=true param) | 21:50 |
| amitgandhi | how does the ttl on it work today (ie how does it get GC when the message TTL expires?) | 21:51 |
| amitgandhi | can we not code the sql driver to delete the message on pop() | 21:51 |
| *** malini_afk is now known as malini | 21:52 | |
| *** cpallares has left #openstack-marconi | 21:53 | |
| *** mpanetta has quit IRC | 21:54 | |
| kgriffs | so, for the mongo driver we just tag each claimed message with a claim ID and TTL, so there is no separate claim resource. We just filter out "claimed" messages on the fly by checking current timestamp against the claim ttl | 21:56 |
| malini | tjanczuk: ping | 21:56 |
| kgriffs | for sql, it is in another table and I haven't looked at that code for a while... | 21:56 |
| tjanczuk | malini: hello | 21:56 |
| kgriffs | but I know messages themselves are deleted on the fly during any message GET operation - not ideal, obviously, and so we had a work item to create a garbage collector | 21:57 |
| malini | tjanczuk: hello..Thank you for your interest in Marconi benchmarks | 21:57 |
| tjanczuk | I am all ears | 21:58 |
| kgriffs | malini: please also reply on the ML for posterity. :) | 21:58 |
| kgriffs | amitgandhi: anyway, if we aren't planning on supporting sqla driver in production deployments, then that optimization is less important | 21:58 |
| malini | tjanczuk:I have some tsung xmls & set up instruction to run the benchmarks. Are you interested in doing some tests with those? | 21:58 |
| amitgandhi | kgriffs: agreed | 21:59 |
| malini | kgriffs: sure..I just wanted to chk if tjanczuk wanted our test scripts | 21:59 |
| kgriffs | amitgandhi: wrt FIFO, I think we may still want to offer it, but it would be optional | 21:59 |
| malini | mainly to make sure if he wants additional operations etc | 21:59 |
| kgriffs | malini: cool | 21:59 |
| *** jmckind has quit IRC | 22:00 | |
| tjanczuk | malini: at some point I may want to use them for private testing; in the meantime I was hoping you would have some numbers to share. | 22:00 |
| malini | tjanczuk: I blew away the tsung servers with the reports after the benchmarks | 22:01 |
| malini | But we can re-run them & share in the ML | 22:01 |
| malini | hopefully within the next 2-3 days | 22:02 |
| tjanczuk | That would be very helpful. Do you recall any numbers off the top of your head? Ballparks? | 22:03 |
| malini | we were able to reach ~ 2000 rps, using 4 webservers | 22:08 |
| malini | The latency was in the milliseconds range…But I dont remmeber accurately | 22:09 |
| *** jcru has quit IRC | 22:15 | |
| amitgandhi | kgriffs: my updates are up on the etherpad for the vision/mission | 22:15 |
| amitgandhi | im heading out now | 22:16 |
| amitgandhi | ttyl | 22:16 |
| kgriffs | tjanczuk: fwiw, we are planning to start benchmarking marconi with a cron job or something and charting the results over time. Performance will be a big focus during Juno. | 22:16 |
| kgriffs | amitgandhi: thanks! | 22:16 |
| amitgandhi | np | 22:16 |
| kgriffs | tjanczuk: we are also looking for independent tests to put up on the wiki | 22:16 |
| *** amitgandhi has quit IRC | 22:17 | |
| tjanczuk | kgriffs: sounds good | 22:17 |
| tjanczuk | malini: re 2000 rps, was it publishing throughput, or end to end? Was it using Mongo as the storage or comething else? How was storage scaled? | 22:18 |
| tjanczuk | Also, how much performance optimization work had been done on Marconi to date? Are bottlenecks understood? | 22:20 |
| malini | tjanczuk: We did the tests with external cluster generating the load . We had a tsung cluster generating traffic. We used Mongo as the backend | 22:21 |
| malini | We used Marconi's sharding to scale. But oz_akan will be the right person to give you more info on how we scaled the DB | 22:22 |
| malini | tjanczuk: We went through a round of perf optimizations last November (tht was when we did the tests) | 22:23 |
| tjanczuk | What was the bottleneck in the test? CPU? Memory? Network? | 22:23 |
| tjanczuk | And at which layer? Transport or DB? | 22:23 |
| malini | It was probably at the DB | 22:23 |
| malini | But I will let flaper87|afk or oz_akan to answer that | 22:24 |
| *** balajiiyer has joined #openstack-marconi | 22:24 | |
| malini | tjanczuk: These are good questions.Can you append that to your email in the ML ? | 22:24 |
| malini | We will attempt to answer all of those along with benchmark numbers from a new test | 22:25 |
| tjanczuk | Will do, thanks. Do you remember what the structure of the test was? One queue or multiple queues, how many publishers and subscribers per queue, message size? | 22:26 |
| malini | tjanczuk: multiple queues.. | 22:26 |
| tjanczuk | Also when you say "latency in millisecond range", what was included in that timing? | 22:26 |
| tjanczuk | Were messages produced and consumed one at a time or in bulk? | 22:27 |
| *** balajiiyer has quit IRC | 22:28 | |
| malini | tjanczuk: Let me just update the github repo with the tsung config file | 22:28 |
| malini | That'll answer most of your questions | 22:28 |
| tjanczuk | Great, can you post a link back here once you do please? | 22:29 |
| malini | On further thoughts..I will do that..But IRC discussion is easier to understand than an XML | 22:29 |
| malini | Updating the repo now | 22:29 |
| *** Kuwagata has quit IRC | 22:30 | |
| *** oz_akan_ has joined #openstack-marconi | 22:31 | |
| *** jergerber has quit IRC | 22:44 | |
| *** oz_akan_ has quit IRC | 22:45 | |
| malini | tjanczuk: See https://github.com/rackerlabs/csi-marconi | 22:49 |
| malini | Let me know if you are interested in additional scenarios beyond what we currently have in tsung.xml | 22:51 |
| tjanczuk | Thanks, I need some time to process this. Once you re-run the tests (in a couple of days?), how are you going to share the results? | 22:55 |
| malini | tjanczuk: mailing list | 22:56 |
| malini | we can chat further in the ML/ here if you need more clarifications | 22:56 |
| *** mwagner_lap has joined #openstack-marconi | 23:02 | |
| tjanczuk | Great, thanks. | 23:02 |
| malini | yw..ttyl | 23:02 |
| *** malini has left #openstack-marconi | 23:04 | |
| *** whenry has joined #openstack-marconi | 23:50 | |
| *** mwagner_ has joined #openstack-marconi | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!