15:00:00 <DinaBelova> #startmeeting Performance Team
15:00:01 <openstack> Meeting started Tue Nov 10 15:00:00 2015 UTC and is due to finish in 60 minutes.  The chair is DinaBelova. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:05 <openstack> The meeting name has been set to 'performance_team'
15:00:07 <DinaBelova> Hello everyone! o/
15:00:12 <DinaBelova> who's here? :)
15:00:19 <kun_huang> +1
15:00:27 <DinaBelova> harlowja - were you able to wake up? :)
15:01:17 <DinaBelova> hope some other channel members are here as well :)
15:01:47 <badari> +1
15:01:49 <DinaBelova> regXboi, SpamapS - are you around as well?
15:02:18 <DinaBelova> ok, let's wait for few more minutes
15:02:18 <Kristian_> so not used to this type of meeting, more of an ATT Connect user, but #it works
15:02:22 <DinaBelova> :D
15:02:40 <Kristian_> #itworks
15:02:47 <DinaBelova> while people are waking up - here is our agenda
15:02:53 <DinaBelova> #link https://wiki.openstack.org/wiki/Meetings/Performance#Agenda_for_next_meeting
15:03:18 <DinaBelova> I decided to start with something related to the passed summit
15:03:19 <DinaBelova> #topic Summit follow-up
15:03:38 <DinaBelova> as you remember we had kick-off session in Tokyo
15:03:40 <DinaBelova> #link https://etherpad.openstack.org/p/mitaka-cross-project-performance-team-kick-off
15:03:45 <regXboi> regXboi is here
15:03:48 <DinaBelova> yay!
15:04:04 <DinaBelova> I see one moment in the etherpad that seems interesting to me and not covered during the summit session.
15:04:12 <DinaBelova> #info What's good for performance test (time to do $thing) vs scale test (behavior of doing $thing in parallel)
15:04:19 * regXboi is also over in the neutron drivers meeting so running between two rooms
15:04:19 <DinaBelova> at Mirantis we use Rally for this purpose with different coefficients.
15:04:28 <DinaBelova> yeah, that's sad...
15:04:41 <DinaBelova> regXboi - hope you were able to go through the etherpad?
15:04:54 <regXboi> unfortunately ... no
15:04:55 <regXboi> :(
15:05:00 <DinaBelova> heh :)
15:05:00 <regXboi> or at least - not yet
15:05:22 <DinaBelova> ok, i'll kindly ask you to do that in some nearest future :)
15:05:33 <regXboi> there is an RFE over in the neutron drivers meeting that I want to reflect up to oslo because I think it has impact to this effort
15:05:49 <patrykw> hi all patrick@intel
15:05:54 <DinaBelova> cool, thank you for visiting it
15:05:57 <DinaBelova> patrykw o/
15:06:20 <regXboi> ok, did a quick scan :)
15:06:27 <DinaBelova> so quick description of what happened on summit and what feedback was collected
15:06:36 <DinaBelova> the main input from different community people on the summit for this team was to start with common methodologies, tool sets, ways to compare different deployments, etc. and eventually work on recommendations for projects, writing up testing methodologies and best deployment practices, and share results of the testing.
15:06:56 <DinaBelova> so we had many people, many tools, lots scenarios, etc.
15:07:11 <DinaBelova> and I decided to start with basic standardisation
15:07:30 <DinaBelova> like "what tools to name as the canonical ones?"
15:07:32 <DinaBelova> etc.
15:07:52 <regXboi> I guess I'd ask "what do people mean by performance?" as that word means different things to different people
15:08:04 <DinaBelova> yes, that's true
15:08:07 <kun_huang> regXboi: agreed...
15:08:08 <regXboi> (sorry for the meta question)
15:08:28 <DinaBelova> at mirantis we use to measure two things during the performance testing
15:08:44 <DinaBelova> control plane performance (operations over OpenStack resources)
15:08:49 <kun_huang> perf = benchmark, tracing, tuning and so on
15:09:07 <kun_huang> there're a lots of tools on for different purposes
15:09:15 <regXboi> DinaBelova: ok, so what do you mean by "control plane performance?"
15:09:19 <badari> DinaBelova: Control plane performance without any data plane load at all?
15:09:19 <DinaBelova> like what time it'll take to boot Vm, assign floating IP, etc. in different load
15:09:20 <kun_huang> rally is good at benchmark of API level
15:09:27 <DinaBelova> and data plane separately :)
15:09:30 <Kristian_> we use Rally now and modify some of the out of the box .json scenarios according to our architecture :)
15:09:35 <DinaBelova> kun_huang I'll agree
15:09:41 <DinaBelova> cool :)
15:10:01 <Kristian_> mainly because its Free and we are saving on test tool licensing.  but it is a flexible, great too;l
15:10:03 <regXboi> well, I'll say I take a slightly different view of control plane performance
15:10:12 <augiemena3> we use Rally as well and make modifications as well for our use
15:10:12 <DinaBelova> and for data plane we use Shaker  https://github.com/openstack/shaker (Functionality is being moved from Shaker to Rally)
15:10:14 <Kristian_> we are looking at jmeter next for load/VM testing as well
15:10:22 <DinaBelova> regXboi - please share the idea
15:10:28 <Kristian_> @DinaBelova good news!
15:10:43 <andreykurilin> hi all:)
15:10:44 <regXboi> I tend to look *very closely* at execution time of operations in the agents/API/etc
15:10:51 <kun_huang> Kristian_:  is there some good points on jmeter other than rally?
15:11:10 <regXboi> because at it's core - the messaging RPC is just a queuing system
15:11:23 <regXboi> er queueing
15:11:28 <DinaBelova> well, it looks like I need to change topic and return back to something else in open discussion
15:11:29 <DinaBelova> #topic Rally as a control plane benchmarking tool agreement
15:11:56 <patrykw> in our team we have different approach, we are simulating LOB applications on cloud and "simulating real" (strange :)) workload by gatling
15:11:56 <augiemena3> I do think it's important to categorize the performance tools:  benchmarks, benchmark harnesses, workloads, profiling/tracing, monitoring, etc
15:12:06 <Kristian_> @regXboi we are looking at it for virtual machine testing, something Rally does not yet do
15:12:15 <kun_huang> augiemena3: +1
15:12:19 <patrykw> applications do not stressing the resources on max
15:12:24 <Kristian_> we need something open-source based and easier to learn, community supported as well
15:12:27 <DinaBelova> augiemena3 +1
15:12:28 <regXboi> ok, so Rally is very good at supplying API load, which lets you stress the API and the input to the RPC
15:12:53 <DinaBelova> Kristian_ - it's going to support that type of testing in future
15:12:55 <patrykw> plus during the test checking with rally how the API is working
15:13:04 <DinaBelova> Kristian_ - workloads testing
15:13:09 <DinaBelova> andreykurilin - am I right?
15:13:11 <regXboi> but AFAICT, OpenStack lacks the ability to report on execution time in the agents
15:13:12 <kun_huang> regXboi: if you just need execution time on agent and rpc level, there would be a lot of tools
15:13:25 <regXboi> and note: I'm not just saying "profiling"
15:13:31 <andreykurilin> DinaBelova: yes
15:13:48 <rohanion> hi everyone
15:13:50 <kun_huang> regXboi: I think boris' osprofiler is design for that
15:13:54 <DinaBelova> ok, so we can assume Rally is good for the API testing
15:14:04 <Kristian_> @DinaBelova that's right, workload as well as VM characteristics measurements we are looking at.  we need to develop SLAs in the integrated cloud , and that is difficult to do w/o a standard set of measurement
15:14:08 <patrykw> +1
15:14:09 <andreykurilin> DinaBelova: kun_huang is also core of rally, so he can answer to all rally related question too :)
15:14:12 <DinaBelova> for profiling and tracing we can eventually use osprofiler
15:14:14 <kun_huang> regXboi: but I just read his writings without real test
15:14:18 <regXboi> kun_huang: it's possible, but it has to be something built into the system and turned off and on with configuration
15:14:30 <regXboi> DinaBelova: please note what I just said :)
15:14:30 <DinaBelova> and eventually rally will support dataplane as well
15:14:44 <DinaBelova> yep, sorry :)
15:14:44 <kun_huang> regXboi: haha, have you tried systemtap/ftrace?
15:15:19 <DinaBelova> regXboi - that's true, although if it will be deeply integrated with openstack projects - why not in fact?
15:15:43 <regXboi> DinaBelova: because I'm trying to get to the point where I can do continuous test for performance
15:16:10 <regXboi> and I think that means the tools have to be there and turned on/off via a simple config switch
15:16:10 <kun_huang> regXboi: "continuous test for performance" +1 for this good point
15:16:42 <regXboi> er sorry ... tools was a bit of a misnomer
15:16:50 <DinaBelova> regXboi - so you want like one place to have this swithed on/off
15:17:07 <regXboi> DinaBelova: so let me give you an example: https://etherpad.openstack.org/p/hyper-scale
15:17:30 <regXboi> right now to do that, I'm having to decorate files manually to get increasing levels of execution time data
15:17:34 <DinaBelova> looking
15:17:35 <regXboi> and anything manual doesn't work
15:17:56 <DinaBelova> yeah, that's true..
15:18:03 <DinaBelova> well, it's one more moment to define
15:18:03 <regXboi> so whatever we use for execution time data has to be built in and has to be easily turned on and off so that we don't pollute logs
15:18:13 <regXboi> once you *have* that
15:18:20 <regXboi> then we get into the whole statistical tool discussion
15:18:32 <DinaBelova> although it more related to the tracing and profiling
15:18:52 <DinaBelova> if we are talking about API benchmarking Rally is pretty good here
15:18:57 <DinaBelova> I suppose we all can agree hee
15:18:59 <DinaBelova> here*
15:19:02 <regXboi> as I said, the RPC is essentially a queueing system
15:19:07 <regXboi> Rally drives the arrival rate
15:19:27 <DinaBelova> yes, so we need to extend it with other tools, but I suppose we still need it
15:19:29 <regXboi> but something needs to document the processing time so that you know what's causing you problems if the RPC starts to break
15:19:38 <DinaBelova> agree
15:19:41 <regXboi> s/if/when/
15:19:45 <DinaBelova> :d
15:19:47 <DinaBelova> :D
15:19:56 <DinaBelova> yes, that's true
15:20:00 <regXboi> so... I agree that Rally is great for the front end :)
15:20:10 <patrykw> testing data plane with Rally ?? really ?
15:20:17 <DinaBelova> and we need something else for deeper analysis
15:20:21 <DinaBelova> patrykw - eventually :)
15:20:30 <DinaBelova> we have rally contributor here :)
15:20:31 <harlowja_at_home> DinaBelova, i made it!
15:20:35 <harlowja_at_home> better late than never :-P
15:20:36 <DinaBelova> harlowja_at_home o/
15:20:41 <harlowja_at_home> :)
15:20:43 <harlowja_at_home> only half zombine
15:20:45 <DinaBelova> patrykw - andreykurilin is Rally core
15:20:45 <Kristian_> question: does Rally log all API objects it creates/destroys (i believe it does) and does it have a separate log report, similar to the reports that it comes pre-packaged with?
15:20:47 <kun_huang> regXboi: https://github.com/openstack/scalpels/blob/master/scripts/rbt-trace.py at least, this script could be used RPC tracing
15:21:15 * regXboi looks
15:21:37 <DinaBelova> Kristian_ - yep, it logs, yes it has separated log report for everything it does
15:21:41 <regXboi> kun_hunag: I'm looking to go deeper
15:21:58 <regXboi> that will tell me about the RabbitMQ, which is certainly useful
15:21:59 <andreykurilin> DinaBelova: we have a lot of rally contributors here: kun_huang, oanufrief
15:22:00 <DinaBelova> kun_huang - thanks for the link, I'll need to go through it further
15:22:10 <Kristian_> @DinaBelova thank you!  looking for a runtime type of report
15:22:11 <DinaBelova> andreykurilin - yep, sure
15:22:15 <andreykurilin> :)
15:22:25 <regXboi> but I'm looking to root cause the delays in processing to get them fixed in the code that serves the RPC
15:22:56 <kun_huang> regXboi: I use dtrace to catch python function call time...
15:23:09 <regXboi> kun_huang: automagically?
15:23:11 <kun_huang> but that is not implemented in Ubuntu
15:23:19 <DinaBelova> so I think we can assume the following: we have Rally now for front-end API testing, it will be able to do dataplane with its help in future, and we need something else to catch what's going on deeper while request processing
15:23:24 <Kristian_> @kun_huang does it provide stress against RabbitMQ?
15:23:28 <kun_huang> regXboi: script only, I'm working on my prototype
15:23:33 <andreykurilin> DinaBelova: +1
15:23:50 <DinaBelova> in fact it's not only RPC, it's various drivers work, it's DB performance, etc
15:23:53 <kun_huang> Kristian_: that rbt-trace.py, don't; it is just rpc log
15:24:02 <DinaBelova> so it looks like it'll be some number of additional tools
15:24:11 <andreykurilin> osprofiler can be really useful for profiling/tracing stuff, but it need more contribution:(
15:24:21 <Kristian_> i see. because that is something also we are looking for, as a stressful/load component against RPC queues and RabbitMQ
15:24:31 <kun_huang> andreykurilin: yep, it is
15:24:42 <DinaBelova> kun_huang, Kristian_, regXboi - would you agree with the points I mentioned?
15:24:58 <kun_huang> andreykurilin: about osprofiler, I think boris need bring more and good demo for users
15:25:16 <kun_huang> DinaBelova: I agree with Rally is a good openstack benchmark framwork
15:25:30 * harlowja_at_home will try to work on osprofiler stuff soon, will do some tweaks to it maybe later this week...
15:25:36 * DinaBelova as well
15:25:38 <andreykurilin> kun_huang: the main problem of osprofiler is a cores of other projects, since osprofiler needs integration with them
15:25:50 <andreykurilin> harlowja_at_home: nice
15:25:53 <regXboi> DinaBelova: I would summarize as a profiler that is built into all OpenStack projects (likely via oslo)
15:25:58 <Kristian_> yes, we need some community tools that do back-end stress to the VMs and databases w/ a front-end manager
15:26:10 <regXboi> and if osprofiler fits the bill, sure
15:26:12 <kun_huang> andreykurilin: I think we should educate them, there fantastic tracing tool has to built into native codes
15:26:14 <DinaBelova> regXboi - agreed
15:26:18 <harlowja_at_home> andreykurilin, yes that, we need to make a joint effort to fix what cores have complained about and figure out why its not integrated yet...
15:26:27 <kun_huang> s/there/every
15:26:29 <DinaBelova> ok, let me write something with >agreed< tag
15:26:42 <regXboi> is osprofiler part of oslo?
15:26:44 <harlowja_at_home> kun_huang, +1 education, and working through the issues they have with it...
15:26:46 <regXboi> if not, it should be
15:26:48 <kun_huang> regXboi: nope
15:26:49 <harlowja_at_home> regXboi, not yet
15:26:50 <regXboi> er it *must* be
15:26:53 <regXboi> ok, that's #1
15:26:57 <patrykw_> ku_huang: from the perspecive of end user, which using VM, the numbers from Rally is what he wanted to see ??, or he just want to have information how well the VM is working and plus if infrastructure is fine
15:27:04 <andreykurilin> harlowja_at_home: as far as I remember, they were against check `if profiler == None`, since it can produce performance issues :D
15:27:18 <harlowja_at_home> andreykurilin, yes, we have to bust that kind of dumb stuff
15:27:31 <harlowja_at_home> but in a politically nice way, ha
15:27:32 <kun_huang> patrykw_: I think you are looking for a "vm workload benchmark framework"
15:27:34 <DinaBelova> #agreed Rally is good benchmarking solution that we want to use widely for OpenStack API benchmarking and dataplane testing in future. Although we need something else for profiling that is built into all OpenStack projects
15:27:52 <kun_huang> patrykw_: I'm not sure ;(
15:28:05 <augiemena3> @DinaBelova +1
15:28:10 <patrykw_> kun_huang: what we want to benchmark ?? only APIs or something more ??
15:28:18 <andreykurilin> kun_huang, patrykw_: we have serie of patches for "vm workload", but they need more reviewers
15:28:31 <andreykurilin> *we = Rally
15:28:34 <patrykw_> :)
15:28:41 <kun_huang> patrykw_: both API level benchmark and vm workload benchmark
15:28:43 <regXboi> DinaBelova, can we go further and say the profiling has to be controlled via config as well as by decoration?
15:28:58 <DinaBelova> regXboi - I'll agree here
15:29:11 <regXboi> I mean, I'd like to have decoration for one offs :)
15:29:27 <regXboi> but CT says config (unless somebody can find a better solution)
15:29:28 <DinaBelova> we need single opportunity to switch this possibility on/off
15:29:35 <patrykw_> andreykurilin: it is fine to use Rally to test API, we are doing this same but vm workload is something else
15:29:59 <DinaBelova> patrykw_ - this is just new commits and new Rally functinality
15:30:04 <DinaBelova> that's in progress now :)
15:30:07 <andreykurilin> patrykw_: What is the difference?)
15:30:08 <kun_huang> patrykw_:  yes it is...
15:30:12 <harlowja_at_home> regXboi, as for oslo, let's work on that, and chat with dims and such, i'd like to capture a list of why it isn't integrated into say nova, keystone... and attempt to resolve those (either by dispelling myths or fixing code) before osprofiler ---> oslo though....
15:30:23 <Kristian_> @DinaBelova - we also need a way to isolate troublesome VMs from a test, that it picks up from statistics during runtime by IP/name/node identification
15:30:23 <regXboi> patrykw_: I suspect that we need to come up with a common vm workload to handle that stressing
15:30:32 <DinaBelova> harlowja_at_home +1
15:30:42 <regXboi> harlowja_at_home: well, that's a bit of a chicken and egg problem
15:31:04 <DinaBelova> Kristian_ - I suppose eventually it's possible, although not sure how to do that right now
15:31:07 <patrykw_> DinaBelova: want to see that :) currently we are using mix of PKB + gatling to achieve some level of workload
15:31:11 <regXboi> I'd like to have that conversation begin with "this is going into oslo, and you really need to integrate to support CT, so what can we do to help with that"
15:31:20 <DinaBelova> patrykw_ brrr...
15:31:31 <kun_huang> I think today's meeting is one hour open discussion ;)
15:31:33 <DinaBelova> yep, it'll be very cool to have everything in one box
15:31:44 <andreykurilin> kun_huang: :D
15:31:49 <Kristian_> @DinaBelova as long as we can get requirement in for future development in Rally or a back-end version of Rally early, i'm fine w/ that answer
15:31:57 <Kristian_> :)
15:32:02 <DinaBelova> kun_huang, andreykurilin - do you have links to the workloads patches?
15:32:04 <andreykurilin> regXboi: there was a spec of moving osprofiler to oslo namespace
15:32:06 <DinaBelova> kun_huang - well, yes
15:32:12 <andreykurilin> DinaBelova: one moment please
15:32:15 <DinaBelova> I was too stupid to create an agenda
15:32:17 <harlowja_at_home> regXboi, sure, i think some of the issues that will be captured aren't 'it isn't in oslo so i won't integrate it'
15:32:18 <regXboi> andreykurilin: is there a #link?
15:32:19 <DinaBelova> andreykurilin sure
15:32:31 <kun_huang> #link https://review.openstack.org/#/q/status:open+project:openstack/rally+branch:master+topic:bp/vm-workloads-framework,n,z
15:32:38 <DinaBelova> kun_huang thank you sir
15:32:40 <andreykurilin> :)
15:32:44 <regXboi> harlowja_at_home: yes I don't want to be blocked by that quote :)
15:33:03 <harlowja_at_home> regXboi, we can resolve that quote if it happens rather quickly imho...
15:33:17 <Kristian_> @DinaBelova you were asking about other tools, we are using jmeter and looking at traffic engineering with SmartBits iTest / Hyperscale solution for scale test
15:33:20 <harlowja_at_home> but saying something like 'shutup u crazy person, it will go into oslo once we resolve the other issues'
15:33:28 <harlowja_at_home> ^ not exactly like that , ha
15:33:29 <Kristian_> we don't have a PoC in place yet
15:33:33 <andreykurilin> regXboi: https://review.openstack.org/#/c/103825/
15:33:37 <DinaBelova> Kristian_ - a-ha, thank you sir
15:33:38 <regXboi> harlowja_at_home: yes, that isn't political (rotflmao)
15:33:46 <harlowja_at_home> :)
15:33:47 <andreykurilin> regXboi: as you can see there were a lot of + for osprofiler:)
15:34:07 <regXboi> andreykurilin: on my list to read and comment on today
15:34:19 <regXboi> and thx :)
15:34:27 <kun_huang> andreykurilin: I'm 100% sure that osprofiler will be a good solution
15:34:29 <patrykw_> i was working on other cloud solutions and main think always was how much load (instances) I can put without issues, issues to the resources, and issues to the management plane
15:34:50 <DinaBelova> so let's write something in OSprofiler topic to have beautiful notes
15:34:51 <regXboi> DinaBelova: I assume we want to put of the "statistical" discussion for another day?
15:34:54 <DinaBelova> #topic Taking part in the OSprofiler initiative
15:34:54 <kun_huang> google's dapper and twitter's zikpin examples
15:34:58 <DinaBelova> regXboi - I suppose yes
15:34:59 <regXboi> er off
15:35:01 <andreykurilin> btw, everyone saw the rexample of osprofiler reports? http://boris-42.github.io/ngk.html
15:35:06 <regXboi> DinaBelova: I'm good with thta :)
15:35:11 * regXboi looking
15:35:33 <patrykw_> will the rally tell us: hold done, you are going to far ?? :)
15:35:34 <DinaBelova> so it looks like during Mitaka there will be some activity on making osprofiler workable enough for all projects
15:35:51 <DinaBelova> harlowja_at_home, me and Boris at least will try to do that
15:36:12 <andreykurilin> DinaBelova: don't forget about me:)
15:36:12 <DinaBelova> and if this solution will be fully ok to be used for tracing, I believe it'll become an oslo standard
15:36:16 <andreykurilin> I'll try to help too
15:36:18 <regXboi> does osprofiler only do db and rpc or can it go to specific python functions?
15:36:18 <DinaBelova> andreykurilin sorry sir :)
15:36:28 <kun_huang> regXboi: yep
15:36:36 <regXboi> kun_hunag: yep == ???
15:36:44 <harlowja_at_home> regXboi, == yes
15:36:49 <andreykurilin> :)
15:36:55 <regXboi> no, I asked an A or B - > yes doesn't answer
15:36:58 <kun_huang> regXboi: osprofiles covers db level things
15:37:03 <regXboi> ok, and RPC
15:37:17 <Kristian_> will chk out osprofiler, thx for the tool tip @andreykurilin
15:37:18 <kun_huang> regXboi: in document, it covers too...
15:37:18 <regXboi> but *does it cover specific functions in the agent code*
15:37:34 <kun_huang> regXboi: no it doesn't
15:37:37 <harlowja_at_home> if u make it do so :)
15:37:43 <DinaBelova> harlowja_at_home ++
15:37:44 <regXboi> ok... *that's* a problem
15:37:58 <regXboi> that means osprofiler doesn't in its current form go deep enough
15:38:05 <harlowja_at_home> it doesn't integrate with pdb to automatically do all the things
15:38:07 <DinaBelova> I thing it can be developed after making current unctioanlity workable
15:38:09 <harlowja_at_home> its not magic ;)
15:38:10 <kun_huang> regXboi: that's the reason I am using systemtap
15:38:19 <DinaBelova> regXboi - right now - yes
15:38:36 <DinaBelova> but if we want unified way of tracing, we need to work on this...
15:38:38 <andreykurilin> regXboi: I suppose osprofiler can be a base for good profiling tool, but it need some work on it
15:39:06 <DinaBelova> regXboi - otherwise everyone will be using something they use to have and the results won't be comparable at all
15:39:13 <regXboi> all - I'm not saying we shouldn't improve it, I'm just working to understand what it would need for this
15:39:20 <andreykurilin> :)
15:39:24 <regXboi> or I should say what I think it would need for this
15:39:27 <DinaBelova> and that's something I want to avoid
15:39:27 <DinaBelova> sure
15:39:50 <DinaBelova> regXboi - I'm very glad to collect your feedback, it's very useful :)
15:40:10 <regXboi> and believe me, I'll make these comments on that commit :)
15:40:19 <regXboi> because what I see, I *mostly* like
15:40:30 <DinaBelova> :D
15:40:30 <regXboi> but it needs some more pieces to make it sing
15:40:51 <regXboi> another question - are we sold on Ceilometer only for reporting?  what about simple logs?
15:41:06 <DinaBelova> ok, so it looks like we have solid opinion here - OSprofiler may become one day the tracing / deeper analysis tool, but it requires some work to be done
15:41:20 <andreykurilin> regXboi: it supports text files
15:41:39 <regXboi> andreykurilin: ok, I missed that on the quick scan
15:41:48 <andreykurilin> also, there is a spec for multi backend - https://github.com/openstack/osprofiler/blob/master/doc/specs/in-progress/multi_backend_support.rst
15:41:50 <DinaBelova> regXboi - one of the boris-42 concerns was to make more collector backends, but simple text was available afair
15:42:00 <andreykurilin> #link http://lists.openstack.org/pipermail/openstack-dev/2015-November/078845.html
15:42:00 <DinaBelova> #link https://github.com/openstack/osprofiler/blob/master/doc/specs/in-progress/multi_backend_support.rst
15:42:19 <andreykurilin> the latest thread related to osprofiler
15:42:35 <DinaBelova> nd all specs that need to be covered at least are placed there as well
15:42:36 <regXboi> andreykurilin: yes I saw that :)
15:43:05 <DinaBelova> ok, so it looks like we're done with osprofiler
15:43:12 <DinaBelova> any more notes about it?
15:43:25 <DinaBelova> ok
15:43:26 <DinaBelova> #topic Data plane testing tools
15:43:34 <andreykurilin> rally!
15:43:35 <andreykurilin> :D
15:43:36 <DinaBelova> ok, so later rally will support it
15:43:38 <DinaBelova> ;)
15:43:42 <andreykurilin> ;(
15:43:54 <DinaBelova> Kristian_ is using jmeter and looking at traffic engineering with SmartBits iTest / Hyperscale solution for scale test
15:44:04 <DinaBelova> for some of the bits
15:44:12 <DinaBelova> Mirantis uses shaker right now
15:44:24 <badari> DinaBelova: we are looking at using "cbtool" currently
15:44:24 <kun_huang> I know vmtp/kloudbuster
15:44:28 <DinaBelova> #link https://github.com/openstack/shaker
15:44:55 <DinaBelova> ok, so it looks like until we'll have nice Rally solution it'll be a complete mess here :)
15:44:55 <regXboi> So, here is where I think it may make sense to come up with a standard image that can be bundled with OpenStack?
15:45:17 <DinaBelova> regXboi - that will be very logical I believe
15:45:28 <Kristian_> @DinaBelova can u explain to me in a nutshell what shaker does?  is it L2/L3 stress tool?
15:45:40 <patrykw_> regXboi :)
15:45:41 <DinaBelova> Kristian_ - in short, yes
15:45:44 <regXboi> DinaBelova: I thought I saw somebody suggest that earlier in the thread (or get most of the way there)
15:45:47 <Kristian_> just heard of it so will read the docs shortly :)
15:45:51 <kun_huang> haha
15:46:02 <DinaBelova> Kristian_ - currently it's only about networking
15:46:17 <DinaBelova> and no need to make it wider due to the current Rally changes
15:46:25 <DinaBelova> regXboi - can you find a thred please?
15:46:31 <Kristian_> so shaker only touches/focuses on neutron component?
15:46:42 <regXboi> DinaBelova: it was in this meeting... let me go back and look
15:46:51 <DinaBelova> regXboi, oh, I missedit :)
15:47:06 <DinaBelova> Kristian_ - yes, that's true :)
15:48:03 <regXboi> It was patrykw_ that got most of the way there: http://eavesdrop.openstack.org/irclogs/%23openstack-performance/%23openstack-performance.2015-11-10.log.html#t2015-11-10T15:29:35
15:48:13 <DinaBelova> a-ha, thank you sir
15:48:20 <DinaBelova> #link http://eavesdrop.openstack.org/irclogs/%23openstack-performance/%23openstack-performance.2015-11-10.log.html#t2015-11-10T15:29:35
15:48:59 <patrykw_> yep, for data plane we are using Gatling + PKB
15:49:01 <DinaBelova> ok, so it looks like we need to enforce standard image creation/assumption
15:49:26 <DinaBelova> and then try to understand if we can compare at least somehow currently gathered data
15:49:43 <regXboi> if the image has what is needed for stressing of the various components, that makes sense
15:49:46 <patrykw_> as data source mediawiki + magentoshop
15:50:09 <DinaBelova> patrykw_ - interesting, thank you sir
15:50:22 <DinaBelova> patrykw_ - do you have this data somewhere published?
15:50:26 <augiemena3> Badari:  yes, https://github.com/ibmcb/cbtool
15:50:36 <DinaBelova> patrykw_ - or is it private info?
15:51:03 <patrykw_> DinaBelova: not yet, currently gathering multinoda data with top bin CPUs
15:51:11 <patrykw_> "multinode"
15:51:36 <DinaBelova> patrykw_ - will you be able to share the results once they'll be collected? I mean, it's interesting experience
15:51:48 <DinaBelova> and personally I would love to take a look on the results
15:52:43 <kun_huang> okay guys, this meeting is close to end; I'm working on my prototype, scalpels, which is debugfs system with systemtap/ftrace support, if you're interested in this, welcome to #openstack-scalpels
15:52:53 <DinaBelova> #topic Open Discussion
15:52:57 <patrykw_> DinaBelova: probably yes, and probably the automated scripts
15:53:04 <DinaBelova> patrykw_ - thank you sir
15:53:17 <DinaBelova> kun_huang - thanks, will join
15:53:59 <DinaBelova> I would ask everyone here who uses rally to prepare and send to this channel scenarios you use to run
15:54:10 <regXboi> where did the hour go :)
15:54:19 <DinaBelova> regXboi :D
15:54:38 <DinaBelova> #action everyone who uses Rally - please share the scenarios you're running
15:54:52 <patrykw_> DinaBelova: +1 good point
15:54:52 <DinaBelova> I think we can try to unify the scenarios list
15:55:10 <regXboi> DinaBelova: can we start an etherpad for that?
15:55:14 <DinaBelova> I think we can use etherpads or whatever for this purpose
15:55:17 <kun_huang> using rally to run complex load to openstack
15:55:23 <DinaBelova> https://etherpad.openstack.org/p/rally_scenarios_list
15:55:34 <regXboi> DinaBelova: thx
15:55:48 <Kristian_> @DinaBelova - if you need inputs on what can be put into "out-of-the-box" scenarios for testing i can do that collectively here @AT^T
15:55:54 <DinaBelova> I've quickly copy-passed the scenarios names, but I'll add their configs somewhere too
15:56:12 <DinaBelova> Kristian_ - this will be perfect!!!!
15:56:19 <Kristian_> for Rally testing in particular
15:56:25 <DinaBelova> Kristian_ - thank you sir
15:56:34 <Kristian_> yw :)
15:57:01 <DinaBelova> #action Kristian_  - have internal att session to collect feedback on what can be added to out-of-box- rally
15:57:15 <DinaBelova> ok, so it looks like we're running out of time
15:57:22 <DinaBelova> any specific points to cover?
15:57:41 <regXboi> other than another channel or two to watch? no :)
15:57:48 <DinaBelova> ;D
15:57:53 <Kristian_> @DinaBelova and group - will do
15:57:55 <kun_huang> DinaBelova: we need a focus next time
15:58:05 <regXboi> yep, time to post an agenda :)
15:58:05 <DinaBelova> kun_huang - that's true
15:58:12 <kun_huang> on benchmark topic, or tracing, or people's need
15:58:16 <Kristian_> agenda would be nice, agreed
15:58:33 <kun_huang> it helps us to find more details
15:58:42 <DinaBelova> please feel free to clear https://wiki.openstack.org/wiki/Meetings/Performance#Agenda_for_next_meeting and fill with your items
15:58:45 <DinaBelova> :)
15:59:02 <Kristian_> context would be good too.  performance testing means different things to different engineers/techies :P
15:59:14 <DinaBelova> ok, thanks everyone, and let's have a bit clearer meeting next tim :)
15:59:19 <DinaBelova> #endmeeting