16:00:14 <DinaBelova> #startmeeting Performance Team
16:00:14 <openstack> Meeting started Tue Mar  1 16:00:14 2016 UTC and is due to finish in 60 minutes.  The chair is DinaBelova. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:17 <openstack> The meeting name has been set to 'performance_team'
16:00:24 <DinaBelova> o/
16:00:33 <DinaBelova> folks, let's wait for a few minutes
16:00:53 <DinaBelova> to see if ilyashakhat will be able to join
16:01:39 <DinaBelova> ilyashakhat rohanion o/
16:01:57 <ilyashakhat> hi!
16:02:01 <rohanion> hi everyone
16:02:21 <DinaBelova> ok, so let's have some minutes to see if somebody else will join
16:02:41 <DinaBelova> we have standard agenda for today
16:02:43 <DinaBelova> #link https://wiki.openstack.org/wiki/Meetings/Performance#Agenda_for_next_meeting
16:04:03 <DinaBelova> #topic Action Items
16:04:13 <DinaBelova> ok, so going through previous week action items
16:04:20 <DinaBelova> #link http://eavesdrop.openstack.org/meetings/performance_team/2016/performance_team.2016-02-23-16.00.html
16:04:54 <DinaBelova> so in fact gokrokve answered that email thread I as going to
16:05:20 <DinaBelova> in short: we're going to publish DVR vs non-DVR openstack topology Rally results to performance-docs
16:05:48 <gokrokve> Yep. I plan to do RST doc this week
16:06:03 <DinaBelova> right now we have groomed results for MOS 7.0 (kilo), and non-groomed for MOS 8.0 (liberty)
16:06:05 <gokrokve> I never did this before so it will take a little bit longer
16:06:21 <DinaBelova> gokrokve did you talk to our QA team about liberty results?
16:06:33 <gokrokve> Nope. I will do this today
16:06:51 <DinaBelova> gokrokve ack, I just wonder if they started work on grooming already
16:07:11 <DinaBelova> I've seen runs, but have no idea if they somehow made them structured
16:07:20 <DinaBelova> ok, cool
16:07:52 <DinaBelova> it looks like kun_huang was not able to go through his action item about https://review.openstack.org/#/c/280843/1/doc/source/labs/huawei_lab.rst
16:08:04 <DinaBelova> so let's keep it open till next meeting
16:08:13 <DinaBelova> #action kun_huang update https://review.openstack.org/#/c/280843/1/doc/source/labs/huawei_lab.rst
16:09:05 <DinaBelova> and about SpamapS testing - last time he joined us at the end of meeting and shared that he was distracted a bit from that task
16:09:30 <DinaBelova> SpamapS - btw, were you able to continue your work on that nova testing we discussed?
16:10:47 <DinaBelova> ok, it looks like this time slot is not comfortable for SpamapS at all :) probably I'll need to have two times for this meeting - maximum PST friendly and maximum Europe/China friendly
16:11:08 <DinaBelova> gokrokve, ilyashakhat, rohanion - what do you think about it btw?
16:11:23 <gokrokve> I support this
16:11:36 <rohanion> fine for me
16:11:40 <ilyashakhat> and I support the one for EU :)
16:11:49 <gokrokve> This time is particularly hard as this is kids drop to school time
16:12:22 <DinaBelova> #action DinaBelova propose to have to times for this meeting to openstack-dev - US and Europe/China friendly and switch them on weekly basis
16:12:26 <DinaBelova> ok, cool
16:12:38 <DinaBelova> so it looks like we went through all our action item
16:12:42 <DinaBelova> items*
16:12:49 <DinaBelova> Let's proceed
16:12:51 <DinaBelova> #topic Test plans statuses
16:13:07 <DinaBelova> ok, so today ilyashakhat and rohanion have joined us
16:13:42 <DinaBelova> and we may ask them to share the status of DB / MQ testing we at Mirantis currently are working on
16:13:50 <DinaBelova> ilyashakhat - will you start?
16:15:10 <ilyashakhat> we've updated MQ test plan and going to use OMsimulator as primary tool
16:16:14 <DinaBelova> ilyashakhat afaik you are using your new tool performa for this testing
16:16:23 <ilyashakhat> DinaBelova: yep
16:16:28 <DinaBelova> were you able to add add public docs to share?
16:16:36 <ilyashakhat> a moment
16:18:18 <ilyashakhat> the new tool is called Performa and it is targeted to simplify test execution and report generation
16:18:39 <ilyashakhat> some very-very short words are published at http://performa.readthedocs.org/en/latest/
16:18:57 <DinaBelova> #link http://performa.readthedocs.org/en/latest/
16:19:21 <DinaBelova> ilyashakhat thanks!
16:19:24 <ilyashakhat> the repo with the code is still in my private github (https://github.com/shakhat/performa)
16:19:45 <DinaBelova> so right now it'll support MQ and DB test plans we've shared already, afaik
16:20:01 <ilyashakhat> yep, but the work is in progress
16:20:03 <DinaBelova> ilyashakhat do you think we can move it to openstack space before next meeting?
16:20:08 <DinaBelova> ilyashakhat sure
16:20:24 <ilyashakhat> I suppose we should do this
16:20:39 <DinaBelova> #action ilyashakhat move Performa to openstack space
16:20:46 <DinaBelova> ok, cool
16:21:13 <ilyashakhat> I also have a sample for the report - http://docs-draft.openstack.org/14/286614/1/check/gate-performance-docs-docs/f8862d3//doc/build/html/test_results/mq/omsimulator/index.html
16:21:14 <DinaBelova> rohanion is working right now on testing ZeroMQ as a oslo.messaging backend for OpenStack
16:21:29 <DinaBelova> #link http://docs-draft.openstack.org/14/286614/1/check/gate-performance-docs-docs/f8862d3//doc/build/html/test_results/mq/omsimulator/index.html
16:21:38 <DinaBelova> ilyashakhat thank you sir!
16:22:06 <ilyashakhat> you;re welcome :)
16:22:19 <ilyashakhat> there gonna be more news soon!
16:22:24 <DinaBelova> cool!
16:22:31 <DinaBelova> rohanion the floor is yours :)
16:22:39 <rohanion> hm
16:23:00 <DinaBelova> rohanion on what scale are you testing it right now and with what OpenStack?
16:24:17 <rohanion> I use OpenStack Liberty deployed with Fuel 8.0 on a 200-node environment
16:25:16 <rohanion> I've finished measuring the number of used sockets and the results are not very fabulous
16:25:37 <DinaBelova> ok, any examples?
16:25:41 <rohanion> sure
16:26:16 <rohanion> here's the number of used sockets on 3 controller nodes with 197 compute nodes running as slaves
16:26:31 <rohanion> 10.20.0.27: 32057
16:26:33 <rohanion> 10.20.0.22: 31899
16:26:35 <rohanion> 10.20.0.28: 33490
16:26:46 <DinaBelova> hehehe
16:26:53 <DinaBelova> and what about computes?
16:27:01 <rohanion> currently we won't make it to 2000 node scale
16:27:01 <DinaBelova> something average for them?
16:27:09 <rohanion> a minute
16:27:13 <DinaBelova> rohanion obviously yes :D
16:27:40 <DinaBelova> btw am i right it's liberty OpenStack but oslo.messaging is mitaka?
16:27:46 <rohanion> no
16:27:53 <rohanion> oslo.messaging is liberty too
16:28:12 <DinaBelova> hm, so no modifications to zeromq driver?
16:28:12 <rohanion> it's a patched version with backported mitaka features
16:28:20 <DinaBelova> ah, I got it
16:28:38 <rohanion> the average number of sockets on compute nodes is approximately 540
16:29:02 <DinaBelova> rohanion ok, so it looks like the bottleneck is for controllers, that was expected
16:29:15 <DinaBelova> rohanion dims - any ideas on how to workaround this?
16:29:54 <rohanion> probably we'll try to use horizontal scaled proxies to minimize the number of connections
16:30:08 <rohanion> but it will significantly affect the overall performance
16:30:09 <dims> rohanion : let's invite ozamiatin here
16:30:22 <rohanion> agree
16:31:03 <DinaBelova> ok, let's wait him to join
16:31:04 <rohanion> apparently he's out of office rn
16:31:21 <DinaBelova> heh, very sad
16:31:34 <DinaBelova> ok, so let's make sure you'll find the way to workaround it
16:31:57 <rohanion> we'll brainstorm it tomorrow, yes
16:32:20 <DinaBelova> #action rohanion share ideas on how to workaround the fact that ZeroMQ eats too many sockets on OpenStack controller nodes
16:32:32 <DinaBelova> so we'll go back to this topic next time
16:32:36 <rohanion> fine
16:32:47 <DinaBelova> ok, so let's proceed
16:33:13 <DinaBelova> I'll skip Huawei lab status this time again as kun_huang is away
16:33:20 <DinaBelova> #topic OSProfiler weekly update
16:33:50 <DinaBelova> so obviously last week we were fixing bugs in profiler and I tried to keep all osprofiler related changes up to date
16:34:17 <DinaBelova> some of them (like updataing heat to use consolidated in profiler options) were merged
16:35:04 <DinaBelova> as for others, there was a stupid issue in openstack client
16:35:21 <DinaBelova> I've introduced with the integration patch
16:35:38 <DinaBelova> #link https://launchpad.net/bugs/1551160
16:35:38 <openstack> Launchpad bug 1551160 in python-openstackclient "interactive openstack console: run_subcommand() takes exactly 2 arguments (1 given)" [Undecided,Fix released] - Assigned to Dina Belova (dbelova)
16:35:44 <DinaBelova> so it was fixed today
16:35:57 <DinaBelova> I'm going to focus on keystone change in nearest future
16:36:07 <DinaBelova> #link https://review.openstack.org/#/c/103368/
16:36:21 <DinaBelova> and try to understand what's going on with failing jobs
16:36:30 <DinaBelova> in fact I have no idea yet what's going on
16:36:50 <DinaBelova> logs are vague :(
16:37:48 <DinaBelova> and I'm going to fix https://review.openstack.org/278155 and https://review.openstack.org/278157 as well, as they can still be merged to mitaka
16:38:16 <DinaBelova> as that's not a feature in fact, just switching to other options place and that's it
16:38:46 <DinaBelova> so that's all from my side for now
16:39:04 <DinaBelova> it looks like boris-42 is not here, so we may close this topic as well
16:39:28 <DinaBelova> #topic Open Discussion
16:39:43 <DinaBelova> gokrokve ilyashakhat rohanion dims - anything else to share?
16:39:59 <dims> DinaBelova : none from me
16:40:15 <gokrokve> nope
16:41:07 <rohanion> hm
16:41:32 <DinaBelova> ok, cool
16:41:37 <DinaBelova> so let's finish then
16:41:41 <DinaBelova> thanks everyone!
16:41:46 <DinaBelova> #endmeeting