16:00:00 <DinaBelova> #startmeeting Performance Team
16:00:01 <openstack> Meeting started Tue Jan 19 16:00:00 2016 UTC and is due to finish in 60 minutes.  The chair is DinaBelova. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:05 <openstack> The meeting name has been set to 'performance_team'
16:00:10 <DinaBelova> o/
16:00:10 <regXboi> o/
16:00:55 <DinaBelova> I wish so much that kun_huang will be here today :) wanted to place him and listomin in one room, and forget the place where the room key is :)
16:01:20 <DinaBelova> todays agenda: https://wiki.openstack.org/wiki/Meetings/Performance#Agenda_for_next_meeting
16:01:57 <DinaBelova> let's wait for a few more moments :) probably harlowja, kun_huang and boris-42 will appear :)
16:02:48 <DinaBelova> #topic Action Items
16:03:01 <DinaBelova> ok, so let's go through the action items first
16:03:05 <DinaBelova> both on me in fact
16:03:20 <DinaBelova> the first one is about pinging kun_huang and the huawei lab doc status
16:03:43 <DinaBelova> in fact he has updated the google doc, but did not publish the info to the performance-docs yet
16:04:07 <DinaBelova> I'll go quickly through what I can see about huawei lab status during the needed section
16:04:27 <DinaBelova> and the second one is about publishing info about mirantis-intel lab
16:04:35 <boris-42> hi DinaBelova
16:04:39 <DinaBelova> boris-42 o/
16:04:54 <DinaBelova> sadly the lab was not prepared ok during the previous week
16:05:04 <DinaBelova> so I'm going to do it this week
16:05:05 <rvasilets> o/
16:05:11 <DinaBelova> AugieMena - probably you'll be interested
16:05:15 <DinaBelova> rvasilets o/
16:05:25 <DinaBelova> AugieMena - Intel-Mirantis lab is now 250 nodes
16:05:45 <DinaBelova> still some networking issues, but I believe we'll be able to start running test plans agains it very soon
16:05:58 <DinaBelova> at least it's more or less available now, etc.
16:06:22 <DinaBelova> #action DinaBelova share details about Mirantis-Intel lab (mailing list + performance-docs)
16:06:31 <DinaBelova> so I'll keep action item on me for this
16:06:39 <DinaBelova> #topic Test plans statuses
16:06:55 <DinaBelova> test plans template was merged to the performance-docs
16:07:12 <DinaBelova> #link https://review.openstack.org/#/c/268236/
16:07:29 <DinaBelova> and the already merged stuff was modified to fit it
16:07:47 <DinaBelova> also we have DB test plan almost merged
16:07:50 <DinaBelova> #link https://review.openstack.org/#/c/263523/
16:07:59 <DinaBelova> I love how it looks now, so I believe we're close :)
16:08:06 <DinaBelova> thanks Jeff :)
16:08:29 <DinaBelova> so I believe that's everything new about the test plans for now
16:08:47 <DinaBelova> #topic Huawei lab status
16:08:52 * regXboi catches up on scrollback
16:08:58 <DinaBelova> regXboi :)
16:09:21 <DinaBelova> about the Huawei lab I can share something very high level without kun_huang who has all info
16:09:32 <AugieMena> Dina - sorry for the late response.  Yes, interested!
16:09:39 <DinaBelova> AugieMena no problem :)
16:09:51 <DinaBelova> AugieMena I'll try to publish info asap
16:10:03 <DinaBelova> really planned to do it before this meeting, but lab was not ok...
16:10:05 <DinaBelova> :)
16:10:10 <kun_huang> Duan and dragonflow team are starting perf testing recently
16:10:17 <DinaBelova> kun_huang yay!
16:10:23 <DinaBelova> nice to see you sir!
16:10:34 <kun_huang> nice to see you ;)
16:10:35 <DinaBelova> kun_huang - hm... I did not see the test plan from them in fact :)
16:10:51 <DinaBelova> kun_huang are they planning to publish it to the performance docs?
16:10:59 <AugieMena> Dina - thx, from our side we need to review the plans and figure out how we can help
16:11:00 <kun_huang> because dragonflow team just fly back
16:11:15 <DinaBelova> AugieMena ack!
16:11:16 <kun_huang> DinaBelova: I could ask them share something
16:11:17 <DinaBelova> kun_huang ok
16:11:20 <kun_huang> it is not diffcult
16:11:26 <DinaBelova> kun_huang I believe that's a good idea
16:11:41 <DinaBelova> because it'll be good to have all stuff like that in on place
16:11:53 <DinaBelova> #action kun_huang ask DragonFly team to share the test plan
16:11:56 <DinaBelova> ok, cool
16:12:09 <kun_huang> yep, we all need make dragonflow more public
16:12:18 <DinaBelova> kun_huang ;)
16:12:21 <DinaBelova> also I can see that the lab has grown during the holidays
16:12:24 <DinaBelova> 46 nodes now
16:12:35 <DinaBelova> very cool!
16:12:41 <kun_huang> there are 40 on the way to Hangzhou
16:12:58 <DinaBelova> kun_huang - ok, and they will be available after Feb 15th?
16:13:26 <DinaBelova> kun_huang - can you please publish the lab doc to the perofrmance docs?
16:13:36 <DinaBelova> kun_huang - I'll do the same for Intel-Mirantis lab
16:13:46 <kun_huang> okay, no problem
16:13:48 <kun_huang> good idea
16:13:51 <DinaBelova> cool
16:14:04 <DinaBelova> #action kun_huang publish info about Huawei lab to performance-docs
16:14:08 <DinaBelova> very nice
16:14:17 <DinaBelova> kun_huang - I wanted to present you listomin
16:14:33 <listomin> kun_huang Hello! :)
16:14:36 <DinaBelova> listomin is one of Mirantis performance team engineers
16:14:48 <kun_huang> listomin: nice to meet you
16:14:55 <DinaBelova> and he is very interested in running the provisioning test plan against your lab
16:15:02 <DinaBelova> when it'll be possible :)
16:15:24 <kun_huang> it will be a nice cooperation
16:15:32 <DinaBelova> so 6 nodes are under the dragonflow testing, and 40 are coming
16:15:42 <DinaBelova> I dunnot when this will be available to be done in fact
16:16:07 <DinaBelova> but I want you guys to coordinate on this topic :)
16:16:20 <kun_huang> one or two is ready I think
16:16:21 <DinaBelova> so just keep in touch
16:16:40 <kun_huang> yep
16:16:43 <DinaBelova> kun_huang - may you please ensure something is available and ping listomin if yes?
16:16:57 <listomin> kun_huang I'll look at Huawei lab docs and probably ask you to configure something. Also I'll try to prepare automations or qcow image to deploy it on your env
16:17:23 <DinaBelova> listomin ok :) so we're waiting public Huawei lab doc
16:18:18 <DinaBelova> ok, cool
16:18:40 <DinaBelova> so let's move forward
16:18:47 <DinaBelova> #topic OSProfiler weekly update
16:19:03 <DinaBelova> the most interesting news are about Nova change :)
16:19:17 <DinaBelova> #link https://review.openstack.org/#/c/254703/
16:19:30 <DinaBelova> during the last week it passed several stages
16:19:39 <DinaBelova> 1 - tests began to work ok
16:19:51 <DinaBelova> 2 - change was modified to gather more info
16:20:07 <DinaBelova> 3 - something else was merged to nova and shown that osprofiler has a bug
16:20:08 <DinaBelova> :)
16:21:17 <DinaBelova> so tests right now will be failing due to the following issue: trace_cls decorator in osprofiler modifies (somehow) the static methods of the class (faving "function type") to not static - having "inctancemethod" type
16:21:32 <DinaBelova> that leads to wrong number of args coming to the methods like that
16:21:44 <DinaBelova> I'm writing the fix at the moment
16:21:51 * DinaBelova hopes to publish it today
16:22:10 <DinaBelova> boris-42 - I'll ping you once the change will be ready
16:22:23 <DinaBelova> please take a look on it when this will happen
16:23:05 <DinaBelova> surprising moment is that special tracing meta class does not have this behaviour, that's really surprising for me
16:23:33 <DinaBelova> also I'm going to grab neutron flag from regXboi :)
16:23:50 <DinaBelova> and start working on osprofiler to neutron change
16:24:02 <DinaBelova> #link https://bugs.launchpad.net/neutron/+bug/1335640
16:24:04 <openstack> Launchpad bug 1335640 in neutron "[RFE] Neutron support for OSprofiler" [Wishlist,Triaged] - Assigned to Dina Belova (dbelova)
16:24:17 <regXboi> yes - I'll pitch in and help, but I'm running around like a one-armed paper hanger
16:24:29 <DinaBelova> regXboi - no worries
16:24:43 <DinaBelova> regXboi, I completely understand your situation
16:25:26 <DinaBelova> I really hope you'll be able to deal with all your current tasks, I don't like the "one-armed paper hanger" metaphor :)
16:25:35 <regXboi> neither do I
16:25:43 <DinaBelova> ack, cool
16:25:55 <DinaBelova> one more interesting change is from dims
16:26:06 <DinaBelova> #link https://review.openstack.org/#/c/269059/
16:26:33 <DinaBelova> that will allow not to copy paste all the stuff to various projects and just use the osprofiler here
16:26:55 <DinaBelova> I'll update it to meet recent comments I got in nova change and I believe we can merged it
16:27:14 <DinaBelova> I guess that's everything interesting about the osprofiler
16:27:22 <DinaBelova> boris-42 - anything to add?
16:27:33 <boris-42> DinaBelova: so
16:27:41 <boris-42> DinaBelova: I am working on devstack-osprofiler integration
16:27:54 <boris-42> DinaBelova: so we should get soon functional job
16:27:59 <DinaBelova> yay!
16:28:09 <boris-42> DinaBelova: and it should be very easy to set up specific version in devstack
16:28:19 <boris-42> #link https://review.openstack.org/#/c/267888/
16:28:29 <boris-42> #link https://review.openstack.org/#/c/267889/
16:28:37 <boris-42> I'll create today branch
16:28:41 <boris-42> patch*
16:28:44 <boris-42> to infra
16:28:50 <DinaBelova> boris-42, very cool
16:29:02 <boris-42> DinaBelova: plus I was debugging yestarday a lot osprofiler
16:29:15 <boris-42> seems like Nova started using greenthreads a lot
16:29:20 <DinaBelova> boris-42 - yes, and I want to say you big THANK YOU
16:29:24 <boris-42> and that is why we are losing our messages
16:29:30 <boris-42> DinaBelova: it works for you as well?
16:29:34 <DinaBelova> the latest nova change looks much better
16:29:44 <DinaBelova> after some fixing in trace_cls yes
16:29:50 <boris-42> DinaBelova: so the only one thing that we need to fix now  is keystone?
16:30:08 <DinaBelova> boris-42 http://dinabelova.github.io/fixed_nova_and_profiler_trace.html
16:30:16 <DinaBelova> boris-42 I believe yes
16:30:28 <boris-42> DinaBelova: so this is bad trace
16:30:53 <boris-42> DinaBelova: are you sure that you put there proper trace?
16:31:10 <boris-42> DinaBelova: there is only one compute point and there should be more
16:31:30 <DinaBelova> boris-42 one?
16:31:39 <DinaBelova> it looks like I see more
16:31:51 <boris-42> DinaBelova: yep only one from nova-compute
16:32:06 <DinaBelova> a-ha, you mean rpc call?
16:32:06 <boris-42> DinaBelova: that's the bug that patching spawn method is addressing
16:32:12 <boris-42> DinaBelova: not rpc call
16:32:15 <DinaBelova> yep, I used your fix
16:32:17 <DinaBelova> one moment
16:32:21 <DinaBelova> lemme check
16:32:21 <boris-42> DinaBelova: I mean call from nova-compute
16:32:31 <boris-42> DinaBelova: there should be zillion calls from nova-compute
16:32:37 <boris-42> DinaBelova: otherwise fix doesn't work
16:32:45 <boris-42> DinaBelova: I believe that this is missing in your patch as ewll https://github.com/boris-42/nova/commit/a3c002e9cb32e98952e18ea7ec447b0e3ae46d63
16:32:57 <DinaBelova> nope, I added it
16:33:10 <DinaBelova> therefore you can see compute_api points
16:34:12 <boris-42> DinaBelova: in any case working profiler means a lot of points from nova-compute
16:34:18 <DinaBelova> https://review.openstack.org/#/c/254703/21/nova/compute/api.py here is nova_compute
16:34:21 <boris-42> DinaBelova: I will test your patch today
16:34:35 <DinaBelova> boris-42 it looks like I really removed for mistake the https://github.com/boris-42/nova/commit/a3c002e9cb32e98952e18ea7ec447b0e3ae46d63
16:34:39 <DinaBelova> interesting
16:34:45 <DinaBelova> was sure I applyed it
16:34:46 <boris-42> DinaBelova: actually that code is executed on osapi-compute
16:34:55 <boris-42> DinaBelova: and node-compute and that name should be fixed
16:35:01 <DinaBelova> I'll retest it
16:35:19 <boris-42> DinaBelova: I am going to renaming of points
16:35:27 <boris-42> DinaBelova: otherwise it becomes a bit unclear
16:35:31 <DinaBelova> boris-42 ack
16:35:35 <DinaBelova> thank you sir
16:35:53 <DinaBelova> #action DinaBelova add https://github.com/boris-42/nova/commit/a3c002e9cb32e98952e18ea7ec447b0e3ae46d63 and retest the nova racing with it
16:35:56 <boris-42> DinaBelova: that's all from my sie
16:35:57 <boris-42> side
16:36:01 <DinaBelova> boris-42 thank you sir
16:36:15 <DinaBelova> ok, it looks like the complete version of osprofiler news
16:36:20 <DinaBelova> good
16:36:26 <DinaBelova> #topic Open Discussion
16:36:36 <DinaBelova> I don't have much to add here
16:36:45 <DinaBelova> the only moment
16:36:47 <DinaBelova> #link https://wiki.openstack.org/wiki/Performance_Team#Mitaka_Roadmap
16:37:06 <DinaBelova> I've added just for the clarification Mitaka roadmap
16:37:26 <DinaBelova> I probably missed something, like one more osprofielr release
16:37:36 <DinaBelova> but more or less it's full now
16:37:58 <DinaBelova> boris-42, kun_huang - please take a look and update in case of any comments
16:38:32 <DinaBelova> and that's all from my side
16:38:59 <DinaBelova> do you guys have something to add?
16:39:46 <rvasilets> no
16:39:53 <DinaBelova> boris-42 ooops https://review.openstack.org/#/c/254703/21/nova/utils.py
16:40:00 <DinaBelova> I've added your fix already
16:40:08 <DinaBelova> I'll recheck it on the env
16:40:27 <DinaBelova> and if it's already added, that means something went wrong
16:40:52 <DinaBelova> so I'll recheck anyway
16:41:05 <DinaBelova> ok, so it looks like no more comments today
16:41:22 <DinaBelova> thanks everyone for attending and have a nice day/evening
16:41:26 <DinaBelova> #endmeeting