| openstackgerrit | Dina Belova proposed openstack/performance-docs: Add new summit recaps part of documentation https://review.openstack.org/393538 | 00:05 |
|---|---|---|
| *** harlowja has joined #openstack-performance | 00:11 | |
| *** dimtruck is now known as zz_dimtruck | 00:20 | |
| *** victorhe has joined #openstack-performance | 00:33 | |
| *** lezbar has quit IRC | 00:44 | |
| *** victorhe has quit IRC | 00:56 | |
| *** markvoelker has quit IRC | 00:57 | |
| *** victorhe has joined #openstack-performance | 00:57 | |
| *** victorhe has quit IRC | 01:00 | |
| *** victorhe has joined #openstack-performance | 01:01 | |
| *** jkilpatr has quit IRC | 01:07 | |
| *** zz_dimtruck is now known as dimtruck | 01:08 | |
| *** lezbar has joined #openstack-performance | 01:37 | |
| *** markvoelker has joined #openstack-performance | 01:57 | |
| *** victorhe has quit IRC | 02:02 | |
| *** markvoelker has quit IRC | 02:02 | |
| *** victorhe has joined #openstack-performance | 02:13 | |
| *** victorhe has quit IRC | 02:28 | |
| *** dimtruck is now known as zz_dimtruck | 02:41 | |
| *** zz_dimtruck is now known as dimtruck | 03:07 | |
| *** dimtruck is now known as zz_dimtruck | 03:25 | |
| *** zz_dimtruck is now known as dimtruck | 03:48 | |
| *** markvoelker has joined #openstack-performance | 03:58 | |
| *** markvoelker has quit IRC | 04:03 | |
| *** dimtruck is now known as zz_dimtruck | 04:19 | |
| *** zz_dimtruck is now known as dimtruck | 04:35 | |
| *** anilvenkata has joined #openstack-performance | 05:48 | |
| *** markvoelker has joined #openstack-performance | 05:59 | |
| *** markvoelker has quit IRC | 06:04 | |
| anilvenkata | DinaBelova, Hi | 06:43 |
| *** markvoelker has joined #openstack-performance | 07:00 | |
| *** markvoelker has quit IRC | 07:04 | |
| *** pcaruana has joined #openstack-performance | 07:33 | |
| *** ad_rien_ has joined #openstack-performance | 08:11 | |
| *** xek has joined #openstack-performance | 08:38 | |
| *** anilvenkata has quit IRC | 09:31 | |
| *** ad_rien_1 has joined #openstack-performance | 09:36 | |
| *** ad_rien_ has quit IRC | 09:40 | |
| *** ggherdov- has quit IRC | 10:42 | |
| *** kun_huang has quit IRC | 10:43 | |
| *** kun_huang has joined #openstack-performance | 10:46 | |
| *** openstackgerrit has quit IRC | 10:47 | |
| *** openstackgerrit has joined #openstack-performance | 10:48 | |
| *** anilvenkata has joined #openstack-performance | 10:51 | |
| anilvenkata | DinaBelova, ping | 10:51 |
| *** ad_rien_1 has quit IRC | 10:57 | |
| *** ad_rien_ has joined #openstack-performance | 10:58 | |
| *** ad_rien_ has quit IRC | 10:58 | |
| *** ggherdov- has joined #openstack-performance | 10:58 | |
| *** rfolco has joined #openstack-performance | 11:05 | |
| *** jkilpatr has joined #openstack-performance | 11:07 | |
| *** klindgren has joined #openstack-performance | 11:10 | |
| *** bvanhav_ has joined #openstack-performance | 11:18 | |
| *** klindgren has quit IRC | 11:42 | |
| *** ad_rien_ has joined #openstack-performance | 11:59 | |
| *** ad_rien_ has quit IRC | 11:59 | |
| *** xek has quit IRC | 12:17 | |
| *** markvoelker has joined #openstack-performance | 12:23 | |
| *** bvanhav_ has quit IRC | 12:54 | |
| *** ad_rien_ has joined #openstack-performance | 13:02 | |
| *** ad_rien_ has quit IRC | 13:02 | |
| *** ad_rien_ has joined #openstack-performance | 13:02 | |
| *** ad_rien_ has quit IRC | 13:02 | |
| *** ad_rien_ has joined #openstack-performance | 13:02 | |
| *** ad_rien_ has quit IRC | 13:03 | |
| *** dimtruck is now known as zz_dimtruck | 13:04 | |
| *** ad_rien_ has joined #openstack-performance | 13:04 | |
| *** ad_rien_ has quit IRC | 13:04 | |
| *** bvanhav_ has joined #openstack-performance | 13:11 | |
| *** zz_dimtruck is now known as dimtruck | 13:16 | |
| *** ad_rien_ has joined #openstack-performance | 13:21 | |
| *** victorhe has joined #openstack-performance | 14:21 | |
| *** hieulq_ has joined #openstack-performance | 14:23 | |
| DinaBelova | anilvenkata o/ | 14:37 |
| anilvenkata | DinaBelova, wanted to check if OSProfiler supports nested functional call tracing | 14:38 |
| anilvenkata | DinaBelova, for example, "neturon port-create" does it record all nested functions(neutron private functions) which are called in plugin.py? | 14:38 |
| DinaBelova | anilvenkata you can add osprofiler points to all functions for instance :) by default osprofiler is logging all api, rpc, DB calls | 14:40 |
| DinaBelova | but if you'd like to debug something specific and trace everything - you'll need to add @trace decorators to interesed funstions or @trace_cls to needed class | 14:40 |
| anilvenkata | DinaBelova, how to enable it to log all private functions? | 14:40 |
| anilvenkata | DinaBelova, do you mean I have to change neutron code i.e add @trace decorators in neutron modules? | 14:41 |
| DinaBelova | anilvenkata yes, because nobody from neutron or whatever other team is wishing to wrap all functions by default :) by default everyone agreed on tracing API, RPC, DB, but not everything | 14:42 |
| DinaBelova | anilvenkata you can add required decorator (either @trace for just function or @trace_cls for the whole class) with trace_private=True passed as an agrument | 14:43 |
| anilvenkata | DinaBelova, adding decorator means changing module, right? | 14:44 |
| DinaBelova | anilvenkata yes, it means | 14:45 |
| anilvenkata | DinaBelova, And it can't be done with changing only neutron config options? | 14:45 |
| DinaBelova | anilvenkata nope, because if everything will be wrapped (even if not working by default) in case of ANY errors in neutron or whatever other project all trace will contain 95% of osprofiler tracing logs | 14:46 |
| DinaBelova | nobody is wishing to see them | 14:46 |
| anilvenkata | ok | 14:47 |
| anilvenkata | DinaBelova, which config file to modify if I want to trace rpc calls and not api? | 14:49 |
| DinaBelova | so of course core teams of openstack project have an agreement on some minimum default amount of enabled (and turned off by default) tracing points (api, rpc, db) and eveything else should be traced manually in case if there is a specific research | 14:49 |
| dalvarezs | ups looks like we're having a simultaneuous conversation sorry DinaBelova :) | 14:49 |
| DinaBelova | dalvarezs it's ok | 14:49 |
| dalvarezs | that makes perfect sense... maybe wrapping around certain points when further profiling info works if depending on certain configuration flag | 14:50 |
| DinaBelova | anilvenkata right now you can turn on separately only DB (trace_sqlalchemy=true in projects configs files) everything else is turned on automatically. the reason is that this will add a complexity to the trace and not projects have, for instance, RPC at all :) keystone does not have for instance | 14:51 |
| *** hieulq_ has quit IRC | 14:51 | |
| DinaBelova | also I can barely imagine a use case when you'd like to see only RPC without API callw between the projects. E.g. you're tracing server create and you're seeing only nova and neutron RPC calls without seeing what API calls were done? | 14:52 |
| DinaBelova | also if you're tracing one project and one request you'll see only one parent API call... | 14:52 |
| anilvenkata | DinaBelova, yes | 14:53 |
| anilvenkata | DinaBelova, In neutron case, any api, like port-create, network-create doesn't depend on agents or other openstack components | 14:53 |
| anilvenkata | DinaBelova, for example port-create creates entries in DB and notifies agent | 14:54 |
| anilvenkata | DinaBelova, if we trace port-create and check which functions are time consuming | 14:55 |
| DinaBelova | anilvenkata that's true, moreover neutron has an issue that agents running on the compute nodes have no idea about API calls that came to the neutron API - that means that you won't see it in the tracing of API call | 14:55 |
| DinaBelova | those agents are just trying to periodically check state on the compute node itself | 14:56 |
| anilvenkata | DinaBelova, yes | 14:56 |
| DinaBelova | with no connections to what was asked from the neutron server | 14:56 |
| DinaBelova | and that's was something we discussed with neutron team for a while ago | 14:56 |
| DinaBelova | and no obvious solution was found | 14:57 |
| anilvenkata | DinaBelova, so for neutron API performance measurement and checking for bottleneck | 14:57 |
| anilvenkata | DinaBelova, do u thing existing OSProfiler will be sufficient? | 14:57 |
| DinaBelova | for API performance you can use osprofiler as is - it should work ok, moreover you'll see nested calls to all neutron processes and DB calls timings | 14:58 |
| DinaBelova | that means you'll see the most influencing parts | 14:59 |
| DinaBelova | or at least would be able to locate a bottleneck | 14:59 |
| anilvenkata | DinaBelova, here we have only one process i.e neutron-server | 14:59 |
| dalvarezs | DinaBelova, and if we need more tracing info, we still have the ability to wrap around the points we really want to explore further, right? | 15:00 |
| DinaBelova | dalvarezs yes | 15:00 |
| anilvenkata | DinaBelova, but OSProfiler wont trace all the nested called functions by default | 15:01 |
| anilvenkata | dalvarezs, ^ | 15:01 |
| anilvenkata | DinaBelova, we have to find and enable them through decorators, right? | 15:01 |
| DinaBelova | anilvenkata api call itself is transfering a call further - to DB for instance... this is something that takes time... | 15:01 |
| DinaBelova | anilvenkata if you want to trace all functions one by one yes.. | 15:01 |
| DinaBelova | the issue is that I don't understand what api functions to you want to trace in fact. every API request has separated function dedicated to go through this request and do further actions - e.g. go to DB and extract the data | 15:03 |
| dalvarezs | DinaBelova, and including that kind of support into osprofiler is beyond the scope and probably rejected by ores | 15:03 |
| DinaBelova | usually this is something that takes time | 15:03 |
| dalvarezs | cores* | 15:03 |
| DinaBelova | dalvarezs osprofiler is just a tracing library, you can trace whatever you want :) the issues is that if you'd like to see all python functions traced one by one this is not going to be accepted by cores - as this will make all requests and all tracebacks too messy | 15:04 |
| anilvenkata | DinaBelova, can u kindly show which changes in https://review.openstack.org/#/c/273951/ done for enabling API trace | 15:04 |
| DinaBelova | so if you want to wrap all python functions by @trace decorator this can be done, but only in the testing environment | 15:04 |
| anilvenkata | DinaBelova, agrre with you | 15:04 |
| anilvenkata | agree | 15:05 |
| DinaBelova | anilvenkata sure | 15:05 |
| dalvarezs | DinaBelova, yes | 15:05 |
| DinaBelova | basically https://review.openstack.org/#/c/273951/28/etc/api-paste.ini to include osprofiler to the pipeline, https://review.openstack.org/#/c/273951/28/neutron/common/profiler.py to add profiler helper, https://review.openstack.org/#/c/273951/28/neutron/pecan_wsgi/app.py to allow needed headers, https://review.openstack.org/#/c/273951/28/neutron/service.py to enable API point | 15:07 |
| DinaBelova | all other places are either adding tracing to some of the RPC managers or DB or agent drivers (for neutron case we agreed those need to be traced as well) | 15:08 |
| anilvenkata | DinaBelova, thanks a lot Dina. Now I understood it better | 15:09 |
| DinaBelova | anilvenkata u'r welcome :) | 15:09 |
| dalvarezs | DinaBelova, thanks indeed | 15:11 |
| *** bvanhav_ has quit IRC | 15:27 | |
| *** ad_rien_1 has joined #openstack-performance | 15:34 | |
| *** ad_rien_ has quit IRC | 15:38 | |
| *** pcaruana has quit IRC | 15:58 | |
| *** anilvenkata has quit IRC | 15:59 | |
| openstackgerrit | Dina Belova proposed openstack/performance-docs: Add new summit recaps part of documentation https://review.openstack.org/393538 | 17:25 |
| *** jkilpatr has quit IRC | 17:42 | |
| *** bvanhav_ has joined #openstack-performance | 17:43 | |
| *** jkilpatr has joined #openstack-performance | 17:43 | |
| *** harlowja has quit IRC | 18:00 | |
| *** harlowja has joined #openstack-performance | 18:03 | |
| *** harlowja has quit IRC | 18:18 | |
| *** johnthetubaguy has quit IRC | 18:28 | |
| *** johnthetubaguy_ has joined #openstack-performance | 18:28 | |
| *** johnthetubaguy_ is now known as johnthetubaguy | 18:30 | |
| *** harlowja has joined #openstack-performance | 18:30 | |
| *** anilvenkata has joined #openstack-performance | 18:35 | |
| *** jkilpatr has quit IRC | 18:50 | |
| *** ad_rien_1 has quit IRC | 18:55 | |
| *** victorhe_ has joined #openstack-performance | 19:02 | |
| *** victorhe_ has quit IRC | 19:03 | |
| *** victorhe_ has joined #openstack-performance | 19:04 | |
| *** jkilpatr has joined #openstack-performance | 19:04 | |
| *** victorhe has quit IRC | 19:05 | |
| *** ad_rien_ has joined #openstack-performance | 19:20 | |
| *** ad_rien_ has quit IRC | 19:22 | |
| openstackgerrit | Dina Belova proposed openstack/performance-docs: Add Newton summit recap https://review.openstack.org/393918 | 19:28 |
| *** bvanhav_ has quit IRC | 19:39 | |
| *** rfolco has quit IRC | 20:00 | |
| *** victorhe_ has quit IRC | 20:02 | |
| *** victorhe has joined #openstack-performance | 20:17 | |
| *** jkilpatr has quit IRC | 20:25 | |
| *** victorhe has quit IRC | 20:33 | |
| *** jkilpatr has joined #openstack-performance | 20:46 | |
| *** victorhe has joined #openstack-performance | 20:48 | |
| *** ad_rien_ has joined #openstack-performance | 20:57 | |
| *** victorhe has quit IRC | 20:58 | |
| *** victorhe has joined #openstack-performance | 20:58 | |
| *** anilvenkata has quit IRC | 21:32 | |
| *** victorhe has quit IRC | 22:15 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!