14:00:59 <boris-42> #startmeeting Rally
14:01:04 <openstack> Meeting started Mon Sep 14 14:00:59 2015 UTC and is due to finish in 60 minutes.  The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:07 <openstack> The meeting name has been set to 'rally'
14:01:09 <ikhudoshyn> o/
14:01:12 <boris-42> ping kun_huang e0ne rvasilets
14:01:24 <kun_huang> hi
14:02:18 <rvasilets> \o/
14:02:22 <amaretskiy> hi
14:02:51 <redixin> -_-
14:03:08 <e0ne> рш
14:03:10 <boris-42> so let's start
14:03:10 <e0ne> hi
14:03:19 <boris-42> #topic Todays Agenda
14:03:41 <boris-42> #link  [kun] argument for using atomic actions or not
14:03:55 <kun_huang> hi
14:04:03 <oanufriev> hi
14:04:11 <kun_huang> there may be some arguments
14:04:34 <boris-42> #topic [kun] argument for using atomic actions or not
14:04:49 <kun_huang> In the patch https://review.openstack.org/#/c/220200/, the output http://paste.openstack.org/show/451164/ of task seems bad, but:
14:05:00 <kun_huang> if not using atomic actions, the output keeps meaningless from real business
14:05:06 <kun_huang> if using atomic actions, the record time represent a combine of multiple calls, how do operators know what bad happened inside it?
14:05:51 <kun_huang> do we have a rule for when to use atomic actions?
14:06:31 <boris-42> kun_huang: so that one example is very bad
14:06:46 <boris-42> kun_huang: we shouldn't call atomic actions in the loop
14:07:09 <kun_huang> got it, that's rule No.1
14:07:40 <boris-42> kun_huang: so I don't know how to check this automatically =(
14:08:17 <kun_huang> check the atomic actions in loop?
14:08:21 <stpierre> tbh, i think a lot of the problem comes from many of the neutron scenarios permitting multiple subnets per network. if they were all set to one subnet, with a separate scenario to test N subnets, then this problem would disappear
14:08:24 <rvasilets> Do we need to write like in murano in such cases? #link https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/murano/environments.py#L64
14:08:56 <stpierre> if we're going to keep N subnets per network in all of the scenarios, then we probably need to rewrite it like that
14:09:54 <stpierre> this particular scenario is still going to be strange, though, since it creates two networks, so there are always going to be multiple timers
14:10:45 <boris-42> stpierre: so I have some ideas
14:11:01 <boris-42> regarding to sub atomic_actions
14:11:12 <stpierre> yeah, if we could nest atomic actions this would look much nicer
14:11:27 <boris-42> stpierre: I found the way to do that
14:11:36 <boris-42> stpierre:  I will make spec after we cut the release 0.1.0
14:11:39 <boris-42> kun_huang: ^
14:11:55 <kun_huang> no problem
14:12:13 <kun_huang> #agreed
14:12:15 <stpierre> but in the mean time we need to either a) get rid of subnets_per_network; or b) rewrite subnet creation/modification/deletion and router plumbing to use a single atomic action per loop
14:12:46 <boris-42> stpierre: so we can create N networks as 1 atomic action
14:12:49 <stpierre> i'd prefer the former -- i think it'd keep things cleaner, and i'm not sure of the value of creating 5 subnets per network in a scenario that's supposed to test creating a router
14:12:58 <boris-42> stpierre: after that create N subnets as another atomic actions
14:13:04 <stpierre> yes, that's option (b)
14:13:07 <stpierre> and i don't like it :)
14:13:55 <boris-42> stpierre: so we may keep it as is for now
14:14:16 <boris-42> okay let's move to the next topic
14:14:34 <boris-42> until we get nested atomic actions we will try to avoid such situation in our scenarios
14:15:02 <stpierre> don't we want to fix the situation as it currently exists, though?
14:15:19 <boris-42> stpierre: ?
14:15:46 <stpierre> in the neutron scenarios, we run atomic timers in loops a lot.
14:15:49 <stpierre> should we fix that?
14:16:02 <boris-42> stpierre: so we can *
14:16:07 <boris-42> stpierre: at least for ports
14:16:10 <kun_huang> I think boris-42 prefer https://github.com/openstack/rally/blob/master/rally/plugins/openstack/scenarios/murano/environments.py#L64 now
14:16:11 <stpierre> every scenario that creates subnets or routers does it. and that's dang near all of them.
14:16:24 <stpierre> that's exactly what i'm trying to discuss
14:16:49 <stpierre> because i think it's preferable to get rid of the subnets_per_network option, whereupon this whole issue disappears
14:17:06 <boris-42> stpierre: even in such case we will have N ports
14:17:17 <kun_huang> stpierre: what's the purpose of first subnets_per_network case?
14:17:23 <kun_huang> anybody remember that?
14:17:30 <stpierre> kun_huang: i have no idea. it seems wrong
14:18:19 <stpierre> boris-42: for scenarios that should be creating multiple resources, obviously we need to move the timer outside the loop. but i'm not convinced we need as many of those as we have
14:19:10 <boris-42> stpierre: so what you are propsing to do?
14:19:27 <stpierre> when benchmarking port creation, for instance, i feel like we should do 100 trials of 1 port rather than 20 trials of 5 ports. very few of the other scenarios (outside of the neutron plugin) do multiple resource creation, and where they do (e.g., boot_and_delete_multiple_servers) it's because it's a separate API call
14:19:43 <stpierre> i think the best solution would be to get rid of subnets_per_network and ports_per_network
14:19:45 <stpierre> entirely
14:20:14 <stpierre> the anti-pattern here is the loop itself
14:20:19 <stpierre> the runner should provide the loop
14:20:28 <boris-42> stpierre: but there is people that use this scenario
14:20:45 <boris-42> stpierre: becasue they really want to create N ports per network which is ok as far as I know
14:21:03 <stpierre> if people are seriously concerned about the performance difference between creating one port per network and two, then we can keep a scenario to do that
14:21:05 <boris-42> stpierre: what is better just to combine together atomic actions
14:21:09 <stpierre> and move the timer outside the loop
14:21:20 <boris-42> stpierre: there can be difference
14:21:31 <stpierre> ok, so  then we can keep a scenario to do that
14:21:58 <boris-42> stpierre: yep we can just reogranize the code to show all creation of ports as one atomic action
14:22:11 <stpierre> for ports we can, sure
14:22:30 <stpierre> but then look at something like create_and_delete_routers, which has three loops, two of which contain (or should contain) two atomic timers each
14:22:52 <stpierre> so now we need to split that out into five loops (create subnets, create routers, add router interfaces, delete router interfaces, delete routers)?
14:23:02 <stpierre> or, we get rid of subnets_per_network and the whole problem disappears
14:23:07 <boris-42> stpierre: so we can create  all networks, all subnete, sall routers, all ports
14:24:03 <stpierre> okay. i think that's going to make the code unnecessarily complicated, and is counter to the way most other plugins work (and in general the way benchmarking should work), but if that's the way we want to go, then i'll respect that
14:24:30 <boris-42> stpierre: so let's talk with neturon guys
14:24:42 <boris-42> stpierre: I will talk today later and will see do they need this
14:24:54 <stpierre> ok
14:25:02 <boris-42> so let's move to the next topic
14:25:14 <boris-42> #topic  [rvasilets] How to use Rally API in our functional tests?
14:25:37 <kun_huang> python api?
14:26:00 <rvasilets> Typing)
14:26:51 <rvasilets> Boris suggested to use Rally API #link https://github.com/openstack/rally/blob/master/rally/api.py in functional tests
14:27:28 <rvasilets> But when I started to add api calls into #link https://github.com/openstack/rally/blob/master/tests/functional/test_task_samples.py
14:27:45 <rvasilets> I found that api not working
14:28:29 <rvasilets> I have made some debug and found that in functional tests there is no deployments table
14:28:43 <rvasilets> http://logs.openstack.org/72/198772/8/check/gate-rally-dsvm-cli/f2114f2/console.html.gz#_2015-08-07_15_13_59_686
14:29:31 <boris-42> rvasilets: api actually works
14:29:35 <rvasilets> This was printed after I have executeв db.deployment_list()
14:29:44 <boris-42> rvasilets: saying that API doens't work means saying that Rally doesn't work at all
14:29:47 <rvasilets> So I don't now what is wrong
14:30:04 <rvasilets> See here #link https://review.openstack.org/#/c/198772/8/tests/functional/test_task_samples.py
14:30:17 <rvasilets> line 65 gives an error
14:30:17 <boris-42> rvasilets: it means that you are doing something wrong*
14:30:30 <rvasilets> And I don;t know what
14:31:15 <rvasilets> Do we use db when is functional tests executed?
14:32:16 <boris-42> rvasilets: Rally doesn't wokr without DB
14:32:33 <boris-42> rvasilets: but not sure where it is created
14:32:47 <boris-42> rvasilets: inside utils.Rally() or when we are exectuing command
14:32:55 <rvasilets> After this #link https://github.com/openstack/rally/blob/master/tests/functional/utils.py#L105
14:32:56 <boris-42> rvasilets: did you talk to redixin ?
14:33:09 <rvasilets> Table deployments must to be exist
14:33:09 <boris-42> rvasilets: it's not DB
14:33:18 <rvasilets> But As we see from error
14:33:29 <rvasilets> sqlalchemy can't find it
14:34:03 <rvasilets> Las time he have suggested to me to add more debug information as I remember
14:35:12 <boris-42> rvasilets: so I don't have quick answer
14:35:23 <boris-42> rvasilets: did you takl to redixin about this?
14:35:40 <rvasilets> Last here said that "we need to research this issue"
14:35:49 <rvasilets> yes
14:36:03 <amaretskiy> boris-42: rvasilets: I think I know the solution
14:36:18 <amaretskiy> boris-42: rvasilets: I will explain this
14:36:51 <amaretskiy> boris-42: rvasilets: We simply have to configure path to sqlite file before any db query
14:37:09 <amaretskiy> I did that recently so I will consult rvasilets
14:37:12 <boris-42> amaretskiy: we do this
14:37:28 <boris-42> amaretskiy: if you take a look at the code
14:37:46 <boris-42> amaretskiy: https://github.com/openstack/rally/blob/master/tests/functional/utils.py#L97-L99
14:39:05 <amaretskiy> this code runs rally-manage and creates db
14:39:14 <boris-42> amaretskiy: yep
14:39:41 <amaretskiy> however we have issue while accessing db running rally as a lib, if db is located in non-standard place
14:39:42 <boris-42> rvasilets: oh I know the issue
14:39:53 <boris-42> rvasilets: yep ^ that is exactly the bug
14:39:54 <amaretskiy> I had investigated this recently
14:40:07 <amaretskiy> and the solution is simple
14:40:08 <boris-42> rvasilets: you don't init CONF properly in case of rally.api
14:40:37 <amaretskiy> rvasilets, boris-42: yes, the CONF is the root cause
14:41:25 <amaretskiy> rvasilets, boris-42:  here is a root cause - https://github.com/openstack/rally/blob/master/rally/common/db/api.py#L51-L52
14:41:57 <amaretskiy> we simply have to run set_defaults
14:42:05 <amaretskiy> the very first time
14:42:14 <amaretskiy> before these lines faired
14:42:53 <boris-42> amaretskiy: yep
14:43:00 <amaretskiy> this can be a dirty way
14:43:10 <amaretskiy> however that was my working case
14:43:11 <boris-42> amaretskiy: or use the rally.conf file
14:43:16 <boris-42> okay
14:43:19 <amaretskiy> i just wanted to test something
14:43:20 <boris-42> let's move to next topic
14:43:37 <boris-42> as far as we know where is the issue
14:43:43 <boris-42> rest can be disucssed in rally chat
14:44:05 <boris-42> #topic [boris-42] Rally Tags for Big Tent
14:44:13 <boris-42> stpierre: kun_huang amaretskiy rvasilets so guys
14:44:45 <boris-42> we were asked by Ops guys to create tags for big tent
14:44:49 <boris-42> related to Rally
14:45:02 <boris-42> I believe that there should be at least few of them
14:45:20 <boris-42> like has-rally-plugins runs-rally-in-gates
14:45:37 <boris-42> works-against-rally =)
14:45:54 <kun_huang> btw, our talk is cheep, if we have a plan for such tag, will that be reviewed by tc team?
14:47:14 <rvasilets> Sorry for nobie but what is the term "big tent" exactly mean?)
14:48:46 <boris-42> rvasilets: https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/the-big-tent-a-look-at-the-new-openstack-projects-governance
14:49:03 <boris-42> kun_huang: so I believe we should make a patch
14:49:12 <boris-42> kun_huang: to governance
14:49:18 <boris-42> kun_huang: or even bettter start mailing thread
14:49:42 <rvasilets> "Just Rally it"
14:50:52 <rvasilets> "rally-for-fast-clouds"
14:51:17 <boris-42> rvasilets: seems like you mislead the idea of tags
14:51:24 <rvasilets> yes
14:51:31 <boris-42> rvasilets: tags are added to the projects
14:51:42 <boris-42> rvasilets: if they pass some criteria
14:51:47 <boris-42> rvasilets: like they have rally plugins
14:51:54 <boris-42> rvasilets: or they use rally in gatting
14:52:04 <boris-42> rvasilets: they works under load of 1k users and so on
14:53:13 <rvasilets> Why we need tags?=) Is this just for Openstack bureaucracy?
14:53:51 <rvasilets> ok. "rally-performance"?
14:54:47 <kun_huang> rvasilets: for example
14:54:58 <rvasilets> How will use this tags?
14:55:14 <kun_huang> the recent ceilometer perfomance issue will be failed with "work in rally"
14:55:29 <kun_huang> because their recently change break rally's job
14:55:34 <kun_huang> on perfomance view
14:55:48 <kun_huang> over 300s for event list api call
14:56:15 <boris-42> kun_huang: it's some kind of bug inside ceilometer
14:56:24 <boris-42> kun_huang: I believe
14:56:32 <kun_huang> boris-42: it is
14:56:48 <kun_huang> there is a very slow sql query
14:57:00 <kun_huang> liusheng is working on it
14:57:22 <boris-42> kun_huang: great
14:58:59 <boris-42> okay we have to finish our meeting
14:59:01 <boris-42> #endmeeting