17:02:35 <boris-42> #startmeeting rally
17:02:35 <openstack> Meeting started Tue Dec 23 17:02:35 2014 UTC and is due to finish in 60 minutes.  The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:38 <openstack> The meeting name has been set to 'rally'
17:02:41 <boris-42> hey hey ho
17:02:44 <boris-42> somebody around?
17:02:46 <oanufriev> yo
17:02:59 <olkonami> hi
17:03:23 <andreykurilin> hi
17:03:36 <rvasilets> Good evening
17:05:08 <msdubov_> Hi!
17:05:20 <amaretskiy> hi
17:05:20 <boris-42> redixin: around?
17:05:27 <redixin> boris-42, yep
17:05:56 <boris-42> #Mirantis Rally CI
17:06:03 <boris-42> #topic Mirnatis Rally CI
17:06:13 <boris-42> redixin: could you please provide some update about current state of work
17:07:32 <redixin> currently we have unittests foy py26 and cli tests vs MOS-5.1 on RallyCI
17:08:01 <boris-42> redixin: btw where is specified task that is run against MOS-5.1?
17:08:22 <redixin> tox -ecli is run against MOS
17:08:36 <boris-42> redixin: ah so our functional tests
17:08:58 <boris-42> redixin: btw can you make a job that runs tasks (I mean like gate-dsvm-rally-*)
17:09:00 <redixin> also job for testing fockerfile is done, but not working -_-
17:09:08 <boris-42> redixin: lol
17:09:14 <boris-42> redixin: any chance that get it working?
17:09:17 <redixin> some strange issues with sqlalchemy versions
17:09:27 <boris-42> redixin: hm interesting
17:09:37 <boris-42> redixin: I rembmeber something related to sqlalchmey and ubuntu
17:09:39 <andreykurilin> redixin: it seems to me, I saw such error
17:09:43 <boris-42> redixin:  we have few questions
17:09:49 <boris-42> redixin: in answersw.launchapd.net
17:10:22 <redixin> boris-42, which questions?
17:10:42 <boris-42> redixin: somebody wasn't able to install rally
17:10:51 <boris-42> redixin: cause of issues with sqlalchemy on ubuntu
17:11:08 <boris-42> redixin: seems like you are facing the same
17:11:32 <redixin> boris-42, ok, Ill try to find it
17:11:52 <boris-42> redixin: okay I am looking forward to get this done
17:12:02 <boris-42> redixin: so we will be able to merge dockerfile and keep it working*
17:12:07 <andreykurilin> redixin: https://review.openstack.org/#/c/141894/1
17:12:35 <andreykurilin> it seems that problem in setuptools
17:12:54 <boris-42> #topic Fix the world
17:13:02 <boris-42> andreykurilin: redixin https://review.openstack.org/#/c/143687/ this fixed our gates
17:13:19 <andreykurilin> +2 from me
17:13:38 <boris-42> andreykurilin: redixin it's latest version. I removed copy pasted code, add filter for plugins (so we don't tests in tests. descibed plugins)
17:13:48 <boris-42> andreykurilin: redixin as well now it shows full name of obj
17:14:18 <boris-42> #topic py34 compability
17:14:29 <boris-42> andreykurilin: could you provide update about stuff related to it?
17:14:37 <andreykurilin> boris-42: sure
17:15:07 <andreykurilin> last week we start to fix issues related to py3 compatibility
17:15:17 <redixin> oh, I saw a lot of bugs reported on launchpad about py34 incompatibles
17:15:22 <andreykurilin> yes)
17:16:05 <andreykurilin> and list of bugs related to py3k are not full in launchpad
17:16:25 <andreykurilin> but we already have some promotions
17:16:30 <redixin> mb we should create one bug "rally is not py34 compatible"?
17:17:00 <boris-42> redixin: nope
17:17:11 <boris-42> redixin: cause various people can work on various issues
17:17:13 <andreykurilin> redixin: imo, one bug per each issue is more user friendly)
17:17:17 <boris-42> redixin: and they can organize they work better
17:17:46 <redixin> i thought it may be fixed by one person in one day
17:17:58 <boris-42> redixin: seems like there is a lot of issues
17:18:36 <andreykurilin> updates of this work are:
17:18:51 <andreykurilin> 1) we found the bug in sqlalchemy and fix it localy
17:19:05 <andreykurilin> 2) fix issues related to iteritems
17:19:33 <andreykurilin> also, fix for StringIO and basestring are sent for review
17:19:45 <andreykurilin> https://review.openstack.org/#/c/143585/
17:19:49 <boris-42> andreykurilin: redixin btw we will need to run "rally using python3"
17:19:50 <andreykurilin> https://review.openstack.org/#/c/143584/
17:19:57 <boris-42> I have some ideas
17:20:11 <andreykurilin> boris-42 ?
17:20:17 <andreykurilin> what ideas&)
17:20:20 <andreykurilin> ?)
17:20:31 <boris-42> #topic Improve performance jobs
17:20:42 <boris-42> redixin: andreykurilin I would like to have suffixes in files
17:20:56 <boris-42> rally_py34_keystonev3
17:21:04 <boris-42> rally_py34_keystonev3.yaml
17:21:13 <boris-42> will use rally-gate-dsvm-rally job
17:21:18 <boris-42> will be used by*
17:21:24 <boris-42> and run if it is specified
17:21:32 <boris-42> so 1 our job can run N files
17:21:33 <redixin> rally_{python_version}_{keystone_version}.yaml?
17:21:53 <boris-42> rally_[{any_from_hardcoded_keys}]
17:22:06 <redixin> how to parse it?
17:22:14 <boris-42> so just split by _
17:22:18 <boris-42> first is file name
17:22:23 <boris-42> all others are keys
17:22:45 <boris-42> and we will have set of set of keys that we are supporitng
17:22:52 <redixin> ok
17:22:59 <boris-42> redixin: this can be done without touching infra
17:23:09 <boris-42> redixin: just fixing ci/*
17:23:35 <boris-42> amaretskiy: can help with formating HTML page for that
17:23:42 <boris-42> redixin: ^
17:23:55 <boris-42> redixin: thougths?
17:23:58 <amaretskiy> i can help with html
17:24:42 <boris-42> amaretskiy: so the idea is that HTML should use some dirty ajax checks
17:24:58 <amaretskiy> yes, i remember we discussed that
17:25:10 <amaretskiy> i'm going to fix that
17:25:13 <boris-42> andreykurilin: in this case we will have N different reports*
17:25:16 <boris-42> amaretskiy: ^
17:25:22 <boris-42> amaretskiy: not 1
17:25:37 <boris-42> amaretskiy: we were discussing about cheking "logs"
17:25:49 <boris-42> amaretskiy: but we can use similiar approach here
17:25:50 <amaretskiy> okay, understood
17:26:00 <redixin> do you want to request /logs/report1.html, /logs/report2.html and so on, until we get 404?
17:26:01 <boris-42> andreykurilin: ok great
17:26:15 <amaretskiy> the approach is dirty but there is probably no another way
17:26:26 <boris-42> amaretskiy: yep there is no other way
17:26:31 <amaretskiy> ok
17:26:34 <boris-42> redixin: yep we will check all files*
17:26:41 <boris-42> redixin: but they will called different
17:27:01 <redixin> actually we can put results.list.txt with list of available reports
17:27:03 <boris-42> redixin: not 1,2,3,4 but report_py34, report_keystonessl
17:27:05 <boris-42> and so on
17:27:21 <boris-42> redixin: yep we can
17:27:26 <boris-42> redixin: actually this is better way
17:28:27 <redixin> we can even add some desctiptions to this list, and parse it with js in browser
17:28:35 <boris-42> redixin: yep yep
17:29:10 <andreykurilin> redixin: good idea
17:29:13 <boris-42> redixin: so can you implement this soon?
17:29:20 <boris-42> redixin: seems like requires a bit bash*
17:29:35 <redixin> boris-42, I can start this tomorrow, after fixing dockerfile job
17:30:03 <andreykurilin> boris-42: "a bit" :D
17:30:47 <boris-42> redixin: ok great
17:31:03 <boris-42> redixin: cause it drastically improves ability of our gates
17:31:35 <boris-42> #topic Greatest Validation Refactoring
17:31:50 <boris-42> redixin: andreykurilin amaretskiy msdubov_ soon guys you'll have to review a lot of my code=)
17:32:02 <boris-42> Let me try to introduce you what I am going to do
17:32:18 <andreykurilin> boris-42: one patch with 100500 LOC again?)
17:32:29 <boris-42> andreykurilin: no 10 patches with 100500 LOC
17:32:33 <boris-42> andreykurilin: in chain=)
17:32:38 <andreykurilin> :D
17:32:44 <boris-42> andreykurilin: so
17:32:55 <boris-42> 1) put in all validators deployment instead of Task
17:33:08 <boris-42> this will allow us to run validations methods without creating Task
17:33:38 <boris-42> 2) add validation levels (depending on mode, validator will be different processed)
17:34:14 <boris-42> e.g. it can be performed for all users, 1 user, 1 user per tenant, admin and so on
17:34:21 <boris-42> or be just static
17:34:32 <boris-42> like cheking that required fileds are specified
17:34:38 <boris-42> This will be huge patch
17:34:52 <boris-42> 3) Unify validation process between all plugins
17:35:02 <boris-42> so we will use always decorators everywhere
17:35:10 <boris-42> context, scenarios, runners, sla...
17:35:35 <boris-42> 4) Make a jsonschema validator (instead of this stuff with "required_field" and number and so on)
17:35:54 <boris-42> So after we implement all this stuff and merge it
17:36:09 <boris-42> We will be able to use "existing users" for benchmarking without admin
17:36:28 <boris-42> andreykurilin: redixin andreykurilin msdubov_ ^
17:36:52 <andreykurilin> boris-42: sounds good, but we need a code:)
17:36:53 <redixin> ok
17:36:58 <boris-42> ok
17:37:15 <boris-42> #topic Adding deployment information to rally task report
17:37:21 <boris-42> amaretskiy: ^
17:37:52 <boris-42> We can collect all infromation about cloud (like what components we have, what services we are running, hypervisors and so on)
17:38:00 <boris-42> and put it in report
17:38:04 <boris-42> redixin: andreykurilin ^
17:38:26 <amaretskiy> sounds good, but i think i did not see that in road map....
17:38:44 <boris-42> amaretskiy: it is new requred
17:38:46 <boris-42> request
17:38:47 <andreykurilin> amaretskiy, we can put it there:)
17:38:48 <amaretskiy> ok
17:38:52 <amaretskiy> sounds very good
17:38:56 <boris-42> amaretskiy: I didn't have a chance to add it
17:39:00 <amaretskiy> i'm ready :)
17:39:01 <boris-42> amaretskiy: but I will
17:39:08 <boris-42> amaretskiy: so put it to your notes
17:39:11 <amaretskiy> ok
17:39:26 <boris-42> amaretskiy: but please first provide fixes for current report that you have in notes
17:39:36 <boris-42> amaretskiy: especially with displaying erros on graphs
17:39:47 <boris-42> okay
17:39:51 <boris-42> #topic FREE discussion
17:39:59 <amaretskiy> there is a big list of notes :)
17:40:03 <boris-42> anybody wants to discuss antyhing?)
17:40:09 <andreykurilin> boris-42: I have a question about collection of cloud information
17:40:17 <boris-42> andreykurilin: sure
17:40:56 <amaretskiy> let's discuss "abstract classes are evil in most cases for rally"
17:41:09 <andreykurilin> boris-42: do you propose to collect it during task launch and store it there, ot have a one infor for deployment?
17:42:00 <andreykurilin> amaretskiy: why you think so?)
17:42:12 <boris-42> andreykurilin: I belive that it should be "rally deployment create"
17:42:27 <boris-42> andreykurilin: so we will store this info in Deployment table
17:42:33 <amaretskiy> 1) can not check arguments 2) no hooks 3) no tests
17:42:59 <amaretskiy> my proposal is "duck typing way"
17:43:15 <andreykurilin> boris-42: but what will happened if clound structure will change? user will not change deployment config, because credentials are left the same
17:44:15 <boris-42> andreykurilin: hm yep you are right
17:44:29 <boris-42> andreykurilin: but collecting on every run can produce various issues...
17:44:44 <boris-42> andreykurilin: maybe we need to add some argument to "task"?
17:44:58 <boris-42> andreykurilin: rally task start --with-deployment-info
17:45:01 <boris-42> andreykurilin: ?
17:45:14 <andreykurilin> boris-42: hm..yeah, it can fix the issues
17:45:24 <boris-42> andreykurilin: but it adds new arugment that suxx
17:46:15 <andreykurilin> boris-42: or we can check the structure before launching task and if it changed-change info in deployment object
17:47:45 <andreykurilin> amaretskiy: 2-3) what tests and hooks do you want?)
17:49:09 <amaretskiy> andreykurilin: take a look at https://review.openstack.org/#/c/103306/96/rally/benchmark/wrappers/network.py,cm
17:49:33 <amaretskiy> and consider how to test https://review.openstack.org/#/c/103306/96/rally/benchmark/wrappers/network.py,cm
17:49:46 <amaretskiy> at least how to do this in good manner
17:49:58 <boris-42> andreykurilin:  I agree with amaretskiy
17:50:03 <amaretskiy> and now take a look at https://review.openstack.org/#/c/103306/89/rally/benchmark/services/network.py,cm
17:50:06 <boris-42> That testing abstract class is some kind of hell
17:50:18 <amaretskiy> see this - we verify arguments
17:50:25 <amaretskiy> and we can put hooks
17:50:31 <amaretskiy> if we want hooks
17:50:53 <amaretskiy> so duck typing is no extra code but a lot of extra flexoibility + tests
17:50:58 <amaretskiy> *flexibility
17:51:13 <amaretskiy> but
17:51:18 <amaretskiy> we can use abc
17:51:30 <amaretskiy> if we provide some classes to user implementation
17:51:58 <amaretskiy> but users usually do not implement something out of our code base
17:52:33 <amaretskiy> usually we create few classes and review them via gerrit
17:53:07 <amaretskiy> and duck typing also checks the implementation
17:53:21 <amaretskiy> so, what do you think?
17:53:22 <andreykurilin> amaretskiy, boris-42: If our abstarct class is not huge, I don't see any problems with testing it, just create FakeImplementaionOfOurClass and test it
17:53:59 <amaretskiy> andreykurilin: this way is real but huge - why do we need it?
17:54:26 <amaretskiy> andreykurilin just try to really test each abstract method
17:54:38 <amaretskiy> and you will se a number of LOC
17:55:48 <amaretskiy> abstract classes even not a language item - that is an extra feature provided by module
17:56:05 <andreykurilin> amaretskiy: imo, if we implement some FakeImplementation, we can check every method of original abstract class. if we don't use abstarct class(use implementation from 89 patch set), we cannot test methods which launch unexisten methods
17:56:38 <boris-42> andreykurilin: amaretskiy there is only one issue without abstracclasses
17:56:50 <amaretskiy> andreykurilin: what if FakeImplementation will implement non-abstarct methods?
17:56:59 <amaretskiy> how we will check that?
17:56:59 <boris-42> andreykurilin: amaretskiy we need to ensure that we implemented extactly "interface" method
17:57:11 <boris-42> andreykurilin: amaretskiy without typos
17:57:15 <boris-42> otherwise it's hell
17:57:46 <amaretskiy> and what about hooks?
17:57:55 <andreykurilin> do we need it now?)
17:57:57 <boris-42> andreykurilin: you mean call _method?
17:58:06 <andreykurilin> boris-42: yes
17:58:07 <boris-42> amaretskiy: ^
17:58:17 <boris-42> amaretskiy: andreykurilin looks dirty
17:58:19 <amaretskiy> https://review.openstack.org/#/c/103306/89/rally/benchmark/services/network.py,cm
17:58:21 <boris-42> but fix situtation
17:58:32 <amaretskiy> if we put something between L59-60
17:59:04 <amaretskiy> and after 60 (move return)
17:59:11 <andreykurilin> amaretski: we can do it, when we will need it
17:59:13 <amaretskiy> so we have a wrapper
17:59:43 <amaretskiy> andreykurilin: id need: 1) abc - reimplementation 2) ducktyping: ready
17:59:51 * morganfainberg waves at boris-42
17:59:53 <boris-42> amaretskiy: andreykurilin let's continue discussion
17:59:53 <amaretskiy> s/id/if/
17:59:59 <boris-42> in rally
18:00:02 <boris-42> #endmeeting