17:00:55 <boris-42> #startmeeting Rally
17:00:55 <openstack> Meeting started Tue Jan 21 17:00:55 2014 UTC and is due to finish in 60 minutes.  The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:56 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:58 <openstack> The meeting name has been set to 'rally'
17:01:04 <hughsaunders> hi boris-42
17:01:09 <boris-42> hughsaunders hi
17:01:16 <boris-42> who is here?)
17:01:22 <olkonami1> hi all!
17:01:24 <miarmak> hi
17:01:25 <stannie> hi
17:01:39 <julienvey> hi
17:02:23 <boris-42> oh seems I am not alone+) I should care what I am saying =)
17:02:49 <boris-42> topic HOT topics
17:02:54 <boris-42> #topic HOT topics
17:02:57 <hughsaunders> boris-42: ~362 lurkers in this channel
17:03:09 <jieryn> #
17:03:10 <tnurlygayanov_> ))
17:03:17 <boris-42> lol=)
17:03:41 <boris-42> So topic list:
17:03:58 <boris-42> 1) Rally & Tempest integrration
17:04:17 <boris-42> 2) Rally & Running benchmarks against cloud (without admin endpoint)
17:04:34 <boris-42> 3) Rally & Multihost deployments
17:04:53 <boris-42> 4) Rally & Smart benchmark scenarios
17:04:56 <boris-42> checkers
17:05:09 <boris-42> 5) Rally as a Service
17:05:20 <boris-42> 6) Improvements in Rally CLI
17:05:34 <boris-42> 7) Rally & benchmarking performance of VMs
17:06:30 <boris-42> 8) Rally use class instances for benchmark scenarions instead of class objects and classmethods
17:06:47 <boris-42> 9) Rally measuring time of atomic actions
17:06:53 <boris-42> ^ Thats all
17:07:04 <boris-42> I will try to use this order in today meeting
17:07:13 <boris-42> #topic Rally & Tempest integration
17:07:25 <boris-42> miarmak pls could you share your results?
17:08:16 <boris-42> (btw guys/girls if you are related to any of this parts prepare your updates)
17:08:21 <miarmak> boris-42: unfortunately, situation has not changed from yesterday
17:08:34 <miarmak> main problem - conf generation
17:08:53 <boris-42> miarmak so you are not able to configure correct conf for tempest ^
17:08:56 <miarmak> maybe anyone know projects, that have smth same?
17:09:00 <boris-42> probably sdague could help us? ^
17:09:47 <hughsaunders> boris-42 mentioned that devstack configures tempest, so this may help: https://github.com/openstack-dev/devstack/blob/master/lib/tempest#L66
17:10:09 <boris-42> hughsaunders ohhhh
17:10:14 <boris-42> hughsaunders how much lines of code..
17:10:30 <sdague> boris-42: if it's not a gate issue, I can't really help you this week
17:10:46 <boris-42> sdague sure understand you, thank you for fixing gate!
17:10:53 <hughsaunders> I'm going to attempt to chef-ify that for the stackforge chef cookbooks project
17:11:14 <boris-42> hughsaunders how to generate proper config for tempest?
17:11:43 <hughsaunders> boris-42: yeah, not started yet, but planning to base it on that devstack function
17:12:01 <boris-42> miarmak hughsaunders  could you probably collaborate around this?
17:12:35 <hughsaunders> Yeah, if I get anywhere I'll ping miarmak with a link
17:12:44 <boris-42> hughsaunders nice
17:12:45 <boris-42> #topic  Rally & Running benchmarks against cloud (without admin endpoint)
17:12:48 <miarmak> ok
17:12:50 <miarmak> thanks
17:13:01 <boris-42> So first of all
17:13:19 <boris-42> #link https://blueprints.launchpad.net/rally/+spec/benchmarking-with-predefined-users
17:13:23 <boris-42> ^ bp
17:13:43 <boris-42> #link https://review.openstack.org/#/q/status:open+project:stackforge/rally+branch:master+topic:bp/benchmarking-with-predefined-users,n,z
17:13:46 <boris-42> patches on review ^
17:13:54 <boris-42> The idea is simple
17:14:06 <boris-42> Now Rally work in next way
17:14:57 <boris-42> get admin endpoints -> create admin clients -> create tmp tenants & users -> run from temp users benchmarks -> cleanup of cloud -> delete tmp users -> FINISH
17:15:19 <boris-42> So it's work until we would like to test Public clouds
17:15:25 <boris-42> where we don't have admin endpoints
17:15:44 <boris-42> So we will add support of running benchmark from bunch of predefined users
17:16:09 <boris-42> input endpoints of non admin users -> create python clients -> run benchmarks -> make non admin cleanups
17:16:31 <boris-42> So if you will have some free time you can spend it reviewing that patches=)
17:16:56 <boris-42> Does anybody has any questions?)
17:17:39 <hughsaunders> seems like a good plan
17:18:10 <boris-42> stannie julienvey  thoughts?
17:18:16 <stannie> seems good to me also
17:18:28 <boris-42> okay nice then we will move just to next topic
17:18:45 <boris-42> #topic 	 3) Rally & Multihost deployments
17:19:07 <boris-42> #link https://blueprints.launchpad.net/rally/+spec/multihost-deploy
17:19:14 <boris-42> bp ^
17:19:38 <boris-42> #link https://review.openstack.org/#/q/status:merged+project:stackforge/rally+branch:master+topic:bp/multihost-deploy,n,z
17:19:40 <boris-42> ^ patches
17:19:45 <boris-42> This work is already done
17:19:48 <boris-42> and should be merged
17:19:54 <boris-42> Idea is next
17:20:12 <boris-42> we are deploying with separated deployers/provider controller and compute/cinder nodes
17:20:37 <boris-42> so if we choose DevStack it will run N times .stacksh
17:20:46 <boris-42> which will have bad perfromance
17:21:10 <boris-42> so we are deploying controller + N type of Nodes only 1 time
17:21:23 <boris-42> (so N+1) call of stack.sh
17:21:30 <boris-42> and then copy pasting LXC containers
17:21:46 <boris-42> which allows us to deploy 1 controller + 128 compute managers for just a 30 minutes
17:22:30 <hughsaunders> boris-42: on a single underlying node?
17:22:41 <boris-42> it could use multiply nodes
17:22:47 <boris-42> or even OpenStack provider
17:22:56 <boris-42> or even any other provider
17:23:03 <hughsaunders> like tripple-o undercloud
17:23:09 <boris-42> hughsaunders but better=)
17:23:36 <boris-42> it doesn't required OpenStack installation
17:23:44 <boris-42> it will work with provider that you put
17:23:56 <boris-42> Dummy/OpenStack/your_custom_cloud_provider
17:24:28 <hughsaunders> so how do you determine in config how many lxc containers are created on each underlying node?
17:24:31 <boris-42> hughsaunders here is the instruction how to try it http://paste.ubuntu.com/6792218/
17:24:34 <stannie> on trello I saw you guys were thinking to integrate Docker, in this case it would simplify this usage. We wouldn't have to copy the LXC containers etc
17:24:48 <boris-42> stannie we need
17:25:06 <boris-42> stannie you should install from custom code your OpenStack installation
17:25:23 <boris-42> stannie with any of Deployers (e.g. DevStack/Anvil/Fuel)
17:25:32 <boris-42> stannie and then copy/paste installed containers
17:25:50 <boris-42> stannie and then fix all copy/pasted lxc containers
17:26:11 <boris-42> stannie there should be fix in nova.conf/cinder.conf/.… to add proper network information (IP/host_name)
17:26:20 <boris-42> stannie and run this components
17:26:20 <stannie> ye and that's can be done easily with Docker
17:26:31 <boris-42> stannie but I don't see any problems to do it with lxc
17:26:37 <stannie> ok
17:26:50 <stannie> it just asks a little more work
17:26:54 <boris-42> stannie https://review.openstack.org/#/c/56222/
17:27:06 <boris-42> stannie 150 lines of code
17:27:24 <stannie> ok
17:27:38 <boris-42> stannie but we are as well experimenting with docker
17:27:55 <boris-42> stannie to make another future e.g. cloud deployment snaphsot
17:27:58 <boris-42> stannie to share cloud+)
17:28:14 <boris-42> feature&
17:28:24 <boris-42> stannie that is the case where docker can help us
17:29:13 <boris-42> stannie hughsaunders  so you can try as well to use this stuff https://review.openstack.org/#/c/56222/
17:29:26 <boris-42> stannie hughsaunders I will try to make installation today and merge it if it works=)
17:29:54 <boris-42> we should move to the next topic
17:29:56 <boris-42> #topic 4) Rally & Smart benchmark scenarios
17:30:06 <boris-42> #topic 4) Rally & Smart benchmark scenarios checkers
17:30:16 <boris-42> olkonami1 could you pls share with your resuls
17:30:27 <boris-42> olkonami1 seems like work is almost done
17:30:51 <olkonami1> https://review.openstack.org/#/c/67157/
17:30:52 <olkonami1> https://review.openstack.org/#/c/68055/
17:31:10 <boris-42> olkonami1 I mean could you expelling what have you done=)
17:31:15 <boris-42> olkonami1 in human way=)
17:31:27 <boris-42> olkonami1 btw you should make new blueprint
17:31:31 <boris-42> olkonami1 for your patches
17:32:11 <hughsaunders> I really like the image_id validator, simple and effective
17:32:46 <boris-42> hughsaunders yep there will be at least 2 more
17:32:53 <boris-42> hughsaunders for flavor checking
17:32:57 <olkonami1> I made decorator that add validators for scenario methods
17:33:04 <boris-42> hughsaunders and checking that image could be use with flavor
17:34:24 <olkonami1> and this validators should be called when runner starts
17:34:46 <stannie> ye that's great, we'll be able to add more validator without adding complexity in the method
17:35:04 <boris-42> stannie yep and stop execution before starting it=)
17:35:15 <olkonami1> now first validator for image_id is implemented
17:35:43 <boris-42> so I hope we will merge this soon
17:36:14 <boris-42> #topic	 5) Rally as a Service
17:36:18 <boris-42> ogelbukh ping
17:36:47 <boris-42> ogelbukh akscram  could you guys shares updates?
17:37:24 <boris-42> I will just make now some small introductions. We would like to support 2 versions of Rally in the same time
17:38:00 <boris-42> 1 that is service (with deamons) and could be used by corporation to have 1 Rally installation integrated in QA process and simple WEB UI for all operations
17:38:15 <boris-42> another one that is like now just CLI application
17:38:44 <boris-42> so ogelbukh akscram are hard working around rally-python client and rally-as-a-service part
17:39:05 <boris-42> #link https://review.openstack.org/#/q/status:open+project:stackforge/rally+branch:master+topic:bp/rally-service-api-v1,n,z
17:39:55 <boris-42> and here is the first v.0.1 rally
17:39:59 <boris-42> #topic http://docs.rallyapi.apiary.io/
17:40:11 <boris-42> #topic  4) Rally & Smart benchmark scenarios checkers
17:40:24 <boris-42> #topic     5) Rally as a Service
17:40:29 <boris-42> omg =)
17:40:37 <boris-42> #link  http://docs.rallyapi.apiary.io/
17:40:53 <boris-42> so if somebody is interstened you are welcome to describe API of Rally
17:41:14 <boris-42> #topic 	 6) Improvements in Rally CLI
17:41:37 <boris-42> stannie julienvey could you share your work
17:42:08 <julienvey> i fixed a small bug on cli output
17:42:18 <julienvey> and currently working on rally use command
17:42:35 <julienvey> (trying to find the bp)
17:43:02 <boris-42> julienvey to find?
17:43:03 <julienvey> https://blueprints.launchpad.net/rally/+spec/rally-use-command
17:43:22 <julienvey> i have had some problem with exporting env vars in python
17:43:28 <boris-42> julienvey bp is approved
17:43:43 <julienvey> boris-42: it was just to share it :)
17:43:55 <boris-42> julienvey =)
17:44:03 <boris-42> julienvey so you are still working around ssh stuff
17:44:08 <julienvey> i should propose it to review tomorow, but i think it will need some rework
17:44:15 <boris-42> julienvey probably just call from python some bash command
17:44:25 <boris-42> julienvey dirty but works=)
17:44:38 <julienvey> actually it doesn't :(
17:44:54 <boris-42> julienvey ah yes..
17:45:18 <boris-42> julienvey there is another way via DB, but not sure that it's okay
17:45:46 <julienvey> i went with the way proposed here https://bugs.launchpad.net/rally/+bug/1266439/comments/3
17:45:48 <boris-42> julienvey or some extra rally tmp fconfig
17:46:08 <julienvey> use .pam_environment file to store the env var
17:46:24 <boris-42> julienvey yep something like that should work
17:46:58 <stannie> I am going to work on "deployment check" subcommand. The subcommand will check keystone auth, get all available services and print names etc.
17:47:02 <stannie> #link https://blueprints.launchpad.net/rally/+spec/deployment-check-subcommand
17:48:24 <boris-42> stannie approved
17:48:29 <stannie> thanks
17:48:53 <boris-42> stannie julienvey  I will be waiting for your patches
17:49:08 <boris-42> stannie julienvey as well it will be good to update/add some extra info on wiki page
17:49:34 <julienvey> boris-42: yes, we have to :)
17:49:57 <boris-42> oaky as we have only 10 minutes moving to next topic
17:50:01 <boris-42> #topic 	 7) Rally & benchmarking performance of VMs
17:50:17 <boris-42> msdubov & me are going to start working around this
17:50:45 <boris-42> I wrote some interesting document
17:50:50 <boris-42> #link https://etherpad.openstack.org/p/Rally_VMs_Benchmarks
17:50:57 <boris-42> The idea is actually as well pretty simple
17:51:15 <boris-42> using the same benchmark engine
17:51:25 <boris-42> but in scenarios we will use heat & heat templates
17:51:46 <boris-42> that will install HPCC, SPEC and other Benchamrks suits on VMs
17:51:48 <boris-42> run them
17:51:56 <boris-42> then collect data
17:52:29 <boris-42> and have for each such benchmark specific type of output data and data processing tool (viewer)
17:52:46 <boris-42> that will show most important data + raw output of these benchmarks
17:53:30 <boris-42> hughsaunders julienvey stannie  ^
17:53:39 <msdubov> boris-42 How should we implement this viewer?
17:53:48 <hughsaunders> boris-42: so will your normalise data before it goes into DB?
17:53:56 <hughsaunders> *you
17:54:12 <boris-42> hughsaunders actually we will go to VMs
17:54:24 <boris-42> hughsaunders then fetch data from bench suuit
17:54:48 <boris-42> hughsaunders then store it in DB as is (but with some kind of marker e.g. specWEB2009)
17:55:06 <boris-42> hughsaunders and we will have viewer that can aggregate this date/or just display as is
17:56:23 <boris-42> so I think that I will publish this to mailing list to get more attention
17:56:31 <boris-42> there is a lot of for discussion
17:56:37 <boris-42> #topic 9) Rally measuring time of atomic actions
17:56:49 <boris-42> Short introduction
17:57:30 <boris-42> just use decorators e.g. @atomic_action("name_of_action")
17:57:31 <boris-42> https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/utils.py#L28
17:57:35 <boris-42> in such places ^
17:57:59 <boris-42> this patch will measure time of call and put this time in special CLS dict
17:58:23 <boris-42> and scenario runner will add extra field https://github.com/stackforge/rally/blob/master/rally/benchmark/runner.py#L60
17:58:27 <boris-42> ^ here
17:58:43 <boris-42> and then improve CLI to display this data as well
17:59:01 <boris-42> as I know stannie is interested in implementing this
17:59:09 <boris-42> stannie  I right?
17:59:15 <boris-42> am I right?*
17:59:31 * hughsaunders has to go, laters
17:59:40 <boris-42> we all have to go=)
17:59:46 <boris-42> hughsaunders bye
18:00:04 <boris-42> ok we can continue discussion in rally chat
18:00:10 <boris-42> #endmeeting