17:01:21 <boris-42_> #startmeeting Rally
17:01:22 <openstack> Meeting started Tue Feb 11 17:01:21 2014 UTC and is due to finish in 60 minutes.  The chair is boris-42_. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:01:26 <openstack> The meeting name has been set to 'rally'
17:01:31 <boris-42_> dkranz
17:01:55 <dkranz> boris-42_: hi
17:02:01 <boris-42_> hughsaunders jorisroovers rediskin  tzabal akscram stannie miarmak meeting
17:02:06 <miarmak> hi
17:02:23 <boris-42_> tnurlygayanov as well meeting time
17:02:27 <tnurlygayanov_> Hi
17:03:13 <tnurlygayanov_> yes, I'm here )
17:03:31 * boris-42_ typing todays topics
17:03:59 <hughsaunders> hi all
17:04:19 <tnurlygayanov_> The document with idea was updated: https://docs.google.com/document/d/1bYdiFiIXKOR2OHeGq2fCm1956CTQVCY-BEZc-irmcxw/edit?usp=sharing
17:04:29 <boris-42_> tnurlygayanov order pls=)
17:04:50 <boris-42_> tnurlygayanov this is weekly meeting about all that happened with Rally and will happen=)
17:05:23 <boris-42_> 1. Atomic actions
17:05:32 <boris-42_> 2. Scenarion runner refactoring
17:05:40 <boris-42_> 3. Rally & Tempest intgeration
17:05:55 <boris-42_> 4. Multi node deployment & LXC engine
17:06:20 <boris-42_> 5. Updates about fixing couple of painful bugs
17:06:36 <boris-42_> 6. Updates in rally.conf
17:06:47 <dkranz> boris-42_: I don't have this wholel hour free so can you tell me when you want to discuss the tempest thing?
17:06:58 <boris-42_> dkranz yep I will just ping you=)
17:07:18 <dkranz> boris-42_: But I have to be away part of the time. How about half-past?
17:07:31 <boris-42_> okay try to make it happen=)
17:07:44 <dkranz> boris-42_: or just ping me and I'll be back soon
17:07:58 <boris-42_> ok
17:08:11 <boris-42_> 7. VM benchmarks
17:08:16 <boris-42_> so let's start
17:08:25 <boris-42_> #topic 1. Atomic actions
17:08:33 <boris-42_> Just short update that this work was finished
17:08:45 <boris-42_> https://github.com/stackforge/rally/commit/b5d24fb3959e9467c4ef8e2924412b37795c6673
17:08:51 <boris-42_> by this patch ^
17:09:11 <hughsaunders> great patch :)
17:09:23 <boris-42_> So now we are able to measure times of atomic actions, e.g. booting VM/deleting VM/snapshoting it and so on
17:09:38 <boris-42_> hughsaunders yep great one=)
17:09:46 <boris-42_> #topic 2. Scenarion runner refactoring
17:09:49 <rediskin> it not quite atomic =)
17:09:59 <boris-42_> rediskin gg
17:10:12 <boris-42_> so we done a lot of work around refactoring our scenario runner
17:10:30 <boris-42_> First of all we improved generic cleanup and creation of users
17:10:48 <boris-42_> now we are using context so cleanup is proceed always
17:11:02 <boris-42_> and as well we removed spaghetti from code
17:11:22 <boris-42_> Second thing is that now we are able to make plugins
17:11:31 <boris-42_> to generate different load
17:11:34 <tnurlygayanov_> cool
17:11:44 <rediskin> plugins?
17:11:48 <tnurlygayanov_> do we have example?
17:11:49 <boris-42_> rediskin https://github.com/stackforge/rally/tree/master/rally/benchmark/runners
17:11:57 <rediskin> oh
17:12:08 <boris-42_> rediskin yep just add new your_load_generator.py
17:12:10 <rediskin> yea, it is easy to write own runner
17:12:23 <boris-42_> There are couple of things that should be also refactored
17:12:32 <hughsaunders> ahh, but not really a plugin, as it has to be in-tree?
17:12:39 <boris-42_> hughsaunders nope
17:12:48 <boris-42_> hughsaunders Olga is working around plugin
17:12:51 <boris-42_> hughsaunders system
17:13:01 <boris-42_> hughsaunders for engines/serverprovider/benchmarks/loaders
17:13:11 <rediskin> https://review.openstack.org/#/c/72679/
17:13:17 <rediskin> ^^ load extra modules
17:13:23 <boris-42_> yep that one
17:13:45 <hughsaunders> could use entrypoints for that?
17:13:57 <boris-42_> hughsaunders actually we are going to use rally.conf
17:14:16 <boris-42_> hughsaunders I don't like steavedoor..
17:14:26 <hughsaunders> ok
17:14:57 <boris-42_> but unfortunately Olga chose wrong place
17:15:00 <boris-42_> to put initialization
17:15:12 <boris-42_> so the idea is that we have some place
17:15:17 <boris-42_> in code
17:15:24 <boris-42_> that will just load external modules
17:15:42 <boris-42_> and our factories approach will fetch them=)
17:16:26 <boris-42_> so next thing that we are going to refactor is this place
17:16:27 <boris-42_> https://github.com/stackforge/rally/blob/master/rally/benchmark/engine.py#L134-L154
17:16:39 <boris-42_> ^ we should put here validation of benchmarks inputs
17:16:50 <boris-42_> i mean image_id/flavor_id and so on
17:17:14 <boris-42_> And thing that is Mike working now
17:17:22 <boris-42_> Support of procreated users
17:17:27 <boris-42_> precreated*
17:17:48 <boris-42_> Btw should we discuss it?)
17:18:14 <rediskin> maybe
17:18:40 <boris-42_> #topic Support of benchmarking with procreated users
17:18:48 <rediskin> how about to create clients-container class
17:19:04 <rediskin> make this object oriented
17:19:27 <boris-42_> but it actually endpoint
17:19:27 <rediskin> do not use all this dicrionaries and diffrerent clients laying around
17:20:02 <boris-42_> ?
17:20:07 <rediskin> it endpoint, but client = make_client(endpoint['username'], endpoint['password']...
17:20:26 <rediskin> everywhere in the code crap like this ^^
17:20:33 <boris-42_> rediskin actually you are speaking about legacy code
17:20:43 <boris-42_> rediskin until we add endpoint object
17:20:56 <boris-42_> rediskin so this part is just not refactored
17:21:24 <boris-42_> I will share with my thoughs
17:21:48 <boris-42_> https://github.com/stackforge/rally/blob/master/rally/osclients.py#L38-L40 this should accept only object endpoint
17:22:03 <boris-42_> and here return it https://github.com/stackforge/rally/blob/master/rally/osclients.py#L139
17:22:29 <boris-42_> this things should be removed https://github.com/stackforge/rally/blob/master/rally/benchmark/runner.py#L33-L36
17:22:35 <boris-42_> ^ they are legacy code =(
17:23:03 <boris-42_> we should pass here https://github.com/stackforge/rally/blob/master/rally/benchmark/runner.py#L39
17:23:08 <rediskin> boris-42_: ok. I'm so looking forward for this refactoring =)
17:23:22 <boris-42_> rediskin so we will refactor it step by step
17:23:37 <boris-42_> rediskin it is hard to make all in one step
17:23:48 <boris-42_> rediskin so we are refactoring it step by step
17:24:01 <boris-42_> And I think that we made already big success
17:24:12 <rediskin> objects Endpoints and Client might be done first
17:24:32 <boris-42_> rediskin I think that it could be done separately
17:24:42 <rediskin> boris-42_: just IMHO
17:24:43 <rediskin> ok
17:24:47 <boris-42_> rediskin ok =)
17:24:56 <boris-42_> does anybody have something to say?)
17:25:48 <boris-42_> #4. Multi node deployment & LXC engine
17:25:56 <boris-42_> #topic 4. Multi node deployment & LXC engine
17:26:03 <boris-42_> lets discuss this before tempest stuff
17:26:16 <boris-42_> hughsaunders rediskin  could you share with updates
17:26:20 <rediskin> I thought it's done, but hughsaunders told it is not
17:26:26 <boris-42_> lol
17:26:59 <rediskin> still some issues. need more tesing.
17:27:04 <rediskin> actually it all works for me
17:27:15 <boris-42_> hughsaunders does it works for you?)
17:28:53 <boris-42_> seems like hughsaunders  is not here..=0
17:28:59 <rediskin> he showd me traceback, but it it not enough. need -d -v options on
17:29:19 <rediskin> anyway there is not much to do
17:29:21 <boris-42_> okay so I hope we will get it soon merged
17:29:41 <boris-42_> #topic 3. Rally & Tempest intgeration
17:29:45 <boris-42_> dkranz around?)
17:29:53 <dkranz> boris-42_: Yes
17:29:54 <boris-42_> miarmak could you share with your updates
17:30:04 <miarmak> yeah
17:30:15 <miarmak> https://review.openstack.org/#/c/70131
17:30:16 <hughsaunders> boris-42_: I have not been able to get lxc or multinode to work yet
17:30:22 <miarmak> patch is ready to review)
17:30:38 <rediskin> miarmak: all tests done?
17:30:42 <boris-42_> miarmak ggg
17:30:45 <miarmak> rediskin: yes
17:30:49 <miarmak> =)
17:30:50 <boris-42_> miarmak could you share in more words
17:30:55 <boris-42_> miarmak what your patch makes
17:30:56 <hughsaunders> but I'm pretty much of the problem is me failing to configure correctly
17:30:58 <miarmak> yoday i have finished
17:31:36 <miarmak> it adds 2 cli commands: rally-manage tempest install to install the Tempest
17:31:50 <miarmak> and 'rally verify start'
17:31:59 <miarmak> to start verifiing process
17:32:26 <miarmak> we can use it with specificated deploy-id
17:32:35 <miarmak> or with default deploy id
17:32:59 <boris-42_> miarmak so and it works with default deployed cloud with devstack?
17:33:17 <miarmak> yes
17:33:25 <rediskin> boris-42_: on devstack almost all tests passed =)
17:33:25 <hughsaunders> brilliant :)
17:33:27 <miarmak> but for now it starts smoke tests
17:33:30 <hughsaunders> haha
17:33:35 <miarmak> to simplifu first steps)
17:33:39 <boris-42_> ahah
17:33:41 <miarmak> simplify*
17:33:53 <boris-42_> okay
17:34:02 <miarmak> all tests pass very long)
17:34:05 <dkranz> miarmak: So devstack is presumably not the real target for rally
17:34:24 <dkranz> miarmak: I made a comment in the review of how I think this really needs to work
17:34:35 <boris-42_> dkranz we agree with you
17:34:36 <rediskin> I gonna test it on cloud deployed bu fuel
17:34:41 <boris-42_> dkranz but step by step
17:34:42 <rediskin> tomorrow..
17:34:45 <dkranz> boris-42_: Of course
17:34:49 <boris-42_> dkranz we would like to get now base in
17:35:00 <dkranz> boris-42_: I am not going to -1 :)
17:35:05 <boris-42_> dkranz so we will be able to simultaneously work on different steps
17:35:22 <boris-42_> dkranz e.g. sets of tests/ storing results/ better config generation
17:35:23 <dkranz> boris-42_: I just thought it would be helpful to understand what is important and missing
17:35:36 <miarmak> dkranz: yeah, thanks)
17:35:41 <boris-42_> dkranz so let me try to describe what we are going to do and why=)
17:35:54 <dkranz> boris-42_: Because devstack is doing stuff that really needs to be in the configure tempest part for real cloud
17:36:08 <boris-42_> dkranz we would like to use Rally against different cloud
17:36:27 <boris-42_> dkranz with fake virtualization, or real big deployments, or private clouds
17:36:33 <dkranz> Sure
17:36:37 <boris-42_> dkranz clouds that are deployed for developers with Rally
17:36:45 <dkranz> I get that
17:36:49 <boris-42_> dkranz so we need mechanism that will understand
17:36:53 <boris-42_> dkranz what we have in cloud
17:37:00 <boris-42_> dkranz and what tests we can actually run
17:37:25 <boris-42_> dkranz so after we merge this patch one of direction will be this
17:37:35 <dkranz> Great
17:37:43 <boris-42_> dkranz when we finish this work
17:37:52 <boris-42_> dkranz we will move this stuff to tempest
17:37:56 <dkranz> boris-42_: Please let me know if you disagree with anything in my comment
17:38:08 <boris-42_> dkranz I mean smart generation of tempest conf
17:38:25 <boris-42_> dkranz even more if you could help us with this work, it will be nice
17:38:47 <boris-42_> dkranz because we don't have enough developers to cover all things… It takes a lot of time to make things simple=)
17:38:51 <dkranz> boris-42_: Yes, but what I was saying is that in order to do smart generation of tempest.conf you also have to create stuff that is expected, and is created by devstack
17:39:16 <boris-42_> dkranz as a first step pre_creation of things could be done by hand
17:39:20 <dkranz> boris-42_: I know and hope we can help. But it will be easier if the initial version submitted to tempest has the form of what I described
17:39:25 <boris-42_> dkranz and passed as extra data
17:39:31 <dkranz> boris-42_: It does not have to be complete.
17:40:38 <boris-42_> dkranz so what's my idea is to split this work
17:40:41 <dkranz> boris-42_: Once an initial version is submitted to tempest I am sure other people will contribute
17:41:16 <boris-42_> dkranz so you would like to start work inside tempest?
17:41:51 <boris-42_> dkranz I am woried a bit the speed of accepting pathces in tempest is less then in Rally… cause it is not top priority
17:41:59 <dkranz> boris-42_: It might be some one else
17:42:09 <dkranz> boris-42_: I can promise this will get high attention
17:42:26 <dkranz> boris-42_: But I do understand your concern
17:42:43 <dkranz> boris-42_: I know there is a lot of interest and this will be reviewed quickly in tempest
17:42:50 <boris-42_> dkranz I mean, the biggest problem is that we don't actually now exactly what we would like to get at the end
17:42:58 <boris-42_> dkranz and it will be faster to make a lot of POC
17:43:08 <boris-42_> dkranz and finial result put to tempest=)
17:43:13 <dkranz> boris-42_: In my comment I tried to describe what this would look like at the end
17:43:35 <dkranz> boris-42_: So it would be fine to push a POC to tempest
17:44:01 <boris-42_> dkranz okay let's try
17:44:16 <dkranz> boris-42_: But please take note of my comment as a guideline
17:44:26 <boris-42_> dkranz I understand that part
17:44:39 <boris-42_> dkranz But it is too high level
17:44:52 <dkranz> boris-42_: And remember that there is already a poc in tempest which you will be replacing so make that clear
17:45:18 <boris-42_> dkranz I mean pre_init of cloud is a bit dangerous thing for Rally
17:45:31 <boris-42_> dkranz because there cases when you don't have admin access
17:45:39 <boris-42_> there are*
17:45:49 <boris-42_> so it should be independet
17:46:01 <boris-42_> e.g. there are part the could prepare cloud for tempest
17:46:07 <boris-42_> and returns some JSON
17:46:17 <boris-42_> and there is a part that should accept this json and make config
17:46:28 <dkranz> boris-42_: I agree they are separate steps but users need to cause the right things to be there
17:46:40 <dkranz> boris-42_: WHether they do it themselves or tell some admin
17:46:53 <dkranz> boris-42_: They should be able to point the admin at a script that does the right thing
17:47:11 <dkranz> boris-42_: There could easily be two separate scripts
17:47:25 <boris-42_> dkranz I hope that we will use python =)
17:47:41 <dkranz> boris-42_: But they are linked in that the "create tempest.conf" is depending on the other one that is creating stuff in the cloud
17:47:57 <dkranz> boris-42_: Why would we not use python?
17:48:06 <dkranz> boris-42_: The POC script in tempest does
17:48:12 <boris-42_> ok nice
17:48:18 <boris-42_> could you just give a link?)
17:48:37 <dkranz> boris-42_: tempest/tools/tempest_auto_config.py
17:48:52 <dkranz> boris-42_: It got totally broken when the sample conf format changed
17:48:55 <boris-42_> dkranz and I let's continue this discussion in mailing list
17:49:09 <dkranz> boris-42_: That's what I was thinking
17:49:21 <boris-42_> dkranz ok I will write some stuff =)
17:49:43 <boris-42_> #topic 5. Updates about fixing couple of painful bugs
17:49:46 <boris-42_> gust short
17:50:05 <boris-42_> There was a bug that freezes Rally if OpenStack doesn't response
17:50:21 <boris-42_> so sometimes it was really impossible to benchmark with Rally
17:50:34 <boris-42_> We fixed it using timeouts in os-python-clients
17:50:59 <boris-42_> And as well I fixed bug around issue gate-26-python that works forever
17:51:08 <boris-42_> by refactoring mocks
17:51:25 <boris-42_> #topic 	 6. Updates in rally.conf
17:51:35 <boris-42_> hughsaunders could you share pls with updates?)
17:51:56 <hughsaunders> I removed lots of magic numbers from scenario utils, and replaced them with config options
17:52:19 <hughsaunders> probably more options than will ever be used, but at least they can be tweaked for different situations - eg short delays for LXC
17:52:59 <boris-42_> hughsaunders  that makes totally sense ^
17:53:19 <boris-42_> ok next topic
17:53:21 <boris-42_> #topic 7. VM benchmarks
17:53:44 <boris-42_> tnurlygayanov_ around?
17:53:50 <boris-42_> tnurlygayanov_ your turn
17:54:54 <boris-42_> Okay I will just say
17:55:13 <boris-42_> that there is work around adding benchmarks that will measure CPU/MEM/IO of VMs
17:55:28 <boris-42_> I think that they will be based on Hugh approach
17:55:52 <boris-42_> but we should just think bit about output
17:55:55 <tnurlygayanov_> yes
17:55:55 <boris-42_> of these scenarios
17:56:03 <boris-42_> to make it clean in Rally arch
17:56:16 <hughsaunders> yeah, at the moment the script is required to ouput json
17:56:28 <boris-42_> hughsaunders yep but we should improve it a bit
17:56:29 <hughsaunders> but we could have some output processor in rally that normalises the json from each script
17:56:45 <boris-42_> hughsaunders but not output data=)
17:57:05 <tnurlygayanov_> and also need to support scenarios when we have several VMs with workload simulation
17:57:05 <boris-42_> hughsaunders so we should think how to process CPU/MEM/IO usage in some common way
17:57:35 <tnurlygayanov_> and run benchmmarks on other VMs
17:57:35 <boris-42_> tnurlygayanov_ eg. IPerf?)
17:57:54 <rediskin> it should be interface like deploy engines/providers
17:58:09 <boris-42_> rediskin we will use for such thing heat
17:58:18 <boris-42_> rediskin to deploy VMs with HPCC
17:58:44 <boris-42_> rediskin tnurlygayanov_  I will think about unifying and making some kind of plugable result processing
17:58:51 <tnurlygayanov_> hm, not only iperf. We want to test performance of cloud under the wrokload
17:59:05 <boris-42_> tnurlygayanov_ it will be really hard to implemnt
17:59:05 <tnurlygayanov_> *workload
17:59:17 <boris-42_> tnurlygayanov_ I was already thinking about it
17:59:23 <boris-42_> tnurlygayanov_ I have a couple of ideas
17:59:29 <boris-42_> tnurlygayanov_ but it won't be simple
17:59:35 <boris-42_> tnurlygayanov_ btw are you familiar with Heat?)
17:59:43 <tnurlygayanov_> yes
17:59:55 <boris-42_> tnurlygayanov_ okay then let's make full things
17:59:56 <tnurlygayanov_> it was the my first idea
18:00:10 <boris-42_> tnurlygayanov_ yep but it will be a bit trickily to integrate it
18:00:11 <tnurlygayanov_> but I think Rully can do it too
18:00:20 <hughsaunders> boris-42_: should probably wrap whole script execution, not just result processing, like in-instance-scenario
18:00:26 <tnurlygayanov_> if we will not delte VMs after the script running
18:00:27 <boris-42_> let's go to rally chat
18:00:31 <stevemar> howdy marekd
18:00:32 <boris-42_> time is out
18:00:36 <tnurlygayanov_> ok
18:00:37 <marekd> stevemar: hey!
18:00:38 <boris-42_> #endmeeting