17:00:32 <boris-42_> #startmeeting Rally
17:00:33 <openstack> Meeting started Tue Feb  4 17:00:32 2014 UTC and is due to finish in 60 minutes.  The chair is boris-42_. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:36 <openstack> The meeting name has been set to 'rally'
17:00:40 <msdubov> hi
17:01:06 <boris-42> nkhare hi
17:01:13 <nkhare> boris-42, hi
17:01:15 <boris-42> akscram meeting time
17:01:23 <nkhare> hello everyone
17:01:28 <miarmak> hi
17:01:32 <boris-42> so okay nice to see you guys let's disscuss what we have
17:01:52 <boris-42> 1. Work around Rally WIKI & docs
17:02:05 <boris-42> 2. Scenario runner refactoring
17:02:17 <boris-42> 2. Ssh refactoring
17:02:46 <boris-42> 4. Mulitnode / LXC engines
17:03:00 <boris-42> 5. Atomic operation times
17:03:32 <boris-42> 6. Free discussion
17:03:48 <boris-42> #topic 1. Work around Rally WIKI & docs
17:04:10 <boris-42_> #topic 1. Work around Rally WIKI & docs
17:04:14 <boris-42_> lol=)
17:04:33 <boris-42_> So I was concentrate on first page of Rally wiki page
17:04:41 <boris-42_> I added tons of different diagrams https://wiki.openstack.org/wiki/Rally
17:05:01 <boris-42_> and will continue adding to describe how Deploy stuff and Benchmark stuff works in details
17:05:19 <boris-42_> Then I am going to refactor Join Rally team (part)
17:05:35 <boris-42_> to describe in more human readable form how to join Rally team
17:05:53 <boris-42_> and then I will concentrate on pages about installation and usage of rally
17:06:12 <boris-42_> we have couple of new features that are not covered by wiki
17:06:23 <boris-42_> e.g. --use stuff / deploy engines (e.g. devstack)
17:06:38 <boris-42_> atomic times and how to write benchmarks
17:06:44 <boris-42_> and make it overall more clear and simple
17:06:48 <redixin> DevstackEngine is not covered by wiki?
17:07:17 <boris-42_> redixin it is not covered in that way that somebody will see it and say YEP I would like to use this stuff
17:07:18 <boris-42_> =)
17:07:53 <boris-42_> so I will try to refactor all this stuff
17:08:16 <boris-42_> does anybody has any question?
17:08:59 <redixin> no
17:09:10 <boris-42_> ok let's move to next topic
17:09:20 <boris-42_> #topic 2. Scenario runner refactoring
17:09:28 <boris-42_> msdubov could you share pls with updates
17:10:01 <redixin> I saw that patch. It works :)
17:10:07 <boris-42_> redixin lol=)
17:10:10 <boris-42_> redixin amazing=)
17:10:14 <msdubov> So its actually the work started by Boris and continued by me. We've issued two patches on this
17:10:34 <msdubov> 1. https://review.openstack.org/#/c/69886/, where we simplify the original ScenarioRunner class
17:10:49 <msdubov> and move some functionaly for creating users/performing cleanup to separate context classes
17:11:21 <msdubov> 2. https://review.openstack.org/#/c/70771/, where we re-implement different benchmark launching strategies in subclasses instead of ScenarioRunner methods
17:11:28 <boris-42_> msdubov actually it is not simplification (as well we are using context, so now in any case we will perform cleanup)
17:11:30 <msdubov> which allows to eliminate the complicated "if"s logic
17:11:55 <boris-42_> msdubov that was serious bug actually as well..
17:12:02 <msdubov> boris-42_ Yep but it is now e.g. not called in multiprocessing Pool, so it simplifies everything a bit
17:12:41 <msdubov> I tested these patches on devstack (launching NovaServers.boot_and_delete_server with small load) and it worked for me
17:12:49 <msdubov> So pls review those pathces
17:13:09 <msdubov> As the next step I need to rebase lots of my code pending review on this
17:13:14 <boris-42_> hughsaunders julienvey stannie  could you take a look at those patches as well?
17:13:30 <msdubov> For example, the DummyEngine refactoring which allows it to work with predefined users instead of generated ones
17:14:24 <boris-42_> msdubov ok thanks for updates
17:14:38 <boris-42_> #topic 	 3. Ssh refactoring
17:14:47 <redixin> https://review.openstack.org/#/c/68063/
17:14:53 <redixin> approve it pls =)
17:15:00 <boris-42_> redixin could you explain in couple of words what you done?)
17:15:14 <boris-42_> redixin and how to test it (for others)
17:16:12 <redixin> just cherry-pick it, and try to make anything that uses ssh access (deploy or run-task-in-instance scenario)
17:16:25 <boris-42_> redixin ok will do ^
17:16:33 <boris-42_> redixin and why this change is required?
17:17:01 <redixin> it is not required, but with this patch using ssh in not painful
17:17:14 <boris-42_> redixin ok =))
17:17:19 <boris-42_> redixin so just cleanup of code
17:17:28 <redixin> api changes
17:17:45 <boris-42_> #topic 4. Multinode / LXC engines
17:18:02 <redixin> mm this pathces is ready for review
17:18:15 <redixin> https://review.openstack.org/#/c/56222/
17:18:15 <redixin> https://review.openstack.org/#/c/57240/
17:18:47 <redixin> tons of test deployment was done with this engines
17:18:56 <boris-42_> redixin ok I will review code
17:18:59 <boris-42_> redixin one more time
17:19:06 <boris-42_> redixin and merge if everything is ok
17:19:17 <redixin> ok thanks
17:19:30 <boris-42_> to others these patches allows us to create Multinode deployments
17:19:41 <boris-42_> at this moment we are supporting only DevStack
17:19:51 <boris-42_> but in future there will be as well Fuel
17:20:00 <redixin> btw
17:20:09 <redixin> FuelEngine is ready for teview
17:20:19 <redixin> https://review.openstack.org/#/c/61963/
17:20:19 <boris-42_> and with LXCEngine we could get very very fast cloud at big scale on small amount of servers
17:20:22 <boris-42_> (even one)
17:20:38 <boris-42_> something like 200mb RAM is required per 1 compute node
17:21:03 <boris-42_> #topic 5. Atomic operation times
17:21:18 <boris-42_> stannie hi, could you share updates?
17:21:30 <boris-42_> stannie btw are you here?)
17:21:32 <redixin> I thought this work was completed
17:22:08 <boris-42_> so seems like stannie is not yet here
17:22:16 <boris-42_> so let me share with updates
17:22:37 <boris-42_> we successfully add support of atomic times for actions
17:22:47 <boris-42_> this allows us to measure times of all atomic actions
17:22:56 <boris-42_> e.g. in Nova.snapshot scneario
17:23:26 <boris-42_> we have next actions: 1) boot VM, snapshot VM, delete VM, start from snapshot VM, delete snapshot, delete VM
17:23:48 <boris-42_> so it becomes really unclear how long works each of operations
17:24:28 <boris-42_> so with this patch https://review.openstack.org/#/c/69828/
17:24:41 <boris-42_> we will store/mesure all times of all atomic actions in DB
17:24:57 <boris-42_> and this one will allow us to see them in CLI https://review.openstack.org/#/c/70362/
17:25:12 <boris-42_> So this finish works around atomic actions
17:25:29 <boris-42_> nkhare are you around?
17:25:40 <nkhare> yes
17:25:48 <boris-42_> #topic Keystone benchmarking
17:26:00 <boris-42_> nkhare could you share some info about your work around keystone benhcmarking
17:26:31 <nkhare> sure. with the current code in review I was able to do create user benchmark
17:27:09 <nkhare> currently writing down the benchmark for pki vs UUID
17:27:46 <boris-42_> nkhare and you are add as well "authentication" benchhmark?
17:27:53 <boris-42_> you added*
17:27:59 <nkhare> yes
17:28:44 <boris-42_> nkhare nice
17:28:54 <nkhare> would be writing down the user-story section once I finish PKI vs UUID benchmark
17:29:04 <nkhare> for keystone
17:29:08 <boris-42_> nkhare ok nice, it will be very interesting to see
17:29:46 <boris-42_> nkhare btw one small question about patch, do we actually need uitls and authenticate files
17:30:05 <boris-42_> nkhare I mean in this case they could be just merged
17:30:14 <boris-42_> nkhare in one single file authenticate
17:31:20 <nkhare> boris-42_, I don't recall why I did that may be just following other benchmarks
17:31:31 <nkhare> we can merge them
17:31:45 <boris-42_> nkhare yep but in another cases there is a reason
17:32:08 <boris-42_> nkhare because there are atomic actions (e.g. create_volume, delete_volume) and you would like to build on top of them scenarios
17:32:46 <boris-42_> nkhare and in this case I don't see any reason to make it complex
17:33:08 <nkhare> boris-42_, as we are evolving the keystone benchmarks so may be we need but for now we can merge them
17:33:44 <hughsaunders> could authenticate be part of the keystone scenario?
17:33:49 <boris-42_> hughsaunders +1
17:34:56 <nkhare> boris-42_, as we discussed earlier we might might want to authenticate with other services.
17:35:30 <nkhare> so we decided to have as different scenario
17:36:32 <boris-42_> nkhare hughsaunders  actually yes it is not absolutely 100% keystone stuff… in case when we are using pyhon clients from different project..
17:36:47 <hughsaunders> ok
17:37:17 <boris-42_> nkhare so i think that it will be enough to merge utils with authenticate (because I don't see any complex logic)
17:37:23 <boris-42_> nkhare that should be splitted
17:37:48 <nkhare> boris-42_, ok. I'll merge them
17:37:56 <boris-42_> nkhare thanks
17:38:55 <boris-42_> #topic free discussion
17:39:07 <redixin> rally deployment use VS rally use deployment
17:39:10 <boris-42_> does anybody has something to say?) some ideas? any questions?)
17:39:32 <miarmak> yes. Tempest verification =)
17:39:35 <boris-42_> redixin holy-war!!!
17:39:40 <boris-42_> ouch
17:39:42 <boris-42_> ffff
17:39:42 <hughsaunders> redixin: which do you prefer?
17:39:43 <redixin> I always do 'rally deployment use deploy_id' >_<
17:40:14 <boris-42_> I don't use always "use", but when I use use ..
17:40:17 <hughsaunders> haha
17:40:22 <redixin> -_-
17:40:43 <hughsaunders> maybe we could have a cli alias for redixin
17:41:29 <redixin> miarmak: what about tempest?
17:41:47 <boris-42_> #topic tempest integration
17:41:51 <hughsaunders> though that would involve duplicating all the args.
17:41:52 <boris-42_> sdague hi!=)
17:41:52 <miarmak> There is a bp: https://blueprints.launchpad.net/rally/+spec/tempest-verification
17:42:23 <boris-42_> miarmak btw I will add some extra info=)
17:42:49 <miarmak> boris-42_: in the bp description?
17:43:03 <boris-42_> miarmak yep about a lot of tasty stuff
17:43:20 <miarmak> boris-42_: ok, it would ve nice)
17:43:33 <miarmak> now, there is only 1 string)
17:43:34 <boris-42_> miarmak so I mean as first step we are going to add command like rally verify install (that installs tempest)
17:43:49 <boris-42_> miarmak and rally verify run (that runs tempest against cloud)
17:44:01 <miarmak> yeah, There is wip patch: https://review.openstack.org/#/c/70131
17:44:06 <miarmak> please, take a look
17:44:16 <boris-42_> redixin msdubov hughsaunders  killem'all
17:44:24 <boris-42_> I mean that patch=)
17:44:39 <miarmak> =)
17:44:51 <boris-42_> So let me share my new ideas about tempest verification
17:44:58 <boris-42_> 1) We should store results in DB
17:45:40 <boris-42_> 2) We should be able to run only sets of tests (that are already procreated), e.g. rally verify nova/cinder/small/medium/big/latest_failed
17:46:28 <boris-42_> 3) we should add command rally verify list/show (that will list all verification stuffs, and  show that will show detailed info and results)
17:47:02 <boris-42_> 4) And one thing that will be super interesting for catching races rally verify run -N <sets>
17:47:14 <boris-42_> that will run N times specified sets
17:47:29 <boris-42_> that's all
17:47:43 <hughsaunders> could tempest be a special case of a benchmark?
17:47:56 <boris-42_> hughsaunders how?)
17:48:13 <redixin> run tempest test N times and measure times
17:48:17 <redixin> as usual
17:48:24 <hughsaunders> yeah
17:48:30 <boris-42_> hughsaunders yep why not
17:48:46 <boris-42_> hughsaunders tempest as a becnhmark scenario lol=)
17:49:13 <hughsaunders> same mechanism, for an alternative purpose
17:49:15 <boris-42_> btw then probably part of verification logic could be moved to benchmark scneario?
17:49:38 <boris-42_> so it could be parametrized
17:49:44 <boris-42_> and we are able to specify times
17:49:49 <boris-42_> and even make a load
17:50:03 <boris-42_> okay seems like a nice idea
17:50:20 <boris-42_> I will add this to BP
17:50:28 <boris-42_> any other great ideas hughsaunders  redixin ?
17:50:39 <redixin> no
17:51:23 <hughsaunders> nope
17:51:24 <boris-42_> #topic holy-wars
17:51:28 <hughsaunders> haha
17:51:39 <hughsaunders> vim +1
17:51:44 <boris-42_> ffff
17:51:49 <boris-42_> sublime is better
17:51:52 <miarmak> PyCharm
17:52:01 <msdubov> "deployment use" vs. "use deployment"
17:52:06 <redixin> cat | sed | awk
17:52:10 <hughsaunders> jq
17:52:23 <boris-42_> omg maniacs!
17:52:31 <boris-42_> nano!!
17:52:39 <boris-42_> the most awful stuff
17:52:41 <boris-42_> =)
17:52:44 <hughsaunders> msdubov: use deployment makes sense, once use is implemented for tasks
17:52:52 <hughsaunders> then can have use task
17:53:00 <boris-42_> hughsaunders yep
17:53:03 <boris-42_> hughsaunders that was idea
17:53:11 <msdubov> hughsaunders yep I know but I always want to do deployment use as redixin =)
17:53:25 <boris-42_> but there is already --use
17:53:29 <boris-42_> msdubov ^
17:53:39 <boris-42_> rally deployment create --use
17:53:40 <boris-42_> =)
17:53:43 <redixin> --use must be set bu default
17:53:53 <hughsaunders> redixin: I argued for that but was overruled
17:53:55 <boris-42_> --don't_use
17:54:01 <boris-42_> lol
17:54:19 <hughsaunders> however --no-use is a good option
17:54:35 <boris-42_> rally task --burn-your-cloud
17:54:41 <boris-42_> =))
17:54:52 <hughsaunders> maybe will submit a review for --no-use and see if it gets in..
17:55:02 <boris-42_> hughsaunders btw
17:55:10 <boris-42_> hughsaunders I forgot to ask about CONF stuff
17:55:18 <hughsaunders> boris-42_: in progress
17:55:28 <boris-42_> hughsaunders pls don't make one patch on 1k lines=)
17:55:29 <hughsaunders> https://github.com/hughsaunders/rally/compare/review?expand=1
17:55:39 <hughsaunders> thats the cinder one
17:55:52 <hughsaunders> will do similar for nova, then submit review
17:55:54 <boris-42_> hughsaunders ok nice
17:55:59 <boris-42_> btw
17:56:03 <boris-42_> one detail
17:56:09 <boris-42_> hughsaunders you should use group
17:56:17 <boris-42_> registre_group ..
17:56:58 <hughsaunders> ok will do
17:57:11 <boris-42_> hughsaunders and what is the proper name for group?
17:57:25 <hughsaunders> one per benchmark type?
17:57:59 <boris-42_> benchmarks
17:58:20 <hughsaunders> ok, one group "benchmarks" then prefix each option with benchmark type
17:58:22 <boris-42_> and inside nova_crate_sleep, nova_delete_sleep
17:58:27 <boris-42_> yep
17:58:34 <hughsaunders> will do
17:58:37 <boris-42_> hughsaunders thanks
17:58:44 <boris-42_> hughsaunders btw pretty important change=)
17:58:57 <hughsaunders> yeah, meant to get it done today, but will have to be tomorrow now
17:59:04 <boris-42_> hughsaunders np
17:59:18 <boris-42_> hughsaunders btw we will need to make a couple of patches (not only for benchmarks)
17:59:26 <boris-42_> hughsaunders to make them configurable..
17:59:57 <hughsaunders> ?
17:59:57 <boris-42_> ok times to end this meeting
18:00:08 <boris-42_> hughsaunders I will show you
18:00:09 <hughsaunders> can continue tomorrw
18:00:10 * ayoung sneaks in
18:00:12 <hughsaunders> bye
18:00:14 <boris-42_> #endmeeting