17:17:14 <boris-42> #startmeeting Rally
17:17:15 <openstack> Meeting started Tue Mar 10 17:17:14 2015 UTC and is due to finish in 60 minutes.  The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:17:16 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:17:19 <openstack> The meeting name has been set to 'rally'
17:17:33 <boris-42> #topic Rally verify as control plane for all kinds of in tree tests
17:17:50 <boris-42> cdent: so I have somewhere real OpenStack installation
17:18:05 <boris-42> I would like to check that developers and MOS patches didn't change anything in API
17:18:18 <boris-42> and would like to run in-tree tests from Ceilometer against this MOS installation
17:18:24 <boris-42> how can I do this?
17:18:50 <boris-42> Download ceilometer, learn how to run gabbi, how to configure it, parse results, do some stuff
17:18:57 <boris-42> then work on script that will automate this stuf
17:19:20 <boris-42> cdent: ^ so instead of making this, let's just do this automation in Rally like we did for tempest
17:19:23 <boris-42> cdent: makes sense?
17:21:35 <cdent> I see where you're coming from, and I think having a way to do that in a unified fashion across the various projects will be handy. Are you saying rally will handle even the download step?
17:21:59 <boris-42> cdent: download/install/running/parsing/storing results
17:22:06 <boris-42> cdent: and future work with results
17:22:19 <boris-42> cdent: like comparing/trending/generating html reports and so on
17:22:44 <boris-42> cdent: this functionality is mostly implemented, but it's a bit hardcoded to tempest
17:22:56 <boris-42> the idea is just to refactor a bit stuff, that will allow to use any tool
17:22:59 <boris-42> like gabbit
17:23:02 <boris-42> gabii*
17:23:30 <cdent> at the moment gabbi is called from within tox targets, so would you just manage a list of tox targets or do some kind of discovery?
17:24:55 <boris-42> cdent: I belive we can analyze tox.ini
17:25:08 <boris-42> cdent: to discover targets
17:25:22 <boris-42> cdent: or even more bruteforce
17:25:31 <boris-42> cdent: have default value that can be passed
17:26:14 <cdent> Okay, this all makes quite a bit more sense now
17:26:23 <boris-42> cdent: great)
17:26:45 <boris-42> Any questions?
17:27:32 <boris-42> okay let's move to the next topic
17:27:49 <meteorfox> boris-42: I wonder how cloudcafe https://github.com/stackforge/cloudroast could integrate with this
17:28:13 <boris-42> meteorfox: so I talk with those guys
17:28:15 <meteorfox> boris-42: written by Rackspace, it was intended to be better than tempest
17:28:19 <boris-42> meteorfox: in integrates in the same way
17:28:41 <meteorfox> and to replace it, but never caught on, at least in the community, we use it here for everthing
17:28:47 <boris-42> meteorfox: we need to do the same  changes in rally like for gabbi and other projects
17:29:12 <meteorfox> boris-42: alright, sounds good
17:30:31 <boris-42> meteorfox: great
17:30:36 <boris-42> okay let's move to the next topic
17:30:50 <boris-42> #topic THE NEW constant runner
17:31:08 <boris-42> We fully refactored how constant runner works
17:31:29 <boris-42> Now it starts N workers (where N is amount of CPU) and uses in each worker threads to create load
17:31:30 <meteorfox> boris-42: do you have the link to the review?
17:31:35 <boris-42> meteorfox: sure
17:31:50 <meteorfox> thanks
17:32:12 <boris-42> meteorfox: https://review.openstack.org/#/c/155225/
17:32:27 <boris-42> meteorfox: so now you can generate big load =)
17:32:37 <boris-42> meteorfox: I tried 1000 threads and it works perfectly fine
17:33:39 <boris-42> any questions?)
17:34:03 <meteorfox> boris-42: it uses multiprocessing.Process?
17:34:04 <arcimboldo> I think it would be useful to also have a command line option or configuration option to decide how many process to create
17:34:36 <boris-42> meteorfox: yep
17:34:42 <boris-42> arcimboldo: what is the case for that?)
17:34:54 <boris-42> arcimboldo: it's easy to add, just curios
17:35:10 <meteorfox> boris-42: so what do you mean that you tried 1000 threads then?
17:35:12 <arcimboldo> just in case someone is running rally on a machine where other services are running
17:35:31 <boris-42> meteorfox: I just put concurrency: 1000 and it worked well
17:35:35 <meteorfox> 1000 process?
17:35:43 <boris-42> meteorfox: read the begging=)
17:35:55 <boris-42> meteorfox: we refactored constant load to not use only Processes
17:36:12 <boris-42> meteorfox: it starts AMOUNT_OF_CORES Processes and then make load using Threads
17:36:31 <boris-42> meteorfox: I have 4 cores so it creates 4 processes each of them generated 250 threads
17:36:36 <boris-42> meteorfox: and it worked well
17:36:48 <meteorfox> boris-42: ok, cool. those threads will be GIL bound right?
17:36:50 <boris-42> arcimboldo: good case
17:37:05 <meteorfox> but not the processes
17:37:17 <boris-42> arcimboldo: I will add it to wishlist this case
17:37:34 <boris-42> arcimboldo: I will add this case to wishlist**
17:37:42 <boris-42> english so hard=)
17:37:44 <meteorfox> boris-42: it pretty much creates 4 interpreters, and they communicate over IPC
17:38:00 <boris-42> meteorfox: yep
17:38:30 <boris-42> meteorfox: they communicated only master-worker
17:38:57 <meteorfox> boris-42: right, with the join and the queue
17:39:39 <boris-42> meteorfox: so for benchmarking things that are taking > 100ms it's quite good=)
17:39:42 <boris-42> choice=)
17:39:58 <boris-42> meteorfox: in case of OpenStack where things are run for seconds and minutes
17:40:04 <boris-42> it's perfectly fine=)
17:41:00 <meteorfox> boris-42: right. I understand
17:42:35 <meteorfox> boris-42: it'd be interesting how consistent can Rally be, at high number of threads
17:42:41 <meteorfox> to see*
17:42:48 <boris-42> #link https://bugs.launchpad.net/rally/+bug/1430461
17:42:48 <openstack> Launchpad bug 1430461 in Rally "Add to constant and rps runneres limits for maximun Core usage" [Wishlist,Triaged]
17:43:05 <boris-42> arcimboldo: ^
17:43:26 <arcimboldo> thnx boris-42
17:43:34 <boris-42> meteorfox: so actually we will integrate rally profiler
17:43:42 <boris-42> meteorfox: that will answer on questions what is the overhead
17:44:08 <boris-42> meteorfox: in any case Runners are plugins they can be rewritten from scratch
17:44:22 <boris-42> meteorfox: at any moment without affecting rest of the code
17:44:28 <boris-42> meteorfox: like we just did=)
17:44:39 <boris-42> Any other questions?
17:44:51 <meteorfox> boris-42: yeah. that's true. About the RPS runner, is that non-blocking?
17:45:02 <boris-42> meteorfox: non-blocking?
17:45:12 <boris-42> meteorfox: you mean will it create more and more threads?)
17:46:05 <meteorfox> boris-42: say I want to model an open workload, that is I want a consistent injection rate, that regardless if the cloud is slowing down, it will keep that rate
17:46:42 <meteorfox> boris-42: usually I've seen that implemented with async io, where the request is decoupled from the response/result
17:46:54 <boris-42> meteorfox: yep
17:46:59 <boris-42> meteorfox: that is why we did rps runner
17:47:10 <boris-42> meteorfox: it just runs every second more "rps" threads
17:47:22 <boris-42> meteorfox: actually every interval where interval is 1/rps
17:48:08 <meteorfox> boris-42: ok, that sounds a little heavy though.
17:48:55 <boris-42> meteorfox: actually aix is working on finish patch from jlk
17:49:06 <boris-42> meteorfox: https://review.openstack.org/#/c/140150/
17:49:15 <boris-42> meteorfox: that limits maximum amount of concurrent working threads
17:49:30 <boris-42> meteorfox:  as well you have --stop-on-sla-failure key for rally task start
17:49:39 <boris-42> meteorfox: http://boris-42.me/rally-tricks-stop-load-before-your-openstack-goes-wrong/
17:49:56 <boris-42> meteorfox: ^ so if things go mad it will stop load at all
17:50:06 <meteorfox> boris-42: ok ok
17:50:21 <meteorfox> boris-42: anyway, that's all I had thanks
17:50:32 <boris-42> meteorfox: great let's move to next topic
17:50:41 <boris-42> #topic Rally in tree-specs
17:50:55 <boris-42> Recently we merged patch that adds Rally in-tree specs and test them
17:51:14 <boris-42> #link https://review.openstack.org/#/c/162433/
17:51:28 <boris-42> ^ we will use them for all big refactoring of Rally
17:51:32 <boris-42> or new features
17:51:55 <boris-42> https://review.openstack.org/#/c/159065/ this is the first spec
17:52:04 <boris-42> Any questions? )
17:52:22 <boris-42> redixin: meteorfox andreykurilin_ msdubov_ ^
17:52:23 <msdubov_> boris-42: What should be added next to the specs?
17:52:34 <boris-42> msdubov_: hm?
17:52:48 <boris-42> msdubov_:  you mean what specs we should add or?
17:52:57 <meteorfox> boris-42: so, are these like acceptance criteria tests?
17:53:03 <msdubov_> boris-42: yes
17:53:20 <meteorfox> boris-42: or acceptance tests
17:54:19 <boris-42> meteorfox: so we have unit test that checks format of new spec
17:54:33 <boris-42> meteorfox: like if there are missing headers or not rst or too longs lines
17:54:38 <boris-42> meteorfox: unit test will fail
17:54:54 <boris-42> msdubov_: your questions is not clear for me=)
17:54:58 <boris-42> question*
17:55:44 <meteorfox> boris-42: ok, so rally will just make sure the specs follows some kind of format?
17:55:51 <boris-42> meteorfox: yep
17:56:06 <boris-42> meteorfox: this format https://github.com/stackforge/rally/blob/master/doc/specs/template.rst
17:56:24 <msdubov_> boris-42: Should we add any specs after this patch?
17:57:29 <meteorfox> boris-42: oh ok. I thought these were like Ruby's Capybara/Cucumber tests
17:57:55 <boris-42> msdubov_: we will see
17:58:06 <boris-42> msdubov_: at least we will do discussion of new Input format
17:58:09 <boris-42> in specs
17:58:31 <meteorfox> boris-42: isn't that a little over-kill. why not a simply python script, called by a git hook?
17:58:45 <boris-42> meteorfox: unit tests are simpler
17:58:47 <boris-42> meteorfox: and unifed
17:59:00 <boris-42> we need to finish meeting
17:59:08 <boris-42> see you in rally chat
17:59:11 <boris-42> #endmeeting