17:10:16 <boris-42> #startmeeting rally
17:10:17 <openstack> Meeting started Tue Dec 17 17:10:16 2013 UTC and is due to finish in 60 minutes.  The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:10:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:10:21 <openstack> The meeting name has been set to 'rally'
17:10:42 <boris-42> msdubov ping
17:10:46 <msdubov> boris-42 hi
17:10:49 <boris-42> jaypipes ping
17:10:56 <boris-42> harlowja_away ping
17:11:28 <boris-42> Alexei_987 ping
17:11:31 <Alexei_987> pong
17:12:10 <boris-42> okay let's start
17:12:27 <boris-42> #topic Benchmark periodic execution
17:12:44 <jaypipes> o/
17:12:55 <boris-42> msdubov pls could you explain what you done and what is the goal and status
17:14:28 <msdubov> boris-42 Well the patch has been actually pending for review for quite a while, but I've rewritten it so that it now launches benchmark scenarios in separate threads and counts periods between two consecutive scenario STARTS, which is different from the previous implementation where the engine counted the period between the end of one scenario and the beginning of the next one
17:14:46 <msdubov> Besides, threading allows the benchmark engine to be error-resistant
17:15:00 <msdubov> so if one of the periodic launches fails, the work still continues
17:15:26 <boris-42> msdubov could you share with url of patch?
17:15:27 <msdubov> Now the patch has been tested on Servers.boot_and_delete()
17:15:31 <msdubov> sure
17:15:38 <msdubov> https://review.openstack.org/#/c/57628/
17:15:41 <boris-42> msdubov did you try to set really small period?
17:15:56 <boris-42> msdubov tol get a lot of threads ?
17:16:29 <msdubov> boris-42 Nope, I tested it with 30 seconds
17:16:39 <msdubov> Is there a well-known boundary for the number of threads?
17:17:00 <Alexei_987> ^ unsigned int lol
17:17:06 <boris-42> msdubov idk, but just try for example to run start_stop every 5 seconds
17:17:26 <boris-42> msdubov 500times
17:17:37 <msdubov> Alexei_987 in Python that shouldn't be a problem?
17:17:43 <msdubov> boris-42 Ok thanks for the advice
17:17:59 <Alexei_987> threads are OS dependent. anyway should be enough for us :)
17:18:12 <msdubov> Alexei_987 Now I see
17:18:27 <boris-42> redixin could you take a look at that patch also ^
17:18:41 <msdubov> boris-42 I think we could just increase the period programmatically to, say, 5 seconds, if it is set to 0.0001 for example
17:18:53 <boris-42> msdubov okay
17:18:53 <msdubov> to avoid such problems
17:19:49 <boris-42> #topic Stress tests
17:20:27 <boris-42> #link https://blueprints.launchpad.net/rally/+spec/benchmark-stress-test-execution
17:20:36 <boris-42> There is already blueprint about it
17:20:48 <msdubov> This is what I'm working on now
17:20:57 <boris-42> So the idea is to simplify stress testing of OpenStack
17:21:24 <boris-42> instead of writing each benchmark with specific "concurrency" we will have more compact record for this
17:21:33 <boris-42> as in a description of BP
17:21:53 <boris-42> msdubov are you going to finish it by the end of next week?
17:22:01 <msdubov> boris-42 Yes
17:22:05 <boris-42> msdubov okay nice
17:22:11 <boris-42> msdubov so next time we will discuss it=)
17:22:24 <boris-42> #topic Fuel based deployer
17:22:31 <boris-42> redixin ping
17:22:43 <boris-42> redixin could you share with us about Fuel based deployer
17:22:55 <redixin> there is 2 patches
17:23:08 <redixin> add Fuel client https://review.openstack.org/#/c/59943/
17:23:21 <redixin> add Fuel engine https://review.openstack.org/#/c/61963/
17:23:36 <boris-42> redixin does that work?
17:23:43 <boris-42> =)
17:23:48 <redixin> yep. with Fuel-4.0-dev
17:24:10 <redixin> which isnt released yet
17:24:20 <boris-42> redixin could you some details and limitations of Fuel deployer?
17:24:39 <boris-42> redixin for example how long it takes to deploy 1 node OS
17:24:51 <boris-42> redixin how long in takes to deploy multimode
17:25:06 <redixin> Fuel is refusing to deploy 1 node OS. 2 nodes minimum
17:25:11 <redixin> it takes about 30 min
17:25:40 <redixin> there is no limitations. only limitation is Fuel.
17:26:10 <boris-42> redixin but we are not able to deploy OS from custom code?
17:26:16 <boris-42> redixin I mean custom nova/cinder/..
17:27:06 <redixin> yes. we can deploy with this provider only whay possible to do with Fuel itself
17:27:27 <boris-42> redixin eh we will need to extend this
17:27:30 <redixin> Fuel can't do this, so FuelEngine also can't do that
17:28:13 <redixin> boris-42: indeed
17:28:30 <boris-42> redixin so with Fuel Engine we are able to deploy on servers provided by any existing server provider multimode OpenStack installation in HA?
17:28:42 <redixin> no
17:28:53 <boris-42> redixin so there are some limitation?
17:28:59 <boris-42> limitations*
17:29:04 <redixin> it can deploy on servers only provisioned by Fuel
17:29:29 <redixin> I said only limitation is Fuel. We can only do things than can do Fuel itself
17:30:01 <redixin> This engine is just client for FuelWeb
17:30:02 <boris-42> redixin but it is useless then, because developers don't have own baremetall servers
17:30:26 <redixin> It is not useless if they have
17:30:36 <boris-42> redixin I thought that we are using only deploy part of FUEL
17:30:41 <boris-42> redixin without provision
17:30:55 <redixin> boris-42: not yet
17:31:06 <boris-42> redixin what means "not yet"
17:31:13 <boris-42> redixin it is the FUEL or our problem?
17:32:33 <redixin> boris-42: theoretically Fuel api can do this, but I not tested it yet
17:33:03 <boris-42> redixin could you test it, because it is super interesting for developers
17:33:11 <boris-42> redixin ?
17:33:20 <redixin> boris-42: sure
17:33:37 <boris-42> redixin thanks
17:33:51 <boris-42> #topic Improvements in DevStack engine
17:34:06 <boris-42> redixin I have couple of thoughts about current DevStack Engine
17:34:23 <boris-42> redixin at this moment we are able to specify only the repository for devstack
17:35:00 <boris-42> redixin but it is extra useful to be able to specify something like take my NovaRepo and all others from current master
17:35:12 <boris-42> redixin so you will be able to get clouds with your patches
17:35:34 <redixin> boris-42: it is possible to build totally custom localrc
17:35:51 <redixin> and in localrc we can specify NOVA_REPO
17:36:08 <boris-42> redixin could you add tutorial/readme/samples about how to do it in rally
17:36:33 <boris-42> redixin we should make it simpler & cleaner then now
17:37:05 <boris-42> redixin ok?
17:37:31 <redixin> boris-42: it is already there AFAIR
17:37:38 <boris-42> redixin where what how?
17:38:23 <boris-42> redixin i see only samples/deployemnts/ directory
17:38:35 <redixin> https://wiki.openstack.org/wiki/Rally/DeployEngines#Configuration_Example_2
17:38:37 <boris-42> redixin and there is nothing about custom Nova
17:38:45 <redixin> see NOVA_REPO in localrc
17:39:14 <boris-42> redixin this should be in rally/doc/samples/deployments as well IMHO
17:39:27 <redixin> boris-42: ok
17:39:38 <boris-42> redixin okay it is nice that we have support of such functionallity
17:39:44 <boris-42> redixin next topic
17:39:55 <boris-42> #topic OpenStack Server provider
17:40:23 <boris-42> When I run e.g. DevStack Engine with OpenStack Server provider
17:41:13 <boris-42> I saw that I don't have access to horizon
17:41:41 <boris-42> neither via fixed neither via floating IP
17:41:46 <redixin> boris-42: oh yes. it is caused by default access rules
17:41:54 <redixin> only port 22 is allowed by default
17:41:59 <boris-42> redixin yep yep
17:42:10 <boris-42> redixin so could we fix this? to make it work out of box?
17:42:25 <redixin> boris-42: why dont you filed bug?
17:42:40 <boris-42> redixin okay I will file it
17:42:53 <boris-42> okay then next topic
17:43:00 <boris-42> #topic Benchmarks for Keystone
17:43:54 <boris-42> So there is a big interest in testing keystone performance with Rally
17:44:07 <boris-42> As we can see from this recent thread http://lists.openstack.org/pipermail/openstack/2013-December/003947.html
17:44:39 <boris-42> This wiki page https://blueprints.launchpad.net/rally/+spec/keystone-benchmark
17:44:51 <boris-42> So it is reasonable to use for such things Rally
17:44:57 <boris-42> instead of bash scripts
17:45:11 <boris-42> So today we get new BP https://blueprints.launchpad.net/rally/+spec/keystone-benchmark
17:45:32 <boris-42> and I am working on INIT patches that will allow us to write benchmarks for keystone
17:45:39 <msdubov> boris-42 You've probably posted a wrong link to wiki
17:46:35 <boris-42> msdubov this one https://wiki.openstack.org/wiki/KeystonePerformance
17:46:54 <boris-42> so we should make a couple of thing to simplify this process
17:47:05 <boris-42> Fix user_init methods
17:47:20 <boris-42> Add admin_init mechanism
17:47:47 <boris-42> Add cleanups for keystone benchmarks (we should delete all created users/projects/...)
17:48:06 <msdubov> boris-42 I'd suggest to leave init() methods named init(), not user_init()
17:48:20 <msdubov> boris-42  Sad that we'll have to introduce additional cleanups
17:48:28 <msdubov> This will make everything more complex
17:48:33 <boris-42> msdubov I think that user_init() admin_init() will be much more simple for understaning
17:48:46 <boris-42> msdubov we only need to improve generic cleanups
17:49:10 <boris-42> msdubov and add some rules about naming resources when we are testing keystone
17:49:20 <msdubov> boris-42 We somehow now need to track created resources?
17:49:23 <boris-42> msdubov like rally-keystone-<uuid>
17:49:32 <boris-42> msdubov no I don't like idea of tracking created users
17:49:35 <msdubov> boris-42 Ok just by using naming rules? That's an oprion
17:49:43 <msdubov> *option
17:49:56 <boris-42> msdubov yep and this hardcoded pattern will be in keystone utils
17:50:15 <boris-42> msdubov so if you create keystone resources through it you don't have any problems
17:50:19 <boris-42> with cleanups
17:50:38 <msdubov> boris-42 I see
17:50:45 <boris-42> msdubov so after this will be done we will be able to add keystone benchmarks
17:50:55 <boris-42> okay seems enough for this topic
17:51:04 <boris-42> #topic OpenStack Profiling
17:51:10 <boris-42> Alexei_987 ping
17:51:17 <Alexei_987> pong
17:51:23 <boris-42> Alexei_987 could you share the latest thoughts and ideas about profiling
17:51:43 <Alexei_987> well currently we have 2 patches: 1 in ceilometer and 1 in oslo waiting for review
17:51:56 <Alexei_987> meanwhile we are working on patches for clients and nova
17:52:04 <Alexei_987> that add profiling capabilities
17:52:23 <boris-42> Alexei_987 what about clients, when we could expect some patches?
17:52:31 <Alexei_987> for clients we'll have to add something like an event framework
17:52:31 <boris-42> Alexei_987 on review
17:52:41 <Alexei_987> based on nova/hooks
17:53:03 <Alexei_987> for this we'll move hooks nova -> olso -> clients
17:53:05 <boris-42> Alexei_987 so we should just move nova python clients hooks from it to oslo
17:53:11 <boris-42> and then back
17:53:14 <Alexei_987> yes
17:53:15 <boris-42> to all clients
17:53:21 <boris-42> Seems like not so super hard
17:53:25 <Alexei_987> it's currently in progress
17:53:27 <boris-42> So do we have already BP?
17:53:34 <boris-42> could you share with link
17:53:35 <boris-42> ?
17:53:45 <Alexei_987> I haven't created a BP for this yet
17:54:08 <boris-42> Alexei_987 okay pls could you creat it tomorrow at morning?
17:54:13 <Alexei_987> ok
17:54:29 <boris-42> Alexei_987 thanks
17:54:34 <Alexei_987> and for nova I hope that you'll help me with proper patches architecture
17:54:48 <boris-42> And also we have one more thing it is visualization of logs
17:54:58 <Alexei_987> cause it requires some solution for dependency injection problem
17:55:06 <boris-42> Alexei_987 ?
17:55:22 <Alexei_987> tie all objects together :)
17:55:35 <boris-42> Alexei_987 objects?
17:55:42 <Alexei_987> profiler + client + other objects
17:55:50 <boris-42> yep
17:55:54 <boris-42> it is not simple task
17:56:02 <boris-42> So here is the part of Rally
17:56:03 <boris-42> http://pavlovic.me/drawer.html
17:56:06 <boris-42> visualisation
17:56:17 <boris-42> I think during this week I will finish work around integrating it into Rally
17:56:25 <boris-42> So we will need only patches in clients
17:56:33 <boris-42> patches for 2 core projects
17:56:36 <boris-42> e.g. nova/glacne
17:56:40 <boris-42> to show the demo
17:56:47 <boris-42> Alexei_987 yep?
17:57:01 <Alexei_987> even 1 project should be enough for start
17:57:08 <Alexei_987> glance is not required I guess :)
17:57:14 <boris-42> Alexei_987 we need 2
17:57:23 <Alexei_987> ok
17:57:25 <boris-42> Alexei_987 to show that cross projects requests works
17:57:41 <boris-42> Alexei_987 not only cross nova services.
17:57:59 <Alexei_987> well we'll be able to show it to display cross request rally -> nova
17:58:14 <boris-42> Alexei_987 it is not enough for public=)
17:58:30 <boris-42> okay seems like times come to the end
17:58:34 <boris-42> #topic free discussions
17:58:42 <boris-42> Any questions/thougths?
17:59:49 <boris-42> Ok if somebody have any questions just ask it in  #openstack-rally channel
17:59:53 <boris-42> #endmeeting