17:10:16 #startmeeting rally 17:10:17 Meeting started Tue Dec 17 17:10:16 2013 UTC and is due to finish in 60 minutes. The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:10:18 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:10:21 The meeting name has been set to 'rally' 17:10:42 msdubov ping 17:10:46 boris-42 hi 17:10:49 jaypipes ping 17:10:56 harlowja_away ping 17:11:28 Alexei_987 ping 17:11:31 pong 17:12:10 okay let's start 17:12:27 #topic Benchmark periodic execution 17:12:44 o/ 17:12:55 msdubov pls could you explain what you done and what is the goal and status 17:14:28 boris-42 Well the patch has been actually pending for review for quite a while, but I've rewritten it so that it now launches benchmark scenarios in separate threads and counts periods between two consecutive scenario STARTS, which is different from the previous implementation where the engine counted the period between the end of one scenario and the beginning of the next one 17:14:46 Besides, threading allows the benchmark engine to be error-resistant 17:15:00 so if one of the periodic launches fails, the work still continues 17:15:26 msdubov could you share with url of patch? 17:15:27 Now the patch has been tested on Servers.boot_and_delete() 17:15:31 sure 17:15:38 https://review.openstack.org/#/c/57628/ 17:15:41 msdubov did you try to set really small period? 17:15:56 msdubov tol get a lot of threads ? 17:16:29 boris-42 Nope, I tested it with 30 seconds 17:16:39 Is there a well-known boundary for the number of threads? 17:17:00 ^ unsigned int lol 17:17:06 msdubov idk, but just try for example to run start_stop every 5 seconds 17:17:26 msdubov 500times 17:17:37 Alexei_987 in Python that shouldn't be a problem? 17:17:43 boris-42 Ok thanks for the advice 17:17:59 threads are OS dependent. anyway should be enough for us :) 17:18:12 Alexei_987 Now I see 17:18:27 redixin could you take a look at that patch also ^ 17:18:41 boris-42 I think we could just increase the period programmatically to, say, 5 seconds, if it is set to 0.0001 for example 17:18:53 msdubov okay 17:18:53 to avoid such problems 17:19:49 #topic Stress tests 17:20:27 #link https://blueprints.launchpad.net/rally/+spec/benchmark-stress-test-execution 17:20:36 There is already blueprint about it 17:20:48 This is what I'm working on now 17:20:57 So the idea is to simplify stress testing of OpenStack 17:21:24 instead of writing each benchmark with specific "concurrency" we will have more compact record for this 17:21:33 as in a description of BP 17:21:53 msdubov are you going to finish it by the end of next week? 17:22:01 boris-42 Yes 17:22:05 msdubov okay nice 17:22:11 msdubov so next time we will discuss it=) 17:22:24 #topic Fuel based deployer 17:22:31 redixin ping 17:22:43 redixin could you share with us about Fuel based deployer 17:22:55 there is 2 patches 17:23:08 add Fuel client https://review.openstack.org/#/c/59943/ 17:23:21 add Fuel engine https://review.openstack.org/#/c/61963/ 17:23:36 redixin does that work? 17:23:43 =) 17:23:48 yep. with Fuel-4.0-dev 17:24:10 which isnt released yet 17:24:20 redixin could you some details and limitations of Fuel deployer? 17:24:39 redixin for example how long it takes to deploy 1 node OS 17:24:51 redixin how long in takes to deploy multimode 17:25:06 Fuel is refusing to deploy 1 node OS. 2 nodes minimum 17:25:11 it takes about 30 min 17:25:40 there is no limitations. only limitation is Fuel. 17:26:10 redixin but we are not able to deploy OS from custom code? 17:26:16 redixin I mean custom nova/cinder/.. 17:27:06 yes. we can deploy with this provider only whay possible to do with Fuel itself 17:27:27 redixin eh we will need to extend this 17:27:30 Fuel can't do this, so FuelEngine also can't do that 17:28:13 boris-42: indeed 17:28:30 redixin so with Fuel Engine we are able to deploy on servers provided by any existing server provider multimode OpenStack installation in HA? 17:28:42 no 17:28:53 redixin so there are some limitation? 17:28:59 limitations* 17:29:04 it can deploy on servers only provisioned by Fuel 17:29:29 I said only limitation is Fuel. We can only do things than can do Fuel itself 17:30:01 This engine is just client for FuelWeb 17:30:02 redixin but it is useless then, because developers don't have own baremetall servers 17:30:26 It is not useless if they have 17:30:36 redixin I thought that we are using only deploy part of FUEL 17:30:41 redixin without provision 17:30:55 boris-42: not yet 17:31:06 redixin what means "not yet" 17:31:13 redixin it is the FUEL or our problem? 17:32:33 boris-42: theoretically Fuel api can do this, but I not tested it yet 17:33:03 redixin could you test it, because it is super interesting for developers 17:33:11 redixin ? 17:33:20 boris-42: sure 17:33:37 redixin thanks 17:33:51 #topic Improvements in DevStack engine 17:34:06 redixin I have couple of thoughts about current DevStack Engine 17:34:23 redixin at this moment we are able to specify only the repository for devstack 17:35:00 redixin but it is extra useful to be able to specify something like take my NovaRepo and all others from current master 17:35:12 redixin so you will be able to get clouds with your patches 17:35:34 boris-42: it is possible to build totally custom localrc 17:35:51 and in localrc we can specify NOVA_REPO 17:36:08 redixin could you add tutorial/readme/samples about how to do it in rally 17:36:33 redixin we should make it simpler & cleaner then now 17:37:05 redixin ok? 17:37:31 boris-42: it is already there AFAIR 17:37:38 redixin where what how? 17:38:23 redixin i see only samples/deployemnts/ directory 17:38:35 https://wiki.openstack.org/wiki/Rally/DeployEngines#Configuration_Example_2 17:38:37 redixin and there is nothing about custom Nova 17:38:45 see NOVA_REPO in localrc 17:39:14 redixin this should be in rally/doc/samples/deployments as well IMHO 17:39:27 boris-42: ok 17:39:38 redixin okay it is nice that we have support of such functionallity 17:39:44 redixin next topic 17:39:55 #topic OpenStack Server provider 17:40:23 When I run e.g. DevStack Engine with OpenStack Server provider 17:41:13 I saw that I don't have access to horizon 17:41:41 neither via fixed neither via floating IP 17:41:46 boris-42: oh yes. it is caused by default access rules 17:41:54 only port 22 is allowed by default 17:41:59 redixin yep yep 17:42:10 redixin so could we fix this? to make it work out of box? 17:42:25 boris-42: why dont you filed bug? 17:42:40 redixin okay I will file it 17:42:53 okay then next topic 17:43:00 #topic Benchmarks for Keystone 17:43:54 So there is a big interest in testing keystone performance with Rally 17:44:07 As we can see from this recent thread http://lists.openstack.org/pipermail/openstack/2013-December/003947.html 17:44:39 This wiki page https://blueprints.launchpad.net/rally/+spec/keystone-benchmark 17:44:51 So it is reasonable to use for such things Rally 17:44:57 instead of bash scripts 17:45:11 So today we get new BP https://blueprints.launchpad.net/rally/+spec/keystone-benchmark 17:45:32 and I am working on INIT patches that will allow us to write benchmarks for keystone 17:45:39 boris-42 You've probably posted a wrong link to wiki 17:46:35 msdubov this one https://wiki.openstack.org/wiki/KeystonePerformance 17:46:54 so we should make a couple of thing to simplify this process 17:47:05 Fix user_init methods 17:47:20 Add admin_init mechanism 17:47:47 Add cleanups for keystone benchmarks (we should delete all created users/projects/...) 17:48:06 boris-42 I'd suggest to leave init() methods named init(), not user_init() 17:48:20 boris-42 Sad that we'll have to introduce additional cleanups 17:48:28 This will make everything more complex 17:48:33 msdubov I think that user_init() admin_init() will be much more simple for understaning 17:48:46 msdubov we only need to improve generic cleanups 17:49:10 msdubov and add some rules about naming resources when we are testing keystone 17:49:20 boris-42 We somehow now need to track created resources? 17:49:23 msdubov like rally-keystone- 17:49:32 msdubov no I don't like idea of tracking created users 17:49:35 boris-42 Ok just by using naming rules? That's an oprion 17:49:43 *option 17:49:56 msdubov yep and this hardcoded pattern will be in keystone utils 17:50:15 msdubov so if you create keystone resources through it you don't have any problems 17:50:19 with cleanups 17:50:38 boris-42 I see 17:50:45 msdubov so after this will be done we will be able to add keystone benchmarks 17:50:55 okay seems enough for this topic 17:51:04 #topic OpenStack Profiling 17:51:10 Alexei_987 ping 17:51:17 pong 17:51:23 Alexei_987 could you share the latest thoughts and ideas about profiling 17:51:43 well currently we have 2 patches: 1 in ceilometer and 1 in oslo waiting for review 17:51:56 meanwhile we are working on patches for clients and nova 17:52:04 that add profiling capabilities 17:52:23 Alexei_987 what about clients, when we could expect some patches? 17:52:31 for clients we'll have to add something like an event framework 17:52:31 Alexei_987 on review 17:52:41 based on nova/hooks 17:53:03 for this we'll move hooks nova -> olso -> clients 17:53:05 Alexei_987 so we should just move nova python clients hooks from it to oslo 17:53:11 and then back 17:53:14 yes 17:53:15 to all clients 17:53:21 Seems like not so super hard 17:53:25 it's currently in progress 17:53:27 So do we have already BP? 17:53:34 could you share with link 17:53:35 ? 17:53:45 I haven't created a BP for this yet 17:54:08 Alexei_987 okay pls could you creat it tomorrow at morning? 17:54:13 ok 17:54:29 Alexei_987 thanks 17:54:34 and for nova I hope that you'll help me with proper patches architecture 17:54:48 And also we have one more thing it is visualization of logs 17:54:58 cause it requires some solution for dependency injection problem 17:55:06 Alexei_987 ? 17:55:22 tie all objects together :) 17:55:35 Alexei_987 objects? 17:55:42 profiler + client + other objects 17:55:50 yep 17:55:54 it is not simple task 17:56:02 So here is the part of Rally 17:56:03 http://pavlovic.me/drawer.html 17:56:06 visualisation 17:56:17 I think during this week I will finish work around integrating it into Rally 17:56:25 So we will need only patches in clients 17:56:33 patches for 2 core projects 17:56:36 e.g. nova/glacne 17:56:40 to show the demo 17:56:47 Alexei_987 yep? 17:57:01 even 1 project should be enough for start 17:57:08 glance is not required I guess :) 17:57:14 Alexei_987 we need 2 17:57:23 ok 17:57:25 Alexei_987 to show that cross projects requests works 17:57:41 Alexei_987 not only cross nova services. 17:57:59 well we'll be able to show it to display cross request rally -> nova 17:58:14 Alexei_987 it is not enough for public=) 17:58:30 okay seems like times come to the end 17:58:34 #topic free discussions 17:58:42 Any questions/thougths? 17:59:49 Ok if somebody have any questions just ask it in #openstack-rally channel 17:59:53 #endmeeting