17:06:23 <boris-42> #startmeeting Rally
17:06:24 <openstack> Meeting started Tue Dec 10 17:06:23 2013 UTC and is due to finish in 60 minutes.  The chair is boris-42. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:06:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:06:27 <openstack> The meeting name has been set to 'rally'
17:07:00 <boris-42> msdubov harlowja jaypipes hi
17:07:12 <msdubov> boris-42 hi
17:07:22 <boris-42> jd__ hi
17:09:00 <boris-42> Alexei_987 hi
17:09:09 <Alexei_987> Hi :)
17:09:15 <boris-42> jog0 ping
17:09:43 <boris-42> Ok let's start
17:10:13 <boris-42> #topic Updates in benchmark engine
17:10:22 <boris-42> msdubov can you share with us
17:10:30 <boris-42> msdubov your recent efforts
17:11:15 <msdubov> boris-42 The main update is no doubt the patch implementing the generic cloud cleanup after launching benchmarks implemented by boden
17:11:40 <msdubov> It has been merged today and required me to put a lot of efforts on rebasing my patches
17:12:25 <msdubov> As for my patches, they are implementing the init() procedure, which is a kind of the opposite thing to cleanup. In init(), one can initialize some resources required for a benchmark scenario, e.g. servers
17:12:57 <boris-42> msdubov I saw some concerns from Alexei_987
17:13:01 <msdubov> I also exemplify it with the patch that implements the benchmark scenario for setting and deleting metadata to a predefined (in init()) server
17:13:21 <msdubov> boris-42 Yep they are addressing the way the benchmark engine works currently
17:13:30 <Alexei_987> boris-42: msdubov yes
17:13:37 <boris-42> Alexei_987 do you have some ideas how to improve it?
17:13:47 <msdubov> The restrictions posed by multiprocessing tool used in the benchmark engine lead to a somewhat ugly way of implementing the engine.
17:13:51 <msdubov> *pool
17:13:51 <Alexei_987> boris-42: msdubov our benchmarking system is a lot alike unittest framework
17:14:06 <boris-42> Alexei_987 not sure
17:14:07 <Alexei_987> and I think we should reuse their code and experience
17:14:12 <Alexei_987> why not?
17:14:24 <Alexei_987> they have setUp/tearDown + run
17:14:41 <eyerediskin> why not using threads?
17:14:42 <boris-42> Alexei_987 yep but they don't have billions of temp users
17:14:50 <Alexei_987> otherwise we'll have to reinvent the wheel all over again
17:14:51 <boris-42> Alexei_987 projects
17:14:58 <boris-42> Alexei_987 resources that should be cleaned
17:15:10 <Alexei_987> why do you think they don't?
17:15:11 <boris-42> Alexei_987 and unit test are run only 1 time
17:15:23 <Alexei_987> unittest is used in many various scenarios
17:15:43 <boris-42> Alexei_987 we already use pytest as a base of our benchmark system
17:15:51 <boris-42> Alexei_987 it was made by Eugeniy
17:15:51 <Alexei_987> anyway I'm not talking about using unittest
17:15:53 <msdubov> eyerediskin I suppose threads will pose the same restrictions on the things we can pass to the called methods? The primary problem actually is we can't pass openstack clients to benchmark scenarios
17:16:15 <Alexei_987> I'm talking about that current design that we have has many issues
17:16:23 <boris-42> Alexei_987 like?
17:16:25 <Alexei_987> related to code organization
17:16:30 <Alexei_987> object creation and so on
17:16:52 <boris-42> Alexei_987 and how are you going to avoid object creation using pytest unit test or something other?
17:16:57 <Alexei_987> I've mentioned one of them to msdubov
17:17:11 <Alexei_987> for example classmethod can only be called under certain conditions
17:17:15 <boris-42> Alexei_987 I mean let's start from task
17:17:30 <boris-42> Alexei_987 we have 1 method that is benchmark scenario
17:17:41 <msdubov> Alexei_987 Yep the possible bug you mentioned indeed can take place, but in fact the benchmark engine manages classes in such a way that this cannot happen
17:17:49 <Alexei_987> ok.. I mean that unittest and other big frameworks have certain way and procedure of when and how to create objects
17:18:05 <boris-42> Alexei_987 could you explain
17:18:12 <boris-42> Alexei_987 on current task
17:18:23 <Alexei_987> for example our problem is that we have to pass objects to other process
17:18:30 <Alexei_987> which is bad
17:18:40 <boris-42> Alexei_987 yep but how are you going to fix it?
17:18:53 <Alexei_987> they should be created where they are actually used
17:19:01 <boris-42> Alexei_987 no
17:19:01 <eyerediskin> create object in process instead of passing it?
17:19:02 <Alexei_987> without passing them all over the system
17:19:08 <Alexei_987> yes
17:19:08 <boris-42> Alexei_987 no
17:19:12 <Alexei_987> why not?
17:19:13 <boris-42> Alexei_987 because the case is another
17:19:23 <eyerediskin> just like setUp method in tests
17:19:26 <msdubov> Alexei_987 But we may launch a benchmark scenario millions of times, and we want to create OpenStack clients only once
17:19:39 <Alexei_987> why?
17:19:48 <boris-42> Alexei_987 because we are testing not abstract shit=)
17:19:52 <eyerediskin> objects are created once thread/process is started
17:19:52 <Alexei_987> creation of a python object takes almost 0 ms
17:20:04 <boris-42> Alexei_987 creation of billions users take a lot of time
17:20:09 <eyerediskin> agreed with Alexei_987
17:20:16 <Alexei_987> users yes but not clients
17:20:25 <Alexei_987> you can pass user credential
17:20:28 <Alexei_987> plain dict
17:20:35 <Alexei_987> and create a complex object later
17:20:36 <boris-42> Alexei_987 and Floting IPs FixedIps
17:20:39 <boris-42> Alexei_987 images
17:20:45 <boris-42> Alexei_987 other stuff that you need in init
17:20:54 <boris-42> Alexei_987 also if you put this to thread
17:21:07 <boris-42> Alexei_987 then you will need to create them in each method
17:21:34 <eyerediskin> this stuff created once per process/thread. created only objects needed by process/thread
17:21:36 <Alexei_987> what do you mean by each method?
17:21:37 <boris-42> Alexei_987 eyerediskin  https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/servers.py#L29-L37
17:22:00 <boris-42> Alexei_987 eyerediskin  this method get's random clients in cls
17:22:21 <Alexei_987> clients?
17:22:26 <Alexei_987> I don't see any clients in there
17:22:26 <boris-42> Alexei_987 yep clients
17:22:31 <msdubov> Alexei_987 (chosen randomly of many client users)
17:22:44 <msdubov> Alexei_987 they are used in utils
17:22:50 <Alexei_987> so it chooses a random user credential
17:22:57 <Alexei_987> please be precise
17:23:00 <boris-42> Alexei_987 https://github.com/stackforge/rally/blob/master/rally/benchmark/scenarios/nova/utils.py#L28-L49
17:23:04 <Alexei_987> client is not tied to user
17:23:16 <Alexei_987> the only things that connects it to the user is auth token
17:24:04 <Alexei_987> again let's separate clients and users
17:24:16 <Alexei_987> user -> auth_token is just a string
17:24:16 <boris-42> Alexei_987 again I don't care about users
17:24:20 <boris-42> Alexei_987 in becnhmarks
17:24:22 <Alexei_987> why not?
17:24:26 <boris-42> Alexei_987 I care about clients
17:24:32 <Alexei_987> clients?
17:24:47 <Alexei_987> then I must have missed something
17:24:48 <boris-42> Alexei_987 glacne, nova, keystone, …. clients for this user
17:24:53 <Alexei_987> so...
17:24:58 <Alexei_987> there is a word user
17:25:13 <boris-42> Alexei_987l so I would like to be able cls.clients("nova").servers.create(server_name, image_id,
17:25:14 <boris-42> flavor_id, **kwargs)
17:25:20 <boris-42> cls.clients("nova")
17:25:27 <boris-42> use from benchmark dirrectly
17:25:30 <Alexei_987> yes but user is hidden inside
17:25:37 <boris-42> Alexei_987 yep
17:25:50 <Alexei_987> so we can change code in such a way that we only pass users
17:25:53 <Alexei_987> and not clients
17:26:00 <boris-42> Alexei_987 why
17:26:11 <boris-42> Alexei_987 to create in each method client every time by hand?
17:26:17 <boris-42> Alexei_987 it is called "copy paste"
17:26:22 <Alexei_987> (facepalm)
17:26:33 <Alexei_987> client manager can handle this
17:26:34 <boris-42> Alexei_987 in each benchmark I need clients
17:26:46 <Alexei_987> and reuse 1 client object for 1000 benchmarks
17:26:55 <Alexei_987> just bind it to certain user before usage
17:27:20 <Alexei_987> the main idea is that we have to define data flow
17:27:30 <Alexei_987> and data dependencies
17:27:42 <boris-42> Alexei_987 hm I don't understand
17:27:42 <Alexei_987> so we won't have to apply dirty hacks to make it work
17:28:16 <boris-42> Alexei_987 we need to be able to pass the same data in every thread
17:28:25 <boris-42> Alexei_987 or even more different data
17:28:29 <Alexei_987> why we should do it?
17:28:32 <boris-42> Alexei_987 in every call of method
17:28:39 <Alexei_987> we should only pass data that is used in that thread
17:28:47 <boris-42> Alexei_987 ?
17:28:54 <boris-42> Alexei_987 and it could be all data from INIT
17:29:05 <boris-42> Alexei_987 that is called only 1 time
17:29:15 <boris-42> Alexei_987 like create 20k fixed IPs
17:29:35 <Alexei_987> and 1 tests uses all 20k fixed ips?
17:29:49 <boris-42> Alexei_987 I don't know what will test use
17:30:17 <Alexei_987> ok.. such discussion is not productive at all
17:30:23 <Alexei_987> need additional digging in here
17:30:29 <boris-42> Alexei_987 lol
17:30:39 <boris-42> Alexei_987 we should talk about user case
17:30:51 <boris-42> Alexei_987 my user case was next
17:30:57 <boris-42> Alexei_987 use case*
17:31:37 <boris-42> Alexei_987 then it will be more clean
17:32:03 <boris-42> msdubov could you write etherpad with full description about our use cases and how they are implemented
17:32:16 <boris-42> msdubov Alexei_987  ^ so we will be able to continue this discussion next week
17:32:37 <boris-42> okay let's close this discussion a got to another topic
17:32:41 <Alexei_987> ok :)
17:32:48 <msdubov> boris-42 ok
17:32:55 <boris-42> #topic Profiling system
17:33:15 <boris-42> Alexei_987 could you say for jd__ and jog0 current status of your work
17:33:16 <Alexei_987> cool :)
17:33:29 <Alexei_987> I have 1 patch in ceilometer on review
17:33:40 <Alexei_987> that adds data collector functionality
17:33:57 <Alexei_987> + I have 1 patch in process of polishing in nova
17:34:13 <Alexei_987> that adds library that will collect and send data
17:34:21 <Alexei_987> that's the current status
17:34:34 <Alexei_987> I also have some problems that we can discuss
17:34:50 <boris-42> Alexei_987 could you share with links to the patches?
17:35:06 <Alexei_987> https://review.openstack.org/#/c/60262/
17:35:13 <Alexei_987> https://review.openstack.org/#/c/60797/
17:35:17 <boris-42> Alexei_987 and what problem do you have?
17:35:38 <Alexei_987> so the problem is: how and where do we store a profiler object on single service
17:36:14 <boris-42> Alexei_987 could you explain?
17:36:16 <Alexei_987> this is kind of cross functionality that should be accessible from anywhere
17:36:29 <Alexei_987> but we should not bind it to singleton of any kind
17:36:48 <Alexei_987> in openstack I didn't find a proper way of doing such a thing
17:37:03 <Alexei_987> it's somewhat common with logging system
17:37:16 <Alexei_987> but logger is created for each file separately
17:37:24 <boris-42> Alexei_987 but why we are not able just to put Profiler class to oslo-incubator
17:37:31 <boris-42> Alexei_987 and copy paste it to all projects?
17:37:42 <Alexei_987> nah.. the code will lie in oslo
17:37:55 <boris-42> Alexei_987 and what is the problem?
17:37:59 <Alexei_987> I mean when we create a profiler object: Profiler(service='nova')
17:38:05 <Alexei_987> where do we put it?
17:38:29 <boris-42> You should call it Profiler(service="oslo") and it will be replaced during sync to ("proect_name")
17:38:32 <boris-42> Alexei_987 ^
17:38:47 <boris-42> Alexei_987 and this could be done in oslo code actually
17:38:58 <Alexei_987> hm the name?
17:39:02 <Alexei_987> or the object?
17:39:10 <Alexei_987> name ok..
17:39:12 <boris-42> Alexei_987 all can be stored there
17:39:15 <Alexei_987> ha ha
17:39:24 <Alexei_987> you should ask Roman Po..
17:39:34 <Alexei_987> how much pain they have now cause of global config object
17:39:53 <boris-42> Alexei_987 first of all they have another problmes
17:40:00 <Alexei_987> yes...
17:40:03 <boris-42> Alexei_987 they would like to move the code in separated lib
17:40:09 <Alexei_987> but library should not create any active objects
17:40:10 <boris-42> Alexei_987 that use another libs
17:40:14 <Alexei_987> that's why it's libraryt
17:40:27 <Alexei_987> library should only provide functionality
17:40:45 <boris-42> Alexei_987 but actually we are going to patch with our lib another libs
17:41:00 <boris-42> Alexei_987 like oslo.log, oslo.messegin?
17:41:09 <Alexei_987> oslo.log - no
17:41:14 <Alexei_987> messaging - maybe
17:41:24 <Alexei_987> but mostly it will be used on higher level code
17:41:30 <Alexei_987> such as controllers, etc.
17:42:12 <boris-42> Alexei_987 so I don't see any problem to just take Profiler() code put it into oslo
17:42:19 <Alexei_987> and I do :)
17:42:20 <boris-42> Alexei_987 then sync a create locally in every code
17:42:28 <boris-42> Alexei_987 project*
17:42:34 <boris-42> Alexei_987 and then use it for tracing
17:42:46 <Alexei_987> code yes
17:42:48 <Alexei_987> objects no
17:42:50 <boris-42> Alexei_987 and have 2 new config options (in oslo or projects)
17:42:59 <boris-42> Alexei_987 you don't need to store any objects
17:43:30 <boris-42> Alexei_987 what is the problem?
17:43:40 <boris-42> Alexei_987 I didn't have any problems with using DB code in all projects
17:43:52 <boris-42> Alexei_987 and DB code is much more complex that this 50lines
17:43:59 <boris-42> Alexei_987 and have much more dependencies
17:44:03 <Alexei_987> there are some problems related to ENGINE being a singleton
17:44:13 <boris-42> Alexei_987 Engine?
17:44:22 <Alexei_987> yes.. connection object
17:44:35 <boris-42> Alexei_987 you are speaking now about DB?
17:44:43 <Alexei_987> (facepalm)
17:44:46 <Alexei_987> 1 sec
17:44:51 <boris-42> Alexei_987 omg
17:45:24 <Alexei_987> https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/sqlalchemy/session.py#L356
17:45:27 <Alexei_987> ENGINE
17:45:31 <Alexei_987> global object
17:45:41 <Alexei_987> cause of it we cannot implement slave reads properly
17:46:17 <Alexei_987> anyway...
17:46:33 <Alexei_987> so there is a problem in this object placement..
17:46:44 <Alexei_987> and I suggest to move on for now
17:46:50 <boris-42> Alexei_987 okay
17:46:52 <Alexei_987> so we don't waste all the time
17:47:09 <boris-42> #topic Deploy engines
17:47:21 <boris-42> eyerediskin hey can you share with your updates
17:47:36 <eyerediskin> Fuel engine is coming soon. One or wo days
17:47:45 <boris-42> eyerediskin oh nice
17:48:23 <boris-42> eyerediskin I saw some changes are checking configs
17:48:31 <boris-42> eyerediskin around checking configs*
17:48:41 <eyerediskin> yep, this is almost done
17:49:06 <boris-42> eyerediskin what is left?
17:49:40 <eyerediskin> config validation in providers. Maxim is working on it
17:50:05 <eyerediskin> https://review.openstack.org/#/c/60275/
17:50:09 <eyerediskin> https://review.openstack.org/#/c/60030/
17:50:22 <eyerediskin> some patches waiting for +2 =)
17:51:01 <boris-42> eyerediskin https://review.openstack.org/#/c/60030/ <- +2ed
17:51:22 <boris-42> eyerediskin and how about LXCEngine?
17:51:49 <boris-42> eyerediskin are you working around it, to make it work inside OS cloud?
17:52:02 <eyerediskin> I gonna continue with it after fuel engine is done
17:52:14 <boris-42> eyerediskin okay nice
17:52:41 <boris-42> okay let's move to the next topic
17:52:42 <eyerediskin> boris-42: I'got solution in my mind. Just need to code it
17:52:51 <boris-42> eyerediskin nice
17:53:01 <boris-42> #topic Rally as a Service
17:53:32 <boris-42> We would like to add Web UI, and make it possible to work together with one Rally instances
17:53:44 <boris-42> So the right way is to make from Rally service
17:53:53 <boris-42> and make something like in nova and other projects
17:54:01 <boris-42> we will build REST with pecan
17:54:30 <boris-42> And make API + Managers (or master + workers) using oslo.messeging
17:54:58 <boris-42> I don't think that these decisions requires discussion
17:55:10 <Alexei_987> we'll use the same async approach with eventlet and hookers?
17:55:11 <boris-42> But there is a discussion how to make it better
17:55:19 <boris-42> Alexei_987 ?
17:55:38 <Alexei_987> nonblocking code and all the stuff like in nova
17:55:50 <Alexei_987> in our service
17:56:10 <boris-42> Alexei_987 There will be API that just make plain operation (get data from DB) or stat task and make record in DB
17:56:25 <boris-42> And call managers that will actually run all other stuff
17:56:30 <Alexei_987> ok.. so no async code and no eventlet :)
17:56:31 <Alexei_987> cool
17:56:32 <boris-42> Alexei_987 so we will use rpc.cast calls
17:56:46 <boris-42> Alexei_987 for long operations
17:57:07 <boris-42> Alexei_987 so yep no eventlet shit
17:57:10 <Alexei_987> ok I got it
17:57:23 <boris-42> And we actually started discussion
17:57:31 <boris-42> About how to make this possible inside Rally
17:57:36 <boris-42> and what changes we should make
17:57:55 <boris-42> https://etherpad.openstack.org/p/rally-service-arch
17:57:58 <boris-42> here is it ^
17:58:18 <boris-42> it is just draft not ready so if everyone is interested it can help with advices
17:58:29 <boris-42> So okay
17:58:30 <boris-42> that is all
17:58:35 <boris-42> #topic free discussions
17:58:56 <boris-42> So if somebody have any questions or something else just ask
17:59:30 <eyerediskin> there is blueprint in devstack
17:59:58 <eyerediskin> https://blueprints.launchpad.net/devstack/+spec/devstack-rally
18:00:08 <eyerediskin> meybe someone wants to implement it
18:00:26 <boris-42> eyerediskin could you specify drafter, assignee and so on?
18:00:32 <boris-42> eyerediskin and series
18:00:42 <boris-42> eyerediskin I think that it will be nice to implement it
18:01:04 <boris-42> eyerediskin especially if we are going to make from Rally service
18:01:49 <boris-42> okay let's finish this meeting
18:01:53 <boris-42> #endmeeting