14:01:45 <boris-42_> #startmeeting Rally
14:01:47 <openstack> Meeting started Mon Dec  7 14:01:45 2015 UTC and is due to finish in 60 minutes.  The chair is boris-42_. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:48 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:50 <openstack> The meeting name has been set to 'rally'
14:01:59 <ikhudoshyn> o/
14:02:04 <oanufriev> hi all
14:02:18 <ylobankov> hi
14:03:14 <boris-42_> bochi-michael: hi
14:03:26 <bochi-michael> boris-42_, hi :)
14:04:16 <chenli> hi
14:05:02 <boris-42_> chenli: hi
14:05:06 <boris-42_> yingjun: hi
14:05:36 <yingjun> boris-42_ hi:)
14:05:43 <boris-42_> #topic Today's Agenda
14:05:52 <boris-42_> #link https://wiki.openstack.org/wiki/Meetings/Rally#Agenda
14:06:02 <boris-42_> #topic [ikhudoshyn] Preparation for a Distributed Runner
14:06:15 <ikhudoshyn> Hi everybody
14:06:18 <boris-42_> #link https://review.openstack.org/#/c/251889/
14:06:22 <boris-42_> ikhudoshyn: hi there
14:07:02 <ikhudoshyn> I'm now working on distributed runner - the addition to Rally that will allow to generate load simultaneously by many nodes
14:07:33 <ikhudoshyn> Before implementing distributed runner itself I expect two preparation steps
14:08:06 <ikhudoshyn> 1. Runner should return chunks (or batches) of TaskResult's rather then individual ones
14:08:43 <ikhudoshyn> 2. Runner should aggregate SLA data and return SLAChecker instances
14:09:03 <ikhudoshyn> by the link https://review.openstack.org/#/c/251889/ there is a patch implementing step #1
14:09:36 <ikhudoshyn> While I'mm working on #2 I'd like to encourage everybody to take a look at it (#1)
14:09:38 <ikhudoshyn> eom
14:09:46 <ikhudoshyn> any questions?
14:10:16 <boris-42_> ikhudoshyn: so we will need to do SLAChecker instances pickable?
14:10:24 <ikhudoshyn> yes
14:10:24 <boris-42_> ikhudoshyn: we are going to send them via network in case of distributed runner?
14:10:32 <ikhudoshyn> yes, exactly
14:11:23 <chenli> ikhudoshyn:  how runners talk with rally ?
14:11:45 <ikhudoshyn> right now Rally support only local runners
14:12:08 <boris-42_> ikhudoshyn: I believe you should send link to your spec
14:12:13 <ikhudoshyn> Runner communicates with TaskEngine via queue
14:12:15 <boris-42_> laurelm: to chenli
14:12:31 <ikhudoshyn> boris-42_, sure, I'll find it right now
14:13:10 <chenli> ikhudoshyn:   queue ? message queue ? RPC like other projects ?
14:13:14 <ikhudoshyn> https://github.com/openstack/rally/blob/master/doc/specs/in-progress/distributed_runner.rst
14:13:30 <boris-42_> chenli: nope
14:13:36 <ikhudoshyn> chenli, no just plain multiprocessing.Queue
14:13:54 <ikhudoshyn> (or it might be Queue.Queue, I forgot)
14:14:00 <chenli> ikhudoshyn: o.... thanks for the link.
14:14:13 <boris-42_> chenli: it's just Queue
14:16:01 <chenli> it is said : communicates with remote RunnerAgents wia message queue (ZeroMQ)
14:16:19 <boris-42_> chenli: yep
14:16:20 <chenli> so the "ZeroMQ" is not the same as rabbitMQ
14:16:32 <boris-42_> chenli: currently we have only Local case
14:16:52 <boris-42_> chenli: it's just few processes
14:17:04 <chenli> boris-42_: o.
14:17:05 <boris-42_> chenli: that are communicating via shared memory
14:17:18 <boris-42_> chenli: using multiprocessing.Queue
14:17:27 <boris-42_> chenli: which you can see from rally.task.engine module
14:17:52 <chenli> boris-42_:  ok
14:19:01 <boris-42_> chenli: so the goal of work that we are working now is to create a distributed Runner
14:19:24 <boris-42_> chenli: it means that we will be able to consume many physical hosts to generate load
14:21:47 <boris-42_> so seems like we have only one item in current agenda
14:21:53 <boris-42_> #topic OpenDiscussion
14:22:06 <boris-42_> somebody has something to discuss?
14:22:09 <ikhudoshyn> there is one more item from my side
14:22:13 <ikhudoshyn> https://review.openstack.org/#/c/253659/
14:22:50 <ikhudoshyn> I'd like other guys join the discussion
14:23:15 <ikhudoshyn> it's a rather small issuse about coding style
14:23:48 <ikhudoshyn> I expected Tom (the author) to join our meeting but looks like he misunderstood me
14:24:28 <ikhudoshyn> putting the issue here is a lil'bit tedious so please come and review
14:24:50 <ikhudoshyn> eom
14:26:17 <boris-42_> ikhudoshyn: looking at it
14:27:15 <boris-42_> ikhudoshyn: so we already have such places
14:27:34 <boris-42_> ikhudoshyn: and we need to do such changes in backward compatible way
14:27:35 <boris-42_> ikhudoshyn: otherwise we will be break installations
14:27:54 <boris-42_> ikhudoshyn: in any case we need to create some guide about how to write plguins
14:27:57 <boris-42_> with such rules
14:28:24 <ikhudoshyn> boris-42_, i see
14:28:26 <boris-42_> ikhudoshyn: however why not using kwargs word?
14:28:35 <boris-42_> ikhudoshyn: I mean cinder_kwargs, nova_kwargs
14:28:40 <boris-42_> ikhudoshyn: seems ok for me
14:28:50 <ikhudoshyn> well my 'kwargs' point is secondary
14:29:27 <ikhudoshyn> I was against using cinder_kwargs for cinder and just **kwargs for nova
14:30:12 <ikhudoshyn> i.e. call method(cinder_kwargs={'param': 'val'}, nova_kwarg='param') looks ugly for me
14:30:40 <ikhudoshyn> i'd like it method(cinder_kwargs={...}, nova_kwargs={..})
14:30:57 <chenli> ikhudoshyn: +1
14:30:59 <boris-42_> so you wrote the same=)
14:31:07 <ikhudoshyn> boris-42_, nope
14:31:14 <boris-42_> ah I see nova_kwargs=''
14:31:17 <ikhudoshyn> gimme a sec, i'll put it in detail
14:31:31 <boris-42_> ikhudoshyn: yep that's ugly
14:32:27 <ikhudoshyn> I realize that changing that will break compatibility with possible custom task configs
14:33:12 <boris-42_> ikhudoshyn: no we must not break compatiblity
14:33:13 <ikhudoshyn> yet it's ugly
14:33:24 <boris-42_> ikhudoshyn: so we can add new argument
14:33:24 <ikhudoshyn> boris-42_, I see
14:33:25 <ikhudoshyn> (
14:33:35 <boris-42_> ikhudoshyn: write deprecation warnigns
14:33:49 <boris-42_> ikhudoshyn: and after few releases remove it
14:34:32 <ikhudoshyn> so we should both add nova_kwargs and keep **kwargs for a while
14:34:42 <ikhudoshyn> and merge the latter two
14:35:03 <ikhudoshyn> that sounds reasonable
14:35:17 <ikhudoshyn> now about '_kwargs' suffix
14:35:42 <ikhudoshyn> this one should get into task configuration
14:36:01 <ikhudoshyn> for me it would look unnatural there
14:36:18 <ikhudoshyn> since it's just python-specific term
14:36:41 <ikhudoshyn> _config or _args looks better
14:36:54 <ikhudoshyn> or _params
14:36:59 <boris-42_> ikhudoshyn: we can make args for example
14:37:00 <ikhudoshyn> like cinder_params
14:37:18 <ikhudoshyn> args are just fine
14:37:31 <ikhudoshyn> cinder_args, nova_args -- look good
14:38:31 <ikhudoshyn> but.. should we have service-specific names or method-specific
14:38:44 <ikhudoshyn> like nova_args or create_vm_args?
14:39:08 <ikhudoshyn> I mean there could be several nova- calls in scenario
14:39:23 <boris-42_> ikhudoshyn: yep params sounds good
14:39:24 <ikhudoshyn> which could require specific params
14:39:41 <boris-42_> ikhudoshyn: we need to standartize this across all plguins in rally
14:39:49 <ikhudoshyn> yes
14:39:52 <boris-42_> ikhudoshyn: could you report a bug?
14:39:57 <ikhudoshyn> thats what I'm talking about
14:40:09 <ikhudoshyn> yes
14:40:16 <ikhudoshyn> will do
14:41:12 <ikhudoshyn> but I guess spec looks more suitable here
14:41:24 <ikhudoshyn> or bug+spec :))
14:42:08 <boris-42_> ikhudoshyn: ok
14:42:13 <ikhudoshyn> deal
14:42:24 <ikhudoshyn> that's all from my side
14:42:38 <chenli> I have a question, about when to use object and when to use dict
14:42:49 <chenli> I'm just a little confused now.
14:43:11 <ikhudoshyn> chenli, we need some more context
14:43:12 <chenli> We have module objects, but we want all db request return dict.
14:43:42 <chenli> and also a a Struct  class in utils.
14:43:58 <rvasilets> o/
14:44:32 <boris-42_> chenli: so objects must work with rally.common.db.api as a with objects
14:44:42 <boris-42_> chenli: we must not use there sqla models
14:45:32 <chenli> So, objects get return value from rally.common.db.api ==> get dict ?
14:47:05 <boris-42_> chenli: yep
14:47:19 <boris-42_> chenli: that was the goal of your task
14:47:25 <chenli> I can understand the usage of objects, because other projects do that too, such as nova and so on. but they would convert return value from db into the objects, and then use objects in other place.
14:48:17 <chenli> I don't really understand how we call objects.xx then get a dict.
14:48:31 <chenli> looks weird.
14:48:54 <boris-42_> chenli: it's not nova objects
14:49:03 <boris-42_> chenli: it's not even close to nova objects
14:49:35 <chenli> boris-42_: o ?
14:49:55 <boris-42_> chenli: did you look at code of nova objects?
14:50:03 <boris-42_> chenli: did you look at code of rally objects?
14:50:10 <boris-42_> chenli: no matching at all
14:50:29 <chenli> boris-42_: a little, it just call db api, and do nothing else.
14:50:31 <boris-42_> chenli: even in use cases
14:50:55 <boris-42_> chenli: in case of rally objects you are doing extra work (called business logic) in objects
14:51:18 <boris-42_> chenli: that doesn't belong to DB layer but that is used many time across the code
14:51:31 <boris-42_> chenli: that is why we have rally.objects
14:52:02 <boris-42_> chenli: rally.common.db.api should execute the DB queries
14:52:03 <boris-42_> chenli: and return dicts
14:52:23 <boris-42_> chenli: rally.objects will work with this dicts
14:52:59 <chenli> boris-42_:  it should return dict too ?
14:53:50 <boris-42_> chenli: nope
14:53:58 <boris-42_> chenli: it should return what it returns
14:54:39 <boris-42_> chenli: so there are just few places in code where "sqla" models are used outside of db.api and we should remove it
14:54:56 <boris-42_> chenli: I am not sure what is unclear here?
14:55:06 <chenli> boris-42_: o, this is link to another question. cli module call api module, api call objects
14:56:25 <chenli> so if objects return a class created by objects, then cli module should not use it
14:56:39 <chenli> boris-42_: right ?
14:57:00 <boris-42_> chenli: it the same topic
14:57:33 <boris-42_> chenli: I said that rally.cli module shouldn't use rally.common.db.api directly and should use rally.common.objects insetad
14:57:54 <boris-42_> chenli: the another thing is rally.common.db.api should return plain dicts instead of sqlamodels
14:58:34 <boris-42_> oh no objects
14:58:35 <boris-42_> sorry
14:58:46 <boris-42_> rally.cli -> rally.api -> rally.bojects
14:58:55 <boris-42_> where rally.cli is on one host
14:59:09 <boris-42_> rally.api and rally.boject and rally.db on another
14:59:13 <boris-42_> this is required to create rally as a service
14:59:26 <boris-42_> okay we have to end meeting
14:59:29 <boris-42_> #endmeeting