| *** wanghao has joined #openstack-zaqar | 00:53 | |
| *** wanghao has quit IRC | 00:53 | |
| *** wanghao has joined #openstack-zaqar | 00:54 | |
| wanghao | flwang: ping | 01:13 |
|---|---|---|
| wanghao | wxy: ping | 01:16 |
| wanghao | hi guys, I'm reviewing the patch of queues actions for pooling. | 01:17 |
| wanghao | I got one question: we keep this 'self._get_controller = self._pool_catalog.get_queue_controller' is for the backend like swift, right? | 01:18 |
| wxy | wanghao: what's the problem? | 01:21 |
| wxy | self._mgt_queue_ctrl = self._pool_catalog.control.queue_controller this one? | 01:22 |
| wanghao | no | 01:22 |
| wanghao | self._get_controller = self._pool_catalog.get_queue_controller' this one | 01:22 |
| wanghao | I didn't get why we keep two queue_controller here. | 01:23 |
| wanghao | Is there different bwteen them | 01:23 |
| flwang | wanghao: the mgt controller is using to talk to the management db for queue actions | 01:24 |
| flwang | and the self._get_controller is used to talk to the pool queue controller to some actions zaqar has to talk to | 01:25 |
| flwang | for example, the stats api | 01:25 |
| wxy | yes, they are different | 01:25 |
| wanghao | but I see the code, for some backend, the mgt controller and pool queue controller are same. Did I understand wrong something? | 01:29 |
| wanghao | like using swift, self._get_controller is getting the queue controller from the self.control_driver.queue_controller. It's same as the mgt controller. | 01:31 |
| *** yanyanhu has joined #openstack-zaqar | 01:37 | |
| flwang | wanghao: what do you mean 'same'? | 01:40 |
| flwang | you can use mongo as management db and message db in the meanwhile | 01:40 |
| flwang | " like using swift, self._get_controller is getting the queue controller from the self.control_driver.queue_controller. It's same as the mgt controller." NO! | 01:40 |
| flwang | when using swift as message store, you have to use either mongodb or sqlalchemey as the management store | 01:41 |
| flwang | so you won't see the queue controller of mgmt is same as the "pool"'s queue controller | 01:41 |
| wanghao | flwang: Look at Catalog class, we have self.control, so self._pool_catalog.control.queue_controller will get management db controller like mongodb or sqlalchemey, right? | 01:45 |
| *** lhx_ has joined #openstack-zaqar | 01:45 | |
| flwang | wanghao: yes, but i haven't check the code | 01:48 |
| wanghao | flwang: okay, then look at the get_queue_controller in Catalog class too, it will call self.lookup, then will call self.get_driver, then will call self._init_driver, and in init_driver, we will load storage driver. | 01:48 |
| wanghao | according the options in pool. | 01:48 |
| wanghao | in utils.load_storage_driver, we pass through the control_driver=self.control. | 01:49 |
| flwang | wanghao: see https://github.com/openstack/zaqar/blob/master/zaqar/storage/pooling.py#L475 | 01:49 |
| flwang | the driver is load based on the url actually | 01:50 |
| wanghao | flwang: yes, I'm talking this code too. | 01:50 |
| flwang | so finally, you will get the correct queue controller of the pool | 01:50 |
| flwang | not the one of control plane | 01:51 |
| wanghao | flwang: if I specify the uri as swift, could I? | 01:51 |
| flwang | wanghao: pls don't just say swift, are you saying you're using swift as a pool, a message store? | 01:52 |
| wanghao | yes | 01:52 |
| flwang | wanghao: yes, it's same | 01:53 |
| flwang | just like the other drivers | 01:53 |
| wanghao | I see your point, for example we can use mongodb as control plane, and then speify redis as the message store. | 01:53 |
| flwang | wanghao: yes | 01:54 |
| wanghao | in pool uri. | 01:54 |
| wanghao | so we will get two queue controllers, one is mongodb, one is redis. | 01:54 |
| wanghao | I'm right? | 01:54 |
| flwang | yep | 01:55 |
| flwang | actually, the 2nd one is not a "real" queue controller | 01:55 |
| flwang | it's https://github.com/openstack/zaqar/blob/master/zaqar/storage/redis/messages.py#L562 | 01:56 |
| *** zhurong has joined #openstack-zaqar | 02:05 | |
| wanghao | flwang: okay I totally got this now, we load messagequeuehandler and redis queue controller both in pipline. | 02:08 |
| wanghao | flwang: but when we invoke 'stats', actually we just call the stats function in messagequeuehandler. | 02:09 |
| wanghao | since function _stats in redis queue controller just pass it. | 02:09 |
| flwang | wanghao: hold on | 02:09 |
| flwang | we load mongodb's queue contorller and messagequeuehandler of redis | 02:10 |
| flwang | yes | 02:10 |
| flwang | and you're statement is also correct "we load messagequeuehandler and redis queue controller both in pipline", i mean this | 02:11 |
| wanghao | https://github.com/openstack/zaqar/blob/master/zaqar/storage/pipeline.py#L142 | 02:11 |
| wanghao | yes | 02:11 |
| flwang | and actually, just like swift, we can get rid of the queue controller of redis as well | 02:11 |
| wanghao | you mean we remove the queue controller from redis? | 02:15 |
| flwang | wanghao: yep | 02:24 |
| flwang | but for now, i don't want to touch it :) | 02:25 |
| wanghao | flwang: but redis can support both management store and data store, why we want to remove the queue controller. | 02:40 |
| wanghao | flwang: we didn't want to use redis as management store in futuher? | 02:41 |
| flwang | wanghao: no | 02:42 |
| flwang | redis never support management db as far as i can tell | 02:42 |
| flwang | wanghao: see https://github.com/openstack/zaqar/blob/master/zaqar/storage/redis/driver.py#L258 | 02:43 |
| flwang | the code may need to be removed as well | 02:44 |
| wanghao | zaqar.control.storage = | 02:47 |
| wanghao | sqlalchemy = zaqar.storage.sqlalchemy.driver:ControlDriver | 02:47 |
| wanghao | mongodb = zaqar.storage.mongodb.driver:ControlDriver | 02:47 |
| wanghao | redis = zaqar.storage.redis.driver:ControlDriver | 02:47 |
| wanghao | faulty = zaqar.tests.faulty_storage:ControlDriver | 02:47 |
| wanghao | But I see driver config in setup.cfg, | 02:47 |
| wanghao | So we also should remove redis from here? | 02:47 |
| flwang | let me see who added it. IMHO, it doesn't make any sense using a memory-based key-value db as the management db of zaqar | 02:57 |
| flwang | wanghao: it's introduced by this https://github.com/openstack/zaqar/commit/76f6e063db982cd45a2b9c75d9a01e1533f905d1 | 03:08 |
| wanghao | yeah, I see | 03:19 |
| *** openstackgerrit has joined #openstack-zaqar | 03:20 | |
| openstackgerrit | Feilong Wang proposed openstack/zaqar master: Redis never supports to be a management store https://review.openstack.org/460811 | 03:20 |
| openstackgerrit | Merged openstack/zaqar master: Fix queues actions for pooling https://review.openstack.org/453440 | 03:20 |
| flwang | wanghao: https://review.openstack.org/#/c/460811/ | 03:21 |
| wanghao | not consider backward-compatibility? | 03:22 |
| flwang | it won't break backward compatibility, because it never works, right? | 03:22 |
| wanghao | hold on, user can use redis to create queue in redis/queues/queueController. | 03:26 |
| flwang | can't see your point | 03:27 |
| wanghao | I mean, when pooling is disable, user also can use redis as the control storage and data storage. | 03:34 |
| flwang | could be, i would say | 03:37 |
| flwang | but not for production | 03:37 |
| *** wanghao_ has joined #openstack-zaqar | 03:37 | |
| flwang | i'm going to abandon it for now anyway | 03:37 |
| *** zhurong has quit IRC | 03:38 | |
| wanghao_ | we can discuss it to see how to change it properly later. | 03:40 |
| flwang | yep | 03:46 |
| openstackgerrit | Feilong Wang proposed openstack/zaqar master: Fix OSProfiler issue for Swift and Redis driver https://review.openstack.org/459926 | 04:14 |
| *** lhx_ has quit IRC | 04:16 | |
| *** zhurong has joined #openstack-zaqar | 05:02 | |
| *** lhx_ has joined #openstack-zaqar | 05:17 | |
| *** lhx_ has quit IRC | 05:26 | |
| *** wanghao_ has quit IRC | 05:56 | |
| *** zhurong has quit IRC | 06:04 | |
| *** lhx_ has joined #openstack-zaqar | 06:05 | |
| *** zhurong has joined #openstack-zaqar | 06:12 | |
| *** zhurong has quit IRC | 06:23 | |
| *** pcaruana has joined #openstack-zaqar | 06:36 | |
| *** yanyanhu has quit IRC | 06:45 | |
| *** lhx_ has quit IRC | 06:49 | |
| *** lhx_ has joined #openstack-zaqar | 06:54 | |
| *** tesseract has joined #openstack-zaqar | 07:05 | |
| *** andreaf has quit IRC | 08:02 | |
| *** andreaf has joined #openstack-zaqar | 08:04 | |
| *** mpanetta has quit IRC | 09:03 | |
| *** mpanetta has joined #openstack-zaqar | 09:04 | |
| *** wanghao has quit IRC | 09:25 | |
| *** jtomasek has quit IRC | 09:34 | |
| *** jtomasek has joined #openstack-zaqar | 09:37 | |
| *** lhx_ has quit IRC | 09:53 | |
| *** lhx_ has joined #openstack-zaqar | 09:53 | |
| *** tesseract has quit IRC | 10:04 | |
| *** XueFeng has quit IRC | 10:07 | |
| *** tesseract has joined #openstack-zaqar | 10:08 | |
| *** tesseract has quit IRC | 11:50 | |
| *** tesseract has joined #openstack-zaqar | 12:05 | |
| *** Kevin_Zheng has quit IRC | 12:27 | |
| *** lhx_ has quit IRC | 12:52 | |
| *** rbrady has joined #openstack-zaqar | 13:31 | |
| *** mpanetta has quit IRC | 14:22 | |
| *** mpanetta has joined #openstack-zaqar | 14:23 | |
| *** openstackgerrit has quit IRC | 15:17 | |
| *** pcaruana has quit IRC | 15:28 | |
| *** mpanetta has quit IRC | 16:37 | |
| *** mpanetta has joined #openstack-zaqar | 16:37 | |
| *** tesseract has quit IRC | 16:45 | |
| *** harlowja has quit IRC | 17:06 | |
| *** harlowja has joined #openstack-zaqar | 18:38 | |
| *** mpanetta has quit IRC | 20:10 | |
| *** exploreshaifali has joined #openstack-zaqar | 20:10 | |
| *** mpanetta has joined #openstack-zaqar | 20:17 | |
| *** mpanetta has quit IRC | 20:18 | |
| *** mpanetta has joined #openstack-zaqar | 20:19 | |
| *** harlowja has quit IRC | 21:17 | |
| *** harlowja has joined #openstack-zaqar | 21:17 | |
| *** harlowja has quit IRC | 21:36 | |
| *** exploreshaifali has quit IRC | 23:14 | |
| *** harlowja has joined #openstack-zaqar | 23:30 | |
| *** harlowja has quit IRC | 23:55 | |
| *** harlowja has joined #openstack-zaqar | 23:59 | |
| *** mpanetta has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!