| openstackgerrit | zengchen proposed openstack/smaug: implement scheduled_operation_state table of db https://review.openstack.org/274995 | 03:46 |
|---|---|---|
| openstackgerrit | zengchen proposed openstack/smaug: implement scheduled_operation_states table of db https://review.openstack.org/274995 | 03:55 |
| *** xiangxinyong has joined #openstack-smaug | 06:07 | |
| *** x00350071_ has quit IRC | 06:09 | |
| *** xiangxinyong__ has quit IRC | 06:10 | |
| *** x00350071 has joined #openstack-smaug | 06:11 | |
| *** x00350071 is now known as xinyong | 06:11 | |
| openstackgerrit | zengchen proposed openstack/smaug: implement scheduled_operation_states table of db https://review.openstack.org/274995 | 06:19 |
| *** gampel has joined #openstack-smaug | 06:59 | |
| openstackgerrit | zengchen proposed openstack/smaug: implement scheduled_operation_states table of db https://review.openstack.org/274995 | 07:13 |
| *** zhonghua-lee has quit IRC | 07:35 | |
| *** xinyong has quit IRC | 07:35 | |
| *** xinyong has joined #openstack-smaug | 07:35 | |
| *** zhonghua-lee has joined #openstack-smaug | 07:35 | |
| openstackgerrit | zengchen proposed openstack/smaug: implement scheduled_operation_logs table of db https://review.openstack.org/275052 | 07:39 |
| *** c00281451 has joined #openstack-smaug | 07:43 | |
| openstackgerrit | zengchen proposed openstack/smaug: implement scheduled_operation_states table of db https://review.openstack.org/274995 | 07:44 |
| c00281451 | saggi: if you online, please tell me. thanks. | 07:49 |
| *** c00281451 is now known as chenzeng | 07:50 | |
| *** gampel1 has joined #openstack-smaug | 08:00 | |
| *** gampel has quit IRC | 08:00 | |
| saggi | chenzeng: ping | 08:00 |
| chenzeng | saggi:yes. | 08:01 |
| saggi | chenzeng: How is it going? | 08:01 |
| chenzeng | yesterday, you said you didn't understand the design of timetrigger. what's your question? | 08:03 |
| chenzeng | I have submitted the codes of dbs that are scheduled_operation_states and scheduled_operation_logs. you can review them. | 08:04 |
| *** zhonghua-lee has quit IRC | 08:04 | |
| *** zhonghua-lee has joined #openstack-smaug | 08:05 | |
| saggi | chenzeng: Yes, I still don't understand why the job store is a member of the trigger. | 08:09 |
| saggi | Could you explain it to me please? | 08:09 |
| chenzeng | do you see the comments I sended you? | 08:10 |
| saggi | chenzeng: Do all triggers get the same job store instance? | 08:11 |
| chenzeng | no | 08:12 |
| saggi | Why do they use different job stores? | 08:12 |
| saggi | isn't it just for queueing things? | 08:12 |
| chenzeng | because different trigger may need diffent jobstore. | 08:13 |
| chenzeng | this jobstore only is used by timetrigger | 08:13 |
| chenzeng | because this jobstore sorts the jobs in ascending order | 08:14 |
| saggi | chenzeng: This is what I don't understand. Why do we need different job stores. What is done differently between different tirggers? | 08:14 |
| saggi | Why isn't the tirgger responsible for queueing a job. Why do we need the store? | 08:15 |
| chenzeng | yes, timetrigger will store its own job by some kind of order, and other trigger will do the same work by its own way. | 08:17 |
| chenzeng | in my timetrigger, it has a jobstore which only store the jobs that are binded with a time trigger. | 08:18 |
| saggi | Why? | 08:19 |
| saggi | Also why is the job store responsible for execution | 08:19 |
| saggi | ? | 08:19 |
| saggi | I'd imagine the trigger will trigger. This will invoker something to get the plan and execute a job in a JobExecutor. The job will periodically store data for consistency and safety through the JobStore. | 08:20 |
| *** gampel1 has quit IRC | 08:20 | |
| *** gampel has joined #openstack-smaug | 08:20 | |
| chenzeng | because diffent jobstores have diffent way to store the job, timetrigger's jobstore should sort its jobs in order to get due jobs quickly. but other jobs don't need to do this work. | 08:21 |
| chenzeng | sorry, other jobstore may don't need to do this work. | 08:21 |
| saggi | The thing I don't understand is that theoretically there are infinite jobs. If you say you want a daily backup you have infinite jobs from now until the end of time. This means you shouldn't create the job until it triggers. | 08:24 |
| saggi | So there isn't a need to even sort them | 08:24 |
| saggi | or queue them | 08:24 |
| saggi | since they are created when they need to execute | 08:24 |
| chenzeng | my timetrigger just trigger the job when the time is up. how to known the job's time is up. my design is that check the job's next_run_time every some time, if the time is up, submit the job to the executor and update the next_run_time, then loop. | 08:30 |
| chenzeng | you design is that if the operation's trigger is ready, the trigger will activate the operation, is that? if so, how to design the trigger? | 08:32 |
| chenzeng | the usage of apscheduler is like your design. but internally, it works like mine, because my implementation is from it. | 08:36 |
| saggi | Triggers all have a reference to some JobExecutor. When they *trigger* the push a job to the executor. The executor manages how many things can run at ones. Jobs can add Log lines whenever they want and persist information for recovery. | 08:36 |
| saggi | Each operation is a new job | 08:37 |
| chenzeng | yes, my design is like this. | 08:38 |
| saggi | So what is the job store? | 08:39 |
| saggi | Sorry for not understanding | 08:39 |
| chenzeng | ok, it doesn't matter. i explain it more distinct. | 08:41 |
| chenzeng | first, there are 3 objects, which are trigger, executor, operation. operation binds with a *trigger*. just take the *time trigger *for exmaple. | 08:45 |
| chenzeng | the responsibility of trigger is compute the time and submit the operation to executor. executor queues the operation and run them orderly. the OperationManager hoold all kinds of operation class objects. when the executor run the operation, it just invoke the OperationManage's *execute_operation* method. | 08:49 |
| saggi | Sounds good | 08:51 |
| saggi | When does JobStore come in to play | 08:51 |
| saggi | ? | 08:51 |
| chenzeng | so, the core question is how does the trigger know when to submit the operation to executor. right? | 08:51 |
| saggi | No, the question is what is the JobStore class for | 08:52 |
| saggi | And why is it different between different triggers | 08:53 |
| chenzeng | ok, we analyze the question step by step. | 08:53 |
| chenzeng | there are at leat 2 kinds of trigger, timetrigger and eventtrigger, right? these triggers are diffent. | 08:54 |
| saggi | Yes | 08:55 |
| chenzeng | ok, suppose there are 3 operations which binds with *time trigger* | 08:56 |
| chenzeng | my implementation is that | 08:56 |
| chenzeng | when create these 3 operations, save them to the DB and register them to my 'TimeTrigger'. 'TimeTrigger' will encapsulate the operation by 'Job' which can computes the next run time of operation, and store the 'Job' in the 'JobStore'. | 09:00 |
| chenzeng | 'JobStore' will sort these 3 'Job' objects by its next_run_time in ascending order. then 'TimeTrigger' will get due jobs from 'JobStore' by comparing the next_run_time of operation with now in a loop function. | 09:03 |
| chenzeng | if 'TimeTrigger' get the due jobs, send them to the executor and udpate its next_run_time, then send back to 'JobStore' and resort it. | 09:07 |
| saggi | OK, so the job store is the triggers way of storing information between runtimes? | 09:08 |
| saggi | between restarts | 09:08 |
| chenzeng | yes | 09:09 |
| saggi | hmmm | 09:11 |
| saggi | How would it store the information. In the DB? | 09:11 |
| chenzeng | no, 'JobStore' just store the 'Job' objects in memory, because the the operation is stored in the DB. we can register the operation again when restarting the operation engine. | 09:13 |
| chenzeng | we don't need to create *Job* of operation when executing it, because the OperationManager can execute the operation by operation type and operation definition. | 09:14 |
| saggi | So JobStore is just an implementation detail | 09:17 |
| saggi | OK | 09:17 |
| saggi | I think got it | 09:17 |
| saggi | Am I correct? | 09:17 |
| chenzeng | yes, you got it. thanks god. | 09:17 |
| chenzeng | sorry, i typed very slowly. | 09:19 |
| xinyong | Hello saggi | 09:28 |
| saggi | xinyong: Hello | 09:28 |
| xinyong | Which email do you use usually? | 09:29 |
| xinyong | I want to send a email to you about the smaug ui | 09:30 |
| openstackgerrit | zengchen proposed openstack/smaug: the field's order of db tables is not same https://review.openstack.org/275088 | 09:37 |
| *** gampel has quit IRC | 09:56 | |
| *** openstackgerrit has quit IRC | 10:17 | |
| *** openstackgerrit has joined #openstack-smaug | 10:17 | |
| openstackgerrit | yinwei proposed openstack/smaug: Basic design doc for Protection Service https://review.openstack.org/261455 | 10:57 |
| *** zengyingzhe has quit IRC | 11:43 | |
| *** gampel has joined #openstack-smaug | 11:48 | |
| *** gampel1 has joined #openstack-smaug | 11:50 | |
| *** gampel has quit IRC | 11:52 | |
| openstackgerrit | yinwei proposed openstack/smaug: Basic design doc for Protection Service https://review.openstack.org/261455 | 11:55 |
| *** gampel has joined #openstack-smaug | 12:06 | |
| *** gampel1 has quit IRC | 12:07 | |
| openstackgerrit | wangliuan proposed openstack/smaug: Implement the basic protection service framework https://review.openstack.org/265760 | 12:43 |
| *** wangliuan has joined #openstack-smaug | 13:10 | |
| wangliuan | hi,saggi | 13:10 |
| saggi | wangliuan: hi | 13:10 |
| wangliuan | I had update yinwei's design doc | 13:11 |
| wangliuan | and my patch | 13:11 |
| wangliuan | I see you review opinion | 13:11 |
| wangliuan | which image you mean that is not up to date | 13:12 |
| wangliuan | it's a little late in china,I think I should check it carefully tomorrow .I am glad to have your review | 13:18 |
| wangliuan | Thank you very much | 13:18 |
| *** go has joined #openstack-smaug | 13:56 | |
| *** go is now known as Guest13883 | 13:56 | |
| *** Guest13883 has quit IRC | 13:57 | |
| *** wanghao_ has joined #openstack-smaug | 14:47 | |
| *** saggi has quit IRC | 16:06 | |
| *** chenzeng has quit IRC | 16:14 | |
| *** chenzeng has joined #openstack-smaug | 16:15 | |
| *** wanghao_ has quit IRC | 16:57 | |
| *** gampel has quit IRC | 17:30 | |
| *** openstackgerrit has quit IRC | 17:32 | |
| *** openstackgerrit has joined #openstack-smaug | 17:32 | |
| *** wanghao has quit IRC | 19:12 | |
| *** yinweiishere has quit IRC | 19:12 | |
| *** yinweiishere has joined #openstack-smaug | 19:12 | |
| *** wanghao has joined #openstack-smaug | 19:12 | |
| *** xiangxinyong_ has joined #openstack-smaug | 19:24 | |
| *** xiangxinyong has quit IRC | 19:27 | |
| *** xiangxinyong_ has quit IRC | 20:19 | |
| *** xiangxinyong_ has joined #openstack-smaug | 20:19 | |
| openstackgerrit | Saggi Mizrahi proposed openstack/smaug: Proposed Smaug API v1.0 https://review.openstack.org/244756 | 23:49 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!