15:01:56 <saggi> #startmeeting karbor
15:01:56 <openstack> Meeting started Tue Nov 29 15:01:56 2016 UTC and is due to finish in 60 minutes.  The chair is saggi. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:00 <openstack> The meeting name has been set to 'karbor'
15:02:00 <saggi> Hi everyone
15:02:11 <xiangxinyong> hi saggi
15:02:17 <leon_wang> saggi: hi saggi
15:02:17 <yuval> hey
15:02:53 <saggi> There are no agenda items AFAIK
15:03:05 <saggi> but I think we can discuss the spec for the protection plugin design
15:03:12 <saggi> since it's a hot matter of contention
15:03:17 <yuval> saggi: yep, I agree
15:03:42 <chenying> ok
15:03:57 <saggi> #topic protection plugin spec
15:04:28 <saggi> https://review.openstack.org/#/c/397156/
15:05:28 <saggi> The main issue of contention seems to be complexity.
15:05:49 <chenying> My oppions is that spilt the backup action to several hooks, it too complex for vendors to integrate their solutions.
15:05:54 <zengchen> saggi:yes
15:05:59 <yuval> From what I understand, this seem complex. As a matter of fact, most of the plugin api is optional, and implementing straight-forward plugins should be relatively easy
15:06:17 <saggi> As the spec says much of the features are optional. There is nothing stopping people from just implementing the main callback
15:06:28 <saggi> it will cause everything to run in parallel
15:06:45 <saggi> The rest of the hooks are for specialized use cases
15:07:55 <saggi> Though we already know of use cases where a protection plugin will have to use some of the hooks
15:08:40 <zengchen> saggi:at present, there is a scenario that protect service can not support.
15:09:01 <saggi> zengchen: what scenario?
15:09:39 <zengchen> saggi:if user want to protect whole vm with their own plugin which just input id of vm.
15:10:19 <zengchen> saggi:karbor will fetch the volumes attached to the vm to protect together.
15:10:52 <zhonghua_> hi
15:10:57 <saggi> In the context of karbor there is no such thing as protecting the whole vm since you don't know what resources are related to the VM. What does make since is being responsible for some of the resources.
15:11:01 <zengchen> saggi:but user just has one plugin that protect vm
15:11:08 <saggi> For example having a plugin for VM and Volumes
15:11:20 <leon_wang> saggi:agree
15:11:54 <saggi> This will work with the paradigm we suggest.
15:12:05 <saggi> The plugin will register for volume and vm
15:12:28 <saggi> when it reaches the prepare stage it will get the sub tree
15:12:45 <saggi> it will get all the information it needs and prepare information for the main phase
15:13:03 <saggi> which could just be sending the instruction somwhere else and waiting on the result
15:13:06 <chenying> saggi: Why do you think the plugins don't know the  resources are related to the VM? As I know, several vendor has the solution to protect all vm, only need the id of vm.
15:13:22 <saggi> The the plugin could just no-op for volume
15:13:43 <saggi> and have the main just track the main VM task
15:14:09 <saggi> or, if the service gives detailed progress. Show progress related to the resource.
15:14:13 <chenying> saggi: There are some workflow in ther backup software like NBU or simpala, to fetch get related volumes about the protected vm.
15:15:18 <zengchen> saggi:i think your description is karbor's solution. it may be right. but it is not appropirate to integrate the vendor's solution.
15:15:35 <yuval> chenying: if the nbu/simpala karbor plugin registers to protect both VM and Volumes, it can send nbu/simpala the VM, and do nothing for volumes or wait until the VM protect is complete
15:16:00 <chenying> In this scenario, the vendors only need dev a plugins  about the main VM task, don't need dev a plugins about volume.
15:16:23 <chenying> yuval I think so
15:16:43 <saggi> zengchen: They need to specify that the volumes are taken care of. Since Karbor needs to know that this type is accounted for.
15:17:20 <leon_wang> chenying: has these vendors provided solutions integrating with VMs now?
15:18:48 <saggi> zengchen: Whatever the vendors protect we need to save information in the checkpoint about the result of the protection.
15:18:56 <leon_wang> chenying: and is it possible for karbor to seamlessly integrate them?
15:19:18 <chenying> leon_wang: simpala plan to integrate the vm protection solution with Karbor,  but I am not when to integrate with karbor.
15:19:25 <zengchen> saggi:maybe we should think more about how to integrate vendor's solution easily. because that is one of karbor's mission.
15:19:58 <leon_wang> zengchen: yeah, also reduce our work.
15:21:08 <saggi> zengchen: That is what we are thinking about. What we know is that different vendors protect different things. This is why we need the ProtectionPlugin to specify what it protects and handle status and progress saving per resource.
15:21:08 <saggi> I don't see a way around this. Let's say that the protection failed because on of the volumes failed. We want to show it to the user. For this we need to know to get information about volumes.
15:21:29 <zhonghua_> hey, sorry, I am late, but where can I find the agenda for this meeting? so that I can caught up with you.
15:21:56 <saggi> zhonghua_: We are talking about this spec [ https://review.openstack.org/#/c/397156/ ]
15:22:07 <zhonghua_> saggi: thanks
15:22:30 <leon_wang> saggi:i think it would be better if the vendor can open these interface to karbor.
15:23:00 <yuval> I think there are two questions here: 1) does this API fits common vendor use cases 2) is this API easy to understand/use
15:23:18 <zengchen> saggi:as i said in that patch, yuval's design is very good. can we define simpler interfaces and move the yuval's desgin and resource dependency analysing etc as karbor's default solution?
15:23:39 <yuval> sometimes, making the API flexible so it fits common use cases, might make it a bit more complex
15:24:06 <yuval> this is why most hooks are optional, and writing a simple plugin requires only one main hook
15:24:06 <saggi> zengchen: I don't understand what you think is complex. The dependency graph building or the ProtectionPlugin interface.
15:24:41 <chenying> leon_wang: some vendor have already open the restAPI about their backupsoftwares, I don't think this is a problow,
15:25:02 <leon_wang> chenying: got it, thanks
15:25:49 <zengchen> saggi:take Heat as an exmple. it just define two interfaces for resource plugin to create resource. they are 'handle_create', 'check_create_complete'
15:26:56 <zengchen> saggi:as yuval's design, one plugin has 5 interfaces and some are optional.
15:27:27 <saggi> zengchen: 4 hooks
15:27:29 <zengchen> saggi:yuval's design is good. but IMO, it is one of solution to protect.
15:27:35 <saggi> zengchen: This is because they don't need the prepare phase.
15:27:44 <saggi> The prepare phase is the most complex
15:27:52 <saggi> but it can wield a lot of benefit
15:27:54 <saggi> for example
15:28:15 <zengchen> saggi:i know, it is flexible. it is good.
15:28:33 <saggi> If you have a VM with 3 volumes. You need to take a snapshot for each and copy the data to 3 different sites.
15:28:46 <saggi> Taking the snapshots should be done at the same time
15:28:53 <saggi> copying can be done later
15:29:03 <saggi> if we have only one hook "main"
15:29:21 <saggi> we put taking the snapshot and copying the data at the same place
15:29:26 <saggi> copying the data could take a lot of time
15:29:46 <saggi> meaning that the snapshots might be made at drastically different points in time
15:30:12 <zengchen> saggi:but for professional preotect software, they just need plugin to send small parameters to it.
15:30:14 <saggi> So you should take the snapshot in prepare and then copy in main
15:30:55 <saggi> zengchen: even some professional solutions might not support copying the data off site natively
15:31:13 <saggi> zengchen: and for this solutions they don't need to implement the prepare phase
15:31:28 <saggi> They only need to implement one hook
15:31:36 <saggi> all hooks are optional
15:32:02 <saggi> But we can't only support vendors that have replication and snapshot built in
15:32:13 <saggi> we need to also support vendors that only have snapshot
15:32:31 <zengchen> saggi:ok, i hold my idea.
15:33:05 <yuval> zengchen: what's your idea? do you think we can minimize the amount of hooks? different api?
15:33:14 <saggi> zengchen: Please rephrase that, I don't understand
15:34:22 <zengchen> saggi:your idea is reasonable. but i still think it is too complex. so i hold my idea.
15:34:44 <saggi> zengchen: How do you suggest to solve the problem I suggested.
15:34:55 <saggi> We need that for the Cinder implementation
15:35:34 <saggi> zengchen: I also don't understand what your idea is. Just only having *main phase* and *complete phase*
15:36:04 <zengchen> saggi:maybe.
15:36:23 <zengchen> saggi:my idea is decribed in the comment.
15:36:43 <zengchen> saggi:you can see it.
15:37:24 <saggi> zengchen: I saw it. I can't see how it can solve the user case I proposed.
15:37:45 <saggi> It will only work for the once case where everything is taken care of by someone else.
15:37:55 <zengchen> saiig:ok, i will think more about your idea.
15:38:39 <xiangxinyong> I am confused with the relationship between this patch and full vm backup?
15:39:05 <saggi> zengchen: We also have the quiesce use case, where we don't want to hold the VM in quiescent state for as little time as possible.
15:39:40 <saggi> xiangxinyong: full vm backup is the easiest use case to support. It will ignore most of the features.
15:40:12 <zengchen> saggi:i didn't refuse you and yuval's idea.
15:40:41 <saggi> zengchen: We also want you to agree :)
15:40:47 <saggi> not just not refuse
15:40:54 <zhonghua_> why not invite some 3rd vendors to get their comments?
15:41:06 <chenying> saggi: IMO, we also need pay more attention to discuss these interfaces with backup software vendor about different resources portection solution.
15:41:31 <xiangxinyong> saggi:ok
15:41:57 <yuval> zhonghua_: chenying: that sounds like a good idea
15:42:18 <zhonghua_> yuval: :)
15:42:34 <xiangxinyong> chenying: As you said, the vendors will only pass the vm id into the plugin to protect whole vm?
15:42:38 <zengchen> saggi:i just suggest that move your idea as the default solution. then user can develop their plugins by inherited default plugin. or the basic plugin.
15:43:56 <chenying> We need the comments about workflow and plugins interfaces form different vendors.
15:44:12 <chenying> xiangxinyong: As I know, it is.
15:44:16 <saggi> zengchen: The way it's designed you don't need any base code. Any function you need you implement any that you don't you don
15:44:18 <saggi> 't
15:45:00 <yuval> chenying: the workflow is karbor internals. The plugin api is the most important part
15:46:01 <zengchen> saggi:sorry, not quite understand.
15:46:59 <chenying> yuval Yes The plugin api is the most important part. Most of time, it is design for vendors, it is also very important for backup vendors.
15:47:42 <zhonghua_> yuval: chenying: +1
15:47:57 <saggi> zengchen: I'll try and cook up some examples for how to do Full VM backup with this inteface
15:48:14 <yuval> chenying: agrees
15:49:26 <zengchen> saggi:ok. maybe we should do it first. then return to check.
15:49:40 <saggi> #topic open discussion
15:49:55 <saggi> Anything else for the last 10 minutes
15:50:39 <yuval> yes
15:51:03 <yuval> zhonghua_: I've seen some students contributing these days
15:51:19 <yuval> zhonghua_: would you like to share about your activities?
15:51:36 <zengchen> i have subbit a spec on resolving token's expiration. so hope every guys can review it.
15:51:52 <zhonghua_> yuval: yeah, sure, but it held by xiangxinyog
15:52:31 <zhonghua_> xiangxinyong: could you please share the communication activities?
15:52:37 <chenying> I think we need add more fulltest about different portect flow and use cases. Now the tests in karbor is too simple.
15:52:49 <zengchen> https://review.openstack.org/#/c/403330/
15:53:36 <chenying> zengchen: I will.
15:53:40 <xiangxinyong> yuval,saggi: you know our team make a hands-on like the summit in a university under zhonghua's lead
15:54:07 <zengchen> chenying:thanks.
15:54:17 <xiangxinyong> and we introduce how to contribute in the openstack and karbor.
15:54:40 <yuval> xiangxinyong: sounds great
15:54:50 <xiangxinyong> and we prepared some blogs for students to operate step by step
15:55:24 <xiangxinyong> This is the team's blog.http://www.cnblogs.com/openstackteam/
15:55:57 <zhonghua_> xiangxinyong: great jobs
15:56:06 <saggi> good work!
15:56:09 <xiangxinyong> zhonghua_: thanks
15:56:23 <leon_wang> xiangxinyong: Great job!
15:56:24 <xiangxinyong> It's our team's work
15:56:29 <zengchen> xiangxinyong:they are all chinese. are you sure saggi & yuval can understand?
15:56:48 <chenying> well done!
15:56:55 <saggi> we'll try and use google translate
15:57:13 <xiangxinyong> because the whole team was arrived the university and help the students to do some practice
15:57:18 <zengchen> saggi::)
15:57:29 <xiangxinyong> zengchen:)
15:57:55 <xiangxinyong> but we need to help the students to raise the experience in OpenStack
15:58:29 <xiangxinyong> zhonghua makes a very good speech about barcelona summit in the university
15:58:42 <saggi> OK, I don't know if you guys know but Yuval will be unavailable next week. Feel free to bug me about things.
15:59:01 <xiangxinyong> and the whole team members helped the students to make pratice
15:59:13 <leon_wang> xiangxinyong: i think it's a good way to help us perfect our use cases.
15:59:17 <xiangxinyong> saggi: sure
15:59:31 <xiangxinyong> leon_wang:)
15:59:33 <saggi> Are window is up
15:59:35 <saggi> our
15:59:44 <saggi> we can continue on #openstack-karbor
15:59:46 <xiangxinyong> bye
15:59:48 <saggi> #endmeeting