14:03:24 <saggi> #startmeeting smaug
14:03:25 <openstack> Meeting started Tue Apr  5 14:03:24 2016 UTC and is due to finish in 60 minutes.  The chair is saggi. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:03:26 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:03:28 <Zhongjun> Hi
14:03:29 <openstack> The meeting name has been set to 'smaug'
14:03:30 <yuval> hi
14:03:34 <zengchen> hi, everyone
14:03:41 <zhonghua-lee> hi
14:03:45 <saggi> how is everybody
14:03:47 <saggi> ?
14:03:49 <chenying_> hi
14:04:27 <saggi> Are we waiting for anyone?
14:04:46 <zhonghua-lee> fine
14:04:54 <gampel> Hi everyone
14:05:22 <zengchen> is yinwei online?
14:05:28 <xiangxinyong> hello
14:05:28 <saggi> with two accounts
14:06:11 <saggi> I pinged her
14:06:19 <saggi> Let's start
14:06:45 <saggi> #topic Announcements
14:07:05 <saggi> API is finally in
14:07:17 <zengchen> great!
14:07:28 <gampel> :)
14:07:44 <saggi> But there are still some issues. Will have fixes for it up today.
14:07:44 <gampel> we still need to merge the operation log API
14:08:03 <saggi> I will have updates for that up too
14:08:37 <chenying_> There are some comments about the operation log api
14:08:50 <saggi> That's it for announcements just wanted to celebrate :)
14:08:54 <saggi> #topic Operation Log
14:09:19 <gampel> #link https://review.openstack.org/#/c/298060/
14:09:36 <saggi> About the operation log. I saw the comments.
14:09:41 <xiangxinyong> saggi, i submit some comments in this agenda
14:09:48 <xiangxinyong> Does it include protect with schedule, protect without schedule and delete?
14:10:04 <saggi> Without schedule there will be no operation.
14:10:11 <saggi> You will just track the checkpoint.
14:10:25 <chenying_> IMO operation log is created when call protect/delete rpc interface.
14:10:31 <saggi> Since checkpoints have built in progress information.
14:11:03 <chenying_> checkpoints have built in progress information YES.
14:11:17 <gampel> So we do not include the checkpoint ID created in the log
14:11:40 <saggi> The log of a protect operation should include the checkpoint it created.
14:12:02 <saggi> And delete should have all the checkpoints it deleted.
14:12:06 <chenying_> what about delete action?
14:12:46 <xiangxinyong> saggi, we think we need to create a operation log in the protect and delete action
14:12:54 <saggi> but after a protect\delete request is made the information is tracked on the entity. This is because the entities can all be scheduled independently.
14:13:05 <saggi> Inside the checkpoint?
14:13:11 <saggi> It will have to be in the bank
14:13:40 <yinweimac> hi, sorry for late
14:13:45 <saggi> chenying: We will need to think about it.
14:13:46 <gampel> Hi yinwei
14:14:03 <xiangxinyong> But when we restore a checkpoint and delete a checkpoint, how to express it?
14:14:15 <gampel> I think on delete we need to add some metadata of the deleted checkpoint
14:14:23 <saggi> We do that.
14:14:29 <saggi> Apart from changing it's status
14:14:48 <saggi> But we don't have logs now. Just changed state.
14:15:03 <saggi> Since logs are very complicated to do on an object store.
14:15:04 <xiangxinyong> saggi: do you consider to add a indepent operation log to show the progress?
14:15:37 <xiangxinyong> saggi: we think the checkpoint status is different form the protect and delete or restore
14:16:04 <xiangxinyong> what's your opinion?
14:16:08 <saggi> xiangxinyong: It also contains failure information and progress about each resource.
14:16:13 <gampel> whats the difference ?
14:16:31 <saggi> but it doesn't contain a message log.
14:16:34 <xiangxinyong> because the protect and delete and restore are a workflow
14:17:00 <xiangxinyong> the protect and delete should have a workflow like restore
14:17:29 <xiangxinyong> checkpoint is just a target
14:17:54 <gampel> what else do you think we should report other then  status and % progress per resource
14:18:08 <saggi> We also put error information
14:18:53 <xiangxinyong> when we protect a checkpoint and delete a checkpoint at the same time,
14:19:06 <chenying_> when is operation log db data created?   before  a protect\delete action rpc request?
14:19:08 <xiangxinyong> it is diffult to disguish the status
14:19:17 <gampel> xiangxinyong: the UI could do the match of the operation log and the operation status metadata in the checkpoint
14:20:02 <gampel> The checkpoint will be locked by the protect and you will not be able to delete it until it is done ?
14:20:12 <saggi> I think we should take it offline. chenying and xiangxinyong please write an email explaining what use cases are not covered and we will try and converge on a solution. If we see it requires more planning we will schedule a meeting just for this. OK?
14:20:33 <gampel> sound good to me
14:20:40 <saggi> You could also comment on the operation log patches if you think it's relevant there
14:20:52 <saggi> chenying, xiangxinyong: ok?
14:20:55 <chenying_> Ok I will review the patch.
14:21:27 <xiangxinyong> ok. saggi.
14:21:31 <saggi> great!
14:21:32 <gampel> we found today we are missing name in the protactable API
14:21:48 <saggi> gampel: I am sending a patch for it later today
14:21:51 <chenying_> Let me see
14:21:56 <saggi> for the API
14:22:01 <saggi> not the actual implementation
14:22:28 <Guest13879> #openstack-meeting
14:22:33 <saggi> We are also missing the API to get dependencies.
14:22:52 <gampel> Ok good we found that it missing today reviewing  the UI with xiangxinyong
14:23:14 <saggi> xiangxinyong also commented on the dependencies api missing.
14:23:17 <saggi> #topic Protectables status
14:23:37 <yuval> Regarding protectables:
14:23:49 <yuval> we already have merged server, image, volume protectables
14:23:56 <yuval> still missing: network and project protectable
14:24:17 <yinweimac> yuval, will you address the image ,volume protectable bug?
14:24:28 <yuval> yinweimac: what bug?
14:24:31 <yinweimac> the dangling volume and image issue, I mean
14:24:34 <gampel> which bug can you sjare the link
14:24:36 <gampel> share
14:24:56 <yinweimac> I haven't fired the bug, but I suppose we have discussed this issue
14:25:07 <gampel> Ok
14:25:09 <yinweimac> let's confirm this is an issue first
14:25:26 <yinweimac> yuval, what do you think?
14:26:42 <yuval> are you speaking about volume and image which are unattached to servers?
14:26:49 <yinweimac> yes
14:27:13 <gampel> I think that we are missing the project protactabole , but what yinwei mean is that when the  parent is projcet we shoudl return all the resources in vol , image etc
14:27:16 <yinweimac> but user chooses project as protection plan resources
14:27:23 <yuval> would that be fixed once we will add a project protectable and add the dependency?
14:27:36 <yinweimac> I don't think so
14:27:48 <yinweimac> it's image and volume protectable business
14:27:51 <gampel> we need to add the dependency in all the other protectable
14:28:28 <yuval> yinweimac: I understand
14:28:55 <yinweimac> just test parent is project, and return any images/volumes under this project will do
14:29:10 <yinweimac> it would be a simple fix
14:29:32 <yuval> I'll handle the project protectable and dangling image/volume
14:29:32 <yinweimac> ok, so we have confirmed this will be fixed by yuval, right?
14:29:42 <yinweimac> thanks!
14:29:47 <yuval> who will handle the network protectable?
14:29:49 <gampel> I think that we need to decide if this is this is the desire flow (select  project return all the resources in that project )
14:30:07 <yinweimac> why not?
14:30:40 <saggi> It might be too cluttered for the user. It's a UI issue in the end but maybe we would want to group by type under project. What do you think xiangxinyong?
14:31:21 <saggi> If the user has thousands of volumes it might just be too much.
14:31:30 <yinweimac> it depends
14:31:36 <yuval> saggi: this affects rest api as well as UI
14:31:41 <yinweimac> user could only choose some volumes
14:31:58 <gampel> It means that if you select the Project you will protect ever thing
14:32:00 <yinweimac> like user only choose volumes by tag
14:32:15 <xiangxinyong> saggi: we can get the resource step by step
14:32:18 <yinweimac> yes, if you select the project we protect everythin
14:32:36 <yinweimac> if you select some of the volumes, we only protect them
14:32:42 <xiangxinyong> saggi:we can also get the resouce in a time
14:32:47 <gampel> I think that we should in the UI make it a special option to select the all project resources
14:32:56 <saggi> The problem is that if you don't protect the project you loose tenant information.
14:33:31 <gampel> We should all ways protect keystone as dependency
14:33:35 <yinweimac> why? I saw chenying made checkpoint under tenant
14:33:36 <saggi> Like username\password which you might want in the restored environment.
14:34:01 <xiangxinyong> gampel: now if select a root keystone project, the all dependent resources will be selected
14:34:15 <gampel> But only if it is selected in the tree explicitly then it will be added as you said
14:34:31 <saggi> What do we even back up in project?
14:34:42 <gampel> But if we do not select it it will be added as dependency  of any resource
14:35:02 <gampel> Not sure probably the Region info
14:35:27 <saggi> Maybe we would make it an empty backup and have any real information as child of project. This means selecting the actual project actually means back up everything.
14:35:33 <yinweimac> why do we need that
14:36:14 <gampel> you mean have keystone resource and Project as two different items in the tree ?
14:36:19 <yinweimac> the keystone info could be input param as the restore api
14:36:49 <yinweimac> I mean why do we need save keystone info? for restore?
14:37:04 <saggi> Since we don't see any information on the project we could leave things as they are since there is no reason to backup a project other than to backup everything. If we find information we want we could have it as a child of project. This means we don't have to change anything now. What do you guys think?
14:37:51 <saggi> yinweimac: Yes, maybe there is tenant information we want to restore. I can't think of something now but I think my suggestion solves both cases.
14:37:52 <gampel> I agree who is  going to do the Keystone protection plugin and investigate this
14:38:21 <gampel> keystone/Project
14:38:59 <yinweimac> confirm: select a project will or will not protect all resources under it?
14:39:25 <gampel> No i think the other way around
14:39:44 <xiangxinyong> i think it will not
14:40:10 <yinweimac> then how to choose all resources of a project to be protected?
14:40:13 <saggi> What I suggested means that it does protect all resources.
14:40:19 <gampel> if I understand  @saggi suggestion is that when selecting a project all the resources in the projcet will be backed up
14:40:31 <yuval> I understood that also
14:40:59 <saggi> I just suggested that if we see information we need for restore we will create child resource for it.
14:41:06 <gampel> Assuming we do not need to protect any think for Project
14:41:12 <yinweimac> ok
14:41:21 <gampel> who is  going to do the Keystone protection plugin and investigate this ?
14:41:30 <saggi> keystone::project -> keystone::ZoneInfo
14:41:37 <gampel> yes
14:41:48 <yinweimac> let me check it with chenzeng&chenhuayi
14:41:51 <saggi> Which will leave all behavior normalized
14:41:58 <gampel> Ok great
14:42:12 <yinweimac> but we need confirm more details about this
14:42:22 <yinweimac> what to be protected here
14:42:43 <gampel> yes that's the research need to be done
14:42:45 <yinweimac> confirm again: select a project will protect all resources under it
14:42:50 <gampel> yes
14:43:08 <yinweimac> so yuval will still adress the bug there?
14:43:20 <yuval> the dangling volume/image? yes
14:43:22 <saggi> yes
14:43:25 <yinweimac> ok
14:43:36 <yinweimac> network protectable needs to be reworked
14:44:01 <yinweimac> but wangliuan is working on integration test of protection service, chenzeng will help here
14:44:30 <yinweimac> and based on our latest discussion, the complexity is in network protection plugin
14:44:35 <gampel> chenzeng need to finish the operation engine patches
14:44:49 <gampel> they are blocking some of the API patches
14:45:05 <zhonghua-lee> yinweimac:it seems chenhuayi works on network protectable
14:45:09 <yinweimac> ok, chenhuayi is willing to help here
14:45:10 <yinweimac> yes
14:45:16 <zhonghua-lee> but he is not here now
14:45:19 <yinweimac> huayi contacted with me today
14:45:30 <saggi> It looks like we moved to tactical stuff. Lets move to the next subject.
14:45:33 <yinweimac> I told him I still need confirm details with saggi and eranl
14:45:52 <saggi> yinweimac: I trust you'll coordinate with everyone about the network protectable
14:46:05 <yinweimac> thanks
14:46:09 <yinweimac> will try
14:46:09 <gampel> thank you
14:46:36 <saggi> #topic CI
14:46:43 <saggi> Who is working on it?
14:47:16 <yuval> I submitted a patch to make smaug python3 compatible
14:47:16 <gampel> I think we need fullstack that will do protect and check the status on the checkpoint
14:47:19 <chenying_> you mean integration testing?
14:47:28 <gampel> yes fullstack on the api level
14:47:37 <xiangxinyong> i agree
14:47:40 <saggi> chenying: automatic tests on gerrit
14:48:13 <yinweimac> liuan is working on the manual integration test, but protection service only
14:48:23 <gampel> who ever is going to work on it I can show example how we did it in other projects
14:48:35 <gampel> we could start with protection only
14:49:06 <chenying_> I can work on it.
14:49:20 <chenying_> Maybe need some help from eran.
14:49:30 <saggi> Do you have any spare time?
14:49:41 <gampel> no problem lets talk about it offline tomorrow morning
14:49:47 <yinweimac> Liuan could work for it as well
14:49:53 <saggi> Great
14:49:55 <yinweimac> I will ask him
14:50:02 <yinweimac> tommorrow
14:50:08 <saggi> Let's make it a point to check for progress next bi-weekly
14:50:24 <saggi> #topic Progress report
14:50:26 <gampel> xiangxinyong: will you be able to share screen-cast of the UI so every one could comment
14:50:37 <saggi> Just in time gampel
14:50:54 <xiangxinyong> by email?
14:51:09 <xiangxinyong> gampel: or some ways?
14:51:11 <gampel> You could sent it to devstack mailing list
14:51:19 <saggi> xiangxinyong has done a lot of work on the UI. We would like as many people to tell us what they think about it.
14:51:36 <xiangxinyong> ok
14:51:40 <saggi> xiangxinyong, is it in a state where people can test it locally?
14:51:50 <zengchen> i am updating the time trigger and two apis. this week i hope they can be merged.
14:51:54 <gampel> so we could get feedback  from the OpenStack community
14:52:26 <xiangxinyong> saggi: i will do as soon as possible
14:52:33 <gampel> Thanks
14:52:41 <saggi> xiangxinyong: great!
14:53:01 <xiangxinyong> <gampel> we found today we are missing name in the protactable API
14:53:01 <xiangxinyong> <saggi> xiangxinyong also commented on the dependencies api missing.
14:53:01 <xiangxinyong> Thanks gampel and saggi.
14:53:01 <xiangxinyong> They are two important functions.
14:53:23 <gampel> Please everyone help with the reviews of all the open patches
14:53:24 <saggi> xiangxinyong: Will have API patches for that later today.
14:53:29 <xiangxinyong> thanks
14:53:44 <saggi> # Open Discussion
14:53:44 <xiangxinyong> by the way. i have another questions to discuss it
14:53:45 <xiangxinyong> http://paste.openstack.org/show/492976/
14:54:16 <gampel> yes
14:54:23 <xiangxinyong> i think we need to add Parameters to every resource in the create plan api
14:54:54 <yuval> what does 'consistency' mean in that context?
14:54:55 <xiangxinyong> please take a look at it.http://paste.openstack.org/show/492976/
14:55:22 <xiangxinyong> eran and saggi, do you agree?
14:55:48 <saggi> xiangxinyong: currently parameters can also be defined for a type
14:55:50 <gampel> You mean local if set and global as override ?
14:55:55 <chenying_> If it is affirmed, I will update the patch.
14:56:31 <saggi> xiangxinyong: The reason we didn't do it originally is because we thought the UI for it will be confusing.
14:56:31 <xiangxinyong> because we need to edit every resource's parameters in the ui
14:56:42 <gampel> or do you mean moving them into the resource >>
14:57:09 <xiangxinyong> yeah, move them into the resources
14:57:25 <gampel> saggi: why not ?
14:57:33 <xiangxinyong> because we need specify parameters for evevy resource
14:57:50 <saggi> xiangxinyong: Currently defining it outside of the resource make them default for auto discovered resources.
14:57:56 <chenying_> IMO I do not think every resource has its own parameters.
14:58:26 <gampel> But if we set some thing for a spesific resource it should be set there
14:58:49 <xiangxinyong> saggi: you mean the outside parameters are the default value
14:59:02 <xiangxinyong> i agree
14:59:03 <saggi> xiangxinyong: Yes
14:59:32 <xiangxinyong> but we need specify in the speicail resouces. is it right?
14:59:33 <gampel> It seem we need the parameters in the resource if selectd and outside as default
14:59:35 <saggi> xiangxinyong: And also you will need to do it a bit differently. since you can't put it inside the resource map since it breaks the type signiture.
14:59:50 <yinweimac> times out
14:59:57 <saggi> yes
15:00:00 <gampel> we are running out of time
15:00:12 <gampel> saggi will you define this for xiangxinyong:
15:00:18 <saggi> xiangxinyong: The idea is good but needs some adjustments.
15:00:19 <xiangxinyong> could we discuss it in our chanel?
15:00:19 <yinweimac> let's switch back to smaug channel
15:00:27 <saggi> #endmeeting