14:00:28 <saggi> #startmeeting smaug
14:00:29 <openstack> Meeting started Tue May  3 14:00:28 2016 UTC and is due to finish in 60 minutes.  The chair is saggi. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:33 <openstack> The meeting name has been set to 'smaug'
14:00:38 <saggi> Hi everyone
14:00:40 <saggi> Who's here?
14:00:44 <chenying_> hi
14:00:47 <xyang1> hi
14:00:52 <yuval> hey
14:00:53 <xiangxinyong> hi
14:00:54 <yinweimac> hi
14:01:13 <sgotliv> o/
14:01:14 * smcginnis is lurking
14:01:20 <saggi> :)
14:01:31 <sgotliv> saggi, I am lurking too
14:02:00 <saggi> OK, so let's start.
14:02:10 <saggi> #topic OpenStack Summit update
14:02:29 <saggi> So I was the only one from that Smaug that was able to get it to the summit.
14:02:57 <saggi> I thought it went well.
14:03:10 <saggi> I tried to go around and shove Smaug in to people's faces
14:03:22 <saggi> I hope I wasn't to obnoxious.
14:03:25 <xiangxinyong> :)
14:03:25 <smcginnis> Lot's of interest and good things said about the project from what I heard.
14:03:51 <saggi> I got to speak to a lot of people and there is some interest stirring.
14:04:03 <saggi> There are a lot of people I need to touch base with
14:04:22 <saggi> but the general feeling is that people understand that this is a piece that is currently missing.
14:04:50 <xyang1> saggi: hi, I talked to you at the cinder session.  will need to talk more later
14:05:11 <saggi> xyang1: sure, no problem
14:05:16 <saggi> I will send a link to the mailing list with the vbrownbag presentation I did.
14:05:35 <saggi> People tell me it's good. I still can't bare looking at my own recordings.
14:05:53 <smcginnis> ;)
14:06:02 <saggi> I got some good feedback from Cinder people.
14:06:07 <xiangxinyong> saggi: great work
14:06:28 <saggi> There are still point of integration that need to get ironed out.
14:06:57 <saggi> Mainly a way for us to understand more about types. Since now the are (purposefully) opaque.
14:07:32 <saggi> And this will cause problem if we ever need to do something vendor specific. Since we will need to pick inside and see which vendor is driving what type.
14:07:49 <xyang1> we are designing replication api in cinder, so good to discuss about how to integrate with smaug
14:08:21 <saggi> Every vendor I spoke with (admittedly not a lot of them) tell me to manually poke around the host and voume and guess what is installed.
14:08:54 <saggi> xyang1: Are there any special irc meetings for that. Or is it in the general Cinder IRC meeting?
14:09:28 <xyang1> saggi: we discussed about it on Friday and wednesday
14:09:54 <saggi> I was on Wednesday but I couldn't make it on Friday. I'll look up the meeting notes.
14:10:00 <xyang1> I have specs, will update them after getting feedback from the summit
14:10:08 <saggi> But I was asking about any on going tracking of the feature.
14:10:27 <xyang1> saggi: we talked about them on Friday afternoon
14:11:16 <smcginnis> saggi: So far just weekly meetings and midcycles. No special meetings planned for it.
14:11:27 <saggi> xyang1: I know, I was going to get there but I there were problems with my flight\luggage I had to sort out.
14:11:42 <saggi> smcginnis: OK,  it's on us to be on top of it.
14:12:02 <xyang1> saggi: ok, let me get the spec link
14:12:13 <smcginnis> saggi: We can try to keep you updated on any changes.
14:12:19 <saggi> smcginnis: Thanks
14:12:33 <smcginnis> saggi: But if there is something smaug needs exposed to be able to integrate well, let us know and we can work out a way to enable it.
14:13:11 <xyang1> https://review.openstack.org/#/c/229722/
14:13:11 <saggi> smcginnis: I'll look up the spec today if it's updated.
14:13:22 <xyang1> this needs to be updated though
14:13:51 <saggi> xyang1: I added myself as a reviewer so I'll get an email when it's updated.
14:14:14 <xyang1> I'd like to describe the flow in the spec on how smaug can use this api
14:14:27 <saggi> xyang1: Go ahead.
14:14:36 <xyang1> ok, thanks
14:15:52 <chenying_> A good news. So far, The integration testing of cinder backup plugins has done. about volume backup, restore and delete action.
14:16:19 <xyang1> chenying_: are those in review or merged?
14:16:21 <saggi> chenying: Are you using v2 or v3?
14:16:41 <chenying_> some patches are in review
14:16:54 <saggi> chenying, please post a link
14:17:03 <chenying_> <saggi>: you means cinder api V2?
14:17:09 <saggi> yes
14:17:21 <chenying_> yes we use cinder api V2
14:17:43 <smcginnis> We've actually moved on the v3 now with the addition of microversion in our API.
14:18:41 <xyang1> saggi: I scanned your repo, it looks like currently there is code for backup and restore, but not for replication yet.  is that right?
14:18:51 <saggi> Yes
14:18:52 <chenying_> Ok I will start the work using cinder api v3.
14:19:39 <saggi> chenying, There are some minor things to keep in mind when using microversions though. I'll speak to you about it afterwards.
14:19:59 <chenying_> Ok
14:20:07 <saggi> xyang1: We want to have the full cycle working with good testing and then start adding more features.
14:20:19 <saggi> cycle==backup->restore
14:20:22 <xyang1> saggi: sure
14:21:00 <saggi> I also, had some conversation with Freezer's PTL. I think I gave a strong arguments about them trying to integrate instead of them trying to do everything. We chose to schedule a specific meeting and see
14:21:08 <saggi> how we can integrate.
14:21:36 <xiangxinyong> saggi: integrate with freezer?
14:21:47 <xyang1> saggi: I also heard that they want to do everything thenselves
14:22:01 <saggi> xyang1: I know :) I tried to be political.
14:22:06 <xyang1> saggi: your approach is better
14:22:47 <saggi> xyang1: I am trying to convince them to concentrate on guest assistance
14:22:55 <yinweimac> they want to do what by themselves? for eg?
14:23:16 <saggi> They currently bypass cinder and use their own implementation for backup\replication
14:23:22 <chenying_> https://review.openstack.org/#/q/project:openstack/smaug  some patches about the plugins are open. Welcome to review them.
14:23:42 <chenying_> We have plans to  integrate with freezer
14:23:53 <chenying_> also about okka
14:23:59 <saggi> and now they also want to do monitoring. Which I think is a bit of spreading a bit too thin.
14:24:13 <saggi> guest agent\assitance is a whole different story
14:24:49 <saggi> I spoke with some operators and vendors and they don't really agree about having an agent or doing it through ssh
14:26:17 <saggi> It's something we need to talk to them about or maybe someone else. I don't know if this should be in our scope or not.
14:26:25 <yinweimac> yes, customers care about security issue. agentless is more preferred
14:27:04 <saggi> I suggested using something similar to Ansible (or just Ansible) that deploys code, runs it and then cleans up afterwards.
14:27:20 <saggi> But this is something I need actual operators to way in.
14:28:51 <saggi> Since it's less technological and more preference standpoint. In the end we will need to run privileged code in the guest.
14:29:02 <saggi> No way around it.
14:29:28 <saggi> Unless people don't want application level consistency
14:29:32 <yinweimac> what they want agent to work for?
14:29:46 <yinweimac> monitor failure status?
14:29:51 <yinweimac> ok
14:29:52 <saggi> Before backup
14:29:54 <yinweimac> got it
14:29:55 <saggi> Tell DB to flush all transactions
14:30:04 <saggi> Get their application to flush in memory infor
14:30:15 <saggi> Application level stuff
14:30:48 <saggi> This is classic use case for Smaug as it requires coordination between multiple OpenStack entities.
14:31:24 <sgotliv> saggi, did you have a chance to talk to the Trove community about backups?
14:32:37 <saggi> No, I did not have a chance to speak with them.
14:33:03 <sgotliv> they may be interested too
14:33:06 <saggi> I did speak briefly with the Manilla guys since they are now working on their backup apis.
14:33:57 <saggi> sgotliv: They are not targeted for Smaug v1. (unless we get more resources) but I'll try and see what they think about the project.
14:33:57 <xyang1> saggi: anyone working on manila plugins in smaug?
14:33:58 <sgotliv> and Trove consider to start using Cinder and Manila to backup databases
14:34:41 <yinweimac> xyang1: not yet
14:35:00 <saggi> xyang1: No, we don't have enough resources for that. We wanted to push them to version 1 but we don't see how we can't get that many things stable.
14:36:05 <saggi> xyang1: https://wiki.openstack.org/wiki/File:Smaug-available-protectables.svg
14:36:54 <yinweimac> but that would be a good suggestion and we will take a look at it later, right? saggi? both manila and trove I mean
14:37:50 <saggi> yinweimac: They were in the original tree. If we get more people early enough in the cycle than we'll add more protectables.
14:38:55 <saggi> I prefer stability right now more than features.
14:38:58 <xiangxinyong> saggi: do you hear about ekko? http://git.openstack.org/cgit/openstack/ekko/
14:39:04 <saggi> Respect is something you earn
14:39:26 <xyang1> saggi: are you using cinder backup and restore APIs?
14:39:37 <yinweimac> yes
14:39:38 <saggi> xyang1: Yes
14:39:51 <xyang1> saggi: great
14:39:55 <yinweimac> currently the cinder plugin is based on cinder backup/restore
14:40:10 <xyang1> yinweimac: thanks, I'll take a look
14:41:04 <saggi> I also met up with sgotliv and Sean Cohen which told me about some use cases they have from users at Red Hat.
14:41:26 <saggi> It seems like we could be a good fit.
14:41:39 <sgotliv> saggi, absolutely
14:43:26 <saggi> There are a lot more stuff that happened. But we're running out of time. I'll compose an email to the team so you are all up to date. yuval already has some stuff to figure out how to solve.
14:43:38 <saggi> #topic Status updates
14:43:58 <saggi> I know that you all worked on getting the full backup\restore running by this week
14:44:07 <saggi> How is that doing?
14:44:09 <saggi> going
14:44:20 <saggi> Are we ready for shipping :)
14:44:21 <yinweimac> it works now
14:44:38 <saggi> nova\cinder\neutron ?
14:44:52 <yinweimac> in shenzhen lab we have integrated smaug with hybrid cloud, and both nova/cinder works
14:45:14 <yinweimac> there we don't need restore network, just binding floating ip will do
14:45:46 <saggi> Are you binding them through restore options?
14:45:53 <yinweimac> and I guess in chengdu lab, chenying also have got one smaug with devstak work. Just only cinder there.
14:46:02 <yinweimac> yes
14:46:26 <yinweimac> you mean restore parameters?
14:46:33 <saggi> yes
14:46:49 <yinweimac> no, that's same floating ip we protected
14:47:00 <saggi> Oh, right since they are proxied
14:47:14 <yinweimac> parameter is the feature we need further enhanced
14:47:22 <saggi> are there tests for everything?
14:47:32 <yinweimac> almost
14:48:02 <yinweimac> create plan, create trigger, schedule operation, create checkpoint, list checkpoint, restore checkpoint, etc.
14:48:13 <saggi> awsome!
14:48:56 <chenying_> chenhuayi is developing the restore action of neutron plugin. After that, we will start integrating test with neutron plugin.
14:49:12 <saggi> great
14:49:22 <saggi> #topic Task Flow improvement
14:49:34 <yinweimac> yes, we will have two categories of plugins, one for original openstack, the other for hybrid cloud
14:49:45 <saggi> That's the plan
14:50:13 <saggi> yuval did a lot of good work getting tasks to work like we want them to
14:50:44 <yuval> hey all :)
14:50:46 <yuval> well:
14:50:50 <yuval> Regarding the operations (protect, restore, delete) workflow
14:51:06 <yuval> In the current design, resource tasks start in order of their dependency, but they might invoke async actions, which can complete after their parent actions are invoked.
14:51:31 <yuval> As you might recall, I sent the suggestion for the workflow to the mailing list
14:51:50 <yuval> The purpose of the design is to let Protection Plugin writers the ability to perform the steps required for the operation in the correct order and in parallel, if possible
14:52:12 <saggi> BTW, async actions are polled actions. actions that we wait on someone else (like cinder backup) and the worflow host doesn't invke itself.
14:52:26 <yuval> correct, thanks
14:52:41 <yuval> The workflow issue is highly important. The design need to support all use-cases and we need to complete it quick, as all Protection Plugins depend on it
14:53:33 <yuval> the suggestion I sent includes two types of activities: SyncActivity and ParallelActivity
14:53:47 <yuval> After discussion with Saggi and his impressions from the summit, I believe there is a need for an additional type of activity, which operates before dependent resources Activity begin
14:54:14 <yuval> (this can be used, for example, to tell the guest agent to complete transactions and flush)
14:54:31 <yuval> This concludes for 3 types of activities, which differ by time they are invoked in respect for other resource activities:
14:55:01 <yuval> 
14:55:02 <yuval> PreActivity (which operates before dependent resources activities are invoked. for example, tell guest agent to flush)
14:55:06 <yuval> WorkerActivity / ParallelActivity (which operates in parallel to other resources WorkerActivities. can be used to perform heavy lifting)
14:55:09 <yuval> PostActivity / SyncActivity (which operates after dependent resources completed all their activities. can be used to perform work which depend on child resources status)
14:55:22 <yuval> 
14:55:25 <yuval> You can view an example of such design (currently lacking the PreActivity, and names may differ): https://review.openstack.org/#/c/310385 https://review.openstack.org/#/c/310386 https://review.openstack.org/#/c/310498
14:55:43 <yuval> </p> :)
14:56:34 <saggi> pre\post are used to bookend some actions like quiesce and unquiesce (if this is the proper term)
14:57:33 <saggi> It's important that (A) people review this so it gets in (B) Everyone understands how it works since all the plugins will use this.
14:58:01 <saggi> Changing this interface later effects all plugins. It's important we get this right.
14:58:53 <saggi> yinweimac, I especially want a review from you.
14:58:54 <yuval> I'll send an updated document tomorrow
14:59:38 <saggi> This is the last minute. So I'd like to thank all the new people that joined us today. We hope you come back for the next meeting.
14:59:52 <yinweimac> sorry for the late response, I was busy with our tdr3 demo and incomming demo next week
15:00:00 <saggi> We do want to make it weekly and have one time for US and one time for China.
15:00:03 <yinweimac> I will try to get time to check it
15:00:25 <yinweimac> that's good
15:00:40 <saggi> smcginnis, xyang1, sgotliv, thank you again for helping.
15:00:51 <xyang1> saggi: thank you
15:00:53 <smcginnis> saggi: +1
15:01:13 <saggi> #endmeeting