14:00:29 #startmeeting freezer 14:00:29 Meeting started Thu May 19 14:00:29 2016 UTC and is due to finish in 60 minutes. The chair is ddieterly. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:30 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:32 The meeting name has been set to 'freezer' 14:02:25 hi ice cubes 14:02:29 hi 14:02:39 attendance? 14:02:42 o/ 14:03:00 :) 14:04:10 just us two? wow 14:04:27 agenda at https://etherpad.openstack.org/p/freezer_meetings 14:04:31 feel free to add to it 14:05:28 o/ 14:05:43 haha 14:05:47 hey daemontool! 14:05:55 big hugs 14:06:00 hi ddieterly who's the chair? 14:06:07 i am 14:06:08 ddieterly, only with harms please 14:06:10 haha 14:06:21 of course 14:06:24 ok 14:06:37 can we get some items on the agenda? 14:06:41 ok 14:06:47 the url please? 14:07:00 this one right https://etherpad.openstack.org/p/freezer_meetings 14:07:05 https://etherpad.openstack.org/p/freezer_meetings 14:07:13 hello 14:07:53 clsacramento, hi 14:08:01 greetings to all 14:08:15 daemontool: buon giorno, come stai? 14:08:21 #topic Tenant resource backup (aka DRaaS) 14:08:21 ddieterly: hi 14:08:39 hi szaher! 14:09:12 daemontool take it away, please 14:10:29 about this Tenant resource backup (aka DRaaS), didn't you find out that there was another project doing it already? 14:10:43 smaug? 14:10:59 yes, I think that was it 14:11:02 I'm here 14:11:05 sorry 14:11:22 clsacramento, ola 14:11:22 so 14:11:26 during the summit 14:11:30 the design session 14:11:39 there were more then one person in the room 14:11:41 from different companies 14:11:53 asking for a complete backup of all the tenant resources 14:11:56 such as 14:12:13 vms, volumes, networks config, identity related etc 14:12:19 to be restored 14:12:27 on the same os platform 14:12:31 or on a new one 14:12:38 it is quite interesting 14:12:49 and more and more users are asking for it 14:13:08 yes, I really like this feature 14:13:20 It looks like vmware vApp and it is quite nice 14:13:24 yes 14:13:28 clsacramento, exactly 14:13:35 that would make us competitive :) 14:13:48 so I've tried to summarize somethign here 14:13:49 https://blueprints.launchpad.net/freezer/+spec/tenant-backup 14:13:51 let me know 14:13:53 any feedback 14:14:10 so everythhing backed up taken from the API, 14:14:17 such as metadata, glance image, snapshots 14:14:20 volumes etc etc 14:14:37 daemontool: Ciao 14:14:57 There is a project from Mirantis called Pumphouse they used to do something similar 14:14:57 https://github.com/Mirantis/pumphouse 14:15:13 I would add not only in case of lost, but in case of willing to replicate a lab too, it is very useful for that 14:15:24 I don't know why they stopped but they were backing up resources then upgrade the cloud then restore resources again 14:15:45 May be we can take a look we might find something useful 14:16:03 wondering about load on the api's and performance of that kind of approach 14:16:08 let's reuse whatever can be reused of course 14:16:18 yes 14:16:38 ddieterly, that's a good point, I think it depends on how many you execute simultaneously 14:16:48 or in a short time windows 14:17:59 i think we need more research on pumphouse and smaug 14:19:21 anyone want to take an action item and report back next week? 14:20:03 daemontool: I was thinking, it is easier on the same cloud because all images, volumes and vm emephemeral disks are already in there backend storage, we can only duplicate them. If it is to recover on a different cloud we would have to download all of that to some other media and make it available on the other cloud... 14:20:24 ddieterly: I can do that 14:20:25 ddieterly: I don't know if we can only backup mysql database which contains all the required metadata and Openstack will automatically create this resources once we restore the database again and only backup volumes, instances, images and so ? 14:20:42 s/this/these/g 14:21:03 szaher: I dont think backing up the database is the best approach. We should be able to get all the info we need to backup from the APIs 14:21:09 clsacramento thanks 14:21:25 #action clsacramento to investigate smaug and pumphouse 14:22:11 clsacramento: we need to know about the drawbacks of using this approach when we have a big deployment with hundreds of tenants and resources 14:22:38 szaher i don't follow your first point 14:23:02 ddieterly: the database approach ? 14:23:11 yes 14:23:39 szaher: for example: if I insert a flavor in MySQL database, openstack automatically creates it for me? 14:24:17 clsacramento: I would say that Openstack stores it's own metadata in mysql database :) and I think yes it will create it automatically 14:24:20 ok 14:24:43 szaher: I dont think it does, but I'll check this out 14:24:44 clsacramento: you can worry about neutron namespaces for example as I am not sure if it will be create automatically or not 14:25:04 we need to get the metadata and data from the API 14:25:08 clsacramento: What about when we take the cloud down then up again ?? How Openstack recovers ? 14:25:19 or we'll have tenants executing actions directly to the db 14:25:21 which is not good 14:25:45 let's have a conversation on the freezer chan for this 14:25:50 and move forward with the topics? 14:25:54 :) 14:25:55 Guys, What I am saying here let's review both approaches :) what will happen at big scales ? 14:26:01 ok 14:26:04 daemontool sounds good 14:26:05 daemontool: OK 14:26:08 ++ 14:26:10 #topic DAR 14:26:34 ok, so currently our incremental restores are busticated under certain conditions 14:26:41 seems to be an issue with TAR 14:26:52 DAR handles the condition quite well 14:27:10 could we investigate a DAR engine as szaher has suggested 14:27:11 ddieterly, yes you are right, in my opinion, I'd remove the deps to binaries.. 14:27:18 asap 14:27:36 I'd focus on that, rather investing time in contingencies solutions 14:27:46 sorry, what's DAR? 14:27:49 not sure what you mean 14:27:55 tar, gzip, dar 14:27:58 are all binaries 14:28:01 yes 14:28:02 openssl... 14:28:21 so, you want to implement that functionality directly in freezer instead of using the binaries? 14:28:21 I think we have to work to remove deps to those binaries 14:28:31 yes, there are modules done 14:28:33 that does that 14:28:34 already 14:28:40 or just let users mix and match which ones to use? 14:28:42 so we'll be more portable 14:29:00 I think if we use the python modules, our life is easier 14:29:03 what modules? 14:29:04 also for windows 14:29:04 reldan and I are working on some sort of refactoring freezer to be pluggable so hopefully we will be able to add engines but we need the refactoring part to be done first 14:29:05 like 14:29:20 bzip, tarfile, crypto.io 14:29:21 and so on 14:29:27 the rsync code 14:29:33 probably the rsync code 14:29:38 can remove tar or dar 14:29:44 by 14:29:49 backing up the data 14:29:51 so there is a python module that implements rsync directly? 14:29:51 only based on inode 14:29:54 npoe 14:30:06 we need to get completed 14:30:07 this 14:30:08 https://review.openstack.org/#/c/290461/ 14:30:12 and we have that 14:30:20 so the difference would be 14:30:30 to backup the whole file if the inode is changed 14:30:36 rather backup block 14:30:45 and the tar approach will be quite the same 14:30:58 so tar, dar openssl 14:30:59 etc 14:31:00 can be removed 14:31:09 but 14:31:14 it is just my opinion 14:31:30 is this your implementation, or is it an existing well-known python module? 14:32:03 for rsync 14:32:06 any module 14:32:09 does not suits 14:32:14 that's why that implementation 14:32:41 if we finish taht 14:32:49 then we have both inode and block based 14:33:05 does the rest of the team have an opinion on this? 14:33:23 i'm still not sure what daemontool tool is saying 14:33:41 ddieterly, tar just check 14:33:46 inode modification 14:33:51 and backup the whole file 14:33:51 are we going to implement the functionality directly in freezer or is there an existing python module that does what tar/dar do? 14:34:23 ddieterly: we need to discuss this before taking a decision 14:34:28 yes 14:34:32 of course 14:34:39 szaher, well the engine approach 14:34:44 has been throughtly discussed 14:34:44 trying to understand what daemontool is saying 14:35:17 daemontool: The engine approach Yes and we agreed we will have mutliple engines and the user can choose what is the best one for him 14:35:20 so, it looks like we could implement another 'engine' that uses dar 14:35:27 ddieterly, well yes 14:35:28 or, we could do what daemontool is suggesting 14:35:42 the rsync engine is being implemented 14:35:45 I just do not have time 14:35:48 to finish the restore 14:35:52 daemontool: I thought we are going to remove tar and dar totally 14:35:54 but the backup there it's working 14:35:58 szaher, yes 14:36:09 that's what I'm advising for 14:36:12 well, we don't have dar yet ;-) 14:36:14 haha 14:36:23 :D 14:36:39 ddieterly, let's have this discussion later in the freezer chan 14:36:44 ? 14:36:47 ok 14:36:50 let's move on 14:36:58 daemontool is such a task master 14:37:12 #topic rsync status 14:38:05 ok 14:38:06 so rsync 14:38:09 I need help guys 14:38:16 the restore needs to be done 14:38:25 if anyone is interested let me know 14:38:32 keep in consideration that is a tough bone 14:38:52 #action let daemontool know if you want to help with rysnc https://review.openstack.org/#/c/290461/ 14:39:06 ddieterly, are you interested? 14:39:21 i am working on hpe hlm most of the time 14:39:21 it is also incredibly inteersting 14:39:23 :-( 14:39:24 ok 14:39:28 np 14:39:47 but, i will take a look at it for sure 14:40:16 ty 14:40:20 next topic? 14:40:39 yes 14:40:50 #topic ATT architecture meeting 14:41:14 ok we need to decide how do we manage cases where Company wants to contribute 14:41:17 without write code 14:41:20 this seems like something pierre should be involved with as he is the PTL 14:41:32 like at Architecture level and so on 14:41:36 yes I had a word with Pierre 14:41:37 about this 14:41:46 he asked me to drive it 14:41:54 well, if they want to contribute w/o writing code, they can submit a bp 14:42:11 we need to have a meeting to explain them what freezer is 14:42:14 what it does etc 14:42:18 I think we should do a video 14:42:24 Freezer Intro Video 14:42:26 something like that 14:42:30 an points new comers to that 14:42:33 that would be great 14:42:39 m3m0, ? 14:42:40 ping 14:42:47 you are good with videos :) 14:42:52 he is on holiday 14:42:54 ah ok 14:43:04 m3m0, and or vannif? 14:43:07 m3m0 will be back next week 14:43:11 ok 14:43:27 I'll engage ATT for that 14:43:27 i'm not photogenic 14:43:28 lol 14:43:39 ddieterly, haha well you need to be videogenic 14:43:51 ok next 14:43:54 ? 14:43:58 sure 14:44:09 #topic freezer overview meeting 14:44:13 what is that? 14:44:21 RH 14:44:24 RH freezer 14:44:27 same as ATT 14:44:30 oh 14:44:32 same deal? 14:44:35 yes 14:44:51 so, are you going to arrange the meetings or is pierre? 14:44:51 so they are evaluating the backup/dr technology 14:44:52 to use 14:44:58 I'm going to do it 14:45:01 have to send an email 14:45:02 are the meetings in europe or via video chat? 14:45:06 but wanted to wait next week 14:45:12 video 14:45:19 hangout I think 14:45:29 frescof__, ping 14:45:38 do we have a last freezer presentation? 14:45:44 we could use the summit one? 14:45:54 is that ok do you think? 14:46:00 daemontool: fresco is not at his desk 14:46:02 clsacramento, szaher ? 14:46:04 ok 14:46:08 szaher, ask him please 14:46:10 i think a video meeting with them going over the slides that arun uses would be good 14:46:15 sorry, what is RH? 14:46:21 RedHat 14:46:21 frescof__ is on holiday 14:46:23 ok 14:46:25 ok 14:46:44 I was thinking of Human Resources in Portuguese :S 14:46:49 haha 14:47:12 ddieterly, can anyone make a plain openstack preso out of that? 14:47:17 without companies logos? 14:47:20 #action daemontool will set up meetings 14:47:22 just openstack and freezer logo? 14:47:23 I can do it 14:47:31 ok 14:47:37 daemontool i think it would be easy to sanitize them and use them 14:47:40 it is public info 14:48:13 ddieterly, ok 14:48:27 someone has to do that thought 14:48:31 daemontool just don't tell anyone ;-) 14:48:34 haha 14:48:34 ok 14:48:47 next topic? 14:48:48 #action ddieterly sanitize something 14:48:51 lol 14:49:00 yea, i'm good at that 14:49:12 especially toilets 14:49:42 #topic Golang 14:49:57 did everybody see this? https://review.openstack.org/#/c/290461/ 14:50:00 yes 14:50:07 oops wrong link 14:50:09 ddieterly, you proposed Java in the mail thread 14:50:09 haha 14:50:22 https://review.openstack.org/#/c/312267/3 14:50:52 so, maybe we could benifit from Go in freezer? 14:50:58 ddieterly, I think so 14:50:59 yes 14:51:03 but 14:51:05 awesome 14:51:07 last year 14:51:14 I've tried freezer executing 14:51:17 actions 14:51:20 under pypy 14:51:25 and it was really fast 14:51:58 so we should look for ways to incorporate Go in places to optimize performance 14:52:36 i can't tell if this passed or not 14:52:40 does anyone know? 14:53:03 ddieterly, I don't know 14:53:10 I guess the workflow is not +1 14:53:14 I think there are taks we perform 14:53:20 intensive 14:53:20 is anyone on freezer opposed to using go? 14:53:23 for rsync 14:53:24 math 14:53:29 I'm in favor 14:53:36 just wonder if we really need it and where 14:53:43 daemontool: +1 14:54:30 ok 14:54:33 all good anyway 14:54:36 for me 14:54:43 #startvote Should we allow Go in Freezer when it makes sense? 14:54:44 Begin voting on: Should we allow Go in Freezer when it makes sense? Valid vote options are Yes, No. 14:54:45 Vote using '#vote OPTION'. Only your last vote counts. 14:54:48 a part the fact tha tI do not know Go 14:54:55 from the last comment on the bp, not sure if approved 14:54:59 #vote yes 14:55:05 #vote yes 14:55:10 #vote yes 14:55:27 using new IRC features lol 14:55:32 #vote yes 14:55:42 i've always wanted to use that! 14:55:44 haha 14:55:57 #showvote 14:56:12 #endvote 14:56:12 I would say yes if we really need it badly 14:56:13 Voted on "Should we allow Go in Freezer when it makes sense?" Results are 14:56:42 #showvote 14:57:09 ddieterly, ok, I'm glad you are enjoying it 14:57:12 ok, so the vote results are logged in the meeting minutes 14:57:15 * daemontool hug ddieterly lol 14:57:35 ok, any other items in the time remaining? 14:57:44 I think we are good 14:57:49 not from me 14:57:51 hpe folks have a standup in a couple of mins 14:58:11 ah guys the upgrade to falcon middleware is merged so you can submit changes to freezer-api 14:58:24 szaher cool 14:58:26 and If you have a change upstream please, submit a recheck 14:58:32 daemontool hugs back to you 14:58:33 szaher, brilliant 14:58:46 ddieterly, not from the back please, that doesn't look good 14:58:48 lol 14:59:20 daemontool ok, you can stop the double entendre please 14:59:39 thanks all 14:59:42 ciao everybody! 14:59:53 ciao ciao :) 14:59:56 daemontool: a presto! 15:00:01 #endmeeting