14:01:50 #startmeeting freezer 14:01:51 Meeting started Thu Mar 31 14:01:50 2016 UTC and is due to finish in 60 minutes. The chair is m3m0. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:52 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:01:55 The meeting name has been set to 'freezer' 14:02:00 o/ 14:02:10 as always, topics are available here: https://etherpad.openstack.org/p/freezer_meetings 14:02:26 who's here for the freezer meeting? hands in the air :) 14:02:28 o/ 14:02:32 o/ 14:02:55 let's wait for more ppl to join 14:03:21 Hi 14:03:30 o/ 14:04:07 PTL 14:04:11 come on 14:05:09 guys before start the meeting, feel free to modify the agenda and add or edit topics 14:05:45 #topic new PTL 14:06:10 well the first thing is to congratulate slashme in his new role 14:06:34 slashme congrats! 14:06:35 under his ruling he will provide food and water to freezer developers 14:06:47 ;) 14:06:49 slashme: Congratulations :) :) 14:07:33 slashme: congrats! 14:07:39 :):) 14:07:47 can we move forward? 14:07:54 get_backup_args 14:08:04 Congratulations 14:08:46 o/ 14:08:54 Make money 14:09:31 freedom 14:10:15 As said in my candidacy, my goal for freezer during the newton release cycle will be to improve Freezer stability, user experience, documentation and interoperability. I really look forward to work with all of you and I think we couldn't have a better team to work on Freezer. :) 14:10:16 hi 14:10:53 wiser words haven't been spoken 14:10:55 I'm also very happy to see more and more people joining the dev team, this is a very good sign. 14:11:25 next topic 14:11:38 #topic integration tests 14:11:52 ddieterly: would you like to comment on this? 14:12:05 sure 14:12:30 the freezer repo and the freezer-api repo both have a gate that runs tempest tests 14:12:59 this review has good examples of how to write tempest tests https://review.openstack.org/#/c/297360/ 14:13:19 so, the infrastructure is all there and there are examples for people to follow 14:13:33 there should be no excuses for not writing tests now ;-) 14:14:03 How are we going to manage integration testing for the freezer-agent? I see a lot of example in other project on how to test api's, but how will we test an "executable" ? 14:14:29 we could test the api with the agent 14:14:33 and the scheduler 14:14:34 you can write the tests without using the tempest rest apis 14:14:52 bash/python ? 14:14:55 just write code under the tempest plugin that follows the examples 14:15:22 the import thing is that there is a gate that will invoke the tests 14:15:28 slashme, yes similarly on how the jobs are crated in hlm 14:15:29 you can write any code you like 14:15:39 So tempest would still be executing them. Seems perfect then. 14:15:49 yes 14:16:08 you can also invoke any other code using the post/pre test hooks in the gate 14:16:52 that code can be outside the tempest tests dir 14:17:31 you all now have the power :-) 14:17:49 thanks for that ddieterly 14:18:01 daemontool my pleasure 14:18:50 ddieterly thanks a lot :) 14:19:29 next? 14:19:41 sure 14:19:53 #topic mitaka branch for ui 14:20:03 slashme any input on this? 14:20:10 daemontool 14:20:18 well we have to do it :) 14:20:21 next week? 14:20:30 sure, that could be great 14:20:38 so we can start the newton development 14:20:40 Mar 28-1 R-1 Final RCs and intermediary releases TC member self-nomination 14:20:40 Apr 4-8 R-0 Mitaka release TC member election 14:21:16 we can work on newton even now 14:21:23 master actually is newton 14:21:29 stable/mitaka is mitaka 14:21:56 but the ui needs that branch and then we can move to newton 14:22:28 we can do that next week 14:22:45 next topic? 14:23:29 #topic pending reviews 14:23:36 guys please review this: 14:23:40 https://review.openstack.org/#/c/297119/ 14:23:40 https://review.openstack.org/#/c/295220/ 14:23:40 https://review.openstack.org/278407 14:23:40 https://review.openstack.org/280811 14:23:41 https://review.openstack.org/291757 14:23:41 https://review.openstack.org/296436 14:23:44 ok 14:23:52 I think we need to change something 14:23:55 in the gate job 14:24:18 we need to differentiate for job execute on master 14:24:23 and previous releases 14:24:28 I'm not sure how to do this 14:24:38 we need to have a conversation with infra 14:24:41 in the infra channel 14:24:45 to understand 14:25:50 agree on that, do you want to have that conversation with them? 14:27:24 I can do it, it's not super urgent 14:27:29 but sooner the better 14:27:56 there's something about the incremental engine 14:28:05 I'd like to get some feedback for 14:28:16 (if we are ok with the previous topic) 14:28:48 tar recognize the data backup file without the need of the metadata 14:29:00 #topic tar metadata 14:29:11 now, how can freezer recognize which engine was used 14:29:17 for the backup 14:29:23 options are 14:29:33 1) read that in the metadata 14:29:48 2) the user has to set --incremental-engine or --engine 14:29:50 then restoring 14:29:59 3) add few bytes in the rsync data blob 14:30:11 like freezer_rsync_v0.1 14:30:25 4) change the current object/directory structure name 14:30:33 prepending tar or rsync at the beginning 14:30:38 like rsync_hostname 14:30:42 and rsync_metadata 14:30:44 same for tar 14:30:51 any thought? 14:31:33 if we are going to store the metadata in the storage as well could be a great place to write in the metadata 14:32:02 because for backups made with the agent we don't have support from the api 14:32:26 ok 14:32:33 each option has pro and cons 14:32:33 I would prefer naming in sub-folders or backup name 14:32:42 reldan, so #4 ? 14:32:55 yes, and I don’t see any drawback for this approach 14:33:12 Well, non-backward compatible 14:33:14 that was the one I thought at the beginning 14:33:30 the only one that is backward compatible 14:33:33 slashme: any approach is not backward compatible 14:33:35 is #3 I think 14:33:46 Yes but it is very bad 14:33:51 :( 14:33:57 tar do that for the metadata file 14:33:57 It is sad, yes 14:34:02 You need to get the object first to have the metadata 14:34:05 is you open the metadata 14:34:11 at the very beginning 14:34:22 there-s GNU TARv1.27 something 14:34:34 but it's also 14:34:36 complicated 14:34:42 to write 14:35:23 cause you have to place the parsing after the data went out of the pipe 14:35:29 I also think #4 is the best option. 14:35:38 so #4? 14:35:52 it's important, cause we need to modify also the remove part 14:36:16 ok 14:36:58 And if using #4, do we go for : engine_backupname_hostname_timestamp_level or engine/backupname_hostname_timestamp_level 14:37:32 or engine/hostname/backupname/timestamp/level ? 14:38:06 it would be different code for swift 14:38:29 reldan, you had an idea about this if I remember well? 14:40:01 I would prefer sub-folders sub-containers 14:40:07 some logical tree from storage 14:41:57 let's do this 14:42:00 let's finish rsync 14:42:04 basically working 14:42:16 and then asap change the object/file format 14:42:39 ? 14:42:48 thoughts? 14:43:12 let me know in case 14:43:35 one more thing, it's important to generate metadata 14:43:41 for the current cinder and nova backup modes 14:43:49 wheter they are native or not 14:43:53 we need to save some metadata 14:45:09 the baas needs to be improved 14:46:00 vannif: any other input for the backup metadata? 14:46:42 i don't like too much the idea of using paths to store information 14:47:56 vannif: any other idea. if only you have is filesystem 14:48:14 a path is somewhat 14:48:18 already there 14:48:32 but I'm not sure we can use a different approach with swift 14:48:44 unless we use SLO rather DSO 14:48:55 that's a whole new discussion anyway 14:48:58 :) 14:49:12 slashme, we really need to add some metadata 14:49:24 to the backup done on VMs and Volumes 14:49:28 reldan, ^^ 14:49:32 very important 14:50:00 guys we have 10 min left 14:50:04 ok 14:50:09 two ways - have something in database, in tree or hashmap 14:50:12 second - path 14:50:20 first - database, index 14:50:25 You mean backup metadata (stored in the api) or engine metadata (stored in a storage media) 14:50:41 backup metadata 14:50:57 that will be displayed by the python-freezerclient 14:50:59 and the web ui 14:51:42 Okay, What should we add ? 14:52:10 if now we execute a backup of a cindervolume 14:52:12 with cinder native 14:52:19 it's not recorded anywhere 14:52:22 afaik 14:52:24 What do we store right now ? Just the volume/instance id ? 14:52:30 I think nothing 14:52:39 we need that 14:52:43 volume/instance id, timestamp 14:53:00 Yes, we need common approach 14:53:03 that basic info at least 14:53:06 for all types of backup 14:53:10 yes 14:53:18 it starts to be critical now 14:54:45 slashme, probably we should define the critical things were we should focus on? 14:55:07 For newton: 14:55:36 1st: freezer-agent plugins layers 14:56:16 ok 14:56:44 guys 3 min left 14:57:00 anyway let's define that 14:57:02 Integration tests. We can't conitue like this where every time we try a feature, it is broken 14:57:08 ok 14:57:39 I'd say these are our top priority. Let's define the rest in an etherpad 14:57:45 ok 14:57:48 baas 14:58:03 so many users ask me for baas and dr 14:58:15 baas top 14:58:50 first we need real multitenancy 14:58:59 Then our main blocker to having baas is oslo policy integrtation 14:59:03 guys let's move to #openstack-freezer channel 14:59:07 ok 14:59:09 slagle, ok 14:59:13 #endmeeting