18:02:10 #startmeeting trove 18:02:11 Meeting started Wed Mar 5 18:02:10 2014 UTC and is due to finish in 60 minutes. The chair is hub_cap. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:11 o/ 18:02:13 o/ 18:02:13 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:13 o/ 18:02:15 o/ 18:02:16 The meeting name has been set to 'trove' 18:02:16 o/ 18:02:16 o/ 18:02:17 what up 18:02:18 heyo troops 18:02:23 o/ 18:02:25 o/ 18:02:27 o/ 18:02:34 o/ 18:02:36 o/ 18:02:41 o/ 18:02:51 okey doke 18:02:55 o\ 18:03:01 o/ 18:03:01 o/ 18:03:04 #link https://wiki.openstack.org/wiki/Meetings/TroveMeeting 18:03:09 o/ 18:03:19 ahh so many head+hand combos 18:03:26 #topic guest agent upgrades 18:03:28 o/ 18:03:29 so whoes up for this? 18:03:40 esp take it away 18:03:44 esp 18:03:44 LETS ROCK! 18:03:48 k 18:03:49 :) 18:03:49 hollah 18:03:53 \m/ 18:04:02 sorry Aliens reference 18:04:12 so I put together a wiki page to describe the work for implementing guest agent upgrades 18:04:22 i think amcrns ascii art was too 18:04:23 link: https://wiki.openstack.org/w/index.php?title=Trove-Guest-Agent-Upgrades 18:04:35 #link https://wiki.openstack.org/w/index.php?title=Trove-Guest-Agent-Upgrades 18:05:05 so what shall we discuss with this, besides the fact that this is an epic awesome wiki page 18:05:14 esp: nice and thorough job on this 18:05:15 +1 18:05:22 esp: did you use visio for this? 18:05:24 hub_cap: I made cartoons for everyone to enjoy 18:05:24 yeah thats what i was thinking. 18:05:33 good job on the wiki page, esp 18:05:33 esp: :) 18:05:33 kevinbenton: omni graffle 18:05:45 ah i was hoping for a flip book 18:05:50 :-P 18:05:52 its going to print cp16net 18:06:00 give it 6wks for first run 18:06:07 esp and the interwebs 18:06:14 and mysql work bench for the DB table 18:06:19 I am guessing not many people had time to review this 18:06:20 Gonna make some time to review it today, and will comment on the wiki page. 18:06:29 i put some comments @ the bottom of the wiki already 18:06:31 yea im skimming it now, but it looks similar to what weve discussed before 18:06:34 and esp kindly responded 18:06:34 and at the mid cycle 18:06:38 juice, +1 18:06:40 juice: no, it was kinda late getting into the agenda 18:06:43 should we have a period for comments - say a week and then close it and call it approved by next week 18:06:59 I guess I have some confusion over process 18:07:01 juice +1 18:07:08 yeah, i'd just ask that those with specific packaging needs add information to the wiki so it's considered appropriately 18:07:15 unless there are some questions right off the top that esp can answer now... 18:07:16 juice, +1 18:07:19 yep, take some time to tear it apart 18:07:29 maybe we need to finish up the talk we started at the summit, but wasn't one idea that the blueprint would link to the wiki, and we'd approve the blueprint first? 18:07:42 grapex: yep 18:07:51 I did link it though.. let me dig up the bp 18:07:57 yea grapex, but im just getting back to the swing of things 18:08:03 i think hub_cap should go over the new process so everyone knows 18:08:05 esp: create a bug describing the missing link to the wiki 18:08:05 hub_cap: Me too :D 18:08:07 o/ 18:08:12 this would be a perfect candidate for part1 and part2 of what we talked about at the mid cycle 18:08:20 kevinconway: lol 18:08:23 part1=bp approve w vision, p2=wiki of awesomeness 18:08:30 true 18:08:37 so we are a bit backwards this time 18:08:41 hub_cap: so when do we approve what's in the wiki? 18:08:51 grapex: id say after we get to the bp vision 18:08:56 but honestly 18:09:01 this one is obvi, we talked about it 18:09:04 so to me the bp is approved 18:09:14 since we had a big duscussion on the mid cycle etc... 18:09:19 *Discussion 18:09:19 Probably it is, I'd just like a chance to read it first, similar to a pull request. 18:09:24 def dude 18:09:25 and then beginning kinda describes what the bp would be 18:09:25 grapex: https://blueprints.launchpad.net/trove/+spec/upgrade-guestagent 18:09:31 #link https://blueprints.launchpad.net/trove/+spec/upgrade-guestagent 18:09:31 lets defer maybe till next wk 18:09:32 Thanks esp! 18:09:35 between the intro and goals 18:09:35 and use this as our poster child 18:09:39 grapex: np 18:09:46 and give esp a chance to dust off the bp 18:09:48 if needed 18:09:48 yeah that sounds good 18:09:53 I'm good with approving the bp. There might be some design details that need to be figured out. 18:09:55 i've not looked over this yet.. 18:09:59 is the bp vision-y enough 18:10:11 So- the whiteboard section- seems like there could be some overlap between that and the wiki 18:10:25 sounds good to me. thx for looking at it. 18:10:28 should we add inspirational quotes to BP to make them more visiony? 18:10:42 :) 18:10:47 yea i hate whiteboard grapex 18:10:52 kevinconway: I thought all Wiki articles were to begin with "In the course of human events.." 18:11:13 So let's remove the whiteboard part for now 18:11:17 Seems redundant 18:11:19 +1 18:11:35 so even tho this bp is somewhat approved in our minds, lets use it as our next wk (mon/tue) discussions w core 18:11:41 esp: ? 18:11:48 k, I will transpose the whiteboard stuff to the wiki 18:11:59 wrong person kevinbenton - sorry 18:12:15 I'm adding it as an agenda item to next weeks meeting. 18:12:23 sweet thx SlickNik 18:12:30 we can use that on monday to discuss bps 18:12:33 for everyone else can we get your comments on this before next weeks meeting? 18:12:43 we still need to discuss the finer points of log delivery too 18:12:43 hub_cap: I have a confession- I haven't read this bp yet! 18:12:45 * grapex blushes 18:12:51 since the wiki is already there, no need to hold up the review of that, right? 18:12:59 * hub_cap giggles w grapex 18:13:12 juice: sure, anyone can review now 18:13:25 we will discuss mon or tue a set of the bps 18:13:28 i believe we said monday 18:13:30 iirc 18:13:39 w might need to consult the etherpad 18:13:43 So about the whiteboard 18:13:52 * amytron rolls her eyes 18:13:55 grapex: it's probably best.. I don't know if the bp and wiki are exactly 1 for 1 18:14:00 Maybe it would work for just notes such as "implemented by the following gerrit item" 18:14:09 instead of sort of defining what the wiki is already saying 18:14:46 lol 18:15:12 grapex: it can work but its VERY Strict in the actual syntax 18:15:13 if we want to discuss the merits of the blueprint this isn't a great candidate because you have to consider what you would like in the blueprint in absence of the wiki 18:15:24 grapex: I'll take a look at it today and see if I can clean up the whiteboard (delete stuff) 18:15:25 grapex: That's what I use it for. eg: https://blueprints.launchpad.net/trove/+spec/trove-tempest 18:15:40 and how much information you as a core team would need in order to send someone off to do the research that esp did for this in order to generate a wiki doc 18:15:43 SlickNik: ++ 18:15:48 just clikc that little help link if anyone has issues 18:15:56 its annoying, but nice to see for muliple patches 18:16:14 oh sorry grapex 18:16:16 im thinking work items 18:16:24 whiteboard is fine for this ++ 18:16:48 ok so lets give esp till mon to clean it up 18:16:56 and we can review it mon as a group in the channel 18:17:16 thx everyone! 18:17:47 so moving on 18:17:57 #topic replication/clustering update 18:18:32 so whos on first? 18:18:33 imsplitbit, i guess its your turn ? 18:18:38 hmm 18:18:42 So just a quick update on this. 18:18:43 ok so what is clear to me 18:18:48 oh good 18:19:07 * imsplitbit waits for SlickNik to tell us 18:19:14 whats the status with meta-data ? 18:19:31 We had a teamspeak meeting about this yesterday. 18:20:03 SlickNik, cool, have you got the meeting minutes ? 18:20:32 i think 3 things were decided, right? 18:20:32 Main outcomes were 1. Imsplitbit to push up the API code with few changes to schema based on comments. 18:21:09 2. Folks from tesora to come up with bp for replication v1 18:21:52 3. And the tesora folks are eager to get started on implementation based on the bp. 18:21:59 horray! 18:22:06 please elaborate p.2 and p.3 18:22:08 wait, is that last one a decision? 18:22:30 also wasn't amcrn doing something moving forward on clustering? 18:22:37 lol kevinconway 18:22:40 well that tesora was gonna work on it was the decision. 18:22:47 * hub_cap glares at amcrn 18:22:53 as discussed at the last meeting we're going to have common meeting in hangout for everyone who interested in taking part in replication implementation 18:22:54 amcrn: are you sending the updated api ? 18:23:02 correct, there's still work being done in fleshing out the rest of the clustering api (building on the replication contract that everyone is preliminarily agreed upon) 18:23:11 vipul: i haven't updated it based on your comments last night yet 18:23:17 i can send it out, tomorrow? 18:23:33 yea sure.. i think imsplitbit and doug_shelley66 shoudl be itnerested in it 18:23:43 heh lets shoot for fri amcrn 18:23:53 we have to prepare for a talk tomorrow 18:23:55 fair enough, today is an all-day deal 18:23:56 :P 18:24:00 as is tomorrow 18:24:02 :D 18:24:04 word 18:24:06 denis_makogon: i think this feature is concise enough that it makes sense to have a dedicated owner 18:24:08 SlickNik, testing using tempest strategy ? can we talk ? 18:24:19 denis_makogon: you can work wth doug_shelley66 to see if there are tasks he'd like you to take on 18:24:31 SnowDust: u should put things on the meeting agenda 18:24:42 lets finish this first before we get started w/ somethign else 18:25:13 hub_cap : thought it was the last .. but will wait for the last in the agenda 18:25:37 ok good to go? (looks like it might be SnowDust ;) ) 18:25:59 imsplitbit: doug_shelley66 SlickNik good? 18:26:08 so where I'm unclear 18:26:08 i'm good 18:26:12 good here. 18:26:15 go on imsplitbit :) 18:26:17 is we never decided on if instances had nodes 18:26:18 or not 18:26:19 Thanks guys! 18:26:34 i think that vote will need to happen enxt week 18:26:38 imsplitbit: lets discuss that next wk 18:26:38 ok 18:26:39 that doesn't impact replication though.. 18:26:42 correct 18:26:45 during the monday bp talk 18:26:48 if the bp is ready 18:27:07 do I own that? 18:27:08 :) 18:27:18 i mean, whoever owns the bp owns it 18:27:23 ok so you folks are going to decide that on monday. so i don't need to study for next weeks trove meeting right? 18:27:28 i think tesora said they were going to take it over? 18:27:38 kevinconway: no, plz study 18:27:43 all wknd 18:27:43 yes the BP for replicatoin v1 18:28:07 we should re-approve all the inflight and new features 18:28:16 ok 18:28:19 amcrn: SlickNik vipul grapex how do yall feel about that? 18:28:29 the inflight juno's and the not impld one 18:28:31 ones 18:28:36 not all on monday 18:28:39 hub_cap: I like it 18:28:39 agreed 18:28:40 but the ones that are worked on 18:28:42 i like that 18:28:44 or are about to be 18:28:44 hub_cap: +1 18:28:49 ok ill clean it up and unapprove 18:28:52 sorry in advance 18:28:56 but ther eis no reason to fret 18:29:04 unless u have no wiki pages!!!!! 18:29:05 ;) 18:29:23 but we wont be terribly strict, lets work thru it liek a happy family :) 18:29:30 ok so now we move on! 18:29:32 good? 18:30:17 but what about... 18:30:38 dont say it 18:30:44 #topic tempest update 18:30:45 :) 18:30:48 SlickNik: go go go 18:31:15 So the initial tempest tests for trove are approved. 18:31:20 hooray 18:31:28 yaaaaaay 18:31:29 It's currently in the merge gate. 18:31:44 #link https://review.openstack.org/#/c/69501/ 18:32:01 that is awesome! 18:32:03 Next steps are to make the job non-experimental 18:32:23 SlickNik, what's the status of dib elements ? 18:32:25 So other projects (devstack!) can gate on it. 18:32:29 how long is it taking SlickNik ? 18:32:36 SlickNik: thatll be SO cool! 18:32:42 currently it's about an hour. 18:32:45 proly wont gate until juno 18:32:46 ouch 18:33:04 but its a good frist step 18:33:10 awesome 18:33:19 cp16net: I need to identify the tests that we actually care about to reduce the execution time. 18:33:26 test-refactoring for tempest .. ideas ? 18:33:29 denis_makogon: That's the next step 18:33:51 hey im putting to open discussion 18:33:56 ill bb in a min 18:34:00 #topic open discussion 18:34:05 sorry, conflicting meetings :/ :/ 18:34:14 SnowDust: We don't have any code to refactor yet. 18:34:37 We need to get _a_ datastore's tests into tempest before we can get multiple datastores. 18:34:37 SlickNik ur bp says so .. https://blueprints.launchpad.net/trove/+spec/trove-tempest 18:34:50 SlickNik: So, about rewriting the clients again. It seems like the desire of the Tempest authors to rewrite the clients stemmed from a need to test both JSON and XML 18:35:17 SlickNik, got ur point 18:35:35 however, since our existing client can do that, and since XML is going away anyway, can we start using the client for the tempest tests? I'd hate to have to duplicate the effort of writing a Python rich client binding for the Tempest tests. 18:36:09 grapex: That's something that I'm going to start looking into. 18:36:15 I think the tempest project doesn't want to be dependant on all the clients 18:36:24 I'd hate to re-implement _all_ of our client functionality. 18:36:40 cause it is a dependancy nightmare 18:36:42 sounds like copy pasta 18:36:43 robertmyers: That would be ironic since if you run Tempest you're essentially dependent on literally all of integrated OpenStack. :p 18:37:01 cp16net: :) 18:37:20 grapex: depends on how you install tempest 18:37:25 robertmyers / grapex: There seem to be different schools of thought on that. And it's a conversation I can't wait to start with some of the Tempest folks :P 18:37:29 I get that technically it's challenging to include a dependency on the client, but since our client doesn't depend on anything else it shouldn't be that bad. 18:37:46 SlickNik: Be sure to CC me when you do 18:37:53 grapex: definitely will 18:38:03 SlickNik: maybe they are afraid of a slippery slope 18:38:13 I have seen some quasi- philosophical reasons for re-creating the clients, such as not being able to trust the original clients 18:38:42 which I think points to a completely different problem. But yeah, let's talk about it at least before we have to start re-writing the entirety of the Trove python client. 18:39:16 agreed 18:39:51 That's pretty much all I had. 18:40:04 I have a topic, can we chat real quick about adding datastore type details into backup metadata for backups stored in swift? 18:40:04 as an update. 18:40:35 This is not done today, but I am not sure if there is a BP for it 18:41:11 demorris: I thought we agreed to add to the docs "users are expected to write down the specific datastore type they used to create each backup so they can remember it later." 18:41:15 grapex: https://github.com/openstack/tempest/blob/master/tempest/clients.py#L434 18:41:18 its fairly straightforward, store the type/version details in the backup metadata, so you can tell what datastore type and version a particular backup belongs to 18:41:21 so i'm wrong 18:41:30 grapex: post-it notes yah? 18:41:43 grapex: they do what you want 18:41:49 grapex: yes, they should write it in sharpie on their hand 18:41:58 imsplitbit: Exactly. We could make them Trove themed and sell them for profit with proceeds going to the OS foundation. :) 18:42:13 demorris, good approach, i think it's very significant since Trove will have multiple datastores 18:42:21 with our trove logo 18:43:18 demorris, we could deal with it, you want to store type/version in metadata or in the backup model ? 18:43:41 demorris: Sounds like a good idea. I haven't seen a bp for it. 18:43:47 someone want to open one? 18:44:03 denis_makogon: true, that might actually be better so then you can query it via Trove 18:44:20 demorris, so, store in the model ? 18:45:38 well I suppose it depends on what we want to accomplish. One of the main reasons for this is so that you can prevent a user from trying to restore a backup to the wrong datastore, but it could also be useful to help with filtering data stores that are valid 18:45:40 demorris, since backup models hold the instance_id we could always retrieve any data from instance model, if instance still exists 18:45:56 it is possible that the instance may be deleted though 18:46:10 yes, thats what i say =) 18:46:31 We do store the strategy in the model, and it's unlikely that different models share the same strategy, so there's that. 18:46:45 demorris, i'll vote for model instead of meta 18:47:38 if i have a percona backup, and want to restore to maria .. that should be allowed 18:47:52 denis_makogon: k, you want to BP it? 18:47:57 it seems that a backup shouldn't be tied to a datastore, rather.. it should be tied to a 'restorer' or whatever 18:48:00 it can get tricky too when you consider MySQL, MariaDB, and Percona…backups via XtraBackup are not strictly supported when backing up with one and restoring to another (but there are cases where it can work) 18:48:04 vipul: Yes, if the strategy is the same, it's likely that the restore will work _across_ datastores. 18:48:31 SlickNik: I think you're right- if the strategy is stored in the model that alone should work for validation 18:48:36 demorris, yeah, i could, but if i'll forget, could you do this ? 18:48:54 so we'd need to someplace in code to tie a strategy to one or more datastores 18:48:59 I think it should support restoring across types/versions, but should be up to operator on what they want to support and not support 18:49:04 #action File a BP for adding datastore type/version to backup model 18:49:10 party time im back 18:49:34 wait so why are we adding datastore type to backups now.. i'm confused 18:49:39 vipul: yes, today this exists only in the ga.conf. But we might want to codify this some more. 18:49:58 SlickNik: yea, mainly want to do validation on a restore 18:50:32 vipul, first approach - block from restoring instance with wrong datastore, that differs from datastore of the instance that was backuped 18:50:57 yes, validation on a restore, and secondarily filtering per datastore (only show me backups for MongoDB, exclude all others") 18:51:00 denis_makogon: we can derive that from the 'Backup stragey' already stored on the backup 18:51:29 i guess i'm not a big fan do de-normalization.. we've got this info that can be derived in several ways 18:51:33 vipul, i guest it'll be the long way 18:51:45 vipul: until you delete the instance 18:51:46 *i guess 18:51:52 robertmyers: it's a soft delete 18:52:12 trove can do a search for any instances that the backup originated from 18:52:27 sure I gues 18:52:48 so am I hearing we can already make the association? 18:52:54 can you drop the backup strategy from the backup model and replace it with the datastore reference...from there you could derive the backup strategy after it has been associated with the datastore 18:52:56 vipul, lets do not relay on soft delete 18:52:56 so then all it would need is a view change 18:53:04 on the details for a backup 18:53:28 robertmyers, agreed 18:53:44 robertmyers: +1 18:53:52 juice: so do we have to work on migration paths for changing backup methods? 18:53:54 so, theres two options: 18:53:59 i agree that there are issues with relying on soft-deletes, because there may be truncates that a deployer wants to do 18:54:22 juice: i think that's the same as what was beign suggested.. add a datastore reference to backups 18:54:27 1. have an ability to retrieve type/version by backup_type. 18:54:47 2. add datastore version and type into the backup model 18:55:00 approach: strict validation on restore/recover 18:55:46 let me file the BP, wiki page and ML for that, and then review it 18:55:48 ok ? 18:55:49 denis_makogon: feel free to BP this, we can disucss it during BP review 18:56:04 i would also put in robertmyers suggestion in the BP, which doesn't require any DB change 18:56:23 I agree that this is something we should do (impl details aside). 18:56:28 So let's file the bp. 18:56:53 robertmyers, i'll file it and then i'll ask you to update the whiteboard 18:57:03 ok 18:57:05 could i take last minutes ? 18:57:08 denis_makogon: use the wiki 18:57:20 ok 18:57:37 remember to use 18:57:40 #link https://wiki.openstack.org/wiki/TroveBlueprint 18:58:04 i'd like to invite all of us into the discussion related to Point it time recovery 18:58:18 denis_makogon: go go go, 3 min 18:58:42 i already filed the BP, did the draft implementation (which was done easily) 18:58:51 and sent the email 18:59:19 so, please, don't be the shy =)) review them and join the ML thread 18:59:36 denis_makogon: can you conver teh google doc to wiki 18:59:44 convert 18:59:49 +1 vipul 18:59:52 then print out the HTML 18:59:56 vipul, yes, i'll do that 18:59:57 feed it to an XLST template 19:00:00 and generate markdown 19:00:02 kevinconway +1 19:00:17 yes all should be wiki 19:00:20 we will review it on monday too 19:00:25 I don't quite understand the issues with looking at point in time recovery based on what was stated on the ML. It is true that there are tons of impl nuances across data stores, but the concept of the API to reference a point in time for a restore both on an existing and new instance is valid 19:00:29 also i'd like to point your attention onto the dblog feature, which wasn't even once reviewed since it was published 19:00:30 and when we post to ML, i don't think it's necessary to copy / paste the entire thing.. i think you shoudl include an overview and then point people to the Wiki 19:00:39 denis_makogon: plz add your log bp and your PIT bp to the bottom of the meeting page 19:00:45 we will use that as the starting point for discussion 19:00:56 ++ vipul 19:01:13 # link https://blueprints.launchpad.net/trove/+spec/mysql-point-in-time-recovery 19:01:33 #link https://blueprints.launchpad.net/trove/+spec/dbinstance-log 19:01:39 #link https://blueprints.launchpad.net/trove/+spec/mysql-point-in-time-recovery 19:01:47 #link https://wiki.openstack.org/wiki/TroveDBInstanceLogOperation 19:01:49 good timing 19:01:54 #endmeeting