18:01:10 #startmeeting trove-bp-meeting 18:01:11 Meeting started Mon Nov 17 18:01:10 2014 UTC and is due to finish in 60 minutes. The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:12 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:14 The meeting name has been set to 'trove_bp_meeting' 18:01:30 Giving folks a few minutes to trickle in 18:01:48 Agenda at 18:01:50 o/ 18:01:51 #link https://wiki.openstack.org/wiki/Meetings/TroveBPMeeting 18:01:57 ./ 18:02:19 o/ 18:03:03 o/ 18:03:35 #topic Add support for DB2 Express-C datastore 18:03:47 this is johnma's BP. 18:03:57 #link https://review.openstack.org/#/c/133856/ 18:04:08 sorry I thought the meeting was at 1 18:04:12 I am here 18:04:19 it is at 1 ; 18:04:20 ;) 18:04:29 1 o'clock somewhere. 18:04:57 I was trying to add some more details to the bp but I will do that after this meeting I guess and include the feeedback 18:05:05 johnma: the meeting is at 18:00 UTC. 18:05:31 yes I get confused all the time. Sorry about that. Thanks SlickNik 18:05:36 Not a problem :) 18:05:56 o/ 18:06:01 so this bp is to add support for DB2-Express C edition 18:06:36 so the idea is to support the default OS for this first version - ubuntu and fedora 18:07:30 and support the following guest api's: - launch , reboot, terminate, backup, restore, resize 18:08:06 there wont be any clustering or replication support in this version since its the Express-C edition 18:08:48 johnma: How close is using DB2-Express C to MySQL? Would you say its a close to it as is Percona and Maria? 18:09:23 the way we create users and grant provileges are very different 18:10:30 between DB2 and MySQL. for one there is no notion of a DB2 user. for creating user, we have to first create an OS user and then DB2 privileges to it 18:11:01 johnma: Another question I had was around obtaining packages for DB2-Express C? Are they freely available on an apt-server somewhere? What is the redist license? 18:11:09 and there isnt a single table like MySQL where we can access and get the privileges for a user. 18:11:44 yes thats very good question SlickNik and one reason why I need to update the spec about the Kilo-1 milestone which I dont think we will make for this reason 18:12:30 this DB2 version is the free version, so there isnt any problem in that front. The problem area is that there is a registration process that user needs to go through first to get the packages 18:13:07 the registration process is free but I have been told that its there to prevent users from embargoed countries from downloading the packages 18:14:13 so we need to figure out a way to work around the registration process. We developed a prototype for the DB2 plugin with most of the features I mentioned above - (we need to implement the resize and backups apis's) which is why I thought we could make the Kilo-1 milestone 18:14:49 but we need to find a work around for the registration process 18:15:23 Okay, all of that is good information to put in the specs (once it's figured out). 18:15:24 johnma: it looks like features such as replication require a yearly subscription? 18:15:36 sure will do that SlickNik 18:15:45 is that restricted by license or by package? 18:16:40 amcrn: Express-C version doesnt support high availability which is why we arent going to implement replication , clustering features for this version. Like you said for those features , user will require a subscription 18:17:06 by license I believe 18:17:08 Merged openstack/trove-specs: Add requirement for APIImpact flag https://review.openstack.org/132346 18:19:38 as far as integration tests, what kind of tests do we usually run? 18:19:49 I have only included unit tests in there 18:20:43 johnma: Tests to test pretty much all the functionality that the datastore offers. 18:20:55 ok 18:20:59 So in this case, that would be provisioning, and backup and restore. 18:21:17 sure, sounds good. thanks SlickNik 18:21:40 One thing I'm working through is how we can leverage an external CI system for third party tests, and what those requirements look like for additional datastores. 18:22:07 So I'll keep you in the loop with that info. 18:22:31 yes, that would be great. Appreciate it. 18:22:39 Thanks johnma! 18:22:53 Any other questions? 18:23:14 I am good on this 18:23:27 #topic Add support for Apache CouchDB datastore 18:23:36 yes thats me again 18:23:42 #link https://review.openstack.org/#/c/133849/ 18:23:59 Pretty much the same discussion applies here as well. 18:24:04 so this is to add support for another datastore - Apache CouchDB 18:24:08 thats right 18:24:39 again for this , we are planning to support the following guest agent api's: - launch 18:24:40 - reboot 18:24:42 - terminate 18:24:43 - resize 18:24:45 - backup 18:24:46 - restore 18:24:48 - replication 18:25:56 couchdb does backups through their replication interface. So they dont have an explicit backup solution. 18:26:35 One question I had is , is there any reason why the other no-sql datastores dont support managing databases and users (create/list/delete 18:26:36 ) 18:27:26 johnma: There's feeling it isn't necessary. Those API calls are actually viewed as an extension to Trove rather than a core feature. 18:27:41 again the plan is to support CouchDB on ubuntu and fedora for this release 18:27:46 johnma would you like to support them for CouchDB? 18:28:22 johnma: Because the implementations chose not to implement that extension. Frankly folks felt it wasn't absolutely necessary, and the users extension has always been somewhat controversial. 18:29:22 I was thinking of implementing those. Again the product team hasnt mandated that. I might have to get back with them and see what their use case is 18:30:17 to be honest I havent gotten that far as to prototyping that so I dont know how much work is going to be involved to implement it 18:30:47 johnma: I'd recommend an initial spec without them, which you can revisit later as part of another spec if you feel that it's really needed. 18:31:02 since it's an extension, it should be easy to separate out. 18:31:07 sounds good. i will take your advice SlickNik 18:32:36 Thats all I had. Any questions or concerns I can address? 18:33:19 We are planning for clustering work for the next release so wont be doing clustering in this one 18:33:50 Sounds good, let's move on to the next item then. Thanks johnma! 18:34:16 #topic Example Snippet Generator 18:34:17 Thank you. Appreciate all the feedback 18:34:55 Hey there, grapex 18:35:10 Yo everyone. If anyone isn't familiar with the Trove API docs, they have these snippets showing what the content of the REST request and response is 18:35:20 For example, here's on on "configurations" 18:35:28 https://github.com/openstack/trove/blob/master/apidocs/src/samples/db-configuration-create-request-json.txt 18:35:28 https://github.com/openstack/trove/blob/master/apidocs/src/samples/db-configuration-create-request.json 18:35:44 https://github.com/openstack/trove/blob/master/apidocs/src/samples/db-configuration-create-response-json.txt 18:36:18 These end up going into the PDFs that get generated on how to use the Trove API 18:36:31 the problem is, they're generated statically and could be nothing but lies 18:36:47 though to be fair they probably are mostly true ;) 18:37:30 So this blueprint adds a test which captures the request and response body of the python-troveclient we all know and (hopefully) love 18:38:07 the tests then does a diff to see if the files in the output of this test is different from the files in source control, and if it is the test fails 18:38:08 meaning 18:38:23 if someone inadvertently changes the API in some minor way that makes a documentation snippet inaccurate, the test fails 18:38:42 they have to update the actual text snippet file to be accurate as part of their pull request to make Jenkins pass 18:38:55 Hopefully that will then trigger a discussion during the code review process. 18:38:59 grapex: this should definitely be helpful, nice work 18:39:13 So lets say, just to make up a fictional example, the configuration had some value in the request change from a number to string 18:39:15 ^^ would have found some issues we've had to back port... 18:39:23 the test would fail 18:39:33 and the person who saw this would either change the code, so the snippet would be the same 18:39:51 or, if that change was intentional, they'd change the snippet. Core reviewers would see it and then decide if the API change was OK or not 18:40:16 the end 18:40:22 cp16net: I was gonna say — wish we had this earlier. 18:40:26 grapex: Nice! This was the same thing that we discussed at the mid-cycle meetup, right? 18:40:28 :-P 18:40:56 SlickNik: Yep. And the same thing that was discussed during the Sprint midcycle in Austin- this is actually one big motivation for the Trove docs living in the Trove repo. 18:40:56 Just in more concrete form. 18:41:12 Hopefully it will also get us in the habit of writing docs as we add features 18:41:28 I've noticed in a lot of code reviews recently, there is push back if some minor nitpick doesn't match what the blueprint said 18:41:36 I might be starting a tangent here 18:41:39 but I don't agree with that 18:41:40 lol 18:41:57 write a blueprint about it ;) 18:41:59 The blueprint is to get people on board with the gist of the change, make sure people don't outright hate it and generally agree the idea is sound 18:42:19 The documentations is where we should actually... documentate what the API does 18:42:30 Merged openstack/trove-specs: Add RSS feed https://review.openstack.org/131253 18:42:31 I can't see users reading the specs repo for behavior 18:43:16 the nitpicking is a bit tangential, but i agree that this will definitely be a boon for users wanting to understand the req/resp model 18:43:36 vs. waiting for labeleled releases with new doc cuts 18:43:42 As work is done, the right thing to do might not be what the blueprint said. I'd like to endorse that as it encourages people to think about edge cases not present in the plain text description of what they thought they would be writing. 18:43:58 amcrn: Agreed 18:44:03 And the blueprint doesn't cover this- 18:44:37 but I've always thought it would be cool to somehow show an example of what the Python code involved to make things happen using the python-trovelcient alongside the generic REST information. 18:45:01 That's probably a bit futuristic at this point, plus it could involve really changing the docs 18:45:05 anyway... thoughts? 18:45:48 I agree with you guys completely on this. 18:46:11 The spec won't be able to cover 100% of the cases anyhow. 18:46:23 And having some latitude for design flexibility is a good thing. 18:46:28 amrith proposed openstack/trove: Rename generic variable named with mysql specific name https://review.openstack.org/129528 18:46:29 But let's try not to get too derailed with this topic in the interest of time for this meeting. :) 18:46:45 +1 to the doc changes. 18:47:05 Okay, any other questions for this? 18:47:07 +1 from my side as well 18:48:02 #topic Flavors per datastore 18:48:14 #link https://blueprints.launchpad.net/trove/+spec/associate-flavors-datastores 18:48:36 So this spec was previously approved for Juno. 18:48:52 But didn't make it in time after the Feature Freeze IIRC. 18:49:05 sgotliv and peterstac had some concerns about the api changes 18:49:35 thought we could discuss about it in the meeting and get some consensus 18:49:47 i've got some concerns too, but i'll yield the floor to sgotliv + peterstac first 18:50:24 neither of them appears to be here 18:50:30 amcrn, why don't you go ahead 18:50:34 I'll try and rustle them up 18:50:37 amcrn: yeah...go ahead :) 18:50:41 gotta page it back into memory first ;) 18:51:18 i haven't had a chance to peer into the code review, but is https://wiki.openstack.org/wiki/Trove/associate-flavors-datastores still up-to-date? if so, my concern is with the default-datastore behavior. 18:51:34 peterstac: your comment on this review might help - https://review.openstack.org/#/c/109824/ 18:51:43 amcrn: yes it is up to date 18:52:07 default_datastore was only introduced to avoid breaking backwards compatibility for those who had deployments that relied on mysql being the only datastore in existence, aka pre icehouse 18:52:19 amcrn: I think that was my concern - if there were no flavors defined, then it looked like it would fail 18:52:23 generally speaking, any new deployment should not set that value 18:52:25 peterstac's comments were - "I have to run the code to be sure, but from my review it looks like this becomes a required feature. Not sure that's what existing customers would want. If it turns out I'm mistaken - or the consensus is that that's what we want, I'll switch my vote (of course)." 18:53:10 peterstac, amcrn: if there are no flavors associated to the datastore version..the default flavors are listed 18:53:11 Unfortunately my env got upside down so I haven't had a chance to run the code yet 18:53:27 in short, i'd like to remove the default flavor listing route 18:53:57 it's an unnecessary shortcut for a cfgopt that shouldn't be used 18:53:57 Riddhi, your most recent comment appears to contradict your response in code review (patch set 9) "The expected behavior is to check if the flavor that is to be used for the instance, has a binding with the datastore_version.id being used for the instance. It throws an exception if not. The code tries to check the same." 18:55:27 okay..one at a time: amcrn: you do not want the list default flavors api call anymore you say? 18:55:48 pretty much. rip out GET /{tenant_id}/flavors and i'm ok with the proposal. 18:56:09 actually, that isn't a proposal, that route already exists... 18:56:34 Wouldn't that be altering the API though? 18:56:34 so if a deployer doesn't have default_datastore set, that behavior has to remain as it is today 18:56:40 amcrn: Wait, I'm confused. I thought you just wanted it so that if a datastore wasn't specified then it would show *all* flavors, rather than the default. Right? 18:56:41 sorry, what? amcrn are you suggesting deleting /{tenant_id}/flavors? 18:56:41 SlickNik: i'm backpedalling 18:56:51 you would need to version the API if you ripped out the default /flavors that currently exists 18:56:51 no the blueprint doesn't state whether the routes are new or changed 18:56:57 yes yes yes 18:57:05 hence the confusion at first 18:57:49 amrith: It's hard to get context on your last comment. Maybe we should just start over and people who are still not ok with this can state what they want? 18:57:53 Let's go with amcrn first 18:58:01 i rescind my comments, because we can't touch GET /{tenant_id}/flavors due to backwards compatibility, it doesn't matter how nonsensical it is 18:58:09 Ah got it — took me a moment to realize it was an existing route as well. 18:58:20 amrith, I'm all set (see amcrn's most recent comment) 18:58:50 Uh-oh, I think someone has hacked into amrith's account and just inadvertently referenced him in the third person! 18:59:09 I talk to myself all the time (in first person). it is boring. 18:59:13 Just to clarify is the existing /{tenant_id}/flavors remaining exactly the same as it is now? 18:59:20 amrith: Tim knows what you mean. 18:59:31 edmondk: I think so, right Riddhi? 18:59:33 amcrn, SlickNik: yes, it is an existing route..the only reason it was outlined in the bp was to maintain the backward compatibility...i think the desc for the /flavors api call is causing the confusion.. 18:59:34 edmondk: Yes, that's an existing API call 19:00:03 got it 19:00:04 Isn't the new one: GET /{tenant_id}/flavors/{datastore_version_id} ? 19:00:14 Riddhi: By pointing out you wouldn't change a feature that was so broadly despised, you actually made everyone think you were changing the API. :) 19:00:18 peterstac: yes..thats the new one 19:00:30 edmondk: I think it's a subtle change — previously you'd get all flavors, after this change you'd presumably get all flavors associated with the default datastore. 19:00:31 #idea ... take existing BP and make it an RST and let people comment on it. 19:00:59 SlickNik: yes thats right 19:00:59 SlickNik: yeah, that would be bad 19:01:15 the new call is /{tenant_id}/flavors/{datastore_version_id} 19:01:17 amrith: I'd normally say yes... but I'd really like to just finish getting to the bottom of these concerns 19:01:35 the trove-manage utility provides the ability to add/delete flavors 19:01:53 amrith: only due to the extreme age of this blueprint. 19:01:53 grapex, I'm proposing it because we are discussing here (in IRC) things that would be better (IMHO) to discuss in review. 19:01:55 associated with these ds versions 19:02:05 but I see your point about the age of this BP 19:02:12 the old flavors route/call still remains the same 19:02:12 it may not survive the replanting 19:02:43 amrith / grapex: I'd like to get an .rst in for spec bookkeeping for kilo though. 19:02:51 SlickNik: Argh 19:02:55 Even if it's just the current wiki page replanted to rst format. 19:02:58 SlickNik: That *is* a good reason though 19:03:13 Still 19:03:22 I feel like I don't understand people's concerns 19:03:26 grapex, I'll do the gardening if it will help ease the workload 19:03:41 Riddhi: make sure to make the id a string and not an integer 19:03:41 grapex, ask Tim about it ;) 19:04:10 amcrn: you mean the flavor id right? 19:04:12 * grapex wonders if he'll go to hell if he also asks Riddhi to write documentation snippets for this... 19:04:13 correct 19:04:31 grapex: will do! 19:04:47 otherwise the id_str hack will have to propigate to a newly introduced api, which would be sad 19:04:47 Riddhi: You are *really* bad at bargaining! 19:04:48 Thanks Riddhi! Really really appreciate it. 19:05:21 The two big concerns right now are from amcrn and amrith right? 19:05:21 Okay, we're done with meeting. We can stay back to close on rst / spec conversation as needed. 19:05:24 Thanks all! 19:05:28 #endmeeting