17:59:52 #startmeeting trove-bp-review 17:59:53 Meeting started Mon Jun 2 17:59:52 2014 UTC and is due to finish in 60 minutes. The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:59:54 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:59:56 The meeting name has been set to 'trove_bp_review' 18:00:24 o/ 18:00:37 o/ 18:00:40 o/ 18:00:47 o/ 18:00:47 o/ 18:00:48 o/ 18:01:02 o/ 18:01:18 o/ 18:01:38 \0/ 18:01:42 o/ 18:01:51 \o 18:02:04 o/ 18:02:16 #topic Trove should use Keystone Trusts for Authentication instead of hard coding the credentials in configuration files. 18:02:17 \o 18:02:53 Not sure who put this one in. Someone with the username dguitarbyte? 18:03:06 o/ 18:03:18 Anyone? 18:03:19 o/ 18:03:19 i like idea of Keystone V3 and Trusts 18:03:20 Blueprint isn't in the correct format 18:03:20 I believe that is Pranav Salunke 18:03:45 if format is wrong - we should skip it 18:04:09 seconded :) 18:04:14 Okay, I don't see him around either. Let's skip it, and I'll try to reach out to him offline. 18:04:22 sounds good 18:04:43 #topic Flavor per datastore association 18:04:51 Hey, so there was an earlier effort https://blueprints.launchpad.net/trove/+spec/service-type-filter-on-flavors 18:04:53 to introduce flavor management and connecting flavors and data stores. This effort got abandoned. I am proposing that we view this as two components: one is introducing flavor management via trove and two is association of flavors and datastore versions. This is the blueprint for the second part: https://blueprints.launchpad.net/trove/+spec/associate-flavors-datastores and is more detailed than the 18:04:55 initial proposal(hence separate bp). It adds additional api calls to display the datastore versions for a given flavor and management calls to associate data stores and flavors 18:05:32 iccha1, spec looks good, but it should be attached to initial BP 18:06:20 iccha1, we need to reach out Sushil Kumar and ask him to re-asign it to you 18:06:29 I do think they can be considered separate items.. i am not sure we need to reuse the BP 18:06:58 iccha1, what problem is this aiming to solve or what new capability does it intend to add? 18:07:03 sorry I'm a bit late 18:07:09 I like the idea of reusing code, but reusing wiki articles gets tricky 18:07:39 amrith: the problem is currently say we need a min flavor 2 for a myql version datastore there is no way to limit that 18:07:59 amrith, and at least flavor with 4Gb and more for Cassandra 18:08:28 so I submit to you that this is a bad idea 18:08:30 here is why 18:08:31 So the one thing about this that I want to bring up is that this proposes to change the GET call /{tenant_id}/flavors to return flavors for the default datastore type (which is different from today, and possibly changes the API contract) 18:08:31 grapex: denis_makogon the first bp was focussed more on adding flavors using trove, and did not have a good mechanism to associate flavors and datastore versions 18:08:42 while the recommendation(s) from vendors is for a produciton machine 18:08:44 vipul: +1 18:08:54 for a dev/test situation, using a micro is fine in many cases 18:09:06 for example, on amazon I use a t1.micro regularly for mysql 18:09:09 iccha1, correct, that's why i just put my thought onto existing BP 18:09:21 amrith: t1.micro may not be suitable for all datastores 18:09:24 making this a minimum would require one to use a larger machine than needed in some cases 18:09:33 amrith, don't think about mysql only 18:09:36 SlickNik: Maybe we could make that configurable 18:09:41 vipul, I was making the case for mysql and according to mysql's documentation t1.micro should not be used 18:09:42 amrith: this is configurable in ur database so ur deployment can allow any flavors 18:09:44 same with others 18:09:53 yeah my oracle enterpise won't run on 128mb 18:10:01 What the need is to add flavors that only show up for some datastores. Its backwards compatable so long as they are new flavors being hidden and not old ones 18:10:09 kevinconway, ++ 18:10:24 iccha1, if this is configurable, I withdraw my objection 18:10:31 SlickNik: yeah was kinda on the fence with that one 18:10:42 kevinconway: Still going on about that Oracle enterprise license! You're just impressed because they shipped it in an actual box with a full color instruction manual 18:10:47 yep.. amrith agreed.. I don't think we are aiming to limit to what vendors would consider suitable.. but we need to ensure that we can specify the possible flavors that would allow a certain datastore to even function 18:10:50 iccha1, can we throw warning to a user if he puts un-suitable flavor 18:10:53 ? 18:11:02 can we discuss Trove-Keystone-Trusts? 18:11:10 vipul: +1 18:11:22 denis_makogon: the instance create process will not start if we have incompatible flavor-datastore version, it ll throw excpetion 18:11:23 ALL: per iccha1's comment that this is configurable, I withdraw my objection. 18:11:37 denis_makogon: I think a warning isn't strong enough- the kind of situations we're talking about, we know the combinations won't work well or maybe even at all 18:11:45 amrith: objection overrulled 18:11:57 kevinconway: +1 18:11:58 i need to turn spell check back on 18:12:01 grapex, iccha1, ok, i'm fine with that 18:12:03 dguitarbite: We discuss that once we're done with other bps. (If there's no time left, let's talk about it offline after the meeting) 18:12:13 SlickNik: ok 18:13:14 so are we all set with this one? time to vote? 18:13:25 i've got question about DB scheme 18:13:35 iccha1 / grapex: perhaps we can return all flavors with an added "datastores" field or something similar (and you could filter based on the query param? 18:13:35 will the flavors for datastore be configurable? For instance, if i have very fast machines or very efficient GAs, the flavor requirement will be different 18:13:46 iccha1, why can't we add flavors table to Datastore table ? 18:13:57 *flavor column 18:14:02 SlickNik: yeah that could be done 18:14:26 denis_makogon: if we want a bunch of flavors, it ll have to be stored as a list(string) and then extracted and processed 18:15:00 iccha1, i guess, it's fine 18:15:09 iccha1, any complains with that ? 18:15:33 #startvote Flavor per datastore association, yes, no, abstain 18:15:34 Unable to parse vote topic and options. 18:15:38 So iccha1, SlickNik- are we settling on having the current list call show everything, while using query params to limit them to specific datastores? 18:15:46 iccha1, also, one question, can we just saw "Hey, you can you everything that grater than this flavor" ? 18:15:47 the ability to have have each flavor as a separate row then we cn do joins 18:16:13 #vote yes 18:16:19 irc://15.185.114.44:5000/#startvote Flavor per datastore association? yes, no, abstain 18:16:20 #vote yes 18:16:26 #vote yes 18:16:35 lol 18:16:51 #vote yes 18:16:54 I messed up the format, didn't I? 18:16:57 yup 18:17:06 #yes 18:17:08 It doesn't matter. We're in agreement I think. 18:17:08 3rd times a charm 18:17:10 #yolo 18:17:23 kevinconway, swag 18:17:45 #vote yes 18:17:46 Okay approved. 18:17:48 denis_makogon: i am happy to answer any more questions after the meeting 18:17:56 iccha1, thanks 18:18:10 #yes 18:18:12 cool 18:18:13 also #late 18:18:33 #vote yes 18:18:57 #topic Conductor phase 2 18:19:00 #link https://blueprints.launchpad.net/trove/+spec/conductor-phase-2 18:19:22 i've spoke with konetzed and Ed Cranford (initial proposers) 18:19:28 SlickNik: i think you need to end the vote before you switch topics 18:19:57 SlickNik: 's the boss he approves 18:20:07 iccha1 ... +1 18:20:10 no, i mean before meetbot will work 18:20:12 kevinconway: The startvote earlier didn't take, cause I messed up the formatting. :( 18:20:25 #endvote 18:20:29 its done 18:20:32 oh wait. true 18:20:43 shouldn't meetbot change the channel topic? 18:20:46 the whole idea is to make conductor the only one entry point 18:20:51 oh wait… not in trove room. sorry 18:20:56 i dont think it does in this 18:20:57 yea.. 18:21:11 to database among all possible trove services 18:21:29 phase 2 will cover taskmanager 18:21:43 i assume this spec pre-dated the template which is why it isn't in the correct format? 18:21:52 18:13 < Sackmann> ekonetzk: had to pull the power. they were in a system halt 18:21:59 lol 18:22:07 dougshelley66, https://wiki.openstack.org/wiki/Trove/Conductor-phase2 18:22:11 oh the pleasure and pain of the middle clickin linux 18:22:19 dougshelley66, it's in BP body 18:22:37 dougshelley66, since spec link leads to conductor feature description 18:22:58 ok i clicked on "read the full spec" and saw the original one, i guess 18:23:14 #link https://wiki.openstack.org/wiki/Trove/guest_agent_communication#Phase_2._Let_taskmanager_speak_with_Trove_backend_through_conductor 18:23:17 So my one issue with this is I don't think we should *requre* to use RPC to talk to conductor. 18:23:24 denis_makogon: Perhaps this is missing a few details: https://wiki.openstack.org/wiki/Trove/Conductor-phase2 18:23:24 so denis_makogon this one is to get the data from the conductor for heartbeats eh? 18:23:28 As is Trove gets the guest status from the database which is pretty fast 18:23:41 grapex: what do u 18:13 < Sackmann> ekonetzk: had to pull the power. they were in a system halt 18:23:44 I think it should just talk to a "conductor API" style class, similar to the guest API 18:23:44 18:13 < Sackmann> ekonetzk: had to pull the power. they were in a system halt 18:23:55 ha 18:24:02 hub_cap: i think 18:13 is the past 18:24:25 i think we need to change the hub_cap bots battery 18:24:27 :P 18:24:52 18:25 < kevinconway> hub_cap: can you clipboard something less embarrassing? 18:25:02 nice 18:25:09 grapex: So the idea would be that the API style class could use RPC or query the DB directly based on a config? 18:25:20 in Phase 2 conductor will receive requests from taskmanager for backed read/write operations 18:25:31 I seem to recall that the point of having the taskmanager etc access db through conductor was to reduce db access. Is this part of this proposal at hand now? 18:25:44 hub_cap: So how's that Chrome book working out for you again? 18:25:50 so, with respect to https://wiki.openstack.org/wiki/Trove/Conductor-phase2 18:25:56 cp16net: I got your feedback, thanks. I'm going through my onboarding with my new company but I'll get on that fix asap 18:25:57 I think this BP isn't ready for review yet 18:26:04 SlickNik: Yes, exactly 18:26:06 the thought behind it is that we can make conductor more of an in memory datastore 18:26:18 grapex: its not a chromebook ;) 18:26:21 hub_cap: +1 where does that fit into these phases 18:26:23 amrith, why? 18:26:41 hub_cap: Right. So I think this needs to be detailed in the spec, no? 18:26:41 vipul: i think its phase two in a way 18:26:47 vgnbkr: for sure 18:26:49 the spec is 1 line 18:26:53 Let taskamanager speak with backend through conductor (Not implemented yet). 18:26:56 ... 18:27:03 unless i missed a link somewhere 18:27:09 amrith: I agree. I see a lot of let's do phase 2, but nothing about what phase 2 entails or is going to solve. 18:27:12 hub_cap, everything is correct 18:27:13 no, you have the right link hub_cap 18:27:23 So I definitely see a lack of details in the spec. 18:27:28 SlickNik: I think like hub_cap said, its just to change the datastore for all the heartbeats and stuff 18:27:43 SlickN1k, I'm particularly curious about things that say "RPC API description will be defined during development" 18:27:48 if the rest of the Trove code doesn't have innate knowledge of how to get the guest status via the database, but just asks a Conductor API for it 18:27:48 that seems to defy the purpose of a bp 18:27:55 Phase 2 will cover communitcation between Trove backend and taskmanager 18:27:57 we could change the status to live in an in-memory database instead 18:28:12 grapex: trove could deploy it! 18:28:20 Now, I personally don't like the idea of having even more RPC calls to conductor from taskmanager if we can avoid it 18:28:28 why don't we just get rid of taskmanager 18:28:31 i thought phase 3 was the in memory datastore 18:28:38 just use conductor 18:28:45 RPC calls aren't like limitless solar energy or something 18:28:50 cp16net, phase 3 is "Let API service speak with backend through conductor (Not implemented yet). " 18:29:07 grapex, that's why conductor should be clusterable 18:29:18 I sort of don't understand why phase 2 and 3 are even different phases 18:29:22 Ok, can we table this one? and have denis_makogon go add more content? 18:29:31 SlickN1k, I move to table this. 18:29:36 +1 that will give more clairty 18:29:37 Phase 2- let one piece of Trove code stop talking to the database and talk to a conductor API. Phase 3- do the same thing with another bit of code 18:29:38 vipul +1 18:29:47 Okay, I think we're going off in different tangents because phase 2 means different things to different people, and the spec doesn't clearly define what it means by phase 2 18:29:49 i like how we don't finalize message protocols until phase 5 18:30:07 it means love and bunnies and garlic to me SlickNik 18:30:27 #startvote Conductor phase 2? yes, no, needs_details 18:30:28 Begin voting on: Conductor phase 2? Valid vote options are yes, no, needs_details. 18:30:29 Vote using '#vote OPTION'. Only your last vote counts. 18:30:39 #vote needs_details 18:30:40 #vote needs_details 18:30:40 #vote needs_details 18:30:41 #vote needs_details 18:30:45 #vote needs_details 18:30:49 #vote needs_details 18:30:49 #vote needs_details 18:30:50 all in all i like it, fwiw 18:30:51 #vote needs_details 18:30:54 #vote needs_details 18:30:55 #vote needs_details 18:30:58 same here. 18:31:00 #vote needs_details 18:31:02 hub_cap: + 18:31:07 its just a bit scary to not know wtf is goin on w/ it 18:31:16 #endvote 18:31:17 Voted on "Conductor phase 2?" Results are 18:31:18 needs_details (11): SlickNik, robertmyers, amrith, vgnbkr, peterstac, cp16net, iccha1, vipul, dougshelley66, grapex, hub_cap 18:31:24 i get it 18:31:27 thanks 18:31:32 i'll update spec 18:31:34 thanks denis_makogon 18:31:35 thx 18:31:43 #topic Add created/updated timestamps and instance count to configuration groups list and details calls 18:31:54 #link https://blueprints.launchpad.net/trove/+spec/minor-config-edits 18:32:06 tvoran is off today, so i am proxying 18:32:13 i've got question 18:32:53 since we can assign one configuration to one instance, can we change "count" field to "assigned: True/False" ? 18:33:17 denis_makogon: is instance count is 0 it is unassigned 18:33:34 denis_makogon a config group can be assigned to N instances 18:33:43 yea id like to see how many i was rebooting if i changed a config 18:33:48 an instance can only have 1 config group 18:33:50 denis_makogon: I believe you can assign a group to any number of instances. 18:33:51 if i had a config N=10000000, and i was updtating it 18:33:56 id think twice 18:34:03 yup exactly 18:34:09 i wouldnt just "joyent" it 18:34:14 i had the count in the original impl i made 18:34:21 i removed it because it wasnt part of the "spec" 18:34:22 lol 18:34:28 too soon? 18:34:39 hub_cap: lol 18:34:41 hub_cap: lol 18:34:47 ok, thanks for explanation 18:35:19 I think it makes sense. 18:35:29 agreed 18:35:31 Is there an easy way to get the actual list of instances that a config group is attached to? 18:35:41 can the database changes be explicitly stated in the spec? 18:35:50 i think the other reason i removed this was because it was expensive to make the count call 18:36:06 using a join with sqlalchemy SUCKED 18:36:37 * hub_cap queues the rainman "bout a thousand dollars" 18:36:39 maybe thw get on a specific configuration could have it, if we are opposed to having to the index call 18:37:04 cp16net: I recall perhaps there was a way with the config groups API; but just wanted to confirm. 18:37:05 i think cp16net 's original proposal had instance ids as well? 18:37:11 yeah its going to be more expensive on the index 18:37:38 so we could keep created and updated on index and move instance count to get alone? 18:37:41 SlickNik: yeah there is /config//instance 18:37:43 do we want to design the solution here or merely surface the requirement? 18:37:51 cp16net: Thanks! 18:37:52 we could make a table that stores the count results and update it every time we create an instance 18:37:54 np 18:38:08 kevinconway: and every time we delete 18:38:29 kevinconway: i'll just send an email to you to update my database for me 18:38:36 kevinconway: iccha1 if we stored all this in redis we could just use counters 18:38:41 SCREW JOINS 18:38:53 amrith: Just surface the requirement; we don't need to design it here. 18:39:01 we should load all the records into python and then count the items with len() 18:39:05 SlickN1k, ++ 18:39:08 kevinconway: ++ 18:39:19 Why reinvent the wheel amirite? 18:39:20 kevinconway: so yer saying an iin memory python hash database? 18:39:22 ++ 18:39:36 I think we're all agreed, but for the record 18:39:40 can we make it eventually consistent too? 18:39:43 hub_cap: We could build it off the test doubles we already have in fake mode 18:39:45 its all the rage 18:39:48 #startvote Add created/updated timestamps and instance count to configuration groups list and details calls? yes, no 18:39:49 grapex: brilliant 18:39:50 Begin voting on: Add created/updated timestamps and instance count to configuration groups list and details calls? Valid vote options are yes, no. 18:39:51 Vote using '#vote OPTION'. Only your last vote counts. 18:40:02 vote #whatever SlickNik votes 18:40:03 #vote yes 18:40:04 #vote yes 18:40:05 #vote yes 18:40:05 #vote yes 18:40:07 #vote yes 18:40:10 #vote yes 18:40:16 #vote yes 18:40:16 Main Tera Hero 18:40:27 #vote yes 18:40:31 vipul ... I know, I've heard that a lot 18:40:32 huh vipul ? 18:40:32 copy paste gone bad lol 18:40:44 #endvote 18:40:44 vipul: whose hero? ;) 18:40:45 Voted on "Add created/updated timestamps and instance count to configuration groups list and details calls?" Results are 18:40:46 yes (8): SlickNik, robertmyers, amrith, cp16net, iccha1, vipul, grapex, hub_cap 18:40:47 a movie i downloaded for my wife ;) 18:40:47 there are lots of abs on that poster 18:41:13 does it have salman khan? 18:41:13 #topic Pluggable conductor manager 18:41:16 vipul: what u mean is a file you illegally downloaded and violated copyright so u could make yer wife watch it with you? 18:41:16 lol 18:41:22 hub_cap: like import math. math.abs? 18:41:30 #link https://blueprints.launchpad.net/trove/+spec/pluggable-conductor-manager 18:41:40 hub_cap: there are no copyright laws in indian movies ;) 18:41:56 boden: around? 18:42:06 SlickNik and others -- I clearly need to create a spec which follows the structure used.. I can do that soon 18:42:08 yes 18:42:11 vipul: sweet /me downloads a bunch of movies 18:42:46 so pluggable conductor manager simply allows the conductor manager class to be exposed in the conf vs being hard-coded in the conductor cmd 18:43:01 wtf its not already? 18:43:02 fail... 18:43:10 all our managers should be conf'able 18:43:17 I checked; it isn't already 18:43:25 boden: hrm.. pluggable managers… external extensions… almost like you have some business use case for openstack 18:43:28 #vote why not 18:43:35 I feel like this is so simple maybe I'm missing something 18:43:37 https://github.com/openstack/trove/blob/master/trove/cmd/conductor.py#L23 18:43:38 #vote sure 18:43:40 fail... 18:43:52 grapex: I think it is simple. 18:43:53 #vote how the frack did we miss this 18:44:05 #doh 18:44:12 #vote Let's do it. 18:44:14 Homer: If you really want something in life you have to work for it. Now quiet, they're about to announce the lottery numbers. 18:44:18 kevinconway -- I think this will make it easier for vendors to deliver custom solutions they choose to not upstream and also easier to perform PoCs and prototypes without upstreaming 1st 18:44:38 boden: dont listen to kevinconway 18:44:38 ever 18:44:40 srsly 18:44:41 ever 18:44:42 boden, I think all of us are in agreement that this was just something that we missed. 18:44:44 EVER 18:44:57 hub_cap 10-4 ;) 18:45:07 #startvote Pluggable conductor manager? yes, no 18:45:08 Begin voting on: Pluggable conductor manager? Valid vote options are yes, no. 18:45:09 Vote using '#vote OPTION'. Only your last vote counts. 18:45:10 #vote yes 18:45:13 #vote yes 18:45:13 #vote yes 18:45:14 #vote yes 18:45:15 #vote yes 18:45:18 #vote yes 18:45:18 #vote si 18:45:19 hub_cap: si is not a valid option. Valid options are yes, no. 18:45:21 #yes 18:45:24 i hate u openstack 18:45:27 #vote yes 18:45:30 #vote yes 18:45:39 iccha1, #vote yes 18:45:51 #endvote 18:45:52 Voted on "Pluggable conductor manager?" Results are 18:45:53 yes (8): SlickNik, boden, robertmyers, amrith, cp16net, vipul, grapex, hub_cap 18:46:36 dguitarbite: around? 18:47:01 #topic Configurable DB plugins 18:47:26 SlickNik -- so it appears the cmd/*.py classes (entry points) have changed since I wrote this... 18:48:08 but in essence there is already plumbing in place to support consumers / extenders to add their own tables to trove db... however its not exposed at the entry point level for some reason 18:48:59 yes 18:48:59 boden: Why would customers want to add their own tables to the trove schema? (Just trying to understand the scenario) 18:49:00 im here 18:49:30 dguitarbite: will discuss the keystone trusts after this bp (if there is still time). 18:49:48 ok 18:49:49 nps 18:49:49 SlickNik - adding custom function to trove which requires new tables... for example in-house / proprietary extensions 18:49:55 how would migrations work for this? 18:50:12 so boden if you add your own migration it could conflict with a new public one 18:50:12 grapex -- trove-manage already supports it 18:50:31 you need a whole new set of migrations 18:50:39 and a new table to manage them 18:50:46 grapex via the --repo_path option 18:51:25 yea... how do the other projects do it? 18:51:27 guys -- sounds like I should take some time to outline this one in more detail? thoughts? 18:51:30 Are there other OpenStack projects that do this? 18:51:37 boden: plz do 18:51:42 boden: Yes, I'd like to take some time to look into it as well. 18:51:43 this seems VERY nonstandard 18:51:45 cp16net and others -- I will do that 18:51:56 aka, if u have extra stuff on top, run it in a postinst 18:51:59 hub_cap -- I can check; as I said the plubming is already there 18:52:00 for your packages 18:52:16 i've seen some weird things in nova that have 5 placeholder migrate scripts before and after the migrate scripts that are created 18:52:22 not sure why tho 18:52:27 cp16net: they do that cuz of gerrit conflicts 18:52:34 its a Queue for the merges so to speak 18:52:41 boden: Thanks. 18:52:44 oh 18:52:44 np 18:52:46 if i understand correctly 18:52:54 * hub_cap is usually not correct 18:53:18 #topic Trove should use Keystone Trusts for Authentication instead of hard coding the credentials in configuration files. 18:53:21 that makes a little more sense tho 18:53:35 #link https://blueprints.launchpad.net/trove/+spec/trove-keystone-trusts 18:54:17 is keystone trusts ready to use? 18:54:27 thought it was experimental 18:54:31 I believe so 18:54:42 can we wait 18:54:44 I'm unsure of this one. 18:54:44 its v3 18:54:48 why rush 18:54:51