18:02:01 <SlickNik> #startmeeting trove_bp_review
18:02:03 <openstack> Meeting started Mon Apr 28 18:02:01 2014 UTC and is due to finish in 60 minutes.  The chair is SlickNik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:02:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:02:06 <openstack> The meeting name has been set to 'trove_bp_review'
18:02:13 <SlickNik> Sorry had an extra '#' in there.
18:02:31 <amrith> \o/
18:02:34 <denis_makogon> щ.
18:02:37 <denis_makogon> o/
18:02:40 <juice> o/
18:02:40 <amcrn> o/
18:02:43 <vipul> o/
18:02:51 <cp16net> |o| gooooooool
18:02:59 <glucas> cp16net, lol
18:03:03 <amrith> ;)
18:03:06 <yogeshmehra> o/
18:03:10 <SlickNik> #link https://wiki.openstack.org/wiki/Meetings/TroveBPMeeting
18:03:11 <SnowDust> ;)
18:03:30 <dougshelley66> o/
18:03:56 <SlickNik> #topic Percona support
18:04:00 <esp> o/
18:04:01 <vgnbkr> \o
18:04:06 <SlickNik> mattgriffin: Around?
18:04:39 <amrith> seems to be online
18:04:58 <denis_makogon> hub_cap, looks like your proposal for design session will be very useful
18:05:12 <denis_makogon> single implementation to rule them all
18:05:14 <SlickNik> Let's give him a minute, if not we can table the discussion it for later.
18:05:25 <denis_makogon> agreed
18:05:33 <juice> looks like mattgriffin's proposal here is to upgrade the percona components
18:05:55 <denis_makogon> juice, it seems like yes
18:05:58 <mattgriffin> SlickNik, hey... sorry. OTP... doh meeting!
18:06:00 <juice> appears that it will benefit the backup since there is direct integration with swift/s3 yes?
18:06:07 <mattgriffin> can we talk about my BPs at the end of the meeting?
18:06:14 <SlickNik> juice: Yes, and I had some questions around backward compat.
18:06:21 <juice> very risky mattgriffin :)
18:06:36 <amrith> move on?
18:06:38 <SlickNik> mattgriffin: okay. But no guarantees the other discussions won't be a black hole.
18:06:38 <denis_makogon> mattgriffin, go-go-go with your BPs
18:06:39 <SlickNik> Yup.
18:06:57 <SlickNik> Let's come back to this.
18:07:12 <SlickNik> #topic Percona support
18:07:23 <SlickNik> oops
18:07:23 <amrith> vertica support, maybe?
18:07:28 <cp16net> lolz
18:07:33 <SlickNik> #topic Support for Vertica
18:07:34 <_shal_kh> Yeah
18:07:34 <denis_makogon> amrith, yup
18:07:39 <yogeshmehra> ok
18:07:45 <SlickNik> yogeshmehra: go for it
18:07:49 <yogeshmehra> yup
18:08:06 <yogeshmehra> This is for adding vertica as a datastore in trove
18:08:19 <denis_makogon> what's the justification for it ?
18:08:40 <denis_makogon> simple googling: Vertica has 600 customers and has been around since 05...
18:08:44 <yogeshmehra> it offers the devs another db to work within trove
18:08:48 <juice> denis_makagon: choices
18:08:57 <vipul> what's the justification for mongo/cassandra/anything
18:09:13 <hub_cap> denis_makogon: :) sry was afk
18:09:14 <amrith> denis_makogon, huh? i don't understand the point you seem to be making
18:09:16 <denis_makogon> vipul, customer envolvement
18:09:26 <grapex> What's the emoticon for a stick figure rushing in late while raising his hand?
18:09:30 <juice> denis_makagon: to be clear this is a product that HP owns and has a vested interest in offering in dbaas
18:09:37 <SnowDust> justification :- it's a big data DB
18:09:47 <yogeshmehra> well..
18:09:49 <hub_cap> is there a free version for testing?
18:09:53 <hub_cap> thats my biggest gripe
18:09:54 <yogeshmehra> there are numerous justifications
18:09:54 <amrith> yes
18:09:57 <denis_makogon> hub_cap, ++
18:09:58 <yogeshmehra> :-)
18:10:01 <hub_cap> if there is no version we can use for validation / verification
18:10:04 <vipul> https://my.vertica.com/community/
18:10:05 <amrith> hub_cap, yes
18:10:08 <SlickNik> So, I'm fine with this given: a. The trove API makes sense for vertica (i.e. including the backup / restore parts), and b. We will have some way of testing this.
18:10:13 <hub_cap> then im a-ok w/ it
18:10:14 <yogeshmehra> the best one is to offer the trove users another option..
18:10:16 <hub_cap> ++ by me
18:10:22 <hub_cap> lets add it in
18:10:29 <grapex> Also it adds flexibility to Trove
18:10:34 <juice> yogeshmehra: does vertica need to be installed as a cluster?
18:10:35 <amrith> so, a question for core
18:10:41 <yogeshmehra> nopes
18:10:47 <amrith> as we add new data stores
18:10:52 <yogeshmehra> the cluster part comes with teh trove cluster support
18:10:53 <amrith> some of them are going to need special options
18:10:56 <cp16net> vipul: i dont want to register tho...
18:10:59 <amrith> in the case of vertica, cluster is one of them
18:11:01 <yogeshmehra> it moves with trove
18:11:15 <amrith> does it make sense to have a notion of "core" datastores and "beta" datastores
18:11:23 <vipul> cp16net: yogeshmehra: there is a apt repo somewhere that doesn't require registration?
18:11:33 <denis_makogon> amrith, we can't, all datastores are equal
18:11:48 <yogeshmehra> vipul:the community edition is free and is available from vertica
18:11:48 <grapex> amrith: I think we shouldn't restrict what datastores run, but we should think before putting in features that no one can easily try or test
18:11:51 <denis_makogon> amrith, we cannot promote or have favorite
18:12:08 <hub_cap> denis_makogon: yes we can
18:12:09 <denis_makogon> grapex, agreed
18:12:09 <vipul> without registering yogeshmehra?
18:12:14 <amrith> because all code won't be equally stable and mature
18:12:15 <hub_cap> if 99% of us work on mysql, mysql is fav
18:12:31 <denis_makogon> hub_cap, its from customer perspective
18:12:35 <SnowDust> denis_makogon : you answered the best justification
18:12:36 <hub_cap> if hp wants to spend a billion dollars and 200 developers on trove+veritca
18:12:40 <yogeshmehra> one time registration is required but the CE can be taken and dropped to apt
18:12:48 <juice> amrith: this sounds like a documentation issue
18:12:48 <hub_cap> then trove+vertica will be the best supported :)
18:13:00 <amrith> juice, not true
18:13:11 <hub_cap> amrith: thats how other projects do it
18:13:21 <amrith> what will adding new datastores do to testing time?
18:13:21 <hub_cap> i ++ juices idea, thats the whole point of the support matrix
18:13:33 <juice> amrith: on the product page we can just state the level of testing/stability
18:13:39 <cp16net> hmm in the vertica docs it doesnt say anything about an apt-repo to install it
18:13:42 <hub_cap> heh amrith ask nova if they test anything besides libvirt ;)
18:13:44 <amrith> it is *also* a documentation issue
18:13:50 <SlickNik> amrith: hub_cap correct me if I'm wrong here: Right now our stance is that the other datastores (other than mysql) are 'experimental'.
18:13:54 <hub_cap> and ask cinder how many backends they test
18:14:00 <grapex> hub_cap: ++
18:14:11 <denis_makogon> hub_cap, heh, nova is a good example =)
18:14:13 <amrith> SlickNik, if that's the case, you have (already) addressed my issue
18:14:17 <juice> we do however need to start working on running tests against the various data stores..I think we have a private one that runs for percona
18:14:27 <hub_cap> SlickNik: that was the case, but im not sure we can say tha tanynmore
18:14:28 <amrith> I suggest we document and move on as juice says
18:14:30 <SnowDust> :) LOL on testing consensus
18:14:33 <hub_cap> its more of they have different levels of implementation
18:14:38 <hub_cap> plz see the support matrix hehe
18:14:43 <amrith> raise glass of juice to toast: +1
18:15:03 <juice> amrith: hopefully not prune juice
18:15:12 <hub_cap> juice: i think thats the general openstack consensus, private tests for other impls
18:15:17 <juice> any other questions for yogeshmehra :)
18:15:18 <denis_makogon> #link https://wiki.openstack.org/wiki/Trove/DatastoreCompatibilityMatrix
18:15:22 <amrith> juice, I agree
18:15:24 <SlickNik> I think we need to discuss this at the Wednesday's meeting.
18:15:24 <hub_cap> but we might need to confirm w/ some peoples at the OS summit
18:15:32 <SnowDust> juice it's most general canberry
18:15:40 <SlickNik> I'll put an agenda item in.
18:15:46 <vgnbkr> Is vertica impl just for testing, or intended for production?
18:15:49 <vipul> the only thing i'd say is.. make the binary available for folks running Trove and we're good.  If someone decides to run Trove with Vertica on their laptop.. the image should build.. it should just work for them
18:15:50 <SlickNik> That'll give us some time to think about it as well.
18:15:52 <amrith> so, back to the BP
18:15:57 <hub_cap> SlickNik: ++ inv mordred and sdague and co plz
18:15:59 <amrith> are we all ok with the BP?
18:16:05 <SlickNik> hub_cap: Will do.
18:16:13 <yogeshmehra> vgnbkr: for productiona s well..
18:16:23 <hub_cap> ++ on the bp
18:16:24 <denis_makogon> there's example of Sahara project
18:16:28 <yogeshmehra> but it is more for devs to taste and enjoy
18:16:31 <yogeshmehra> first
18:16:38 <SlickNik> #startvote Add vertica implementation? yes, no
18:16:38 <vgnbkr> Does it make sense to use vertica on a single instance, or did you have something else in mind?
18:16:38 <openstack> Begin voting on: Add vertica implementation? Valid vote options are yes, no.
18:16:39 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
18:16:50 <amrith> #vote yes
18:16:52 <vipul> yes
18:16:55 <vipul> boo
18:16:58 <vipul> #vote yes
18:16:58 <denis_makogon> Mirantis has putted hbase/hadoop packages into the our out repo and it's 100% available
18:17:01 <SlickNik> #vote yes
18:17:06 <denis_makogon> #vote yes
18:17:07 <SnowDust> #vote yes
18:17:09 <grapex> #vote yes
18:17:13 <juice> #vote yes
18:17:26 <cp16net> #vote yes
18:17:27 <amrith> final tally, 8 yes, 1 boo
18:17:36 <SlickNik> Okay enough data points.
18:17:40 <yogeshmehra> cool
18:17:41 <SlickNik> #endvote
18:17:42 <openstack> Voted on "Add vertica implementation?" Results are
18:17:43 <openstack> yes (8): SlickNik, amrith, denis_makogon, grapex, juice, cp16net, vipul, SnowDust
18:17:50 <yogeshmehra> vipul: yes + boo  :-)
18:17:54 <vipul> lol
18:17:56 <juice> denis_makogon: what is the take away/point you just made?
18:17:59 <SlickNik> Let's move on.
18:18:06 <denis_makogon> juice, after meeting
18:18:07 <juice> ok let's move on
18:18:23 <yogeshmehra> thanks guys
18:18:29 <SlickNik> #topic Resource management driver
18:18:36 <denis_makogon> it's mine
18:18:40 <denis_makogon> At current state Heat support in Trove is nothing else than experimental. Trove should be able to fully support Heat as resource management driver.
18:18:54 <denis_makogon> Why is it so important? Because Trove should not do what it does now (cloud service orchestration is not the part of the OS Database Program). Trove should delegate all tasks to Cloud Orchestration Service (Heat).
18:19:32 <denis_makogon> Why is it needed? The first answer is: To split-out two completely different resource management engines. Nova/Cinder/Neutron engine etc. called �NATIVES� and Heat engine called �ORCHESTRATOR�. As you can all know they cannot work together, because they are acting with resources in their own manners. But both engines are sharing more than enough common code inside the Trove.
18:19:42 <hub_cap> are u just copy/pasting from the bp denis_makogon ?
18:19:43 <vipul> so are we talking about refactoring the "use_heat" code path.. to be plugin driven?
18:19:52 <denis_makogon> hub_cap, i got some notes
18:19:58 <denis_makogon> vipul, yes
18:20:01 <hub_cap> ok those notes should be on bp :)
18:20:08 <hub_cap> and there is no need for a plugin
18:20:11 <hub_cap> we should use heat and heat only
18:20:14 <denis_makogon> hub_cap, it's on the 2 ML's
18:20:16 <hub_cap> and deprecate the old way
18:20:24 <hub_cap> and then remove the old way
18:20:28 <hub_cap> just like weve talked about since day one
18:20:31 <denis_makogon> hub_cap, heat is not 100% ready
18:20:36 <hub_cap> then make it 100% ready
18:20:37 <grapex> denis_makogon: ++
18:20:43 <hub_cap> im not saying we do i tnow
18:20:47 <yogeshmehra> hub_cap: ++
18:20:51 <hub_cap> im saying instead of a silly plugin approach
18:20:52 <hub_cap> fix heat
18:20:53 <denis_makogon> it doesn't support volume resizes
18:20:54 <hub_cap> deprecate old
18:21:04 <hub_cap> then add that to heat
18:21:13 <yogeshmehra> making heat a resident trovian...
18:21:16 <grapex> Well here's the thing- I think we're already at the point where we need an interface in Task Manager we can switch out for different strategies
18:21:18 <hub_cap> how would a plugin model help that?
18:21:29 <denis_makogon> So, there are three valid options:
18:21:35 <denis_makogon> use NATIVES if there's no available Heat;
18:21:35 <grapex> We can say "don't waste time on that, fix heat" but fixing heat is a much larger effort than refactoring our code to read better.
18:21:39 <SlickNik> Do we need a plugin approach if the goal is for heat to be the only way of doing things?
18:21:39 <denis_makogon> use ORCHESTRATOR to work with Heat only;
18:21:53 <hub_cap> grapex: no one is touching that code :)
18:22:04 <grapex> Also... I'd prefer we not use "plugins" but just a simple strategy pattern as we do in dozens of other places in Trove
18:22:05 <denis_makogon> SlickNik, of course we need
18:22:14 <hub_cap> if there are defeciencies in heat, we should fix them
18:22:15 <denis_makogon> current code looks strange and messy
18:22:22 <SlickNik> denis_makogon: Don't just say of course, please list the reasons.
18:22:23 <hub_cap> rather than do 70% heat, 30% trove homegrown
18:22:42 <hub_cap> just like the other projects, we need to use heat or dont use it
18:22:48 <yogeshmehra> denis_makogon: why not enlist the improvements required in the heat integration and work them out
18:22:54 <hub_cap> we cant use heat for some thinkgs, then use non heat, and make heat out of sync
18:23:02 <denis_makogon> yogeshmehra, it's already done
18:23:05 <hub_cap> if w edont have volume resize in heat
18:23:06 <hub_cap> and we resize
18:23:11 <hub_cap> then our heat volume size is wrong
18:23:20 <yogeshmehra> denis_makogon: then second step, to fix them :-)
18:23:21 <hub_cap> so when u call get info what do we use for the volume?
18:23:31 <SlickNik> So the takeaway I'm getting from this is that we need to work with devs on the heat team to fix heat to make sure it can support all our use cases.
18:23:31 <hub_cap> itll get _more_ confusing
18:23:35 <amcrn> as hub_cap said, the long-term goal is to have heat as the only option, but since the existing path will continue to exist for awhile (as deprecated), there's no reason to add yet another path to workaround heat "deficiencies" considering the deprecated path will still exist.
18:23:35 <grapex> hub_cap: I agree with what you're saying long term
18:23:37 <hub_cap> yes SlickNik
18:23:52 <grapex> but I think denis_makogon's want here is to refactor some of the code-
18:23:54 <hub_cap> grapex: and short term, no one use heat till the long term is done :)
18:23:59 <denis_makogon> all i say, resource provisioning should be done through heat
18:24:09 <hub_cap> id rather see us 1) fix heat, 2) refactor our code
18:24:11 <denis_makogon> grapex, that's the first step
18:24:12 <grapex> I guess I'm with him on this- though maybe we'll need to talk more on the particulars of how it looks.
18:24:20 <hub_cap> than 1) refacotr our code, 2) fix heat 3) refactor _again_
18:24:29 <denis_makogon> grapex, in parallel implement required use cases in heat
18:24:32 <amcrn> +1 hub_cap
18:24:37 <hub_cap> 1) dont touch anything 2) fix heat 3) refactor once
18:24:37 <denis_makogon> 3d step - re-use it in trive
18:24:38 <SlickNik> grapex: I think the question is prioritization; i.e. what do we do first.
18:24:44 <grapex> This is why we can't have nice things. :)
18:24:48 <denis_makogon> hub_cap, no refactoring after
18:24:49 <vipul> amcrn: +1 i don't really care to refactor our code that we're goign to deprecate, but i also don't see getting out of the two code paths anytime soon
18:24:50 <hub_cap> lol grapex
18:24:59 <hub_cap> denis_makogon: thats bs there will def be refactoring after
18:25:04 <hub_cap> once u add volume resize
18:25:06 <denis_makogon> hub_cap, no
18:25:09 <hub_cap> u will need to refactor the code to use it
18:25:11 <hub_cap> boom, refactor
18:25:21 <denis_makogon> hub_cap, you would need just to write new method - nothing else
18:25:22 <hub_cap> once u add proper flavor resize
18:25:23 <hub_cap> same
18:25:27 <SnowDust> hub_cap  +1 on approach
18:25:27 <hub_cap> thats a refactor denis_makogon
18:25:36 <denis_makogon> hub_cap, that's feature coverage
18:25:38 <ramashri> if resource provisioning can be done using heat why have another strategy for provisioning
18:25:38 <grapex> It's like living next to a creek bed everyone dumps garbage into; we keep saying "that land is scheduled for development so don't bother." I think we should at least be open to the idea- though I don't know if it should be an entire blueprint
18:25:43 <hub_cap> if u move from using A to B and not chanign how the api looks its a refactor
18:25:47 <hub_cap> rewriting existing code
18:26:04 <SlickNik> denis_makogon: Are you looking to make the heat changes needed for trove?
18:26:13 <vipul> denis_makogon: why don't you fix the missing things in Heat and then we can revisit this
18:26:15 <denis_makogon> SlickNik, yes
18:26:30 <SlickNik> vipul: ++
18:26:31 <denis_makogon> SlickNik, i filed and delegated several BPs in Heat to my college
18:26:40 <hub_cap> denis_makogon: why not just do that work?
18:26:40 <amrith> vipul, fix the missing things or identify them?
18:26:48 <hub_cap> if u want it done so bad, do it! :)
18:26:52 <SnowDust> yogeshmehra : how is yor bp different from denis_makogon s BP being discussed
18:26:58 <vipul> amrith: i am assuming they have been identified..
18:27:09 <amrith> vipul, ok; thx.
18:27:21 <amrith> vipul, I wasn't making that assumption.
18:27:24 <yogeshmehra> SnowDust: denis_makogon wants to abstract the exact orchestration/resoruce-provisioning out
18:27:28 <vipul> i hope :) amrith
18:27:30 <yogeshmehra> and use heat just as an implementation
18:27:46 <yogeshmehra> my BP was about makinng complete heat/trove integration
18:27:49 <denis_makogon> current heat support is nothing else than experimental
18:28:02 <yogeshmehra> its not complete, true
18:28:11 <juice> denis_makogon: i think what most folks are saying here is that when Heat has the features we need, then rip out the old code and go with a pure Heat timplmentation
18:28:18 <SnowDust> that's for no benefits if orchestration will b thru heat
18:28:18 <juice> until then, don't change the code
18:28:36 <denis_makogon> juice, what about backward compatibility ??
18:28:49 <amcrn> denis_makogon: the existing path will still exist as hub_cap stated, it will just be deprecated.
18:28:56 <denis_makogon> we can't just rip out use of native services
18:29:05 <yogeshmehra> denis_makogon: i think we need to go incremental on this and need to see how it goes along the clustering also
18:29:08 <hub_cap> right, no one will use heat until it works for the use cases, if not we wil have to do data migrations every time we add "somehting" in heat
18:29:09 <SlickNik> Okay, I think the consensus is that we should revisit this bp only once denis_makogon has the gaps in heat which are needed by trove (resize, et al) fixed.
18:29:15 <hub_cap> denis_makogon: think about this
18:29:26 <hub_cap> youve done all of the "core" stuff, now someoen adds volume resize
18:29:33 <hub_cap> well all your existing voumes are out of sync
18:29:37 <hub_cap> so u have to write a migration in heat as well
18:29:44 <juice> what about denis_makogon's question on backward compatibility
18:29:46 <hub_cap> on top of changing what trove is referencing
18:30:08 <denis_makogon> hub_cap, heat would not do migration
18:30:16 <juice> if you have some instances that are currently running that were provisioned the old way, how would the new heat only approach work
18:30:21 <denis_makogon> hub_cap, that's why Trove needs think about migration
18:30:30 <denis_makogon> hub_cap, it's only our problem
18:30:45 <vipul> we'd migrate them juice by creating whatever's needed in Heat
18:30:45 <hub_cap> its only our problem if we do it yoru way
18:30:47 <denis_makogon> juice, let me show
18:30:51 <esp> I think we said there could be a migration script one day that would populate heat’s db (backfill)
18:30:51 <hub_cap> its no ones problem if u fix heat first
18:31:07 <SnowDust> ;)
18:31:08 <SlickNik> juice: That's something we need to think about before we deprecate the old way.
18:31:19 <denis_makogon> juice, https://github.com/denismakogon/trove/blob/bp/resource-manager-interface/trove/taskmanager/resources/migration.py
18:31:22 <hub_cap> does anyone else feel like the horse is beaten to death? i want to -2 this
18:31:34 <hub_cap> and say the prereq is to make sure heat works for our use cases
18:31:41 <hub_cap> and encourage denis_makogon to fix heat to work for us
18:32:00 <SlickNik> Okay, time for vote
18:32:04 <vipul> hub_cap: +1
18:32:06 <denis_makogon> hub_cap, i'm not working at heat, tasks were deletegated
18:32:07 <SnowDust> -2
18:32:12 <yogeshmehra> -1
18:32:26 <amcrn> #vote no
18:32:37 <SlickNik> #startvote Resource management driver? yes, no, postpone_after_heat
18:32:38 <openstack> Begin voting on: Resource management driver? Valid vote options are yes, no, postpone_after_heat.
18:32:39 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
18:32:41 <hub_cap> denis_makogon: and they are being worked on now by people? u cant take them? like someone is actively coding on them?
18:32:43 <grapex> I'm for this blueprint being drastically reduced in scope, but as is it's too big
18:32:45 <hub_cap> #vote no
18:32:46 <amcrn> #vote no
18:32:52 <cp16net> #vote no
18:32:54 <vipul> #vote no
18:33:04 <SnowDust> #vote no
18:33:05 <denis_makogon> hub_cap, yes
18:33:14 <SlickNik> #vote postpone_after_heat
18:33:17 <juice> #vote no
18:33:19 <hub_cap> then ask them if u can help :)
18:33:26 <hub_cap> #vote postpone_after_heat
18:33:29 <juice> #vote yes
18:33:31 * hub_cap changes his vote
18:33:36 <hub_cap> lol juice your vote is yes now
18:33:49 <hub_cap> u cant make a dependent vote
18:33:56 <juice> I saw the post pone after heat :)
18:34:00 <hub_cap> i know ;)
18:34:07 <juice> #vote no
18:34:08 <denis_makogon> hub_cap, tasks already being spreaded, i can only wait
18:34:22 <SlickNik> anyone else?
18:34:33 <SlickNik> #endvote
18:34:33 <openstack> Voted on "Resource management driver?" Results are
18:34:34 <openstack> postpone_after_heat (2): SlickNik, hub_cap
18:34:35 <openstack> no (5): amcrn, juice, cp16net, vipul, SnowDust
18:34:38 <hub_cap> denis_makogon: plz send me the blueprints
18:34:44 <hub_cap> so i can talk to the people about it
18:34:57 <hub_cap> and make sure that they are all being worked on _immediately_
18:35:10 <denis_makogon> hub_cap, search in your mailbox, i sent at least 2+ emails
18:35:24 <denis_makogon> with [Trove] tag
18:35:35 <SlickNik> denis_makogon: It would be good if you could drive that work and make sure that all the scenarios needed for trove are covered in heat.
18:35:48 <SlickNik> Okay, moving on.
18:35:49 <hub_cap> denis_makogon: and the have the heat bps in them ?
18:35:56 <denis_makogon> SlickNik, thanks what i'm dling
18:35:58 <denis_makogon> *doing
18:36:03 <denis_makogon> hub_cap, yues
18:36:07 <hub_cap> wonderful thx
18:36:26 <SlickNik> #topic Managed Instances
18:36:35 <juice> so we talked a few weeks back about locking down nova vms which are managed by Trove.  The first proposal was to have Trove Admin own the Instances in Nova.  That was shot down.  I went back to the nova team to discuss
18:36:52 <juice> phil day proposed using the lock/unlock feature in Nova (wow!)
18:37:13 <grapex> juice: lock / unlock?
18:37:20 <juice> the basic concept is that Trove Admin will put a lock on instances preventing users from compromising the integrity of the vm
18:37:23 <esmute> wow!
18:37:30 <juice> grapex  - yes
18:37:34 <hub_cap> denis_makogon: you do realize youre assigned to one of the bps right?
18:37:39 <hub_cap> https://blueprints.launchpad.net/heat/+spec/handle-update-for-security-groups
18:37:43 <juice> basically there is a user level lock and an admin level lock
18:37:46 <hub_cap> assignee denis m
18:37:52 <hub_cap> so u can do that work :)
18:37:52 <amrith> juice, what does a "lock" do? in reality?
18:37:59 <denis_makogon> hub_cap, yes, i haven't update it yet
18:38:07 <hub_cap> well u should do that work!
18:38:08 <juice> amrith: it just puts a flag in the db
18:38:14 <kevinconway> ¡mom
18:38:20 <juice> nova checks this flag on just about every modification call
18:38:34 <hub_cap> so we just need to send lock=true for it?
18:38:40 <esmute> juice: will it hide it from users when doing a nova list
18:38:41 <esmute> ?
18:38:41 <amrith> juice, and it will lock down ssh and things as well? or is that at a different level of protection?
18:38:46 <juice> if the instance is locked, it rejects the call (assuming that your role is lower than the lock holder)
18:38:59 <juice> esmute: no
18:39:26 <juice> amrith: no but the user cannot ssh into the instance because their key does not exist on the server
18:39:36 <amrith> juice, thx
18:39:47 <juice> unfortunately security groups are still modifiable but just about everything else is locked down
18:39:52 <amcrn> if this won't protect security-groups, won't protect volumes, and won't protect swift objects (backups), i'd be hesitant to vote yes for this if the scope of work is high.
18:40:01 <juice> Trove Admin will be the only thing that can delete these
18:40:17 <vgnbkr> Should this (locking) be an optional feature?
18:40:24 <juice> it will protect volumes since there is not much you can do to a volume that is already attache
18:40:34 <juice> and you can't detach the volume from a locked server
18:40:38 <amcrn> juice: it'll protect a detach? oh, neat.
18:40:46 <amcrn> i take back my comment :)
18:40:49 <SlickNik> vgnbkr: It is being proposed as optional.
18:40:52 <juice> vgnbkr: it will be optional
18:41:10 <juice> just a flag that say's "lock the instances" or some such
18:41:10 <vipul> juice: does neutron support any locking?
18:41:28 <juice> vipul: not that I am aware of
18:41:32 <vgnbkr> Sorry, I looked through the bp and didn't see it being optional.  OK, thanks.
18:41:41 <SlickNik> vipul: Nope, I haven't seen anything in neutron corresponding to this.
18:41:43 <juice> vipul: let me look at my api printout sheet :)
18:41:52 <vipul> so when we manage networks, secgroups in neutron, will that cause this to be disjoint?
18:42:07 <vipul> meaning, i can delete a network from underneath the lock
18:42:29 <juice> vipul: can you give me an example of where you see some disjointedness
18:42:32 <vipul> or is that ok? since they own the network
18:42:35 <SlickNik> vipul: neutron won't let you delete a network that's in use.
18:42:44 <SlickNik> if I recall correctly.
18:42:45 <juice> the network they own they better have access to
18:42:51 <esmute> vipul: i think it works like volume-attached..
18:42:58 <juice> I don't think we would want to prevent them from making changes there
18:43:00 <esmute> it wont let you delete it..
18:43:01 <vipul> I see.. ok then this might be fine
18:43:26 <juice> it may mean that we need to revisit the network attachment in trove-api
18:43:41 <juice> currently, you specify the attached networks only in the create.
18:43:44 <vgnbkr> For the optional thing, I meant more like the root setting, not a global setting.
18:43:57 <juice> I don't know if we allow for that network to be detached to the old and attached to new
18:44:29 <mattgriffin> SlickNik, i'm here now :) when we're ready
18:44:34 <juice> vgnbkr: I think this would be a global thing - as global as the api and task manager instances are concerned
18:45:12 <mat-lowery> Doesn't Heat itself have this same problem (can't protect its resources)? Also all other projects that are Heat-like (i.e. manage OpenStack "primitives") or use Heat (e.g. Sahara maybe): are they attempting to solve this?
18:45:18 <SlickNik> mattgriffin: Okay.
18:45:28 <juice> mat-lowery: good question
18:45:34 <vgnbkr> juice: so how does this differ from root-enable?  Both are to prevent the user from messing us up, but one is set global and one per-instance?  Doesn't seem constent.
18:45:40 <juice> I heard rumors of a project called Service VMs
18:45:56 <juice> there is something forming from dust currently - perhaps another project
18:46:05 <grapex> juice: Sounds mythological.
18:46:17 <SlickNik> juice: How will the locking work in the case we're provisioning using heat?
18:46:25 <juice> grapex: its in early planetary formation
18:46:59 <juice> SlickNik: the instance will need to be locked outside of Heat invocation
18:46:59 <yogeshmehra> SlickNik: good question...
18:47:12 <grapex> juice: Is it currently in the hot gaseous state?
18:47:14 <grapex> :p
18:47:23 <juice> grapex: exactly
18:47:29 <juice> but it has some momentum
18:47:39 <SlickNik> Okay, I think it's time to vote.
18:47:40 <grapex> juice: Cool
18:47:49 <esmute> vote for juice!
18:47:53 <juice> vgnbkr: I'mo not sure this would be inconsistent with root-enable
18:47:57 <juice> esmute :)
18:48:09 <SlickNik> #startvote Managed Instance? yes, no, still_have_questions
18:48:10 <openstack> Begin voting on: Managed Instance? Valid vote options are yes, no, still_have_questions.
18:48:11 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
18:48:20 <juice> root-enable is on the mysql db not the vm
18:48:31 <juice> vgnbkr ^ ^
18:48:49 <yogeshmehra> #vote yes
18:48:52 <juice> very small optional change guys and gals
18:48:54 <amcrn> #vote yes
18:48:56 <esmute> #vote yes
18:48:57 <juice> there we go we have a yes
18:48:59 <juice> another yes
18:49:01 <hub_cap> #vote yes
18:49:01 <SlickNik> #vote yes
18:49:03 <juice> do go I get one more
18:49:05 <juice> yes!
18:49:06 <hub_cap> juice: thx for the commentary
18:49:06 <grapex> #vote yes
18:49:07 <juice> :)
18:49:11 <hub_cap> im pretty sure that the bot will tell us too ;)
18:49:12 <vipul> #vote yes
18:49:14 <SlickNik> lol hub_cap
18:49:16 <hub_cap> #vote no
18:49:19 <juice> I recently went to an auction
18:49:20 <hub_cap> just cuz i wanna piss juice off
18:49:32 <juice> hub_cap: I can always count on you
18:49:38 <SlickNik> #endvote
18:49:39 <openstack> Voted on "Managed Instance?" Results are
18:49:40 <openstack> yes (6): SlickNik, yogeshmehra, amcrn, esmute, vipul, grapex
18:49:41 <openstack> no (1): hub_cap
18:49:57 <grapex> hub_cap: harsh
18:50:03 <SlickNik> #topic List all datastore types and versions by a single API call
18:50:05 <juice> there it's on record
18:50:07 <SlickNik> NehaV1: around?
18:50:15 <NehaV1> hey i am
18:50:45 <SlickNik> take it away
18:50:48 <NehaV1> yes i have added this bp https://blueprints.launchpad.net/trove/+spec/list-datastore-type-and-versions
18:50:49 <grapex> NehaV1: Right off the bat I must object
18:50:56 <NehaV1> why?
18:51:01 <grapex> I was able to read through this and understand it in 15 seconds
18:51:07 <grapex> That's not long and overly verbose enough
18:51:14 <cp16net> lol
18:51:19 <grapex> NehaV1: I keed, I keed
18:51:35 <NehaV1> cheese
18:51:36 <SlickNik> lol
18:51:49 <grapex> I do wonder about the request url though- "datastoreandversions"
18:51:56 <SlickNik> I don't like the endpoint name, but I'm not gonna bikeshed over it.
18:52:01 <NehaV1> yes i dont like it either
18:52:05 <cp16net> i think the idea is a valid but i am not sure about he uri and the structure of the data
18:52:07 <grapex> SlickNik: But that's the funnest part!
18:52:26 <grapex> I wonder if instead we could pass an extra query parameter to the datastore list call to make it do a verbose list
18:52:31 <vipul> yea let's change the URL
18:52:32 <NehaV1> we can think about augmenting the existing get datastores call
18:52:35 <grapex> or would that not be RESTy enough?
18:52:40 <kevinconway> this all be easier if all requests went to "/" and we just used some field in the JSON blob to let trove know what we really wanted
18:52:52 <grapex> kevinconway: I like it
18:53:00 * amcrn glares at kevinconway
18:53:02 <vipul> would it be that bad to just change the existing /datastores call
18:53:03 <SnowDust> grapes+1on extra request parameter
18:53:09 <amcrn> :P
18:53:13 <vipul> and also reply with all versions
18:53:14 <cp16net> oh and everything keyed off query paramters
18:53:17 <SlickNik> I don't like that we have to make 2 calls in the first place. I'd be fine just changing the existing API to return both.
18:53:21 <NehaV1> vipul +1
18:53:26 <SnowDust> grapex
18:53:29 <robertmy_> vipul: +1
18:53:42 <SlickNik> vipul: you quicker than me :) +1
18:53:52 <vipul> :) SlickNik
18:53:58 <cp16net> hmm would that be changing the contract we have on the call tho?
18:53:59 <grapex> SlickNik: That wouldn't be too bad, but then we lose a lighter weight list call for just the datastore types.
18:54:01 <amcrn> i'll point out this is indicative of the need to merge datastore + datastore-version into one field
18:54:13 <NehaV1> so the existing call returns the default version for every db type. i just want to make sure we will be able to do the same with this call
18:54:37 <kevinconway> amcrn: so just create a typeversion field?
18:54:54 <grapex> amcrn: Are you saying there'd just be one resource instead of two?
18:55:13 <cp16net> i feel like in order to not break our contract we have on the other URis we could do something like /datastores/versions
18:55:27 <cp16net> that doesnt have a path yet
18:55:38 <vipul> it's kinda like /flavors vs. /flavors/detail
18:55:40 <vipul> they return the same thing
18:55:41 <cp16net> and its pretty simple at the same time
18:55:42 <amcrn> grapex kevinconway: in horizon it's being folded into one field, now we're adding a convenience uri to return it as if they're one (because it's inconvenient for n +1 calls); i'm seeing a pattern.
18:56:01 <NehaV1> thats right cp16net
18:56:13 <grapex> amcrn: I think nesting them still makes sense for management though.
18:56:14 <cp16net> amcrn: yeah
18:56:19 <kevinconway> amcrn: i don't think we should tailor the API to fit one specific application like horizon. if there are uses for seperate fields then let horizon make two requests
18:56:23 <grapex> amcrn: I do get your point.
18:56:41 <SlickNik> okay, so we all like this.
18:56:45 <grapex> Maybe the REST call should be one field, while the call to manage it should be two
18:56:56 <grapex> SlickNik: Agreed, let's vote.
18:57:04 <amcrn> SlickNik: well, can't the concept of a default version be a separate bp? it really has nothing to do with adding a uri for convenience?
18:57:13 <robertmyers> what are the benefits of haveing two different resources for datastores?
18:57:14 <kevinconway> if the cost of two API calls is significant in terms of response time then maybe we could combine, but if the only cost is a dev has to make to calls in their app i find that perfectly acceptable
18:57:25 <amcrn> robertmyers: absolutely nothing as far as I can see
18:57:41 <robertmyers> thats what I thought
18:58:00 <cp16net> yeah seems to just cause is problems
18:58:08 <robertmyers> can we just deprecate it?
18:58:12 <vipul> let's combine them.. but leave the API urls in-tact for backwards compat
18:58:15 <kevinconway> i thought the point of having seperate type and version is that some of our abstractions are best targeted at a type and others at a version
18:58:22 <kevinconway> like backups vs datastore upgrade
18:58:29 <grapex> kevinconway: I agree
18:58:41 <amcrn> kevinconway: the problem is that everything thus far has been tied to a version.
18:58:41 <grapex> but it does seem like increasingly few things in Trove do *everything* based exclusively on type
18:58:48 <grapex> they seem to always need a version too
18:58:52 <amcrn> grapex: yep
18:59:17 <grapex> Never say never though. It could be useful to keep them as seperate objects in Trove at least. And I think it's easier to deal with them seperately using Trove manage
18:59:24 <grapex> for the API calls though I agree, one call would be easier
18:59:26 <amcrn> either way, back specifically to this bp: the ability to set a version as a default for that type does not exist, i'd like to see this implemented separately from the convenience uri being spoken about.
18:59:38 <amcrn> actually it does, i'm an idiot
18:59:38 <cp16net> maybe as separate obejcts but not as separate api calls to get them
18:59:42 <amcrn> nevermind.
18:59:46 <NehaV1> for purposes of keeping it clean, i will suggest that we have db type and version both stored separately
18:59:57 <kevinconway> would the convenience call replace the existing calls or simply live along side them?
19:00:33 <grapex> kevinconway: the current blueprint is for it to live alongside
19:00:36 <cp16net> kevinconway: i think that was debated... i was for it living along side
19:00:37 <SlickNik> Okay. So let's vote on this.
19:00:39 <vipul> my vote would be to replace existing /datastores call and make it 'convenient'
19:00:52 <SlickNik> And we can have the combination of type/version as a separate bp.
19:01:05 <grapex> vipul: No one at Rax is really using it- *yet*. This is pretty late to totally change the API though.
19:01:18 <esmute> i wonder if we can do something with the datastore views to present the versions nicely
19:01:21 <vipul> even if it's just an API response? grapex
19:01:38 <vipul> we're not changing the requests or removing anything
19:01:41 <vipul> just addign more info
19:01:45 <kevinconway> especially if it's an API response. that's our primary contract with application developers
19:01:50 <hub_cap> we cannot _change_ the api
19:01:53 <hub_cap> its done
19:01:55 <hub_cap> we are integrated
19:01:57 <hub_cap> period
19:02:03 <hub_cap> we can only add to it
19:02:06 <vipul> you cannot add more to the reponse?
19:02:07 <SlickNik> #startvote List all datastore types and versions by a single API call? yes_with_new_route, yes_and_change_existing, no
19:02:09 <openstack> Begin voting on: List all datastore types and versions by a single API call? Valid vote options are yes_with_new_route, yes_and_change_existing, no.
19:02:10 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
19:02:10 <amcrn> hub_cap: i think vipul is saying if you add an "optional" field to the existing response payload, you havent' broken the contract
19:02:11 <hub_cap> yes vipul u can add to it
19:02:13 <grapex> #vote yes
19:02:13 <openstack> grapex: yes is not a valid option. Valid options are yes_with_new_route, yes_and_change_existing, no.
19:02:23 <hub_cap> yes absolutely amcrn (sorry i just came back and read what grapex said)
19:02:25 <vipul> well there we go.. so that's all i'm suggesting
19:02:42 <hub_cap> ok so is "yes and change existing" really "yes and add to existing?"
19:02:44 <grapex> So hub_cap is saying we can't change existing
19:02:51 <hub_cap> we cannt modify existing
19:02:52 <cp16net> #vote yes_with_new_route
19:02:53 <hub_cap> we can add to
19:03:07 <SlickNik> #endvote
19:03:08 <hub_cap> {a, b} can become {a, b, c} but cannot become {a, d}
19:03:08 <openstack> Voted on "List all datastore types and versions by a single API call?" Results are
19:03:09 <vipul> #vote add_to_existing
19:03:10 <openstack> yes_with_new_route (1): cp16net
19:03:17 <hub_cap> thx SlickNik lets fix the vote options
19:03:19 <SlickNik> #startvote List all datastore types and versions by a single API call? yes_with_new_route, yes_and_add_to_existing, no
19:03:20 <openstack> Begin voting on: List all datastore types and versions by a single API call? Valid vote options are yes_with_new_route, yes_and_add_to_existing, no.
19:03:21 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
19:03:28 <vipul> #vote add_to_existing
19:03:29 <openstack> vipul: add_to_existing is not a valid option. Valid options are yes_with_new_route, yes_and_add_to_existing, no.
19:03:29 <amcrn> vipul: your "add_to_existing" is assuming the extra ?parameter correct?
19:03:41 <SlickNik> Okay, to be clear I changed the options.
19:03:44 <grapex> #vote this_is_silly
19:03:45 <openstack> grapex: this_is_silly is not a valid option. Valid options are yes_with_new_route, yes_and_add_to_existing, no.
19:03:51 <SlickNik> lol @ grapex
19:03:52 <vipul> amcrn: no, just change it as-is
19:03:56 <grapex> #vote yes_and_add_to_exisiting
19:03:56 <openstack> grapex: yes_and_add_to_exisiting is not a valid option. Valid options are yes_with_new_route, yes_and_add_to_existing, no.
19:04:00 <grapex> #vote yes_and_add_to_existing
19:04:04 <vipul> keep URI intact.. no query param needed amcrn
19:04:10 <vipul> #vote yes_and_add_to_existing
19:04:14 <amcrn> vipul: cool, just clarifying for myself and others :)
19:04:15 <robertmyers> #vote yes_and_add_to_existing
19:04:21 <SlickNik> #vote yes_and_add_to_existing
19:04:22 <amcrn> #vote yes_and_add_to_existing
19:04:29 <esmute> #vote yes_and_add_to_existing
19:04:31 <NehaV1> #vote yes_and_add_to_existing
19:04:49 <SlickNik> Any more votes?
19:04:49 <yogeshmehra> #vote yes_and_add_to_existing
19:04:55 <amcrn> #vote lunch
19:04:55 <juice> #vote yes_and_add_to_existing
19:04:55 <openstack> amcrn: lunch is not a valid option. Valid options are yes_with_new_route, yes_and_add_to_existing, no.
19:05:04 <SlickNik> #endvote
19:05:04 <openstack> Voted on "List all datastore types and versions by a single API call?" Results are
19:05:05 <openstack> yes_and_add_to_existing (9): SlickNik, NehaV1, robertmyers, amcrn, juice, yogeshmehra, esmute, vipul, grapex
19:05:06 <kevinconway> #vote wait
19:05:07 <juice> amcrn :)
19:05:08 <vipul> denied! juice
19:05:12 <hub_cap> lol
19:05:13 <kevinconway> awe...
19:05:13 <SlickNik> #endmeeting