00:00:06 <devananda> #startmeeting db
00:00:07 <openstack> Meeting started Fri Nov  2 00:00:06 2012 UTC.  The chair is devananda. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:00:09 <openstack> The meeting name has been set to 'db'
00:00:16 <dragondm> Hello!
00:00:20 <devananda> hi all!
00:00:20 <dripton> hi
00:00:37 <devananda> so who all is here?
00:00:44 <jog0> o/
00:00:46 * dragondm waves
00:01:26 * devananda hopes others trickle in
00:01:49 * dragondm thinks likewise
00:01:53 <devananda> there are 4 big topics from the summit, and jog0 added 2 items to the agenda
00:02:04 <devananda> lets see how quickly we can go :)
00:02:13 <devananda> #topic db-common / oslo
00:02:26 <devananda> anyone here to update on db-common?
00:03:15 <devananda> :(
00:03:39 <devananda> let's try this another way since it's just 3 of us for now
00:03:50 <devananda> #topic open discussion
00:03:51 <sdague> <= just got here
00:04:48 <jog0> lets start with an easy one
00:05:04 <jog0> nova/db/api allows for more then one nova-compute managing a single host
00:05:12 <jog0> do we still need that? https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L380
00:06:13 <dripton> Looks like nobody is sure.
00:06:14 <dragondm> *blink* I can't think of a good use case...
00:06:19 <sdague> I'd guess no, where's it actually used?
00:06:28 <devananda> i can't think of any reason for it either
00:06:38 <dripton> Is there significant benefit to removing it?
00:06:55 <sdague> that's got to have feedback from some of the nova core folks though. I'd say suggest a remove patch and see what people say :)
00:07:03 <jog0> dripton: lets confusing code
00:07:09 <jog0> sdague: I agree
00:07:21 <dripton> agreed, but it's a compatibility change so you might get pushback.  Worth a try.
00:07:21 <devananda> ack-grep for it... lots of hits outside of db/api
00:07:25 <dragondm> It's used in a few places, but many parts of the code just do [0] on the return
00:07:49 <dragondm> i.e. nova/compute/api.py
00:08:09 <devananda> scheduler/manage and scheduler/driver as well
00:08:30 <jog0> right
00:08:40 <devananda> i'm not sure if it's in their plans, but i wonder if it would have any use to baremetal driver
00:08:44 <jog0> I can propose a patch and see how it goes
00:08:53 <jog0> AFAIK bare metal goes the other way
00:08:55 <dragondm> Plus I can think of things that would produce odd results architecturally if there was more than 1 compute on the host.
00:09:17 <jog0> instead of multiple services controlling a single node, its a one service controlling multiple nodes
00:09:21 <dragondm> Yah, I can see multiple hosts on a compute, but not the reverse.
00:09:30 <devananda> jog0: yes, they do that currently
00:09:40 <devananda> and had to add an extra "node" field to accomodate that
00:09:47 <devananda> however i can see a need for the opposite at some point
00:09:55 <sdague> I guess that was actually a use case that came up, instead of doing the bare metal the way it's done, just run a bunch of computes on one host, each talking to a physical node
00:10:00 <devananda> ie, >1 compute node sharing control over the same group of hw resources
00:10:22 <devananda> otherwise 1 n-cpu instance is a SPoF for a whole _lot_ of hw
00:10:41 <devananda> afaik NTT isn't doing that yet, but i don't see another solution, and I know they eventually want HA
00:11:07 <devananda> sdague: yea, that came up but is terrible :)
00:11:08 <jog0> devananda: true, but although the DB supports this model the code doesn't at all
00:11:12 <devananda> try scaling that to 1000's of hw nodes
00:11:24 <dragondm> don't want to do that.
00:11:46 <sdague> devananda: hey, just saying :)
00:11:52 <dragondm> (That's the architecture of the pre-Nova rackspace code. It dosen't scale)
00:12:01 <devananda> fair 'nuf :)
00:12:03 <devananda> moving on?
00:12:05 <jog0> devananda: wouldn't it make more sense to do the bare metal provisioning nodes  as a  RR queue?  like the scheduler or network?
00:12:35 <dragondm> +1
00:12:52 <devananda> jog0: afaik different groups want to do schedulng in different ways for baremetal...
00:13:00 * devananda is easily distracted...
00:13:10 <jog0> +1 moving on
00:13:21 * dragondm nods
00:13:35 <dripton> next topic?
00:13:37 <devananda> db-archiving and db-cleanup are the most interesting to me right now :)
00:13:52 <devananda> #topic db-cleanup
00:14:14 <devananda> i've posted a doc with some guidelines on using session and transactions
00:14:32 <devananda> and starting to work through db/sqlalchemy/api.py bringing things into line with that
00:14:47 <dripton> do you have a link to the doc?
00:14:59 <devananda> https://review.openstack.org/#/c/14966/
00:15:10 <dragondm> It would be *very* nice to not create a new session for each db call.
00:15:21 <devananda> where to put the doc was briefly discussed in nova meeting earlier today
00:15:43 <devananda> my feeling is it should become part of db-common
00:15:49 <devananda> so for now, living in session.py makes sense to me
00:16:02 <devananda> as that will, aiui, get absorbed by db-common
00:16:11 <sdague> devananda: so most of that low level mapping is to mysql, any idea on how some of those things map in the pgsql case?
00:16:28 <devananda> sdague: not really, no
00:16:33 <devananda> i'd love input from someone who does
00:17:14 <devananda> dragondm: yes. and to avoid creating transactions just for a single write :)
00:17:30 <sdague> honestly, I don't know either. But as we try to make pgsql a first class citizen I'm a little concerned about overoptimizing the way sqlalchemy works to get mysql behavior right
00:17:35 <dragondm> Plus the whole detatched object issue...
00:18:22 <devananda> any pgsql guys are welcome to chime in // help with that optimization
00:18:29 <devananda> i'm just doing what i know, which is mysql ;)
00:18:33 <sdague> fair :)
00:18:34 <dragondm> I am familliar w/ both
00:18:53 <devananda> dragondm: sounds like you just volunteered :)
00:18:59 <dragondm> I can look it over.
00:19:22 <devananda> #action dragondm to make sure session optimizations wont kill pgsql
00:19:33 <sdague> hopefully that gets easier soon, as there is devstack patch I need to review that will let it install pgsql
00:19:47 <dripton> sdague: excellent idea
00:19:51 <devananda> sdague: ++
00:20:25 <devananda> anything else on db-cleanup?
00:20:36 <jog0> yes
00:20:43 <sdague> #link https://review.openstack.org/#/c/15224/ if anyone is interested in helping validate it
00:20:53 <jog0> two things
00:21:16 <jog0> the db api has been added piecemeal over time
00:21:30 <dragondm> It's a junk drawer
00:21:43 <jog0> and can use a good makeover
00:21:59 <jog0> such as remove duplicate DB APIs
00:22:04 <dragondm> +1000
00:22:11 <jog0> or ones that have overlapping functionality etc
00:22:19 <dripton> Can we do it safely without breaking out-of-tree users of the API?
00:22:32 <dripton> Or do we need a deprecation period?
00:22:44 <jog0> dripton: nova/db/api is not a stable public API
00:22:53 <jog0> so AFAIK we can do whatever we want
00:22:55 <dripton> ok
00:22:55 <dragondm> also +1000
00:23:26 <jog0> so to start with perhaps we can break down the API to categories and review  the code and make a list of what can be removed
00:24:02 <jog0> the  calls are broken down into categories with '#####' between them
00:24:23 <devananda> db/sqlalchemy/api is already fairly well organized
00:24:23 <dragondm> eventually to be moved to their own modules?
00:24:24 <sdague> jog0: is there a blueprint in for it? Agree it would be good to get technical debt out of there
00:24:35 <jog0> I count 43 dividers in the code
00:24:45 <jog0> sdague: for now its in db-cleanup
00:24:53 <jog0> unless we want to break it out
00:25:04 <sdague> it probably makes sense to break up db-cleanup into more discrete tasks
00:25:15 <devananda> #link https://blueprints.launchpad.net/nova/+spec/db-cleanup
00:25:52 <jog0> sdague: I agree, db-cleanup is a catch all until we are able to break down everything
00:26:04 <devananda> sdague: ++. so far i see 2 sub-tasks
00:26:16 <devananda> sessions/transactions and api cruft
00:26:27 <jog0> ++
00:26:31 <sdague> blueprints can have deps, so you could make other blueprints that are the prereqs for cleanup
00:26:37 <dripton> I'll volunteer to look for any unused functions in api.py and make a patch to remove them.
00:26:53 <sdague> so you still get them collected together, but they are small enough that you could say one was done at grizzly-2, for instance
00:27:02 <jog0> dripton:  I don't think too many are unused -- I did that test a while back, but some may be able to be consolidated
00:27:02 <devananda> #action dripton hunt for unused api calls
00:27:05 <devananda> dripton: thanks :)
00:27:40 <dripton> jog0: if I don't find any then that's okay -- it won't take too long
00:27:49 <jog0> devananda: another action item for what sdague  said
00:27:57 <jog0> dripton: agreed
00:28:09 <devananda> another aspect of api cleanup jog0 brought up earlier is objects leaking out
00:28:21 <devananda> #action create add'l bps and link them under db-cleanup
00:28:52 <jog0> the plan  there was to add a debug version of nova/db/api that would introspect all return values and hunt for sqlalchemy objects
00:29:34 <jog0> unless someone has a better idea ...
00:30:05 <jog0> I can write the patch and get a list of all the abstraction violations for next week
00:30:14 <devananda> ++
00:30:49 <devananda> #action jog0 find db/api abstraction violations
00:31:00 <devananda> moving on, since cleanup is an unfortunately huge topic ...
00:31:10 <devananda> #topic db-archiving
00:31:33 <devananda> archiving was a pretty lively session at the summit :)
00:31:41 <devananda> i think at least 3 different solutions were proposed
00:31:49 <jog0> if you thought db cleanup was big… archiving is even bigger
00:32:02 <devananda> in my eyes, archiving is a part of cleanup ;)
00:32:14 <devananda> there's code cruft, and there's data cruft ...
00:32:31 <devananda> deleted=1 needs to go
00:32:37 <dripton> Is there anyone who actually likes soft deletes?
00:32:38 <dragondm> +1
00:32:50 <dragondm> And real constraints would be nice.
00:32:52 <dripton> Someone put them in, so someone must like them?
00:32:54 <devananda> yes!! +++
00:33:26 <devananda> i believe Paul Carlton was presenting their way of using an EVENT that copies deleted=1 records to a shadow table
00:33:46 <dripton> and then deletes the rows after they're archived?
00:33:53 <devananda> no
00:34:06 <devananda> turns out some large companies believe they can't ever throw away records
00:34:07 <dragondm> And there are some places where old data can just be deleted.
00:34:17 <sdague> dripton: I think it's part of the eventual consistency strategy. There are plenty of times where things communicate through the database, and if the row just isn't there, it would go weird
00:34:18 <dkehn> seems like a way to offer auditing, but over time emcumbers the db
00:34:38 <dragondm> Nova is not a historical record.
00:34:47 * dripton nods
00:34:49 <dkehn> and definately will slow down the db if you don't move them to backup tables
00:35:00 <dkehn> true
00:35:15 <devananda> sdague: so that is an interesting point
00:35:27 <devananda> sdague: do you know what parts of nova actually need to read deleted=1 records?
00:35:37 <devananda> as in, what would actually break if things were deleted?
00:35:58 <sdague> devananda: I know at one point there was a reference to flavors that broke because of that
00:36:08 <jog0> devananda:  I think that was one of our action items from the summit
00:36:33 <sdague> I don't know of an active one at the moment, but I'm guessing they are hiding in there
00:36:41 <sdague> no-db-compute makes it better
00:36:44 <dragondm> Perhaps that info should simply be copied to the instance on create?
00:36:53 <jog0> one is simplue usage report
00:36:57 <jog0> simple *
00:37:13 <dripton> So do we wait until after no-db-compute is done, then try to change soft deletes to real deletes?
00:37:25 <sdague> but there at least needs to be some audit to figure out how busted things would get
00:37:42 <devananda> dripton: i'm not sure how no-db-compute helps. that is only abstracting the db access away from n-cpu to some middle tier
00:37:47 <devananda> the same db access is needed, aiui
00:37:54 <jog0> sdague: sounds like you volunteered
00:38:12 <sdague> jog0: sadly, too many tasks on my plate right now :P
00:38:34 <sdague> once I'm not behind the 8 ball on v3 api, I'll consider :)
00:39:54 <devananda> ok, seems like no action on that for now, moving on?
00:40:09 <jog0> git grep "read_deleted='yes'" | wc -l = 21
00:40:20 <devananda> jog0: look for "true" as well
00:40:39 <jog0> git grep "read_deleted=\"yes\"" | wc -l = 33
00:40:59 <devananda> now you're just making me sad :(
00:41:19 <jog0> lets move on for now, lets talk shadow table trigger
00:41:30 <jog0> I think we can punt on some of the read_deleted cases for now
00:41:40 <jog0> and hit the low hanging fruit first
00:42:21 <devananda> jog0: you have a good idea what said low-hanging fruit is?
00:43:48 <jog0> devananda:  sadly no, will have to go through the tables and correlate the read_deleted
00:44:27 <devananda> jog0: think it would be worth just spec'ing it and letting it sit for someone to pick up?
00:44:42 <jog0> dripton:  perhaps you can look for those while hunting for unused API calls
00:44:54 <dripton> Most of the read_deleted are in api.py
00:45:02 <dripton> I'll take a look while in there.
00:45:23 <jog0> sweet
00:45:32 <devananda> great
00:45:46 <devananda> moving on
00:46:03 <devananda> #topic db-common
00:46:15 <devananda> i haven't been invovled much in ^ so i'll just open the floor :)
00:46:16 <jog0> devananda: action item for dripton ^
00:46:26 <devananda> jog0: ah, thx
00:46:49 <devananda> #action dripton look for read_deleted low-hanging-fruit while reviewing db/api.py
00:47:55 <devananda> anything on db-common? if not, i'll move on to no-db-compute
00:48:08 <devananda> #topic no-db-compute
00:48:30 <devananda> again, open floor
00:48:48 <devananda> since i haven't done anything with no-db-compute yet
00:49:22 <dripton> I know rbryant is working on it, but he's not here.
00:49:41 <jog0> devananda: can you propse a draft of the shadow table trigger?  for db-archiving?
00:50:15 <devananda> jog0: that'll be a pretty big migration, creating N tables and N triggers
00:50:39 <jog0> how about just for one table
00:50:43 <devananda> that i can do :)
00:50:46 <jog0> just a WIP version to get some feedback on
00:50:55 <jog0> see if people like it etc.
00:51:04 <dkehn> devananda, and getting triggers to be compatible across differenets DBs (i.e. psql) careful
00:51:17 <dripton> I don't currently see any triggers in the migrations directory.
00:51:25 <devananda> #action devananda make draft review of on-delete trigger for just one table
00:51:39 <devananda> dkehn: how familiar are you with pgsql? want to help? :)
00:51:43 <dragondm> (we barely have indexes :( )
00:51:43 <devananda> #topic db-archiving
00:51:54 <devananda> dragondm: as of folsom, there should be a _lot_ more indexes
00:52:17 <dragondm> True. We probably need more, ho.
00:52:22 <dkehn> devananda, I have build migrations tools between the two in the paste , yes I can help, but I wouldn't say I'm and expert
00:52:31 <dkehn> s/paste/past
00:52:46 <devananda> dragondm: about mid-folsom, i reviewed all the queries in db/api and proposed indexes for all of them that were missing
00:53:01 <devananda> dragondm: so unless a lot of new db access patters were added, we /should/ be pretty good right now
00:53:08 <devananda> *patterns
00:53:15 <dragondm> Ah
00:53:43 <devananda> #action dkehn assist making on-delete trigger compatible with pgsql
00:54:18 <devananda> #topic open discussion
00:54:19 <dragondm> devananda: right, sorry, long day :P
00:54:41 <jog0> what about a different meeting time?
00:54:53 <devananda> ++
00:55:01 <dkehn> ++
00:55:04 <dripton> +
00:55:11 <devananda> jog0: what do you have in mind?
00:55:43 <jog0> how about one hour earlier?
00:55:43 <devananda> (answering myself) 2hr earlier looks clear on the OS iCal
00:55:49 <jog0> or 2
00:55:59 <dripton> Does the nova meeting often run long?
00:55:59 <clarkb> for those in the US the time change this weekend will move it an hour
00:56:06 <dripton> They are 3 hours before us.
00:56:10 <clarkb> (US less Arizona and Hawaii)
00:56:30 <devananda> dripton: they ran a few minutes long today, but that's just one data point.
00:57:02 <jog0> devananda: usually they run very short
00:57:05 <clarkb> I think it moves back an hour so you get at least one hour for free
00:57:10 <dripton> Then 2 hours earlier should be safe.
00:57:35 <jog0> well all meetings are scheduled on UTC
00:57:52 <jog0> so  the time between meetings is always absolute
00:57:52 <clarkb> ya so the shift is relative to you not UTC
00:57:59 <devananda> #action devananda move meeting up two hours, update wiki and iCal, notify -dev list
00:58:05 <dragondm> +1
00:58:12 <jog0> +1
00:58:16 <dripton> +
00:58:20 <dkehn> +
00:58:51 <sdague> +1
01:00:24 <devananda> ok, thanks all
01:01:47 <devananda> #endmeeting