19:03:19 <mtaylor> #startmeeting
19:03:20 <openstack> Meeting started Tue Apr 10 19:03:19 2012 UTC.  The chair is mtaylor. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:03:21 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:03:31 <mtaylor> who wants to talk about CI?
19:03:52 <LinuxJedi> *tumbleweed*
19:03:57 <mtaylor> excellent
19:04:01 <jeblair> should we start by resuming the conversation sdague started?
19:04:06 <mtaylor> well - first of all - let me introduce Shrews
19:04:23 <mtaylor> who has joined the HP team working on this stuff
19:04:48 <Shrews> hi all
19:04:54 <jeblair> hi Shrews
19:04:57 <LinuxJedi> hey Shrews
19:05:02 <sdague> hey Shrews
19:05:11 <mtaylor> Shrews worked with LinuxJedi and I on Drizzle back in the day, and with us at MySQL before that
19:05:23 <mtaylor> ok. that's enough of that
19:05:25 <LinuxJedi> well, I replace Shrews at MySQL ;)
19:05:49 <mtaylor> sdague: so - about testing mysql stuff
19:06:25 <sdague> sure, I was looking into adding a unit test to nova to test whether we were forcing all the tables to innodb
19:06:50 <sdague> and then I realized that the nova unit tests aren't really set up to run mysql in default configuration
19:07:15 <sdague> so I have the following suggestion, based on something I did in a previous project with multiple databases that it supported
19:08:00 <mtaylor> your idea of trying and then skipping if mysql isn't present isn't terrible, but I'm a little concerned that it's a decent amount of work to make sure that the jenkins builders provide an environment in which those tests run
19:08:02 <sdague> setup the unit tests so they try to run mysql (and later postgresql) tests, but only if they find a specific database / user / password combo
19:08:20 <sdague> like openstack_novaunittests for all 3
19:09:19 <sdague> mtaylor: over how many machines to the builders run?
19:09:33 <sdague> do the builders run
19:10:37 <LinuxJedi> mtaylor: something we can get puppet to configure?
19:10:37 <jeblair> it's undefined; they're cloud servers, and we add and remove them as needed
19:11:04 <jeblair> so yeah, we'd have to add the database install to our puppet configuration for them.
19:11:15 <jeblair> we'd also need to make sure things are re-set appropriately after test runs
19:11:28 <jeblair> since, at the moment, unit tests are run on somewhat long-lived hosts.
19:11:39 <jeblair> (though someday they will probably be run on single-use cloud servers)
19:11:43 <mtaylor> ++
19:12:01 <sdague> jeblair: at least in the test_migration piece I was in, it resets during the setup phase
19:12:21 <sdague> so there might need to be work to make sure the setup / teardown for the unit tests is right
19:12:36 <sdague> to leave the db blank at the end, and blank it at the beginning just in case
19:13:03 <sdague> otherwise it's just the creation of a single mysql user with a grant. Should be pretty easy to add to puppet configs (though my puppet is a little rusty)
19:13:09 <mtaylor> indeed
19:13:15 <jeblair> yeah; this will probably need some buy-in from the nova core devs, since if it fails, it'll snag unrelated changes
19:13:48 <sdague> jeblair: ok, should I start an ML thread on this? kick it off that way?
19:13:52 <jeblair> sdague: i'm not too worried about creating a mysql user in puppet; i wrote that yesterday.  :)
19:13:58 <sdague> jeblair: :)
19:14:10 <sdague> see, easy as cake ;)
19:15:15 <jeblair> the other thing we should realize this would change is the principal of unit tests being easy for developers to run in the virtualenv
19:15:47 <jeblair> and maybe it's worth it and it's time to branch out from that.  just noting that as a change.  :)
19:15:48 <sdague> jeblair: not really, because if the right database isn't found, it just skips out
19:15:48 <mtaylor> well - potentially this would just skip the mysql tests if they weren't there - but yeah, that's a concern of mine
19:16:06 <mtaylor> problem is too - we'd be adding python-mysqldb to the pip-requires
19:16:09 <mtaylor> to make that work
19:16:17 <jeblair> or the test-requires
19:16:19 <sdague> yes, that would need to change
19:16:24 <mtaylor> which means you'd have to have a working libmysqlclient on the machine
19:16:44 <mtaylor> because the virtualenv build is going to have to be able to build python-mysqldb
19:17:06 <mtaylor> not saying no - just saying it does raise the number of things required to be a  devs box
19:17:15 <mtaylor> now they have to have mysql libs even if they're not doing anthing with mysql
19:17:25 <jeblair> should this sort of test be in tempest?
19:17:33 <mtaylor> can't really be in tempest
19:17:41 <mtaylor> tempest only does black-box api level testing
19:17:58 <jeblair> ok
19:18:03 <mtaylor> and although we could add an exercise.sh script in devstack to look at the db
19:18:09 <mtaylor> that's not going to run through all of the db migrations
19:18:26 <mtaylor> so this is REALLY about testing that the db migration code works across mutliple database backends
19:18:51 <jeblair> it does seem to fit into the unit test space fairly well.
19:18:53 <mtaylor> since testing that the service _works_ across multiple backends could be done by configuring devstack to use different databases
19:18:58 <mtaylor> yeah. it's just ugly :)
19:19:17 <sdague> I guess if with think that having mysql libs is too high a bar, can we add a "--livedb" or some other flag to run_tests.sh that adds things to the venv?
19:19:31 <mtaylor> sdague: I want to make run_tests.sh go away
19:19:40 <mtaylor> sdague: but
19:19:43 * LinuxJedi wonders why there is no native Python client library (ie. not dependant on libmysqlclient)
19:19:52 <mtaylor> LinuxJedi: actually - good point, there is
19:20:08 <LinuxJedi> isn't that what Geert does now?
19:20:33 <LinuxJedi> it would get rid of a few headaches at least anyway
19:20:44 <jeblair> assuming it works with sqlalchemy
19:20:51 <mtaylor> it does
19:20:56 <mtaylor> but it's not on pypi
19:20:57 <LinuxJedi> jeblair: we'll make it work! ;)
19:20:58 <mtaylor> fail
19:21:03 <LinuxJedi> :(
19:21:04 <jeblair> anyone remember what it's called?
19:21:08 <sdague> LinuxJedi: though we'd also want to solve it for postgresql as well
19:21:14 <jeblair> that's true
19:21:30 <Shrews> MySQL Connector/Python
19:21:35 <mtaylor> we could add tox targets, but that's ugly
19:22:02 <LinuxJedi> jeblair: https://launchpad.net/myconnpy
19:22:21 <mtaylor> I was wrong
19:22:23 <Shrews> mtaylor: http://geert.vanderkelen.org/post/825/
19:22:24 <mtaylor> thanks Shrews
19:22:27 <mtaylor> http://pypi.python.org/pypi/MySQL-Connector/
19:22:35 <jeblair> py-postgresql is pure python, but only python 3.0+.  we'll have to move openstack to python3.
19:22:42 <mtaylor> jeblair: hahaha. yeah.
19:22:46 <sdague> heh
19:22:58 <LinuxJedi> jeblair: mtaylor, Shrews and I have all worked with Geert so if it is broke we can kick him ;)
19:24:21 <mtaylor> so - let's take this one bit at a time ...
19:24:40 <jeblair> mtaylor: we could have a third -requires file; optional-test-requires
19:24:48 <jeblair> or test-optionals.  :)
19:25:01 <mtaylor> if we get something working with mysql-connector that skips tests if mysql isn't there and works consistently
19:25:10 <mtaylor> we'll at least be _somewhere_
19:25:15 <sdague> mtaylor: agreed
19:25:17 <jeblair> yes, good first step
19:25:20 <LinuxJedi> ++
19:25:22 <sdague> I'm happy to go tackle that
19:25:34 <mtaylor> sweet.
19:26:05 <mtaylor> sdague: if you get that working locally, and then send us info on what the needs are in terms of having the right mysql resource on the box, we can work on getting an available mysql laying around
19:26:13 <sdague> sure will do
19:26:21 <jeblair> i can do the puppet config for that
19:26:45 <mtaylor> cool
19:27:30 <jeblair> i'll wait for sdague to send me specs
19:27:42 * LinuxJedi smokes a cigar and says "I love it when a plan comes together"
19:27:45 <sdague> as I start to get something you want to see things go through gerrit, or start with something like a github branch ?
19:28:24 <mtaylor> well, if you get jeblair the db specs, we can add those to the slaves before the other code is ready
19:28:32 <mtaylor> and at that point, it can just go through gerrit as normal
19:28:34 <mtaylor> hrm
19:28:36 <mtaylor> one thing though
19:28:53 <mtaylor> how do we test that the "skip if it isn't there" works and continue to work?
19:29:22 <mtaylor> because if all of a sudden local tests start failing for people but work on a special jenkins node, that sucks too
19:29:42 <mtaylor> jeblair: it's starting to feel like we just picked up another matrix dimension
19:30:01 <jeblair> _if_ we want to test that, we could have another job without the credentials.
19:30:08 <sdague> yeh
19:30:23 <jeblair> we could also assume it will keep working and that developers will yell and fix it if it breaks locally.
19:30:27 <LinuxJedi> mtaylor: wouldn't the standard python tests test that?
19:30:36 <sdague> actually you don't even need another job
19:30:48 <jeblair> LinuxJedi: i've been operating under the assumption that this would happen in the standard python tests
19:31:00 <mtaylor> sdague: no?
19:31:05 <LinuxJedi> jeblair: ah, ok.  I thought we were just talking Devstack, sorry :)
19:31:08 <sdague> just force a bad pass and test the skip function itself
19:31:51 <mtaylor> sdague: ah, that makes sense - and then assume that skipping tests itself is a feature of nose that we don't have to test
19:31:59 <sdague> I'm going to encapsulate the "skip_mysql" to a function, so that can be tested to return correctly independent of actually mysql tests
19:32:36 <mtaylor> great. ok. I'm on board with that - I don't want to start blowing out crazy matrix combinations until jclouds-plugin is actually live and doing caching :)
19:32:44 <sdague> :)
19:32:51 <jeblair> cool, so to be clear
19:33:11 <jeblair> this will all happen in one run of the unit tests, and there will be a unit test that checks that skipping tests works correctly?
19:33:34 <sdague> yes
19:33:34 <jeblair> (even in an environment where they won't actually be skipped?)
19:33:41 <sdague> correct
19:33:45 <jeblair> groovy
19:34:04 <sdague> ok, now I just have to go and implement it all :)
19:34:20 <mtaylor> jeblair: what we really need is the ability to override and/or inject deps into a tox run
19:34:31 <jeblair> sdague: can you file a bug about this on launchpad in the openstack-ci project?
19:34:39 <sdague> jeblair: yes, will do
19:35:02 <mtaylor> jeblair: which would allow us to remove the jenkins* stanzas, and would also allow a mechanism to have tox --also-require=pypostgres be something a dev could run locally if they wanted to
19:35:39 <mtaylor> jeblair: we could make shrews fix tox for us...
19:35:52 <jeblair> mtaylor: yes; as silly as it is, having a third -requires file still might be a good idea to keep track of such optional test dependencies
19:35:57 <sdague> jeblair: in the openstack project under OpenStack Core Infrastructure, or somewhere else?
19:36:19 <jeblair> https://bugs.launchpad.net/openstack-ci/
19:37:32 <jeblair> mtaylor: then tox --also-requires=tools/optional-test-requires
19:37:59 <mtaylor> jeblair: yeah
19:38:39 <mtaylor> jeblair: and that way we could do our .cache.bundle thing in a way that doesn't have to be encoded into the tox.ini file itself
19:39:17 <jeblair> sounds great
19:40:53 <jeblair> sdague: assuming we knock out mysql as a first step, are you interested in working on similar postgres tests later on?
19:42:28 <sdague> jeblair: probably. Honestly, if I do this right, it should just be a matter of a slightly different skip function to make it work, and the right requires
19:42:47 <sdague> let me get over bridge one first though :)
19:42:52 <jeblair> ya
19:44:34 <mtaylor> jeblair: speaking of test-requires ... should we take the apts files from devstack and put them into the projects themselves? so that it's documented somewhere?
19:45:05 <mtaylor> jeblair: or have I lost my mind?
19:46:24 <jeblair> i think that sounds fine, but it also sounds like a great think to talk about with jesse next week
19:46:31 <mtaylor> jeblair: ++
19:48:13 <mtaylor> okie. anybody else got anything else?
19:48:26 <mtaylor> LinuxJedi, jeblair anything you particularly want to share with the fine folks at home?
19:48:46 <jeblair> i'm hoping we'll be able to move to a new gerrit server and upgrade to gerrit 2.3 this week
19:48:52 <mtaylor> w00t!
19:48:57 <mtaylor> that's the best news I've heard in a long time
19:49:00 * LinuxJedi has been off most of the last week.  But done a bit of work on Jenkins Jobs.  Plus there is a fix for the Gerrit theme
19:49:13 <LinuxJedi> which means we can delete stuff again ;)
19:49:16 <jeblair> i'm testing it now; found some surprises, but so far no showstoppers.
19:49:27 <mtaylor> LinuxJedi: did you get what you needed in terms of those reverted patches?
19:49:29 <jeblair> i used the force to find the delete button
19:49:54 <LinuxJedi> mtaylor: I still can't find the email so I'm just going to start over :)
19:50:23 <LinuxJedi> jeblair: well that one-liner that is pending will reveal them all again
19:50:43 <mtaylor> https://review.openstack.org/#change,6399
19:51:13 <LinuxJedi> that's the one
19:53:48 <mtaylor> k. I approved it. I'll let jeblair cast the final vote
19:55:12 <jeblair> i'll do that as soon as i revert my hosts file.  :)
19:55:17 <mtaylor> hehe
19:55:39 <mtaylor> ok. I thnk that's a full tuesday!
19:55:46 <jeblair> woo
19:55:47 <mtaylor> thanks everybody
19:55:59 <mtaylor> #endmeeting