19:01:54 <mtaylor> #startmeeting
19:01:55 <openstack> Meeting started Tue Aug 16 19:01:54 2011 UTC.  The chair is mtaylor. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:56 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:02:18 <mtaylor> #topic Actions from last meeting
19:02:30 <mtaylor> #action jaypipes design upgrade path jenkins job with mtaylor
19:02:37 <mtaylor> (that's my way of saying we still haven't done that)
19:02:53 <mtaylor> jeblair did do an auto-closing pull request hook though
19:03:03 <mtaylor> jeblair: is that deployed now?
19:03:03 <jeblair> i did...
19:03:20 <jeblair> here it is: https://github.com/openstack/openstack-ci/blob/master/gerrit/close_pull_requests.py
19:03:35 <jeblair> and yes, it's deployed.  it should be running in production for keystone and glance
19:03:43 <mtaylor> #link  https://github.com/openstack/openstack-ci/blob/master/gerrit/close_pull_requests.py
19:03:46 <mtaylor> sweet
19:03:53 <jeblair> it runs out of cron every 5 mins
19:04:11 <mtaylor> that's all from last week then
19:04:21 <mtaylor> #topic Discuss automated testing and configuration coverage
19:04:27 <jeblair> someone last week volunteered to take that and see about adding support for creating gerrit changes
19:04:39 <mtaylor> #link http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-08-15-20.07.log.html
19:04:42 <mtaylor> oh yeah - who was that
19:04:45 <mtaylor> anybody remember?
19:04:53 <mtaylor> heckj: was that you?
19:05:27 <jeblair> 19:44:36 <_0x44> vi, and bengrue is volunteering to write the second-round hook to auto-integrate gerrit and github.
19:05:29 <heckj> mtaylor: sorry, no
19:05:44 <mtaylor> bengrue. that's him
19:06:03 <mtaylor> #action bengrue write the second-round gerrit github hook
19:06:09 <mtaylor> how's that? :)
19:06:14 <jeblair> :)
19:06:29 <jaypipes> oh, cool, I got an action item...
19:06:35 <mtaylor> so - soren, jaypipes and deshantm: what are we supposed to pow-wow about?
19:06:44 <jeblair> future bengrue, if you're reading this, let me know if you need anything. :)
19:06:46 <soren> gimme a sec, catching up.
19:07:03 <soren> caught up.
19:07:05 * jaypipes reading back..
19:07:31 <deshantm> I think most of the things have been discussed. Xen.org/Citrix is here to help with automation/testing
19:07:39 <mtaylor> sweet. we're in favor of that
19:07:41 <deshantm> working thing into our workflow as needed
19:07:43 <soren> mtaylor: Is that the action item from yesterday?
19:07:49 <mtaylor> yeah
19:07:53 <soren> Ok.
19:08:00 <jaypipes> deshantm: have you and mtaylor chatted about how to set up a jenkins slave?
19:08:15 <deshantm> jaypipes: not yet
19:08:23 <mtaylor> we theoretically have a citrix slave, actually :)
19:08:25 <deshantm> we run jenkins at Citrix and Xen.org
19:08:30 <mtaylor> although it's been offline for a bit
19:08:55 <jaypipes> deshantm: yup, understood. we'd like to have our master jenkins box trigger builds on your local jenkins.
19:08:55 <soren> mtaylor: I must admit, I'm not completely sure. I wasn't following the meeting, my name was just mentioned and I chimed in, but I didn't completely get the context.
19:09:09 <deshantm> I have limited experience, but have created a job successfully
19:09:17 <mtaylor> jaypipes: just to be clear (and because people keep saying similar words) ...
19:09:21 <deshantm> jaypipes that neat
19:09:25 <jaypipes> deshantm: no worries, mtaylor can help explain that process.
19:09:31 <mtaylor> we're actually NOT interested in having our master jenkins trigger jobs on other people's jenkins
19:09:34 <mtaylor> because that doesn't work
19:09:41 <jaypipes> Bus, meet mtaylor. :P
19:09:45 <mtaylor> we're interested in having people provide jenkins slaves, and running jobs there
19:09:47 <jaypipes> mtaylor: no?
19:09:54 <mtaylor> jaypipes: no. it is not an option
19:10:14 <jaypipes> mtaylor: hmm, that's odd. I thought that was the whole purpose of having a slave builder?
19:10:21 <deshantm> mtaylor and I can take the details of the discussion offline though
19:10:22 <mtaylor> jaypipes: yes. a slave builder
19:10:33 <soren> jaypipes: slave builders are "dumb".
19:10:36 <deshantm> is there anything at a high level that we need to discuss though?
19:10:37 <jaypipes> mtaylor: sorry, that's what I was referring to..
19:10:39 <mtaylor> jaypipes: but a slave builder is not another jenkins ... it's a slave hanging off our our jenkins
19:10:44 <deshantm> I'm dropping off in 5
19:10:45 <soren> jaypipes: They're not full Jenkins installs, just workers.
19:10:50 <jaypipes> mtaylor: sorry, got my semantics wrong...
19:10:53 <jaypipes> soren: got it.
19:10:54 <mtaylor> jaypipes: I figured - I just wanted to be clear here because some folks have been getting confused :)
19:11:05 <jaypipes> mtaylor: appreciate that :)
19:11:06 * soren included
19:11:08 <soren> :)
19:11:17 <mtaylor> deshantm: cool - well let's definitely talk offline about getting some useful jobs set up on your slave
19:11:35 * jaypipes replaces slave with builder in his mind...
19:11:38 <mtaylor> deshantm: I'm also working with alandman and primeminiterp from msft/novell with their slave
19:11:58 <mtaylor> deshantm: so hopefully we can put together a generic/reusable process here
19:14:04 <deshantm> ok sounds good. we may already be testing useful things that can help with automated testing for things like XenAPI support
19:14:13 <mtaylor> I'd love that
19:14:16 <deshantm> I'll have to check
19:14:27 <mtaylor> #action dshantm and mtaylor will talk jenkins slaves
19:14:44 <deshantm> the end goal is to fill in that hypervisor support matrix automagically
19:14:59 <mtaylor> ooh - that _would_ be magical
19:15:04 <soren> Yeah. I'm working on that, too.
19:15:07 <soren> Differently, though.
19:15:16 <deshantm> ok, i'm off
19:15:19 <deshantm> thanks guys
19:15:20 <deshantm> take care
19:15:28 <jeblair> thanks!
19:15:35 <soren> So, I have a stack of tests that exercise all the various things hypervisor drivers are supposed to do.
19:15:38 <soren> (unit tests)
19:16:01 <soren> Whenever a NotImplementedError is raised, I log that.
19:16:21 <soren> ...so when this massive clean-up task is done, that should provide reliable information to fill in such a matrix.
19:16:40 <soren> ...and then it's just a matter of extracting the info from these logs and turn it into beautiful html.
19:16:50 <mtaylor> mmmm
19:16:59 * mtaylor thinks that sounds wonderful
19:17:10 <soren> It's coming along reasonably well.
19:17:11 <mtaylor> soren: this may sound like an odd question, but ...
19:17:14 <soren> Shoot.
19:17:30 <mtaylor> soren: do you actually have information on installed cluster setup prereqs for running those tests?
19:17:43 <soren> 19:15 < soren> (unit tests)
19:17:51 <mtaylor> ah
19:17:54 <soren> :)
19:17:57 <mtaylor> sorry - missed that
19:18:04 <mtaylor> I was about to get wet
19:18:20 <mtaylor> and on that note ...
19:18:29 <mtaylor> #topic Open Discussion
19:18:52 <mtaylor> anybody gots anythings?
19:19:05 <soren> I can go into a bit more detail about this stuff I'm doing if anyone wants it?
19:19:30 <soren> ..but maybe it's more suitable for an ml post.
19:19:39 <bengrue> (sorry about being late; this meeting is right at lunchtime here.)
19:19:49 <bengrue> Anything new of note?
19:19:53 <mtaylor> soren: I'm happy to listen - but also happy to read mailing list post
19:20:01 <mtaylor> bengrue: there's an action with your name on it :)
19:20:07 <bengrue> oh?
19:20:28 <bengrue> url?
19:20:28 <mtaylor> bengrue: jeblair made the auto-closing pull-request hook thing - so we made you an action for poking at the create-a-gerrit-review bit
19:20:46 <bengrue> where is gerrit installed
19:20:46 <bengrue> ?
19:21:17 <mtaylor> it'll be in the meeting notes. gerrit is installed at review.openstack.org ... the hook is at https://github.com/openstack/openstack-ci/blob/master/gerrit/close_pull_requests.py
19:21:24 <jeblair> we have a dev box: review-dev.openstack.org
19:21:43 <bengrue> will I need an account on the dev box?
19:21:49 <jeblair> nope
19:22:06 <jeblair> you can do everything you need to externally
19:22:28 <jeblair> its has a test project set up on it
19:22:32 <bengrue> is the dev box auto-updated on push?
19:22:37 <jeblair> that's linked to https://github.com/gtest-org/test
19:23:04 <jeblair> so what you might do is send pull requests to that project, and then have your script check for them and create gerrit changes
19:23:25 <jeblair> you can submit the changes as your own gerrit user (it syncs with launchpad same as production)
19:24:07 <jeblair> and yes, both dev and production run the latest code out of the github repo
19:24:18 <jeblair> so when you're done, and propose the change in gerrit, when it's approved, it'll go into production
19:24:29 <bengrue> Okay.
19:24:40 <jeblair> lemme know if you need anything
19:24:44 <bengrue> Other than that, were any decisions made/discussed today?
19:24:47 <bengrue> Sure thing.
19:25:43 <mtaylor> not really
19:28:28 <mtaylor> jeblair: can you think of anything else we should talk with folks about?
19:29:27 <jeblair> nope
19:34:15 <bengrue> So, I have some general questions about CI workflow and openstack.
19:34:43 <mtaylor> great! bring em on
19:34:57 <bengrue> What tests are the set of acceptance tests for something to be put into trunk?
19:35:24 <bengrue> acceptance tests being the set of tests that are required to be green.
19:35:52 <mtaylor> currently, it's the unit tests in the tree
19:36:08 <bengrue> Because I've been having issues with the smoketests on my dev box, and they seem to be the only ones that actually integrate various subsystems.
19:36:11 <bengrue> I see.
19:36:14 <jaypipes> bengrue: for glance, it is the set of unit and functional tests inside glance/tests/
19:36:16 <mtaylor> that is correct
19:36:28 <mtaylor> we're currently working on getting smoketests added to the list of things that are acceptance tests
19:36:39 <bengrue> So, are there any full integrstion tests between nova/glance/stance?
19:36:42 <bengrue> ...
19:36:46 <bengrue> swift.
19:36:48 <bengrue> stance, wtf.
19:37:18 <mtaylor> hehe. stance
19:38:10 <mtaylor> bengrue: also currently in work
19:38:11 <bengrue> Also, what's the level of code coverage?  I've been removing whole functions from my dev and the unit tests have still been passing.
19:38:56 <bengrue> Which has been frustrating, because I was removing the functions to see the set of tests that went red to learn about said functions.
19:39:10 <mtaylor> bengrue: https://jenkins.openstack.org/view/Nova/job/nova-coverage/
19:39:24 <jaypipes> bengrue: where are you putting your code?
19:39:35 <mtaylor> bengrue: is is also an open task to get the code coverage jobs to block coverage decreasing
19:40:20 <bengrue> where am I putting my code?  My code is currently local, I didn't commit this stuff.
19:41:12 <dprince> I'm not the biggest fan of blocking commits if code coverage decreases.
19:41:15 <bengrue> The one that comes to mind is the iscsi discovery code.  I made it return at the top with a false to see what tests were covering it.
19:41:19 <bengrue> (none were)
19:41:24 <jaypipes> bengrue: ok. there are a number of projects on github that are trying to address integrated testing: https://github.com/cloudbuilders/kong and https://github.com/rackspace-titan/stacktester/
19:42:04 <bengrue> I'm not a fan of code coverage as a hard metric either, but I was surprised that such systems didnt seem to have tests at all, so I was hoping to get a high level picture of how safe I should feel because of the tests.
19:42:07 <dprince> jaypipes: Smokestack runs stacktester now.
19:42:20 <dprince> Also. Mtaylor. I can make nova-vpc run it too if your interested.
19:42:27 <jaypipes> dprince: figured as much :)
19:42:28 <mtaylor> dprince: please do
19:42:28 <bengrue> I'm working at integrating kong with piston's workflow right now.
19:42:39 <dprince> Will take a bit of work to make those tests pass for libvirt though.
19:42:41 <mtaylor> dprince: blocking based on decreasing coverage was a request from NTT from last ODS
19:42:45 <jaypipes> bengrue: could you elaborate on what Piston's workflow is?
19:43:05 <dprince> Yeah. I was at that session during the conference. Just saying it worries me.
19:43:36 <dprince> Seems like it motivates people to write tests for coverage which wouldn't always be good tests that we need.
19:43:55 <bengrue> right now, jay, it's being invented!
19:44:01 <jaypipes> dprince: soren is working on un-f-king the unit tests and removing the excessive use of stubs...
19:44:13 <dprince> great!
19:44:40 <dprince> Still. I hate to block a commit on code coverage as a metric.
19:44:51 <jaypipes> bengrue: OK, well I want to make sure that we all work together and that all these new tests get brought under a single umbrella project that everyone can benefit from :) too little communication going on up until now, so there's a myriad projects duplicating efforts..
19:45:35 <heckj> jaypipes: a thousand flowers blooming?
19:45:42 <bengrue> I can elaborate more in a few days; I'm currently working on a CI project internally so we can work on our proprietary stuff alongside the open stuff.
19:45:47 <jaypipes> dprince: I sent an email to Gabe this morning about trying to get Kong and stacktester merged and everyone working on the same stuff. Sorry, didn't include you because it was more high-level discussion trying to get everyone on same page first...
19:45:57 <jaypipes> heckj: indeed, my friend :)
19:47:44 <mtaylor> wait - there are flowers?
19:48:01 <dprince> jaypipes: Sure. As far as test suites go I'm Okay with there being multiple options out there. I sort of see them as weapons. If they work well at finding bugs people will use them. Right now I'm running 3 suites: Ruby Openstack Compute v1.0, Nova Smoke Tests, StackTester v1.1
19:48:33 <dprince> So essentially all 3 API/versions are getting some coverage on every commit and/or branch we run in SmokeStack.
19:48:43 <dprince> Haven't heard of Kong but I'll take a look.
19:50:01 <dprince> I appears to be 5 days old. (Initial release).
19:51:16 <soren> ?!? I thought we adjourned this meeting 20 minute ago?
19:51:18 <soren> Doh.
19:51:38 <mtaylor> soren: whatever gave you that idea?
19:51:49 <dprince> Sorry. I was late. And people were still talking. So...
19:53:25 <dprince> mtaylor: On another topic. The nova tarmac job is getting hung every couple days.
19:53:59 <dprince> If I see its been running for more than 10 hours in the morning I kill it, a new one starts, and code gets merged again.
19:54:18 <mtaylor> dprince: yeah - I've been seeing that. have I mentioned how much I can wait to move from tarmac to gerrit so that we can actually see _where_ it's getting hung?
19:54:35 <mtaylor> s/can/can't/
19:54:42 <dprince> Sure. Just trying to keep things moving along until we make the switch.
19:55:17 <soren> mtaylor: the 5 minutes of silence, perhaps? :)
19:55:32 <mtaylor> dprince: thank you very much! if you kill the job, probably looking at the console output of the job for the last branch attempted to merge and setting that to work in progress with a note about hanging would prevent the re-hang (I would hope)
19:56:13 <dprince> I've done that a couple times. Sometimes the branch it hangs on appears to already have merged though. Its wierd.
19:56:24 <mtaylor> very weird
19:56:32 <mtaylor> I will very much look forward to tracking that down better
19:57:09 <dprince> Also. I've been gone.... Last week or a couple of weeks ago I saw you made a comment about the nova-vpc job being red a lot. The test_004_metadata test had some issues (I think those have been fixed now).
19:57:39 <dprince> Anyway. The job is red today... because of another nova bug. So I think it is working pretty well.
19:57:58 <dprince> The nova smoke tests still need a bit of tuning IMHO.
19:58:24 <mtaylor> ++
19:58:40 <ttx> 2min left
19:58:52 <mtaylor> dprince: well, the question is - once we can do this (post gerrit) - do you think that nova-vpc is solid enough to add to the pre-merge blockers?
19:58:58 <dprince> We may want to wait a bit longer for an instance to boot. Adding an option to increase the instance boot timeout would do the trick. But its getting to the point where I would consider gating commits on a job like this possible.
19:59:02 <mtaylor> dprince: or do you think it shoudl stay post-merge
19:59:07 <dprince> Its close.
19:59:11 <mtaylor> dprince: cool
19:59:26 <mtaylor> well - we're at least 3 weeks out from being able to make that choice anyway - so sweet
20:00:02 <mtaylor> alright - I think we're out of time here
20:00:09 <mtaylor> thanks everybody!
20:00:11 <mtaylor> #endmeeting