19:07:00 <mtaylor> #startmeeting CI
19:07:01 <openstack> Meeting started Tue Sep 18 19:07:00 2012 UTC.  The chair is mtaylor. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:07:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:07:03 <openstack> The meeting name has been set to 'ci'
19:07:10 <mtaylor> anybody still around?
19:07:14 <fungi> only mostly
19:07:16 <dkehn> yepper
19:07:19 <clarkb> yup
19:07:29 <mtaylor> whee!
19:07:53 <jeblair> o/
19:07:56 <mtaylor> so - fungi, how's that CLA stuff going?
19:08:18 <fungi> looks like we're ready to merge #link https://review.openstack.org/13058 if jeblair has a chance to look
19:08:21 <jeblair> #topic gerrit/foundation CLA integration
19:08:30 <jeblair> i can't topic, can i?
19:08:32 <mtaylor> #topic gerrit/foundation CLA integration
19:08:34 <mtaylor> I can
19:08:48 <fungi> should allow us to demo it on review-dev.openstack.org while we wait for the foundation to set up a server for the contact store
19:09:01 <fungi> it all works as expected on my test gerrit
19:09:06 <mtaylor> anybody know how that's going on the foundation side?
19:09:08 <mtaylor> reed: ?
19:10:04 <jeblair> i think he said earlier that he had no updates.
19:10:08 <mtaylor> ok. cool
19:10:12 <fungi> he did indeed say that
19:10:20 <mtaylor> well, testing through on our side is still good
19:10:36 * mtaylor doesn't like being blocked on things other than his own laziness
19:12:07 * fungi notices it suddenly got very quiet in here
19:12:07 <mtaylor> but that seems to be in good shape from our end
19:12:10 <mtaylor> fungi: hehe
19:12:18 <mtaylor> fungi: anything else interesting in your world?
19:12:58 <fungi> continuing to read and watch, and also poking at git-patch-id to see what i can do for a --no-whitespace-really-matters option
19:13:12 <fungi> nothing terribly exciting yet
19:14:18 <mtaylor> kk
19:14:43 <mtaylor> clarkb: how about you? anything fun this past week?
19:15:05 <clarkb> I did the things that were stuck to me last tuesday. Bug reports and such.
19:15:14 <clarkb> The most exciting thing is the translation jobs though
19:15:27 <mtaylor> #topic translations
19:16:14 <clarkb> so we got the jobs mostly working on Nova. To get there we needed to copy ssh keys to tx.slave.o.o and give the jenkins transifex user admin privs on transifex for nova
19:16:34 <clarkb> This worked the first time around and updated translations were merged and everything was happy.
19:17:51 <clarkb> But I had to use git review 1.17 which tries to rebase which fails on current runs of the translation proposal job so everything is no longer happy. I need to rewrite the script to use the commit message of existing changes, but work atop master instead of the old patchset
19:18:16 <jeblair> it's not the rebasing, per se, that's the problems --
19:18:40 <clarkb> its that we want to be working off the latest code but don't when an existing patchset is on gerrit
19:18:55 <clarkb> the rebasing pointed this problem out to us though
19:19:00 <jeblair> exactly
19:19:25 <clarkb> so Nova is almost there. Horizon is a different story
19:19:39 <jeblair> when a rebase is necessary and there's a conflict, there's no human to fix it, so it semes like the better approach is just sort of start over each time.
19:19:49 <mtaylor> ++
19:20:06 <mtaylor> I'm curious about this django translations setup that's superior
19:21:02 <clarkb> so ya Horizon. I attempted to setup Horizon like Nova and pushed a change to Horizon in Gerrit to make it fit in and GabrielHurley said no because django does things differently and better
19:21:26 <clarkb> so I have removed the Horizon translation jobs from Jenkins and abandoned that change that would make it work.
19:21:36 <mtaylor> ok. that's so great
19:21:47 <clarkb> Horizon is special because it is a django project and django comes with its own translation tools.
19:21:53 <mtaylor> what is it about how django does things that's better? and can we learn something from it?
19:21:56 <mtaylor> or is it just different
19:22:06 <mtaylor> and integrated into django land in some way that's good for django people
19:22:07 <clarkb> I think the better part is it uses the gnu gettext tools
19:22:20 <mtaylor> doesn't babel also use the gnu gettext tools?
19:22:21 <clarkb> and it uses them in a way that fits into django
19:22:33 <clarkb> no, I believe babel is a from scratch implementation
19:22:36 <mtaylor> awesome
19:22:50 <mtaylor> should we ditch babel and use gnu gettext tools?
19:23:01 <clarkb> one major difference is django uses multiple translation domains
19:23:09 <clarkb> there is a django domain and a djangojs domain
19:23:25 <mtaylor> do all of the strings go into the django domain?
19:23:32 <mtaylor> so, like, there isn't a horizon domain?
19:23:39 <mtaylor> weird...
19:23:42 <clarkb> mtaylor: that is how it is currently setup
19:24:12 <clarkb> django has other weirdness like not supporting translations for languages that the django core does not support and so on
19:24:19 <mtaylor> well, that would certainly allow django code to find things
19:24:33 <clarkb> so I can see where GabrielHurley is coming from when saying that it is better to use the django tools
19:24:37 <mtaylor> yeah
19:25:06 <mtaylor> so you were suggesting we add a set of tox calls to do things so that we can have CI things work the same everywhere?
19:25:35 <clarkb> that would be one way to get around the problem. It might also be possible to switch to gnu gettext for everything and mimic the django tools when working with Horizon
19:26:20 <mtaylor> hrm
19:26:42 <mtaylor> we could also stop thinking of horizon as being python language
19:26:49 <mtaylor> and think of it as a django language thing
19:27:13 <clarkb> GabrielHurley and I should probably get together at the summit and get a better idea of what their needs are
19:27:39 <mtaylor> ++
19:27:50 <mtaylor> #action clarkb solve django with GabrielHurley
19:28:21 <mtaylor> clarkb: anything else there?
19:28:27 <clarkb> keystone is configured like nova so in general I think the current setup should work
19:28:31 <clarkb> and that is it
19:28:35 <mtaylor> cool
19:29:03 <mtaylor> jeblair: how's bcn?
19:29:28 <jeblair> it might rain.  apparently.  we'll see.
19:30:08 <mtaylor> neat
19:30:17 <jeblair> mtaylor: #topic jenkins job builder
19:30:24 <mtaylor> #topic jenkins job builder
19:30:30 <jeblair> #link http://ci.openstack.org/jenkins-job-builder/
19:30:48 <mtaylor> that's really nice
19:30:58 <jeblair> that was a lot of typing.
19:31:04 <jeblair> that's most of what i've been working on
19:31:20 <jeblair> there's a bit of sphinx magic to make it easy to use autodoc
19:31:35 * mtaylor likes the yaml autodoc sphinx stuff
19:31:40 <jeblair> so whenever we add a new component (builder, publisher, whatever), we can just write docstrings
19:31:44 <jeblair> and it'll show up
19:32:23 <jeblair> clark and i should have several more gating jobs for it now
19:32:31 <jeblair> pep8, pyflakes, and docs
19:32:51 <clarkb> jeblair: might be a good idea to add a link to http://ci.openstack.org/jenkins-job-builder/ under http://ci.openstack.org/jenkins_jobs.html
19:32:52 <jeblair> it doesn't have any unit tests, (though it does have the XML comparison test, which is more important)
19:33:03 <jeblair> clarkb: yes
19:33:26 <mtaylor> btw - there are a set of helper utilities in testtools that lifeless is going to split out into their own library that would help us use pyflakes in more places
19:33:38 <mtaylor> (the pyflakes ref above made me think of it)
19:33:40 <jeblair> cool
19:33:55 <mtaylor> specifically, a thing that does the try: import foo except: foo = None thing that pyflakes really hates
19:34:25 <jeblair> so i think jenkins job builder is basically up to standards at this point
19:34:39 <jeblair> and we've been getting contributions from internal HP users too, which is nice
19:34:43 <mtaylor> yeah. I'm pretty pleased about that
19:35:06 <jeblair> mtaylor: there was a tempest disucssion recently where people were disatisfied with nose
19:35:28 <jeblair> mtaylor: maybe testtools would be of interest to them
19:35:42 <mtaylor> jeblair: ++
19:35:43 <jeblair> jaypipes: ^ know about testtools?
19:35:44 <mtaylor> jaypipes: ^^^
19:35:47 <mtaylor> :)
19:36:18 <mtaylor> jaypipes: there's a couple of things that lifeless has been working on that might be handy: testtools and testrepository
19:36:19 <jaypipes> jeblair: yes, mtaylor showed me testtools and fixtures about a month ago
19:36:23 <mtaylor> cool
19:37:27 <clarkb> is nose too unittesty?
19:37:28 <mtaylor> jaypipes: I'm going to make an ODS session about some of that ... hopefully today
19:37:33 <mtaylor> nose is not unittesty enough
19:37:40 <mtaylor> it injects evil into the things it runs
19:38:17 <mtaylor> #action mtaylor submit ODS session about testtools/testrepository
19:38:23 <jeblair> one other thing: i've made a "dev" branch of zuul, since we're trying to keep ci in a soft freeze.
19:39:16 <jeblair> someone pointed out that it depended on our apache git mirror.  i fixed that in the dev branch.  if i knew who that user was, i'd love to tell him.  :)
19:39:59 <jeblair> that's about it for me.
19:40:36 <mtaylor> oh yeah, I remember that
19:41:37 <clarkb> I should probably mention jclouds
19:42:22 <clarkb> I have jclouds working on jenkins-dev with hp cloud and rackspace
19:42:40 <mtaylor> jeblair: do we have a time in mind for when soft freeze is over?
19:42:43 <mtaylor> #topic jclouds
19:43:26 <jeblair> mtaylor: i imagine after the release ?
19:43:51 <mtaylor> good topic thought clarkb
19:44:04 <clarkb> rackspace does not configure the base host in a way that the latest facter version can return an fqdn (I have complained about this and it looks like the current version of facter on github fixes part of the problem).
19:44:16 <mtaylor> wow
19:44:22 <mtaylor> that's really special
19:45:05 <clarkb> I thought so too :) to get around that I am simply doing `echo "domain localdomain" >> /etc/resolv.conf` before running puppet
19:45:36 <clarkb> you can't reassign the value of a variable in another scope so I can't do $::fqdn = $hostname in the puppet expression
19:46:05 <clarkb> this was all necessary because puppetlabs-mysql uses the ::fqdn value
19:46:37 <clarkb> but that issue is sorted and jclouds can create oneiric and precise hosts on hpcloud and rackspace now.
19:46:43 <mtaylor> awesome
19:47:02 <mtaylor> but it still has enough issues with things like load balancing and creating that we still want some static hosts, yeah?
19:47:13 <clarkb> yes
19:47:51 <clarkb> it doesn't load balance. It uses provider A until it hits the threshold you set for the number of VMs then switches to provider B
19:47:56 <jeblair> yeah, i'm thinking the long view is maybe we drop down to about enough static hosts to run one set of jobs
19:48:05 <jeblair> and then use this to burst
19:48:42 <clarkb> and I think in some cases it just gives up with jobs waiting in the queue. I haven't been able to nail that down yet though
19:48:45 <jeblair> in practice, i expect it will ramp up in the US morning, and the dynamic hosts will stay around most of the day
19:49:14 <jeblair> clarkb: have you seen anything that would cause a job to fail?
19:49:43 <clarkb> jeblair: I am beginning to think I may have been hitting thesholds on the hpcloud side that didn't show up in the jenkins logs
19:49:54 <jeblair> (inefficiency we can tolerate, but actually failing jobs is bad)
19:50:03 <mtaylor> jeblair: ++
19:50:03 <clarkb> oh, no it doesn't fail jobs that do run
19:50:13 <jeblair> ok
19:50:15 <clarkb> at least not the jobs i was running (nova unittests)
19:50:52 <mtaylor> I'm game to try it again
19:50:52 <clarkb> so I think we can try using this again in production when ttx relaxes on the change freeze
19:50:57 <mtaylor> yeah
19:51:23 <clarkb> I do think we want to put a different label on the nodes and do something like node: precise || hpcloud-precise || rackspace-precise in the jenkins job defs
19:51:27 <mtaylor> in the mean time - should we put the main config.xml into puppet with hiera values for the secrets?
19:51:43 <jeblair> clarkb: why?
19:52:03 <clarkb> jeblair: I think that will make it easier to control where things are run
19:52:17 <clarkb> we could optionally stop using hpcloud or rackspace for particular jobs
19:52:34 <jeblair> ok.  i hope we don't need to use that.  :)
19:52:49 <jeblair> nice to have knobs and levers though.
19:53:14 <jeblair> +1 config.xml in git
19:53:22 <clarkb> mtaylor: are we managing the global jenkins config in puppet yet?
19:53:27 <mtaylor> we are not
19:53:53 <clarkb> adding secrets to hiera and managing the config through puppet would be awesome
19:53:54 <mtaylor> and for a while I was thinking we should yaml/job-builder it - but I think jeblair has convinced me that it changes infrequently enough that just storing the xml is ok ...
19:53:56 <mtaylor> oh, wait
19:53:58 <mtaylor> I remember now
19:54:05 <mtaylor> the global config xml is going to suck
19:54:13 <mtaylor> because slaves get stored into it
19:54:20 <jeblair> oh, right
19:54:30 <jeblair> we, uh, change slaves often.
19:54:36 <clarkb> might need puppet parsed file provider
19:54:39 <mtaylor> so we'll need to write a tool that will store _most_ of the global config.xml in puppet and then merge them
19:54:41 <mtaylor> yeah
19:55:15 <mtaylor> clarkb, fungi: one of you guys have the bandwidth to think about that? or perhaps you can each thing about different parts of it?
19:55:20 <mtaylor> or just tell me to go shove it
19:55:42 <fungi> i can look into it
19:55:45 <clarkb> I may have bandwidth in a day or two. need to dig out from under translations
19:56:02 <clarkb> fungi: be warned ruby is probably involved
19:56:11 <fungi> i'll need to be pointed to some more background, but it doesn't sound too dreadful
19:56:20 * mtaylor hits fungi in the face with ruby
19:56:24 <fungi> ruby is tolerable
19:56:35 <fungi> just don't slap me around with c#
19:56:38 <clarkb> hey everyone! give fungi the ruby stuff :)
19:56:50 <mtaylor> #agreed fungi is the new ruby guy
19:57:02 * fungi sighs
19:57:08 <mtaylor> and with that ...
19:57:13 <mtaylor> I believe we are at time
19:58:27 <mtaylor> #endmeeting