19:02:42 <jeblair> #startmeeting ci
19:02:43 <openstack> Meeting started Tue Nov 20 19:02:42 2012 UTC.  The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:02:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:02:45 <openstack> The meeting name has been set to 'ci'
19:02:57 <jeblair> #topic actions from last meeting
19:03:18 <jeblair> fungi: it seems like progress with the foundation server, care to give an update?
19:03:23 <fungi> sure
19:03:52 <fungi> toddmorey has set up a dummy interface on the foundation server and we've repointed review-dev at that now
19:04:22 <fungi> he's requested saner error handling for gerrit's remote http calls, so i'm trying to dissect the code paths those take
19:04:50 <jeblair> fungi: we can move forward with your "static string update" patch, right?
19:04:57 <fungi> though i'm not really savvy with java, so i'm about to the point of bringing it up on #gerrit
19:04:59 <fungi> yes
19:05:00 <jeblair> fungi: (I don't want the perfect solution to hold us up)
19:05:03 <fungi> that'll be a temporary workaround
19:05:31 <jeblair> cool, so assuming "making error handling nicer is an out-of-band project", what's next?
19:05:56 <fungi> he's still working on integration on the foundation side of things
19:06:14 <fungi> i've got patches more or less ready for review.o.o once we're ready topull the trigger
19:06:24 <fungi> just need a couple of trivial updates and a revbase
19:06:26 <fungi> er, rebase
19:06:49 <fungi> we've got a new official keypair generated and implemented for the contact info encryption
19:07:03 <fungi> that's more or less it on that topic unless there are questions
19:07:39 <jeblair> fungi: do you have a feeling for when this might be ready?
19:08:19 <fungi> it's entirely dependent on when the remaining pieces come together on the foundation end
19:08:30 <jeblair> and toddmorey isn't here.  :(
19:08:35 <fungi> hopefully no more than another week or so, but not under my control
19:08:47 <jeblair> so before we roll this out in production, we need a few more things
19:09:15 <clarkb> but it sounds like things are rolling on the other end which is ++
19:09:16 <jeblair> doc updates to the wiki for new contributors (and old, who will be required to re-agree)
19:09:33 <jeblair> if we want to stop running the sync script at the same time, we'll also need:
19:09:53 <jeblair> doc updates to tell people how to manage the core and drivers groups in gerrit
19:09:59 <jeblair> acl changes to allow self-management of those groups
19:10:20 <jeblair> and changes to the bug/blueprint scripts to not assume lp/gerrit username equality
19:10:46 <jeblair> and a nice announcement explaining all this and scheduling a time for the change
19:10:56 <jeblair> none of those things are very hard, i just wanted to put them out there.  :)
19:11:28 <jeblair> personally, i'd like to stop running the sync script when we make this change
19:11:33 <fungi> agreed. i think we really ought to be 100% positive review-dev is working the way we want before we announce/schedule however
19:11:53 <jeblair> any differing opinions on ending the sync script?
19:12:08 <jeblair> the main downside i see is that there are still a few groups that have overlap in LP and gerrit...
19:12:20 <clarkb> jeblair: stopping the sync script is a big win. no opposition here
19:12:21 <jeblair> the release groups, and -drivers groups...
19:12:39 <fungi> doesn't the sync script have some bearing on openid fixups though?
19:12:46 <jeblair> but i think the (distributed!) task of dual-maintenance of those groups is small, esp compared to the costs of running the sync script.
19:12:59 <jeblair> fungi: with group self-management, we don't really care anymore.
19:13:29 <jeblair> we only care that the openids match users so that we can make sure the right gerrit users are in the -core groups, which at the moment, are managed in lp
19:13:45 <jeblair> by making the group management happen in gerrit, that goes away...
19:14:00 <jeblair> -core members just have to add the correct gerrit user to their -core gerrit group
19:14:20 <jeblair> (which is no more difficult than identifying the correct lp user, really, possibly less)
19:14:59 <jeblair> fungi: answer your question?
19:15:12 <fungi> okay, so the lp user/multiple openid situation isn't really at issue as long as groups aren't also managed via lp. got it
19:15:45 <fungi> and yes, my lack of timely responses is due only to the tin cans and string over which my internet connection is travelling at the moment
19:15:56 <fungi> i'm good with it
19:16:04 <jeblair> yeah. it also affects the bug/blueprint scripts, but we can have them look up gerrit-username->openid->launchpad-person mapping when they go to do an update.
19:16:42 <jeblair> (they'll need gerrit database read access, but i don't think that's a problem)
19:17:33 <jeblair> #action toddmorey continue to integrate gerrit contact store with foundation server
19:18:29 <jeblair> anyone want to (a) write docs on the transition (b) update bug/blueprint scripts?
19:19:18 <fungi> put me down for those
19:19:38 <fungi> i haven't looked at the bug/bp scripts, but those can't be too complicated right? ;)
19:19:41 <jeblair> #action fungi write docs on CLA/group changes
19:19:50 <clarkb> fungi: the lp stuff is pretty straightforward
19:19:58 <clarkb> (bugs, blueprints, etc)
19:20:01 <fungi> k
19:20:09 <jeblair> #action fungi update bug/blueprint scripts to use openids to look up lp usernames
19:20:25 <jeblair> fungi: i can help out with that too
19:20:32 <fungi> jeblair: appreciated
19:20:36 <clarkb> maybe put the wiki changes in an etherpad
19:20:52 <fungi> clarkb: yeah, i figure we'll want to stage those changes somewhere
19:21:08 <jeblair> also, we can actually make the bug/blueprint changes before everything else
19:21:20 <jeblair> (ie, it should work if we put it in place right now)
19:21:29 <fungi> i'll put together a comprehensive etherpad to track the time-sensitive changes
19:21:45 <jeblair> #action mordred bugify summit actions
19:21:54 <jeblair> ^ that's going to get reassigned real soon now
19:21:58 <uvirtbot> jeblair: Error: "that's" is not a valid command.
19:22:00 <clarkb> jeblair: he said earlier today that he started that process
19:22:05 <jeblair> oh good!
19:22:12 <clarkb> not done, but he is working on it
19:22:13 <jeblair> #action everyone collect action items from other summit session etherpads and register as bugs
19:22:26 <jeblair> and i know i haven't done that one yet, so, oops.
19:23:12 <jeblair> "clarkb look into subunit/testtools with coverage"
19:23:22 <jeblair> clarkb: had time to look into that yet?
19:23:41 <clarkb> jeblair: lifeless and I have chatted about it and I have a plan, but haven't implemented anything yet
19:23:57 <clarkb> besically we can run subunit under coverage when testr runs
19:24:23 <jeblair> clarkb: does that involve post-run merging?
19:24:26 <clarkb> initially that will be done without parallel subunit execution to avoid the need for merging results
19:24:29 <jeblair> ah
19:25:02 <clarkb> long term lifeless' suggestion is to include coverage info in the subunit stream then testr can do the merging
19:25:19 <clarkb> short term will be equivalent to what we have today. long term will be a step up
19:25:29 <jeblair> yep
19:25:41 <jeblair> #action clarkb continue to look into subunit/testtools with coverage
19:25:50 <jeblair> "clarkb document and test project creation change"
19:26:01 <jeblair> clarkb: i think that has happend, yeah?
19:26:24 <clarkb> it did. I think we/I may end up needing to write a little more documentation, but the puppet + python script is working
19:26:34 <jeblair> clarkb: where is documentation lacking?
19:26:51 <clarkb> stackforgers are a primary consumer of the new tool and the docs don't directly address stackforge
19:27:06 <clarkb> need a if you are adding a stackforge project do foo and bar section
19:27:32 <jeblair> clarkb: yeah, i think a bit of a reorg may be in order...
19:27:48 <jeblair> clarkb: i'm thinking a lot of the new stuff could go into a "howto: create a new project" section
19:28:02 <jeblair> clarkb: (which isn't gerrit-specific, like the current one)
19:28:34 <clarkb> and that should cover things like gitreview, zuul and JJB, and launchpad (or maybe gerrit groups)
19:28:40 <jeblair> clarkb: exactly.
19:28:41 <mordred> jeblair, clarkb: ++
19:28:51 <mordred> (lurking, but in an in-person meeting)
19:29:10 <jeblair> mordred: this one's cooler.  and also has persons.  ;)
19:29:21 <jeblair> clarkb: want to volunteer for that?
19:29:26 <clarkb> jeblair: sure
19:29:26 * fungi is not a person
19:29:56 <clarkb> #action clarkb write a howto: create a new project in openstack-ci docs
19:30:09 <jeblair> "jeblair finish updates to sync script"
19:30:16 <jeblair> so that's basically done...
19:30:42 <fungi> just in time to deprecate it?
19:30:44 <jeblair> but then i got sick and haven't done final testing/polishing on it.
19:30:54 <jeblair> and then it looks like we might actually be able to deprecate ti...
19:30:56 <jeblair> it
19:31:18 <jeblair> so i'm thinking i'll just sit on that and see where we are next week.
19:31:22 <clarkb> jeblair: to test it you should be able to drop it into review-dev
19:31:26 <fungi> if nothing else, your investigation into the lp/openid api additions will help the bug/bp script updates
19:31:37 <clarkb> review-dev runs the sync script as well
19:32:04 <jeblair> clarkb: yeah, i've been testing it locally with a db dump from review dev.
19:33:00 <jeblair> anyway, like i said, if it looks like it may be useful next week, i'll finish it up.  otherwise, i'm happy to be rid of it.  :)
19:33:18 <clarkb> sounds good to me
19:33:29 <jeblair> fungi: and yeah, there's some code/experience there that should make the bug script updates easy
19:33:56 <jeblair> "jeblair propose a system for linking reverifies to bugs"
19:33:56 <fungi> i will feast on whatever part of your brains contains that experience
19:34:11 <jeblair> i have not done that.  but i have thought about it.  :)
19:34:26 <jeblair> #action jeblair propose a system for linking revierifies to bugs
19:34:31 <clarkb> jeblair: do we want it to be more robust than simply updating the regex in layout.yaml?
19:34:43 <jeblair> clarkb: yeah, i think the main component is the reporting...
19:34:59 <clarkb> jeblair: the comment contents already end up in the zuul logs
19:35:08 <fungi> though i suppose the bug linking can come first and reporting second
19:35:09 <clarkb> maybe reporting is an offline grep through the logs?
19:35:29 <jeblair> clarkb: which i think should be something that watches for reverifies and produces a web page with like the top 10 bugs
19:35:41 <clarkb> (more robust is definitely better though, could check the bug is valid in LP and matches and openstack project for example)
19:35:50 <clarkb> jeblair: ++
19:36:23 <fungi> yes, some way of actually forcing people to have a real lp bug open before being able to reverify would be good
19:36:28 <jeblair> i think the reporting is what makes it less "annoying thing the ci people are making us do" and more "useful thing to direct testing and developer resources at known problems"  :)
19:37:15 <fungi> so in that regard, i guess not annoying the devs until we can give them useful reports would be good after all
19:37:34 <fungi> or at least a promise of reports rsn
19:37:59 <jeblair> yeah.  i don't think that'll be too hard though.  gerritlib script that watches the event stream, totals bug links, then spits out some html to a static web page.
19:38:22 <jeblair> _or_ we could try to pretend that's in-scope for zuul.  :)  i'm not convinced.
19:39:01 <jeblair> #topic grenade/quantum
19:39:10 <fungi> i'd be interested to see gerritbot mention the reverifies in #-infra maybe
19:39:28 <clarkb> fungi: ++
19:39:29 <jeblair> I've restarting work with nachi on quantum.  it needs some more changes before it's ready, but progress is being made
19:39:31 * fungi curses his flaky internets
19:39:39 <jeblair> fungi: +1
19:39:58 <jeblair> i'll probably not get to revisiting grenade until next week
19:40:21 <jeblair> anyone have other topics?
19:40:29 <clarkb> pypi uploads
19:40:45 <jeblair> #topic pypi uploads
19:41:17 <clarkb> last week it became apparent that stackforgers could make use of jenkins jobs to upload their packages to pypi
19:41:59 <clarkb> we can't let them run the existing jobs because arbitrary code on hosts with passwords, so to better accomodate them and stop trusting openstack projects I have added two new jenkins job templates to make this more secure
19:42:25 <clarkb> basically create an sdist on a normal jenkins slave, copy that to a slave with pypi credentials and use curl to upload the sdist to pypi
19:42:32 * jeblair cheers
19:42:38 <clarkb> now no more arbitrary code is executed to upload changes to pypi
19:43:02 <pabelanger> Nice!
19:43:08 <clarkb> #link https://review.openstack.org/#/c/16501/
19:43:26 <clarkb> is the last change needed to make it work in the way that is expected (upload meta data being the current missing piece)
19:43:58 <jeblair> clarkb: cool, so we should be able to update all the projects after that's merged
19:44:11 <jeblair> but maybe we'll make one more jjb release first.  :)
19:44:22 <clarkb> jeblair: yes, that is probably a good thing
19:44:30 <clarkb> and we keep getting more JJB patches
19:44:40 <jeblair> yay more jjb contributors!
19:44:49 <pabelanger> woot
19:44:59 <fungi> clearly it fills a previously unaddressed niche
19:45:45 <jeblair> anything else?
19:46:01 <fungi> jeblair: you were wanting to also do a g-r release via traditional methods too, yes? or wait for the pipy automation?
19:46:19 <jeblair> #topic git-review
19:46:29 <jeblair> fungi, saper, and I have tested git-review HEAD
19:46:51 <jeblair> we haven't seen any major regressions, which is expected since currently it only contains one patch over 1.19
19:46:55 <jeblair> which fixes the upgrade bug
19:47:06 <jeblair> so I'll release that today,
19:47:16 <jeblair> then we'll merge documentation of the testing we've done
19:47:27 <jeblair> and open git-review to more substantial changes
19:47:29 <clarkb> jeblair: saper mentioned a bug, in -infra. Did yo usee that?
19:48:06 <jeblair> we still expect it to work on every commit, but we'll be doing manual regression testing before we make another release
19:48:06 <clarkb> looks like its a bug when downloading a change from a different project
19:48:08 <clarkb> maybe that is out of scope
19:48:11 <fungi> oh, that reminds me from this morning... hashar found the previoys commit for the -W option, so i'll see if i can port that to a 1.21 target if nobody beats me to it
19:48:36 <jeblair> clarkb: yeah, if that's what it is, then i'd say it can wait for the next release
19:49:47 <fungi> seems like that bug saper found has probably been hiding in there a while
19:50:06 <jeblair> #topic stackforge
19:50:57 <jeblair> considering the large number of new stackforge projects that have recently been or are expected to be added...
19:51:49 <jeblair> i thought it'd be a good time to remind people that the primary goal of the infrastructure and ci teams is to support the openstack project
19:52:21 <fungi> to that end, a semi-official statement on what stackforge is and isn't might be in order
19:52:47 <jeblair> stackforge is an important area that the ppb has asked us to support as well, but not at the expense of attention and resources for the openstack projects
19:54:09 <jeblair> i expect new stackforge projects to be completely self-sufficient, and i also expect them not to divert time from the infrastructure team, at least, not without giving something back, preferably in the form of people with general interest in the openstack project infrastructure and ci
19:54:53 <clarkb> speaking of zykes- added a simple jenkins job to support readthedocs (FYI to other projects that want to use RTD)
19:56:03 <fungi> jeblair: that sounds entirely reasonable to me
19:56:34 <ttx> jeblair: +1
19:56:48 <jeblair> so i hope that helps people prioritize work and reviews appropriately.  i know some people may not have been around when stackforge started, and it may be helpful to know that concern and decision on prioritization was explicit.  :)
19:56:50 <jeblair> ttx was there.  :)
19:57:16 <jeblair> and since he's here now, i think it's probably time to
19:57:18 <jeblair> #endmeeting