19:03:32 <SpamapS> #startmeeting tripleo
19:03:33 <openstack> Meeting started Tue Jan 13 19:03:32 2015 UTC and is due to finish in 60 minutes.  The chair is SpamapS. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:03:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:03:36 <openstack> The meeting name has been set to 'tripleo'
19:03:51 <SpamapS> #link https://wiki.openstack.org/wiki/Meetings/TripleO
19:04:03 <SpamapS> #topic agenda
19:04:25 <bnemec> o/
19:04:34 <shardy> o/
19:04:35 <SpamapS> Normally I paste the agenda blindly, but I want everyone to take a moment to confirm we need all these and that there aren't other agenda items we need. So I'll mention each one and pause for comments.
19:04:40 <SpamapS> * bugs
19:06:04 <SpamapS> ok we'll keep that one. :)
19:06:09 <SpamapS> * reviews
19:06:14 <SpamapS> also one I think we need to discuss
19:06:40 <SpamapS> * Projects needing releases
19:07:12 <SpamapS> I feel like this one is usually just a nomination for somebody to volunteer and it's done.
19:07:18 <SpamapS> But it seems to be working so why change it?
19:07:21 * SpamapS taps the mic
19:07:23 <SpamapS> is this thing on?
19:07:25 <jdob> sure, its reasonably quick
19:07:29 <jdob> to select a volunteer
19:07:36 <jdob> hell, I'll make this one easy this week: I'll do it
19:07:37 <jdob> done
19:08:15 <Ng> can we just automate the releases? :D
19:08:25 <SpamapS> btw I may go dark for a moment if the people installing my new HVAC unit clip my coax with their crane.. I'll come back on 4G tether if it happens.
19:08:37 <SpamapS> Ng: I often wonder that. ;)
19:08:49 <SpamapS> * CD Cloud status
19:08:53 <SpamapS> I feel like this one needs rethinking
19:09:03 <SpamapS> we don't really run it like we used to
19:09:13 <SpamapS> and there are really a bunch of TripleO clouds that don't really operate using CD
19:09:39 <SpamapS> So I wonder if we can morph this into "Live Cloud Status" and offer it as a chance for those running our TripleO clouds to speak up.
19:10:15 <bnemec> Do those people come to the meeting?
19:10:29 <Ng> do we want it to just be about the infrastructure clouds we're running for ourselves, or do we mean "anyone running a tripleo cloud can come and complain in this agenda item"?
19:11:11 <dprince> bnemec; no way, they are off fixing things
19:11:22 <bnemec> :-)
19:11:40 <SpamapS> As somebody who has struggled to bring up a TripleO cloud, I'd like it to be an agenda item where we can discuss known things affecting real clouds at a higher level than bugs.
19:11:52 <Ng> +1
19:12:02 <TheJulia> +1
19:12:22 <SpamapS> Also poor tchaypo has been fighting hp2 for months alone, I want it to be a place where we offer support to tripleo-cd-admins. ;)
19:12:59 <dprince> +1, we need a post-mortem
19:13:25 <bnemec> I'm not opposed to it, but if we want to do that we need to make sure the right people can be here is all.
19:14:06 <SpamapS> #action SpamapS to change "CD Cloud Status" to "Live Clouds Status" in wiki
19:14:21 <Ng> s/Live/Production/ maybe?
19:14:29 <SpamapS> w/e
19:14:30 <greghaynes> My only fear is same as bnemec, that the participants for the most part might not be here but it doesnt hurt to try - worst case well just move along quickly
19:14:39 <greghaynes> so +1 :)
19:14:39 <SpamapS> greghaynes: why aren't they here?
19:14:56 <Ng> tchaypo is often very timezone afflicted
19:14:58 <SpamapS> I say unto all of you.. ;)
19:15:07 <greghaynes> I think most cloud operators dont want to be told to wait until the meeting to bring up their issues
19:15:08 <SpamapS> We have an alt TZ meeting which will share this agenda.
19:15:16 <SpamapS> greghaynes: this is not like that
19:15:36 <Ng> SpamapS: indeed. Does anyone apart from him and StevenK go to those? ;)
19:15:55 <SpamapS> I'm saying if we're talking about something affecting a real cloud in #tripleo or on the ML or in person, we should encourage people to share here so that developers are present to discuss.
19:16:04 <SpamapS> Ng: dunno. :/
19:16:28 * SpamapS hears the crane flexing
19:16:37 <greghaynes> ok, so in addition to the normal issue-bringing-up
19:16:51 <SpamapS> * CI
19:17:12 <SpamapS> vital, don't change it, IMO
19:17:24 <SpamapS> * Specs
19:17:29 <SpamapS> also vital, need to do more here actually
19:17:36 <SpamapS> and finally * Open discussion
19:17:39 <SpamapS> duh, keeping that
19:17:42 <SpamapS> ok so on to bugs?
19:18:29 <bnemec> +1
19:21:16 * bnemec wonders if SpamapS got craned ;-)
19:21:33 <greghaynes> Yes, so bugs, lets make less of those ;)
19:21:45 <bnemec> And fix the ones we do have
19:21:47 <SpamapS> no crane
19:21:49 <SpamapS> #topic bugs
19:21:52 <SpamapS> but almost :)
19:22:18 <SpamapS> #link https://bugs.launchpad.net/tripleo/
19:22:18 <SpamapS> #link https://bugs.launchpad.net/diskimage-builder/
19:22:18 <SpamapS> #link https://bugs.launchpad.net/os-refresh-config
19:22:18 <SpamapS> #link https://bugs.launchpad.net/os-apply-config
19:22:18 <SpamapS> #link https://bugs.launchpad.net/os-collect-config
19:22:21 <SpamapS> #link https://bugs.launchpad.net/os-cloud-config
19:22:23 <SpamapS> #link https://bugs.launchpad.net/os-net-config
19:22:26 <SpamapS> #link https://bugs.launchpad.net/tuskar
19:22:28 <SpamapS> #link https://bugs.launchpad.net/python-tuskarclient
19:22:56 <bnemec> Can we close https://bugs.launchpad.net/tripleo/+bug/1405908 ?
19:22:57 <uvirtbot> Launchpad bug 1405908 in tripleo "Setuptools 8.4 fails to upgrade/install packages" [Critical,Confirmed]
19:23:08 <bnemec> We're multiple major versions past 8.4 now.
19:23:15 <SpamapS> sounds like it
19:23:23 <SpamapS> bnemec: seems like it's resolved for us anyway
19:23:23 <greghaynes> bnemec: yep, the revert to that fix has even been marged
19:23:49 <SpamapS> bug 1374626 is still stalled on me being too busy
19:23:50 <uvirtbot> Launchpad bug 1374626 in tripleo "UIDs of data-owning users might change between deployed images" [Critical,Triaged] https://launchpad.net/bugs/1374626
19:24:06 <SpamapS> bug 1401617 is similarly stalled
19:24:08 <uvirtbot> Launchpad bug 1401617 in tripleo "Flavor based ramdisk/kernel ID is deprecated in Ironic since Juno" [Critical,In progress] https://launchpad.net/bugs/1401617
19:24:35 <SpamapS> greghaynes: whats going on with bug 1385346
19:24:36 <uvirtbot> Launchpad bug 1385346 in tripleo "upstart service unreliable after introducion of pipe to logger" [Critical,Fix committed] https://launchpad.net/bugs/1385346
19:24:53 <greghaynes> SpamapS: Its been fix commited - its as good as its going to get basically
19:25:16 <greghaynes> we no longer do the | logger hack so aside from fixing upstart...
19:25:52 <SpamapS> greghaynes: we are fixing upstart by moving to systemd. ;)
19:26:17 <greghaynes> Out of the furnace ...
19:26:42 <greghaynes> https://bugs.launchpad.net/tripleo/+bug/1401300 is stalled on me
19:26:43 <uvirtbot> Launchpad bug 1401300 in tripleo "Keystone Private Key not securely sent to host" [Critical,Confirmed]
19:26:46 <SpamapS> I don't see any other criticals.
19:26:57 <greghaynes> I plan to do a hacky temp fix for it very soon
19:27:00 <SpamapS> any other bugs we need to discuss?
19:29:11 <greghaynes> dont think so
19:29:14 <SpamapS> #topic reviews
19:30:25 <SpamapS> #info There's a dashboard linked from https://wiki.openstack.org/wiki/TripleO#Review_team - look for "TripleO Inbox Dashboard"
19:30:28 <SpamapS> #link http://russellbryant.net/openstack-stats/tripleo-openreviews.html
19:30:31 <SpamapS> #link http://russellbryant.net/openstack-stats/tripleo-reviewers-30.txt
19:30:35 <SpamapS> #link http://russellbryant.net/openstack-stats/tripleo-reviewers-90.txt
19:30:38 <SpamapS> So, I think I'm going to do a meta-review of tchaypo
19:30:55 <SpamapS> seems like he'd make a good addition to the core review team
19:30:55 <Ng> SpamapS: very +1
19:30:57 <bnemec> +1
19:30:59 <greghaynes> yaya
19:31:09 <SpamapS> Pretty sure a review of his reviews will confirm that.
19:31:17 <SpamapS> THe stats are a little bit meaningless right now
19:31:33 <SpamapS> I move we ignore them.
19:32:49 <bnemec> We had talked about sending a reminder in January that we would be potentially pruning the core list once everyone has had a chance to get back from the holidays.
19:34:00 <SpamapS> bnemec: right, so it's hard to judge the core list with the remnants of December still dogging the stats.
19:34:03 <SpamapS> bnemec: so maybe 2 more weeks?
19:34:36 <SpamapS> anything else? Anybody want something specific reviewed, or feel a spec is ready for +A?
19:35:28 <dprince> I'd like eyes on my swift puppet stuff
19:36:03 <dprince> I'm also finding some really good fixups cleanups as I'm reviewing things for puppet support as in https://review.openstack.org/#/c/146293/1
19:36:08 <SpamapS> awesome
19:36:09 <dprince> and https://review.openstack.org/#/c/146292/1
19:36:23 <bnemec> I'll put in a plug for giulio's Cinder HA spec: https://review.openstack.org/101237
19:36:24 <SpamapS> Really good to see that progressing.
19:36:30 <dprince> I'm not a swift expert though. So eyes :)
19:36:30 <bnemec> It's pretty much what we discussed at summit.
19:36:41 <SpamapS> Yeah I feel like the cinder HA spec is pretty good and will be ready for +A once it has a few more reviews.
19:36:57 <SpamapS> also I'm a ceph fanboi so doooo it ;)
19:37:55 <SpamapS> ok moving on?
19:38:19 <SpamapS> #topic Projects needing releases
19:38:34 * SpamapS spins the wheel of developers
19:38:42 <SpamapS> tick... tick.......... greghaynes ?
19:38:46 <greghaynes> sure
19:38:52 <SpamapS> fantastic
19:38:59 <dprince> greghaynes: you won!
19:39:02 <greghaynes> \O/
19:39:05 <SpamapS> A NEW CAR!
19:39:14 <greghaynes> Best releases ever
19:39:18 <SpamapS> and by car I mean release of diskimage-builder
19:39:25 <greghaynes> oh :(
19:39:32 <SpamapS> #topic Live Clouds Status
19:39:57 <SpamapS> I've been trying to run a cloud on blades backed by SAN and it's been rough
19:40:38 <SpamapS> I've resorted to pre-formatting the SAN LUN's with an ext4 label and then overriding block-device-mapping.ephemeral0 to point at /dev/disk/by-label/....
19:40:58 <SpamapS> seems like it might work, multipath issues notwithstanding
19:42:52 <SpamapS> I also had a situation where my blades were set to 250 days ago in their hwclocks
19:42:52 <SpamapS> so had to go through and reset all the clocks
19:42:53 <SpamapS> anybody else want to share?
19:42:53 <SpamapS> (Feels like group therapy ... ;)
19:42:53 <SpamapS> Going to move on then.
19:43:29 <bnemec> SpamapS: Your last five lines all came in within two seconds for me.
19:43:47 <bnemec> Not that I have anything to share anyway :-)
19:43:49 <SpamapS> bnemec: which is why I pause. :)
19:43:56 <SpamapS> #topic CI
19:44:10 <Ng> SpamapS: too late for you now, but https://review.openstack.org/#/c/145743/ should have prevented that time horror
19:44:48 <SpamapS> Ng: ah right maybe my cloud is old. ;)
19:45:01 <SpamapS> I don't know how CI is doing lately.
19:45:08 <SpamapS> Seems like it is stable right now
19:45:16 <greghaynes> our failure rate seems pretty high ATM, I keep seeing networkey issues
19:45:27 <SpamapS> #link http://goodsquishy.com/downloads/s_tripleo-jobs.html
19:45:27 <greghaynes> this is just based on my random reviewing though
19:45:39 <SpamapS> yeah ow
19:45:52 <bnemec> Actually, I'm seeing all red for our precise jobs atm: http://goodsquishy.com/downloads/tripleo-jobs.html
19:45:59 <SpamapS> if people can devote a little time this week to debugging those fails.. it can get off the rails fast if we don't stay on top of it.
19:46:17 <SpamapS> #link http://goodsquishy.com/downloads/tripleo-jobs.html
19:46:19 <bnemec> W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/precise-security/main/source/Sources  503  Service Unavailable
19:46:28 <SpamapS> eeeenteresting
19:46:48 <greghaynes> and OUTPUT=ERROR (ConnectionError): ('Connection aborted.', error(113, 'No route to host'))
19:47:06 <greghaynes> anywho, something to debug
19:47:11 <bnemec> Yep
19:47:42 <SpamapS> #topic Specs
19:48:55 <Ng> SpamapS: (blah, that was poorly worded. that change hasn't landed yet. I meant it should prevent that, once landed)
19:52:32 <SpamapS> Ng: ahh1
19:52:37 <SpamapS> #topic open discussion
19:52:42 <SpamapS> Meetup!
19:52:52 * greghaynes just booked
19:52:52 <SpamapS> We may have a final attendee list at this point.
19:53:10 <jdob> can you link the etherpad for that again?
19:53:13 <SpamapS> #link https://etherpad.openstack.org/p/kilo-tripleo-midcycle-meetup
19:53:15 <shardy> SpamapS: is there still space for additional attendees?
19:53:37 <SpamapS> shardy: yes we have space for 30 and I count 13.
19:53:57 <shardy> SpamapS: ok, thanks, just wondering as stevebaker and I were discussing it
19:54:01 <SpamapS> o/' bring it on down to tripleo-ville o/'
19:54:07 <geoffarnold> What's the cutoff date?
19:54:19 <SpamapS> geoffarnold: day of. :)
19:54:26 <Ng> hopefully someone will eat an inadvisable amount of BBQ, in my absence :D
19:54:50 <SpamapS> Curious where those who have booked are staying.
19:54:52 <SpamapS> I'm in the Westin
19:54:55 <geoffarnold> thanks.
19:55:05 <greghaynes> Im at crowne plaza
19:55:26 <greghaynes> which I believe is literally across the street
19:55:36 <TheJulia> I'm in the Hilton
19:55:47 <greghaynes> optimal dispersement achieved!
19:55:58 <SpamapS> Sheraton is across the street from the office. Crowne Plaza is a bit down toward Pioneer Square IIRC but I may be misremembering.
19:56:16 * dprince is in Vintage
19:56:33 <SpamapS> awesome we'll be able to escape eachother properly then. ;)
19:56:42 <dprince> like a fine wine
19:56:47 <TheJulia> greghaynes: I think your like 5 blocks from the office
19:57:08 <greghaynes> what!
19:57:21 <greghaynes> the CWT map must have been lies if thats the case
19:57:25 <SpamapS> If people can start filling out the agenda that would be great.
19:58:01 <SpamapS> I think hammering HP2 into submission and pushing on the HA job stability are at the top of my list.
19:58:26 <SpamapS> also getting puppety things landed
19:58:36 <dprince> SpamapS: ++
19:58:39 <SpamapS> anyway, let's wrap up. Thanks everyone!!!
19:58:43 <SpamapS> #endmeeting