| ttx | maybe it's just a minority, but I've got lots of reports since ~August 5 | 00:00 |
|---|---|---|
| jeblair | ttx: do you have some specific examples? | 00:02 |
| jeblair | ttx: (i imagine you are correct; it'll be easier to figure out what's wrong with some specific cases) | 00:02 |
| ttx | jeblair: I'll forward you my weekly report. usually those are all human errors that I try to fix. This time there were lots of "failed to mark it so I did it". | 00:03 |
| *** lifeless has joined #openstack-infra | 00:03 | |
| ttx | jeblair: sent. You shall find a few examples in that list... for cinder nova... | 00:04 |
| ttx | (this script just looks at when a human sets FixCommitted or FixReleased) | 00:05 |
| ttx | There are a few weird occurences in there where the script mentioned the commit but did NOT set TO fixCommitted. | 00:05 |
| ttx | anyway, I plan to investigate one of those days... just checking if you heard similar reports. | 00:06 |
| *** UtahDave has quit IRC | 00:07 | |
| *** fbo is now known as fbo_away | 00:09 | |
| *** sarob_ has joined #openstack-infra | 00:13 | |
| ttx | jeblair: looking at recent changes they look alright. Maybe it was just a transient thing | 00:14 |
| fungi | ttx: the ones i've seen weren't using the closes-bug header in the commit message and used wording other than the handful of backward-compatible patterns (things like "this fixes the issues in bug #XXXXXX" | 00:14 |
| *** erfanian has joined #openstack-infra | 00:15 | |
| fungi | the fallback solution in the newer pattern match for it assumes the few things it didn't know for sure meant "closes-bug" should be interpreted as "related-bug" | 00:15 |
| ttx | fungi: I suspect we had transitional things where the review link was posted before the transition and the fixcommitted failes to show | 00:15 |
| fungi | ttx: agreed | 00:15 |
| ttx | we should be good now | 00:16 |
| fungi | ttx: we're a little over a week with the new version, so stuff from before that is much more likely to not close bugs when merged | 00:16 |
| openstackgerrit | Alessandro Pilotti proposed a change to openstack-dev/pbr: Fixes issue with command escaping on Windows https://review.openstack.org/41283 | 00:17 |
| *** sarob has quit IRC | 00:17 | |
| fungi | though if they had "fixes bug #whatever" on a line by itself (which i think was the old documented example) it still gets picked up as closes-bug from what i've seen | 00:17 |
| fungi | and there were a few permutations of that which also got included for backwards-compat | 00:18 |
| *** sarob_ has quit IRC | 00:18 | |
| *** pcrews has joined #openstack-infra | 00:18 | |
| ttx | yep | 00:18 |
| *** emagana has quit IRC | 00:25 | |
| *** dina_belova has joined #openstack-infra | 00:29 | |
| *** lcestari has joined #openstack-infra | 00:30 | |
| *** lcestari has quit IRC | 00:30 | |
| *** dina_belova has quit IRC | 00:31 | |
| *** dkliban has quit IRC | 00:33 | |
| pabelanger | I guess tarballs.openstack.org does not fall under the realm of -infra? | 00:36 |
| *** jhesketh has quit IRC | 00:36 | |
| *** dkliban has joined #openstack-infra | 00:36 | |
| pabelanger | was wanting to see some things in puppet, but looks like the host is not managed | 00:36 |
| *** jhesketh has joined #openstack-infra | 00:36 | |
| jeblair | pabelanger: it does; we just haven't gotten around to rsyncinc the content to static.o.o | 00:36 |
| *** nati_ueno has quit IRC | 00:37 | |
| pabelanger | jeblair, Ah, I see | 00:38 |
| pabelanger | cool, thanks | 00:38 |
| *** nati_ueno has joined #openstack-infra | 00:40 | |
| *** pcm__ has quit IRC | 00:41 | |
| *** rfolco has joined #openstack-infra | 00:42 | |
| *** rfolco has quit IRC | 00:42 | |
| *** nati_ueno has quit IRC | 00:48 | |
| *** nati_ueno has joined #openstack-infra | 00:49 | |
| *** nati_uen_ has joined #openstack-infra | 00:51 | |
| *** nati_ueno has quit IRC | 00:53 | |
| fungi | eek, tarballs.o.o is still on 10.04 lts even | 00:55 |
| *** mriedem has joined #openstack-infra | 00:55 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gearman-plugin: Rework starting/stopping executors https://review.openstack.org/41488 | 00:56 |
| jeblair | fungi: yes, we should move it. | 00:57 |
| jeblair | clarkb, zaro: ^ that should take care of the problem zaro pointed out. | 00:57 |
| jeblair | oh, right unit tests... | 00:57 |
| openstackgerrit | Tom Fifield proposed a change to openstack-infra/config: Add build job for Japanese Operations Guide https://review.openstack.org/39846 | 00:59 |
| openstackgerrit | Tom Fifield proposed a change to openstack-infra/config: Add build job for Japanese Operations Guide https://review.openstack.org/39846 | 01:02 |
| *** ^d has quit IRC | 01:07 | |
| *** yaguang has joined #openstack-infra | 01:08 | |
| *** jerryz has quit IRC | 01:12 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/gearman-plugin: Rework starting/stopping executors https://review.openstack.org/41488 | 01:13 |
| *** erfanian has quit IRC | 01:18 | |
| *** vipul is now known as vipul-away | 01:26 | |
| *** dina_belova has joined #openstack-infra | 01:29 | |
| *** dina_belova has quit IRC | 01:34 | |
| *** yaguang has quit IRC | 01:35 | |
| *** vipul-away is now known as vipul | 01:41 | |
| *** mjfork has quit IRC | 01:45 | |
| *** markmcclain has quit IRC | 01:48 | |
| *** yaguang has joined #openstack-infra | 01:54 | |
| *** mrodden1 has quit IRC | 01:55 | |
| *** mrodden has joined #openstack-infra | 01:56 | |
| *** changbl has joined #openstack-infra | 01:57 | |
| *** adalbas has quit IRC | 02:02 | |
| *** anteaya has quit IRC | 02:03 | |
| *** mriedem has quit IRC | 02:09 | |
| openstackgerrit | benley proposed a change to openstack-infra/jenkins-job-builder: Add support for parameter filters in copyartifact https://review.openstack.org/41582 | 02:10 |
| *** nati_uen_ has quit IRC | 02:11 | |
| clarkb | got distracted for far too long | 02:11 |
| clarkb | jog0: still around? re the py3k thing. The print statement in the example is print ('foo') + 'some string' which is not compat | 02:12 |
| fungi | whereas print ('foo' + 'some string') would be fine | 02:16 |
| clarkb | ya, and I am not sure that is something that hacking should try to catch as it is a runtime error | 02:16 |
| clarkb | but meh | 02:16 |
| fungi | i'm similarly on the fence about hacking acting as a proxy for a python 3.x interpreter | 02:19 |
| fungi | we now have the ability to just try and parse the entire project with python 3.3 and catch waaaay more than something as simple as print statements | 02:20 |
| jog0 | clarkb fungi: no I am not around | 02:28 |
| fungi | jog0: awesome, neither am i ;) | 02:28 |
| jog0 | but yeah now that we have py33 running I am inclined to say just do it that way | 02:28 |
| *** dina_belova has joined #openstack-infra | 02:30 | |
| *** melwitt has quit IRC | 02:34 | |
| *** dina_belova has quit IRC | 02:35 | |
| *** yaguang has quit IRC | 02:40 | |
| jgriffith | sdague: around? | 02:55 |
| fungi | jgriffith: i believe he has declared himself officially vacationing as of some hours ago now | 02:57 |
| jgriffith | fungi: ahh... good for him :) | 03:01 |
| fungi | not so good for the rest of us though | 03:02 |
| fungi | s'okay... i'll be doing the same soon | 03:02 |
| jgriffith | fungi: Ummm... I wasn't informed of this? | 03:02 |
| jgriffith | fungi: sorry, but I think that's going to be terribly inconvenient | 03:03 |
| fungi | next you're going to tell me you need me to come into work on saturday | 03:04 |
| jgriffith | fungi: haha.. no need to tell you, you always do anyway | 03:04 |
| jgriffith | fungi: I've just come to expect it :) | 03:05 |
| fungi | touché | 03:05 |
| jgriffith | fungi: anything fun/exciting planned or just some down time? | 03:05 |
| jgriffith | fungi: or none of jgriffith 's biz | 03:06 |
| fungi | showing my girlfriend around seattle for a week since she's never been, then getting on a boat up to alaska and back for a week after that | 03:06 |
| jgriffith | Wow! That sounds pretty darn good! | 03:06 |
| * jgriffith is a bit jealous | 03:06 | |
| fungi | planning to try to be useful while in seattle, though probably not as useful as i'd like | 03:06 |
| jgriffith | fungi: take a real vacation! | 03:07 |
| fungi | but the following week i'll be (un)lucky if i can find an internet anywhere | 03:07 |
| jgriffith | fungi: well, here's to hoping you don't find internet access | 03:07 |
| jgriffith | :) | 03:07 |
| fungi | anyway, it'll be a cool way to escape the 100f heat here in nb | 03:07 |
| fungi | nc | 03:08 |
| fungi | (not nb) | 03:08 |
| jgriffith | ewww... | 03:08 |
| jgriffith | hot AND humid | 03:08 |
| fungi | yeah, it'll be nice to get away from | 03:08 |
| jgriffith | indeed... hope you enjoy and hope that you don't find any internet access | 03:08 |
| fungi | thanks! | 03:08 |
| jgriffith | When are you heading off? | 03:09 |
| fungi | plane to seattle this coming saturday, then getting on the boat the next friday | 03:10 |
| jgriffith | nice.. I'l be sure NOT to ping you between those dates :) | 03:10 |
| fungi | eh, it's okay. if i really can't help you i just won't answer ;) | 03:10 |
| jgriffith | fungi: haha.. fair enough :) | 03:10 |
| *** yaguang has joined #openstack-infra | 03:25 | |
| *** dina_belova has joined #openstack-infra | 03:30 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/jenkins-job-builder: Document escaping braces in template literals https://review.openstack.org/40973 | 03:31 |
| *** dina_belova has quit IRC | 03:35 | |
| *** tianst has joined #openstack-infra | 03:40 | |
| *** jfriedly has quit IRC | 03:42 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Wrap long lines https://review.openstack.org/40800 | 03:50 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Wrap long lines https://review.openstack.org/40800 | 03:51 |
| *** SergeyLukjanov has joined #openstack-infra | 04:05 | |
| *** mberwanger has quit IRC | 04:05 | |
| *** changbl has quit IRC | 04:05 | |
| *** changbl has joined #openstack-infra | 04:06 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Migrate to pbr. https://review.openstack.org/35486 | 04:07 |
| *** tianst has quit IRC | 04:07 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Implement integration tests. https://review.openstack.org/35104 | 04:13 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Implement integration tests. https://review.openstack.org/35104 | 04:20 |
| *** locke105 has quit IRC | 04:29 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Pin hacking <0.6 https://review.openstack.org/40799 | 04:30 |
| *** dina_belova has joined #openstack-infra | 04:31 | |
| *** locke105 has joined #openstack-infra | 04:31 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/git-review: Implement integration tests. https://review.openstack.org/35104 | 04:35 |
| *** dina_belova has quit IRC | 04:35 | |
| *** dina_belova has joined #openstack-infra | 04:42 | |
| *** dina_belova has quit IRC | 04:48 | |
| *** nati_ueno has joined #openstack-infra | 04:52 | |
| *** nati_ueno has quit IRC | 05:02 | |
| *** nati_ueno has joined #openstack-infra | 05:02 | |
| *** ruhe has joined #openstack-infra | 05:08 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Reword H404 description to match check https://review.openstack.org/41240 | 05:10 |
| *** vogxn has joined #openstack-infra | 05:14 | |
| *** SergeyLukjanov has quit IRC | 05:18 | |
| *** nayward has joined #openstack-infra | 05:21 | |
| openstackgerrit | Sergey Kolekonov proposed a change to openstack-infra/jenkins-job-builder: Existing plugins improvements https://review.openstack.org/41375 | 05:22 |
| *** SergeyLukjanov has joined #openstack-infra | 05:30 | |
| *** SergeyLukjanov has quit IRC | 05:31 | |
| *** nicedice_ has quit IRC | 05:43 | |
| *** sarob has joined #openstack-infra | 05:46 | |
| *** dina_belova has joined #openstack-infra | 05:49 | |
| *** nati_uen_ has joined #openstack-infra | 05:50 | |
| *** dina_belova has quit IRC | 05:53 | |
| *** nati_ueno has quit IRC | 05:54 | |
| *** sarob has quit IRC | 06:03 | |
| *** sarob has joined #openstack-infra | 06:03 | |
| *** morganfainberg is now known as morganfainberg_a | 06:04 | |
| *** sarob has quit IRC | 06:07 | |
| *** ruhe has quit IRC | 06:12 | |
| *** ruhe has joined #openstack-infra | 06:14 | |
| *** ruhe has quit IRC | 06:18 | |
| *** odyssey4me has joined #openstack-infra | 06:25 | |
| *** lyz has quit IRC | 06:49 | |
| *** lyz has joined #openstack-infra | 06:50 | |
| *** nati_uen_ has quit IRC | 07:03 | |
| *** nati_ueno has joined #openstack-infra | 07:04 | |
| *** morganfainberg_a is now known as morganfainberg | 07:07 | |
| *** dnavale has quit IRC | 07:09 | |
| *** sarob has joined #openstack-infra | 07:14 | |
| *** sarob has quit IRC | 07:18 | |
| *** vogxn has quit IRC | 07:33 | |
| *** ruhe has joined #openstack-infra | 07:35 | |
| *** dguitarbite has joined #openstack-infra | 07:35 | |
| *** fbo_away is now known as fbo | 07:35 | |
| *** psedlak has joined #openstack-infra | 07:37 | |
| *** nati_uen_ has joined #openstack-infra | 07:52 | |
| *** nati_ueno has quit IRC | 07:55 | |
| *** afazekas has joined #openstack-infra | 07:55 | |
| *** afazekas has quit IRC | 07:55 | |
| *** afazekas has joined #openstack-infra | 07:55 | |
| *** vogxn has joined #openstack-infra | 07:57 | |
| *** boris-42 has joined #openstack-infra | 08:08 | |
| *** jpich has joined #openstack-infra | 08:09 | |
| *** zul has quit IRC | 08:13 | |
| *** yaguang has quit IRC | 08:16 | |
| *** yaguang has joined #openstack-infra | 08:28 | |
| *** dguitarbite has quit IRC | 08:29 | |
| *** markmc has joined #openstack-infra | 08:37 | |
| *** zehicle has quit IRC | 08:45 | |
| *** vogxn has quit IRC | 08:53 | |
| *** yaguang has quit IRC | 09:00 | |
| *** koobs has quit IRC | 09:07 | |
| *** koobs has joined #openstack-infra | 09:07 | |
| *** vogxn has joined #openstack-infra | 09:08 | |
| *** yaguang has joined #openstack-infra | 09:17 | |
| *** mrmartin has joined #openstack-infra | 09:24 | |
| *** SergeyLukjanov has joined #openstack-infra | 09:49 | |
| openstackgerrit | Serg Melikyan proposed a change to openstack-infra/config: Added murano-common project https://review.openstack.org/41634 | 09:50 |
| chmouel | mordred: is it expected? http://pastie.org/pastes/8232222/text?key=vvw9rh55njzzvqtvoji5rg | 09:57 |
| chmouel | mordred: running tox -epy27 on nova try to install stuff as root | 09:57 |
| chmouel | mordred: running tox -epy27 on nova try to install stuff in /usr/local | 09:57 |
| *** dina_belova has joined #openstack-infra | 09:59 | |
| *** ruhe has quit IRC | 10:01 | |
| *** koolhead11 has quit IRC | 10:08 | |
| *** nayward has quit IRC | 10:11 | |
| *** nayward has joined #openstack-infra | 10:13 | |
| *** vogxn has quit IRC | 10:22 | |
| *** mrodden1 has joined #openstack-infra | 10:31 | |
| *** nati_ueno has joined #openstack-infra | 10:32 | |
| *** cp16net_ has joined #openstack-infra | 10:33 | |
| *** tianst20 has joined #openstack-infra | 10:34 | |
| *** comstud_ has joined #openstack-infra | 10:35 | |
| *** lifeless_ has joined #openstack-infra | 10:36 | |
| *** samalba_ has joined #openstack-infra | 10:39 | |
| *** SergeyLukjanov has quit IRC | 10:40 | |
| *** nati_uen_ has quit IRC | 10:40 | |
| *** mrodden has quit IRC | 10:40 | |
| *** lifeless has quit IRC | 10:40 | |
| *** ladquin has quit IRC | 10:40 | |
| *** sdake_ has quit IRC | 10:40 | |
| *** tian has quit IRC | 10:40 | |
| *** cp16net has quit IRC | 10:40 | |
| *** ianw has quit IRC | 10:40 | |
| *** yamahata has quit IRC | 10:40 | |
| *** dansmith has quit IRC | 10:40 | |
| *** harlowja has quit IRC | 10:40 | |
| *** Vivek has quit IRC | 10:40 | |
| *** samalba has quit IRC | 10:40 | |
| *** akscram has quit IRC | 10:40 | |
| *** maurosr has quit IRC | 10:40 | |
| *** comstud has quit IRC | 10:40 | |
| *** uvirtbot has quit IRC | 10:40 | |
| *** cp16net_ is now known as cp16net | 10:40 | |
| *** samalba_ is now known as samalba | 10:40 | |
| *** ianw has joined #openstack-infra | 10:45 | |
| *** ruhe has joined #openstack-infra | 10:45 | |
| *** sdake_ has joined #openstack-infra | 10:45 | |
| *** sdake_ has joined #openstack-infra | 10:45 | |
| *** uvirtbot has joined #openstack-infra | 10:46 | |
| *** ladquin has joined #openstack-infra | 10:46 | |
| *** dansmith has joined #openstack-infra | 10:46 | |
| *** Vivek has joined #openstack-infra | 10:46 | |
| *** akscram has joined #openstack-infra | 10:46 | |
| *** maurosr has joined #openstack-infra | 10:46 | |
| *** yamahata has joined #openstack-infra | 10:46 | |
| *** SergeyLukjanov has joined #openstack-infra | 10:46 | |
| *** yaguang has quit IRC | 10:52 | |
| *** ruhe has quit IRC | 10:53 | |
| openstackgerrit | Serg Melikyan proposed a change to openstack-infra/config: Added murano-common project https://review.openstack.org/41634 | 10:55 |
| *** fbo is now known as fbo_away | 10:56 | |
| *** vogxn has joined #openstack-infra | 10:56 | |
| *** zehicle has joined #openstack-infra | 10:57 | |
| *** mjfork has joined #openstack-infra | 10:58 | |
| *** pcm_ has joined #openstack-infra | 11:00 | |
| *** vogxn has quit IRC | 11:00 | |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Add groups to gerrit https://review.openstack.org/41643 | 11:02 |
| *** SergeyLukjanov has quit IRC | 11:02 | |
| openstackgerrit | Zane Bitter proposed a change to openstack-infra/reviewstats: Update Heat core team https://review.openstack.org/41644 | 11:06 |
| openstackgerrit | Zane Bitter proposed a change to openstack-infra/reviewstats: Make reviewers output line up https://review.openstack.org/41645 | 11:06 |
| *** lcestari has joined #openstack-infra | 11:21 | |
| *** dina_belova has quit IRC | 11:21 | |
| openstackgerrit | Serg Melikyan proposed a change to openstack-infra/config: Fix ACL for Murano projects https://review.openstack.org/41650 | 11:21 |
| *** dina_belova has joined #openstack-infra | 11:24 | |
| *** ruhe has joined #openstack-infra | 11:25 | |
| *** woodspa has joined #openstack-infra | 11:25 | |
| *** adalbas has joined #openstack-infra | 11:26 | |
| *** vogxn has joined #openstack-infra | 11:34 | |
| *** SergeyLukjanov has joined #openstack-infra | 11:37 | |
| *** fbo_away is now known as fbo | 11:43 | |
| *** ArxCruz has joined #openstack-infra | 11:47 | |
| *** dina_belova has quit IRC | 11:57 | |
| Alex_Gaynor | Oooh, I like the styling of the completness bars on zuul status | 11:58 |
| *** rfolco has joined #openstack-infra | 11:59 | |
| *** yaguang has joined #openstack-infra | 11:59 | |
| *** dina_belova has joined #openstack-infra | 12:01 | |
| *** dina_belova has quit IRC | 12:05 | |
| openstackgerrit | will soula proposed a change to openstack-infra/jenkins-job-builder: Adding support for the Warnings plugin https://review.openstack.org/40621 | 12:08 |
| *** zul has joined #openstack-infra | 12:11 | |
| *** mriedem has joined #openstack-infra | 12:13 | |
| openstackgerrit | Ryan Petrello proposed a change to openstack-infra/config: Add pecan project. https://review.openstack.org/41552 | 12:16 |
| *** Ryan_Lane has joined #openstack-infra | 12:17 | |
| *** weshay has joined #openstack-infra | 12:19 | |
| *** mberwanger has joined #openstack-infra | 12:20 | |
| openstackgerrit | will soula proposed a change to openstack-infra/jenkins-job-builder: Adding support for the Warnings plugin https://review.openstack.org/40621 | 12:22 |
| *** ruhe has quit IRC | 12:22 | |
| openstackgerrit | Ryan Petrello proposed a change to openstack-infra/config: Add pecan project. https://review.openstack.org/41552 | 12:22 |
| *** dina_belova has joined #openstack-infra | 12:24 | |
| *** dnavale has joined #openstack-infra | 12:24 | |
| *** sandywalsh has quit IRC | 12:25 | |
| openstackgerrit | will soula proposed a change to openstack-infra/jenkins-job-builder: Adding support for the Warnings plugin https://review.openstack.org/40621 | 12:33 |
| *** dina_belova has quit IRC | 12:34 | |
| *** dina_belova has joined #openstack-infra | 12:34 | |
| *** psedlak has quit IRC | 12:35 | |
| *** zul has quit IRC | 12:37 | |
| *** sandywalsh has joined #openstack-infra | 12:39 | |
| *** zul has joined #openstack-infra | 12:40 | |
| *** afazekas has quit IRC | 12:41 | |
| *** prad_ has joined #openstack-infra | 12:56 | |
| *** zul has quit IRC | 13:02 | |
| *** prad_ has quit IRC | 13:03 | |
| *** mberwanger has quit IRC | 13:03 | |
| *** zul has joined #openstack-infra | 13:04 | |
| *** anteaya has joined #openstack-infra | 13:05 | |
| *** vogxn has quit IRC | 13:05 | |
| *** dprince has joined #openstack-infra | 13:07 | |
| *** ruhe has joined #openstack-infra | 13:10 | |
| *** psedlak has joined #openstack-infra | 13:12 | |
| *** zul has quit IRC | 13:15 | |
| *** prad_ has joined #openstack-infra | 13:16 | |
| *** Ryan_Lane1 has joined #openstack-infra | 13:17 | |
| *** Ryan_Lane has quit IRC | 13:17 | |
| *** burt has joined #openstack-infra | 13:18 | |
| *** dina_belova has quit IRC | 13:19 | |
| *** dina_belova has joined #openstack-infra | 13:22 | |
| *** ruhe has quit IRC | 13:28 | |
| *** boris-42 has quit IRC | 13:29 | |
| *** boris-42 has joined #openstack-infra | 13:30 | |
| *** changbl has quit IRC | 13:34 | |
| *** pentameter has joined #openstack-infra | 13:35 | |
| *** jpeeler has joined #openstack-infra | 13:37 | |
| *** afazekas has joined #openstack-infra | 13:37 | |
| *** jpeeler has quit IRC | 13:37 | |
| *** jpeeler has joined #openstack-infra | 13:37 | |
| *** anteaya has quit IRC | 13:41 | |
| *** anteaya has joined #openstack-infra | 13:41 | |
| *** nati_ueno has quit IRC | 13:45 | |
| *** ruhe has joined #openstack-infra | 13:51 | |
| *** dkliban has quit IRC | 13:57 | |
| *** Ryan_Lane1 has quit IRC | 14:00 | |
| *** markmcclain has joined #openstack-infra | 14:00 | |
| *** blamar has joined #openstack-infra | 14:02 | |
| *** zul has joined #openstack-infra | 14:02 | |
| mordred | chmouel: no. it should not try to install stuff in /usr/local - that would be a bug if it's doing that | 14:02 |
| mordred | chmouel: but that paste looks like the local sdist command in /opt/stack/nova is trying to run in a directory where devstack has already run python setup.py develop - so the egg-info dir is now owned by root | 14:03 |
| mordred | chmouel: I thought I put a patch in to devstack already to chown those directories back once it's done | 14:03 |
| fungi | mordred: couple quick pbrish questions to bounce off you when you have a sec... | 14:04 |
| *** ruhe has quit IRC | 14:04 | |
| mordred | fungi: shoot | 14:06 |
| fungi | i was toying with using git to do tarball validation by checking out the ref on which the tarball is based and then untarring it into the working tree and having git report changed files | 14:07 |
| fungi | tested with nova and it *almost* works perfectly, with a couple packaging-related differences | 14:07 |
| fungi | one is that PKG-INFO appears in the tarball. should it be safe to start adding that to everyone's .gitignore? | 14:07 |
| *** fifieldt has quit IRC | 14:08 | |
| fungi | the other is that something in the sdist is making whitespace-only changes to setup.cfg... what would you guess is the culprit, is it necessary, and if so do we want a hacking rule to check that people are formatting their setup.cfg correctly? | 14:09 |
| fungi | and before you ask why i care, i was pondering the fact that we build our release artifacts on untrusted jenkins slaves (because we run code from those repos to generate them), but if we get around to setting up some signing infrastructure it would be great to have a trusted slave test that the tarballs actually haven't been tampered with | 14:10 |
| *** ruhe has joined #openstack-infra | 14:11 | |
| openstackgerrit | Mark McLoughlin proposed a change to openstack/requirements: Allow use of oslo.messaging 1.2.0a3 https://review.openstack.org/41696 | 14:11 |
| *** rnirmal has joined #openstack-infra | 14:12 | |
| chmouel | mordred: I think I haven't done more than a stack.sh and python setup.py develop in nova, nothing as root, so perhaps it's in devstack | 14:13 |
| mordred | chmouel: yeah - devstack does sudo python setup.py develop | 14:13 |
| mordred | chmouel: (this is another thing that would be made easier by having a global venv) | 14:14 |
| *** dina_belova has quit IRC | 14:14 | |
| chmouel | mordred: yeah so is there technichal impediment to global venv? | 14:14 |
| mordred | chmouel: speaking of which - https://review.openstack.org/#/c/40534/ is a patch for that which fails when it gets to swift | 14:15 |
| mordred | chmouel: basically, getting the patch to work is step one :) | 14:15 |
| *** yolanda has joined #openstack-infra | 14:15 | |
| chmouel | mordred: :) | 14:15 |
| mordred | fungi: yes. add PKG-INFO to everyone's gitignore | 14:15 |
| *** cppcabrera has joined #openstack-infra | 14:16 | |
| mordred | I believe setuptools 'helpfully' makes changes to setup.cfg (remind me to see if I can stub out the part of setuptools that does that in pbr) | 14:16 |
| mordred | fungi: but if you can figure out what it is that setuptools wants to do differently in our setup.cfg, then sure, a hacking rule sounds sensible | 14:17 |
| mordred | and that seems like a sensible thing to do - especially when you consider that we do have signatures on the git tags too | 14:17 |
| fungi | mordred: great, i'll open a pbr bug about the "helpful" setup.cfg normalization and, if someone deems fixing it is impossible/unreasonable then i'll write a hacking rule | 14:17 |
| chmouel | mordred: looking at it, do you have by any chance the stack trace? digging into the logs is like doing the same jump as that guy http://openstackreactions.enovance.com/wp-content/uploads/2013/08/6721699501.gif | 14:17 |
| yolanda | hi, i'm doing a test deployment with zuul, and having this error: ERROR zuul.Gearman: Job <gear.Job 0x7f37f005af50 handle: None name: build:gate-noop unique: 7fc91b2bb38047a8831f13f83a8dbfb1> is not registered with Gearman | 14:19 |
| fungi | mordred: right, eventual workflow i imagine is that the trusted upload slave has a list of keys of known signers so it can validate the tag, then retrieves and compares the tarball to make sure it's not tampered with, then generates a signature of the tarball with a dedicated archive signing key, then uploads the tarball and detached signature to pypi and wherever else we like | 14:19 |
| yolanda | do i need to configure gearman in some way, do i need to configure the jobs? | 14:19 |
| mordred | chmouel: not yet - I haven't been able to sort out where it's unhappy yet - I'll see if I can narrow it down and send you more specific thoughts | 14:19 |
| chmouel | mordred: thanks let me know if i can be any help | 14:20 |
| mordred | fungi: yah | 14:20 |
| fungi | yolanda: i believe you configure the jenkins-gearman plugin once installed in jenkins to tell it where to find zuul, at least | 14:20 |
| yolanda | fungi, i did it, installed jenkins with gearman plugin, and i configured that properly | 14:21 |
| yolanda | and i have a job in jenkins called gate-noop, not sure if that's right | 14:21 |
| fungi | yolanda: lemme refresh my memory of our zuul config to see if it needs to know to expect gearman connections from specific addresses | 14:22 |
| *** dnavale has quit IRC | 14:22 | |
| yolanda | fungi, i configured zuul to setup gearman address like that | 14:22 |
| yolanda | [gearman] | 14:22 |
| yolanda | server=10.55.32.117:4730 | 14:22 |
| yolanda | [gearman_server] | 14:22 |
| yolanda | start=false | 14:22 |
| yolanda | gearman is correctly listening in that port | 14:23 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Add testr parallel run to gate pipeline nonvoting. https://review.openstack.org/41698 | 14:24 |
| fungi | yolanda: i believe that tells zuul where to find the gearman server (we simply set ours to 127.0.0.1 letting it know to connect to itself) | 14:24 |
| * fungi reviews the zuul config documentation | 14:25 | |
| yolanda | fungi, we have our gearman server in a separate instance | 14:27 |
| *** datsun180b has joined #openstack-infra | 14:27 | |
| fungi | yolanda: current doc at http://ci.openstack.org/zuul/zuul.html#configuration suggests that server is the dns name or ip address of the gearman server to which you wish to connect, and then there's a separate port option if you want it to use something other than the default gearman service port number | 14:27 |
| fungi | and i guess if you're using an external gearman server, then you probably also want to make sure you don't set gearman_server.start in the zuul.conf | 14:28 |
| *** rcleere has quit IRC | 14:28 | |
| fungi | though i don't think it would break anything to have it running, it wouldn't be needed' | 14:30 |
| *** cppcabrera has left #openstack-infra | 14:30 | |
| yolanda | fungi, i'll update that in config file | 14:30 |
| fungi | and presumably your jenkins-gearman plugin is configured to connect to the same gearman server as you told zuul to connect to | 14:30 |
| yolanda | gearman_server is set to start=false, but server and port were wrong | 14:30 |
| yolanda | and yes, jenkins-gearman-plugin is connecting to the same gearman-server | 14:31 |
| fungi | yolanda: start=false is the default, according to the documentation, so you could also just leave the entry out of the config entirely | 14:31 |
| fungi | whatever your preference | 14:31 |
| yolanda | fungi, so that should be enough? do we need to define the jobs in jenkins, for example the gate-noop? | 14:33 |
| fungi | yolanda: yes, jenkins needs to know what the job should do. we use a separate tool called jenkins-job-builder to create/update jobs on our jenkins masters from a controlled set of templatized yaml | 14:34 |
| *** mrmartin has quit IRC | 14:35 | |
| yolanda | fungi, i configured manually a gate-noop job that simply does an echo "SUCCESS" but it doesn't seem to be called | 14:35 |
| fungi | but the no-op job we have just tells a slave to run /bin/true (so that we at least exercise that a worker got the job and ran something) | 14:36 |
| anteaya | chmouel: I see you are speaking in gifs, I have to say I like gif-speak better than acronyms | 14:37 |
| yolanda | fungi, so even that gate-noop isn't running then | 14:37 |
| fungi | yolanda: it's probably time to start peering into the zuul logs then to see whether it got a trigger it associates with that job, whether the job got queued, et cetera | 14:38 |
| yolanda | fungi, now that we have the infrastructure more clear, we'll retry the change | 14:39 |
| fungi | jeblair may also have some more specific troubleshooting suggestions | 14:39 |
| yolanda | thx | 14:39 |
| fungi | you're welcome | 14:39 |
| fungi | and also, i'm very excited to see yet another organization starting to try out and possibly use zuul | 14:40 |
| jeblair | yolanda: try "telnet 10.55.32.117 4730" and then type "status" | 14:40 |
| fungi | there are only a few so far | 14:40 |
| jeblair | yolanda: that should list the functions defined on the gearman server | 14:40 |
| *** changbl has joined #openstack-infra | 14:41 | |
| *** mrmartin has joined #openstack-infra | 14:41 | |
| yolanda | build:gate-noop002 | 14:41 |
| yolanda | . | 14:41 |
| yolanda | so that gate-noop is defined | 14:41 |
| jeblair | yolanda: then it sounds like a problem with zuul connecting with the gearman server; have you restarted since fixing the configuration? | 14:43 |
| *** dkliban has joined #openstack-infra | 14:43 | |
| yolanda | jeblair, we will test a new patchset now | 14:44 |
| *** dina_belova has joined #openstack-infra | 14:44 | |
| yolanda | http://91.189.93.38:8080/job/gate-noop/1/ : SUCCESS in 0s | 14:46 |
| yolanda | better now | 14:46 |
| jeblair | yolanda: yay! | 14:47 |
| *** ruhe has quit IRC | 14:47 | |
| yolanda | jeblair, is there any minimum requirement for gearman-server? i tried yesterday with old version from precise package and even wasn't triggering error messages. I manually built gearman-server now with the latest tarball and we saw some progress | 14:48 |
| Bobba | does Zuul pick up a change that has been moved from draft to published without any other changes? | 14:48 |
| fungi | Bobba: you need to leave a comment of "recheck no bug" on the published patchset | 14:49 |
| fungi | Bobba: the version of gerrit we're running does not emit any events related to draft changes, not even an event to signal that a draft has been published | 14:49 |
| Bobba | fair enough | 14:49 |
| jeblair | yolanda: it actually needs (works best with) the 'gear' server (which is what it runs if you tell it to use the 'internal' server, or you can run it separately with geard'). there are two reasons: | 14:50 |
| Bobba | ohhhhhh okay | 14:50 |
| fungi | Bobba: subsequent patchsets, if uploaded normally (not as drafts) will be tested as usual | 14:50 |
| jeblair | yolanda: cancel queue items -- that should be in the latest tarball | 14:50 |
| jeblair | (of the gearman server) | 14:50 |
| Bobba | thanks fungi | 14:50 |
| fungi | Bobba: we also recommend using the "work in progress" button to indicate a patch is not ready for review, instead of drafts (because they are broken in a number of ways) | 14:51 |
| jeblair | yolanda: and strict fifo ordering of the queue -- if you use gearman-server and enable round-robin, it should be mostly okay, but it still won't quite run jobs in the order zuul tells it to | 14:51 |
| yolanda | jeblair, so you don't use gearman-job-server? | 14:52 |
| jeblair | yolanda: so basically -- the latest gearman-server tarball should mostly work (but is untested) and should be fine if you don't have much contention for slaves, but if you do, the jobs may run in a less-optimal server | 14:52 |
| Bobba | fungi: can git review upload a WIP change? I was uploading them as draft | 14:52 |
| Bobba | fungi: or is the recommendation to upload then press the button ;) | 14:52 |
| jeblair | yolanda: no, we use geard; it's included in python-gear -- it's a simple server that just does enough for zuul. | 14:52 |
| chmouel | is anyone knows how i can have the equivalent of -s in testr? | 14:52 |
| chmouel | anteaya: :) | 14:52 |
| anteaya | :D | 14:52 |
| jeblair | yolanda: i would like to patch gearman-job-server to support fifo queues, but have not had time | 14:53 |
| yolanda | jeblair, good to know, thx | 14:53 |
| fungi | Bobba: we have a wishlist bug open against git-review to put a wip option in, but since we're hopefully soon upgrading to a newer gerrit, we're heavily reworking how wip works anyway and would need to change the implementation in git-review to match | 14:53 |
| Bobba | understood. Thanks fungi | 14:54 |
| fungi | Bobba: there is also a gerrit ssh api call which can set wip status, if you want to alias that command. i'll dig up the link to the documentation | 14:54 |
| Bobba | nah - I can do it manually the few times I do it1 | 14:54 |
| mordred | fungi: there _used_ to be a gitreview feature | 14:55 |
| mordred | it got lost in the undo of the git-review refactoring | 14:55 |
| *** rcleere has joined #openstack-infra | 14:56 | |
| anteaya | chmouel: what funtionality are you looking for from -s? (I don't know the context) | 14:56 |
| mordred | but I agree - waiting for new WIP would be better | 14:56 |
| chmouel | anteaya: have testr show the prints | 14:56 |
| anteaya | ah | 14:56 |
| mordred | chmouel: so, all of the prints are going into a detail record and sould be in the output if the test _fails_ | 14:57 |
| mordred | however ... there's a couple of things you can do (and we're working on getting this better served by the ui) | 14:57 |
| chmouel | mordred: nice thanks. | 14:58 |
| mordred | chmouel: if you want to run just a single test and not capture any of the output: | 14:58 |
| mordred | chmouel: python -m testtools.run python.path.to.test.function | 14:58 |
| chmouel | perfect that works | 14:59 |
| *** mrodden1 has quit IRC | 14:59 | |
| *** mrmartin has quit IRC | 15:04 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Create repo when we create the cgit config file https://review.openstack.org/41279 | 15:07 |
| mordred | pleia2: fixed your reviews ^^ thans! | 15:08 |
| mordred | fungi: pleia2 says she's tested that that does what we expect it to | 15:08 |
| mordred | pleia2: did you test (or do we know) if this is going to create the repos with the right owner? | 15:08 |
| *** _TheDodd_ has joined #openstack-infra | 15:09 | |
| anteaya | hey _TheDodd_ | 15:10 |
| _TheDodd_ | Hey, anteaya! | 15:10 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Update Heat core team https://review.openstack.org/41644 | 15:10 |
| _TheDodd_ | How have you been? | 15:10 |
| anteaya | good thanks | 15:10 |
| anteaya | busy as always, and yourself? | 15:11 |
| fungi | mordred: too awesome--i meant to review it, so now intend to do so when the call i'm on wraps up | 15:11 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Add requirements to note that paramiko is needed https://review.openstack.org/39874 | 15:11 |
| _TheDodd_ | Great! Busy as well. | 15:11 |
| *** mrodden has joined #openstack-infra | 15:11 | |
| anteaya | good | 15:11 |
| anteaya | when you get a chance, skim the channel logs | 15:11 |
| _TheDodd_ | Any thing wild and crazy going on these days, anteaya? | 15:12 |
| _TheDodd_ | Ok, will do. | 15:12 |
| anteaya | I do believe that jeblair has some thoughts about the bug labelling patch | 15:12 |
| _TheDodd_ | -infra or ref? | 15:12 |
| anteaya | -infra | 15:12 |
| _TheDodd_ | That's cool. | 15:12 |
| anteaya | thought you would want to know | 15:12 |
| _TheDodd_ | Definitely. | 15:12 |
| anteaya | :D | 15:13 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Make reviewers output line up https://review.openstack.org/41645 | 15:15 |
| anteaya | _TheDodd_: timestamps 2013-08-12T23:55:06 to 2013-08-13T00:18:28 | 15:15 |
| *** hughsaunders has quit IRC | 15:16 | |
| anteaya | _TheDodd_: those are UTC timestamps | 15:18 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/reviewstats: Add TripleO projects https://review.openstack.org/41708 | 15:22 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/reviewstats: Updated Ironic core team https://review.openstack.org/41709 | 15:22 |
| jpmelos | Guys, is there a way to retrigger jobs in Zuul for patches that are already open in Gerrit? | 15:22 |
| jpmelos | Or manually trigger them. =) | 15:22 |
| Bobba | jpmelos: recheck no bug | 15:22 |
| jpmelos | Bobba: I didn't understand. | 15:23 |
| Bobba | jpmelos: add it as a comment to the gerrit review | 15:23 |
| dstufft | mordred: I'm fucking with SAT solvers to try and make a real dep resolver for pip | 15:23 |
| jpmelos | I just need to add that string as a comment in Gerrit? | 15:24 |
| Bobba | yes | 15:24 |
| fungi | dstufft: the entire python world will worship you if you make that happen | 15:24 |
| dstufft | fungi: current status is trying to understand SAT solvers :V I think I have a small basic example figured out though | 15:25 |
| mordred | dstufft: oh my. that's gota be fun | 15:25 |
| jpmelos | Bobba: Didn't seem to work. | 15:25 |
| anteaya | jpmelos: can you post the url of the patch in question, please? | 15:26 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Add TripleO projects https://review.openstack.org/41708 | 15:26 |
| jeblair | mordred, fungi, clarkb: a review on this: https://review.openstack.org/#/c/41565/ would help set the direction of my work today. | 15:26 |
| mordred | jeblair: I will review it right now! | 15:26 |
| jpmelos | anteaya: Unfortuantely, I can't. It's inside the company VPN only. | 15:26 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Updated Ironic core team https://review.openstack.org/41709 | 15:26 |
| dstufft | mordred: it's.. interesting. as far as I can tell all the SAT solvers are written in things like C or C++, so I'm trying to decipher one of them and write a python implementation. | 15:27 |
| mordred | jpmelos: we have the retrigger functionality expressed in our zuul layout config | 15:27 |
| Bobba | jpmelos: ahhhh - the rules that cause zuul to trigger a new job are in the config file - perhaps they aren't set up in the same way as for OpenStack? | 15:27 |
| jeblair | mordred, fungi, clarkb: https://review.openstack.org/#/c/41517/ is step 2 of "get rid of d-g inprogress jobs" | 15:27 |
| jpmelos | Bobba: Haha, OK. I'll take a look at openstack layout and mimic that. | 15:28 |
| *** ruhe has joined #openstack-infra | 15:28 | |
| mordred | jeblair: +2 on nodepool from me | 15:28 |
| mordred | jeblair: and I love the idea of it being its own thing | 15:28 |
| jeblair | mordred: good, i hoped you would. i'm not sure where the dib elements should end up living; do you have thoughts on that? | 15:29 |
| jeblair | mordred: i expect to put the equivalent scripts in the nodepool repo for now. | 15:30 |
| anteaya | jpmelos: ah okay, well for conversation's sake, if you were to add the recheck no bug comment, and I had the url of the patch, I could go to zuul's status page at http://status.openstack.org/zuul and confirm your findings | 15:30 |
| mordred | jeblair: I think the elements can probably live with the scripts until we're happy with how it all works for expediency - but I think that eventually the elements probably just want to be their own repo? | 15:31 |
| jeblair | mordred: ok. or maybe even a directory in infra/config? i'm thinking we may accumulate several elements for several different node types | 15:32 |
| mordred | jeblair: yes. I'm imagining we will - and I also thought about infra/config ... although I'd love to continue the thought of breaking that up more so that we could give more fine grained access (like I'd love to give russellb and pabelanger +2 on the asterisk stuff) | 15:33 |
| mordred | jeblair: but my feelsings are 100% mushy on this topic | 15:33 |
| *** hughsaunders has joined #openstack-infra | 15:33 | |
| russellb | mordred: can always just give honorary +2 via +1 | 15:34 |
| russellb | mordred: oh russellb reviewed it already, so i don't really have to look quite as closely, or whatever | 15:34 |
| mordred | russellb: your +1 are always honorary +2 :) | 15:34 |
| russellb | heh | 15:34 |
| russellb | sounds like less work that way, heh | 15:35 |
| russellb | not that I want to stifle ifra improvements! you guys work magic | 15:35 |
| jeblair | mordred: yeah. it's a bit hard for me to imagine that we'd end up with core teams dedicated to a single dib template each; and that they would change enough for that to be an issue; maybe infra/config or a single new repo as a starting point and see how it goes. anyway, that's still like 10 steps ahead of where we are. | 15:35 |
| mordred | jeblair: so many steps! | 15:35 |
| mordred | jeblair: wow. I clearly have not been following your work closely enoguh | 15:35 |
| mordred | "Use the new gearman-plugin single-use feature" | 15:36 |
| *** vogxn has joined #openstack-infra | 15:36 | |
| jeblair | mordred: :) it marks a node offline after the job completes. | 15:37 |
| mordred | jeblair: remember when we wanted the jclouds plugin to be able to do that? | 15:37 |
| jeblair | mordred: it should make things nicer with our current scripts (less load and race conditions with the inprogress jobs), and will also make the nodepool workflow better. | 15:38 |
| jeblair | mordred: good times | 15:38 |
| mordred | yeah. I love the idea. I have +2'd your patches | 15:39 |
| jeblair | mordred: you should subscribe to my newsletter | 15:39 |
| pleia2 | good morning | 15:39 |
| mordred | jeblair: can I use that to purchase your goods and services? | 15:39 |
| anteaya | pleia2: morning | 15:42 |
| mordred | morning pleia2 | 15:42 |
| pleia2 | mordred: it's logging in with the cgit user for replication over ssh and that's the owner we want, should be good | 15:44 |
| mordred | pleia2: great! | 15:47 |
| pleia2 | and hopefully replication is actually trying to put them in the project.git directory for each project, that's the only thing I'm not 100% on | 15:48 |
| Bobba | Any chance I could get reviews of https://review.openstack.org/#/c/41464/? I (very embarassingly) broke XenServer support in DevStack so getting the fix in quickly would help me save a little face ;) | 15:48 |
| *** pcrews has quit IRC | 15:49 | |
| *** ruhe has quit IRC | 15:51 | |
| anteaya | Bobba: is that patch restricted to certain users? because I am unable to view it | 15:57 |
| openstackgerrit | Jason Meridth proposed a change to openstack-dev/hacking: Adds ability to ignore H231, python3x except compatability check https://review.openstack.org/41713 | 15:57 |
| *** ruhe has joined #openstack-infra | 15:58 | |
| pleia2 | mordred: patch looks good but I think it needs a recheck | 15:59 |
| Bobba | very weird anteaya ... | 15:59 |
| pleia2 | anteaya: if you copied the ? at the end of the url too, it won't work | 16:00 |
| pleia2 | anteaya: just https://review.openstack.org/#/c/41464/ | 16:00 |
| anteaya | that was it, I had just selected the link and clicked it | 16:00 |
| anteaya | removing the ? at the end shows me the patch, thanks pleia2 | 16:00 |
| Bobba | anteaya: Sorry! | 16:01 |
| clarkb | morning | 16:01 |
| pleia2 | morning clarkb | 16:01 |
| clarkb | jeblair will review shortly | 16:01 |
| Bobba | my fault :/ of course ?'s are part of a URL! | 16:01 |
| Bobba | :) | 16:01 |
| anteaya | Bobba: we both learned something, no worries | 16:01 |
| *** nicedice_ has joined #openstack-infra | 16:01 | |
| anteaya | morning clarkb | 16:01 |
| jgriffith | mordred: ping | 16:02 |
| mordred | jgriffith: it wasn't me! | 16:03 |
| *** sarob has joined #openstack-infra | 16:03 | |
| jgriffith | mordred: haha.. probably not, but maybe you can help :) | 16:03 |
| jgriffith | mordred: http://paste.openstack.org/show/44044/ | 16:03 |
| jgriffith | mordred: I've been having fits the past week with installing devstack | 16:03 |
| jgriffith | mordred: thought I had most of it sorted... but now I'm hitting this one | 16:03 |
| mordred | jgriffith: it's been a bit of a nightmare - looking | 16:03 |
| anteaya | Bobba: I would like to test this before I vote, do I have time? | 16:04 |
| mordred | jgriffith: is this on an old devstack machine which you may have unstack/restacked a few times? | 16:04 |
| Bobba | anteaya: of course :) There's always time for testing | 16:04 |
| anteaya | Bobba: k, thanks. Testing. | 16:05 |
| jgriffith | mordred: Not really | 16:05 |
| jgriffith | mordred: So here's the deal | 16:05 |
| jgriffith | mordred: my vagrant fails due to the chmod issue | 16:05 |
| jgriffith | mordred: so I log in do the chmod and run stack.sh again | 16:05 |
| jgriffith | mordred: so it's kinda what you suspect | 16:05 |
| jgriffith | mordred: the re-stacking has been the crux of issues at this point | 16:05 |
| *** dripton_ is now known as dripton | 16:06 | |
| mordred | oh. blerg. well that's unpleasant. try this and tell me if it fixes it, and if so I'll try to add it to devstack logic | 16:06 |
| mordred | whatI want to know is why the pbr in /opt/stack/pbr is reporting: pbr 0.5.17.10.gce11c73 (/opt/stack/pbr), | 16:07 |
| jgriffith | mordred: FYI, I think I can work around it by 'pip unisntall pbr' and rerun, but not sure it's 100% | 16:07 |
| jgriffith | mordred: lemme look | 16:07 |
| mordred | jgriffith: I'd actually try "cd /opt/stack/pbr ; rm -rf *egg-info; git pull --ff-only ; sudo pip install -e ." | 16:07 |
| mordred | but more interesting is why you have old pbr in /opt/stack/pbr | 16:08 |
| jgriffith | mordred: k | 16:08 |
| jgriffith | mordred: yeah, that I don't know :( | 16:08 |
| mordred | if this is a new vagrant thing - are you caching repos anywhere? | 16:08 |
| jgriffith | mordred: I thought that might be the issue so I did a nuke on my entire vagrant dir | 16:08 |
| jgriffith | mordred: but it's possible it's pickign something up somewhere else | 16:09 |
| *** mrodden has quit IRC | 16:09 | |
| * jgriffith has so much crap on his laptop who knows | 16:09 | |
| mordred | what does the vagrant file thing look like? (I've never gotten vagrant to work- but I can tak e a look and see if I see anything) | 16:09 |
| *** psedlak has quit IRC | 16:10 | |
| jgriffith | mordred: hmmm "You asked me to pull without telling me which branch...." | 16:11 |
| jgriffith | mordred: that's different | 16:11 |
| mordred | oh. fascintating. that means it's not on a branch | 16:11 |
| mordred | git branch output? | 16:11 |
| jgriffith | mordred: indeed... but it git status says master | 16:11 |
| *** markmc has quit IRC | 16:11 | |
| mordred | wait - why is that in /opt/stack? shouldn't it be in /opt/stack/new? | 16:12 |
| jgriffith | mordred: ha! sure'nuff | 16:12 |
| jgriffith | mordred: nahh.. not running grenade | 16:12 |
| mordred | oh. k | 16:12 |
| jgriffith | mordred: so you're correct though, "git branch output" shows nada | 16:12 |
| mordred | no - I jus tmeant git branch | 16:13 |
| mordred | like "show me the output of 'git branch'" | 16:13 |
| mordred | sorry - I'm being too terse | 16:13 |
| jgriffith | derp! | 16:15 |
| jgriffith | says it's on master | 16:15 |
| jgriffith | now I've created "output" hehe | 16:15 |
| * jgriffith is just copy/pasting instructions mindlessly | 16:16 | |
| mordred | hehe. it's what we all do around devstack breaks | 16:16 |
| jgriffith | mordred: so I can delete and reclone if that's useful | 16:16 |
| mordred | yeah. try that | 16:16 |
| jgriffith | kk | 16:16 |
| *** comstud_ is now known as comstud | 16:16 | |
| mordred | and then setup.py develop or pip install -e it | 16:16 |
| *** mrodden has joined #openstack-infra | 16:18 | |
| jgriffith | mordred: well the version appears to be updated at least :) | 16:18 |
| mordred | yay! | 16:18 |
| mordred | now - why was it old before... | 16:18 |
| jgriffith | Version: 0.5.21.6.ge2821d1 | 16:18 |
| jgriffith | that I don't know... looking in vagrant cache | 16:19 |
| *** blamar has quit IRC | 16:20 | |
| *** hughsaunders has left #openstack-infra | 16:20 | |
| *** sarob has quit IRC | 16:21 | |
| jgriffith | mordred: must be vagrant related | 16:22 |
| anteaya | jgriffith: what version of vagrant are you using? | 16:22 |
| jgriffith | mordred: talked to thingee and he did a fresh spin up with no issues | 16:22 |
| jgriffith | anteaya: 1.2 | 16:22 |
| anteaya | I just downloaded 1.2.7 yesterday | 16:23 |
| anteaya | so far so good here | 16:23 |
| jgriffith | anteaya: .7 ??? | 16:23 |
| jgriffith | hmmm | 16:23 |
| jgriffith | I'm on 1.2.2 | 16:23 |
| anteaya | 'tis new | 16:23 |
| anteaya | don't know if that is it or not | 16:23 |
| jgriffith | anteaya: and you're not seeing any issues? | 16:23 |
| anteaya | since yesterday, not so far | 16:23 |
| fungi | dstufft: from what i've seen sat solvers are usually written in low-level languages for performance reasons. they tend to be very resource-intensive when solving for large sets | 16:23 |
| jgriffith | anteaya: did you add the chmod to your vagrant files? | 16:23 |
| anteaya | I didn't add chmod to vagrant files | 16:24 |
| mordred | jgriffith: I'm wondering if it's doing anything smart like looking for git repos on your local machien and pre-copying those in? | 16:24 |
| anteaya | just increased the RAM available | 16:24 |
| jgriffith | mordred: I suspect that may be the case | 16:24 |
| jgriffith | mordred: I'm going to nuke my cache directory again and see what happens, but it's odd I've not encountered this before | 16:24 |
| jgriffith | although now i want to try 1.2.7 thanks to anteaya :) | 16:25 |
| anteaya | ha ha ha, I'm a bad influence | 16:25 |
| jgriffith | indeed :) | 16:25 |
| dstufft | fungi: yea | 16:25 |
| dstufft | fungi: But that's unacceptable for my use cases :) Needs to be pure python. And luckily the sets aren't going to be very large | 16:25 |
| anteaya | here is my list of instructions I follow for vagrant: https://gist.github.com/anteaya/dedca991f761c48bb262 | 16:26 |
| anteaya | not very pretty, so be cautious | 16:26 |
| fungi | dstufft: i'm definitely eager to see the resulting implementation, and see what performance you get from it | 16:26 |
| Bobba | gah! Sorry dtroyer ... I hate hard tabs! | 16:26 |
| jgriffith | anteaya: ahhhh, I actually have cook-books to do devstack, that could be my problem :) | 16:27 |
| Bobba | Can't we have a check for them? :) And what sneaky side effects are you thinking of that we might have to look for? | 16:27 |
| anteaya | jgriffith: perhaps, would have to see the cook-books | 16:27 |
| jgriffith | mordred: anywho... deleted cache and running again | 16:27 |
| jgriffith | mordred: I'll let ya know what happens | 16:27 |
| mordred | jgriffith: cool. I'll be interested to know! | 16:28 |
| jgriffith | anteaya: https://github.com/j-griffith/vagrant_devstack | 16:28 |
| *** sarob has joined #openstack-infra | 16:28 | |
| jgriffith | anteaya: there's a sub-repo there for the cookbooks | 16:28 |
| anteaya | grand, while devstack is stacking, I shall peruse | 16:29 |
| anteaya | jgriffith: are you on OSX? | 16:30 |
| jgriffith | anteaya: nahh... ubuntu | 16:31 |
| anteaya | jgriffith: k, did you install vagrant as a gem? | 16:31 |
| jgriffith | anteaya: nope, pkg | 16:32 |
| anteaya | k | 16:33 |
| *** cppcabrera has joined #openstack-infra | 16:33 | |
| anteaya | same as me so far, I installed both virtualbox and vagrant from the .debs | 16:33 |
| *** cppcabrera has left #openstack-infra | 16:33 | |
| *** sarob has quit IRC | 16:34 | |
| pleia2 | fungi: if you have some time later, I could use some guidance for getting my slides committed, and we are we planning to have an index at http://docs.openstack.org/infra/publications/ right? | 16:34 |
| *** sarob has joined #openstack-infra | 16:34 | |
| fungi | pleia2: there is a script already in place which should generate it, but i think it still may not be 100% working | 16:34 |
| pleia2 | fungi: ok, I thought it was something like that, I'll take a look | 16:34 |
| clarkb | dstufft: fungi: too bad z3 isn't licensed in a friendly way and I think it depends on C :/ | 16:35 |
| fungi | pleia2: the script to generate the index lives in the master branch, the different presentations live in their own branches, and the theory is that when you give a talk you tag the commit on that branch (which may or may not include further changes on top of the previous time it was given) | 16:36 |
| dstufft | clarkb: fungi it's ok I found ministat which is 900 lines of C++, so I'm gonna try and port that to Python :V | 16:36 |
| fungi | pleia2: i don't think we've gotten to the point of tagging anything yet though | 16:37 |
| pleia2 | fungi: oh, so just standard branching we do with commits anyway, but we'll actually continue using them for each presentation | 16:37 |
| anteaya | fwiw I find chef recipes easier to grok than puppet manifests, on first glance | 16:37 |
| clarkb | anteaya: probably because chef recipes are just ruby | 16:38 |
| *** psedlak has joined #openstack-infra | 16:38 | |
| fungi | i find ansible playbooks to be the most readable of any config management system i've seen so far | 16:39 |
| anteaya | clarkb: yes, that is probably it, I am finding myself just reading them | 16:39 |
| clarkb | jeblair: https://review.openstack.org/#/c/41517/ LGTM and mordred and fungi. I will let you approve when you are ready | 16:39 |
| anteaya | jgriffith: I don't do any of the actions in this file when I set up my vagrant: https://github.com/j-griffith/vagrant-openstack/blob/master/templates/default/hostname.erb | 16:39 |
| Bobba | dprince: ping! | 16:40 |
| *** Bobba is now known as BobBall | 16:40 | |
| *** ruhe has quit IRC | 16:41 | |
| dprince | BobBall: hi | 16:41 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add nodepool project https://review.openstack.org/41565 | 16:42 |
| BobBall | Looking at that smokestack failure... I'm very confused... Is it every run now, or is it only a subset? i.e. does it seem random? | 16:42 |
| anteaya | jgriffith: it is my understanding that vagrant sets the permissions inside the vm the way it needs, this cookbook was last edited a year ago, I wonder if vagrant has changed the way it sets permissions since then? | 16:42 |
| *** gyee has joined #openstack-infra | 16:42 | |
| BobBall | dprince: It seems to be sporadic but I can't explain that | 16:42 |
| BobBall | dprince: so I'm hoping I'm reading the status pages wrong or it might be per-host or something like that | 16:43 |
| *** svarnau has joined #openstack-infra | 16:43 | |
| *** ruhe has joined #openstack-infra | 16:43 | |
| *** jerryz has joined #openstack-infra | 16:44 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Remove unneeded node label parameters https://review.openstack.org/41516 | 16:44 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Make all devstack-vm jobs single-use https://review.openstack.org/41517 | 16:44 |
| *** yaguang has quit IRC | 16:44 | |
| *** HenryG_ has quit IRC | 16:44 | |
| dprince | BobBall: Yeah. I'm perplexed by this one. | 16:45 |
| *** HenryG has joined #openstack-infra | 16:45 | |
| BobBall | dprince: other question - how did you get the SS URL in the email? Did you write it by hand or is there a magic link on smokestack.openstack.org I've not found yet that I can copy :D | 16:45 |
| BobBall | dprince: oh... and now I see it | 16:45 |
| BobBall | stupid Bob | 16:45 |
| fungi | clarkb: i took a stab at your textwrap suggestion for git-review last night... https://review.openstack.org/40800 | 16:45 |
| BobBall | sorry | 16:45 |
| anteaya | jgriffith: here is my current Vagrantfile: https://gist.github.com/anteaya/4c96a59008d26110f948 | 16:45 |
| pleia2 | fungi: hm, actually, I see, overview is it's own branch and not merged with master like we typically would, how exactly would my commit differ from normal? | 16:46 |
| *** ^d has joined #openstack-infra | 16:46 | |
| fungi | pleia2: is your update for the overview presentation? | 16:46 |
| *** emagana has joined #openstack-infra | 16:46 | |
| fungi | pleia2: or a different presentation? | 16:46 |
| anteaya | jgriffith: these are the only edits I made to the file: https://gist.github.com/anteaya/4c96a59008d26110f948#file-my-vagrantfile-L47-L53 | 16:46 |
| BobBall | dprince: so it's all hosts, but only a subset of the nova changes - even though all of them should have that change in... | 16:47 |
| pleia2 | fungi: new presentation | 16:47 |
| dprince | BobBall: as far as I could tell yes. | 16:47 |
| dprince | BobBall: it has definately occurred on multiple hosts | 16:48 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Give tag upload permissions for os-apply-config https://review.openstack.org/41731 | 16:49 |
| fungi | pleia2: we should come up with a (preferably one-word) short name for it which we can use as a branch name | 16:49 |
| openstackgerrit | A change was merged to openstack-infra/config: Add nodepool project https://review.openstack.org/41565 | 16:49 |
| BobBall | dprince: do you restart XAPI on each run? | 16:49 |
| fungi | pleia2: and then we'll branch from a commit in your commit history common with the existing repo (likely your parent commit?) and then you can push to that branch | 16:50 |
| *** ladquin is now known as ladquin_afk | 16:50 | |
| mordred | fungi, jeblair: ^^ tag upload permission missing for the os-*-config projects and SpamapS needs to just a release | 16:50 |
| *** mrodden has quit IRC | 16:50 | |
| *** mrodden1 has joined #openstack-infra | 16:50 | |
| *** ftcjeff has joined #openstack-infra | 16:50 | |
| dprince | BobBall: not that I remember | 16:51 |
| clarkb | fungi: reviewed | 16:51 |
| dprince | BobBall: should we? | 16:51 |
| pleia2 | fungi: hm, that last bit is where I need some help (and to document it) | 16:51 |
| *** ruhe has quit IRC | 16:52 | |
| fungi | mordred: does os-apply-config-core expect to grow its membership substantially? should there be a smaller group of people who are familiar with signing and pushing tags to release it? | 16:52 |
| jgriffith | mordred: what version of PBR are you expecting? | 16:52 |
| BobBall | dprince: maybe - but I'm not clear how that'd cause this... thinking that it's been cached somewhere, possibly with an address or something which might account for the seemingly randomness... but then that isn't quite borne out by the code I'm looking at | 16:52 |
| mordred | jgriffith: 0.5.21 | 16:52 |
| mordred | jgriffith: is what's in tip of trunk | 16:53 |
| *** pcrews has joined #openstack-infra | 16:53 | |
| jeblair | fungi, mordred: right, we normally use the -ptl group for that. | 16:53 |
| anteaya | BobBall: I'm requesting a rebase | 16:53 |
| mordred | jeblair: should we go ahead and make one of those? | 16:53 |
| jeblair | mordred: yes | 16:53 |
| mordred | k. amending. I will also go learn more things | 16:53 |
| jeblair | mordred: to avoid cleaning up the result of bad tags | 16:53 |
| jgriffith | mordred: so interestingly enough, I nuked my cache and it appears as though that was in fact the issue | 16:53 |
| fungi | mordred: can't hurt. it mainly helps us not to have to help people unscrew their screwed-up release mistakes | 16:53 |
| *** fbo is now known as fbo_away | 16:53 | |
| fungi | er, what jeblair said | 16:53 |
| mordred | jgriffith: excellent. the cache logic seems unhappy then | 16:53 |
| BobBall | anteaya: Sure - done! | 16:54 |
| jgriffith | mordred: indeed, wonder if it doesn't go past the major number?? | 16:54 |
| jgriffith | mordred: anywho... thanks for walking through it with me | 16:54 |
| jeblair | mordred: the members of the -ptl group need to read and understand the instructions in https://wiki.openstack.org/wiki/GerritJenkinsGithub#Tagging_a_Release | 16:54 |
| mordred | SpamapS: ^^ | 16:54 |
| jgriffith | mordred: I'll let ya know if I discover anything else interesting on the topic | 16:54 |
| anteaya | BobBall: great, I'm destroying and starting again, will ping with results | 16:55 |
| mordred | jgriffith: my pleasure! and I'm glad it was something we were able to find | 16:55 |
| *** sarob has quit IRC | 16:55 | |
| jgriffith | anteaya: thanks to you as well :) | 16:55 |
| *** ftcjeff has quit IRC | 16:56 | |
| SpamapS | mordred: ok so -ptl isn't just the PTL | 16:56 |
| anteaya | glad to help, jgriffith | 16:56 |
| SpamapS | it is the PTL's peeps? | 16:56 |
| clarkb | jeblair: what is the difference between a computer and a node in jenkins? | 16:56 |
| jeblair | clarkb: beats the hell out of me | 16:56 |
| SpamapS | clarkb: a node may also be filled with puss? | 16:56 |
| mordred | SpamapS: yeah, well, really, the one thign the ptl grou pcan do is make releases | 16:56 |
| jeblair | clarkb: there are words in the api documentation | 16:56 |
| mordred | SpamapS: so, it's the people the ptl feels can handle the extra duties | 16:56 |
| jeblair | clarkb: they mean very little to me | 16:56 |
| fungi | SpamapS: we've used the term "ptl delegates" | 16:56 |
| SpamapS | sounds good to me | 16:57 |
| jeblair | clarkb: http://javadoc.jenkins-ci.org/hudson/model/Computer.html | 16:57 |
| jeblair | clarkb: http://javadoc.jenkins-ci.org/hudson/model/Node.html | 16:57 |
| mordred | ok. I'll do it with a ptl group for now, and we'll let you and lifeless duke it out over whether you should be in it or now | 16:57 |
| *** dkliban is now known as dkliban_afk | 16:57 | |
| fungi | SpamapS: so it could be just the ptl, or addtionally anyone to whom the ptl wishes to delegate release duties | 16:57 |
| clarkb | jeblair: thanks | 16:57 |
| jeblair | clarkb: i mean, i understand the sentences there; but if you can actually make heads or tails of why they are so different and yet nearly interchangable, i'd love to be enlightened. :) | 16:57 |
| SpamapS | mordred: I like my chances.. 15:22 <+lifeless> SpamapS: cut a release? | 16:58 |
| clarkb | seems fairly subtle, ut computers are more tightly related to executors which is what we are dealing with | 16:58 |
| SpamapS | hrm, the ? makes it less like an imperative | 16:58 |
| *** UtahDave has joined #openstack-infra | 16:58 | |
| SpamapS | :p | 16:58 |
| jeblair | clarkb: yeah, but then jenkins goes and asks whether a "node" can run a job. why did they chose that? a node without a computer can't run a job. | 16:58 |
| BobBall | anteaya: pls hold on a min - I'm rebasing again - somehow the sneaky tab got back in! | 16:58 |
| anteaya | BobBall: sneaky tab! okay thanks, will stand by | 16:59 |
| anteaya | just bringing up a fresh ubuntu | 16:59 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Set tag uploading permissions for TripleO https://review.openstack.org/41731 | 16:59 |
| fungi | mordred: No distributions at all found for pkginfo (from -r /home/jenkins/workspace/gate-jeepyb-pep8/requirements.txt (line 6)) | 16:59 |
| fungi | mordred: did we screw up jeepyb somehow and slip it past the gate? http://logs.openstack.org/79/41279/2/check/gate-jeepyb-pep8/585a90d/console.html | 17:00 |
| mordred | jeblair, fungi: ^^ that should be more like it - I also added dib while I was at it | 17:00 |
| clarkb | jeblair: and we still use the node to deal with project labels | 17:00 |
| BobBall | anteaya: Done! | 17:01 |
| fungi | mordred: so dib isn't going to use milestone-proposed workflow but just direct release tagging? | 17:01 |
| mordred | fungi: is it? I thought they were going to operate more like the client libs? | 17:02 |
| anteaya | BobBall: great, thanks | 17:02 |
| *** danger_fo_away is now known as danger_fo | 17:02 | |
| mordred | fungi: I'm willing to wait for lifeless to get up and verify | 17:02 |
| mordred | fungi: it's possibly I simple am mistaken | 17:02 |
| fungi | mordred: no idea--that's why i'm asking, since you removed their milestone-proposed management in the dib acl | 17:02 |
| fungi | just wanted to make sure that was their plan | 17:02 |
| mordred | fungi: good call. | 17:03 |
| mordred | lifeless_: ping when you get up | 17:03 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Test that custom functions are live-reloaded https://review.openstack.org/39551 | 17:03 |
| SpamapS | dib is going to be more like the client libs AFAIK too | 17:03 |
| fungi | mordred: i'm cool with taking SpamapS's word for it in this case, but up to you | 17:04 |
| *** ruhe has joined #openstack-infra | 17:05 | |
| *** mrodden has joined #openstack-infra | 17:05 | |
| clarkb | jeblair: for the test that is broken, is commenting it out preferable to removing it? | 17:05 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Fix json syntax errors https://review.openstack.org/41736 | 17:06 |
| jeblair | clarkb: i feel like it would be a useful test -- it would be nice actually if it could be used to validate, say, that this change doesn't completely break assumptions about node labels, etc. | 17:06 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Fix json syntax errors https://review.openstack.org/41736 | 17:06 |
| clarkb | jeblair: do you intend to try and fix it in a new patchset? | 17:07 |
| jeblair | clarkb: however, the test does break, and yet jenkins appears to operate correctly in reality. | 17:07 |
| jeblair | clarkb: i don't, but perhaps someone who knows more about jenkins testing might. | 17:07 |
| *** mrodden1 has quit IRC | 17:07 | |
| clarkb | ok | 17:08 |
| SpamapS | fungi: we can wait 3 hours to see what lifeless has to say. | 17:08 |
| fungi | k | 17:08 |
| *** openstackgerrit has quit IRC | 17:09 | |
| *** openstackgerrit has joined #openstack-infra | 17:09 | |
| *** nati_ueno has joined #openstack-infra | 17:10 | |
| clarkb | jeblair: I will take a quick look at it and bug zaro as needed | 17:11 |
| jeblair | fungi: re-eval? https://review.openstack.org/#/c/41516/ | 17:15 |
| fungi | jeblair: yep, was just in the middle of pulling that one back up | 17:16 |
| jeblair | fungi: cool, thx | 17:16 |
| *** vogxn has quit IRC | 17:16 | |
| *** afazekas has quit IRC | 17:16 | |
| zaro | clarkb: http://maven.apache.org/surefire/maven-surefire-plugin/examples/single-test.html | 17:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove unneeded node label parameters https://review.openstack.org/41516 | 17:18 |
| jeblair | mvn -Dtest=GearmanPluginUtilTest | 17:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Make all devstack-vm jobs single-use https://review.openstack.org/41517 | 17:18 |
| *** BobBall is now known as BobBall_Away | 17:18 | |
| jeblair | clarkb, zaro: ^ that's what i was running yesterday | 17:18 |
| jeblair | clarkb, zaro: the problem seems to be that in the test, the slave does not have a computer | 17:19 |
| fungi | clarkb: responded to your comment on https://review.openstack.org/40800 but in short that was intentional to deal with a shortcoming in the wrap() function | 17:19 |
| *** harlowja has joined #openstack-infra | 17:22 | |
| openstackgerrit | Paul Belanger proposed a change to openstack-infra/config: Properly load modules.conf https://review.openstack.org/41740 | 17:23 |
| pabelanger | russellb, ^ | 17:23 |
| pabelanger | just noticed the typo | 17:23 |
| mordred | fungi: I'm also ok with taking SpamapS word | 17:23 |
| *** dina_belova has quit IRC | 17:23 | |
| russellb | pabelanger: so ... | 17:23 |
| mordred | fungi: I'm pretty sure he knows the consequences of attempting to thwart lifeless_ | 17:23 |
| russellb | pabelanger: how is it working now? | 17:23 |
| fungi | mordred: relentless trolling for the remainder of his mortal existence? | 17:24 |
| pabelanger | russellb, opposite of what you implemented. We need to explicitly load modules, out side the default was used. | 17:24 |
| mordred | fungi: well, he gets that anyway | 17:24 |
| pabelanger | then, once asterisk see a file change, it will auto reload specific modules | 17:24 |
| *** dina_belova has joined #openstack-infra | 17:24 | |
| pabelanger | in this case, if modules.conf.d changes, the asterisk service is stopped , then started again | 17:24 |
| pabelanger | to force a reload of modules | 17:24 |
| russellb | pabelanger: oh have there been other changes i haven't seen? | 17:25 |
| *** vipul is now known as vipul-away | 17:25 | |
| *** dkliban_afk is now known as dkliban | 17:25 | |
| pabelanger | russellb, I guess, we merged reload support the other day | 17:25 |
| *** yolanda has quit IRC | 17:25 | |
| russellb | oh ok, didn't see it | 17:25 |
| russellb | so this is a fix to recent changes | 17:26 |
| russellb | i was just confused since obviously confibridge is loaded now | 17:26 |
| pabelanger | right | 17:26 |
| russellb | +1 then | 17:26 |
| *** vipul-away is now known as vipul | 17:27 | |
| fungi | mordred: do you have any idea what's going on with pip not finding pkginfo in the tests for https://review.openstack.org/41279 ? i'm going to recheck it because i can't see anything wrong with jeepyb's requirements and am hoping it was just transient pypi.p.o suck | 17:27 |
| anteaya | BobBall_Away: reviewed, devstack installed this time | 17:29 |
| BobBall_Away | woohoo | 17:29 |
| anteaya | was there something else I can do to evaluate the patch? or is a successful installation the goal? | 17:29 |
| anteaya | already +1'd | 17:30 |
| openstackgerrit | Paul Belanger proposed a change to openstack-infra/config: Clean up warning messages for asterisk https://review.openstack.org/41743 | 17:31 |
| dhellmann | hi, folks | 17:31 |
| pabelanger | russellb, ^ another | 17:31 |
| *** UtahDave has quit IRC | 17:32 | |
| dhellmann | we're ready to move pecan over to stackforge, and my co-worker has put together a patch. could you take a look, when you have a moment? https://review.openstack.org/#/c/41552/ | 17:32 |
| *** UtahDave has joined #openstack-infra | 17:32 | |
| fungi | clarkb: to your question yesterday about reordering the find command line in https://review.openstack.org/41544 , what do you think about adding -O3 to those finds instead? | 17:34 |
| fungi | clarkb: they're long-running enough they would almost certainly benefit from full optimization | 17:34 |
| *** cyeoh has quit IRC | 17:35 | |
| jeblair | dhellmann: lgtm | 17:35 |
| dhellmann | thanks, jeblair | 17:35 |
| *** cyeoh has joined #openstack-infra | 17:35 | |
| pabelanger | clarkb, jeblair: not sure if you wanted to us confbridge for the meeting today, but https://review.openstack.org/#/c/41740/ will need to be merged. Otherwise, we can manually load the modules | 17:36 |
| fungi | dhellmann: is the pecan-core group small enough that you expect everyone in it to participate in tagging releases? | 17:36 |
| dhellmann | fungi: yes, there's just a few of us for now | 17:36 |
| dhellmann | 4-5? | 17:36 |
| dhellmann | if that grows, we can create a separate release group later, right? | 17:36 |
| jeblair | fungi: drat i keep missing that1 | 17:36 |
| fungi | jeblair: i'm just great at being a parrot, that's all ;) | 17:37 |
| jeblair | dhellmann: so we really really don't like it when a lot of people can tag | 17:37 |
| dhellmann | ok, we'll go ahead and change it to use a separate group | 17:37 |
| jeblair | dhellmann: because tagging something incorrectly means we have to go and do lots of manual cleanup | 17:37 |
| jeblair | dhellmann: i don't think we need to insist on it for pecan.... | 17:37 |
| dhellmann | fair enough | 17:37 |
| jeblair | dhellmann: but it might still be a good idea; basically, the people who can tag... | 17:38 |
| fungi | dhellmann: pecan-ptl can be the group with the project lead and any delegates who are intimately familiar with https://wiki.openstack.org/wiki/GerritJenkinsGithub#Tagging_a_Release | 17:38 |
| dhellmann | well, we'd like to get some non-dreamhost folks as core at some point, so I can see creating a separate group | 17:38 |
| dhellmann | fungi: so call it "pecan-ptl"? | 17:38 |
| fungi | dhellmann: that's been the convention so far for ptl delegate groups entrusted with release tagging | 17:39 |
| fungi | dhellmann: so yes, preferably | 17:39 |
| dhellmann | sure, no problem, just double checking | 17:39 |
| fungi | great! | 17:40 |
| fungi | i'll update the review with comments to that effect too | 17:40 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add pecan project. https://review.openstack.org/41552 | 17:40 |
| *** sarob has joined #openstack-infra | 17:40 | |
| *** jfriedly has joined #openstack-infra | 17:40 | |
| dhellmann | jeblair, fungi: updated ^^ | 17:41 |
| * fungi cancels his review comment in that case | 17:41 | |
| dhellmann | fungi: if you add me to those groups, I can add ryan and show him how to add the other members when he returns from lunch | 17:43 |
| fungi | dhellmann: will do. remind me in about 30 minutes if i don't tell you i've done it already ;) | 17:44 |
| dhellmann | fungi: sounds good, thanks! | 17:44 |
| *** sarob has quit IRC | 17:46 | |
| *** vipul is now known as vipul-away | 17:46 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add pecan project. https://review.openstack.org/41552 | 17:46 |
| mordred | new thing learned for today: | 17:47 |
| mordred | git diff HEAD | flake8 --patch --select=E121,E122,E123,E124,E126 | 17:47 |
| mordred | will test flake8 things on just the stuff you changed | 17:47 |
| mordred | so potentially useful for turning on checking/gating of rules where the codebase is _not_ clean, but you don't want to introduce new breakage | 17:48 |
| *** nati_ueno has quit IRC | 17:48 | |
| *** dina_belova has quit IRC | 17:49 | |
| *** dina_belova has joined #openstack-infra | 17:49 | |
| * anteaya goes for a walk | 17:53 | |
| *** zul has quit IRC | 17:55 | |
| fungi | mordred: neat. i assume that only works for things where flake8 does not need context? or does it go digging for more context to do static analysis things as needed? | 17:55 |
| mordred | fungi: I believe it gets some context | 17:55 |
| fungi | nifty | 17:56 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Sort mailing lists in alpha order https://review.openstack.org/41756 | 17:56 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add marketing-content mailing list https://review.openstack.org/41757 | 17:56 |
| mtreinish | clarkb, jeblair, fungi: so I'm trying to track the success rate of the parallel testr tempest job vs the serial job using graphite. But I don't see any data in the graphs for the testr runs. | 17:57 |
| mtreinish | is there something that I need to do? | 17:57 |
| *** sarob has joined #openstack-infra | 17:59 | |
| *** vipul-away is now known as vipul | 17:59 | |
| jeblair | mtreinish: do you have a url to a graph you've been working on? | 18:00 |
| mtreinish | jeblair: this is what I have so far just trying to plot the data for the jobs: http://graphite.openstack.org/graphlot/?from=00:00_20130811&until=23:59_20130813&target=stats.zuul.job.gate-tempest-devstack-vm-testr-full&target=stats_counts.zuul.job.gate-tempest-devstack-vm-testr-full | 18:01 |
| mtreinish | nothing fancy yey | 18:01 |
| mtreinish | *yet | 18:01 |
| *** boris-42 has quit IRC | 18:02 | |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Create repo when we create the cgit config file https://review.openstack.org/41279 | 18:04 |
| *** blamar has joined #openstack-infra | 18:05 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Allow url to be updated https://review.openstack.org/41020 | 18:07 |
| *** ryanpetrello has joined #openstack-infra | 18:08 | |
| jeblair | mtreinish: http://tinyurl.com/n5fxmjk | 18:10 |
| jeblair | mtreinish: we moved some of those stats recently; i think i need to restart graphite or carbon for the old paths to disappears | 18:11 |
| jeblair | mtreinish: (i usually use the composer, but you can s/composer/graphlot/ in the url) | 18:11 |
| mtreinish | jeblair: ah, ok cool | 18:12 |
| mtreinish | that actually looks pretty close to ready for realtime (just need to wait longer to see if the trend holds) | 18:12 |
| mtreinish | thanks | 18:13 |
| mtreinish | while I've got you ear can you take a look at: https://review.openstack.org/41698 | 18:13 |
| mtreinish | so we can get testr parallel non voting on the gate queue too | 18:14 |
| *** jhesketh has quit IRC | 18:17 | |
| *** mrmartin has joined #openstack-infra | 18:17 | |
| *** ruhe has quit IRC | 18:17 | |
| *** boris-42 has joined #openstack-infra | 18:22 | |
| jeblair | mtreinish: good, that'll get better data | 18:22 |
| jeblair | mtreinish: you'll notice that the graphite paths have the pipeline in them, so with your zuul config patch, you'll be able to just look at gate data | 18:23 |
| mtreinish | jeblair: cool, yeah that'll be good to remove the bad patches on the check queue from the plots | 18:24 |
| mtreinish | jeblair: yeah hopefully the trend will be clean and come friday or early next week we can make the switch. | 18:24 |
| jeblair | mtreinish: !! :) | 18:25 |
| mtreinish | everyone will like a 50% reduction in tempest gate job runtime | 18:25 |
| dhellmann | fungi: ping | 18:26 |
| fungi | dhellmann: yep, i nabbed bug 1211912 but noticed the -ptl group didn't get created so am rerunning manage-projects while i try to sort out what happened there | 18:26 |
| uvirtbot | Launchpad bug 1211912 in openstack-ci "Create initial gerrit core member for Pecan" [Wishlist,In progress] https://launchpad.net/bugs/1211912 | 18:26 |
| fungi | thanks for the reminder too | 18:27 |
| dhellmann | fungi: ok, cool. I noticed that, too, but wasn't sure if the groups were created by hand and you stopped in the middle. :-) | 18:27 |
| fungi | created by the hand of a very lazy and easily distracted script, it seems | 18:27 |
| dhellmann | :-) | 18:28 |
| *** dina_belova has quit IRC | 18:28 | |
| clarkb | jeblair: I have a fix for that test, should I update your chage? | 18:28 |
| clarkb | jeblair: or push a new patchset instead? | 18:28 |
| *** SergeyLukjanov has quit IRC | 18:28 | |
| clarkb | also it is a very simple change and super annoying once you figure it out | 18:28 |
| clarkb | but setNodeName is only meant to be called when cloning an instance and not when creating a new one | 18:29 |
| *** ruhe has joined #openstack-infra | 18:29 | |
| ryanpetrello | thanks for your help, fungi | 18:29 |
| ryanpetrello | sorry I'm late to the party :) | 18:29 |
| jeblair | clarkb: awesome! i'm fine with you updating my change (unless it's substantial and you want git credit) | 18:29 |
| *** ruhe has quit IRC | 18:29 | |
| fungi | dhellmann: worked that time. want me to put you or ryanpetrello in one or both groups? | 18:29 |
| clarkb | jeblair: it isn't substantial. I am fine with updating your change. will do that now | 18:29 |
| fungi | dhellmann: how about i add you and then assign ryanpetrello's bug to you | 18:30 |
| dhellmann | works for me | 18:30 |
| *** zul has joined #openstack-infra | 18:30 | |
| ryanpetrello | sounds good | 18:30 |
| openstackgerrit | A change was merged to openstack-infra/config: Add testr parallel run to gate pipeline nonvoting. https://review.openstack.org/41698 | 18:31 |
| comstud | who maintains paste.openstack? | 18:31 |
| fungi | dhellmann: ryanpetrello: it's up to you folks now. happy hacking | 18:31 |
| comstud | it seems hosered. | 18:31 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/gearman-plugin: Rework starting/stopping executors https://review.openstack.org/41488 | 18:31 |
| clarkb | jeblair: ^ there you go | 18:32 |
| fungi | comstud: we do. checking for degree of hosx0rd now | 18:32 |
| comstud | thanks | 18:32 |
| comstud | i, for instance, can't get http://paste.openstack.org/show/44012/ to load | 18:32 |
| jeblair | clarkb, zaro: do you like the change? | 18:32 |
| jeblair | fungi: let me know if you want another set of eyes/hands | 18:32 |
| comstud | tho i guess, just the main page won't load for me, either | 18:32 |
| dhellmann | fungi: looks good, thanks again for your help | 18:32 |
| fungi | "The RSA host key for paste.openstack.org has changed" | 18:32 |
| fungi | jeblair: clarkb: i can't remember, that was just rebuilt recently right? | 18:33 |
| comstud | hah, that's not good | 18:33 |
| clarkb | jeblair: yes, I am much happier with the change now that we are not commenting out a test | 18:33 |
| clarkb | fungi: it was to move off of oneiric in may iirc | 18:33 |
| jeblair | fungi: relatively; current key is in my known hosts | 18:33 |
| fungi | jeblair: thanks, fc:30:14:d3:60:44:79:4a:70:9a:06:ee:63:6d:95:8e hopefully? | 18:33 |
| comstud | ah | 18:33 |
| comstud | i got a Proxy Error | 18:33 |
| comstud | if i wait long enough | 18:34 |
| fungi | seems the paste.o.o page took a while for me to load but did eventually come up | 18:34 |
| comstud | yeah, it loads now | 18:34 |
| fungi | may have been a brief load spike | 18:34 |
| *** ryanpetrello has quit IRC | 18:34 | |
| clarkb | fungi: it is deep into swap | 18:35 |
| fungi | gah | 18:35 |
| *** ryanpetrello has joined #openstack-infra | 18:35 | |
| fungi | clarkb: drizzled! | 18:36 |
| fungi | apparently drizzle wants more than a gig to itself | 18:36 |
| jeblair | fungi: yes, that matches the fp in my known hosts (as well as the key on the host in /etc) | 18:36 |
| fungi | jeblair: yeah, once i logged in and w showed you and clarkb logged in i assumed it was probably safe | 18:37 |
| *** nati_ueno has joined #openstack-infra | 18:37 | |
| fungi | maybe after the meeting i focus on helping canonical patch cacti for precise so we can install a hand-built security update and start looking at graphs again while we wait for someone to sponsor it into the archive | 18:38 |
| fungi | but my guess is slow leak in drizzled | 18:38 |
| clarkb | probably a reasonable guess | 18:38 |
| zaro | jeblair: looking thru it now. almost done. | 18:39 |
| clarkb | LinuxJedi: is around here somewhere maybe he can tell us to upgrade | 18:39 |
| fungi | so paste has probably been chewing through ram gradually for the past couple months | 18:39 |
| clarkb | the version of drizzle that we are running is pretty ancient | 18:39 |
| jeblair | fungi: jeez. do we need to start running debian to get timely security updates? (it's like 10 years ago but backwards) | 18:39 |
| jeblair | clarkb: i think we identified that as a good first thing to move to trove. | 18:40 |
| jeblair | as soon as we have backups worked out. | 18:40 |
| jeblair | which mordred is working on | 18:40 |
| fungi | jeblair: well, i think cacti is in "multiverse" which is community-supported security fixes only | 18:40 |
| fungi | on lts at least | 18:40 |
| fungi | jeblair: clarkb any objections to an emergency restart of drizzle on paste.o.o for now? | 18:41 |
| jeblair | fungi: please | 18:41 |
| clarkb | not from me, but you may need to kick apache afterwards to restart paste as well | 18:42 |
| clarkb | iirc that was an issue when we moved the host | 18:42 |
| clarkb | (it won't reconnect cleanly or something) | 18:42 |
| *** nayward has quit IRC | 18:46 | |
| * anteaya is back | 18:48 | |
| fungi | yeah, i had to shut down drizzle and apache and then kill an old python process for lodgeit which was still hanging around | 18:48 |
| fungi | now that it's started back up, memory pressure hasn't really let up much but i can't see what's squatting in it now | 18:48 |
| *** melwitt has joined #openstack-infra | 18:48 | |
| mrmartin | hi | 18:49 |
| jeblair | mrmartin: hello! | 18:49 |
| mrmartin | jeblair: i have a merge conflict during zuul/layout update. I rebased the dev branch and solved to conflict. Is it enought to include change-id in the commit message for the review process? | 18:50 |
| koolhead17 | pleia2, around? | 18:50 |
| jeblair | mrmartin: that's exactly right. | 18:51 |
| mrmartin | jeblair, ok I'll give it a try | 18:51 |
| zaro | jeblair: approved. thanks for that much needed fix. | 18:51 |
| jeblair | zaro: cool, thanks! | 18:51 |
| openstackgerrit | A change was merged to openstack-infra/gearman-plugin: Rework starting/stopping executors https://review.openstack.org/41488 | 18:53 |
| fungi | jeblair: clarkb: well, paste.o.o seems nice and responsive after restarting drizzle and apache, but we still don't have enough ram on that vm by half. any guess how much ram the previous paste.o.o had? | 18:54 |
| pleia2 | koolhead17: hey | 18:54 |
| fungi | jeblair: clarkb: wondering if maybe we scaled it down when we replaced it | 18:54 |
| jeblair | fungi: the other host still exists in the openstack account | 18:55 |
| fungi | ahh, perfect | 18:55 |
| jeblair | fungi: i read 60% free | 18:55 |
| jeblair | fungi: (most of it is in buffers/cache, which doesn't count) | 18:56 |
| *** SergeyLukjanov has joined #openstack-infra | 18:56 | |
| fungi | jeblair: well, yeah but trying to figure out why it's preferring to leave ~.5g swapped out in favor of buffers/cache | 18:56 |
| fungi | not detrimental, but a good bit more than is typical for most systems | 18:57 |
| fungi | based on overall percentage | 18:57 |
| fungi | especially since i restarted everything which should be consuming any substantial amount of memory, so i would have expected that to get freed | 18:58 |
| *** datsun180b_ has joined #openstack-infra | 19:00 | |
| clarkb | meeting time | 19:00 |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Add groups to gerrit https://review.openstack.org/41643 | 19:01 |
| *** datsun180b has quit IRC | 19:01 | |
| *** datsun180b_ is now known as datsun180b | 19:01 | |
| fungi | looks like the old paste.o.o also only had 1g ram and similar memory usage patterns, but had 2g of swap instead of just 1g | 19:02 |
| openstackgerrit | Jason Meridth proposed a change to openstack-dev/hacking: Adds ability to ignore H231, python3x except compatability check https://review.openstack.org/41713 | 19:02 |
| mrmartin | jeblair, patch set 2 is done. Basically we want to keep just the custom changes of the Drupal Commons codebase in the git repo, as an installation profile. | 19:03 |
| mrmartin | jeblair: so we can use make drush to build a distribution tarball from this profile, and it pulls down contrib modules from drupal.org site. It have the advantage, that drupal.org provides a lots of check / audit on contrib modules, so we can keep custom codes in a minimal level. | 19:05 |
| ttx | mordred: all clear to play with merge-milestone-proposed-back-to-master on swift 1.9.1 | 19:07 |
| ttx | I'm holding on removing the branch | 19:08 |
| dhellmann | fungi: we're having a permission issue with pecan-core, could you lend a hand when you're done with the infra team meeting? | 19:09 |
| fungi | dhellmann: sure | 19:09 |
| *** datsun180b has quit IRC | 19:10 | |
| dhellmann | fungi: pecan-core members can only +1 on changesets for the repo, such as https://review.openstack.org/#/c/41764/ | 19:10 |
| *** datsun180b has joined #openstack-infra | 19:10 | |
| dhellmann | fungi: no rush | 19:10 |
| fungi | dhellmann: i see the issue. rerunning manage-projects one more time to get the acl applied properly | 19:14 |
| fungi | i'll let you know when it fiishes | 19:14 |
| dhellmann | fungi: thanks! | 19:14 |
| *** nati_ueno has quit IRC | 19:15 | |
| *** markmc has joined #openstack-infra | 19:16 | |
| *** adalbas has quit IRC | 19:17 | |
| *** gyee has quit IRC | 19:25 | |
| fungi | dhellmann: should be all set now. try again | 19:26 |
| dhellmann | fungi: yep, looks good | 19:26 |
| *** vipul is now known as vipul-away | 19:27 | |
| *** cthulhup has joined #openstack-infra | 19:27 | |
| *** odyssey4me has quit IRC | 19:28 | |
| *** dina_belova has joined #openstack-infra | 19:28 | |
| *** adalbas has joined #openstack-infra | 19:29 | |
| *** chuckieb has quit IRC | 19:30 | |
| *** dina_belova has quit IRC | 19:31 | |
| ryanpetrello | fungi: for pecan, it looks like we've got an old branch laying around that's related to our old review process | 19:36 |
| ryanpetrello | https://github.com/stackforge/pecan/tree/next | 19:36 |
| ryanpetrello | is there a process for removing that branch? | 19:36 |
| ryanpetrello | it's not really necessary now that we're using gerrit | 19:36 |
| fungi | ryanpetrello: yeah, i can delete it for you | 19:37 |
| fungi | just a moment | 19:37 |
| ryanpetrello | okay, awesome | 19:37 |
| ryanpetrello | thanks | 19:37 |
| *** dina_belova has joined #openstack-infra | 19:38 | |
| *** sarob has quit IRC | 19:40 | |
| *** sarob has joined #openstack-infra | 19:41 | |
| *** dina_belova has quit IRC | 19:43 | |
| *** cthulhup has quit IRC | 19:44 | |
| russellb | jeblair: pabelanger: for after the meeting since i missed the topic ... we don't *need* a large group to load test. we could generate calls that randomly occasionally play a sound prompt from another rackspace instance in the same region. and then just a couple people call in to actually hear any effects to the quality. | 19:45 |
| pabelanger | russellb, yup, that works too. I figured people actually wanted to talk to each other | 19:49 |
| pabelanger | :p | 19:49 |
| *** dina_belova has joined #openstack-infra | 19:50 | |
| jeblair | russellb: that's a good idea; can you take care of that? | 19:50 |
| jeblair | pabelanger, russellb: we may still want to get at least several people calling in to increase the diversity of inputs a bit. | 19:51 |
| lifeless_ | mordred: pong | 19:52 |
| *** lifeless_ is now known as lifeless | 19:52 | |
| mordred | lifeless_: it was a question about tripleo-ptl team and releases | 19:52 |
| *** ruhe has joined #openstack-infra | 19:53 | |
| *** pcm_ has quit IRC | 19:53 | |
| lifeless | mordred: go on | 19:55 |
| *** cthulhup has joined #openstack-infra | 19:55 | |
| russellb | jeblair: yep | 19:55 |
| mordred | lifeless: the normal pattern for projects is that there is a group, $project-ptl which has the acls to push tags | 19:55 |
| russellb | jeblair: and by take care of it, i mean i'll make pabelanger do it | 19:56 |
| mordred | this can include the ptl and other people the ptl can handle the extra responsibilty aroud the tag/release process | 19:56 |
| lifeless | cool | 19:56 |
| mordred | lifeless: because you have multiple projects with delegated core groups, such as os-apply-config-core | 19:56 |
| mordred | we wanted to check in with you before doing what seemed like the 'right' thing - which was making a tripleo-core and putting you and Spamaps in it | 19:56 |
| mordred | so that he could cut releases of os-*-config | 19:57 |
| lifeless | uhm, me SpamapS and devananda right now; I want to chat with Ng_holiday, who has spent quite a while maintaining projects too | 19:57 |
| pabelanger | russellb, jeblair: sure, I know how to hack the gibbson | 19:57 |
| jeblair | mrmartin: tripleo-ptl ? | 19:57 |
| jeblair | ack | 19:57 |
| jeblair | mordred: tripleo-ptl ? | 19:57 |
| jeblair | mrmartin: sorry | 19:57 |
| mordred | jeblair: yes. sorry | 19:59 |
| mordred | lifeless: tripleo-ptl | 19:59 |
| mrmartin | jeblair, not yet ;) | 19:59 |
| * mordred needs to afk again | 20:00 | |
| clarkb | jeblair: http://paste.openstack.org/show/44068/ is the exception. It looks like fileinput.input is a global object and somehow we are abusing that in a bad way | 20:01 |
| *** sandywalsh has quit IRC | 20:01 | |
| fungi | ryanpetrello: deleted your "next" cruft branch from stackforge/pecan as requested, leaving only master | 20:01 |
| clarkb | jeblair: I have a patch that uses distinct objects per file | 20:02 |
| *** hub_cap has joined #openstack-infra | 20:02 | |
| ryanpetrello | fungi: thanks for your help :) | 20:02 |
| fungi | ryanpetrello: any time | 20:03 |
| clarkb | I think it may be related to the logstash stuff hitting logs.o.o a lot more than logs-dev which causes race conditions | 20:03 |
| hub_cap | do we have some implicit requirements like pexpect? i dont see it in global-requirements, and when we tried to add it to a review it puked with the message "Duplicate requirement in HEAD: pexpect" | 20:03 |
| clarkb | hub_cap: can you link to the review that is failing? | 20:04 |
| hub_cap | clarkb: https://review.openstack.org/#/c/40859/ | 20:04 |
| fungi | clarkb: as i mentioned earlier, not sure if it helps but the terribly simple mysqldump script i run from cron on my personal systems is http://paste.openstack.org/show/44069/ | 20:04 |
| lifeless | mordred: yes: me, SpamapS, devananda please. | 20:04 |
| lifeless | mordred: thanks for asking! | 20:04 |
| clarkb | fungi: thanks | 20:04 |
| *** zul has quit IRC | 20:04 | |
| fungi | clarkb: doesn't version them or anything (i separately snapshot the filesystem later on) | 20:05 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/jeepyb: Add chown of git repo directories https://review.openstack.org/41781 | 20:05 |
| fungi | so probably fairly similar to what we'd want for bup's purposes anyway | 20:05 |
| pleia2 | ^^ looked for a more pythony way to do it, but it turns out recursive chown in python is not awesome | 20:06 |
| uvirtbot | pleia2: Error: "^" is not a valid command. | 20:06 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Create distinct fileinput objects in htmlifier. https://review.openstack.org/41782 | 20:06 |
| fungi | pleia2: yes, you'll see manage_projects.py shells out | 20:06 |
| clarkb | jeblair: ^ potential fix for the log stuff. completely untested | 20:06 |
| pleia2 | fungi: ah, indeed it does | 20:07 |
| zaro | clarkb: would you be willing to approve? https://review.openstack.org/#/c/41307/ | 20:07 |
| * devananda waves | 20:08 | |
| *** datsun180b_ has joined #openstack-infra | 20:08 | |
| clarkb | hub_cap: weird | 20:10 |
| jeblair | clarkb: yeah, actually what we should probably do is extend the current bup class | 20:10 |
| hub_cap | isint it? | 20:10 |
| * hub_cap is confused (but whats new) | 20:10 | |
| jeblair | clarkb: so it takes an argument about whether to run mysqldump; and then just have the cron script run that before bup (if the arg is set) | 20:10 |
| clarkb | jeblair: ok | 20:10 |
| *** datsun180b has quit IRC | 20:11 | |
| *** datsun180b_ is now known as datsun180b | 20:11 | |
| fungi | jeblair: clarkb: that actually sounds really good. takes care of in-progress races and minimizes the window between dump and archiving | 20:12 |
| *** dprince has quit IRC | 20:12 | |
| clarkb | hub_cap: its breaking when checking the trove change and not openstack/requirements | 20:12 |
| jeblair | hub_cap: it's in both requirements and test-requirements | 20:13 |
| *** sandywalsh has joined #openstack-infra | 20:13 | |
| hub_cap | ya somethign about a dup in our file right clarkb? | 20:13 |
| jeblair | hub_cap: if you want to move it to requirements, remove it from test-requirements | 20:13 |
| hub_cap | yes test requirements is requirements + any testing stuffs right? | 20:13 |
| hub_cap | as in, if we need in prod, put it in requirements and when we test, itll already be there | 20:14 |
| jeblair | hub_cap: right | 20:14 |
| hub_cap | kk thx | 20:14 |
| clarkb | jeblair: I don't see it in requirements | 20:14 |
| jeblair | clarkb: https://review.openstack.org/#/c/40859/3/requirements.txt | 20:14 |
| *** vipul-away is now known as vipul | 20:14 | |
| clarkb | oh that change is adding it to requirements but it is already in test-requirements? | 20:14 |
| jeblair | clarkb: correct | 20:14 |
| clarkb | gotcha | 20:15 |
| clarkb | also using expect for anything production like is uhm scary | 20:15 |
| hub_cap | heh of course it is :) | 20:15 |
| hub_cap | tell apt that | 20:15 |
| clarkb | why does apt need expect? | 20:15 |
| jeblair | expect: failure | 20:15 |
| hub_cap | we expect that a package will be installed, so to speak | 20:15 |
| clarkb | can't you just query apt after to see if it is installed? | 20:16 |
| hub_cap | well yes, but thats easy | 20:16 |
| hub_cap | ;) | 20:16 |
| * hub_cap files a bug | 20:16 | |
| jog0 | why can't I sort my review.openstack.org page by date? | 20:18 |
| jog0 | also need oen mre review on https://review.openstack.org/#/c/41523/ | 20:20 |
| *** sandywalsh has quit IRC | 20:20 | |
| jog0 | if any one is jonesing to do a review | 20:20 |
| fungi | clarkb: i've used expect quite happily in production in years past, though prefer the python pexpect module as a saner solution which doesn't require plumbing the depths of my tcl memory | 20:26 |
| fungi | clarkb: to your question yesterday about reordering the find command line in https://review.openstack.org/41544 , what do you think about adding -O3 to those finds instead? | 20:33 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add start/complete custom functions https://review.openstack.org/41785 | 20:33 |
| *** sandywalsh has joined #openstack-infra | 20:33 | |
| jeblair | fungi, clarkb, mordred: if any of you have a minute, can you take a look at that ^ | 20:34 |
| jeblair | fungi, clarkb, mordred: i have some design questions i'd like to bounce around | 20:34 |
| *** cthulhup has quit IRC | 20:35 | |
| fungi | ooh, hooks! | 20:35 |
| *** woodspa has quit IRC | 20:35 | |
| jeblair | basically, i'm not entirely sure putting that into 'configuration' is the right way to go. otoh, why not? :) | 20:36 |
| fungi | exim certainly has no fear of turning its configuration into software ;) | 20:37 |
| *** sgviking has quit IRC | 20:37 | |
| jeblair | other ideas i've had include actually making entrypoints and calling real hooks that way -- elegant, but really heavy weight (i mean, then we'd have to install a plugin just to do this...) | 20:37 |
| fungi | jeblair: is the thought that this would also get used to update stats? | 20:37 |
| jeblair | or going with functions like i have there, but limiting them so that they are exclusively used to run gearman jobs (with custom parameters)... | 20:39 |
| *** dina_belova has quit IRC | 20:39 | |
| fungi | seems reasonable | 20:39 |
| *** vipul is now known as vipul-away | 20:39 | |
| jeblair | the thinking there is that i feel like i'm breaking some api barriers by passing the _scheduler_ into the custom functions, because then they could do anything, most of which would be bad. but then if i do what i just suggested, then it's assuming a gearman launcher, which is weird. | 20:40 |
| jeblair | fungi: i would like to use these to call gearman functions that are implemented by the nodepool worker | 20:40 |
| jeblair | fungi: so this will replace the inprogress/complete jobs in jenkins | 20:40 |
| *** ruhe has quit IRC | 20:41 | |
| jeblair | yet another option would be to put zmq support in zuul, but i actually like gearman for this because i want these jobs to be queued | 20:41 |
| clarkb | fungi what were you using to test the htmlifier stuff re https://review.openstack.org/#/c/41782/ ? | 20:42 |
| clarkb | though I am happy enough to approve that and test it as is as the current solution is broken | 20:42 |
| clarkb | jeblair: looking | 20:42 |
| fungi | i really don't feel like it's a violation of any implied api contract there, as long as the interface between zuul and the custom function modules is clearly defined, but it does of course allow them to circumvent that interface by mucking with the namespace contents directly | 20:42 |
| jeblair | the advantage of the patchset i uploaded is that it doesn't assume any knowlede about zuul or the environment -- because it lets the custom functions do anything. | 20:42 |
| jeblair | (and you can see that in my test functions, i abused knowledge of internals to do what i needed; which is roughly what we'd be doing in production) | 20:43 |
| clarkb | jeblair: what if the api was in gearman? | 20:43 |
| fungi | clarkb: testing the htmlifier stuff before it was in production was easier because i was just manually applying changes to the dev chost | 20:44 |
| clarkb | jeblair: eg register workers for these jobs if you want them to do work? otherwise zuul will not submit the jobs | 20:44 |
| fungi | clarkb: er, dev vhost | 20:44 |
| *** ruhe has joined #openstack-infra | 20:45 | |
| *** vipul-away is now known as vipul | 20:45 | |
| *** adalbas has quit IRC | 20:45 | |
| *** ruhe has quit IRC | 20:46 | |
| fungi | clarkb: i suppose you could put a renamed copy of that script in a puppet development environment, modify the logs-dev vhost to point its filter at that name, and then apply on static.o.o from that environment to test | 20:47 |
| jeblair | clarkb: that's an interesting idea, though i think in that case, the jobs would probably have to always run for every build (the current patch lets you specify that they should only be run for certain jobs) | 20:47 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/jeepyb: Add chown of git repo directories https://review.openstack.org/41781 | 20:47 |
| jeblair | clarkb: that would be fine for us; nodepool will quickly disregard things it doesn't know about | 20:48 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Document escaping braces in template literals https://review.openstack.org/40973 | 20:48 |
| clarkb | jeblair: going back to mysql backups. I don't think we should make them too tightly coupled to bup. Instead have the module do the mysqldump to a configurable dir which will be logrotated by the mysql backup module on some configurable interval. Then bup can just backup that dir | 20:48 |
| jeblair | clarkb: okay; why's that? i suppose it's true that having a definabel resource per database would make it easier when we're using trove. | 20:49 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/jeepyb: Add chown of git repo directories https://review.openstack.org/41781 | 20:49 |
| fungi | pleia2: any particular reason for the separate chgrp pass instead of just passing '%s:%s'(CGIT_USER, CGIT_GROUP) as a parameter? | 20:50 |
| clarkb | jeblair: if we ever go to something other than bup we won't have to figure out mysqldump again, and we can do it per database | 20:50 |
| jeblair | clarkb: sounds good then | 20:51 |
| fungi | pleia2: er, '%s:%s' % (CGIT_USER, CGIT_GROUP) | 20:51 |
| ryanpetrello | fungi: for stackforge projects that already have external docs (e.g., readthedocs), is there a best practice towards building these with releases? | 20:51 |
| pleia2 | fungi: could do that, just didn't think of it in time :) kept trying different quoting and then gave up | 20:51 |
| ryanpetrello | I know the existing infra stuff will publish to openstack.org somewhere, but we probably don't want that | 20:51 |
| clarkb | jeblair: the custom functions to submit jobs that d-g/nodepool can consume is fine with me. I was just offering an alternative to stick with a more gearman specific interface | 20:52 |
| clarkb | fungi: I think I may just approve the fileinput change and if that doesn't help revert the logs.o.o vhost change | 20:52 |
| fungi | ryanpetrello: we have a rtfd hook, though i believe we need to know the rtfd id number and your rtfd project needs to allow access from our account | 20:52 |
| jeblair | fungi: i don't think there's an account involved | 20:52 |
| fungi | clarkb: that seems very low-risk. at most people won't be able to view the logs for a few minutes | 20:52 |
| fungi | jeblair: oh, no permissions settings needed? i think we have a doc on it anyway, checking | 20:53 |
| jeblair | ryanpetrello: https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/projects.yaml#L70 | 20:54 |
| clarkb | the puppetlabs mysql module has a backup manifest but I think it attempts to be too smart | 20:54 |
| jeblair | ryanpetrello: should just need to do the rtfd bits from that section, substituting your own rtfd id | 20:54 |
| ryanpetrello | jeblair: awesome, thanks for the reference :) | 20:54 |
| *** ArxCruz has quit IRC | 20:54 | |
| fungi | looks like maybe we don't have actual documentation contributed for the rtfd hook, so that's something which would probably fit well in the stackforge.rst | 20:55 |
| jeblair | fungi: ++ | 20:55 |
| *** ArxCruz has joined #openstack-infra | 20:57 | |
| clarkb | what is a good default mysql backup dir? | 20:57 |
| fungi | clarkb: i tend to just use /var/backups | 20:57 |
| fungi | it has historical precedent | 20:57 |
| *** boris-42 has quit IRC | 21:00 | |
| ryanpetrello | jeblair: is there anything special we need to do in RTFD to authorize this (doc publication)? | 21:01 |
| jeblair | ryanpetrello: afaik no | 21:01 |
| *** ArxCruz has quit IRC | 21:01 | |
| *** moted has quit IRC | 21:02 | |
| *** vipul is now known as vipul-away | 21:02 | |
| *** moted has joined #openstack-infra | 21:02 | |
| *** boris-42 has joined #openstack-infra | 21:02 | |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/jeepyb: Add chown of git repo directories https://review.openstack.org/41781 | 21:02 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add start/complete custom functions https://review.openstack.org/41785 | 21:04 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/jeepyb: Add chown of git repo directories https://review.openstack.org/41781 | 21:05 |
| *** boris-42 has quit IRC | 21:07 | |
| jeblair | clarkb, fungi: https://jenkins01.openstack.org/computer/devstack-precise-hpcloud-az1-9345/ | 21:11 |
| jeblair | "Offline due to Gearman request" | 21:11 |
| jeblair | yay :) | 21:11 |
| fungi | ooh! | 21:11 |
| *** rfolco has quit IRC | 21:12 | |
| *** jpich has quit IRC | 21:14 | |
| clarkb | \o/ | 21:15 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: Don't relabel devstack-gate nodes https://review.openstack.org/41789 | 21:15 |
| jeblair | fungi, clarkb: step 3 ^ :) | 21:16 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add new mysql_backup module. https://review.openstack.org/41790 | 21:16 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Switch etherpad_lite backups to mysql_backup. https://review.openstack.org/41791 | 21:16 |
| clarkb | jeblair: fungi ^ that is also completely untested, but it is relatively simple and I included an example of how we could use it | 21:16 |
| clarkb | wanted more feedback before I started applying it to things (I intend on running that second change on etherpad_dev to test) | 21:17 |
| clarkb | fungi: does https://review.openstack.org/#/c/41782/1 look good to you? if so can you approve it? I will babysit and propose the logs.o.o vhost revert if necessary | 21:18 |
| fungi | clarkb: lgtm, done | 21:19 |
| clarkb | fungi: ty | 21:19 |
| clarkb | jeblair: for step three, are you using the custom functions in the cahnge you pointed out earlier? | 21:19 |
| clarkb | jeblair: or is it just that other custom function which sets the build env var? | 21:20 |
| jeblair | clarkb: the second thing | 21:20 |
| jeblair | clarkb: which is happening now (and gearman plugin is taking those nodes offline because of it) | 21:20 |
| clarkb | jeblair: what need do the custom functions in the second thing accomplish that the current setup does not? | 21:21 |
| openstackgerrit | A change was merged to openstack-infra/config: Create distinct fileinput objects in htmlifier. https://review.openstack.org/41782 | 21:21 |
| jeblair | clarkb: they are for nodepool -- to trigger its worker (so we can get rid of the devstack-gate jobs entirely) | 21:21 |
| clarkb | gotcha | 21:21 |
| *** markmc has quit IRC | 21:23 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Document RTFD hook job template in StackForge doc https://review.openstack.org/41793 | 21:23 |
| *** ryanpetrello has quit IRC | 21:24 | |
| *** vipul-away is now known as vipul | 21:25 | |
| *** nati_ueno has joined #openstack-infra | 21:25 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Switch etherpad_lite backups to mysql_backup. https://review.openstack.org/41791 | 21:25 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add new mysql_backup module. https://review.openstack.org/41790 | 21:25 |
| fungi | clarkb: quick eyeballs on https://review.openstack.org/41781 and approve if you're cool with that? once it's in place i'll approve something which touches review_projects.yaml.erb if there is anything ready to go (or i'll fix the minor issue with https://review.openstack.org/41643 and put that through) | 21:29 |
| clarkb | fungi: I just approved it :) | 21:30 |
| fungi | too cool | 21:30 |
| * fungi disappears to eat something briefly. brb | 21:31 | |
| clarkb | mtreinish: super excited about the testr work and potential switch to it in the gate. | 21:31 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Add chown of git repo directories https://review.openstack.org/41781 | 21:32 |
| *** cp16net has left #openstack-infra | 21:32 | |
| *** gyee has joined #openstack-infra | 21:33 | |
| mtreinish | clarkb: yeah, hopefully it'll stay stable | 21:33 |
| *** cthulhup has joined #openstack-infra | 21:35 | |
| *** weshay has quit IRC | 21:36 | |
| *** dina_belova has joined #openstack-infra | 21:39 | |
| clarkb | mtreinish: ya, I am all for switching now >_> to help avoid new problems sneaking in | 21:40 |
| clarkb | jeblair: fungi I am able to get logs through the htmlifier on logs.o.o now | 21:41 |
| fungi | clarkb: great! | 21:41 |
| fungi | clarkb: also i replied to your question on https://review.openstack.org/40800 | 21:42 |
| mtreinish | clarkb: well I'd like that too and just the fight races when they come up. But, my only concern with doing that is doesn't really have an established track record of working for that long. My fix for the last big race only got merged yesterday. | 21:43 |
| *** dina_belova has quit IRC | 21:44 | |
| *** cthulhup has quit IRC | 21:44 | |
| clarkb | mtreinish: I understand. I am just slightly more adventurous considering I don't have a lot of buckets in H3 >_> | 21:44 |
| mtreinish | clarkb: well feel free to share that opinion on the ML thread, maybe the consensus is that people don't care about the proven stability weighed against raw speed and we can push it through now :) | 21:45 |
| clarkb | fungi: responded :) | 21:45 |
| clarkb | mtreinish: I have sent mail, I guess it hasn't gone through yet | 21:46 |
| mtreinish | clarkb: no, I just fail at looking at new mail notification | 21:46 |
| *** sgviking has joined #openstack-infra | 21:47 | |
| *** dina_belova has joined #openstack-infra | 21:49 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Switch etherpad_lite backups to mysql_backup. https://review.openstack.org/41791 | 21:50 |
| fungi | clarkb: on https://review.openstack.org/40800 i can switch it back to two print statements but it's not going to alter the output (more specific example added inline) | 21:51 |
| clarkb | oh because of the explicit newlines? | 21:52 |
| clarkb | wow I am derping hard. | 21:52 |
| clarkb | fungi: updated my review. thank you for your patience | 21:53 |
| fungi | clarkb: yeah, i just figured while i was collapsing repeated print statements i'd go ahead and clean that up | 21:53 |
| jeblair | clarkb, fungi: maybe the new zuul functions are premature. maybe nodepool should be a jenkins zmq listener... | 21:53 |
| jeblair | at least, for now... | 21:53 |
| jeblair | clarkb, fungi: mostly because if we restart zuul while jobs are running (which we do), nodepool will miss notifications | 21:53 |
| *** dina_belova has quit IRC | 21:54 | |
| jeblair | (it'll eventually clean them up, but those nodes would stick around for quite a while) | 21:54 |
| jeblair | (if it's a zmq listener, it could miss notifications when _nodepool_ isn't running, but hopefully that won't be very often or for very long) | 21:56 |
| clarkb | jeblair: we can actually work around that second issue as well | 21:58 |
| clarkb | jeblair: zmq pub sub allows us to queue jobs in memory on the sender side if the receiver goes away | 21:58 |
| clarkb | jeblair: however I have avoided doing that as it has memory consideratinos, but totally doable if we decide that is a need | 21:58 |
| *** rnirmal has quit IRC | 21:59 | |
| jeblair | clarkb: yeah, i also don't know where we'd want to put this for non-jenkins workers; perhaps nodepool should keep a zmq listener for jenkins and be a gearman worker for other nodes... | 21:59 |
| jeblair | clarkb: (perhaps even the gearman plugin for jenkins should get the ability to put these jobs on the gearman queue) | 22:00 |
| jeblair | clarkb: but the more i think about it, the more i think that right now we should stick with zmq because it's the best match for current operations, and we should still have the flexibility to change in the future. | 22:00 |
| jeblair | (ie, nodepool can have as many of these inputs as we want) | 22:00 |
| clarkb | ++ | 22:01 |
| *** beagles has quit IRC | 22:01 | |
| fungi | and what would be the longer-term fit in a post-jenkins world? | 22:01 |
| jeblair | fungi: if we have our own workers, it makes a little more sense for them to put jobs on the gearman queue themselves, so they could trigger these on their own. | 22:02 |
| *** datsun180b has quit IRC | 22:03 | |
| *** burt has quit IRC | 22:04 | |
| *** dkliban has quit IRC | 22:04 | |
| *** SergeyLu_ has joined #openstack-infra | 22:04 | |
| fungi | got it. and at that point the vestigial zmq support just gets phased out/dropped entirely | 22:05 |
| jeblair | yeah. it's _a_ plan. :) | 22:05 |
| openstackgerrit | Marton Kiss proposed a change to openstack-infra/config: Add groups to gerrit https://review.openstack.org/41643 | 22:05 |
| *** SergeyLukjanov has quit IRC | 22:06 | |
| *** jerryz has quit IRC | 22:06 | |
| *** beagles has joined #openstack-infra | 22:06 | |
| *** mrmartin has quit IRC | 22:09 | |
| *** SergeyLu_ has quit IRC | 22:09 | |
| *** pentameter has quit IRC | 22:11 | |
| *** sarob has quit IRC | 22:12 | |
| *** sarob has joined #openstack-infra | 22:12 | |
| *** sarob has quit IRC | 22:17 | |
| fungi | approved ^ that, so once it merges i'll keep an eye on git.o.o to see i it populates accordingly | 22:20 |
| openstackgerrit | A change was merged to openstack-infra/config: Add groups to gerrit https://review.openstack.org/41643 | 22:21 |
| * fungi rubs hands together | 22:22 | |
| * pleia2 opens up dashboard | 22:23 | |
| *** mrodden has quit IRC | 22:23 | |
| *** _TheDodd_ has quit IRC | 22:28 | |
| *** changbl has quit IRC | 22:31 | |
| *** dripton has quit IRC | 22:31 | |
| *** mriedem has quit IRC | 22:32 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: WIP: Initial commit https://review.openstack.org/41805 | 22:33 |
| *** ryanpetrello has joined #openstack-infra | 22:34 | |
| *** dripton has joined #openstack-infra | 22:35 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Gate nodepool on pep8. https://review.openstack.org/41806 | 22:35 |
| jeblair | clarkb, fungi: ^ aprv 41806 at will | 22:36 |
| *** beagles has quit IRC | 22:36 | |
| fungi | ooh | 22:36 |
| jeblair | clarkb, fungi, mordred: now might be a good time to add nodepool to your watched projects list | 22:36 |
| clarkb | jeblair: will do | 22:37 |
| *** sarob has joined #openstack-infra | 22:38 | |
| pleia2 | fungi: https://git.openstack.org/cgit/openstack/api-site/ | 22:38 |
| pleia2 | empty \o/ | 22:38 |
| fungi | pleia2: yep, and this time the permissions check out | 22:40 |
| pleia2 | now hopefully gerrit can properly replicate to it | 22:40 |
| pleia2 | great | 22:40 |
| fungi | pleia2: i forced full replication via the gerrit ssh api, so let's see if it fills them in | 22:40 |
| openstackgerrit | A change was merged to openstack-infra/config: Gate nodepool on pep8. https://review.openstack.org/41806 | 22:41 |
| *** cthulhup has joined #openstack-infra | 22:41 | |
| jeblair | 3bdfffc9 push cgit@git.openstack.org:/var/lib/git/opensta | 22:41 |
| jeblair | that's so helpful. | 22:41 |
| *** jhesketh has joined #openstack-infra | 22:41 | |
| fungi | pleia2: you can watch the count of replication jobs queued for it with 'ssh -p 29418 review.openstack.org gerrit show-queue | grep -c cgit' | 22:41 |
| fungi | jeblair: yeah, love the truncation there | 22:41 |
| pleia2 | fungi: neat, thanks | 22:41 |
| *** beagles has joined #openstack-infra | 22:42 | |
| jeblair | fungi: it's an openstack project. | 22:42 |
| fungi | i guess we can say for certain 3bdfffc9 does not belong to a stackforge project | 22:42 |
| pleia2 | hehe | 22:42 |
| jeblair | it probably belongs to nova | 22:44 |
| pleia2 | I sure hope so, it's taking a while | 22:44 |
| fungi | statistically speaking, it probably does belong to nova | 22:46 |
| *** ladquin_afk has quit IRC | 22:48 | |
| *** cthulhup has quit IRC | 22:49 | |
| *** dina_belova has joined #openstack-infra | 22:50 | |
| clarkb | does this mean cgit is mostly working now? | 22:50 |
| jeblair | i don't think so... | 22:53 |
| jeblair | there are still 170 cgit jobs | 22:53 |
| jeblair | perhaps none of them have completed | 22:54 |
| jeblair | they are the only ones left in the nqueue | 22:54 |
| *** dina_belova has quit IRC | 22:55 | |
| samalba | does anyone have the answer of that: https://answers.launchpad.net/devstack/+question/233734 ? | 22:55 |
| samalba | I have the same issue | 22:55 |
| samalba | and not super familiar with cinder | 22:55 |
| *** ladquin_afk has joined #openstack-infra | 23:00 | |
| *** ladquin_afk has joined #openstack-infra | 23:00 | |
| *** svarnau has quit IRC | 23:05 | |
| *** ryanpetrello has quit IRC | 23:07 | |
| fungi | samalba: you might have more luck asking in #openstack or #openstack-dev | 23:08 |
| fungi | samalba: the -infra channel is the people running the community infrastructure systems | 23:08 |
| samalba | I thought it was a devstack issue (like config problem), that's why I asked here | 23:09 |
| samalba | but I'll try on -dev | 23:09 |
| pleia2 | so, looks like replication is still unhappy | 23:09 |
| pleia2 | sorted permissions, ssh keys should be ok | 23:10 |
| fungi | pleia2: yes, tcpdump on git.o.o does show an established ssh connection from review.o.o, just very, very minimal traffic | 23:10 |
| jog0 | mordred: I assume you saw this article http://arstechnica.com/information-technology/2013/08/why-ubuntus-creator-still-invests-his-fortune-in-an-unprofitable-company/ | 23:10 |
| * pleia2 headscratch | 23:10 | |
| fungi | pleia2: i also checked /var/log/secure and it didn't show any authentication issues | 23:13 |
| *** sdague has quit IRC | 23:13 | |
| openstackgerrit | A change was merged to openstack/requirements: Add note to test-requirements saying its local only https://review.openstack.org/41524 | 23:15 |
| pleia2 | browsing replication docs to see if there is something I'm missing for ssh | 23:16 |
| fungi | pleia2: the gerrit log is showing constant ssh exceptions for "connection reset by peer" (typically that indicates a tcp/rst on an open socket) | 23:16 |
| pleia2 | huh | 23:17 |
| fungi | though i'm not seeing rst set on any of the packets going back to the gerrit server from git.o.o | 23:17 |
| jeblair | fungi: i believe that's normal gerrit behavior, indicating that an ssh connection is finished. | 23:18 |
| *** blamar has quit IRC | 23:18 | |
| fungi | ha | 23:18 |
| fungi | jeblair: that's downright sad | 23:18 |
| jeblair | fungi: it's sad. it makes the error log very useless. | 23:18 |
| pleia2 | doesn't seem like there are any special things in the configuration we need, just the url line should be enough | 23:19 |
| fungi | netstat -nt shows four ssh connections established inbound from review.o.o to git.o.o | 23:20 |
| jeblair | fungi: but none of them are running git, correct? | 23:21 |
| fungi | jeblair: yeah, no git processes | 23:22 |
| *** ryanpetrello has joined #openstack-infra | 23:26 | |
| jeblair | fungi: how about i kill one of those and we see if gerrit logs anything? | 23:28 |
| fungi | jeblair: yeah, i wonder if these are still there from a previous failing replication attempt and all the in progress ones need killing | 23:29 |
| *** jhesketh has quit IRC | 23:29 | |
| *** jhesketh has joined #openstack-infra | 23:29 | |
| jeblair | oh, i was thinking of killing the ssh process on git.o.o, but yeah, actually, let's try removing them from the gerrit queue first | 23:29 |
| fungi | yeah, i thought you meant the gerrit api's kill command | 23:30 |
| jeblair | killed 3bdfffc9 | 23:31 |
| fungi | oh, in fact i should have paid attention to the timestamps on the ssh processes for the cgit user | 23:31 |
| fungi | they were all from 19:32 utc | 23:31 |
| jeblair | yse they are old aren't they | 23:31 |
| jeblair | well, they are 1 hour old | 23:32 |
| fungi | i make that 4 hours old | 23:32 |
| *** ryanpetrello has quit IRC | 23:32 | |
| jeblair | i am bad at math | 23:32 |
| fungi | it's okay ;) | 23:32 |
| * fungi is awful at math | 23:33 | |
| fungi | the easier the math, the more likely i am to get it wrong | 23:33 |
| jeblair | so i killed the other 3 that were running; the others are still waiting; strangely... | 23:33 |
| jeblair | nothing has changed on git.o.o | 23:33 |
| fungi | huh | 23:33 |
| fungi | i'll kill those ssh sessions then | 23:33 |
| jeblair | sounds good | 23:33 |
| fungi | killed | 23:33 |
| fungi | and no new ones appearing (yet) | 23:34 |
| jeblair | ok, next how about i kill all the waiting gerrit processes | 23:34 |
| clarkb | Alex_Gaynor: I would check with zul if cffi is an actual problem. It might be cloud archived if you need it | 23:34 |
| fungi | jeblair: please do. at least the ones for cgit@git anyway | 23:34 |
| jeblair | ya | 23:35 |
| *** ^d has quit IRC | 23:35 | |
| jeblair | fungi: done | 23:36 |
| fungi | the start times on the ssh processes were from when i tried to trigger a full replication run during the infra meeting, back when repo permissions were wrong, so maybe it was just confused | 23:37 |
| jeblair | fungi: why don't you trigger a single-project replication now ? | 23:37 |
| fungi | and yeah, no new ssh sessions appearing yet. i'll try to replicate something small like gear | 23:37 |
| fungi | 3c1b62a7 23:38:08.922 push git@github.com:openstack-infra/gear.git | 23:38 |
| jeblair | i wonder if it's permanently stuck wrt git.o.o | 23:38 |
| jeblair | fungi: how about we restart gerrit | 23:39 |
| fungi | could be, though that replication job vanished from the queue | 23:39 |
| *** mriedem has joined #openstack-infra | 23:39 | |
| fungi | but yeah, /var/lib/git/openstack-infra/gear.git/refs/heads/ is still empty | 23:39 |
| jeblair | fungi: yeah, but that was for github; i never saw one for git.o.o | 23:39 |
| fungi | oh | 23:40 |
| fungi | right, i had tunnel vision there | 23:40 |
| fungi | restarting gerrit | 23:40 |
| fungi | it's up again now | 23:41 |
| fungi | and it has "Replicate All Projects" in its queue at the moment | 23:41 |
| *** rcleere has quit IRC | 23:42 | |
| pleia2 | it's going! | 23:42 |
| jeblair | yay! | 23:42 |
| pleia2 | https://git.openstack.org/cgit/openstack-dev/openstack-qa/ :D | 23:42 |
| fungi | git processes! | 23:42 |
| fungi | i see them in ps | 23:42 |
| fungi | we (i?) clearly confused gerrit earlier | 23:42 |
| jeblair | [2013-08-13 23:42:31,240] ERROR com.google.gerrit.server.git.PushReplication : Cannot replicate to git@github.com:openstack-infra/meetbot.git | 23:42 |
| fungi | that's... exciting | 23:43 |
| jeblair | (there are some other ones that are excpected (melange, pypi-mirror), but that one i think is an actual problem) | 23:43 |
| * fungi looks at its settings on github | 23:43 | |
| pleia2 | heading out to the train station to get to a meetup in mt view, will check in on the train | 23:44 |
| jeblair | pleia2: cgit seems to be working :) | 23:44 |
| fungi | pleia2: great work! | 23:44 |
| pleia2 | yes, I'm very pleased! | 23:44 |
| clarkb | hurray | 23:44 |
| pleia2 | and looks like it's all done syncing | 23:45 |
| clarkb | we should s/https/git/ in the gerrit git prep script soonish (I think we need zuul to check that replication to git.o.o has happened first) | 23:45 |
| pleia2 | good to know that it's quick | 23:45 |
| jeblair | except for nova | 23:45 |
| pleia2 | git clone from git:// works too, so hooray for git-daemon | 23:45 |
| jeblair | clarkb: well, erm... | 23:46 |
| fungi | i don't see anything out of the ordinary with openstack-infra/meetbot.git on github | 23:46 |
| pleia2 | hah, yeah, nova | 23:46 |
| jeblair | clarkb: it should be able to fetch most things from git.o.o, then fetch changes from zuul | 23:46 |
| clarkb | jeblair: I have given my email address to someone that can give me the email address of the people to ask about whether or not it is safe to use that hpcloud account | 23:46 |
| clarkb | jeblair: that works too | 23:46 |
| fungi | nova finished finally, at least | 23:47 |
| jeblair | clarkb: it shouldn't need to check that replication has happened, because zuul has refs for critical things | 23:47 |
| clarkb | jeblair: gotcha, we just need to get enough of a seed there that we don't spend a bunch of time with https | 23:47 |
| jeblair | yep | 23:49 |
| jeblair | clarkb: we should also, very very soon, change our vcsrepo refs in puppet :) | 23:50 |
| clarkb | ++ | 23:50 |
| clarkb | pleia2: do you want to make the first pass at updating infra related things? | 23:50 |
| *** dina_belova has joined #openstack-infra | 23:50 | |
| fungi | the tip of master on github and gerrit for meetbot match shas, so i don't think there's anything wrong with the repo there | 23:51 |
| jeblair | fungi: weird | 23:51 |
| fungi | and the project config for it on github matches the way we have our other projects set from what i can tell | 23:51 |
| clarkb | maybe some other underlying problem? maybe git fsck it? | 23:52 |
| fungi | mmm, maybe. fscking | 23:53 |
| *** dina_belova has quit IRC | 23:53 | |
| fungi | fscks clean on my checkout. is it safe to git fsck the copy in ~gerrit on review.o.o? | 23:53 |
| fungi | er, in ~gerrit2 | 23:54 |
| jeblair | fungi: yes -- however, there is a nightly repack already | 23:54 |
| fungi | eh | 23:54 |
| fungi | i'm going to suspect a transient, passing issue on github | 23:54 |
| jeblair | so it might detect or correct errors. | 23:54 |
| jeblair | fungi: that seems reasonable for now. | 23:54 |
| clarkb | fungi: would not be surprised if that was the issue | 23:55 |
| fungi | anyway, fsck runs clean on gerrit's copy as well | 23:55 |
| fungi | so really, no sign of anything broken, and this would not be the first time github randomly rejected something we tried which normally works | 23:56 |
| fungi | if the repo for it starts getting behind on github, then i'll worry | 23:56 |
| *** ryanpetrello has joined #openstack-infra | 23:58 | |
| clarkb | jeblair: why does the graphite graph that you gave mtreinish earlier break if I s/check/gate/ ? | 23:59 |
| jeblair | clarkb: no data in gate yet? | 23:59 |
| jeblair | did we merge that change? | 23:59 |
| fungi | i think i approved it | 23:59 |
| * fungi looks | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
