| anteaya | pleia2: I finally got impress to do what I want | 00:00 |
|---|---|---|
| *** banix has quit IRC | 00:00 | |
| jog0 | horizon appears to be failing grenade | 00:00 |
| jog0 | http://status.openstack.org/elastic-recheck/ | 00:00 |
| anteaya | I'll probably be asking you for a review of my talks when we meet | 00:00 |
| anteaya | after beach | 00:00 |
| * mikal is trying to think of a way of being findable, apart from handing out his phone number | 00:00 | |
| pleia2 | anteaya: oh! I meant beach following saturday, I dont' get in until sunday | 00:00 |
| mikal | nononon | 00:01 |
| anteaya | ah okay | 00:01 |
| mikal | You have a meetup on Saturday! | 00:01 |
| pleia2 | anteaya: happy to review your slides this week though | 00:01 |
| anteaya | I get in Saturday at 2:10pm | 00:01 |
| *** banix has joined #openstack-infra | 00:01 | |
| anteaya | it appears pleia2 gets in Sunday | 00:01 |
| anteaya | pleia2: working on it, like I said I finally figured out how to talk to impress so it does what I want | 00:02 |
| anteaya | basically I have 2.5 rough cuts | 00:02 |
| anteaya | needs more polish | 00:02 |
| jog0 | http://logs.openstack.org/83/64383/7/gate/gate-grenade-dsvm/f9cbc0b/console.html#_2013-12-30_23_40_21_759 | 00:03 |
| mikal | Is anyone else seeing rabbitmq-server having problems starting on Ubuntu 12.04 with devstack? | 00:04 |
| pleia2 | anteaya: impress take some practice :) | 00:04 |
| mikal | Or is 12.04 just too old now and I should build a new image? | 00:04 |
| anteaya | pleia2: I got what I want, took me 3 days | 00:04 |
| jeblair | mikal: all our testing is on 12.04 | 00:04 |
| mikal | Hmmm | 00:04 |
| jog0 | jeblair: where are the horizon logs? | 00:05 |
| mikal | I consistently get rabbitmq-server start errors after a fresh install on a clean image | 00:05 |
| jog0 | jeblair: http://logs.openstack.org/83/64383/7/gate/gate-grenade-dsvm/f9cbc0b/console.html#_2013-12-30_23_40_21_759 | 00:05 |
| mikal | jeblair: to dyou have cloudarchive installed? | 00:05 |
| jeblair | mikal: i _think_ so. | 00:05 |
| * mikal tries that | 00:05 | |
| lifeless | we had it disabled because of mongo for a while | 00:05 |
| clarkb | jog0: I don't think horizon is starting | 00:05 |
| lifeless | mikal: rabbit blowing up - how ? | 00:06 |
| clarkb | jog0: so there is no log | 00:06 |
| jog0 | clarkb: we don't upgrade horizon in grenade AFAIK | 00:06 |
| jog0 | we just use the old one | 00:06 |
| clarkb | jog0: really? | 00:06 |
| *** dims has quit IRC | 00:06 | |
| jog0 | clarkb: http://logs.openstack.org/83/64383/7/gate/gate-grenade-dsvm/f9cbc0b/logs/old/screen-horizon.txt.gz' | 00:06 |
| *** croberts has quit IRC | 00:06 | |
| clarkb | jog0: right so it started there correct? | 00:06 |
| clarkb | jog0: or we would've seen the echo'd error | 00:07 |
| clarkb | then after everything is upgraded and services are restarted horizon doesn't restart | 00:07 |
| jog0 | the devtack old horizon.sh test passed | 00:07 |
| *** rongze has joined #openstack-infra | 00:07 | |
| clarkb | hmm looks like http://logs.openstack.org/42/64442/3/check/check-grenade-dsvm/1ba796a/logs/ doesn't have a horizon log in new either and that test passed | 00:08 |
| jog0 | clarkb: devstack old sets up horizon and grenade pulls everything but horizon up | 00:08 |
| clarkb | so maybe we don't upgrade horizon (why wouldn't we upgrade everything?) | 00:08 |
| jog0 | its not even stopped | 00:08 |
| jog0 | clarkb: AFAIK its a bug | 00:08 |
| clarkb | jog0: willing to bet old horizon can't talk to new whatever or has its libs changed under it and it falls over | 00:09 |
| jog0 | clarkb: yeah | 00:09 |
| jog0 | but I think the issue here is we aren't collecting apache2 logs or something | 00:09 |
| jog0 | hmm we are only collecting error logs | 00:09 |
| jog0 | weird | 00:09 |
| clarkb | well the log file in old should have it unless old is collected earlier in the job and not at the end | 00:10 |
| jog0 | cd /opt/stack/old/hori | 00:10 |
| jog0 | zon && sudo tail -f /var/log/apache2/horizon_error.log || echo "horizon failed t | 00:10 |
| jog0 | o start" | tee "/opt/stack/old/status/stack/horizon.failure" | 00:10 |
| clarkb | oh it is being tailed in a screen and I bet that screen is completely killed | 00:11 |
| clarkb | so new should do the same thing imo | 00:11 |
| anteaya | jog0: do we have a bug filed for this yet? | 00:12 |
| jog0 | anteaya: not that I know of | 00:12 |
| anteaya | k | 00:12 |
| jog0 | yeah that screen is killed | 00:12 |
| *** rongze has quit IRC | 00:12 | |
| jog0 | clarkb: one of my patches changes that | 00:12 |
| clarkb | there is a bug for it | 00:12 |
| clarkb | chmouel: submitted one earlier this afternoon | 00:12 |
| jog0 | https://review.openstack.org/#/c/63965/4 | 00:13 |
| anteaya | https://bugs.launchpad.net/grenade/+bug/1265057 | 00:13 |
| uvirtbot | Launchpad bug 1265057 in openstack-ci "gate/check-grenade-dsvm: Horizon front page not functioning!" [Undecided,Incomplete] | 00:13 |
| anteaya | yup, just found it | 00:13 |
| jog0 | clarkb: ^ I'll run recheck on my patch and if we hit it horizon logs should have output there | 00:13 |
| *** banix has quit IRC | 00:14 | |
| clarkb | jog0: the problem with that change is it doesn't put the logs in new/ | 00:14 |
| clarkb | which may be confusing | 00:14 |
| clarkb | though better than nothing at this point I suppose | 00:14 |
| jog0 | clarkb: long term solution is to add a upgrade-horizon section to grenade | 00:15 |
| jog0 | shouldn't be hard, I can crank it out this afternoon | 00:17 |
| jog0 | err later this afternoon | 00:17 |
| *** dims has joined #openstack-infra | 00:18 | |
| *** dstanek has joined #openstack-infra | 00:20 | |
| *** rakhmerov has joined #openstack-infra | 00:29 | |
| *** rakhmerov has quit IRC | 00:34 | |
| *** dstanek has quit IRC | 00:41 | |
| *** ^d has quit IRC | 00:43 | |
| *** ^d has joined #openstack-infra | 00:44 | |
| *** banix has joined #openstack-infra | 00:45 | |
| *** b3nt_pin has quit IRC | 00:45 | |
| *** senk has joined #openstack-infra | 00:47 | |
| *** dstanek has joined #openstack-infra | 00:48 | |
| *** banix has quit IRC | 00:48 | |
| *** dcramer_ has quit IRC | 00:49 | |
| *** adalbas has quit IRC | 00:49 | |
| *** ^d has quit IRC | 00:49 | |
| *** hogepodge has quit IRC | 00:50 | |
| *** b3nt_pin has joined #openstack-infra | 00:51 | |
| *** dstanek has quit IRC | 00:52 | |
| *** Ryan_Lane has quit IRC | 00:53 | |
| *** mfer has joined #openstack-infra | 00:55 | |
| *** senk has quit IRC | 01:01 | |
| *** sarob has quit IRC | 01:03 | |
| *** sarob has joined #openstack-infra | 01:03 | |
| *** sarob has quit IRC | 01:08 | |
| *** rongze has joined #openstack-infra | 01:08 | |
| *** pcrews_ has quit IRC | 01:10 | |
| *** mfer has quit IRC | 01:12 | |
| *** rongze has quit IRC | 01:12 | |
| *** MarkAtwood has quit IRC | 01:12 | |
| *** senk has joined #openstack-infra | 01:15 | |
| *** senk has quit IRC | 01:16 | |
| *** senk has joined #openstack-infra | 01:18 | |
| *** dcramer_ has joined #openstack-infra | 01:20 | |
| *** banix has joined #openstack-infra | 01:22 | |
| *** praneshp has quit IRC | 01:23 | |
| *** praneshp has joined #openstack-infra | 01:25 | |
| *** rakhmerov has joined #openstack-infra | 01:30 | |
| *** senk has quit IRC | 01:33 | |
| *** rakhmerov has quit IRC | 01:34 | |
| *** senk has joined #openstack-infra | 01:36 | |
| *** nati_ueno has quit IRC | 01:39 | |
| *** nati_ueno has joined #openstack-infra | 01:40 | |
| *** rongze has joined #openstack-infra | 01:41 | |
| *** rongze has quit IRC | 01:41 | |
| *** rongze has joined #openstack-infra | 01:41 | |
| *** senk has quit IRC | 01:41 | |
| *** dstanek has joined #openstack-infra | 01:43 | |
| *** rnirmal has quit IRC | 01:43 | |
| *** nati_ueno has quit IRC | 01:44 | |
| *** malini_afk is now known as malini | 01:47 | |
| *** dstanek has quit IRC | 01:49 | |
| *** ^d has joined #openstack-infra | 01:55 | |
| *** ^d has quit IRC | 01:59 | |
| *** senk has joined #openstack-infra | 02:04 | |
| *** dstanek has joined #openstack-infra | 02:05 | |
| *** ryanpetrello has joined #openstack-infra | 02:05 | |
| *** senk has quit IRC | 02:08 | |
| *** ^d has joined #openstack-infra | 02:09 | |
| *** ^d has joined #openstack-infra | 02:09 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/devstack-gate: Gather horizon/apache2 logs https://review.openstack.org/64490 | 02:11 |
| *** banix has quit IRC | 02:11 | |
| *** ^d has quit IRC | 02:14 | |
| jog0 | clarkb: don't you hate it when things won't fail | 02:15 |
| *** dstanek has quit IRC | 02:16 | |
| clarkb | it is the worst | 02:17 |
| *** dstanek has joined #openstack-infra | 02:22 | |
| *** banix has joined #openstack-infra | 02:25 | |
| *** dstanek has quit IRC | 02:28 | |
| jog0 | clarkb sdague: first pass at horizon patch to grenade https://review.openstack.org/64491 | 02:28 |
| *** AaronGr is now known as AaronGr_Zzz | 02:28 | |
| fungi | does anybody recall the consensus on when project-specific tempest jobs get to vote on tempest changes in the gate (to prevent asymmetry)... during incubation or only after graduation? | 02:30 |
| *** rakhmerov has joined #openstack-infra | 02:30 | |
| mikal | jog0: grenade is broken at the moment, yes? | 02:33 |
| *** praneshp has joined #openstack-infra | 02:34 | |
| jog0 | mikal: mostly yes | 02:34 |
| jog0 | mikal: see https://bugs.launchpad.net/grenade/+bug/1265057 | 02:34 |
| uvirtbot | Launchpad bug 1265057 in grenade "gate/check-grenade-dsvm: Horizon front page not functioning!" [Undecided,Confirmed] | 02:34 |
| mikal | jog0: cool. No problems, it just means I wont dig into this failure I am seeing for now. | 02:35 |
| *** rakhmerov has quit IRC | 02:35 | |
| jog0 | mikal: well wedn't know why its broken yet actually | 02:35 |
| jog0 | trying to figure that out and have a grenade patch to attempt to fix what I think is the root caue | 02:36 |
| jog0 | cause* | 02:36 |
| jog0 | mikal: LCA will be fun with most of infra down there and all | 02:37 |
| mikal | Yeah, I'm looking forward to it | 02:38 |
| anteaya | jog0: are you speaking, I just looked at the schedule, if you are I missed your talk in the schedule | 02:41 |
| *** praneshp has quit IRC | 02:42 | |
| mikal | Sigh | 02:44 |
| mikal | I'm triaging nova bugs, and I keep seeing packagers who are just plain rude | 02:44 |
| mikal | Its an interesting way to work with upstream | 02:44 |
| jog0 | anteaya: at the OpenStack mini-conf | 02:46 |
| anteaya | jog0: ah ha | 02:46 |
| anteaya | mikal: that is a shame | 02:47 |
| mikal | The "I don't want to use tox" one is particularly awesome | 02:47 |
| anteaya | mikal: wow | 02:47 |
| anteaya | the packagers personal decision should affect all those using the packages? | 02:48 |
| anteaya | packager's | 02:48 |
| mikal | https://bugs.launchpad.net/nova/+bug/1235955 | 02:48 |
| uvirtbot | Launchpad bug 1235955 in nova "testsuite needs upgrading" [Undecided,Incomplete] | 02:48 |
| mikal | I think what he's really saying is "please ignore this bug" | 02:48 |
| anteaya | hmmmm, he will be at lca | 02:49 |
| anteaya | he spent some time in -neutron | 02:50 |
| anteaya | we talked | 02:50 |
| jog0 | mikal: whoa toxicity hehe | 02:50 |
| jog0 | and gentoo lolz | 02:50 |
| anteaya | he has struck me as a good tester | 02:51 |
| anteaya | but for some reason takes a powerful stance in one direction | 02:52 |
| mikal | And is rude? | 02:52 |
| jog0 | anteaya: thats the gentoo talking | 02:52 |
| mikal | LOL | 02:52 |
| anteaya | which unfortunately results in obstacles materializing where there doesn't need to be any | 02:52 |
| *** MarkAtwood has joined #openstack-infra | 02:53 | |
| mikal | anteaya: I can't find him registered for LCA under either his name or his email address | 02:53 |
| jog0 | mikal: btw can you do a quick review of https://review.openstack.org/#/c/64235/ | 02:53 |
| jog0 | lifeless: ^ | 02:54 |
| jog0 | I want to unwege the gantt efforts | 02:54 |
| mikal | jog0: I will require an explaination of what gantt is before I can help much | 02:54 |
| anteaya | mikal: oh, he said he would be there | 02:54 |
| anteaya | perhaps he has changed his mind | 02:54 |
| mikal | jog0: I have zero relevant emails containing the word "gantt" | 02:54 |
| jog0 | mikal: nova-scheduler repo clone | 02:55 |
| mikal | Oh nice. I AM GANTT-CORE. | 02:55 |
| mikal | HP better send me another badge. | 02:55 |
| anteaya | mikal: ha ha ha | 02:55 |
| jog0 | mikal: its nova-core + lifeless | 02:55 |
| mikal | Heh | 02:56 |
| anteaya | you'd think someone would tell you when they make you core | 02:56 |
| mikal | jog0: I think my point is that nova-core don't know what it is, so they wont be looking at it much | 02:56 |
| jog0 | mikal: that may be true | 02:56 |
| mikal | anteaya: hey -- I was ironic core without realizeing | 02:56 |
| mikal | I'm kind of a big deal in openstack you see | 02:56 |
| anteaya | mikal: hahaha | 02:56 |
| anteaya | apparently | 02:57 |
| *** praneshp has joined #openstack-infra | 02:57 | |
| anteaya | you should be getting badges aplenty | 02:57 |
| jog0 | russellb made a quite announcement somewhere about that | 02:57 |
| anteaya | about mikal being a big deal in openstack? | 02:57 |
| jog0 | mikal: well until my patch (or one like it) is merged it doesn't matter who notices since nothing will merge | 02:57 |
| anteaya | mikal: I guess I would just leave that bug as incomplete, and give it another chunk of time | 02:59 |
| mikal | anteaya: the gentoo one? Yeah, I remembered that I was only triaging for fun, and skipped it | 02:59 |
| anteaya | cool | 02:59 |
| anteaya | triaging for fun is a good approach | 02:59 |
| *** ^d has joined #openstack-infra | 03:00 | |
| *** ^d has joined #openstack-infra | 03:00 | |
| mikal | jog0: have a +2 on that patchset | 03:01 |
| mikal | Cause I care | 03:01 |
| jog0 | mikal: thanks | 03:02 |
| *** malini is now known as malini_afk | 03:02 | |
| jog0 | this is step 1 on my plan to prod people into working on gantt | 03:02 |
| jog0 | by saying oh look you can now merge code | 03:02 |
| mikal | Heh | 03:03 |
| mikal | What we need to do is lock a couple of people in a room | 03:03 |
| mikal | Its where bikeshedders go it seems | 03:03 |
| *** rakhmerov has joined #openstack-infra | 03:04 | |
| jog0 | sigh | 03:04 |
| jog0 | http://docs.openstack.org/infra/ is not working for me | 03:04 |
| jog0 | unless I am just doing it wrong | 03:04 |
| mikal | I get forbidden | 03:04 |
| mikal | Perhaps I'm not a big enough deal in infra... | 03:04 |
| jog0 | mikal: same forbidden | 03:04 |
| mikal | jog0: yeah, you're not invited either | 03:05 |
| jog0 | and http://docs.openstack.org/infra/elastic-recheck doesn't work either | 03:05 |
| mikal | We should go back to nova where they love us | 03:05 |
| jog0 | ^_^ | 03:05 |
| mikal | nova is where the cool people is at | 03:05 |
| mikal | He says, being over there and all | 03:05 |
| anteaya | I'm forbidden | 03:05 |
| jog0 | clarkb: ^^ docs.o.o/infra/ | 03:05 |
| mikal | anteaya: to nova with you! | 03:06 |
| mikal | Mutiny! | 03:06 |
| *** ryanpetrello has quit IRC | 03:07 | |
| *** dstanek has joined #openstack-infra | 03:08 | |
| anteaya | mikal: okay | 03:11 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: Collect list of installed packages at end of run https://review.openstack.org/63551 | 03:12 |
| *** harlowja is now known as harlowja_away | 03:20 | |
| *** rongze has quit IRC | 03:22 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Explicitly document requirements for 3rd party testing https://review.openstack.org/63478 | 03:22 |
| *** rongze has joined #openstack-infra | 03:23 | |
| greghaynes | dddd | 03:27 |
| greghaynes | s | 03:27 |
| *** rongze has quit IRC | 03:27 | |
| fungi | mikal: jog0: it doesn't appear the elastic-recheck-docs job has ever run... have any e-r changes merged since that job was added? | 03:31 |
| greghaynes | sorry :) | 03:32 |
| *** reed has quit IRC | 03:37 | |
| *** nati_ueno has joined #openstack-infra | 03:44 | |
| anteaya | jog0: commented | 03:45 |
| *** michchap has quit IRC | 03:46 | |
| *** michchap has joined #openstack-infra | 03:47 | |
| jog0 | fungi: not that I know of | 04:03 |
| *** praneshp has quit IRC | 04:03 | |
| jog0 | anteaya: awesome! thanks | 04:04 |
| lifeless | jog0: its not quite that, | 04:23 |
| lifeless | mikal: yes, we're not expecting nova core to look at it, only the folk that volunteered, for now | 04:23 |
| *** SergeyLukjanov has joined #openstack-infra | 04:35 | |
| *** SergeyLukjanov has quit IRC | 04:35 | |
| *** SergeyLukjanov has joined #openstack-infra | 04:37 | |
| *** vogxn has joined #openstack-infra | 04:46 | |
| *** jerryz has quit IRC | 04:46 | |
| *** praneshp has joined #openstack-infra | 04:48 | |
| *** dguitarbite has joined #openstack-infra | 04:53 | |
| *** ^d has quit IRC | 05:06 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Setup tooling and jobs for openstack Gerrit ver 2.8 and beyond https://review.openstack.org/60348 | 05:15 |
| *** SergeyLukjanov has quit IRC | 05:15 | |
| *** SergeyLukjanov has joined #openstack-infra | 05:16 | |
| *** chandankumar has joined #openstack-infra | 05:23 | |
| *** senk has joined #openstack-infra | 05:31 | |
| *** hogepodge has joined #openstack-infra | 05:55 | |
| *** SergeyLukjanov has quit IRC | 05:57 | |
| *** ArxCruz has quit IRC | 06:11 | |
| *** banix has quit IRC | 06:12 | |
| *** dguitarbite has quit IRC | 06:12 | |
| *** nati_uen_ has joined #openstack-infra | 06:14 | |
| *** nati_ueno has quit IRC | 06:15 | |
| *** hogepodge has quit IRC | 06:22 | |
| *** rongze has joined #openstack-infra | 06:25 | |
| *** rongze has quit IRC | 06:30 | |
| *** dguitarbite has joined #openstack-infra | 06:34 | |
| *** garyk has joined #openstack-infra | 06:51 | |
| *** dstanek has quit IRC | 06:56 | |
| *** HenryG_ has quit IRC | 07:19 | |
| *** rongze has joined #openstack-infra | 07:27 | |
| *** loq_mac has joined #openstack-infra | 07:32 | |
| *** rongze has quit IRC | 07:32 | |
| *** Ryan_Lane has joined #openstack-infra | 07:41 | |
| *** HenryG has joined #openstack-infra | 07:48 | |
| *** SergeyLukjanov has joined #openstack-infra | 07:49 | |
| *** thomasbiege has joined #openstack-infra | 07:50 | |
| *** thomasbiege has quit IRC | 07:53 | |
| *** thomasbiege has joined #openstack-infra | 07:56 | |
| *** thomasbiege has quit IRC | 07:57 | |
| *** MarkAtwood has quit IRC | 08:10 | |
| *** loq_mac has quit IRC | 08:11 | |
| *** Ryan_Lane has quit IRC | 08:24 | |
| *** rongze has joined #openstack-infra | 08:28 | |
| *** rongze has quit IRC | 08:32 | |
| *** SergeyLukjanov has quit IRC | 08:34 | |
| *** loq_mac has joined #openstack-infra | 08:43 | |
| *** loq_mac has quit IRC | 08:44 | |
| *** rakhmerov has quit IRC | 08:46 | |
| *** rakhmerov1 has joined #openstack-infra | 08:47 | |
| *** loq_mac has joined #openstack-infra | 08:49 | |
| *** loq_mac has quit IRC | 08:51 | |
| *** loq_mac has joined #openstack-infra | 08:52 | |
| *** rakhmerov1 has quit IRC | 08:59 | |
| *** loquacities has joined #openstack-infra | 08:59 | |
| *** rakhmerov has joined #openstack-infra | 09:00 | |
| *** loq_mac has quit IRC | 09:01 | |
| *** flaper87|afk is now known as flaper87 | 09:06 | |
| *** flaper87 is now known as flaper87|afk | 09:12 | |
| *** loquacities has quit IRC | 09:20 | |
| *** flaper87|afk is now known as flaper87 | 09:20 | |
| *** rongze has joined #openstack-infra | 09:29 | |
| *** rongze has quit IRC | 09:33 | |
| *** loq_mac has joined #openstack-infra | 09:37 | |
| *** praneshp has quit IRC | 09:43 | |
| *** praneshp has joined #openstack-infra | 09:55 | |
| *** praneshp has quit IRC | 10:10 | |
| *** loq_mac has quit IRC | 10:32 | |
| *** sileht has quit IRC | 10:32 | |
| *** loq_mac has joined #openstack-infra | 10:32 | |
| *** loq_mac has quit IRC | 10:33 | |
| *** thomasbiege has joined #openstack-infra | 10:39 | |
| *** thomasbiege1 has joined #openstack-infra | 10:40 | |
| *** vogxn has quit IRC | 10:42 | |
| *** thomasbiege has quit IRC | 10:43 | |
| *** thomasbiege1 has quit IRC | 10:48 | |
| *** sileht has joined #openstack-infra | 10:55 | |
| chmouel | if someone else find that useful https://gist.github.com/chmouel/8195145 | 10:56 |
| chmouel | parse the failed jobs and save the consoel.html in /tmp | 10:56 |
| *** flaper87 is now known as flaper87|afk | 11:07 | |
| *** SergeyLukjanov has joined #openstack-infra | 11:10 | |
| *** SergeyLukjanov has quit IRC | 11:19 | |
| *** xchu has joined #openstack-infra | 11:26 | |
| *** adam_g has quit IRC | 11:28 | |
| *** ArxCruz has joined #openstack-infra | 11:28 | |
| *** adam_g has joined #openstack-infra | 11:28 | |
| *** adam_g has quit IRC | 11:28 | |
| *** adam_g has joined #openstack-infra | 11:28 | |
| *** rongze has joined #openstack-infra | 11:31 | |
| *** hashar has joined #openstack-infra | 11:34 | |
| *** rongze has quit IRC | 11:36 | |
| *** adam_g has quit IRC | 11:50 | |
| *** adam_g has joined #openstack-infra | 11:50 | |
| *** adam_g has quit IRC | 11:50 | |
| *** adam_g has joined #openstack-infra | 11:50 | |
| *** flaper87|afk is now known as flaper87 | 12:06 | |
| *** adam_g has quit IRC | 12:11 | |
| *** adam_g has joined #openstack-infra | 12:11 | |
| *** adam_g has quit IRC | 12:11 | |
| *** adam_g has joined #openstack-infra | 12:11 | |
| *** adam_g has quit IRC | 12:17 | |
| *** adam_g has joined #openstack-infra | 12:17 | |
| *** adam_g has quit IRC | 12:18 | |
| *** adam_g has joined #openstack-infra | 12:18 | |
| *** CaptTofu has quit IRC | 12:38 | |
| *** CaptTofu has joined #openstack-infra | 12:38 | |
| *** flaper87 is now known as flaper87|afk | 12:49 | |
| *** adalbas has joined #openstack-infra | 13:06 | |
| *** flaper87|afk is now known as flaper87 | 13:10 | |
| *** dguitarbite has quit IRC | 13:27 | |
| *** dguitarbite has joined #openstack-infra | 13:27 | |
| *** che-arne has quit IRC | 13:31 | |
| *** CaptTofu has quit IRC | 13:36 | |
| *** CaptTofu has joined #openstack-infra | 13:36 | |
| *** che-arne has joined #openstack-infra | 13:37 | |
| *** hashar has quit IRC | 13:41 | |
| *** hashar has joined #openstack-infra | 13:44 | |
| *** smarcet has joined #openstack-infra | 13:45 | |
| *** chandankumar has quit IRC | 14:07 | |
| *** hashar has quit IRC | 14:20 | |
| *** che-arne has quit IRC | 14:29 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/zuul: Update the url for the zuul git repo https://review.openstack.org/62676 | 14:32 |
| *** rakhmerov has quit IRC | 14:34 | |
| *** rongze has joined #openstack-infra | 14:35 | |
| *** che-arne has joined #openstack-infra | 14:35 | |
| *** rongze has quit IRC | 14:39 | |
| *** hashar has joined #openstack-infra | 14:40 | |
| *** dstanek has joined #openstack-infra | 14:46 | |
| *** SergeyLukjanov has joined #openstack-infra | 14:46 | |
| *** banix has joined #openstack-infra | 14:49 | |
| *** rakhmerov has joined #openstack-infra | 14:51 | |
| *** SergeyLukjanov has quit IRC | 14:53 | |
| *** croberts has joined #openstack-infra | 14:55 | |
| *** banix has quit IRC | 14:55 | |
| *** croberts has quit IRC | 14:57 | |
| *** croberts has joined #openstack-infra | 14:57 | |
| *** ryanpetrello has joined #openstack-infra | 15:00 | |
| *** malini_afk is now known as malini | 15:00 | |
| *** ryanpetrello has quit IRC | 15:05 | |
| *** DennyZhang has joined #openstack-infra | 15:10 | |
| *** CaptTofu has quit IRC | 15:10 | |
| *** CaptTofu has joined #openstack-infra | 15:10 | |
| *** rongze has joined #openstack-infra | 15:14 | |
| *** rongze has quit IRC | 15:19 | |
| *** malini has left #openstack-infra | 15:23 | |
| *** rongze has joined #openstack-infra | 15:24 | |
| *** garyk has quit IRC | 15:25 | |
| *** xchu has quit IRC | 15:30 | |
| mordred | oh look. it's greghaynes | 15:31 |
| *** ryanpetrello has joined #openstack-infra | 15:32 | |
| *** ryanpetrello has quit IRC | 15:33 | |
| *** milki has quit IRC | 15:34 | |
| *** rnirmal has joined #openstack-infra | 15:39 | |
| *** DennyZha` has joined #openstack-infra | 15:39 | |
| *** DennyZhang has quit IRC | 15:41 | |
| mordred | mikal: I have responded to https://bugs.launchpad.net/nova/+bug/1235955 | 15:42 |
| uvirtbot | Launchpad bug 1235955 in nova "testsuite needs upgrading" [Undecided,Incomplete] | 15:42 |
| chmouel | mordred: well answered :) | 15:44 |
| mordred | chmouel: thanks! | 15:47 |
| * chmouel is looking for a osreactions for that ;) | 15:47 | |
| *** w_ is now known as olaph | 15:47 | |
| mordred | chmouel: do you think that perhaps we need more prominent instructions on how to run tests outside of a venv? | 15:47 |
| mordred | chmouel: :) | 15:47 |
| chmouel | mordred: I don't think so for most developers but I guess for packagers | 15:48 |
| *** dmitrydz has joined #openstack-infra | 15:48 | |
| chmouel | mordred: but even zigo is happy about the current state of how we are having the tests ;) | 15:48 |
| mordred | chmouel: I consider that an excellent achievement by us! | 15:51 |
| *** hashar has quit IRC | 15:53 | |
| *** smarcet has left #openstack-infra | 15:55 | |
| *** rongze has quit IRC | 15:56 | |
| *** rongze has joined #openstack-infra | 15:57 | |
| dkranz | mordred: I frequently run into the fact that a lot of people out there use nose, and that the OpenStack project thinks it sucks and does not care about making it easy or even possible to use nose. | 15:58 |
| mordred | dkranz: well, the problem isn't that we think it sucks | 15:58 |
| dkranz | mordred: Is there some blog post or something that can be pointed to explaining why this is so? | 15:58 |
| mordred | it's that it doesn't follow the python unittest protocol | 15:59 |
| dkranz | mordred: I understand. | 15:59 |
| dkranz | mordred: Just thought it would be helpful to have something to point people at whenever this isssue comes up. | 15:59 |
| mordred | dkranz: I agree though- we need a blog post or something on the subject | 15:59 |
| mordred | ++ | 15:59 |
| dkranz | mordred: Thanks! | 15:59 |
| dkranz | mordred: I would volunteer but am a bit too ignorant about this | 15:59 |
| chmouel | what's the python untittest protocol? | 16:00 |
| *** DennyZha` has quit IRC | 16:00 | |
| chmouel | i mean is there a PEP definiting it? | 16:00 |
| mordred | chmouel: one sec... | 16:00 |
| mordred | chmouel: http://docs.python.org/2/library/unittest.html#load-tests-protocol | 16:01 |
| *** dmitrydz has quit IRC | 16:01 | |
| mordred | chmouel: this is the standard way to support programmatic injection of new tests into the discovery system | 16:01 |
| *** croberts has quit IRC | 16:01 | |
| chmouel | ah great thanks for the link | 16:02 |
| *** croberts has joined #openstack-infra | 16:02 | |
| *** nati_uen_ has quit IRC | 16:02 | |
| mordred | chmouel: nose does not support it - so using things like testscenarios or testresources become teh suck | 16:02 |
| chmouel | mordred: it would be nice there was a PEP for that so we can just point to people that nose doesn't support that pep | 16:04 |
| mordred | hrm. nod | 16:05 |
| mordred | lifeless: ^^ do you know if nick made a pep around load_tests? | 16:05 |
| *** boris-42 has joined #openstack-infra | 16:09 | |
| *** zigo has quit IRC | 16:10 | |
| *** HenryG has quit IRC | 16:10 | |
| *** garyk has joined #openstack-infra | 16:12 | |
| *** milki has joined #openstack-infra | 16:15 | |
| *** sdague_ has joined #openstack-infra | 16:18 | |
| *** briancli1e has joined #openstack-infra | 16:19 | |
| *** dcramer__ has joined #openstack-infra | 16:22 | |
| *** morganfainberg has quit IRC | 16:23 | |
| *** persia has quit IRC | 16:23 | |
| *** persia has joined #openstack-infra | 16:23 | |
| *** briancline has quit IRC | 16:23 | |
| *** sdague has quit IRC | 16:23 | |
| *** vishy has quit IRC | 16:23 | |
| *** dcramer_ has quit IRC | 16:23 | |
| *** freyes_ has quit IRC | 16:23 | |
| *** morganfainberg has joined #openstack-infra | 16:23 | |
| *** sdague_ is now known as sdague | 16:24 | |
| *** freyes_ has joined #openstack-infra | 16:24 | |
| *** vishy has joined #openstack-infra | 16:24 | |
| *** persia is now known as Guest13984 | 16:24 | |
| *** morganfainberg has quit IRC | 16:24 | |
| *** morganfainberg has joined #openstack-infra | 16:24 | |
| openstackgerrit | Monty Taylor proposed a change to openstack/requirements: Add python-openstackclient https://review.openstack.org/64562 | 16:26 |
| *** banix has joined #openstack-infra | 16:30 | |
| *** dcramer__ has quit IRC | 16:31 | |
| *** senk has joined #openstack-infra | 16:31 | |
| *** jcooley_ has joined #openstack-infra | 16:31 | |
| *** reed has joined #openstack-infra | 16:33 | |
| *** mika has quit IRC | 16:33 | |
| *** changbl has quit IRC | 16:34 | |
| *** SergeyLukjanov has joined #openstack-infra | 16:34 | |
| *** SergeyLukjanov has quit IRC | 16:34 | |
| *** mika has joined #openstack-infra | 16:34 | |
| *** mika has joined #openstack-infra | 16:34 | |
| *** banix has quit IRC | 16:35 | |
| *** AaronGr_Zzz is now known as AaronGr | 16:36 | |
| *** changbl has joined #openstack-infra | 16:37 | |
| *** ^d has joined #openstack-infra | 16:43 | |
| *** ^d has joined #openstack-infra | 16:43 | |
| *** sileht_ has joined #openstack-infra | 16:45 | |
| *** morganfainberg has quit IRC | 16:47 | |
| *** senk has quit IRC | 16:49 | |
| *** senk has joined #openstack-infra | 16:52 | |
| *** mikap has joined #openstack-infra | 16:53 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Add niceness gate for operations-guide https://review.openstack.org/64567 | 16:54 |
| *** senk has quit IRC | 16:55 | |
| *** senk has joined #openstack-infra | 16:56 | |
| *** sileht has quit IRC | 16:56 | |
| *** morganfainberg has joined #openstack-infra | 16:56 | |
| *** bknudson has quit IRC | 16:56 | |
| *** mika has quit IRC | 16:56 | |
| *** morganfainberg has quit IRC | 16:56 | |
| *** bknudson has joined #openstack-infra | 16:56 | |
| *** morganfainberg has joined #openstack-infra | 16:56 | |
| *** Guest13984 has quit IRC | 16:56 | |
| *** persia_ has joined #openstack-infra | 16:56 | |
| *** bknudson has quit IRC | 16:56 | |
| *** bknudson has joined #openstack-infra | 16:56 | |
| *** mikap is now known as mika | 16:56 | |
| *** morganfainberg has quit IRC | 16:57 | |
| *** morganfainberg has joined #openstack-infra | 16:57 | |
| *** flaper87 is now known as flaper87|afk | 16:58 | |
| *** persia_ is now known as Guest65285 | 16:58 | |
| *** morganfainberg has quit IRC | 16:59 | |
| *** morganfainberg has joined #openstack-infra | 17:02 | |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Add niceness gate for operations-guide https://review.openstack.org/64567 | 17:05 |
| *** praneshp has joined #openstack-infra | 17:10 | |
| *** fbo is now known as fbo_away | 17:11 | |
| *** AaronGr is now known as AaronGr_Zzz | 17:24 | |
| *** senk has quit IRC | 17:30 | |
| *** krotscheck has joined #openstack-infra | 17:34 | |
| *** praneshp has quit IRC | 17:38 | |
| *** Hefeweizen has joined #openstack-infra | 17:39 | |
| *** rcarrillocruz1 has quit IRC | 17:40 | |
| *** senk has joined #openstack-infra | 17:41 | |
| *** senk has quit IRC | 17:43 | |
| *** senk has joined #openstack-infra | 17:43 | |
| *** rongze has quit IRC | 17:44 | |
| *** rongze has joined #openstack-infra | 17:45 | |
| *** rongze has quit IRC | 17:49 | |
| *** tma996 has joined #openstack-infra | 17:52 | |
| zaro | morning | 17:59 |
| krotscheck | yup | 18:00 |
| mordred | look, it's a zaro and a krotscheck | 18:01 |
| *** praneshp has joined #openstack-infra | 18:07 | |
| *** lcestari has quit IRC | 18:12 | |
| greghaynes | mordred: hello hello :) | 18:12 |
| *** rongze has joined #openstack-infra | 18:16 | |
| jesusaurus | oh theres a greghaynes in here now | 18:16 |
| *** adam_g has quit IRC | 18:18 | |
| *** adam_g has joined #openstack-infra | 18:18 | |
| *** adam_g has quit IRC | 18:18 | |
| *** adam_g has joined #openstack-infra | 18:18 | |
| *** rongze has quit IRC | 18:20 | |
| *** adam_g has quit IRC | 18:20 | |
| *** adam_g has joined #openstack-infra | 18:20 | |
| *** adam_g has quit IRC | 18:20 | |
| *** adam_g has joined #openstack-infra | 18:20 | |
| *** rongze has joined #openstack-infra | 18:22 | |
| clarkb | morning | 18:23 |
| *** AaronGr_Zzz is now known as AaronGr | 18:23 | |
| zaro | clarkb: we gonna try the jenkins upgrade again? | 18:24 |
| clarkb | zaro: I think we should however, I think we should leave that up to fungi as a bunch of us will disappear at the end of the week leaving fungi with root | 18:25 |
| clarkb | a wild greghaynes has appeared | 18:26 |
| jeblair | clarkb: is it on jenkins-dev? | 18:26 |
| * mordred imagines greghaynes running around with crazy hair arms and legs akimbo foaming at the mouth | 18:26 | |
| clarkb | jeblair: yes zaro installed his scp plugin patch on jenkins-dev. Though we should probably upgrade jenkins-dev to whatever the current jenkins release is | 18:27 |
| clarkb | we are probably one or two behind at this point | 18:27 |
| jeblair | clarkb: the elastic recheck bot runs on logstash; does it need any special access (iptables, etc)? i want to move it to status.o.o. | 18:27 |
| greghaynes | haha | 18:27 |
| jeblair | clarkb: is jenkins-dev in sync with the other 3? | 18:28 |
| clarkb | jeblair: I don't think so, it consumes the public elasticsearch api | 18:28 |
| clarkb | jeblair: no jenkins-dev is out of sync because we are testing new jenkins with scp plugin | 18:28 |
| jeblair | clarkb: it's running a different jenkins version? | 18:28 |
| clarkb | correct | 18:28 |
| jeblair | how did it end up with a different version? | 18:29 |
| clarkb | because we upgraded the other three and they broke | 18:29 |
| clarkb | so we downgraded those and kept -dev for testing | 18:29 |
| jeblair | ah right, i'm caught up. | 18:29 |
| jeblair | if there isn't a compelling reason to upgrade jenkins past what's on dev, i'd stick with that | 18:30 |
| jeblair | (as the production target); to avoid introducing new problems | 18:30 |
| clarkb | let me check release notes | 18:30 |
| *** rongze has quit IRC | 18:30 | |
| *** LinuxJedi has quit IRC | 18:31 | |
| clarkb | we are one release behind. The newest release 1.544 has a handful of bug fixes, nothing that stands out as important to us | 18:32 |
| *** LinuxJedi has joined #openstack-infra | 18:32 | |
| *** rongze has joined #openstack-infra | 18:33 | |
| *** salv-orlando has quit IRC | 18:34 | |
| jeblair | clarkb: i'd lean toward upgrading one of the jenkins servers now; there's enough of this week left for us to either upgrade the others or downgrade according to fungi's wishes before we leave | 18:34 |
| jeblair | clarkb: (and we should even be able to get through the next two days on one if it comes to that) | 18:34 |
| clarkb | should we merge zaros pull request first? | 18:35 |
| jeblair | clarkb: how about after we production-test it? | 18:35 |
| clarkb | sure | 18:35 |
| zaro | you want to build on your own or would you like me to share my built hpi? | 18:36 |
| jeblair | we can copy from jenkins-dev | 18:36 |
| clarkb | zaro: I can just grab the one off of jenkins-dev | 18:36 |
| zaro | ohh yeah. that would work. | 18:37 |
| *** jcooley_ has quit IRC | 18:37 | |
| clarkb | give me a few minutes to settle in here and I will get things sorted | 18:37 |
| *** jcooley_ has joined #openstack-infra | 18:37 | |
| *** rongze has quit IRC | 18:37 | |
| clarkb | fungi: did you use package pins to downgrade or did you dpkg -i a deb? | 18:38 |
| clarkb | I'm guessing fungi is enjoying the beach | 18:38 |
| jeblair | clarkb: sudo dpkg -i jenkins_1.525_all.deb | 18:39 |
| *** jcooley_ has quit IRC | 18:42 | |
| *** jerryz has joined #openstack-infra | 18:43 | |
| *** yfried has left #openstack-infra | 18:44 | |
| clarkb | putting jenkins02 in shutdown mode now | 18:46 |
| clarkb | zaro: I see a jpi, you installed via the web interface? | 18:49 |
| *** dims has quit IRC | 18:49 | |
| clarkb | zaro: I guess I will need the hpi file | 18:49 |
| jeblair | clarkb: they are the same | 18:49 |
| jeblair | clarkb: hpi==jpi (hudson -> jenkins) | 18:50 |
| clarkb | jeblair: gotcha | 18:50 |
| *** tma996 has quit IRC | 18:50 | |
| *** dims has joined #openstack-infra | 18:52 | |
| fungi | clarkb: yeah, sorry, was lunching... reading scrollback in just a sec | 18:55 |
| clarkb | just waiting for 02 to be quiet now | 18:55 |
| zaro | clarkb: getting you one shortly. | 19:00 |
| clarkb | zaro: its fine | 19:00 |
| clarkb | zaro: jeblair pointed out that the j and h are for jenkins and hudson. The contents are the same | 19:01 |
| fungi | clarkb: jeblair: zaro: i'm cool with upgrading one of the jenkins masters again, checking the problem we know we hit last time, and then seeing what else crops up. i did the last downgrade quite easily, so not hard to repeat again if needed | 19:01 |
| jeblair | fungi: cool | 19:02 |
| fungi | clarkb: i downgraded with a simple dpkg -i of a manually-downloaded deb | 19:04 |
| fungi | probably still in my homedir | 19:04 |
| clarkb | fungi: thanks, I will upgrade using the same process | 19:05 |
| clarkb | 02 is not running any jobs I am upgrading it now | 19:05 |
| *** rakhmerov has quit IRC | 19:07 | |
| fungi | i am in a car at the moment, or else i would help more. sorry | 19:08 |
| clarkb | np | 19:08 |
| anteaya | fungi: for when you feel like it, can I get my username on review-dev changed from akuno to anteaya? | 19:10 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Move elastic-recheck bot to status.o.o https://review.openstack.org/64574 | 19:12 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Move elastic-recheck bot to status.o.o https://review.openstack.org/64574 | 19:15 |
| clarkb | so far it is fine, but it has copied relatively small files for python-cinderclient jobs | 19:17 |
| *** moted has joined #openstack-infra | 19:18 | |
| clarkb | there are a bunch of tempest jobs in flight those will give us proper information | 19:18 |
| *** senk has quit IRC | 19:18 | |
| *** jcooley_ has joined #openstack-infra | 19:18 | |
| anteaya | fungi: the account id is 4931 | 19:20 |
| *** jcooley_ has quit IRC | 19:21 | |
| zaro | **** review-dev.o.o will be down for testing ***** | 19:25 |
| anteaya | okay | 19:27 |
| clarkb | http://logs.openstack.org/80/63680/6/gate/gate-tempest-dsvm-large-ops/3de5a99/console.html 2.2MB of console.html | 19:27 |
| jeblair | clarkb: it has a bottom! | 19:28 |
| *** senk has joined #openstack-infra | 19:29 | |
| *** CaptTofu has quit IRC | 19:30 | |
| *** CaptTofu has joined #openstack-infra | 19:31 | |
| *** jcooley_ has joined #openstack-infra | 19:32 | |
| fungi | anteaya: i didn't have any luck finding your account in the database, but i'll check again when i'm not in the car | 19:33 |
| *** rongze has joined #openstack-infra | 19:34 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Move elastic-recheck bot to status.o.o https://review.openstack.org/64574 | 19:35 |
| jeblair | okay that ^ looks good in noop mode on status.o.o | 19:36 |
| jeblair | and logstash | 19:38 |
| *** rongze has quit IRC | 19:38 | |
| *** ryanpetrello has joined #openstack-infra | 19:40 | |
| *** rakhmerov has joined #openstack-infra | 19:41 | |
| anteaya | fungi: hmmmm, okay I was able to sign in and post a public key | 19:44 |
| anteaya | I have signed out since review-dev is down for testing now | 19:45 |
| anteaya | odd | 19:45 |
| *** rfolco has quit IRC | 19:47 | |
| fungi | anteaya: oh! on review-dev, not review.o.o. sorry, misread | 19:48 |
| jeblair | anteaya: you might want to wait a bit; zaro will be dropping and recreating the database, so a change may be undone. | 19:49 |
| fungi | anteaya: making changes in the db on review-dev is probably not helpful for now, since the table contents are likely to get restored from backup (probably multiple times) in the coming days | 19:49 |
| fungi | or what jeblair said ;) | 19:49 |
| *** thingee has quit IRC | 19:49 | |
| jeblair | that too :) | 19:49 |
| anteaya | okay | 19:56 |
| anteaya | thanks | 19:56 |
| anteaya | my timing is bad | 19:56 |
| anteaya | I finally have time to focus on my lca talk, I will move in another direction | 19:56 |
| anteaya | so later then | 19:56 |
| clarkb | http://logs.openstack.org/73/62173/3/gate/gate-grenade-dsvm/28e4ba4/ more complete logs | 19:57 |
| *** rfolco has joined #openstack-infra | 19:59 | |
| *** jcooley_ has quit IRC | 20:03 | |
| *** jcooley_ has joined #openstack-infra | 20:03 | |
| *** thingee has joined #openstack-infra | 20:04 | |
| *** ryanpetrello has quit IRC | 20:13 | |
| *** jcooley_ has quit IRC | 20:14 | |
| *** jcooley_ has joined #openstack-infra | 20:14 | |
| *** rakhmerov has quit IRC | 20:14 | |
| *** jcooley_ has quit IRC | 20:19 | |
| *** krotscheck has quit IRC | 20:21 | |
| *** senk has quit IRC | 20:22 | |
| *** thingee has quit IRC | 20:34 | |
| *** rongze has joined #openstack-infra | 20:35 | |
| *** nicedice_ has joined #openstack-infra | 20:35 | |
| *** nicedice has quit IRC | 20:36 | |
| fungi | excellent | 20:37 |
| *** senk has joined #openstack-infra | 20:38 | |
| *** rongze has quit IRC | 20:39 | |
| *** thingee has joined #openstack-infra | 20:41 | |
| *** salv-orlando has joined #openstack-infra | 20:45 | |
| lifeless | mordred: nick did not make a pep. michael foord would be the guy for that, and he JFDI'd it based on our discussions | 20:49 |
| *** reed has quit IRC | 20:53 | |
| *** senk has quit IRC | 20:53 | |
| *** jcooley_ has joined #openstack-infra | 20:56 | |
| *** senk has joined #openstack-infra | 20:56 | |
| *** CaptTofu has quit IRC | 20:57 | |
| *** CaptTofu has joined #openstack-infra | 20:57 | |
| *** jcooley_ has quit IRC | 21:00 | |
| *** senk has quit IRC | 21:06 | |
| *** rakhmerov has joined #openstack-infra | 21:11 | |
| *** markmcclain has joined #openstack-infra | 21:13 | |
| *** banix has joined #openstack-infra | 21:14 | |
| *** rakhmerov has quit IRC | 21:14 | |
| *** banix has quit IRC | 21:21 | |
| *** ryanpetrello has joined #openstack-infra | 21:22 | |
| *** salv-orlando has quit IRC | 21:23 | |
| *** banix has joined #openstack-infra | 21:25 | |
| *** ryanpetrello has quit IRC | 21:32 | |
| *** rongze has joined #openstack-infra | 21:35 | |
| *** rongze has quit IRC | 21:40 | |
| *** praneshp has quit IRC | 21:41 | |
| *** krotscheck has joined #openstack-infra | 21:41 | |
| *** praneshp has joined #openstack-infra | 21:42 | |
| *** CaptTofu has quit IRC | 21:44 | |
| *** CaptTofu has joined #openstack-infra | 21:44 | |
| *** ryanpetrello has joined #openstack-infra | 21:46 | |
| zaro | fungi, clarkb : available to help with review-dev.o.o? | 21:47 |
| clarkb | sort of, eating lunch | 21:47 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/config: Explicitly document requirements for 3rd party testing https://review.openstack.org/63478 | 21:48 |
| *** ryanpetrello has quit IRC | 21:50 | |
| zaro | clarkb: ohh perfect. try logging into review-dev.o.o as an admin and try to view a change. | 21:50 |
| *** jcooley_ has joined #openstack-infra | 21:50 | |
| zaro | clarkb: can you access? | 21:50 |
| clarkb | nope | 21:51 |
| clarkb | The page you requested was not found, or you do not have permission to view this page. | 21:51 |
| *** salv-orlando has joined #openstack-infra | 21:55 | |
| *** salv-orlando has quit IRC | 21:55 | |
| *** jcooley_ has quit IRC | 21:55 | |
| *** salv-orlando has joined #openstack-infra | 21:56 | |
| *** rnirmal has quit IRC | 21:58 | |
| zaro | clarkb: thanks. odd that there's no errors in the log. | 21:59 |
| clarkb | zaro: not in the error log? | 21:59 |
| zaro | clarkb: yep. | 22:00 |
| clarkb | zaro: I wonder if the proxy settings need updating | 22:02 |
| *** che-arne has quit IRC | 22:03 | |
| clarkb | I can't open anything that hits deferredjs | 22:03 |
| clarkb | zaro: check the apache logs? | 22:03 |
| *** krotscheck has quit IRC | 22:03 | |
| *** jcooley_ has joined #openstack-infra | 22:03 | |
| *** krotscheck has joined #openstack-infra | 22:05 | |
| zaro | clarkb: the apache log has latest change date of 12/30 so it's not gonna be there. | 22:09 |
| zaro | clarkb: so here's what i've done. maybe you can help. | 22:09 |
| zaro | clarkb: i think this is the bug that was causing the error during upgrade.. https://code.google.com/p/gerrit/issues/detail?id=2046 | 22:11 |
| zaro | clarkb: i did the truncate table thing and that seemed to make the error go away. | 22:12 |
| *** rakhmerov has joined #openstack-infra | 22:12 | |
| zaro | clarkb: i double check the table that was truncated after upgrade and it was restored so the table still looks good. | 22:13 |
| clarkb | ya if you look under settings group membership shows properly, I think this is related to the reverse proxy | 22:14 |
| zaro | clarkb: when i go to start gerrit post upgrade it fails because gerrit2 no longer has permissions to access the database... http://paste.openstack.org/show/59097/ | 22:14 |
| *** che-arne has joined #openstack-infra | 22:14 | |
| clarkb | zaro: its not using a password for some reason | 22:15 |
| clarkb | zaro: could that be the problem? | 22:15 |
| zaro | clarkb: i circumvent that by giving gerrit2 a new password so now gerrit starts post upgrade. | 22:16 |
| zaro | clarkb: now i'm not sure if the error is because i circumvented the password. | 22:17 |
| *** rakhmerov has quit IRC | 22:17 | |
| zaro | clarkb: i mean the error that users can't view changes. | 22:17 |
| zaro | clarkb: what makes you say that it's not using a password? | 22:18 |
| clarkb | zaro: in your log is says using password no | 22:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Move elastic-recheck bot to status.o.o https://review.openstack.org/64574 | 22:19 |
| zaro | clarkb: i a see, that log was right after the upgrade so that's probably why i was getting the access denied errors. I don't know why it stopped using password after running upgrade script. | 22:20 |
| clarkb | did the config change? | 22:20 |
| zaro | clarkb: no, the original config didn't specify a password. | 22:20 |
| clarkb | zaro: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/gerrit/templates/secure.config.erb#n2 it does | 22:21 |
| clarkb | for the reverse proxy try changing http://git.openstack.org/cgit/openstack-infra/config/tree/modules/gerrit/templates/gerrit.vhost.erb#n58 [P] to [NE,P] | 22:22 |
| jeblair | mordred: the gear installation on logstash.o.o is broken; i assume it's something related to pbr | 22:23 |
| jeblair | err: /Stage[main]/Log_processor/Package[gear]/ensure: change from 0.3.1 to 0.5.0 failed: Could not update: Execution of '/usr/local/bin/pip install -q --upgrade gear' returned 1: Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/gear | 22:23 |
| jeblair | Storing complete log in /root/.pip/pip.log | 22:23 |
| zaro | clarkb: ohh it's in secure.config not gerrit.config that's why i was confused. | 22:24 |
| jeblair | mordred: i ran the get-pip snippet from install_puppet.sh and that seems to have fixed it. | 22:26 |
| jeblair | clarkb, fungi, mordred: elasticrecheck is running on status.o.o now and i've manually removed it from logstash.o.o | 22:28 |
| clarkb | jeblair: roger | 22:29 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add devstack-gate unit test job https://review.openstack.org/64582 | 22:33 |
| *** loq_mac has joined #openstack-infra | 22:35 | |
| *** rongze has joined #openstack-infra | 22:36 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-dev/hacking: Add H904: don't wrap lines using a backslash https://review.openstack.org/64584 | 22:40 |
| *** rongze has quit IRC | 22:41 | |
| mordred | lifeless: great - well, chmouel was suggesting that when people come in and bitch that nose doesnt' work with openstack, it would probably be nice to be able to point them at something | 22:43 |
| *** salv-orlando has quit IRC | 22:43 | |
| mordred | lifeless: specifically, a) something we have written as to why openstack has chosen what it's chosen, but then b) if there is a deficiency in nose, we should probably point to the specific deficiency too | 22:43 |
| mordred | jeblair: yeah - sorry, I really need to get the pip patch finished for o-i/config | 22:44 |
| mordred | jeblair: high on my list of things to finish when I'm actually back and useful | 22:44 |
| lifeless | mordred: I think thats reasonable. It's worth noting that nose is *well known* for not being unittest. | 22:45 |
| lifeless | mordred: expectations that it just works, anywhere, are misplaced :(. | 22:45 |
| *** jcooley_ has quit IRC | 22:45 | |
| *** krotscheck has quit IRC | 22:45 | |
| mordred | lifeless: well, misplaced as they may be, they are also seemingly quite common | 22:45 |
| lifeless | mordred: I agree. | 22:45 |
| mordred | lifeless: did you read the lovely bug referenced in here earlier? :) | 22:45 |
| lifeless | mordred: thus my saying I think it's reasonable :) | 22:45 |
| lifeless | mordred: no | 22:45 |
| lifeless | mordred: what bug? | 22:45 |
| mordred | lifeless: https://bugs.launchpad.net/nova/+bug/1235955 | 22:46 |
| uvirtbot | Launchpad bug 1235955 in nova "testsuite needs upgrading" [Undecided,Incomplete] | 22:46 |
| *** jcooley_ has joined #openstack-infra | 22:46 | |
| lifeless | [there is way too much chatter to read backscroll every day, let alone on holiday^Wpaper-writing days. | 22:46 |
| mordred | yah | 22:46 |
| *** krotscheck has joined #openstack-infra | 22:49 | |
| lifeless | mordred: fun, lol. | 22:49 |
| *** crank has quit IRC | 22:50 | |
| *** jcooley_ has quit IRC | 22:50 | |
| zaro | clarkb: ok. i looked around a little but can't figure out what mean with changing the reverse proxy. can you elaborate? | 22:54 |
| clarkb | zaro: I linked the line earlier, it containts a rewrite rule with a flag of [P] that says reverse proxy | 22:54 |
| clarkb | zaro: I think newer gerrit gets cranky when things are escaped for it and the NE flag needs to be apssed as well | 22:55 |
| zaro | clarkb: i assume you mean i should change it in some file on review-dev. if so what would that file be? | 22:57 |
| *** jaypipes has quit IRC | 22:57 | |
| *** ^d has quit IRC | 22:57 | |
| zaro | zaro: i can't tell from the puppetry. | 22:57 |
| clarkb | zaro: its the review dev vhost file /etc/apache2/sites-enabled/50-something | 22:58 |
| *** ryanpetrello has joined #openstack-infra | 22:58 | |
| zaro | clarkb: ahh, that helps. taking a look. | 22:58 |
| clarkb | zaro: after editing apache will need to be restarted | 22:59 |
| *** adam_g has quit IRC | 23:02 | |
| *** adam_g has joined #openstack-infra | 23:02 | |
| *** adam_g has quit IRC | 23:02 | |
| *** adam_g has joined #openstack-infra | 23:02 | |
| jeblair | clarkb: didn't david have a change for that? | 23:05 |
| fungi | zaro: back now--reading... | 23:06 |
| zaro | clarkb: ok, change made apache restarted. | 23:06 |
| clarkb | jeblair: maybe, but I think that host is running without puppet currently | 23:06 |
| jeblair | https://review.openstack.org/#/c/60893/1/modules/gerrit/templates/gerrit.vhost.erb | 23:06 |
| jeblair | clarkb: the change hasn't merged; i wasn't quite sure about it | 23:07 |
| clarkb | jeblair: it won't work | 23:07 |
| clarkb | jeblair: that change assumes proxypass and not rewriterule | 23:07 |
| clarkb | I will comment on it | 23:07 |
| clarkb | also NoDecode is a newer apache setting iirc | 23:07 |
| clarkb | nevermind | 23:08 |
| *** banix has quit IRC | 23:09 | |
| *** croberts has quit IRC | 23:09 | |
| zaro | clarkb: i still view changes. what would hit deferredjs? | 23:10 |
| zaro | i mean can't view | 23:10 |
| clarkb | zaro: the group listing for example | 23:10 |
| clarkb | zaro: that works now | 23:10 |
| clarkb | but changes don't | 23:11 |
| zaro | well group list is viewable. | 23:11 |
| *** dcramer_ has joined #openstack-infra | 23:12 | |
| *** rakhmerov has joined #openstack-infra | 23:12 | |
| clarkb | jeblair: oh I see if we set the slash encoding to be nodecode then we don't need nocanon/NE? | 23:13 |
| zaro | clarkb: so after upgrade the secure.config no longer has any effect. would you have any idea why that might be the case? | 23:14 |
| clarkb | zaro: maybe gerrit stopped loading that file? | 23:14 |
| clarkb | zaro: https://review-dev.openstack.org/Documentation/config-gerrit.html#_file_code_etc_secure_config_code looks like it should work | 23:15 |
| zaro | well that feature is still available in gerrit-2.8 so it should load. | 23:15 |
| clarkb | zaro: maybe it needs the username too? | 23:15 |
| clarkb | check david's changes maybe he updated for that? | 23:15 |
| *** jcooley_ has joined #openstack-infra | 23:16 | |
| * zaro checks it out. | 23:16 | |
| *** rakhmerov has quit IRC | 23:17 | |
| jeblair | clarkb: ok, so current hypothesis is that we should be able to use "NoDecode" boolean-or "[NE]"; and that should have the desired effect, right? | 23:18 |
| clarkb | jeblair: yes | 23:18 |
| clarkb | [NE] will also ignore # and ? which may or may not be necessary | 23:19 |
| jeblair | clarkb, zaro: I lean toward trying 'NE' first; i like the proximity to the actual proxy config | 23:19 |
| *** senk has joined #openstack-infra | 23:20 | |
| *** jcooley_ has quit IRC | 23:20 | |
| *** dcramer_ has quit IRC | 23:21 | |
| clarkb | jeblair: today isn't a good day to stress test jenkins02 but comparing current thread counts for jenkins01 and jenkins02 looks good | 23:21 |
| jeblair | clarkb: yeah; zuul is _idle_. | 23:21 |
| jeblair | i can't remember the last time i've seen it completely idle. | 23:22 |
| *** ryanpetrello has quit IRC | 23:22 | |
| *** banix has joined #openstack-infra | 23:22 | |
| clarkb | jeblair: it is tempting to upgrade 01 as well, but there doesn't seem to be any harm in waiting either | 23:25 |
| clarkb | jeblair: https://bugs.launchpad.net/openstack-ci/+bug/1265125 will probably interest you | 23:27 |
| zaro | clarkb: from what i can gather it looks like a permissions issue. when i chage secure.config from 600 to 644 it works. | 23:27 |
| uvirtbot | Launchpad bug 1265125 in openstack-ci "Grenade clones all repos in the gate, should use repo cache instead" [Undecided,New] | 23:27 |
| clarkb | zaro: what user is gerrit running as? we don't want that file to be 644 because it contains sensitive info | 23:28 |
| zaro | clarkb: i assume gerrit2 is the account that runs the service? | 23:29 |
| clarkb | zaro: it should be, you can check with ps | 23:29 |
| zaro | clarkb: i just run the service with sudo. | 23:29 |
| clarkb | zaro: you are running it directly or with the init script? | 23:30 |
| clarkb | the init script should set user details appropriately | 23:31 |
| zaro | clarkb: running directly i guess. | 23:32 |
| zaro | i'll try the script. | 23:32 |
| clarkb | zaro: you shouldn't run it direclty. Just run the init script | 23:32 |
| *** dcramer_ has joined #openstack-infra | 23:32 | |
| *** banix has quit IRC | 23:33 | |
| *** blamar has quit IRC | 23:33 | |
| pleia2 | on https://wiki.openstack.org/wiki/GitCommitMessages#Summary_of_GIT_commit_message_structure shouldn't it say first line should be 50 not 72? | 23:35 |
| mordred | pleia2: yes | 23:36 |
| clarkb | and note the period thing is contentious... I would just remove it all together | 23:36 |
| pleia2 | clarkb: hah, there is that | 23:36 |
| pleia2 | I'll just fix the s/72/50 for now | 23:37 |
| *** rongze has joined #openstack-infra | 23:37 | |
| jeblair | clarkb: let's give jenkins a bit more time to burn in before we upgrade 01 | 23:37 |
| jeblair | clarkb: where are git clones happening? | 23:38 |
| *** jcooley_ has joined #openstack-infra | 23:38 | |
| jeblair | clarkb: i only see a single failed "git clone /horizon" | 23:38 |
| clarkb | jeblair: you know after you say that I realize I may have completely misread that log | 23:39 |
| jeblair | clarkb: nb: "git clone" != "git_clone" | 23:39 |
| clarkb | jeblair: it is saying git_clone github/something then deciding it doesn't actually need to clone? | 23:39 |
| jog0 | pleia2: we want the commit message to be under 50 but hard limit is 72 | 23:40 |
| jeblair | clarkb: yeah, git_clone is a function that might clone | 23:40 |
| clarkb | jeblair: arg I will close that bug sorry for the bother | 23:40 |
| pleia2 | jog0: seems to me that the docs should reference the "want" (it's what vim syntax highlights on :)) | 23:40 |
| jog0 | pleia2: true, but then people go out of there way to hit the limit if they are 52 chars | 23:41 |
| jog0 | resulting in funny looking commit messages | 23:41 |
| *** loq_mac has quit IRC | 23:41 | |
| jog0 | that are missing vowels and whatnot | 23:41 |
| jog0 | pleia2: http://docs.openstack.org/developer/hacking/#commit-messages | 23:41 |
| pleia2 | jog0: they're doing it wrong | 23:41 |
| * pleia2 nods | 23:41 | |
| jog0 | First, provide a brief summary of 50 characters or less. Summaries of greater then 72 characters will be rejected by the gate. | 23:41 |
| *** rongze has quit IRC | 23:41 | |
| zaro | clarkb: so ps tells me that gerrit2 is the gerrit process owner. | 23:42 |
| pleia2 | jog0: updated to something similar, thanks | 23:42 |
| mordred | jog0: I never submit with summaries over 50 characters because of vim highlighting | 23:42 |
| pleia2 | plane landing, bbiab (I was in upgraded first class too, I feel like mordred today!) | 23:43 |
| zaro | clarkb: so when i run using the script and set secure.config to 600 gerrit fails to load the file. | 23:43 |
| *** loq_mac has joined #openstack-infra | 23:43 | |
| mordred | pleia2: nice! | 23:43 |
| jog0 | mordred: I submit 52 many times | 23:43 |
| jog0 | but I try not to go an extra word over | 23:44 |
| *** jcooley_ has quit IRC | 23:44 | |
| mordred | jog0: it's amazing how much editor highlighting is helpful | 23:44 |
| jog0 | mordred: yup! | 23:44 |
| jog0 | mordred: have you moved yet? | 23:44 |
| mordred | jog0: in the midst of hte move | 23:44 |
| jog0 | neato | 23:44 |
| mordred | we're in dallas tonight | 23:44 |
| mordred | will be doing a dinner at my friend ben's popup restaurant | 23:45 |
| clarkb | zaro: check what user the process is running as | 23:45 |
| zaro | clarkb: ps says 'gerrit2' user. | 23:45 |
| clarkb | zaro: who owns the file? | 23:45 |
| zaro | clarkb: root. | 23:45 |
| clarkb | oh so that would be a problem, | 23:46 |
| zaro | clarkb: that makes sense right? don't know how it worked previously. | 23:46 |
| clarkb | zaro: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/gerrit/manifests/init.pp#n282 should be gerrit2 | 23:46 |
| clarkb | zaro: no gerrit2 should own the file | 23:46 |
| clarkb | zaro: note the comment for that resource though. It could be something related to that | 23:47 |
| clarkb | maybe they changed the init script? that would be unfortunate | 23:47 |
| zaro | clarkb: yeah, that would make sense. but it didn't when run with gerrit 2.4 | 23:47 |
| zaro | ok, so i think instead of chaning the permission i'll set the file ownship to gerrit2 instead. | 23:49 |
| zaro | s/chanig/changing | 23:49 |
| clarkb | zaro: right, but we should sort out if init is doing something funny now | 23:49 |
| zaro | clarkb: ohh wait. i took a look at the backed up gerrit site folder. the file was set to 'gerrit2' ownership. | 23:52 |
| zaro | clarkb: so i think something happened during the upgrade to change the ownership. | 23:53 |
| clarkb | zaro: yes it should be gerrit2 ownership according to puppet, but if you read the comment on that file resource in puppet is says that init is picky about it | 23:53 |
| zaro | clarkb: let me try the upgrade from scratch again and see. | 23:54 |
| zaro | clarkb: i did run it with sudo so that might be the problem. | 23:54 |
| clarkb | zaro: run it however puppet is running it | 23:54 |
| *** loq_mac has quit IRC | 23:54 | |
| fungi | yeah, that file's definitely been touched today, and it shouldn't be root-owned from what i can see | 23:55 |
| zaro | clarkb: is there a way i can make myself gerrit2? | 23:55 |
| clarkb | zaro: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/gerrit/manifests/init.pp#n485 | 23:55 |
| zaro | clarkb: or run upgrade as gerrit2? | 23:55 |
| fungi | i mean, it *is* root owned, but it's not supposed to be and wasn't previously | 23:55 |
| clarkb | zaro: sudo su - gerrit2 or sudo -i -u gerrit2 | 23:55 |
| clarkb | or sudo -u some command | 23:56 |
| *** senk has quit IRC | 23:57 | |
| *** adalbas has quit IRC | 23:57 | |
| zaro | ok, thanks. that's gonna get me a long ways i think. | 23:58 |
| fungi | zaro: yeah, you can see here that puppet does the upgrade as the gerrit2 user, not root/sudo: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/gerrit/manifests/init.pp#n486 | 23:58 |
| fungi | and my initial upgrade tests were done su'd to gerrit2 before running anything | 23:58 |
| fungi | (so as to be consistent with puppet) | 23:59 |
| zaro | yes, i think that was my problem. i'll try a fresh upgrade just using gerrit2 instead. | 23:59 |
| fungi | zaro: well, note that there may now be files in there owned by root, so gerrit2 will be unable to modify them | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!