Tuesday, 2013-09-10

*** dims has joined #openstack-infra00:02
*** hemna is now known as hemnafk00:04
*** dkehn_ has joined #openstack-infra00:05
*** dkehn has quit IRC00:07
*** matsuhashi has joined #openstack-infra00:09
*** changbl has joined #openstack-infra00:12
*** changbl has quit IRC00:23
hasharfungi: are pip commands using an internal openstack mirror of pip ?00:30
clarkbhashar: it depends where the job is running, but in many cases yet00:30
clarkbs/yet/yes/00:31
hasharJenkins Job Builder test-requires has sphinxcontrib-programoutput as a new addition.  Some job is apparently unable to install that python module00:31
hasharso I guess I need to get the python module added to the mirror00:31
fungii don't think jjb's jobs use the mirror...00:32
hasharhttp://logs.openstack.org/15/45715/3/check/jenkins-job-builder-compare-xml/9cabd2a/console.html  tells me """ Could not find any downloads that satisfy the requirement sphinxcontrib-programoutput """00:32
HenryGCan anyone help me with this dependency error I am getting with devstack? oslo.messaging vs. pbr ... http://paste.openstack.org/show/46399/00:32
*** marun has quit IRC00:32
clarkbhashar: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/select-mirror.sh#n54 should have it fall back on pypi proper00:33
clarkbfungi: JJB does but it should use pypi proper too00:33
fungiaah00:33
clarkbHenryG: your global version of pbr is too old and setup_requires won't upgrade it00:33
clarkbHenryG: if you manually upgrade pbr gobally it should make the error go away00:33
hasharclarkb: or can I add the missing module to the mirror ?00:34
clarkbhashar: no, the change merging should add it to the mirror00:34
clarkbhashar: this is why we allow it to go to pypi, can you link to the job that failed?00:35
hasharhttps://review.openstack.org/#/c/45715/   the check on patchset 300:35
hasharor did you mean console log?  that is http://logs.openstack.org/15/45715/3/check/jenkins-job-builder-compare-xml/9cabd2a/console.html00:36
HenryGclarkb: aha, I thought I had done that. But the upgrade is failing because it can't uninstall the old version.00:36
clarkbhashar: now that that change has merged the new requirement should end up on the mirror once the mirror jobs run00:37
clarkbunless00:37
HenryGclarkb: 'pip uninstall pbr' says "No files were found to uninstall."00:37
clarkbfungi: do we run the job for not the openstack mirror?00:37
clarkbHenryG: you may need to manually remove it, old versions of pip are bad at uninstalling stuff00:37
fungiclarkb: no, that mirror is empty00:37
clarkbfungi: so maybe select mirror needs to stop using that mirror until we can build it properly?00:38
fungiassuming you mean the openstack-infra mirror tree00:38
fungiclarkb: is it causing issues?00:38
clarkboh we fall back on http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/select-mirror.sh#n5300:39
clarkbfungi: see hashar's failed job, I htink pypi is failing us00:39
hasharfungi: http://logs.openstack.org/15/45715/3/check/jenkins-job-builder-compare-xml/9cabd2a/console.html :)00:39
hasharfungi: JJB can't find the module  sphinxcontrib-programoutput  which has been added to tools/test-requires00:39
HenryGclarkb: thanks. It seems that /opt/stack/pbr was the old version. Deleting that dir exposed the new version installed.00:40
clarkbshould install https://pypi.python.org/pypi/sphinxcontrib-programoutput/0.8 but pip isn't able to isntall it00:40
*** nosnos has joined #openstack-infra00:40
clarkbhashar: pip can't find the package on pypi00:40
clarkbhashar: I think JJB itself is fine00:40
*** thomasm has joined #openstack-infra00:40
hasharmaybe it has hit an outdated pypi mirror00:41
fungiis pypi.p.o broken? cdn fail?00:41
clarkbfungi: the log isn't verbose enough00:41
fungi:/00:41
hasharat least I managed to install it manually but I have no idea which mirror I have hit :D00:41
clarkbI think just typicaly pypi derpage00:41
fungitried to recheck?00:42
hasharnop :/00:42
clarkbthe change has already merged00:42
fungioh00:42
clarkbleading more weight to once in a while derpage00:42
hasharI will submit a new change to find out what happens00:42
clarkbhashar: I wouldn't worry about it too much00:42
clarkbhashar: but yes, give another change a shot and see if this is consistent00:43
fungiwell, yeah, probably a fastly cdn cache miss or similar00:43
openstackgerritGiulio Fidente proposed a change to openstack-infra/devstack-gate: add c-bak to MY_ENABLED_SERVICES  https://review.openstack.org/4579700:43
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: can Jenkins / pip install sphinxcontrib-programoutput ??  https://review.openstack.org/4579800:44
fungistill not finding any obvious common thread between the three mis-cloned repos on zuul00:45
hasharsame :( No distributions at all found for sphinxcontrib-programoutput    https://jenkins01.openstack.org/job/jenkins-job-builder-compare-xml/56/console00:45
clarkbhashar: it is weird that none of the other jobs are running into that00:45
hasharmaybe something is misconfigured in jjb00:46
clarkboh, I think I know what the issue is00:47
clarkbthe compare job is not running select-mirror so if the last job that ran select mirror was for an openstack/ project it won't be allowed to talk to the world00:48
*** mriedem has joined #openstack-infra00:48
fungiha. that's awesome00:48
fungithis must be jjb's first requirement not in the openstack/* mirror00:49
clarkbI will push a fix00:49
openstackgerritClark Boylan proposed a change to openstack-infra/config: Select mirror when diffing JJB XML files.  https://review.openstack.org/4580100:54
clarkbfungi: hashar ^ that should do it00:54
*** Ryan_Lane has quit IRC00:57
fungiderp in the commit message? (not that it especially matters)00:57
clarkbfungi: where?00:57
*** zehicle has quit IRC00:58
*** zehicle has joined #openstack-infra00:58
*** thomasm has quit IRC00:59
hasharclarkb: crazy stuff :-]00:59
* mordred did not accomplish as much today as he might have wanted01:00
fungiclarkb: the filenames01:00
clarkbfungi: gah yes01:00
clarkbI will fix because why not01:00
fungisure01:00
clarkbmordred: me too, but at least the review queue is much smaller and review backups have been kickstarted01:00
openstackgerritClark Boylan proposed a change to openstack-infra/config: Select mirror when diffing JJB XML files.  https://review.openstack.org/4580101:01
*** giulivo has joined #openstack-infra01:02
mordredclarkb: yay!01:04
mordredclarkb: my outstanding patches list is less than a browser page!!!01:05
hasharI am off, thanks clarkb!01:07
*** dkehn_ has quit IRC01:07
clarkbhashar: have fun in SF01:08
fungiclarkb: what was the reasoning beyond running it in the job shell fragment instead of the script file? we seem to do far more of the latter (everything except hacking and requirements)01:08
hasharbeers.each do |beer| { throw_at('hashar') }01:08
fungier, s/beyond/behind/01:09
clarkbfungi: because that is what the old jobs did01:09
*** sarob has joined #openstack-infra01:09
clarkbfungi: and each one is different ...01:09
fungiah01:10
*** changbl has joined #openstack-infra01:10
*** dkehn has joined #openstack-infra01:10
*** hashar has quit IRC01:11
fungiright, passes the project name01:15
*** yaguang has joined #openstack-infra01:21
*** ericw has joined #openstack-infra01:32
*** blamar has quit IRC01:32
*** senk has quit IRC01:35
*** dkehn has quit IRC01:38
*** xchu has joined #openstack-infra01:38
*** sarob has quit IRC01:38
*** sarob has joined #openstack-infra01:39
*** fandikurnia02 has quit IRC01:40
*** adalbas has quit IRC01:58
*** vipul has quit IRC02:01
*** vipul has joined #openstack-infra02:02
*** sarob_ has joined #openstack-infra02:02
*** sarob has quit IRC02:02
*** ericw has quit IRC02:04
*** portante|afk is now known as portante02:13
openstackgerritZhongyue Luo proposed a change to openstack-dev/hacking: Add check for deprecated method assert_  https://review.openstack.org/4580702:17
*** sarob has joined #openstack-infra02:19
*** sarob_ has quit IRC02:19
*** zehicle has quit IRC02:20
*** zehicle has joined #openstack-infra02:25
*** senk has joined #openstack-infra02:26
*** sarob has quit IRC02:28
*** sarob has joined #openstack-infra02:29
*** mriedem has quit IRC02:29
*** dims has quit IRC02:31
*** senk has quit IRC02:31
*** sarob has quit IRC02:33
*** wenlock has quit IRC02:35
*** wenlock has joined #openstack-infra02:35
*** pcrews has joined #openstack-infra02:43
*** mestery_ is now known as mestery02:44
*** anteaya has quit IRC02:47
*** melwitt has quit IRC02:50
*** KLevenstein has joined #openstack-infra02:50
*** ghena1986 has joined #openstack-infra02:50
*** ghena1986 has left #openstack-infra02:50
*** zhiyan has joined #openstack-infra02:56
*** zhiyan has left #openstack-infra02:58
*** wenlock has quit IRC03:00
*** dcramer_ has joined #openstack-infra03:12
*** DennyZhang has joined #openstack-infra03:30
*** nati_ueno has quit IRC03:32
*** nati_ueno has joined #openstack-infra03:32
*** nati_ueno has quit IRC03:33
*** nati_ueno has joined #openstack-infra03:33
*** boris-42 has joined #openstack-infra04:03
*** vogxn has joined #openstack-infra04:07
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Merge duplicate bugs in recheckwatch  https://review.openstack.org/4551104:10
*** boris-42 has quit IRC04:10
*** dkehn has joined #openstack-infra04:20
*** nati_uen_ has joined #openstack-infra04:21
*** dkliban has quit IRC04:22
*** nati_uen_ has quit IRC04:22
*** nati_uen_ has joined #openstack-infra04:23
*** nati_ueno has quit IRC04:25
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Remove closed bugs from recheckwatch  https://review.openstack.org/4581304:30
jog0some tweaks to recheckwatch04:30
jog0to make life easier during the stabilization period04:34
*** SergeyLukjanov has joined #openstack-infra04:38
*** dkehn has quit IRC04:47
*** Ryan_Lane has joined #openstack-infra04:48
*** boris-42 has joined #openstack-infra04:50
*** UtahDave has quit IRC04:52
*** HenryG has quit IRC04:53
openstackgerritAlex Gaynor proposed a change to openstack-infra/config: Run the troveclient tests under Pypy  https://review.openstack.org/4581404:56
*** dguitarbite has joined #openstack-infra05:04
*** sjing has joined #openstack-infra05:05
*** dkehn has joined #openstack-infra05:13
*** pblaho has joined #openstack-infra05:27
*** nicedice has quit IRC05:33
*** nosnos has quit IRC05:35
*** nosnos has joined #openstack-infra05:36
*** SergeyLukjanov has quit IRC05:55
*** SergeyLukjanov has joined #openstack-infra06:04
*** notmyname has quit IRC06:05
*** Shrews has quit IRC06:06
*** oubiwann has quit IRC06:06
*** Shrews has joined #openstack-infra06:06
*** Guest66487 has quit IRC06:06
*** annegentle_ has quit IRC06:06
*** retr0h has quit IRC06:07
*** ttx has quit IRC06:07
*** notmyname has joined #openstack-infra06:08
*** retr0h has joined #openstack-infra06:08
*** retr0h has joined #openstack-infra06:08
*** mordred has quit IRC06:08
*** stevebaker has quit IRC06:09
*** annegentle has joined #openstack-infra06:09
*** SergeyLukjanov has quit IRC06:10
*** comstud has quit IRC06:10
*** DennyZhang has quit IRC06:11
*** comstud has joined #openstack-infra06:11
*** jlk has quit IRC06:12
*** russellb has quit IRC06:12
*** stevebaker has joined #openstack-infra06:13
*** xchu has quit IRC06:13
*** jlk has joined #openstack-infra06:13
*** jlk has joined #openstack-infra06:13
*** vishy has quit IRC06:13
*** vishy has joined #openstack-infra06:14
*** lillie has joined #openstack-infra06:15
*** lillie is now known as Guest1168506:16
*** vogxn has quit IRC06:16
*** nijaba has joined #openstack-infra06:20
*** russellb has joined #openstack-infra06:21
*** jhesketh has quit IRC06:22
*** xchu has joined #openstack-infra06:26
*** vogxn has joined #openstack-infra06:29
*** yaguang has quit IRC06:29
*** odyssey4me has joined #openstack-infra06:30
*** oubiwann has joined #openstack-infra06:42
*** afazekas_zz is now known as afazekas06:51
*** yaguang has joined #openstack-infra06:53
*** ttx has joined #openstack-infra07:09
*** ttx has quit IRC07:09
*** ttx has joined #openstack-infra07:09
*** jhesketh_ has quit IRC07:11
*** flaper87|afk is now known as flaper8707:14
*** vogxn has quit IRC07:21
*** shardy_afk is now known as shardy07:22
*** afazekas has quit IRC07:24
*** fbo_away is now known as fbo07:33
*** alexpilotti has joined #openstack-infra07:39
*** Bada has joined #openstack-infra07:41
*** Ryan_Lane has quit IRC07:44
*** nosnos has quit IRC07:52
*** xchu has quit IRC07:52
*** tstevenson has quit IRC07:55
*** vogxn has joined #openstack-infra07:55
*** amotoki has joined #openstack-infra08:00
*** afazekas has joined #openstack-infra08:01
*** jpich has joined #openstack-infra08:02
saschpe_zul: hi, any update on python-lesscpy packaging?08:02
*** mordred has joined #openstack-infra08:05
*** xchu has joined #openstack-infra08:06
*** KLevenstein has quit IRC08:08
ttxjog0: for testing launchpad bugs you can use http://staging.launchpad.net/08:11
ttxjog0: can be a bit slow but at least it does not pollute the main db08:12
ttx(context: https://bugs.launchpad.net/openstack-ci/+bug/1223159)08:12
uvirtbotLaunchpad bug 1223159 in openstack-ci "fake bug 2" [Undecided,New]08:12
*** Ryan_Lane has joined #openstack-infra08:14
*** Ryan_Lane has quit IRC08:21
*** derekh has joined #openstack-infra08:33
*** SergeyLukjanov has joined #openstack-infra08:37
*** sjing has quit IRC08:47
*** odyssey4me has quit IRC08:51
*** mgagne1 has joined #openstack-infra09:02
*** mgagne1 has joined #openstack-infra09:02
*** afazekas_ has joined #openstack-infra09:07
*** jhesketh_ has joined #openstack-infra09:11
*** afazekas has quit IRC09:11
*** amotoki has quit IRC09:11
*** pabelanger has quit IRC09:11
*** mgagne has quit IRC09:11
*** jpeeler has quit IRC09:11
*** mtreinish has quit IRC09:11
*** markmcclain has quit IRC09:11
*** comstud has quit IRC09:11
*** dcramer_ has quit IRC09:11
*** dragondm has quit IRC09:11
*** SpamapS has quit IRC09:11
*** roaet has quit IRC09:11
*** saper has quit IRC09:11
*** carlp has quit IRC09:11
*** EntropyWorks has quit IRC09:11
*** jesusaurus has quit IRC09:11
*** danger_fo_away has quit IRC09:11
*** rainya has quit IRC09:11
*** derekh has quit IRC09:11
*** nijaba has quit IRC09:11
*** pblaho has quit IRC09:11
*** zehicle has quit IRC09:11
*** soren has quit IRC09:11
*** david-lyle has quit IRC09:11
*** ianw has quit IRC09:11
*** LinuxJedi has quit IRC09:11
*** locke105 has quit IRC09:11
*** koolhead17 has quit IRC09:11
*** pleia2 has quit IRC09:11
*** matsuhashi has quit IRC09:11
*** jcooley has quit IRC09:11
*** jeremyb has quit IRC09:11
*** rockstar has quit IRC09:11
*** zaro has quit IRC09:11
*** chmouel has quit IRC09:11
*** yaguang has quit IRC09:11
*** nati_uen_ has quit IRC09:11
*** pcrews has quit IRC09:11
*** mestery has quit IRC09:11
*** salv-orlando has quit IRC09:11
*** sdague has quit IRC09:11
*** saschpe_ has quit IRC09:11
*** niska has quit IRC09:11
*** briancline has quit IRC09:11
*** samalba has quit IRC09:11
*** paul-- has quit IRC09:11
*** uvirtbot has quit IRC09:15
*** uvirtbot has joined #openstack-infra09:15
*** afazekas_ is now known as afazekas09:15
*** jhesketh__ has joined #openstack-infra09:16
*** pabelanger_ has joined #openstack-infra09:17
*** yaguang has joined #openstack-infra09:17
*** nati_uen_ has joined #openstack-infra09:17
*** pcrews has joined #openstack-infra09:17
*** mestery has joined #openstack-infra09:17
*** salv-orlando has joined #openstack-infra09:17
*** sdague has joined #openstack-infra09:17
*** saschpe_ has joined #openstack-infra09:17
*** niska has joined #openstack-infra09:17
*** briancline has joined #openstack-infra09:17
*** samalba has joined #openstack-infra09:17
*** Ryan_Lane has joined #openstack-infra09:19
*** jpeeler has joined #openstack-infra09:20
*** mtreinish has joined #openstack-infra09:20
*** derekh has joined #openstack-infra09:20
*** nijaba has joined #openstack-infra09:20
*** comstud has joined #openstack-infra09:20
*** pblaho has joined #openstack-infra09:20
*** dcramer_ has joined #openstack-infra09:20
*** zehicle has joined #openstack-infra09:20
*** matsuhashi has joined #openstack-infra09:20
*** rainya has joined #openstack-infra09:20
*** danger_fo_away has joined #openstack-infra09:20
*** jesusaurus has joined #openstack-infra09:20
*** EntropyWorks has joined #openstack-infra09:20
*** carlp has joined #openstack-infra09:20
*** saper has joined #openstack-infra09:20
*** roaet has joined #openstack-infra09:20
*** SpamapS has joined #openstack-infra09:20
*** dragondm has joined #openstack-infra09:20
*** pleia2 has joined #openstack-infra09:20
*** koolhead17 has joined #openstack-infra09:20
*** locke105 has joined #openstack-infra09:20
*** LinuxJedi has joined #openstack-infra09:20
*** ianw has joined #openstack-infra09:20
*** david-lyle has joined #openstack-infra09:20
*** soren has joined #openstack-infra09:20
*** chmouel has joined #openstack-infra09:20
*** zaro has joined #openstack-infra09:20
*** rockstar has joined #openstack-infra09:20
*** jeremyb has joined #openstack-infra09:20
*** jcooley has joined #openstack-infra09:20
*** paul-- has joined #openstack-infra09:20
*** amotoki has joined #openstack-infra09:20
openstackgerritThierry Carrez proposed a change to openstack-infra/odsreg: Expose Comments in admin site  https://review.openstack.org/4583209:23
openstackgerritA change was merged to openstack-infra/odsreg: Expose Comments in admin site  https://review.openstack.org/4583209:24
*** mkerrin has joined #openstack-infra09:24
*** Ryan_Lane has quit IRC09:24
*** vogxn has quit IRC09:29
*** matsuhashi has quit IRC09:34
*** matsuhashi has joined #openstack-infra09:36
*** SergeyLukjanov has quit IRC09:45
*** vogxn has joined #openstack-infra09:45
*** Ryan_Lane has joined #openstack-infra09:49
*** dguitarbite has quit IRC09:50
*** Ryan_Lane has quit IRC09:54
*** lifeless has quit IRC09:56
*** xchu has quit IRC09:56
*** pcm_ has joined #openstack-infra09:59
*** pcm_ has quit IRC10:01
*** afazekas is now known as afazekas_food10:01
*** pcm_ has joined #openstack-infra10:01
*** matsuhashi has quit IRC10:03
*** vogxn has quit IRC10:12
*** vogxn has joined #openstack-infra10:14
*** Ryan_Lane has joined #openstack-infra10:20
*** Ryan_Lane has quit IRC10:25
*** salv-orlando has quit IRC10:30
*** dizquierdo has joined #openstack-infra10:43
*** yaguang has quit IRC10:46
*** rfolco has quit IRC10:48
*** SergeyLukjanov has joined #openstack-infra10:48
*** Ryan_Lane has joined #openstack-infra10:50
*** vogxn has quit IRC10:54
*** Ryan_Lane has quit IRC10:55
*** ArxCruz has quit IRC10:56
*** vogxn has joined #openstack-infra10:58
*** lcestari has joined #openstack-infra11:03
openstackgerritZhongyue Luo proposed a change to openstack-dev/hacking: Add check for equality operator use for singleton comparsion  https://review.openstack.org/4582411:12
*** DennyZhang has joined #openstack-infra11:12
*** lifeless has joined #openstack-infra11:15
*** pblaho has quit IRC11:16
*** adalbas has joined #openstack-infra11:20
*** tian has quit IRC11:21
*** vogxn has quit IRC11:29
*** pblaho has joined #openstack-infra11:35
*** ArxCruz has joined #openstack-infra11:44
*** weshay has joined #openstack-infra11:45
*** DennyZhang has quit IRC11:46
*** shadower has quit IRC11:47
*** afazekas has joined #openstack-infra11:48
*** markmcclain has joined #openstack-infra11:52
*** rfolco has joined #openstack-infra11:58
*** yaguang has joined #openstack-infra12:00
*** sandywalsh has joined #openstack-infra12:13
*** dcramer_ has quit IRC12:15
*** zeus has joined #openstack-infra12:15
*** thomasm has joined #openstack-infra12:23
*** sridevi has joined #openstack-infra12:25
*** kaushikc has joined #openstack-infra12:26
srideviHi, could someone help me with https://review.openstack.org/#/c/45822/12:26
sridevifailure12:26
*** dcramer_ has joined #openstack-infra12:27
openstackgerritJulien Danjou proposed a change to openstack-infra/config: Run Tempest tests against Ceilometer w/ MongoDB  https://review.openstack.org/4586212:33
*** salv-orlando has joined #openstack-infra12:34
*** kaushikc has quit IRC12:37
openstackgerritJulien Danjou proposed a change to openstack-infra/config: Run Tempest tests against Ceilometer w/ MongoDB  https://review.openstack.org/4586212:37
*** sridevi has quit IRC12:38
*** markmcclain has quit IRC12:39
*** weshay has quit IRC12:40
*** yaguang has quit IRC12:55
*** dprince has joined #openstack-infra12:55
*** weshay has joined #openstack-infra12:55
*** anteaya has joined #openstack-infra13:03
*** ericw has joined #openstack-infra13:04
*** whayutin_ has joined #openstack-infra13:06
zulsaschpe_:  should be this week13:07
saschpe_zul: that'd be great, thanks13:07
*** KLevenstein has joined #openstack-infra13:08
*** weshay has quit IRC13:08
*** dims has joined #openstack-infra13:10
*** mriedem has joined #openstack-infra13:13
*** boris-42 has quit IRC13:18
*** boris-42 has joined #openstack-infra13:19
*** dkliban has joined #openstack-infra13:22
*** rfolco has quit IRC13:23
*** yaguang has joined #openstack-infra13:37
*** burt has joined #openstack-infra13:43
*** rfolco has joined #openstack-infra13:45
*** cppcabrera has joined #openstack-infra13:46
*** thedodd has joined #openstack-infra13:47
*** prad has joined #openstack-infra13:49
*** vogxn has joined #openstack-infra13:51
*** dkehn has quit IRC14:01
*** dkehn has joined #openstack-infra14:01
*** tstevenson has joined #openstack-infra14:02
*** vogxn has quit IRC14:02
*** zeus has quit IRC14:02
*** nati_uen_ has quit IRC14:05
*** erfanian has quit IRC14:08
*** mrodden has joined #openstack-infra14:11
*** rnirmal has joined #openstack-infra14:12
*** erfanian has joined #openstack-infra14:15
*** markmcclain has joined #openstack-infra14:17
sdaguewhat would people think about a new recheck tag for freshness of a patch?14:24
sdaguegiven our rate of change, I find that after a week nothing is mergable, so before reviewing code I like to actually make sure the patch is going to still merge in jenkins before doing so14:24
sdagueI've been doing a lot of recheck no bug on old patches, but that seems a little overuse of that tag14:25
*** kaushikc has joined #openstack-infra14:26
*** dcramer_ has quit IRC14:30
anteayasdague: what would the new recheck tag be called?14:30
*** jhesketh_ has quit IRC14:33
*** vogxn has joined #openstack-infra14:33
*** kaushikc has quit IRC14:35
fungii'm not seeing how that's overuse of "recheck no bug"14:37
fungiyou're rechecking, and there's no bug... unless the goal is to make it clear to other reviewers why you're rechecking it14:38
fungii agree leaving a second comment explaining the non-bug reason for the recheck is extra makework. how about extending the no bug syntax to allow an optional comment after? something like "recheck no bug: just want to see if this is still working"14:39
*** marun has joined #openstack-infra14:39
fungiand not track the comments or anything, just match the no bug pattern on "recheck no bug(:.+)?"14:40
fungior rather, "^recheck no bug(:.+)?$"14:41
*** ttx has quit IRC14:42
*** oubiwann has quit IRC14:43
*** dcramer_ has joined #openstack-infra14:43
jeblairthat's one of the actual reasons for recheck no bug.  also, fun zuul project: https://bugs.launchpad.net/zuul/+bug/107926814:45
uvirtbotLaunchpad bug 1079268 in zuul "periodic recheck of reviews" [Wishlist,Triaged]14:45
*** mriedem has quit IRC14:45
sdaguefungi: yeh, optional comment would be nice14:47
*** mriedem has joined #openstack-infra14:48
jeblairpart of the reason for keeping it simple (and not allowing, say, a full sentence) was to avoid accidental triggers.  if you extend the syntax (which is already confusing enough for people), please keep that in mind.14:49
*** oubiwann has joined #openstack-infra14:49
openstackgerritAlex Gaynor proposed a change to openstack-infra/config: Run the troveclient tests under PyPy  https://review.openstack.org/4581414:50
jeblairBobBall: can you attend the infra team meeting today (at 1900 utc)?14:50
*** dkranz has joined #openstack-infra14:51
*** UtahDave has joined #openstack-infra14:52
anteayajeblair: do you want a salt category added to today's meeeting agenda?14:55
*** KLevenstein has quit IRC14:55
jeblairanteaya: feel free14:56
anteayaokay thanks14:56
*** nati_ueno has joined #openstack-infra14:58
*** dkranz has quit IRC14:58
*** dcramer_ has quit IRC15:01
*** pycabrera has joined #openstack-infra15:02
anteayaso is today bug day for -infra?15:05
anteayaif so, I'll do what I can to help, if not I will keep working on puppet-dashboard15:06
*** cppcabrera has quit IRC15:06
jeblairanteaya: yes, at 1700 utc15:06
anteayaah okay15:07
*** pycabrera is now known as cppcabrera15:07
*** mrodden has quit IRC15:09
*** ttx has joined #openstack-infra15:10
*** ttx has quit IRC15:10
*** ttx has joined #openstack-infra15:10
*** vogxn has quit IRC15:10
*** yaguang has quit IRC15:14
*** nati_ueno has quit IRC15:14
*** dkranz has joined #openstack-infra15:15
*** pblaho has quit IRC15:16
fungii definitely want to go over the salt stuff too--can we try to get to it earlyish in the agenda? i'm probably going to have to bail at about 1945z so i can get to red hat and find parking (our monthly local opestack meetup landed on a tuesday this time)15:17
*** mrodden has joined #openstack-infra15:20
*** marun has quit IRC15:20
*** mrmartin has joined #openstack-infra15:22
*** marun has joined #openstack-infra15:25
*** sridevi has joined #openstack-infra15:29
*** Bada has quit IRC15:30
*** nati_ueno has joined #openstack-infra15:31
*** nati_ueno has quit IRC15:31
*** vogxn has joined #openstack-infra15:32
*** dkehn has quit IRC15:34
*** dcramer_ has joined #openstack-infra15:37
*** KLevenstein has joined #openstack-infra15:39
jd__fungi: where should I look to see that the precise-node are using Ubuntu Cloud Archive?15:39
*** KLevenstein has quit IRC15:40
fungijd__: i'd git grep in openstack-infra/config for the url, as a start15:40
*** KLevenstein has joined #openstack-infra15:40
jd__fungi: tried that already :)15:40
jd__I didn't find anything about sources.list handling actually15:41
*** jhesketh__ has quit IRC15:41
fungijd__: i just checked directly on one of the precise slaves with 'grep -ir cloud /etc/apt' and found nothing15:42
*** pabelanger_ has quit IRC15:42
jd__fungi: damn, thanks, I thought it was already in use for whatever reason15:42
*** pabelanger has joined #openstack-infra15:42
*** dprince has quit IRC15:43
fungiwas pretty sure that never got implemented because originally we were talking about moving jobs to quantal, which didn't have cloud archive support, but should resurrect the cloud archive work now that we decided to stick with precise until the next lts15:43
jd__makes sense15:43
fungiespecially now that we have python 3.3 and pypy solutions on precise15:44
jeblairit shouldn't be too hard to do, i think we already have some 'apt::repo' usage puppet15:44
* fungi agrees15:44
*** rfolco has quit IRC15:45
jd__I'm looking into it15:45
*** sridevi has quit IRC15:50
openstackgerritJulien Danjou proposed a change to openstack-infra/config: Use Ubuntu Cloud Archive on precise nodes  https://review.openstack.org/4589615:53
openstackgerritJulien Danjou proposed a change to openstack-infra/config: Run Tempest tests against Ceilometer w/ MongoDB  https://review.openstack.org/4586215:53
jd__my try at it :)15:54
anteayafungi: I had put salt at the end of the agenda just before open discussion since I thought it was etiquette to append new items to the end15:55
jeblairi'm given to understand the chair has a habit of reordering items as warranted.15:55
fungii have in fact seen the chair do that15:56
anteayago chair15:56
zaromorning15:56
anteayamorning zaro15:56
*** nati_ueno has joined #openstack-infra15:57
anteayacan puppet-dashboard connect to a puppet client or does it have to connect to a puppet master?16:02
* mordred reorders the chair16:02
*** reed has joined #openstack-infra16:03
*** hashar has joined #openstack-infra16:04
*** gyee has joined #openstack-infra16:04
anteayamorning mordred16:04
openstackgerritA change was merged to openstack-infra/nodepool: Add image-update command  https://review.openstack.org/4492016:04
openstackgerritA change was merged to openstack-infra/nodepool: Add alien-list command  https://review.openstack.org/4544816:05
*** dkehn has joined #openstack-infra16:05
*** thedodd has quit IRC16:05
openstackgerritA change was merged to openstack-infra/config: Update nodepool init script to nodepoold  https://review.openstack.org/4455716:06
openstackgerritDavid Boucha proposed a change to openstack-infra/config: Give Jenkins the ability to trigger puppet runs  https://review.openstack.org/4589816:06
openstackgerritA change was merged to openstack-infra/nodepool: Add alien-image-list command  https://review.openstack.org/4544916:07
*** vogxn has quit IRC16:07
openstackgerritA change was merged to openstack-infra/nodepool: Add a hold command  https://review.openstack.org/4545316:07
openstackgerritA change was merged to openstack-infra/nodepool: Add a delete command  https://review.openstack.org/4545416:07
openstackgerritA change was merged to openstack-infra/nodepool: Add image-delete command  https://review.openstack.org/4545516:07
pleia2anteaya: I16:07
anteayamorning pleia216:07
pleia2anteaya: I'll take a closer look, but I believe each node reports in to the dashboard (not mater)16:07
anteayaokay the documentation I am reading leads me to believe that dashboard fetches the info, so if the client nodes report it, I would like to understand that16:08
zaropretty busy meeting agenda today.16:08
pleia2bug day in an hour too :)16:08
* pleia2 prepping etherpad16:09
pleia2anteaya: oh ok, maybe it's the other way around then :)16:09
zaropleia2: is there an alert system for bug day?  i can never seem to remember the schedule.16:09
pleia2dashboard goes to nodes16:09
anteayaokay16:09
*** pycabrera has joined #openstack-infra16:09
openstackgerritA change was merged to openstack-infra/config: Select mirror when diffing JJB XML files.  https://review.openstack.org/4580116:09
pleia2zaro: in the weekly meeting announcement16:09
pleia2I also mentioned it at the meeting last week ;)16:10
zaropleia2: thx.  i'll remember to read it going forward.  i thought it was just boiler plate reminder.16:10
*** markmcclain has quit IRC16:10
pleia2yeah, I probably should have modified the subject too16:10
*** vogxn has joined #openstack-infra16:10
zaropleia2: missed last meeting.16:10
zaropleia2: imo, would be better as a seperate email.16:11
*** pycabrera is now known as cppcabrera_afk16:12
*** cppcabrera has quit IRC16:12
*** dkehn has quit IRC16:12
pleia2zaro: ok :)16:12
*** nati_uen_ has joined #openstack-infra16:13
*** SergeyLukjanov has quit IRC16:15
*** dkranz has quit IRC16:15
*** nati_ueno has quit IRC16:16
*** openstackstatus has quit IRC16:18
anteayait got buried in the avalanche but here is UtahDave's inital offering for a salt patch: https://review.openstack.org/#/c/45898/116:18
mordredI was just looking at that16:19
mordredit seems so simple16:19
anteayayay16:19
*** rev has joined #openstack-infra16:19
anteayawhen you know what button to push, yeah16:19
fungipinch of salt16:21
UtahDavefungi: I didn't include the change to enable the salt-master and salt-minion services16:22
fungiUtahDave: cool. i can manually test this first once i'm done reading through it16:22
UtahDavefungi: that would be great!  Let me know if you have any questions or if you think something could be done in a better way.16:23
fungiUtahDave: reenabling should be discussed on #openstack-meeting between 19:00-20:00 utc, to confirm we're comfortable that the runaway memory utilization and process build-up is ironed out in a later release than we were running16:23
*** dachary has joined #openstack-infra16:24
fungii went digging in the release notes a little but didn't spot right off which version fixes it16:24
*** dkehn has joined #openstack-infra16:25
jeblairmordred: do you have a few minutes to bring django-openstack-auth up to standards? https://review.openstack.org/#/c/45655/16:28
*** cppcabrera has joined #openstack-infra16:28
openstackgerritA change was merged to openstack-infra/config: link to jjb cache documentation in the handbook .  https://review.openstack.org/4573816:28
*** dkranz has joined #openstack-infra16:28
*** cppcabrera_afk has quit IRC16:28
jeblairmordred: (those are non-voting; it's not urgent; just something we should do at some point, and i know it's your favorite thing to make projects conform)16:29
*** cppcabrera is now known as cppcabrera_afk16:29
*** vogxn has quit IRC16:31
openstackgerritA change was merged to openstack-infra/config: link git sha in commit comments to the gerrit review  https://review.openstack.org/4546416:35
*** nhm has joined #openstack-infra16:36
jog0how can I make hacking-pypy non-gating?16:37
clarkbmorning16:37
jog0as it doesn't work16:37
jog0morning clarkb16:37
clarkbjog0: in the layout.yaml file make it non voting under the jobs sections near the top of the file16:37
anteayamorning clarkb16:37
clarkbjog0: should be a lot of examples of that in that section of the file16:37
jeblairjog0: though we should ask Alex_Gaynor about that16:37
clarkbjeblair: jog0 yes he said it was working16:38
jog0https://review.openstack.org/#/c/45761/16:38
Alex_Gaynorhacking tests totally passed for me under PyPy, can you link a failign run?16:38
jeblair(we wouldn't have merged it if he hadn't tested it)16:38
mordredjeblair: I'll take a look16:38
jog0in general I don't like going from no test to gating without a non-voting phase to confirm16:38
*** nicedice has joined #openstack-infra16:38
Alex_Gaynor(looking)16:39
*** yolanda has joined #openstack-infra16:40
*** fbo is now known as fbo_away16:41
yolandahi, i have a question about zuul/jenkins communication. I'm finding that zuul tries to access jenkins as anonymous user, that can be a problem with security when defining roles. Is there any way to define some user/password to access jenkins with a named user?16:41
jeblairyolanda: what version of zuul?16:41
*** SergeyLukjanov has joined #openstack-infra16:43
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Make gate-hacking-pypy non-gating  https://review.openstack.org/4590516:43
jog0Alex_Gaynor clarkb ^16:43
jog0I propose we make this non-gating for now until we can resolve it16:43
Alex_Gaynorjog0: so pypy totally passes locally for me :/16:43
yolandajeblair, i'm just grabbing the last one from https://github.com/openstack-infra/zuul.git16:44
jog0even if resolve means it works in 2 hours16:44
jog0Alex_Gaynor: I hate when that happens16:44
jeblairyolanda: so you're using the gearman plugin?16:44
Alex_Gaynorjog0: maybe try a recheck no bug?16:44
yolandajeblair, yes16:44
jog0Alex_Gaynor: it failed 2x in a row16:44
jog0so don't think that will help16:44
*** nati_uen_ has quit IRC16:44
Alex_Gaynorjog0: grumble, sounds like a thing then :/16:45
clarkbjog0: lgtm16:45
jog0Alex_Gaynor: I am running on https://review.openstack.org/#/c/45824/ just to confirm its not my patch somehow16:45
jeblairyolanda: we use the project-based matrix authorization strategy, and they only perms granted to anonymous are "overall-read" and "job-read"16:46
jeblairs/they/the/16:46
openstackgerritA change was merged to openstack-infra/config: Make gate-hacking-pypy non-gating  https://review.openstack.org/4590516:47
yolandajeblair, that's enough to be able to trigger jobs and retrieve results?16:47
jog0Alex_Gaynor: did you try running the tests locally for https://review.openstack.org/#/c/45761/116:47
Alex_Gaynorjog0: no, let me do that16:47
jeblairyolanda: i believe so because the gerrit-plugin bypasses the authorization system16:47
jeblairer16:47
jeblairgearman-plugin16:47
clarkbgearman-plugin escalates itself iirc16:47
anteayawow, awesome work on the etherpad round-up, pleia216:47
clarkboh yes bug day16:48
Alex_Gaynorjog0: hmm, results in a "VersionConflict: (pyflakes 0.7.3 (/Users/alex_gaynor/.virtualenvs/23e6d2eafccc0dc9/hacking/.tox/pypy/site-packages), Requirement.parse('pyflakes==0.7.2'))" which doesn't really seem relevant16:48
* clarkb finds link to etherpad16:48
yolandajeblair, i was having some issues with "Connect" permissions denied for anonymous, but i had anonymous access totally locked16:48
anteayahttps://etherpad.openstack.org/cibugreview-september201316:48
yolandai don't want anonymous to be able to read, but if there is no way to specify that, we will try to deal with it16:48
pleia2anteaya: I'm not done yet, it's not 17:00! :)16:49
jeblairyolanda: i don't know if that is required for gearman-plugin -- we have it set because that's what we want (it's public).16:49
clarkbpleia2: I won't look at it yet then :)16:49
jeblairyolanda: can you pastebin the error message you get?16:49
anteayadon't let me distract, pleia216:49
Alex_Gaynorjog0: so tests pass on that other one you rechecked, I wonder if it's somehow your patch!16:49
jog0Alex_Gaynor:  :(16:49
Alex_Gaynorjog0: for the life of me I can't see how though16:49
*** dkehn_ has joined #openstack-infra16:49
jog0Alex_Gaynor: you get it to fail locally16:50
*** dkehn has quit IRC16:50
hasharclarkb: hi :)  Seems like the JJB xml comparison job is working again. I have abandoned my test change 45798 and left a message on the puppet change 45801.   Thank you very much!16:50
pleia2clarkb: everyone can start working on it, I'm about halfway through Mediums16:50
yolandajeblair: INFO: While serving http://xxxx/computer/ciaas-jenkins-slave-0/slave-agent.jnlp: hudson.security.AccessDeniedException2: anonymous is missing the Connect permission16:50
yolandaand also zuul was reporting that the job didn't exist16:50
Alex_Gaynorjog0: yup, it totally reproduces locally on your patch16:51
yolandaremoving security worked16:51
jog0Alex_Gaynor: :( can you do a git bisect o nit16:51
jog0on it16:51
Alex_Gaynorjog0: how do you mean?16:51
jog0good news is its reproducable16:51
jeblairyolanda: i'm not a jenkins expert, but that kind of looks like a problem launching the slave, not a gearman-plugin problem;16:52
jog0find a patch in that branch where things work, then use git-bisect16:52
*** hashar has quit IRC16:52
jeblairyolanda: it looks like you're using jnlp to launch the slave, which requires the slave make a connection to jenkins, which is denied16:52
clarkbjeblair: yolanda: that is what it looks like to me. slave-agent.jnlp is one way slaves talk to the master16:52
yolandajeblair, not sure if this log was related anyway, but the zuul one was16:52
jeblairyolanda: (it will affect gearman-plugin because without the slave online, the job won't be registered)16:52
Alex_Gaynorjog0: can you try rebasing that patch on master? if I do that it works ok16:52
clarkbyolanda: you can try running a job on the master with zuul and see if that suffers the same problem?16:53
jog0Alex_Gaynor: that is already in master16:53
jeblairyolanda: (but there's nothing gearman-plugin can do about it)16:53
Alex_Gaynorjog0: no, I mean on https://review.openstack.org/#/c/4576116:53
yolandaclarkb, i can't try it at the moment, sorry16:53
jog0Alex_Gaynor: that is cherry-pick to a stable branch16:54
jog0so can't rebase on master16:54
clarkboh maybe stable branch doesn't work? we can make pypy run only on master and new stable branches16:54
*** nati_ueno has joined #openstack-infra16:54
jog0try running pyp on the 0.7.x branch16:54
yolandajeblair, so gearman-plugin is bypassing all security rules, but if slave-master relation fails by some security issues, jobs won't be registered, right?16:54
jog0pypy16:54
mordredwhat clarkb said16:54
yolandajust trying to get some clues16:54
jeblairyolanda: yes16:54
*** UtahDave has quit IRC16:55
yolandathx16:55
jog0clarkb: not sure why that would be the case16:55
jog0this is a new stable branch16:55
jog0as of yesterday16:55
mordredjog0: we should really make that branch stable/0.7.x next time we get a chance16:55
mordredor maybe we shouldn't16:55
mordredhrm16:55
Alex_Gaynorjog0: ah ah ah, this patch is against stable/0.7.x, not master16:55
mordreddon't pay attention to me - I'm still drinking coffee16:55
jog0mordred: I forgot why we didn't use stable in it but there was a reason16:55
clarkbmordred: fungi https://review.openstack.org/#/c/45788/ getting that in soonish would be awesome16:56
jog0sdague and I chatted about it16:56
mordredbecause that means it becomes stable-maint16:56
*** UtahDave has joined #openstack-infra16:56
jog0here stable just means no new tests16:56
mordredclarkb: done16:56
clarkbmordred: danke16:57
sdagueyeh, that's why, the core teams get all weird if we made it stable/16:57
fungiclarkb: we double-tapped it apparently ;)16:57
*** Ryan_Lane has joined #openstack-infra16:57
clarkbsdague: jog0: we can override that behavior in your acl with exclusive access if you wish16:57
*** derekh has quit IRC16:57
sdaguewe could, it's not really a big deal though16:57
openstackgerritA change was merged to openstack-infra/config: Use bup to backup review-dev to an offsite server  https://review.openstack.org/4578816:58
openstackgerritTom Fifield proposed a change to openstack-infra/jeepyb: Update doc impact to only make bugs on merge  https://review.openstack.org/4590716:58
jog0Alex_Gaynor: well for now pypy is non-gating and when we sort this out (today I hope) we can make it gating again16:58
*** fifieldt has joined #openstack-infra16:58
jog0I am thinking maybe its a funny deps issue16:58
sdagueman, today is so full of meetings.... I can't wait to get back to hacking on things tomorrow16:58
Alex_Gaynorjog0: can we make it only non-gating on the 0.7.x branch?16:58
Alex_Gaynorjog0: since it gates fine on master16:59
jog0Alex_Gaynor: perhaps, lets see if we can make it gate on 0.7.x branch as well though16:59
openstackgerritTom Fifield proposed a change to openstack-infra/jeepyb: Update doc impact to only make bugs on merge  https://review.openstack.org/4590716:59
jog0Alex_Gaynor: but I am all for that, if you put the patch up I will +116:59
jeblairpleia2: ready?17:00
jog0Alex_Gaynor: I wonder how much faster hacking/flake8 is with pypy17:00
jog0because for nova running flake8 can be slow17:01
fungione two three etherpad17:01
jog0I.E. if its faster I want to switch to using pypy hacking on my own bux17:01
jog0box17:01
Alex_Gaynorjog0: is http://bpaste.net/show/131157/ what I want?17:01
jog0Alex_Gaynor: I *think* so17:02
Alex_Gaynorclarkb: ^17:02
jog0yeah ask them17:02
Alex_GaynorI'll put it up for review17:02
clarkbAlex_Gaynor: you want to remove the voting false and invert the regex to be negative lookahead on that string17:02
Alex_Gaynorclarkb: ah ok17:02
pleia2jeblair: yeah I just have some low ones to review17:03
clarkbAlex_Gaynor: asserting that we don't run that job on that branch, but everything else is fair game17:03
pleia2https://etherpad.openstack.org/cibugreview-september201317:03
pleia2happy bug day :)17:03
openstackgerritAlex Gaynor proposed a change to openstack-infra/config: Only make hacking-pypy non-voting on stable/0.7.x  https://review.openstack.org/4590817:03
*** mrmartin has quit IRC17:03
jog0so the non-gating patch didn't work  https://review.openstack.org/#/c/45761/117:03
jog0clarkb: ^17:03
clarkbjog0: patience :) that change is pushed out by puppet which is on a slightly weird interval17:04
clarkbsource updates every 15 minutes, puppet runs about every 10 minutes17:04
jog0clarkb: ohh ahh17:04
jeblairmordred, clarkb, fungi, anyone-else: bug triage time https://etherpad.openstack.org/cibugreview-september201317:04
* fungi is already digging in17:05
jeblairclarkb: you should take the backup bug, you seem to just about have that wrapped up17:05
clarkbjeblair: will do17:05
*** changbl has quit IRC17:05
clarkbmordred: the second bug on that list has your name written all over it17:05
jeblairclarkb: did notmyname or chmouel ever respond about https://bugs.launchpad.net/openstack-ci/+bug/107648117:06
uvirtbotLaunchpad bug 1076481 in openstack-ci "Add Swift functional tests to full test run" [High,In progress]17:06
pleia2fungi: I think we can close this one https://bugs.launchpad.net/openstack-ci/+bug/120484317:08
uvirtbotLaunchpad bug 1204843 in openstack-ci "No Content-Type for logs on eavesdrop.openstack.org" [Medium,In progress]17:08
clarkbjeblair: yes, I think we went ahead and made them gating17:08
fungipleia2: oh, right. closing17:08
anteayapabelanger: this one is currently assigned to you: https://bugs.launchpad.net/openstack-ci/+bug/933843 is it still on your radar, or what would you like to do?17:08
uvirtbotLaunchpad bug 933843 in openstack-ci "Use upstream puppet modules where possible" [Medium,In progress]17:08
jeblairzaro: ping17:08
*** kpepple has joined #openstack-infra17:09
*** dprince has joined #openstack-infra17:09
jeblairi spoke to david yesterday, and told him about the tag-based releases for the clouddocs plugin17:10
jeblairi'll start an email thread and we can coordinate17:10
jeblairupdated bug 108281217:10
uvirtbotLaunchpad bug 1082812 in openstack-ci "Move rackspace/clouddocs-maven-plugin to openstack-dev/clouddocs-maven-plugin" [High,In progress] https://launchpad.net/bugs/108281217:10
clarkbjeblair: ty17:10
jeblairpleia2: is 'incomplete' not a closed state?17:12
*** cppcabrera_afk is now known as cppcabrera17:13
pleia2jeblair: I manually added the list of ones included here, I can add Incomplete, sec17:13
pleia2hm, actually I have LPSTATUS = ('New', 'Incomplete', 'Incomplete (with response)',17:14
jeblairpleia2: just wondering why https://bugs.launchpad.net/openstack-ci/+bug/1129172 is in the list17:14
uvirtbotLaunchpad bug 1129172 in openstack-ci "devstack-gate - Unable to get pack index" [High,Incomplete]17:14
pleia2jeblair: I guess "High" trumps "Incomplete"17:14
pleia2let's see what it does if I remove Incomplete17:15
*** sarob has joined #openstack-infra17:15
*** sarob has quit IRC17:16
*** sarob has joined #openstack-infra17:17
*** pmoosh has joined #openstack-infra17:17
*** sarob has quit IRC17:18
pleia2jeblair: yeah, so I was specifically including incomplete in my list, I can not do that if we want17:18
*** sarob has joined #openstack-infra17:18
zarojeblair: pong.  sorry i didn't realized that i lost my network connection.17:18
jeblairzaro: see https://launchpad.net/bugs/108281217:18
uvirtbotLaunchpad bug 1082812 in openstack-ci "Move rackspace/clouddocs-maven-plugin to openstack-dev/clouddocs-maven-plugin" [High,In progress]17:18
clarkbjeblair: how does recent update on https://bugs.launchpad.net/openstack-ci/+bug/1010621 look?17:19
uvirtbotLaunchpad bug 1010621 in openstack-ci "important servers should have backups" [Critical,In progress]17:19
*** dizquierdo has left #openstack-infra17:20
Alex_Gaynorjog0: so I'm pretty confused, at this point `git diff master` from 0.7.x is tiny, and there's nothing in there that obviously would cause pypy incompat, but it really doesn't pass on 0.7.x and does on master :/17:20
jog0 Alex_Gaynor any deps that are different?17:20
jeblairclarkb: lgtm17:20
mordredclarkb: yes. the second bug is all me17:20
zarojeblair: i can take ownership of that one if someone else isn't already working on it.17:20
Alex_Gaynorjog0: six vs. six<1.4.017:20
*** jcoufal has joined #openstack-infra17:21
jog0Alex_Gaynor: that patch is going into 0.7.x next17:21
jeblairzaro: that would be great, thanks17:21
Alex_Gaynorjog0: cool, we'll see if that changes anything I guess, I can't for the life of me imagine it would17:21
jog0https://review.openstack.org/#/c/45764/17:21
zaroclarkb: took ownership of https://launchpad.net/bugs/1082812  should be easy to do since infra is already in place.17:22
uvirtbotLaunchpad bug 1082812 in openstack-ci "Move rackspace/clouddocs-maven-plugin to openstack-dev/clouddocs-maven-plugin" [High,In progress]17:22
clarkbzaro: its mostly done17:23
Alex_Gaynorjog0: grumble, pypy still fails there, still one less thing for me to look at17:23
clarkbzaro: we ended up putting it in stackforge iirc17:23
clarkbzaro: I haven't had a chance to update the bug since mordred merged the import change :P17:23
jog0Alex_Gaynor: good luck17:23
clarkbmordred: ^ you pulled the trigger too soon17:23
Alex_Gaynorjog0: I wonder if git will let me bisect between 0.7.x and master in a sane way17:24
clarkblooks like jeblair updated the bug, good17:24
mordredclarkb: sorry17:24
clarkbmordred: its ok, we will just need to force push to update the repo to the latest version before they start using gerrit17:24
*** Ryan_Lane has quit IRC17:25
jeblairhe also has a second plugin he wants to move after this one is done17:25
jog0Alex_Gaynor: looks like it does17:25
jog0'git bisect start master 0.7.x'17:25
Alex_Gaynorjog0: ok cool, I'll try that in a minute17:25
*** hub_cap has joined #openstack-infra17:26
hub_caphey is there a way to reference a dependency on another project in gerrit? besides just a self-review w a message?17:26
clarkbhub_cap: no17:26
hub_capok so then is the rule of thumb just to add a comment?17:27
hub_capw a link17:27
clarkbhub_cap: yeah, documentation17:27
fungihub_cap: bug 117239917:27
uvirtbotLaunchpad bug 1172399 in zuul "support cross-repo-dependencies in zuul" [Undecided,New] https://launchpad.net/bugs/117239917:27
clarkbhub_cap: you can also potentially express the relationship in your test suite17:27
*** matty_dubs has joined #openstack-infra17:27
clarkbhub_cap: such that you can only merge in one way with the gate, but that is often more trouble than it is worth17:27
mordredok. I'm off the phone. where are we?17:28
hub_capyes clarkb i see17:28
hub_capbut i like the fungi link :)17:28
* mordred always seems to get stuck with heavy call days on bug review days17:28
jeblairhub_cap: sorry it's short notice, but can you drop by the infra team meeting today (starts at 1900 utc) ?17:28
clarkbmordred: https://etherpad.openstack.org/cibugreview-september201317:28
hub_capjeblair: yar17:28
clarkbmordred: work down the list :)17:28
jeblairhub_cap: great thanks!17:28
hub_capcan u do me a solid jeblair? and just ping me when it starts?17:28
hub_capim lurking17:28
jeblairhub_cap: will do17:29
hub_cap<3<317:29
*** Ryan_Lane has joined #openstack-infra17:30
zaroohh. did you want to keep ownership of 1082812 then?17:30
jeblairbug 108281217:31
uvirtbotLaunchpad bug 1082812 in openstack-ci "Move rackspace/clouddocs-maven-plugin to openstack-dev/clouddocs-maven-plugin" [High,In progress] https://launchpad.net/bugs/108281217:31
jeblairzaro: no, you can have it; the only other thing is that a core needs to do a force push.  it's not a big deal.17:31
jeblairzaro: but i'll start an email thread17:31
fungiclarkb: jeblair: are we clear to delete the old paste.o.o vm? we've been on the new one a while17:31
clarkbfungi: isn't that still paste.drizzle.org?17:32
jeblairmordred: ^ ?17:32
fungiahh, right17:32
jeblairhonestly, i think we've given them quite enough time.17:32
mordredyeah. oh - brian and mark said they don't care if that disappears17:32
fungidns for paste.drizzle.o definitely still points to the old vm17:33
* mordred double checking17:33
clarkbjeblair: https://bugs.launchpad.net/openstack-ci/+bug/1093812 should I go ahead and remove openstack-ci from that bug? no progress from nova and unless we want to fix the nova tree ourselves I don't see much reason for tracking it17:33
uvirtbotLaunchpad bug 1093812 in openstack-ci "nova API version doc should point to Nova developer doc" [High,Confirmed]17:33
mordredbtw - bug 2 in the list is still waiting on this: https://review.openstack.org/#/c/41945/17:33
mordredclarkb: I don't think that's a thing nova can fix17:34
clarkbmordred: it is, the links are on nova/doc/source17:34
clarkbs/on/in/17:34
mordredclarkb: the redirects from nova.o.o?17:34
mordredah - I see17:35
mordrednevermind17:35
mordredclarkb: actually, I think this is a "we need to re-run the docs jobs for the previous tags"17:35
zarojeblair: is this done? https://bugs.launchpad.net/openstack-ci/+bug/108280317:36
uvirtbotLaunchpad bug 1082803 in openstack-ci "Manage jenkins global config and plugin list" [Medium,In progress]17:36
mordredclarkb: http://docs.openstack.org/developer/nova/2013.1/ works just fine - but that's because of how we publish docs on tag events, right?17:36
jeblairzaro: partially; mgagne1's summary is the current status17:36
jeblairzaro: it should be left open because there's still more to do17:36
fungiwhich may not work if the docs jobs are not backward compatible to the old states represented by those tags17:36
mordredfungi: verified. brian aker does not care if paste.drizzle.org ceases to exist17:36
*** mgagne1 is now known as mgagne17:36
clarkbmordred: developer/nova is off of master17:36
fungimordred: righteous. i'll blow away the old paste.o.o17:37
zarojeblair: i'm not sure what else is needed.  could you please update?17:37
mordredhttp://docs.openstack.org/developer/nova/2013.1/ is off the 2013.1 tag17:37
clarkbmordred: I think there may be mutliple issues expressed in that bug and it isn't clearly described17:37
jeblairzaro: the global config is not managed17:37
mordredclarkb: THAT I agree with17:37
fungimordred: clarkb: jeblair: also should be safe to delete the old wiki.o.o vm and the old-wiki.o.o dns entry i guess. any objections?17:37
mordredfungi: I'm fine with it - I think we've been running on new wiki for quite a while now17:38
jeblairfungi: there's a bug assigned to me to move stuff off that server17:38
clarkbmordred: 1. nova doc/source/* have/had hard coded links to places with versions that are old 2. we don't delete old stuff when doing the ftp upload because doing so leaves a small window where stuff will 40417:38
fungijeblair: oh, right that17:38
funginevermind17:38
fungii keep forgetting we had other sites on that system17:38
jeblairi'm still pretty bummed that we're going to lose a lot of content when we shut it down17:38
jeblairi had to use it just the other day to find some etherpads :(17:39
clarkbmordred: I think the solution to 2 is to stop hosting the docs wherever they are hosted so that we can be smarter about the file copying17:39
mordredjeblair: blerg17:39
mordredclarkb: they are currently hosted in cloud sites - perhaps that's what the CI bug should be about?17:39
jeblairmordred, clarkb: i don't object, but how does that help?17:40
jeblairmy understanding is that many docs jobs are designed around leaving files in certain directories, eg, branches17:40
Alex_Gaynorjog0: this doesn't appear to work, git bisect appears to have gotten itself into a loop of some sort, it never actually makes progress17:40
clarkbmordred: anteaya's example is great http://docs.openstack.org/developer/nova/object.model.html isn't in latest master (it is an orphaned file that refers to old stuff on the irght hand side17:40
jeblairthe file copy mechanism doesn't appear to change that.17:40
*** pabelanger has quit IRC17:40
jeblairclarkb: is the bug about "obsolete files are left in place"?17:41
mordredno, the bug is about "links to old tags 404"17:41
jeblairclarkb: because i thought it was about entire missing deployments of docs17:41
clarkbjeblair: the bug should be split in two. obsolete files are left in place and in some parts of nova docs we hard code links to old stuff17:41
mordredclarkb: there is a third thing17:41
mordredit's not about the stale links in the nova docs17:41
mordredit's about the fact that the thing they want to point to does not exist anywhere17:41
jog0Alex_Gaynor: :( it was worth a short17:41
mordredso the link can't be updated17:41
jeblairthu bug says "Currently nova.openstack.org redirects to docs.openstack.org/developer/nova.17:42
mordredbecause there is no location17:42
jeblairBut the /versionnumber sites (http://nova.openstack.org/2011.2 ), points 404."17:42
jeblairthat seems pretty clear17:42
fungittx: do you still need the old summit.o.o vm at 50.57.121.48 or can it be deleted?17:42
jog0I assume after running each test you say git bisect good or bad17:42
jeblairi don't see anything about individual files that stick around17:42
clarkbjeblair: then read the description17:42
clarkbjeblair: it is a very confusing bug and the exmaple doesn't support the title and so on which is why at least a couple different bugs are expressed in it17:42
jog0Alex_Gaynor: are only a few patches that are missing in 0.7.x so shouldnt' be hard to manually do17:42
clarkbjeblair: anteaya's comment points out the thing about individual files sticking around17:43
notmynamejeblair: did you get an answer about swift functional tests?17:43
jeblairclarkb: are you saying that 'object.model.html' isn't a current file?17:43
jeblairnotmyname: yes17:43
clarkbjeblair: yes17:43
notmynamejeblair: kk. anything needed from our side?17:43
notmynamejeblair: or any changes we should know about?17:43
mordredthree bugs: orphan files, stale links in nova docs, missing publications of old tags of nova17:43
jeblairnotmyname: not that i'm aware of, thanks; we're just cleaning up old bugs17:43
clarkbjeblair: so the bug there isn't that we link to a thing that doesn't exist its that we still host content that is dead17:44
*** nati_uen_ has joined #openstack-infra17:44
jeblairmordred: my evolving understanding of the problem is now that 'stale links in nova docs' doesn't exist, because the stale links are only in files that shouldn't exist, so that should be folded into "orphan files"17:44
jeblairmordred, clarkb: ^ is that wrong?17:45
jog0Alex_Gaynor: I try running 'pypy /usr/local/bin/flake8' I get no module named pkg_resources17:45
*** nati_ueno has quit IRC17:45
mordredjeblair: yes. I believe I agree with that17:45
Alex_Gaynorjog0: yeah, you need to have pypy installed into its own virtualenv, `virtualenv -p `which pypy``17:45
*** nati_uen_ has quit IRC17:46
jog0eww17:46
*** atiwari has joined #openstack-infra17:46
jeblairclarkb: can you take 1168455 as part of your etherpad hatchet job?17:48
jeblairbug 116845517:48
uvirtbotLaunchpad bug 1168455 in openstack-ci "Etherpad export function always exports another document" [Medium,Triaged] https://launchpad.net/bugs/116845517:48
jeblairclarkb: i think the understanding is that we should drop our rewrite rule17:48
jeblairclarkb: (but consider putting in a redirect, if it's not too much trouble)17:49
clarkbjeblair: yes I can take it, and yes export still doesn't work with new etherpad when we have our rewrite rules17:49
clarkbjeblair: import does work however... I tested that as part of the hatchet job17:49
Alex_Gaynorjog0: ce4f1f35 fixes it, I have no clue how why17:49
clarkbjeblair: redirect https://etherpad.openstack.org/\([^p][^/]\) to https://etherpad.openstack.org/p/$1 ?17:50
jog0Alex_Gaynor: weird17:50
*** pycabrera has joined #openstack-infra17:50
jeblairclarkb: do you need to exclude the etherpad js, css, images and such dependencies?17:51
clarkbjeblair: yes17:51
notmynameclarkb: jeblair: mordred: anything going on right now that would cause networking errors on the swift functests gate?17:52
*** cppcabrera has quit IRC17:52
portanteand along with those failing tests are other tempest jobs as well17:53
notmynameportante: ah? it's not just our tests?17:53
portantewell17:53
portantethe tempest jobs fail often17:53
portanteso I can't tell17:54
clarkbnotmyname: portante: can you link to examples?17:54
portantehttp://logs.openstack.org/43/44943/6/check/gate-swift-devstack-vm-functional/9c1f08017:54
notmynameclarkb: specivic http://logs.openstack.org/97/45897/1/gate/gate-swift-devstack-vm-functional/7cb7029/console.html17:54
notmynameclarkb: but I see 8 patches with it right now17:54
portanteclarkb: https://bugs.launchpad.net/swift/+bug/122346617:54
uvirtbotLaunchpad bug 1223466 in swift "Functional test failures occurring sporadically" [Undecided,New]17:54
jeblairmordred: i think you should close 119203917:54
jeblairmordred: i think you should close bug 119203917:54
uvirtbotLaunchpad bug 1192039 in openstack-ci "Merge release tags back into master" [Medium,In progress] https://launchpad.net/bugs/119203917:54
zaroclarkb: is Review in progress at https://review.openstack.org/#/c/43263/17:55
portantethat bug shows a failing functional test on man page changes. :)17:55
jeblairmordred: ha, sorry, you should not close it, nm17:55
*** pycabrera is now known as cppcabrera17:55
clarkbzaro: no, see jeblair's comment17:55
zaroclarkb: is bug 1200215 an upstream bug?17:55
uvirtbotLaunchpad bug 1200215 in openstack-ci "Gerrit revert does not include Change-Id in commit message" [Medium,Triaged] https://launchpad.net/bugs/120021517:55
jeblairmordred: i'm confused by why that didn't lik17:56
clarkbzaro: no17:56
jeblairlink17:56
*** melwitt has joined #openstack-infra17:56
clarkbzaro: that is a quirk in how git hooks work17:56
clarkbfungi: has lots of details17:56
mordredjeblair: I am also confused by that17:56
*** sarob has quit IRC17:56
clarkbportante: notmyname: that is networking failing against localhost17:56
clarkbthat should just work (tm_17:56
notmynameclarkb: thus our confusion ;-)17:56
*** sarob has joined #openstack-infra17:56
jog0Alex_Gaynor: so pypy version of hacking/flake8 seems take the same amount of time17:57
clarkbnotmyname: portante: are those tests using swiftclient? at least a few of them appear to17:58
jog0oh wait that was cold cache17:58
Alex_Gaynorjog0: find a larger codebase, and see if its gets faster ;)17:58
clarkbnotmyname: portante: perhpas we have a non symmetric gate with swift and swiftclient that allowed swiftclient to break swift ?17:58
zaroclarkb: is it even a bug at all?17:58
jog0Alex_Gaynor: in nova flake8 takes 72 seconds or so17:58
jeblairclarkb: also, why are there no logs for that job?17:59
notmynameclarkb: the last patch merged to python-swiftclient was Aug2917:59
notmynameclarkb: IOW, no changes recently, and the problem only surfaced since I got on the bus this morning ;-)17:59
portantethese functional test failures seemed to start today18:00
portantewhat he said18:00
* portante SF buses are probably not the root cause, though18:00
clarkbjeblair: I am not sure, probably due to the way the job uses the d-g hooks18:00
clarkbnotmyname: portante: are the failures reproducable?18:01
clarkb*reproduceable locally?18:01
notmynameclarkb: no. works for me locally18:01
jeblairso i wonder about a devstack or other dependency change18:01
clarkbjeblair: possibly in the swift(client) dependencies as well18:01
Alex_Gaynorjog0: oh.18:02
Alex_Gaynorjog0: can we just backport this to 0.7.x ;)18:02
clarkbjeblair: actually, the job doesn't have the scp stuff for devstack jobs18:02
clarkbjeblair: so we aren't copying what logs may be there18:02
*** pblaho has joined #openstack-infra18:02
notmynameportante: the bus I got on this morning was surprisingly clean. that plus the gate failures are 2 weird things that have happened already today18:03
jog0Alex_Gaynor: looks like pypy does result ina 20% speedup afterall18:03
portante;)18:03
jeblairclarkb: i'll fix the job defn18:03
Alex_Gaynorjog0: oh, cool18:03
*** salv-orlando has quit IRC18:03
*** pabelanger has joined #openstack-infra18:03
clarkbportante: in your local env are you starting from a fresh devstack isntall so that dependencies are installed as in the gate?18:04
clarkber notmyname ^18:04
*** salv-orlando has joined #openstack-infra18:04
jog0Alex_Gaynor: yeah so if you want to backport that change to 0.7.x i will aprove it but it sounds like the wrong way to fix it18:04
*** markmcclain has joined #openstack-infra18:04
Alex_Gaynorjog0: it is totally wrong, but I completely don't understand :/18:04
notmynameclarkb: not devstack, but I've installed all the deps from test-requirements18:04
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use correct gate hook for swift functional tests  https://review.openstack.org/4591718:04
jog0Alex_Gaynor: maybe you can file a bug with pypy to deal with later18:05
*** cppcabrera has quit IRC18:05
Alex_Gaynorjog0: ahhh, I think I found a bug maybe18:05
clarkbnotmyname: devstack and devstack gate are going to install deps from openstack/requirements/global-requirements.txt to make sure everyone works on the same page however we are asymmetric with this test so that may be related18:05
clarkbmordred: ^18:05
*** cppcabrera has joined #openstack-infra18:06
portanteclarkb: link to that file?18:06
clarkbportante: https://git.openstack.org/cgit/openstack/requirements/tree/global-requirements.txt18:06
Alex_Gaynorjog0: ok I understand why <something> doesn't work, I have no idea how it only affects pypy though18:07
anteayaclarkb: could this patch be playing a role? https://review.openstack.org/#/c/45283/118:07
clarkbanteaya: no, that is for gating global-requirements and shouldn't affect anything else18:07
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Add swift-functional test to client and reqs  https://review.openstack.org/4591818:07
anteayaclarkb: k18:07
jog0Alex_Gaynor: haha18:08
clarkbanteaya: actual changes to global-requirements or the unpinned dependencies in global-requirements would potentially be causes18:08
*** salv-orlando_ has joined #openstack-infra18:08
notmynameclarkb: perhaps, but has that changed in the last 2 hours?18:08
anteayak, I'll look again18:08
anteayais it just within the last two hours?18:08
notmynameanteaya: ish. it started just a few hours ago. I don't have an exact start time18:09
Alex_Gaynorjog0: yeah, I'm confused :/ I'll put up the cherry-pick18:09
Alex_Gaynorjog0: what's the procedure for cherry-picking a thing18:09
jog0Alex_Gaynor: kk, when/if yo u find the issue I am interested in hearing what it was18:09
jog0'git cherry-pick -ex sha1'18:10
notmynameanteaya: the oldest test failure is at 9:23am (pacific). so just under 2 hours ago18:10
jog0that keeps the original change-id18:10
*** cppcabrera has quit IRC18:10
Alex_Gaynorjog0: so part of hte issue is that when you have `return` in a generator it return an empty generator, it doesn't actually return None, and moving the thing outside into a different funciton really does make it return None18:10
anteayanotmyname: hmmm okay18:10
portantenotmyname, clarkb: certainly swift's requirements.txt needs to be reconciled against the global one18:10
*** salv-orlando has quit IRC18:11
*** salv-orlando_ is now known as salv-orlando18:11
clarkbhttps://jenkins.openstack.org/job/periodic-mirror-python26/29/console18:11
anteayanotmyname: what was the last passing test prior to that?18:11
portanteglobal is missing swift's requirement of pastedeploy18:11
clarkbkeyring was updated18:11
anteayakeyring, good old keyring18:11
openstackgerritJames E. Blair proposed a change to openstack-infra/devstack-gate: Add a pip freeze invocation to save with the logs  https://review.openstack.org/4591918:11
Alex_Gaynorjog0: https://review.openstack.org/#/c/45920/ let me know if I scrwed that up18:12
portanteswift wants dnspython >= 1.10.0, while global wants 1.9.418:12
jog0Alex_Gaynor: add a little blurb saying why we are backporting18:12
Alex_Gaynorjog0: ok18:12
portantethose are the only two requirements that look out of sync18:13
portantenotmyname,clarkb: ^18:13
jog0Alex_Gaynor: once this merges lets turn pypy gating back on18:13
notmynameanteaya: hard to say. the next oldest patch is from 8:13 am. (but there may have been passing earlier runs in the ones that are marked failed now)18:13
anteayanotmyname: okay18:14
clarkbthe changes jeblair has proposed should give us a lot more info18:14
Alex_Gaynorjog0: yup, going to go abandon my other patch18:14
clarkbit may be valuable to reproduce using an env similar to the one used in the gate18:15
notmynameportante: doesn't look like our requirements or test requirements has changed since april18:15
mordrednotmyname: I know one of the reasons for dnspython pinning is around ipv6 stuff- it's possible that testnodes are starting to do somethign with ipv6 on localhost or something?18:15
*** dcramer_ has quit IRC18:15
notmynamemordred: again, is that something that changed in the last few hours?18:16
clarkbnotmyname: right, but in the devstack jobs we don't use your requirements18:16
mordrednotmyname: it's possible - I don't mean in our config18:16
mordrednotmyname: I mean in the underlying machiens we're getting from the cloud18:16
clarkbfalcon was also semi recently updated https://jenkins.openstack.org/job/post-mirror-python26/172/console18:16
Alex_Gaynorjog0: ok as soon as that's landed I'll put the re-enable gate up18:16
*** cppcabrera has joined #openstack-infra18:16
jog0Alex_Gaynor: awesome18:16
notmynameclarkb: so related (but perhaps distracting, for now) question: when upstream requirements are updated, how do we get them? like I said, our requirements files haven't changed since april18:17
mordrednotmyname: there is a pending patch which will propose changes to your repo when that happens18:17
clarkbnotmyname: you update your requirements file to match18:17
clarkbmordred: I thought you decided that you wouldn't do that?18:17
jog0jeblair: when is it possible to get https://review.openstack.org/#/c/45813/ running?18:17
mordrednotmyname: in the mean time, cloning openstack/requirements and using its update.py script is the best bet currently18:17
mordredclarkb: no, I just decided to deprioritize getting it finished18:18
notmynamemordred: clarkb: ok, so there is not outstanding patch for swift to update the requirements. so something in that process is broken if the 2 are out of sync18:18
jeblairsdague: can you look at https://review.openstack.org/#/c/45813/ and jog0's other recheckwatch patch when you have a moment?18:18
mordredsince it's not an operational need atm18:18
mordrednotmyname: no - the job doesn't exist yet18:18
mordrednotmyname: at the moment, it's purely a thing that projects can consume when they choose18:18
mordredwell, purely is the wrong word18:19
sdaguejeblair: sure18:19
jog0jeblair: good idea, sdague thanks18:19
mordredconsuming it via changes to your repo is at the project's choice of velocity18:19
notmynamemordred: "when they choose" == when something upstream changes and causes gate failures?18:19
openstackgerritPierre Rognant proposed a change to openstack-infra/jenkins-job-builder: Fix plot plugin support  https://review.openstack.org/4575418:19
mordrednotmyname: often that is a motivating factor, yes18:19
*** UtahDave has quit IRC18:19
sdagueso my concern is that closed bugs might actually be fair recheck bugs18:20
mordredthus why I do still want to get the auto-propose things going18:20
notmynamemordred: but perhaps this is distracting. it hasn't yet been established that this is the root cause yet18:20
mordrednotmyname: indeed18:20
openstackgerritA change was merged to openstack-infra/config: Use correct gate hook for swift functional tests  https://review.openstack.org/4591718:20
mordredpotentially many moving bits18:20
sdaguewhich is why I left them in, but aged them more18:20
clarkbnotmyname: right, hopefully jeblair's changes give us enough info to stab at this less blindly18:20
*** UtahDave has joined #openstack-infra18:20
clarkbjeblair: mordred fungi https://bugs.launchpad.net/openstack-ci/+bug/1129172 safe to mark that something stronger than incomplete with the new git.o.o in place?18:21
uvirtbotLaunchpad bug 1129172 in openstack-ci "devstack-gate - Unable to get pack index" [High,Incomplete]18:21
mordredclarkb: looking18:22
openstackgerritPierre Rognant proposed a change to openstack-infra/jenkins-job-builder: Fix plot plugin support  https://review.openstack.org/4575418:22
jeblairclarkb: what would you suggest?  pleia2 was explicitly including incomplete bugs in her list; i think that was the problem.18:22
clarkbjeblair: hmm, ya I think that is valid18:23
mordredclarkb: the logs are gone, so I don't know enough about that bug to say otherwise18:23
clarkbjeblair: so stick to incomplete and update etherpad populator?18:23
mordred++18:23
clarkbpleia2: ^18:23
pleia2already patched my script for next time18:24
pleia2need to push it to the githubs18:24
Alex_Gaynorjog0: so who do I bribe for the second review :)18:24
jog0Alex_Gaynor: mordred  or clarkb18:24
mordredAlex_Gaynor: which review?18:24
Alex_Gaynormordred: https://review.openstack.org/#/c/45920/18:24
Alex_Gaynormordred: backport of a thing to get hacking passing on 0.7.x against pypy18:24
openstackgerritA change was merged to openstack-infra/config: Add swift-functional test to client and reqs  https://review.openstack.org/4591818:24
jog0sdague:  if a closed bug is called in recheck we can re-open it18:25
mordreddone18:25
Alex_Gaynormordred: thanks!18:25
jeblairmordred: https://bugs.launchpad.net/openstack-ci/+bug/1214366  i'm sure that's fixed; i'm just not sure why; can you resolve that?18:25
jog0sdague: think the tradeoff of a more concise recheck page is worth this edge case18:25
uvirtbotLaunchpad bug 1214366 in openstack-ci "Could not find any downloads that satisfy the requirement markupsafe" [Undecided,New]18:25
sdaguejog0: that's fine, I zeroed it as a review18:25
mordredjeblair: got it18:26
sdagueplus it will mean I'm not the last person to touch that :)18:26
clarkbnotmyname: portante: if you recheck tests in about 15 minutes we should get a lot more info18:27
portanteclarkb: okay, thx18:27
notmynameclarkb: ah, ok. I already set one to reverify18:27
jog0sdague: heh18:27
clarkbnotmyname: https://jenkins02.openstack.org/job/gate-swift-devstack-vm-functional/368/ will capture the pip freeze but not the other things jeblair added18:28
clarkbnotmyname: still potentially useful18:28
notmynameok18:28
clarkbthough we may not copy it off of the host... beause jenkins18:28
jeblairclarkb: we need the log copy to get the pip freeze18:28
jog0sdague: also I figure someone will just open a new bug or search the bug list and find it18:28
*** UtahDave has left #openstack-infra18:28
jog0sdague: there is this one too https://review.openstack.org/#/c/45511/18:28
*** UtahDave has joined #openstack-infra18:28
jeblairnotmyname, portante, clarkb: should we hold that node and try to debug on-host?18:29
jog0Merge diplciate bugs18:29
openstackgerritAlex Gaynor proposed a change to openstack-infra/config: Re-enable gating for hacking on PyPy  https://review.openstack.org/4592318:29
openstackgerritPierre Rognant proposed a change to openstack-infra/jenkins-job-builder: Fix plot plugin support  https://review.openstack.org/4575418:29
Alex_Gaynorjog0: ^ yay18:29
clarkbjeblair: yes please18:29
openstackgerritPierre Rognant proposed a change to openstack-infra/jenkins-job-builder: Fix plot plugin support  https://review.openstack.org/4575418:30
clarkbmordred: https://bugs.launchpad.net/openstack-ci/+bug/1118469 why not simply push a .md5 file?18:30
uvirtbotLaunchpad bug 1118469 in openstack-ci "Upload integrity: jenkins should publish expected MD5" [Medium,Triaged]18:30
clarkbmordred: your suggestion seems complicated18:30
mordredclarkb: I think my suggestion is very old18:30
jog0Alex_Gaynor: now I just need one more review to push out the 0.7.1 tag https://review.openstack.org/#/c/45764/18:31
Alex_Gaynorjog0: seems like you need to ask clarkb :)18:31
jeblairclarkb: http://paste.openstack.org/show/46509/18:31
clarkbok I will update a suggestion for uploading a .md5 file. That seems super simple. Also I think that can be low-hanging-fruit in that it should be straightforward for not us to do18:31
mordredclarkb: we could totally publish an md5 file - however, there is no SSL path to the tarballs.o.o- so it's still tough to verify that you didn't get mitm'd18:31
clarkbmordred: is jenkins https much better? I suppose you only need to make a cert exception once18:32
mordredclarkb: well.... when I made the suggestoin, there was only one jenkins and it had a real cert18:32
mordredclarkb: BUT - I like the md5 file idea18:32
clarkbjeblair: I really liked how cookiecutter the nodepool command additions were18:32
mordredclarkb: I'm just saying we should consider adding an ssl cert on things we tell people to download from18:33
notmynameclarkb: just got to the swift tests part. they failed again18:33
mordredclarkb: you know, for the reasons18:33
clarkbmordred: ++18:33
jeblairclarkb: yeah, i think i need to set the default config file location; i had to say "nodepool -c /etc/.... hold ..." instead of just "nodepool hold ..." which was the intent.  but other than that...18:33
mordredjeblair: yes. I am a fan of those18:33
jeblairmordred: we don't tell people to download from tarballs.o.o18:34
jeblairmordred: we publish releases to launchpad18:34
*** dcramer_ has joined #openstack-infra18:34
mordredjeblair: although, as I was reading them, it made me think "wow, these should be a rest API and a client lib" ...18:34
mordredjeblair: ttx publishes to launchpad after we upload the artifacts to tarballs.o.o18:34
jeblairyes, but where ttx gets the tarball is different than where "people" get them.  :)18:34
mordredjeblair: seems like there is a hole in the trust chain there for ttx18:35
mordredjeblair: sure. I misspoke then when I said "people" - I forgot, ttx isn't human ;)18:35
jeblairmordred: indeed there is, but solving that problem for ttx is not the same as solving it for everyone18:35
jeblairmordred: which is, in fact, why you stated your original idea the way you did18:35
mordredalso, we do have tarballs.o.o links for oslo.config in our repos mid-cycle18:35
mordredjeblair: but point well taken18:36
jeblairmordred: and we serve git repos over the insecure git:// protocol18:36
mordredalso, since the publish jobs run on jenkins.o.o18:36
mordredthe original suggestion should still work18:36
clarkbmordred: you shouldn't calculate on publish though18:36
clarkbmordred: you should calculate in the -tarball job18:37
mordredthis is true18:37
jog0clarkb: can you review https://review.openstack.org/#/c/45764/ its one line18:37
mordredalso, we can attach md5 files to our pypi uploads as well, iirc18:37
jeblairmordred: yep.  i'm not saying i like the original suggestion or that we shouldn't put an https cert on tarballs; i just don't want to lose sight of the actual problem18:37
mordredjeblair: ++18:37
jog0and then the 0.7.x branch is ready for hackign18:37
clarkbmordred: we already do md5 stuff with pypi18:37
*** MarkAtwood has joined #openstack-infra18:38
clarkbjog0: approved18:38
clarkbjog0: see comment on the change though18:39
adam_ganyone know sup /w the 'run.py: error: no such option: -t' errors when running troveclient tests? https://jenkins.openstack.org/view/All/job/gate-python-troveclient-python27/52/console18:39
clarkbadam_g: testr does a test discovery pass that involves importing the python modules that it can find. I am willing to bet importing run.py creates that outpt18:40
jog0clarkb: lets talk about that hacking requirements check later18:41
jog0I like the idea18:41
jog0there maybe some gotchas though18:41
clarkbjeblair: mordred: is there a reason that people should consume releases from launchpad?18:41
jeblairclarkb: because that's where our release manager publishes releases18:42
clarkbthat seems like an implementation detail of how we do it and not necessarily a requirement (but I may be missing something important like gpg key stuff)18:42
mordredclarkb, adam_g: that is correct18:42
*** thomasm has quit IRC18:42
clarkbadam_g: easy work around is to restrict where testr looks for tests. You do that in .testr.conf18:43
jeblairclarkb: it means that tarballs.o.o is not required to have the sorts of things you would expect for an official release channel for a project18:43
clarkbjeblair: thats fair18:43
jeblairclarkb: we can add those things to tarballs.o.o, and then it could be offered as an option18:43
mordredclarkb, adam_g: or, fix the actual fail in that repo18:43
*** dmakogon_ has joined #openstack-infra18:43
*** adalbas has quit IRC18:44
mordredwhich is a) importing run shouldn't try to do whatever it's trying to do18:44
mordredand b) see a18:44
* fungi just had a thought... an openstack/releases git repo where people can submit detached openpgp signatures for review, which end up being published alongside the tarballs to which they correspond. would be trivial to implement18:45
clarkbfungi: mordred jeblair I left a comment on the bug that solicits more discussion18:46
*** pcrews has quit IRC18:47
fungitheoretical workflow would involve retrieving the tarball, signing it locally after verifying what needs veryfying, then committing the detached signature to the releases repo, and submitting for review18:47
* fungi will mull over that for a bit18:47
clarkbhttps://bugs.launchpad.net/openstack-ci/+bug/1129436 seems like that could be left open until we run our own apt mirror18:47
uvirtbotLaunchpad bug 1129436 in openstack-ci "devstack failed: dpkg was interrupted" [Medium,Triaged]18:47
jeblairfungi: neat18:48
clarkbmaybe make that a wishlist item?18:48
fungithat would also allow anyone who wanted to attest to a tarball to submit a signature for possible addition. this is similar to some of the suggestions which have been batted around for attestation on the next generation of pypi18:49
*** dachary has quit IRC18:51
clarkbjeblair: the devstack slave you held unpings18:52
*** thomasm has joined #openstack-infra18:52
clarkbjeblair: did the completion of testing mark it used or delete and it went away?18:52
*** nicedice has quit IRC18:53
openstackgerritA change was merged to openstack-infra/devstack-gate: Add a pip freeze invocation to save with the logs  https://review.openstack.org/4591918:53
*** nicedice has joined #openstack-infra18:53
notmynameclarkb: jeblair: https://review.openstack.org/#/c/44570/ just failed, but still no logs18:53
*** dachary has joined #openstack-infra18:53
clarkbnotmyname: yeah the job change to grab logs merged after that job started18:53
clarkbnotmyname: at this point everything should be in. I would try again18:54
notmynamek18:54
jeblairclarkb: yeah, i think delete must have ignored hold, sorry. :(18:54
clarkbjeblair: oh well18:54
jeblairnext run should get logs, yeah?18:54
chmouelclarkb, notmyname: I launched those functional tests on a fresh devstack which update the dependences and I could not reproduce either18:55
clarkbjeblair: yes18:55
clarkbchmouel: thanks good to know18:55
chmouelI don't use localhost tho but the public IP of my vm18:55
adam_gclarkb, mordred thanks18:55
*** vipul is now known as vipul-away18:56
* jog0 just pushed the 0.7.1 tag for hacking18:57
*** adalbas has joined #openstack-infra19:00
*** fbo_away is now known as fbo19:00
*** jhesketh has joined #openstack-infra19:01
*** dcramer_ has quit IRC19:01
jeblairmeeting time!19:01
notmynameclarkb: jeblair: perhaps different error, but still no logs https://jenkins02.openstack.org/job/gate-swift-devstack-vm-functional/369/19:03
*** thomasm has quit IRC19:04
clarkbnotmyname: the logs are copied offsite19:04
clarkbnotmyname: I think there is a new issue as a result of one of jeblairs changes ...19:05
clarkb.functests: line 6: nosetests: command not found because we are running in the test and not post test portion of devstack-gate19:05
clarkbjeblair: &19:05
clarkbI think we need to revert that particular change19:06
*** dkehn_ is now known as dkehn19:06
hub_capjeblair: are yall in openstack-meeting or -alt?19:07
pleia2-meeting19:07
hub_capn/m i was scrolled up and didnt notice :)19:07
hub_capthx pleia219:07
jeblairclarkb: go for it.  it's still the wrong hook to use though. :)  i wouldn't revert the log copying though19:07
* hub_cap is dumb19:07
clarkbjeblair: I htink it is the correct hook bceause it needs devstack to actually run19:07
jeblairclarkb: ah, ok.19:09
*** vipul-away is now known as vipul19:10
openstackgerritClark Boylan proposed a change to openstack-infra/config: Partial revert of 257b10d37405e28f3bf292e1041bf2c3010c02c6  https://review.openstack.org/4592819:10
clarkbthere we go19:10
*** salv-orlando has quit IRC19:12
*** dcramer_ has joined #openstack-infra19:14
*** prad has quit IRC19:15
pabelangeranteaya, Yes, you better re-assign it. Haven't had much free time lately to hack away on puppet modules19:16
*** vipul is now known as vipul-away19:17
*** vipul-away is now known as vipul19:17
anteayapabelanger: very good thanks, I'll take it then, can you unassign yourself please?19:17
*** fbo is now known as fbo_away19:21
*** HenryG has joined #openstack-infra19:24
*** Ryan_Lane has quit IRC19:26
*** gyee has quit IRC19:27
mordredclarkb: yes. I believe that is the reason I did it originally in post-test19:28
clarkbmordred: can haz approval?19:28
jeblairBobBall: ping19:28
*** fbo_away is now known as fbo19:30
*** markmcclain has quit IRC19:31
ttxjeblair: how is 'where ttx gets the tarball is different than where "people" get them.' ? I download it from tarballs.o.o19:32
*** vipul is now known as vipul-away19:32
clarkbttx: but then you push to launchpad and that is where people consume them with signatures and SSL19:32
*** markmcclain has joined #openstack-infra19:32
ttxclarkb: right -- although the tarball copy is more of a convenience. We could just publish signatures/ssl19:33
ttx(in a non-LP setup)19:33
clarkbttx: right jeblair touched on that when I asked him why LP was required19:34
clarkbttx: and basically it depends on being able to provide the things we need that LP provides19:34
*** sandywalsh has quit IRC19:34
ttxclarkb: agreed19:34
notmynamejeblair: clarkb: when the global requirements are updated, are they gated on all of the gate tests?19:36
ttxclarkb: pypi is another interesting "copy", basically where we push libraries19:36
clarkbnotmyname: not all of them, only the devstack tests (we had missed your functional tests previously, but jeblair just added them so we are symmetric now)19:37
*** yolanda has quit IRC19:37
*** pmoosh has quit IRC19:37
*** sergmelikyan has quit IRC19:42
*** ruhe has joined #openstack-infra19:44
*** MarkAtwood has quit IRC19:46
*** mdenny has quit IRC19:47
*** mdenny has joined #openstack-infra19:48
*** thomasm has joined #openstack-infra19:48
*** sandywalsh has joined #openstack-infra19:49
*** pblaho has quit IRC19:51
*** annegentle has quit IRC19:53
openstackgerritAndreas Jaeger proposed a change to openstack-infra/config: Move admin-guide-cloud to docs.o.o/admin-guide-cloud/content  https://review.openstack.org/4593819:53
*** annegentle has joined #openstack-infra19:53
*** toddmorey has joined #openstack-infra19:54
*** MarkAtwood has joined #openstack-infra19:55
*** afazekas has quit IRC19:55
*** dolphm has joined #openstack-infra19:57
openstackgerritA change was merged to openstack-infra/config: Partial revert of 257b10d37405e28f3bf292e1041bf2c3010c02c6  https://review.openstack.org/4592819:59
jog0jeblair: now that sdague commented on https://review.openstack.org/#/c/45813/ ...20:01
jog0I am hoping to get the changes into recheck so we can use them during stablization20:01
jheskethjeblair: good meeting20:02
clarkbzaro: you might just try hacking something together for the other ACL categories20:02
notmynameportante: clarkb: jeblair: failure logs http://logs.openstack.org/97/45897/1/gate/gate-swift-devstack-vm-functional/9207838/20:02
jeblairjhesketh: thanks!  glad you could stop by!20:02
jheskethjeblair: I was hoping to discuss some log stuff with you during it but didn't quite realise how it would work20:02
clarkbnotmyname: 45928 should actually get the tests to run and fail as before20:03
jeblairnotmyname: next puppet update should fix that20:03
jheskethmaybe we should discuss it here first before in such a meeting anyway20:03
jeblairjhesketh: no problem; if you ever want to discuss anything, the agenda is on a wiki page that you may feel free to edit https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting20:04
dolphm:( is it possible for requirements/global-requirements.txt to be bypassed somehow?20:04
jeblairjhesketh: i think it might be best for me to send an email about the log thing20:04
jheskethyeah, I decided since I hadn't attended before I should get a feel first before adding an item (although I have been reading the logs+minutes for a few meetings)20:04
clarkbdolphm: not supposed to be. what is wrong?20:04
dolphmglobal-requirements.txt specified six<1.4.0 but keystone is failing checks because 1.4.1 is installed https://bugs.launchpad.net/ceilometer/+bug/1219694/comments/320:05
uvirtbotLaunchpad bug 1219694 in ceilometer/grizzly "Jenkins broken due to incompatibility between six 1.4.0 and wsme 0.5b2" [Critical,Confirmed]20:05
jeblairjhesketh: it's been meetings all morning for me, and i haven't had a chance to do anything with email yet20:05
jheskethjeblair: sure, I was meaning to do the same20:05
jheskethno worries, and definitely no hurry20:05
clarkbjog0: is dolphm's thing related to hacking?20:05
jeblairjhesketh: are you on the openstack-infra@lists.o.o mailing list?20:05
clarkbdolphm: I think that is an issue that jog0 is working to correct20:06
clarkbdolphm: and may even be fixed at this point20:06
clarkbjhesketh: jeblair: log thing?20:06
zaroclarkb: that's going to be interesting.  but i don't see another alternative.20:07
jheskethjeblair: to be honest, I can't remember and I'm on the wrong computer :p20:07
jheskethbut I believe so20:07
jheskeththe archives look familiar20:07
jheskethclarkb: so there's this vague thing where zuul should handle logs20:07
jog0Lear it may be, but hacking 0.7.1 is out which fixes that20:07
dolphmclarkb: there's 3 failed runs showing in zuul right now20:07
jheskethI think jeblair has some more formed thoughts on it but it has been something I've been considering too20:08
*** rfolco has joined #openstack-infra20:08
clarkbhttp://pypi.openstack.org/openstack/hacking/ new version is there20:08
jeblairclarkb: mikal mentioned to me that he and jhesketh have log storing needs and it sounded like their needs might be very similar to the 'log storing/retrieving' thing (to replace scp-to-static.o.o)20:08
jheskethspecifically because I've been working on a non-jenkins based worker for zuul20:08
portanteclarkb: so where are we at with swift functional test failures?20:08
jeblairwhich we also need for non-jenkins based workers for zuul :)20:08
*** melwitt1 has joined #openstack-infra20:08
*** melwitt has quit IRC20:08
clarkbportante: pushed a fix to one of the fixes :) now waiting for that to get applied so that we get real logs20:09
jeblairjhesketh, clarkb: so i'll send an email with a summary of things we've brainstormed about on the subject to the openstack-infra list20:09
clarkbjeblair: interesting20:09
portanteokay, so will that take another 15 minutes?20:09
clarkbportante: you may already be good at this point20:09
clarkbportante: jobs started now I expect to have the new config20:09
jeblairi think the list is a good place for that, as it might be a bit of an evolving design discussion20:09
portanteokay20:09
clarkbjeblair: ++20:10
jheskethjeblair: sounds good20:10
clarkbapparently we don't do a pip freeze in the pep8 job :/20:11
clarkbjog0: https://jenkins01.openstack.org/job/gate-keystone-pep8/706/console according to timestamps there and on the mirror that job should've used hacking 0.7.120:12
clarkboh unless maybe old hacking was installed because of a pin?20:12
jog0clarkb: it shouldn't be we pin between 0.7 and 0.820:13
jog0but not sure if everything is there yet20:13
*** vipul-away is now known as vipul20:13
clarkbportante: notmyname: https://jenkins02.openstack.org/job/gate-swift-devstack-vm-functional/372/console just started. I am watching it20:13
*** sarob has quit IRC20:14
mordredjog0: we should make a gate job for hacking that runs proposed hacking change against trunk of all projects that gate on hacking20:14
jog0clarkb: the pinning looks right20:14
jog0mordred: we have20:14
mordredjog0: we have?20:14
jog0http://logs.openstack.org/24/45824/3/check/gate-hacking-integration-nova/7e3fed0/console.html20:15
jog0it only does nova right now20:15
jog0but we can add others20:15
jog0mordred: it doesn't do any deps checks though20:15
clarkbdolphm: I am trying to reproduce locally to get a better handle on this20:16
jog0Clark not surwe whats going on20:16
clarkbjog0: yeah I am going to work on reproducing20:17
dolphmclarkb: thank you20:17
jog0clarkb: 2013-09-10 20:03:17.120 | pkg_resources.VersionConflict: (six 1.4.1 (/home/jenkins/workspace/gate-keystone-pep8/.tox/pep8/lib/python2.7/site-packages), Requirement.parse('six<1.4.0'))20:17
jog0so yeah we pinned six because ...20:18
jog0https://review.openstack.org/#/c/44656/20:19
*** sarob has joined #openstack-infra20:19
jog0and https://review.openstack.org/#/c/45641/20:20
jog0dolphm clarkb ^20:20
*** dprince has quit IRC20:22
*** ArxCruz has quit IRC20:23
* jog0 hates dependency hell20:24
mordredjog0: you LOVE dependency hell20:24
clarkbjog0: fails that way with new hacking (0.7.1)20:25
jog0mordred: we should get to the point where hacking  is autosynced with global openstack reqs20:25
clarkbsomething else has unpinned six20:25
mordredjog0: ++20:25
clarkbjog0: it is easy, just add the job :)20:25
* jog0 looks ata mordred to make the magic20:25
*** markmcclain has quit IRC20:25
jog0well we need the job for the stable branch20:25
* mordred wrote the job already20:25
jog0'stable'20:25
clarkbhttp://paste.openstack.org/show/46535/ somewhere in that list of stuff we pull in six > 1.4020:26
mordredhttps://review.openstack.org/#/c/40068/20:27
mordredjog0: ++20:27
mordredjog0: ^^ I mean20:27
jog0mordred: hacking 0.7.x branch is missing from that20:27
clarkbhttp://git.openstack.org/cgit/openstack/oslo.config/tree/requirements.txt?id=1.2.0a320:27
clarkbI think we have a winner20:27
mordredjog0: yah. I'm just saying, those are the mechanics20:27
jog0mordred: pretty slick I like it20:28
clarkbI am going to propose something different20:28
clarkbwe revert the openstack/requirements six requirement20:28
clarkband make ceilometer and wsme play nice20:28
clarkbbecause20:28
clarkbceilometer + wsme is a small web20:28
clarkbsix is a massive web20:28
* dhellmann perks up his ears20:28
dhellmann"play nice"?20:29
portanteclarkb: notmyname: job failed, not sure what was supposed to look different in the logs ...20:29
portante         https://jenkins02.openstack.org/job/gate-swift-devstack-vm-functional/372/console20:29
clarkbdhellmann: https://review.openstack.org/#/c/44656/20:29
jog0clarkb: any solution sounds good to me20:29
mordreddhellmann: we're stubbing our toes on six pinning20:29
*** salv-orlando has joined #openstack-infra20:29
clarkband I know that getting a new oslo.config release out is going to be hard particularly with markmc busy20:29
dhellmannclarkb: I think we have a fix for that in wsme, and just need to cut another release20:29
jeblairportante: we new have logs: http://logs.openstack.org/44/45944/1/check/gate-swift-devstack-vm-functional/877a4a9/logs/20:30
jeblairs/new/now/20:30
portantehuh, xattr at 0.6.2 ...20:31
*** sdake_ has quit IRC20:32
portanteso we test with the latest python-*client versions (-dev, -master, etc.)20:33
clarkbportante: yes20:33
clarkbportante: so that a release doesn't break everything20:33
*** gyee has joined #openstack-infra20:34
*** changbl has joined #openstack-infra20:34
portantek20:34
dimsclarkb, what do you need with oslo.config?20:34
*** bswrchrd has joined #openstack-infra20:34
clarkbdims: possibly to pin six, but I am working with ceilo folks to take a different direction and remove the six pin20:35
dimsclarkb, ok. if something is needed in oslo, i can pitch in20:36
*** Ryan_Lane has joined #openstack-infra20:36
jog0clarkb: is this breaking everything right now? if so lets unpin in hacking right now20:36
clarkbjog0: not everything, just things that have dependencies that install six before a thing with six pinned is installed20:37
boris-42guys does anybody know about this bug20:37
boris-42 pkg_resources.VersionConflict: (six 1.4.1 (/home/jenkins/workspace/gate-rally-pep8/.tox/pep8/lib/python2.7/site-packages), Requirement.parse('six<1.4.0'))20:37
jog0clarkb: so is it worth it to get a quick dirty fix hacking wise now20:38
jog0boris-42: ^20:38
mordredboris-42: yup. it's the error of the moment and is being worked20:38
boris-42oh nice thanks guys!!20:38
portanteclarkb: what added information is in those logs, the pip freeze?20:38
jeblairportante: also all of the service logs; the entire http://logs.openstack.org/44/45944/1/check/gate-swift-devstack-vm-functional/877a4a9/logs/ directory20:39
*** sdake_ has joined #openstack-infra20:39
*** sdake_ has joined #openstack-infra20:39
clarkbI have sent mail to ceilo folks asking about the feasibility of fixing this on the ceilo side so that we can unpin six everywhere20:40
fungiclarkb: 1700 utc this saturday works fine for me. we'll get 'em moved then20:40
clarkbfungi: great20:40
jeblairfungi: thanks20:40
* fungi re-lurks for meetup, already in progress ;)20:40
jog0revert six pin in 0.7.x: https://review.openstack.org/4595020:42
jog0clarkb mordred ^ once merged will cut 0.7.220:42
clarkbjog0: lets hold off on merging until we know if we can fix this in ceilo20:43
portanteclarkb: can we get those functional tests runs to happen where nose saves the stdout and stderr of each test and prints it out with the failures?20:43
jog0clarkb: fine with me20:43
clarkbjog0: otherwise we may need to make a great big pinning everywhere :)20:43
jog0-1 or 2 the patch20:43
portanteclarkb: and can we get nose to use -v there?20:43
jog0i just wanted it up just in case20:43
jog0figure break one project instead of >120:43
clarkbAlex_Gaynor: ^ can you take a look at the six stuff? any opinions?20:44
mordredportante: we basically just run .functest - so whatever that does20:45
clarkbportante: we should be able to by updating the job http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/swift-functional.yaml#n22 assuming the functests script knows how to do that20:45
clarkbportante: also we should work on getting you guys on testr once Icehouse opens as we currently do all of that for all of the testr test runs20:46
mordred++20:47
mordredwe'll also want that so that we can do "zuul-processes-test-run-output-streaming" everywhere20:47
clarkbportante: looks like you will need to update .functests to get that info20:48
clarkbit isn't very configurable20:48
clarkbactually it does use $@20:49
clarkbso .functests -v20:50
clarkbaccording to the nose docs stdout capture is enabled by default20:50
clarkbwe don't seem to override it in setup.cfg20:50
*** sarob has quit IRC20:50
portantehmm, wonder where that output is coming from then20:51
clarkbportante: stderr? looking at a nose issue they don't capture stderr20:52
clarkbI would link to it but I had to open it in google's cache because the direct link won't open20:52
portanteclarkb, mordred: can you update the yaml to add -v to functests20:53
clarkbportante: I think we needt to set --verbosity=320:54
*** ericw has quit IRC20:54
clarkb-v is the same as 2 which is set in setup.cfg. I will propose that change20:54
portantek thanks20:54
portanteclarkb, in the mean time, I am going to modify our func test client to report the errors that it is encountering20:55
*** adalbas has quit IRC20:57
annegentlehey infra team, we're seeing a ton of FATAL ERROR cannot assign instance of org.apache.fop.fonts.FontTriplet to field org.apache.fop.fonts.FontTriplet.style20:57
annegentleit's causing a lot of rechecks. An example is at http://logs.openstack.org/24/45924/1/gate/gate-openstack-manuals-validate/94d5ee2/console.html20:58
openstackgerritKhai Do proposed a change to openstack-infra/config: Diff between installed packages and requirements  https://review.openstack.org/4532921:00
openstackgerritClark Boylan proposed a change to openstack-infra/config: Run swift functional tests verbosely.  https://review.openstack.org/4595321:00
clarkbjeblair: fungi mordred ^21:01
mordredcall for tech is 621:01
mordredactor call at 721:01
mordredgah21:01
mordredwell, you all know when the actor call is now21:01
openstackgerritKhai Do proposed a change to openstack-infra/config: Diff between installed packages and requirements  https://review.openstack.org/4532921:01
*** prad has joined #openstack-infra21:01
clarkbannegentle: that stacktrace is being generated in the cloud docs plugin21:02
clarkbannegentle: I would check with David21:02
annegentleclarkb: okay will do21:02
*** bodepd has quit IRC21:02
clarkbannegentle: 2013-09-10 18:58:29.951 | at com.rackspace.cloud.api.docs.builders.PDFBuilder.postProcessResult(PDFBuilder.java:245) being the important line in the long21:03
clarkbs/long/log/21:03
*** bodepd has joined #openstack-infra21:03
annegentleclarkb: oo thanks for spelunking for me21:04
jgriffithanything funky going on, or is this just a reverify no bug:  http://logs.openstack.org/68/41868/19/gate/gate-cinder-pep8/b6ad019/console.html21:04
clarkbjgriffith: major funkyness21:04
jgriffithclarkb: can't figure out where the req's conflict came in from the rebase21:05
clarkbjgriffith: long story short we pinned six in openstack/requirements to < 1.4.0 to accomodate ceilometer, but in doing so made the hurt spread elsewhere21:05
*** rfolco has quit IRC21:05
jgriffithAhh :)21:05
clarkbjgriffith: currently working to unpin six and fix the problems on the ceilo side so that everyone else can continue on as usual21:05
jgriffithclarkb: sounds good, thanks for the details21:06
jog0clarkb: lets mege this https://review.openstack.org/#/c/45950/21:06
clarkbjog0: ok it has my +2 I will let you hit the go button21:06
*** matty_dubs is now known as matty_dubs|gone21:07
jog0clarkb: kk21:07
clarkbportante: https://review.openstack.org/#/c/45953/1/modules/openstack_project/files/jenkins_job_builder/config/swift-functional.yaml your review on that is valuable as well21:07
*** dmakogon_ has quit IRC21:08
*** rockyg has joined #openstack-infra21:09
mordredstevebaker, dstufft: pip mucking with file mode things21:09
stevebakerdstufft: hey21:10
*** adalbas has joined #openstack-infra21:11
stevebakeroh, hang on. It looks like https://github.com/pypa/pip/blob/develop/pip/util.py#L590 already has some mode fixing21:11
stevebakerlet me test that21:12
stevebakerok, the problem is fixed in the develop branch, nice!21:13
* stevebaker waits for 1.4.221:13
*** dolphm has left #openstack-infra21:15
*** sarob has joined #openstack-infra21:15
notmynameportante: changes are pending to verbosely run functional tests? is that where we are?21:17
jeblairnotmyname: yes https://review.openstack.org/#/c/45953/121:18
*** vipul is now known as vipul-away21:18
jeblairnotmyname: we would like a swift person to look at that21:18
jeblairportante: may have been working on a similar change to .functests itself, from what i read21:19
*** adalbas has quit IRC21:19
notmynamejeblair: lgtm21:20
*** Ryan_Lane has quit IRC21:22
*** Ryan_Lane1 has joined #openstack-infra21:22
* jog0 cut hacking 0.7.2 now with no six pinning21:24
*** Ryan_Lane1 is now known as Ryan_Lane21:24
*** pcm_ has quit IRC21:24
*** Ryan_Lane has quit IRC21:24
*** Ryan_Lane has joined #openstack-infra21:24
notmynamejeblair: so that needs to land then a test suite rerun?21:24
jeblairnotmyname: yes; i approved it; puppet will need to reconfigure the job (so it'll take a bit)21:24
notmynamejeblair: bit=5 minutes? bit=1 hour?21:25
jeblairnotmyname: (for immediate results, propose a change to swift that makes that the default; you don't have to merge it, the check run will use the change)21:25
jeblairnotmyname: 0-25 minutes21:25
notmynamejeblair: k21:26
jeblairnotmyname: some folks are working on making that instantaneous21:26
openstackgerritA change was merged to openstack-infra/config: Run swift functional tests verbosely.  https://review.openstack.org/4595321:26
notmynamejeblair: I'll wait for the job21:26
*** KLevenstein1 has joined #openstack-infra21:26
*** KLevenstein has quit IRC21:30
clarkbjeblair: I also think some of the log weirdness may be related to devstack-gate logging21:31
jeblairclarkb: log weirdness?21:31
clarkbjeblair: I am trying to read up on how it works to make sure we are doing the right thing for post_test_hook21:31
*** sarob has quit IRC21:31
clarkbjeblair: http://logs.openstack.org/01/44901/4/check/gate-swift-devstack-vm-functional/9d8245c/console.html around 2013-09-10 20:33:24.988 we appear to have overlapping streams21:31
clarkbjeblair: and the actual call to .functest isn't logged21:32
clarkbI don't think this is the reason for the failures, just makes it a little harder to read what is going on21:32
*** HenryG has quit IRC21:32
clarkbjeblair: I think we might want to redirect the output of pre and post hooks and set -oxtrace like we do for setupworkspace21:33
jeblairclarkb: wfm21:33
*** jhesketh has quit IRC21:33
*** thomasm has quit IRC21:34
*** flaper87 is now known as flaper87|afk21:37
*** vipul-away is now known as vipul21:38
*** prad has quit IRC21:39
*** mriedem has quit IRC21:41
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Log pre and post test hooks to file and stdout.  https://review.openstack.org/4595621:42
clarkbjeblair: something like ^21:42
clarkbjeblair: any chance we can get that nodepool hold bug fixed as well so that holding a slave is an optio?21:43
jeblairclarkb: sure, i'll work on that in a sec.21:43
jeblairclarkb: we should gate d-g on the functests run21:43
clarkbjeblair: ++ but I think we should get them passing again (debugging may be hard otherwise)21:44
*** ruhe has quit IRC21:44
jeblairyes; just thinking that it will help exercise this part of the script21:44
*** fbo is now known as fbo_away21:47
*** Ryan_Lane has quit IRC21:47
*** Ryan_Lane1 has joined #openstack-infra21:47
*** flaper87|afk is now known as flaper8721:47
*** flaper87 is now known as flaper87|afk21:48
*** thomasm has joined #openstack-infra21:49
*** Ryan_Lane has joined #openstack-infra21:49
*** Ryan_Lane1 has quit IRC21:49
*** cppcabrera has left #openstack-infra21:52
*** prad has joined #openstack-infra21:54
*** rockyg has quit IRC21:55
*** amotoki_ has joined #openstack-infra22:00
*** amotoki_ has quit IRC22:00
*** amotoki_ has joined #openstack-infra22:00
*** prad has quit IRC22:01
*** krtaylor has joined #openstack-infra22:02
*** burt has quit IRC22:02
*** boris-42 has quit IRC22:03
clarkbnotmyname: portante: I just started https://jenkins02.openstack.org/job/gate-swift-devstack-vm-functional/375/console22:03
clarkbthat should include verbose functests22:04
*** whayutin_ has quit IRC22:04
*** toddmorey has quit IRC22:06
*** dkranz has quit IRC22:06
*** SergeyLukjanov has quit IRC22:06
notmynameclarkb: thanks22:06
*** kgriffs has joined #openstack-infra22:10
openstackgerritKiall Mac Innes proposed a change to openstack-infra/jenkins-job-builder: Ensure Conditional Builder produces valid class paths  https://review.openstack.org/4596022:10
*** sarob has joined #openstack-infra22:10
clarkbnotmyname: the output looks a lot better now22:10
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Add default location for config file  https://review.openstack.org/4596122:11
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Fix fake provider  https://review.openstack.org/4596222:11
notmynameclarkb: boom22:11
openstackgerritJames E. Blair proposed a change to openstack-infra/nodepool: Fix HOLD state  https://review.openstack.org/4596322:11
*** jhesketh_ has joined #openstack-infra22:11
*** Ryan_Lane has quit IRC22:12
*** Ryan_Lane has joined #openstack-infra22:13
clarkbis it trying to talk to keystone for the account info?22:13
*** UtahDave has quit IRC22:15
notmynameperhaps if devstack uses keystone22:16
*** sarob has quit IRC22:16
*** lcestari has quit IRC22:16
*** prad has joined #openstack-infra22:17
*** rnirmal has quit IRC22:17
jeblairthis is starting to sound like an integration test22:19
*** SergeyLukjanov has joined #openstack-infra22:19
clarkbjeblair: I am trying to decide if it is, there is code that creates account containers not sure if that is providing user auth through swift directly or not22:19
*** SergeyLukjanov has quit IRC22:20
notmynamejeblair: clarkb: for swift dev environments, we use tempauth for this (that's why we have tempauth--to lower dependency and integration issues).22:20
jeblairclarkb, mordred: if one/both of you could review a few quick changes: https://review.openstack.org/#/q/status:open+project:openstack-infra/nodepool,n,z22:20
jeblairclarkb, mordred: i'll restart nodepool to fix the hold bug22:20
notmynamebut, tempauth is not The OpenStack Way (tm)22:20
clarkbjeblair: looking22:21
*** prad has quit IRC22:21
*** lcestari has joined #openstack-infra22:21
*** torgomatic has joined #openstack-infra22:22
clarkbjeblair: lgtm. nice and simple22:22
openstackgerritA change was merged to openstack-infra/nodepool: Fix fake provider  https://review.openstack.org/4596222:25
openstackgerritA change was merged to openstack-infra/nodepool: Add default location for config file  https://review.openstack.org/4596122:25
openstackgerritA change was merged to openstack-infra/nodepool: Fix HOLD state  https://review.openstack.org/4596322:25
notmynameclarkb: even so, it looks like the keystone endpoints it's using (actually, it looks like it got back a good service catalog) are at localhost22:26
*** thomasm has quit IRC22:26
clarkbnotmyname: https://review.openstack.org/#/c/45201/ any chance that is related22:27
clarkbthere are ceilometer warnigns in the proxy log22:27
*** melwitt1 has quit IRC22:29
*** melwitt has joined #openstack-infra22:29
clarkbI wonder if that extra sed is breaking the config line22:31
clarkbjeblair: can you hold a swift functional test test node once you have nodepool updated?22:32
jeblairclarkb: will do22:32
jeblairclarkb: the important one of those 3 changes has not shown up here: https://git.openstack.org/cgit/openstack-infra/nodepool/log/22:33
jeblairclarkb: it _has_ appeared here: https://github.com/openstack-infra/nodepool/commits/master22:33
notmynameclarkb: what cm errors are you seeing? in http://logs.openstack.org/01/44901/4/check/gate-swift-devstack-vm-functional/9d8245c/logs/screen-s-proxy.txt.gz there are lots of proxy responses that seem to indicate that good things are happening22:34
clarkbnotmyname: its jus ta warning right at the beginning of the log22:34
jeblair(i'm looking at gerrit's error log)22:34
*** jpich has quit IRC22:36
*** thomasm has joined #openstack-infra22:39
jhesketh_jeblair: at the moment with the logs, do we apply any formatting to any of them?22:39
clarkbjhesketh_: we do22:39
*** jcoufal has quit IRC22:39
jeblairjhesketh_: the on-the-fly htmlization in that mod_python script22:39
clarkbjhesketh_: http://logs.openstack.org/01/44901/4/check/gate-swift-devstack-vm-functional/9d8245c/logs/screen-n-crt.txt.gz for example22:39
jhesketh_righto, which logs/gates does that apply to22:40
notmynameclarkb: I don't think it's cm22:40
notmynameclarkb: "Sep 10 20:33:59 devstack-precise-hpcloud-az2-245229 rsyslogd-2177: imuxsock lost 4065 messages from pid 21437 due to rate-limiting" is odd (in the syslog)22:40
*** sarob has joined #openstack-infra22:40
clarkbjhesketh_: all of them, the script will bail if it doesn't recognize the format22:40
jhesketh_ah okay22:40
jhesketh_what would you say is the largest text file that actually gets formatted?22:41
clarkbnotmyname: I think that is syslog complaining that it got ratelimited22:41
notmynameclarkb: right. but it means that we might not have all the data22:41
clarkbjhesketh_: nova cpu, nova api, or nova computer. They can get into the several megabytes22:41
clarkbnotmyname: well we don't use syslog22:41
jhesketh_okay22:41
clarkbnotmyname: for this reason22:41
jhesketh_clarkb: so I22:42
jhesketh_*eh, fail22:42
jhesketh_so I have been working on some private CI testing that currently just stores logs on disk22:42
jhesketh_and serves them as is22:42
*** harlowja has quit IRC22:42
jhesketh_however then I use javascript to apply formatting using regex22:43
*** thomasm has quit IRC22:43
jhesketh_this way I don't need to store mutliple copies and I figure javascript is good enough these days to handle it (coupled with fast CPU's)22:43
jhesketh_they are much much smaller in size though22:43
jhesketh_but it could be an option up to a certain size before a log server would need to cache html formatted versions22:44
clarkbjhesketh_: I have killed my browser with these log files and that is without javascript doing stuff with them22:44
*** dkliban has quit IRC22:44
jhesketh_but then agian, I'm not sure we care about disk space22:44
clarkbhowever js might be smart enough to only transform a window22:44
jhesketh_heh, fair enough22:44
*** changbl has quit IRC22:44
clarkbso that we aren't doing it for the whole thing22:44
jhesketh_clarkb: what do you mean?22:44
jhesketh_oh right, I see22:44
*** mrodden has quit IRC22:44
jhesketh_that's not a terrible idea22:45
clarkbjhesketh_: yeah I can see us doing that22:45
*** kgriffs has left #openstack-infra22:46
*** MarkAtwood2 has joined #openstack-infra22:46
*** MarkAtwood has quit IRC22:47
notmynameclarkb: portante: I've got about 10 more minutes today to look at this. any ideas on next steps?22:51
jeblairclarkb: http://paste.openstack.org/show/46559/22:51
jeblairrun https://jenkins02.openstack.org/job/gate-swift-devstack-vm-functional/377/console22:51
clarkbnotmyname: I am poking at making make_requests log actual useful info22:52
*** amotoki_ has quit IRC22:52
openstackgerritSteve Baker proposed a change to openstack/requirements: Add diskimage-builder and tripleo-image-elements  https://review.openstack.org/4596722:52
*** pcrews has joined #openstack-infra22:52
notmynameclarkb: once that run finishes and we still have access to the box, then running the functests by hand would be a good exercise22:53
clarkbnotmyname: yup22:53
*** Ryan_Lane has quit IRC22:54
*** Ryan_Lane has joined #openstack-infra22:55
mikaljeblair: I just want to confirm... It was linuxcon europe that you and mordred are going to, right?22:55
clarkbnotmyname: portante https://review.openstack.org/45970 something like that22:57
jeblairmikal: yes22:57
mikaljeblair: sweet22:57
openstackgerritPierre Rognant proposed a change to openstack-infra/jenkins-job-builder: Fix plot plugin support  https://review.openstack.org/4575422:58
jeblairclarkb: 2013-09-10 23:00:57,430 INFO nodepool.NodeCompleteThread: Node id: 245709 is complete but in HOLD state23:01
jeblairshould be good to go23:01
clarkbjeblair: woot23:01
clarkbnotmyname: did you want to jump on that host?23:01
clarkbportante: ^23:01
stevebakerhey, could I please have some eyeballs on https://review.openstack.org/#/c/45967/ so I can continue with heat-slow testing?23:01
notmynameclarkb: do you want that upstream? you could patch it locally on the hold node. I'd also recommend adding the try_count and the request itself23:02
notmynameclarkb: I don't think I have creds there23:02
clarkbnotmyname: I think it is valuable upstream to help debug future problems (as is the error is basically useless)23:02
clarkbnotmyname: not yet, but I can add your ssh key :)23:02
notmynameclarkb: right, but patching locally is a lot faster :-) (actually we can't even merge upstream yet)23:03
notmynameclarkb: pub key https://gist.github.com/notmyname/c5bb54d03a802cff2f9d23:04
jhesketh_So currently if I want to find a log for a job, is the only way to do that following the links from gerrit?23:04
notmynameclarkb: I can look at it for just a few more minutes23:04
clarkbnotmyname: ssh jenkins@15.185.234.17323:05
clarkbjhesketh_: the url is deterministic, so you can construct it23:05
clarkbjhesketh_: but otherwise yes follwoing links from gerrit is easiest23:05
notmynameclarkb: where is the swift code on there?23:05
*** nati_ueno has joined #openstack-infra23:05
jeblairnotmyname: /opt/stack/new/swift23:06
jhesketh_thanks23:07
*** nati_ueno has quit IRC23:07
*** sandywalsh has quit IRC23:08
*** alexpilotti has quit IRC23:08
Alex_Gaynorjeblair: if you've got a sec would you mind reviewing https://review.openstack.org/#/c/45923/23:10
jeblairjog0: ^23:11
Alex_Gaynorthanks23:11
Alex_GaynorAlso did someone in here ping me a while ago?23:11
clarkbAlex_Gaynor: yes wanted feedback on unpinning six23:12
Alex_Gaynorclarkb: that's waiting on a WSME release23:13
clarkbAlex_Gaynor: right, but fixing ceilometer broke everyone else23:14
Alex_Gaynorclarkb: how so?23:14
clarkbAlex_Gaynor: lots of things don't pin six, as soon as they install before pinned six you get a dependency conflict in pip23:14
clarkbAlex_Gaynor: because pip dependency resolution needs dstufft's awesomeness23:14
Alex_Gaynorclarkb: ah, so we needed to send review requests to all sub projects or something?23:15
jog0jeblair: loos good to me23:15
*** w_ has joined #openstack-infra23:15
clarkbAlex_Gaynor: no, WSME release is supposed to happen rsn23:15
clarkbAlex_Gaynor: then we unpin I think23:15
Alex_Gaynorright23:15
Alex_Gaynorclarkb: I've got a patch up that's waiting on a WSME release23:15
clarkbAlex_Gaynor: I think if we really need to pin it we have to put six first on the dependency list23:16
clarkbthat way the pinned version is installed before anything else. but right now I think we can limp by until WSME release23:16
Alex_Gaynorclarkb: in global-requirements.txt?23:16
openstackgerritA change was merged to openstack-infra/config: Re-enable gating for hacking on PyPy  https://review.openstack.org/4592323:17
clarkbAlex_Gaynor: no in the individual projects23:17
Alex_Gaynorclarkb: ah, yes23:17
*** thomasm has joined #openstack-infra23:17
*** olaph has quit IRC23:17
stevebakerwhat is needed to get diskimage-builder onto the devstack image /var/cache/pip? Is it enough to just have it in requirements/global-requirements.txt?23:19
Alex_Gaynorstevebaker: global-reqiurements.txt + the nightly job which caches stuff23:19
stevebakerAlex_Gaynor: ok, so it doesn't need to be in some project's requirements.txt too?23:20
Alex_Gaynorstevebaker: you eventually need it in your project's requirements.txt so it gets installed23:20
clarkbstevebaker: it doesn't if you install it in the image build, however we don't like doing that23:20
clarkbstevebaker: we like devstack being able to fully configure itself23:20
stevebakerclarkb: I currently do, see https://review.openstack.org/#/c/45969/23:21
stevebakerBut it is not obvious to me which project should depend on diskimage-builder in this context - most likely to me is tempest23:22
*** sandywalsh has joined #openstack-infra23:22
stevebakersince tempest needs the images to do some tests23:23
*** Ryan_Lane has quit IRC23:23
*** Ryan_Lane has joined #openstack-infra23:23
clarkbstevebaker: is it going to be used for heat testing? I think it would be fine to put it in heat23:23
stevebakerdoes putting it in test-requirements.txt lead to it being installed?23:24
*** Ryan_Lane has quit IRC23:26
*** Ryan_Lane has joined #openstack-infra23:26
clarkbstevebaker: good question, we are pip installing stuff so I think the answer is yes23:26
clarkbmordred: ^23:26
jeblairstevebaker: it sounds like tempest is probably a good place, since tempest will be the thing using it23:26
clarkbso the problem with using tempest and maybe it isn't really a problem is that we depend on site packages for it in the gate23:27
clarkbany stragglers not installed already should get installed though23:27
jeblairthe alternative, i think would be in devstack23:28
jeblairbut it doesn't seem to be big on installing dependencies these day23:29
jeblairs23:29
clarkbjeblair: just dependencies out of apt/yum23:29
stevebakerits good to know the solution is not obvious ;)23:31
jeblairalso, as an aside, the pip cache isn't terribly important on devstack slaves atm; i'm not sure there's anything in it, as fetching from the openstack mirror is quite fast.  though i wouldn't mind rsyncing the mirror if that ever gets set up.23:32
stevebakerIt looks like I can't pip install diskimage-builder without it being in the pip cache http://logs.openstack.org/69/45969/1/check/gate-tempest-devstack-vm-full/f555c61/console.html23:33
*** HenryG has joined #openstack-infra23:33
clarkbstevebaker: I think it is not in the mirror23:33
clarkbstevebaker: so you will need to add it to openstack/requirements/global-requirements.txt23:33
stevebakerclarkb: https://review.openstack.org/#/c/45967/23:34
stevebakerI'm not sure if that gate-requirements-install fail is due to the change23:34
clarkbstevebaker: it isn't23:35
*** thomasm has quit IRC23:35
clarkbstevebaker: I think your change is fine, the tests are just derpy right now23:35
stevebakerok23:36
notmynameclarkb: found it23:36
notmynameclarkb: it's the ceilometer middleware in the pipeline. please revert that patch from devstack ASAP23:37
notmynameclarkb: torgomatic is going to look at their code to see where it breaks23:37
clarkbnotmyname: great thanks23:37
clarkbnotmyname: https://jenkins01.openstack.org/job/gate-swift-devstack-vm-functional/419/console is very verbose :)23:37
notmynameclarkb: tldr, it swallows the error response that the proxy returns23:37
clarkbresulting in a BadStatusLine exception?23:38
notmynameclarkb: because it changes a 404 http resonse into a sero byte reply23:38
clarkbnotmyname: yup BadStatusLine("\'\'",)23:39
notmynameclarkb: ok, I gotta go prep for a meetup. I leave you to it. please revert ASAP and tickle the outstanding patches in swift, if possible. torgomatic will figure out why it's bad middleware23:39
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Gate devstack and devstack-gate on swift-func  https://review.openstack.org/4597623:39
notmynameclarkb: jeblair: thanks for your help23:39
jeblairand that ^ should fix the asymetric gate that let this in23:39
koolhead17mordred: are you at openstack day india by any chance?23:39
jeblairnotmyname: you're welcome, thanks for finding the actual problem.  :)23:40
clarkbjeblair: https://review.openstack.org/4597723:41
*** branen__ has joined #openstack-infra23:41
clarkbnotmyname: ^ and I assume we can destroy the slave we held?23:41
*** dims has quit IRC23:41
clarkbjeblair: do you want to recheck 45977 after 45976 goes in?23:41
*** branen__ has quit IRC23:42
*** branen has joined #openstack-infra23:42
jeblairclarkb: let's do that23:42
clarkbok 45976 lgtm23:42
jeblairclarkb: i'll push the config update through so we can get started23:43
clarkbsounds good23:43
*** sarob has quit IRC23:44
openstackgerritA change was merged to openstack-infra/config: Gate devstack and devstack-gate on swift-func  https://review.openstack.org/4597623:44
clarkbI am going to take this as an opportunity to talk home (while we wait for puppet)23:45
jeblairi'm running puppet now.  but go on home, i'll cancel and recheck.  :)23:45
clarkbjeblair: thanks.23:45
*** branen_ has quit IRC23:45
*** dkliban has joined #openstack-infra23:46
jeblairbtw, apparently virtualenv '1.10.1' is about to be installed everywhere23:46
jeblairnotice: /Stage[main]/Openstack_project::Base/Package[virtualenv]/ensure: ensure changed '1.10' to '1.10.1'23:46
*** MarkAtwood2 has quit IRC23:48
*** jhesketh_ has quit IRC23:48
jeblairerm, i didn't need to abort anything because the reload automatically lanuched that job.  oops.23:49
*** dims has joined #openstack-infra23:53
*** atiwari has quit IRC23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!