| *** liusheng has quit IRC | 00:00 | |
| jeblair | fungi: i only knew about the star wars band-aids! | 00:00 |
|---|---|---|
| *** nadya has joined #openstack-infra | 00:00 | |
| pabelanger | jeblair: will look shortly | 00:01 |
| *** liusheng has joined #openstack-infra | 00:01 | |
| jeblair | anyway, a quick test of the concept in that patc suggests we may be able to reduce geard load by 63%. | 00:01 |
| ianw | hmmm : rm -f /etc/apt/sources.list.d/backports.list /etc/apt/sources.list.d/default.list <- something in debootstrap has maybe moved away from /etc/apt/sources.list to split out files ... then we helpfully "clean things up" | 00:01 |
| jeblair | i haven't tested it locally yet, will do that tomorrow. | 00:01 |
| *** xiangxinyong has quit IRC | 00:04 | |
| openstackgerrit | Doug Hellmann proposed openstack-dev/pbr: fix post-version parsing to allow the trailing git sha https://review.openstack.org/327424 | 00:04 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Revert "Revert "Migrate tripleo to centos-7"" https://review.openstack.org/327425 | 00:05 |
| *** nadya has quit IRC | 00:05 | |
| openstackgerrit | Simon Gomizelj proposed openstack-infra/jenkins-job-builder: Fix broken skip-push on docker-build-publish https://review.openstack.org/327426 | 00:06 |
| *** asettle has joined #openstack-infra | 00:06 | |
| *** banix has quit IRC | 00:07 | |
| *** xarses has quit IRC | 00:07 | |
| *** nadya has joined #openstack-infra | 00:10 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Revert "Revert "Migrate tripleo to centos-7"" https://review.openstack.org/327425 | 00:10 |
| *** asettle has quit IRC | 00:11 | |
| *** nadya has quit IRC | 00:14 | |
| dmsimard | jeblair: how do you manage to not have ansible in either requirements or test-requirements for Zuul ? | 00:14 |
| *** jamesmcarthur has joined #openstack-infra | 00:15 | |
| dmsimard | magic ? | 00:16 |
| *** nadya has joined #openstack-infra | 00:19 | |
| notmorgan | jeblair: hm. jeblair looking at it | 00:19 |
| pabelanger | dmsimard: https://review.openstack.org/#/c/316387/ | 00:19 |
| *** r-daneel has quit IRC | 00:19 | |
| pabelanger | dmsimard: basically, we are not expecting users outside of -infra to run zuulv2.5. | 00:20 |
| pabelanger | dmsimard: since we want to move to zuulv3 asap | 00:20 |
| dmsimard | pabelanger: it's okay, but I was wondering how you were integration/unit testing ansible bits | 00:20 |
| pabelanger | dmsimard: production | 00:20 |
| dmsimard | lol :) | 00:20 |
| notmorgan | jeblair: so that looks readonable from a base standpoint. | 00:21 |
| pabelanger | dmsimard: our motivation is to get off jenkins, and learn things with ansible. Then, for v3, start up on integration testing again | 00:21 |
| notmorgan | dmsimard: tests? did you not see the "revert revert revert revert revert revert ...." patch for toggling it on/off :) | 00:21 |
| *** psilvad has quit IRC | 00:21 | |
| *** roxanaghe has quit IRC | 00:22 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Don't clear-out all repos in puppet cleanup https://review.openstack.org/327434 | 00:22 |
| *** Swami has quit IRC | 00:23 | |
| *** nadya has quit IRC | 00:23 | |
| ianw | pabelanger: ^ that should fix debian-jessie build | 00:24 |
| pabelanger | ianw: looking | 00:25 |
| dmsimard | notmorgan: oh, yeah, that one was funny. | 00:25 |
| *** bpokorny has quit IRC | 00:26 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: More encode/decode fixes https://review.openstack.org/327435 | 00:26 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 fix: GC final state inspection https://review.openstack.org/327436 | 00:26 |
| *** piet has quit IRC | 00:27 | |
| pabelanger | ianw: WFM | 00:27 |
| notmorgan | jeblair: I have some hello-kitty bandaids around here i'll toss into the pile ;) | 00:27 |
| *** piet has joined #openstack-infra | 00:27 | |
| *** shashank_hegde has quit IRC | 00:27 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Don't clear-out all repos in puppet cleanup https://review.openstack.org/327434 | 00:27 |
| ianw | now with fixed syntax | 00:28 |
| *** nadya has joined #openstack-infra | 00:28 | |
| notmorgan | pabelanger: i feel like i'm in wonderland with all this py3 stuff. | 00:28 |
| notmorgan | pabelanger: :( | 00:28 |
| *** gouthamr_ has quit IRC | 00:30 | |
| pabelanger | notmorgan was a tireless worker for justice | 00:30 |
| notmorgan | pabelanger: though i think i'm down to ~33 failures in py3 unit tests, all are related to the build queues/histories (so likely more encode/decode weirdness) | 00:31 |
| dmsimard | Oh wow, https://python3wos.appspot.com/ is much greener than when I last looked | 00:32 |
| pabelanger | notmorgan: Ya, I'd be head deep into the manually trying to figure out why the changes. | 00:32 |
| *** Daisy has joined #openstack-infra | 00:32 | |
| *** nadya has quit IRC | 00:32 | |
| notmorgan | pabelanger: sadly pdb doesn't play well with threads, so it's manual exception throwing/hacking code in to report things. | 00:33 |
| *** mtanino has quit IRC | 00:33 | |
| pabelanger | dmsimard: suds, no surprise there. Used that about 5 years ago to write a banking IVR system with asterisk | 00:33 |
| *** bpokorny has joined #openstack-infra | 00:34 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: More encode/decode fixes https://review.openstack.org/327435 | 00:34 |
| pabelanger | notmorgan: good to know | 00:34 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 fix: GC final state inspection https://review.openstack.org/327436 | 00:34 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: More encode/decode fixes https://review.openstack.org/327435 | 00:35 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 fix: GC final state inspection https://review.openstack.org/327436 | 00:35 |
| notmorgan | pabelanger: well to be more exact, not playing well with the zuul threading. it seems to be mostly ok when debugging nodepool | 00:36 |
| notmorgan | pabelanger: but there is a serious difference in the volume of things happening between the two services. | 00:36 |
| *** anteaya has quit IRC | 00:37 | |
| *** nadya has joined #openstack-infra | 00:37 | |
| *** nadya has quit IRC | 00:42 | |
| *** liusheng has quit IRC | 00:42 | |
| *** Daisy has quit IRC | 00:42 | |
| *** sdake has quit IRC | 00:42 | |
| *** Daisy has joined #openstack-infra | 00:42 | |
| *** sputnik13 has quit IRC | 00:43 | |
| *** amitgandhinz has joined #openstack-infra | 00:44 | |
| *** nadya has joined #openstack-infra | 00:46 | |
| *** Daisy has quit IRC | 00:47 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: DNM: experimenting with local.conf https://review.openstack.org/326585 | 00:47 |
| *** piet has quit IRC | 00:48 | |
| *** amitgandhinz has quit IRC | 00:49 | |
| *** tphummel has quit IRC | 00:49 | |
| *** nadya has quit IRC | 00:51 | |
| *** pvaneck has quit IRC | 00:52 | |
| *** salv-orlando has quit IRC | 00:53 | |
| *** salv-orlando has joined #openstack-infra | 00:53 | |
| *** nadya has joined #openstack-infra | 00:55 | |
| *** Sukhdev has quit IRC | 00:55 | |
| *** mriedem has joined #openstack-infra | 00:56 | |
| *** vhosakot has quit IRC | 00:56 | |
| *** vhosakot has joined #openstack-infra | 00:57 | |
| *** nadya has quit IRC | 01:00 | |
| *** jamesmcarthur has quit IRC | 01:02 | |
| *** vhosakot has quit IRC | 01:02 | |
| *** baoli has joined #openstack-infra | 01:03 | |
| *** scottda has quit IRC | 01:05 | |
| *** jamesmcarthur has joined #openstack-infra | 01:06 | |
| *** jamesmcarthur has quit IRC | 01:07 | |
| *** Apoorva has quit IRC | 01:07 | |
| *** jamesmcarthur has joined #openstack-infra | 01:08 | |
| *** yamahata has joined #openstack-infra | 01:09 | |
| *** Apoorva has joined #openstack-infra | 01:10 | |
| *** jamesmca_ has joined #openstack-infra | 01:10 | |
| *** vhosakot has joined #openstack-infra | 01:10 | |
| *** jamesmcarthur has quit IRC | 01:12 | |
| *** nadya has joined #openstack-infra | 01:13 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Fix broken skip-push on docker-build-publish https://review.openstack.org/327426 | 01:14 |
| *** jamesmca_ has quit IRC | 01:15 | |
| *** cody-somerville has joined #openstack-infra | 01:16 | |
| *** Apoorva has quit IRC | 01:17 | |
| openstackgerrit | Merged openstack-infra/project-config: horizon job to use tempest plugin in gate https://review.openstack.org/327226 | 01:17 |
| *** nadya has quit IRC | 01:18 | |
| *** rhallisey has quit IRC | 01:18 | |
| *** bpokorny_ has joined #openstack-infra | 01:19 | |
| *** bpokorny has quit IRC | 01:22 | |
| *** nadya has joined #openstack-infra | 01:22 | |
| *** roxanaghe has joined #openstack-infra | 01:23 | |
| *** bpokorny_ has quit IRC | 01:23 | |
| *** cody-somerville has quit IRC | 01:24 | |
| *** nadya has quit IRC | 01:27 | |
| *** thorst has quit IRC | 01:27 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/irc-meetings: Modify Smaug meeting time https://review.openstack.org/322713 | 01:27 |
| *** thorst has joined #openstack-infra | 01:27 | |
| *** roxanaghe has quit IRC | 01:28 | |
| *** thorst_ has joined #openstack-infra | 01:29 | |
| *** EricGonczer_ has joined #openstack-infra | 01:30 | |
| *** nadya has joined #openstack-infra | 01:31 | |
| *** thorst_ has quit IRC | 01:32 | |
| *** thorst_ has joined #openstack-infra | 01:33 | |
| *** thorst has quit IRC | 01:33 | |
| jlvillal | Ironic team would love a review of https://review.openstack.org/#/c/316662/ Has one +2. It is the final patch needed to get Grenade working for Ironic :) | 01:35 |
| *** aysyd has quit IRC | 01:36 | |
| jlvillal | It is a devstack-gate patch | 01:36 |
| *** nadya has quit IRC | 01:36 | |
| *** nadya has joined #openstack-infra | 01:41 | |
| *** thorst_ has quit IRC | 01:41 | |
| *** nwkarsten has quit IRC | 01:44 | |
| *** amitgandhinz has joined #openstack-infra | 01:45 | |
| *** nadya has quit IRC | 01:45 | |
| *** mtanino has joined #openstack-infra | 01:49 | |
| *** amitgandhinz has quit IRC | 01:49 | |
| *** nadya has joined #openstack-infra | 01:50 | |
| *** sarob has quit IRC | 01:52 | |
| *** vhosakot has quit IRC | 01:53 | |
| *** vhosakot has joined #openstack-infra | 01:54 | |
| *** nadya has quit IRC | 01:54 | |
| *** r-mibu has quit IRC | 01:57 | |
| *** r-mibu has joined #openstack-infra | 01:57 | |
| *** banix has joined #openstack-infra | 01:58 | |
| *** nadya has joined #openstack-infra | 01:59 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Modify Smaug meeting time https://review.openstack.org/322713 | 01:59 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add meeting chair to Congress meetings https://review.openstack.org/325066 | 01:59 |
| *** PollyZ has joined #openstack-infra | 02:01 | |
| *** yamamoto_ has joined #openstack-infra | 02:01 | |
| *** nadya has quit IRC | 02:04 | |
| *** savihou has quit IRC | 02:04 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Replace the tox macro by shell to pre-export env variable https://review.openstack.org/327439 | 02:05 |
| *** cody-somerville has joined #openstack-infra | 02:05 | |
| *** cody-somerville has joined #openstack-infra | 02:05 | |
| *** savihou has joined #openstack-infra | 02:05 | |
| *** vhosakot has quit IRC | 02:07 | |
| *** vhosakot has joined #openstack-infra | 02:07 | |
| *** nadya has joined #openstack-infra | 02:08 | |
| *** yamahata has quit IRC | 02:08 | |
| *** EricGonczer_ has quit IRC | 02:09 | |
| *** cody-somerville has quit IRC | 02:10 | |
| *** nadya has quit IRC | 02:12 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/project-config: Add swift bitrot/periodic jobs https://review.openstack.org/327440 | 02:14 |
| *** mriedem has quit IRC | 02:14 | |
| *** xarses has joined #openstack-infra | 02:14 | |
| *** nadya has joined #openstack-infra | 02:17 | |
| *** ayoung has quit IRC | 02:17 | |
| *** cody-somerville has joined #openstack-infra | 02:18 | |
| *** alex_xu has quit IRC | 02:21 | |
| *** PollyZ has quit IRC | 02:21 | |
| *** larainema has quit IRC | 02:21 | |
| *** nadya has quit IRC | 02:21 | |
| *** alex_xu has joined #openstack-infra | 02:23 | |
| *** ayoung has joined #openstack-infra | 02:25 | |
| *** tphummel has joined #openstack-infra | 02:25 | |
| *** nadya has joined #openstack-infra | 02:26 | |
| *** woodster_ has quit IRC | 02:28 | |
| *** ayoung has quit IRC | 02:29 | |
| *** gouthamr has joined #openstack-infra | 02:31 | |
| *** nadya has quit IRC | 02:31 | |
| *** yamamoto_ has quit IRC | 02:32 | |
| *** nadya has joined #openstack-infra | 02:35 | |
| *** cody-somerville has quit IRC | 02:36 | |
| *** yamamoto_ has joined #openstack-infra | 02:36 | |
| *** ramishra has quit IRC | 02:38 | |
| *** thorst has joined #openstack-infra | 02:39 | |
| *** ramishra has joined #openstack-infra | 02:39 | |
| *** nadya has quit IRC | 02:40 | |
| *** jgriffith_away is now known as jgriffith | 02:42 | |
| *** kgiusti has left #openstack-infra | 02:43 | |
| *** yamahata has joined #openstack-infra | 02:44 | |
| *** nadya has joined #openstack-infra | 02:44 | |
| *** amitgandhinz has joined #openstack-infra | 02:45 | |
| *** lvdongbing has joined #openstack-infra | 02:46 | |
| *** thorst has quit IRC | 02:47 | |
| *** nadya has quit IRC | 02:49 | |
| *** lvdongbing has quit IRC | 02:50 | |
| *** yamahata has quit IRC | 02:50 | |
| *** yuanying has quit IRC | 02:50 | |
| *** amitgandhinz has quit IRC | 02:50 | |
| *** lvdongbing has joined #openstack-infra | 02:52 | |
| *** nadya has joined #openstack-infra | 02:53 | |
| *** tphummel has quit IRC | 02:54 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Remove ansible integration tests from ARA https://review.openstack.org/327446 | 02:55 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Remove ansible integration jobs from ARA https://review.openstack.org/327446 | 02:55 |
| *** iurygregory_ has joined #openstack-infra | 02:56 | |
| *** nadya has quit IRC | 02:58 | |
| iurygregory_ | Hi people, we have move a puppet-congress to openstack, but the repo does not contain the .gitreview file. How can we send changes? if there is something i can do please let me know (https://github.com/openstack/puppet-congress) | 02:59 |
| *** gyee has quit IRC | 03:01 | |
| *** nadya has joined #openstack-infra | 03:02 | |
| *** nwkarsten has joined #openstack-infra | 03:06 | |
| *** nadya has quit IRC | 03:07 | |
| *** lvdongbing has quit IRC | 03:07 | |
| *** piet has joined #openstack-infra | 03:09 | |
| *** sdake has joined #openstack-infra | 03:10 | |
| *** shashank_hegde has joined #openstack-infra | 03:10 | |
| *** nwkarsten has quit IRC | 03:11 | |
| *** lvdongbing has joined #openstack-infra | 03:11 | |
| *** nadya has joined #openstack-infra | 03:11 | |
| *** jerryz has quit IRC | 03:14 | |
| *** nadya has quit IRC | 03:16 | |
| *** iurygregory_ has quit IRC | 03:17 | |
| *** piet has quit IRC | 03:20 | |
| *** banix has quit IRC | 03:20 | |
| *** nadya has joined #openstack-infra | 03:21 | |
| *** gouthamr has quit IRC | 03:21 | |
| ianw | iurygregory_: make a .gitreview file? | 03:22 |
| *** roxanaghe has joined #openstack-infra | 03:24 | |
| *** piet has joined #openstack-infra | 03:25 | |
| dmsimard | ianw: Hey, any idea how I could overcome this setuptools version error ? It only happens on CentOS, not on Ubuntu. I made sure that setuptools>=17.1 was at the top of my test-requirements and you see it being setup just above.. I also don't carry site packages with tox. | 03:25 |
| dmsimard | http://logs.openstack.org/52/327352/13/check/gate-ara-integration-ansible-centos-7/567c579/console.html#_2016-06-09_03_00_26_509 | 03:25 |
| *** nadya has quit IRC | 03:25 | |
| dmsimard | I'm running out of ideas, bashed my head against this a good part of the night. Seems trivial alright but the only place I have seen that error happen is in the gate :) | 03:25 |
| *** sdake has quit IRC | 03:26 | |
| ianw | dmsimard: looking .... | 03:26 |
| ianw | so this is just a tox environment? | 03:27 |
| mgagne | this error could come from one of your dependency which has conflicting requirements | 03:28 |
| dmsimard | ianw: yeah, just tox.. on Ubuntu everything installs just fine: http://logs.openstack.org/52/327352/13/check/gate-ara-integration-smoke-ubuntu-trusty/5dd4b73/console.html#_2016-06-09_02_59_57_461 | 03:29 |
| dmsimard | fwiw I have no problems running tox on either my local laptop (fedora) or a fresh centos machine | 03:29 |
| *** roxanaghe has quit IRC | 03:29 | |
| *** nadya has joined #openstack-infra | 03:30 | |
| dmsimard | mgagne: that setuptools 0.9.8 is definitely not within any bounds in the requirements I provide, ansible requires >=13ish and mock requires >=17.1 | 03:30 |
| dmsimard | I presume it's installed by default, or something | 03:30 |
| mgagne | dmsimard: how about other downsteam deps? | 03:30 |
| mgagne | that could be it too. I had those errors with: site packages and dependencies of dependencies. and wrong setuptools on the host | 03:31 |
| dmsimard | I don't know -- the reqs are the same for Ubuntu and CentOS but it'd fail on CentOS only ? | 03:32 |
| dmsimard | mgagne: but with tox I don't pass site-packages: true.. so it's not supposed to be creating virtualenvs with site packages in it | 03:32 |
| dmsimard | another reason why I'm not understanding this | 03:32 |
| dmsimard | it's complaining from a version in site-packages | 03:32 |
| mgagne | is tox the latest on all platforms? tox used to enable site packages by default | 03:32 |
| dmsimard | It did ? Doc says it defaults to false.. let me explicitely define it and see what happens | 03:33 |
| mgagne | or was it venv? can't remember | 03:33 |
| ianw | site-packages does seem to be leaking in there, but that's not usual ... | 03:34 |
| *** nadya has quit IRC | 03:34 | |
| mgagne | found in logs: "using tox-2.3.1" | 03:34 |
| *** lvdongbing has quit IRC | 03:36 | |
| dmsimard | ianw: can we download the centos images you guys use from somewhere or do we need to dib it ourselves ? | 03:36 |
| dmsimard | trial and error in the gate isn't fun | 03:36 |
| ianw | dmsimard: we can't download it, i don't have a recent one at this moment, but i can build it. i'm not sure if anyone is around to hold a node for us atm | 03:37 |
| ianw | i think the issue might be with the virtualenv install just above this | 03:37 |
| *** kushal has joined #openstack-infra | 03:37 | |
| dmsimard | ianw: bah, it's alright, I'll try and figure it out | 03:37 |
| mgagne | how about rpm -qa and pip freeze to get an idea of versions installed? | 03:37 |
| dmsimard | yeah I was thinking just now I could remove that | 03:37 |
| *** asettle has joined #openstack-infra | 03:38 | |
| ianw | sudo PATH=/usr/sbin:/sbin:/usr/local/bin:/usr/bin yum install -y libffi-devel mercurial python-virtualenv tree | 03:38 |
| ianw | where's that coming from? | 03:38 |
| dmsimard | other-requirements.txt | 03:38 |
| ianw | ahh, yes, i see | 03:38 |
| mgagne | isn't python-virtualenv preinstall already? | 03:38 |
| ianw | mgagne: it is, but we probably overwrite it with all the latest pip/upstream | 03:39 |
| *** nadya has joined #openstack-infra | 03:39 | |
| ianw | so reinstalling the package might break things | 03:39 |
| mgagne | sure but is other-requirements.txt messing up with the preinstalled package on slave? | 03:39 |
| dmsimard | I removed python-virtualenv, I'll check what happens. There were some fun hacks to do to get some "gate jobs" to work with Travis CI (bleh) | 03:40 |
| ianw | mgagne: good point ... we may need to ensure we install that *before* pip gets to it. dmsimard -- probably worth a run without that to see if that's a smoking gun | 03:40 |
| *** amotoki has quit IRC | 03:40 | |
| dmsimard | mgagne: otherwise, also working late ? :) | 03:40 |
| *** piet has quit IRC | 03:41 | |
| mgagne | working on our new ci infra =) | 03:41 |
| mgagne | I say it's mostly completed at a very acceptable level | 03:41 |
| dmsimard | Cool stuff. We just started using software factory for RDO stuff. | 03:42 |
| mgagne | hehe | 03:42 |
| mgagne | we (well, I) choose to use Ansible instead =) | 03:42 |
| *** asettle has quit IRC | 03:43 | |
| dmsimard | Yeah, of course. You know, Red Hat dogfooding and all that. | 03:43 |
| mgagne | people are really starting to move projects to this new infra | 03:43 |
| mgagne | hehe | 03:43 |
| mgagne | true, forgot you inherited it from enovance | 03:43 |
| * dmsimard nods | 03:43 | |
| dmsimard | looks like python-virtualenv was the culprit | 03:44 |
| ianw | dmsimard: if it works without python-virtualenv, we'll want to hack around https://git.openstack.org/cgit/openstack/diskimage-builder/tree/elements/pip-and-virtualenv/install.d/pip-and-virtualenv-source-install/01-install-pip | 03:44 |
| *** nadya has quit IRC | 03:44 | |
| dmsimard | stuff went through | 03:44 |
| mgagne | hehe | 03:44 |
| ianw | pip v packages once again | 03:45 |
| *** thorst has joined #openstack-infra | 03:45 | |
| dmsimard | ianw: it's okay, in this case I don't really need the packaged version | 03:45 |
| dmsimard | it was an "artifact" migrated from travis ci | 03:45 |
| ianw | dmsimard: ok, well we should definitely "fix" it anyway ... if we pre-install python-virtualenv, then use pip to overwrite it, everyone is happy -- yum/dnf thinks the package is installed and pip owns it | 03:46 |
| *** amitgandhinz has joined #openstack-infra | 03:46 | |
| *** links has joined #openstack-infra | 03:46 | |
| *** yamamoto_ has quit IRC | 03:47 | |
| *** bpokorny has joined #openstack-infra | 03:47 | |
| *** yamamoto_ has joined #openstack-infra | 03:47 | |
| ianw | however, since it's in a dib element i'll have to get it in there, and make a release, then we wait for the rebuilt images | 03:47 |
| *** yuanying has joined #openstack-infra | 03:48 | |
| *** nadya has joined #openstack-infra | 03:48 | |
| dmsimard | up to you if you think there's a fix to be made, wfm with removing something I didn't need so I'm not going to complain | 03:49 |
| *** lvdongbing has joined #openstack-infra | 03:50 | |
| ianw | dmsimard: yeah, work fixing, save me having to debug it again next time :) actually i think we can move some of system-config:install_puppet.sh's workarounds into the virtualenv element ... i'll have a play | 03:50 |
| *** amitgandhinz has quit IRC | 03:51 | |
| *** thorst has quit IRC | 03:51 | |
| *** yamamoto_ has quit IRC | 03:52 | |
| *** nadya has quit IRC | 03:53 | |
| *** psachin has joined #openstack-infra | 03:53 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Remove ansible integration jobs from ARA https://review.openstack.org/327446 | 03:53 |
| *** amotoki has joined #openstack-infra | 03:54 | |
| *** vhosakot has quit IRC | 04:00 | |
| *** amotoki has quit IRC | 04:01 | |
| *** lvdongbing has quit IRC | 04:02 | |
| *** sflanigan has quit IRC | 04:02 | |
| *** pahuang has quit IRC | 04:03 | |
| *** amotoki has joined #openstack-infra | 04:05 | |
| *** nadya has joined #openstack-infra | 04:06 | |
| *** lvdongbing has joined #openstack-infra | 04:06 | |
| dmsimard | ianw: If you have a second to review ^ would appreciate it. Going to bed here, getting late. Thanks for the help tonight. | 04:08 |
| *** nadya has quit IRC | 04:11 | |
| *** sshnaidm|afk has quit IRC | 04:13 | |
| ianw | dmsimard: will do, see you later! | 04:13 |
| *** sputnik13_ has joined #openstack-infra | 04:14 | |
| *** sflanigan has joined #openstack-infra | 04:14 | |
| *** shashank_hegde has quit IRC | 04:15 | |
| *** nadya has joined #openstack-infra | 04:15 | |
| *** liusheng has joined #openstack-infra | 04:16 | |
| *** baoli has quit IRC | 04:17 | |
| *** lvdongbing has quit IRC | 04:19 | |
| *** pahuang has joined #openstack-infra | 04:20 | |
| *** nadya has quit IRC | 04:20 | |
| *** akshai has quit IRC | 04:24 | |
| *** nadya has joined #openstack-infra | 04:24 | |
| *** salv-orl_ has joined #openstack-infra | 04:25 | |
| *** lvdongbing has joined #openstack-infra | 04:26 | |
| *** Sukhdev_ has joined #openstack-infra | 04:26 | |
| *** roxanaghe has joined #openstack-infra | 04:26 | |
| *** shashank_hegde has joined #openstack-infra | 04:26 | |
| *** gongysh has joined #openstack-infra | 04:26 | |
| *** maishsk has joined #openstack-infra | 04:27 | |
| *** salv-orlando has quit IRC | 04:28 | |
| *** nadya has quit IRC | 04:29 | |
| *** roxanaghe has quit IRC | 04:30 | |
| *** amotoki has quit IRC | 04:31 | |
| *** amotoki has joined #openstack-infra | 04:31 | |
| *** maishsk has quit IRC | 04:32 | |
| *** kushal has quit IRC | 04:32 | |
| *** sree has joined #openstack-infra | 04:32 | |
| *** maishsk has joined #openstack-infra | 04:33 | |
| *** akshai has joined #openstack-infra | 04:33 | |
| *** maishsk has quit IRC | 04:33 | |
| *** amotoki has quit IRC | 04:33 | |
| *** yamamoto_ has joined #openstack-infra | 04:33 | |
| *** nadya has joined #openstack-infra | 04:34 | |
| *** guardianJ has joined #openstack-infra | 04:34 | |
| *** yamahata has joined #openstack-infra | 04:35 | |
| *** akshai has quit IRC | 04:36 | |
| *** larainema has joined #openstack-infra | 04:36 | |
| *** sree has quit IRC | 04:37 | |
| *** lvdongbing has quit IRC | 04:38 | |
| *** sree has joined #openstack-infra | 04:38 | |
| *** dimtruck is now known as zz_dimtruck | 04:38 | |
| *** sputnik13_ has quit IRC | 04:39 | |
| *** nadya has quit IRC | 04:39 | |
| *** lvdongbing has joined #openstack-infra | 04:41 | |
| *** sree has quit IRC | 04:42 | |
| *** nadya has joined #openstack-infra | 04:43 | |
| *** amotoki has joined #openstack-infra | 04:43 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 fix: GC final state inspection https://review.openstack.org/327436 | 04:44 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: More encode/decode fixes https://review.openstack.org/327435 | 04:44 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python3 Fix: tools use six.moves.urllib https://review.openstack.org/327459 | 04:44 |
| notmorgan | mordred, jeblair, ^ If that goes well... py34 should pass there. | 04:45 |
| *** bpokorny has quit IRC | 04:47 | |
| *** amitgandhinz has joined #openstack-infra | 04:47 | |
| *** jaosorior has joined #openstack-infra | 04:47 | |
| *** nadya has quit IRC | 04:47 | |
| *** greghaynes has quit IRC | 04:48 | |
| *** thorst has joined #openstack-infra | 04:49 | |
| *** xarses has quit IRC | 04:50 | |
| *** salv-orl_ has quit IRC | 04:51 | |
| *** salv-orlando has joined #openstack-infra | 04:51 | |
| *** amitgandhinz has quit IRC | 04:51 | |
| *** nadya has joined #openstack-infra | 04:52 | |
| *** yujunz has joined #openstack-infra | 04:52 | |
| *** lvdongbing has quit IRC | 04:55 | |
| *** harlowja_at_home has joined #openstack-infra | 04:56 | |
| *** harlowja_at_home has quit IRC | 04:56 | |
| *** lvdongbing has joined #openstack-infra | 04:56 | |
| *** harlowja_at_home has joined #openstack-infra | 04:56 | |
| *** nadya has quit IRC | 04:56 | |
| *** thorst has quit IRC | 04:56 | |
| notmorgan | mordred, jeblair: we will need a gear update and we're golden. patch proposed will bug you tomorrow for the merge/release :) | 04:57 |
| *** yujunz has quit IRC | 04:58 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: WIP: Add web-based console log streaming https://review.openstack.org/320563 | 04:58 |
| AJaeger | project-config cores, could you review https://review.openstack.org/327021 and https://review.openstack.org/324548 , please? These are two simple translation script changes. | 04:59 |
| *** tlian has quit IRC | 04:59 | |
| *** greghaynes has joined #openstack-infra | 04:59 | |
| *** salv-orlando has quit IRC | 05:00 | |
| *** nadya has joined #openstack-infra | 05:01 | |
| *** nadya_ has joined #openstack-infra | 05:02 | |
| *** nadya has quit IRC | 05:02 | |
| *** guardianJ has quit IRC | 05:04 | |
| *** savihou has quit IRC | 05:04 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove ansible integration jobs from ARA https://review.openstack.org/327446 | 05:05 |
| openstackgerrit | Merged openstack-infra/project-config: Enable functional test job in openstack ci for networking-ovn https://review.openstack.org/316983 | 05:05 |
| *** yamahata has quit IRC | 05:06 | |
| *** yamahata has joined #openstack-infra | 05:07 | |
| *** harlowja_at_home has quit IRC | 05:08 | |
| AJaeger | ianw, could you review https://review.openstack.org/326039 as well, please? I don't agree with the -1... | 05:12 |
| *** rlandy|bbl is now known as rlandy | 05:12 | |
| *** roxanaghe has joined #openstack-infra | 05:15 | |
| *** rcernin|off has joined #openstack-infra | 05:15 | |
| *** Sukhdev_ has quit IRC | 05:16 | |
| *** lvdongbing has quit IRC | 05:17 | |
| *** LaT0rtue has quit IRC | 05:18 | |
| *** rlandy has quit IRC | 05:18 | |
| *** xarses has joined #openstack-infra | 05:19 | |
| *** mhickey has joined #openstack-infra | 05:19 | |
| *** sree has joined #openstack-infra | 05:22 | |
| *** ams__ has quit IRC | 05:23 | |
| *** jed56 has quit IRC | 05:23 | |
| *** fmccrthy has quit IRC | 05:23 | |
| *** GheRivero has quit IRC | 05:23 | |
| *** vdrok has quit IRC | 05:23 | |
| *** mgkwill has quit IRC | 05:23 | |
| *** simonmcc has quit IRC | 05:23 | |
| *** lindsayk has quit IRC | 05:23 | |
| *** mwhahaha has quit IRC | 05:23 | |
| *** samyewell has quit IRC | 05:23 | |
| *** petems has quit IRC | 05:23 | |
| *** warpc has quit IRC | 05:23 | |
| *** igorbelikov has quit IRC | 05:23 | |
| *** betherly has quit IRC | 05:23 | |
| *** dirk has quit IRC | 05:23 | |
| *** jpmaxman has quit IRC | 05:23 | |
| *** lindsayk has joined #openstack-infra | 05:24 | |
| *** ameade has quit IRC | 05:24 | |
| *** evgenyl has quit IRC | 05:24 | |
| *** fdegir has quit IRC | 05:24 | |
| *** rajinir has quit IRC | 05:24 | |
| *** tpeoples has quit IRC | 05:24 | |
| *** madhuvishy has quit IRC | 05:24 | |
| *** DuncanT has quit IRC | 05:24 | |
| *** kgalanov has quit IRC | 05:24 | |
| *** MarkAtwood has quit IRC | 05:24 | |
| *** maximov has quit IRC | 05:24 | |
| *** edwarnicke has quit IRC | 05:24 | |
| *** patricku has quit IRC | 05:24 | |
| *** hrybacki has quit IRC | 05:24 | |
| *** dougwig has quit IRC | 05:24 | |
| *** rpodolyaka has quit IRC | 05:24 | |
| *** jraim has quit IRC | 05:24 | |
| openstackgerrit | Merged openstack-infra/project-config: Don't clear-out all repos in puppet cleanup https://review.openstack.org/327434 | 05:24 |
| *** iremizov has quit IRC | 05:25 | |
| *** rguillebert has quit IRC | 05:25 | |
| *** andreykurilin__ has quit IRC | 05:25 | |
| *** serverascode has quit IRC | 05:25 | |
| *** Ng has quit IRC | 05:25 | |
| *** flaviof has quit IRC | 05:25 | |
| *** yonglihe has quit IRC | 05:25 | |
| *** zxiiro has quit IRC | 05:25 | |
| *** ashtokolov has quit IRC | 05:25 | |
| *** NobodyCam has quit IRC | 05:25 | |
| *** xgerman has quit IRC | 05:25 | |
| *** sridhar_ram has quit IRC | 05:25 | |
| *** tdurakov has quit IRC | 05:25 | |
| *** cargonza has quit IRC | 05:25 | |
| *** mattfarina has quit IRC | 05:25 | |
| *** jaranovich has quit IRC | 05:25 | |
| *** aimeeu has quit IRC | 05:25 | |
| *** dham1 has quit IRC | 05:25 | |
| *** kozhukalov has quit IRC | 05:25 | |
| *** briancurtin has quit IRC | 05:25 | |
| *** boris-42 has quit IRC | 05:25 | |
| *** zhiyan has quit IRC | 05:25 | |
| *** sweston has quit IRC | 05:25 | |
| *** skath has quit IRC | 05:25 | |
| *** abregman has joined #openstack-infra | 05:25 | |
| *** xarses has quit IRC | 05:25 | |
| *** zaro has quit IRC | 05:25 | |
| *** ogelbukh has quit IRC | 05:25 | |
| *** ggherdov- has quit IRC | 05:25 | |
| *** electrical has quit IRC | 05:25 | |
| *** nikhil has quit IRC | 05:25 | |
| *** patrickeast has quit IRC | 05:25 | |
| *** zhenguo_ has quit IRC | 05:25 | |
| *** andrewbogott has quit IRC | 05:25 | |
| *** oanson has joined #openstack-infra | 05:25 | |
| *** rcernin|off has quit IRC | 05:26 | |
| *** igorbelikov has joined #openstack-infra | 05:27 | |
| *** hrybacki has joined #openstack-infra | 05:27 | |
| *** tdurakov has joined #openstack-infra | 05:28 | |
| *** ashtokolov has joined #openstack-infra | 05:28 | |
| *** NobodyCam has joined #openstack-infra | 05:28 | |
| *** rpodolyaka has joined #openstack-infra | 05:28 | |
| *** fdegir has joined #openstack-infra | 05:28 | |
| *** GheRivero has joined #openstack-infra | 05:28 | |
| *** edwarnicke has joined #openstack-infra | 05:28 | |
| *** xgerman has joined #openstack-infra | 05:28 | |
| *** kgalanov has joined #openstack-infra | 05:28 | |
| *** DuncanT has joined #openstack-infra | 05:28 | |
| *** yonglihe has joined #openstack-infra | 05:28 | |
| *** patricku has joined #openstack-infra | 05:28 | |
| *** jaranovich has joined #openstack-infra | 05:29 | |
| *** ggherdov- has joined #openstack-infra | 05:29 | |
| *** briancurtin has joined #openstack-infra | 05:30 | |
| *** MarkAtwood has joined #openstack-infra | 05:30 | |
| *** evgenyl has joined #openstack-infra | 05:30 | |
| *** vdrok has joined #openstack-infra | 05:30 | |
| *** zhenguo_ has joined #openstack-infra | 05:30 | |
| *** patrickeast has joined #openstack-infra | 05:30 | |
| *** ogelbukh has joined #openstack-infra | 05:30 | |
| *** sweston has joined #openstack-infra | 05:30 | |
| *** kozhukalov has joined #openstack-infra | 05:30 | |
| *** dham1 has joined #openstack-infra | 05:30 | |
| *** simonmcc has joined #openstack-infra | 05:30 | |
| *** samyewell has joined #openstack-infra | 05:30 | |
| *** jed56 has joined #openstack-infra | 05:31 | |
| *** ams__ has joined #openstack-infra | 05:31 | |
| *** zxiiro has joined #openstack-infra | 05:31 | |
| *** iremizov has joined #openstack-infra | 05:31 | |
| *** jraim has joined #openstack-infra | 05:32 | |
| *** aimeeu has joined #openstack-infra | 05:32 | |
| *** ameade has joined #openstack-infra | 05:32 | |
| *** mattfarina has joined #openstack-infra | 05:32 | |
| *** andrewbogott has joined #openstack-infra | 05:32 | |
| *** electrical has joined #openstack-infra | 05:32 | |
| *** sridhar_ram has joined #openstack-infra | 05:32 | |
| *** maximov has joined #openstack-infra | 05:32 | |
| *** zaro has joined #openstack-infra | 05:32 | |
| *** tpeoples has joined #openstack-infra | 05:32 | |
| *** andreykurilin__ has joined #openstack-infra | 05:32 | |
| *** gildub has quit IRC | 05:32 | |
| *** rguillebert has joined #openstack-infra | 05:33 | |
| *** petems has joined #openstack-infra | 05:33 | |
| *** patricku has quit IRC | 05:33 | |
| *** yonglihe has quit IRC | 05:33 | |
| *** shamail has joined #openstack-infra | 05:33 | |
| *** zhiyan has joined #openstack-infra | 05:33 | |
| *** skath has joined #openstack-infra | 05:33 | |
| *** rajinir has joined #openstack-infra | 05:33 | |
| *** Ng has joined #openstack-infra | 05:33 | |
| *** serverascode has joined #openstack-infra | 05:34 | |
| *** boris-42 has joined #openstack-infra | 05:34 | |
| *** fmccrthy has joined #openstack-infra | 05:34 | |
| *** mgkwill has joined #openstack-infra | 05:34 | |
| *** nikhil has joined #openstack-infra | 05:35 | |
| *** yonglihe has joined #openstack-infra | 05:35 | |
| *** madhuvishy has joined #openstack-infra | 05:35 | |
| *** dougwig has joined #openstack-infra | 05:36 | |
| *** jpmaxman has joined #openstack-infra | 05:36 | |
| *** patricku has joined #openstack-infra | 05:36 | |
| *** flaviof has joined #openstack-infra | 05:36 | |
| *** cargonza has joined #openstack-infra | 05:36 | |
| *** mwhahaha has joined #openstack-infra | 05:37 | |
| *** warpc has joined #openstack-infra | 05:37 | |
| *** dirk has joined #openstack-infra | 05:37 | |
| *** betherly has joined #openstack-infra | 05:37 | |
| *** dchen has joined #openstack-infra | 05:39 | |
| *** nadya_ has quit IRC | 05:40 | |
| *** flepied has joined #openstack-infra | 05:41 | |
| *** mohankumar_ has joined #openstack-infra | 05:46 | |
| mohankumar_ | Team , could you guide how to raise release request for release independent deliverables | 05:46 |
| *** shamail has quit IRC | 05:47 | |
| *** amitgandhinz has joined #openstack-infra | 05:48 | |
| *** fawadkhaliq has joined #openstack-infra | 05:50 | |
| *** fawadkhaliq has quit IRC | 05:50 | |
| *** fawadkhaliq has joined #openstack-infra | 05:51 | |
| *** kdas__ has joined #openstack-infra | 05:51 | |
| *** amitgandhinz has quit IRC | 05:52 | |
| AJaeger | mohankumar_: which repo? | 05:53 |
| *** kdas__ has quit IRC | 05:53 | |
| *** thorst has joined #openstack-infra | 05:54 | |
| AJaeger | mohankumar_: Best talk to release team in #openstack-release channel | 05:54 |
| AJaeger | mohankumar_: the releases repository is used for repos handled by the release team. | 05:55 |
| *** pt_15 has quit IRC | 05:56 | |
| *** savihou has joined #openstack-infra | 05:59 | |
| *** itisha has quit IRC | 06:00 | |
| *** savihou has quit IRC | 06:01 | |
| *** thorst has quit IRC | 06:01 | |
| *** savihou has joined #openstack-infra | 06:01 | |
| mohankumar_ | AJaeger : https://github.com/openstack/networking-huawei | 06:02 |
| mohankumar_ | AJaeger : thanks ! will talk to #openstack-release | 06:03 |
| *** jianghuaw has joined #openstack-infra | 06:05 | |
| *** nadya has joined #openstack-infra | 06:05 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: [WIP] install pip/virtualenv packages https://review.openstack.org/327472 | 06:05 |
| *** yolanda has joined #openstack-infra | 06:05 | |
| *** jpmaxman has quit IRC | 06:06 | |
| *** cargonza has quit IRC | 06:07 | |
| *** javeriak has joined #openstack-infra | 06:07 | |
| *** mwhahaha has quit IRC | 06:07 | |
| *** nikhil has quit IRC | 06:07 | |
| *** dirk has quit IRC | 06:08 | |
| *** warpc has quit IRC | 06:08 | |
| *** madhuvishy has quit IRC | 06:08 | |
| *** betherly has quit IRC | 06:08 | |
| *** jpmaxman has joined #openstack-infra | 06:08 | |
| *** nikhil has joined #openstack-infra | 06:09 | |
| *** yolanda has quit IRC | 06:10 | |
| *** cargonza has joined #openstack-infra | 06:10 | |
| *** yolanda has joined #openstack-infra | 06:10 | |
| *** madhuvishy has joined #openstack-infra | 06:10 | |
| *** warpc has joined #openstack-infra | 06:10 | |
| *** dirk has joined #openstack-infra | 06:11 | |
| *** nadya has quit IRC | 06:11 | |
| *** betherly has joined #openstack-infra | 06:12 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/327476 | 06:13 |
| *** pfallenop has quit IRC | 06:14 | |
| *** pfallenop has joined #openstack-infra | 06:14 | |
| *** rcernin|off has joined #openstack-infra | 06:15 | |
| *** yolanda has quit IRC | 06:16 | |
| *** yolanda has joined #openstack-infra | 06:18 | |
| *** nadya has joined #openstack-infra | 06:18 | |
| *** abregman has quit IRC | 06:20 | |
| *** mwhahaha has joined #openstack-infra | 06:20 | |
| yolanda | good morning | 06:20 |
| *** nadya_ has joined #openstack-infra | 06:21 | |
| *** rcernin|off is now known as rcernin | 06:21 | |
| *** nadya has quit IRC | 06:23 | |
| notmorgan | yolanda: mornin! | 06:23 |
| *** nadya has joined #openstack-infra | 06:23 | |
| *** flepied has quit IRC | 06:24 | |
| notmorgan | yolanda: :) | 06:24 |
| yolanda | hi notmorgan , how are you? | 06:24 |
| notmorgan | yolanda: fantastic! | 06:24 |
| notmorgan | yolanda: had a very productive day. | 06:24 |
| notmorgan | and now... winding down and relaxing | 06:24 |
| *** aeng has quit IRC | 06:24 | |
| yolanda | notmorgan, that's good. I'm just now starting | 06:24 |
| *** fawadkhaliq has quit IRC | 06:24 | |
| notmorgan | yolanda: ^_^ enjoy your day! | 06:25 |
| *** nadya_ has quit IRC | 06:26 | |
| *** maishsk has joined #openstack-infra | 06:27 | |
| *** nadya has quit IRC | 06:28 | |
| *** baoli has joined #openstack-infra | 06:29 | |
| *** sflanigan_ has joined #openstack-infra | 06:32 | |
| *** sflanigan_ has joined #openstack-infra | 06:32 | |
| *** sflanigan has quit IRC | 06:32 | |
| *** baoli has quit IRC | 06:33 | |
| *** dchen has quit IRC | 06:34 | |
| *** andreas_s has joined #openstack-infra | 06:37 | |
| *** pahuang has quit IRC | 06:37 | |
| AJaeger | mohankumar_: networking-huawei is not part of Big Tent, so release team will not handle it. | 06:38 |
| AJaeger | mohankumar_: In that case follow http://docs.openstack.org/infra/manual/drivers.html#tagging-a-release | 06:39 |
| AJaeger | HI yolanda, notmorgan ! | 06:39 |
| yolanda | hi! | 06:40 |
| AJaeger | yolanda: could you review https://review.openstack.org/326039 as well, please? | 06:40 |
| yolanda | sure | 06:40 |
| AJaeger | and also https://review.openstack.org/#/c/327021/ , please | 06:41 |
| AJaeger | yolanda: for 326039 the dependent job is currently in the gate - I'll +A later myself, just ask for second +2 | 06:41 |
| AJaeger | ianw, thanks for review! | 06:41 |
| yolanda | ok going to start with the review queue | 06:42 |
| mohankumar_ | AJaeger : i did tagging , after tagging do i raise patch to "openstack/releases/deliverables/_independent/ " like https://review.openstack.org/#/c/322505/ ? | 06:44 |
| AJaeger | mohankumar_: your project is not in big tent - so releases repository is not used for it. | 06:45 |
| AJaeger | mohankumar_: follow the infra manual and push the tag to gerrit | 06:45 |
| mohankumar_ | AJaeger : ohh ok , thanks ! | 06:46 |
| *** sorantis has joined #openstack-infra | 06:46 | |
| *** dchen has joined #openstack-infra | 06:47 | |
| *** sree has quit IRC | 06:48 | |
| *** amitgandhinz has joined #openstack-infra | 06:48 | |
| *** pahuang has joined #openstack-infra | 06:51 | |
| *** LaT0rtue has joined #openstack-infra | 06:53 | |
| *** amitgandhinz has quit IRC | 06:53 | |
| *** roxanaghe has quit IRC | 06:53 | |
| *** asettle has joined #openstack-infra | 06:54 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/327476 | 06:56 |
| *** thorst has joined #openstack-infra | 07:00 | |
| *** sarob has joined #openstack-infra | 07:05 | |
| *** thorst has quit IRC | 07:06 | |
| *** matrohon has joined #openstack-infra | 07:08 | |
| *** sarob has quit IRC | 07:09 | |
| *** pcaruana has joined #openstack-infra | 07:09 | |
| *** nmagnezi has joined #openstack-infra | 07:09 | |
| *** dchen has quit IRC | 07:11 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: make all sahara-dashboard jobs voting https://review.openstack.org/326474 | 07:12 |
| *** flepied1 has joined #openstack-infra | 07:12 | |
| *** zeih has joined #openstack-infra | 07:13 | |
| *** amoralej|off is now known as amoralej | 07:15 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix building nodes panel for zuul launchers https://review.openstack.org/327421 | 07:15 |
| *** mtanino has quit IRC | 07:15 | |
| openstackgerrit | Merged openstack-infra/project-config: Add swift bitrot/periodic jobs https://review.openstack.org/327440 | 07:15 |
| openstackgerrit | Merged openstack-infra/project-config: Move puppet-tacker to Openstack https://review.openstack.org/327178 | 07:15 |
| *** vincentll has joined #openstack-infra | 07:17 | |
| openstackgerrit | Merged openstack-infra/project-config: Add infra structure for install guide publishing https://review.openstack.org/326039 | 07:17 |
| openstackgerrit | Merged openstack-infra/project-config: Ironic: fetch tempest plugin from git for non-master https://review.openstack.org/326137 | 07:17 |
| *** pahuang has quit IRC | 07:18 | |
| *** dtantsur|afk is now known as dtantsur | 07:18 | |
| *** ihrachys has joined #openstack-infra | 07:19 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config: Make grenade-dsvm-ironic non voting https://review.openstack.org/319336 | 07:20 |
| *** arxcruz has joined #openstack-infra | 07:20 | |
| openstackgerrit | Merged openstack-infra/project-config: Add releasenotes job for manila-ui https://review.openstack.org/320718 | 07:22 |
| openstackgerrit | Merged openstack-infra/infra-ansible: Wait for server to be ssh reachable before bootstrap/configuration https://review.openstack.org/326957 | 07:22 |
| openstackgerrit | Merged openstack-infra/project-config: Run tempest tests from sahara-tests for sahara jobs https://review.openstack.org/326980 | 07:23 |
| openstackgerrit | Merged openstack-infra/project-config: Skip release notes jobs for pre-Mitaka os-brick and python-cinderclient https://review.openstack.org/327264 | 07:23 |
| *** mikelk has joined #openstack-infra | 07:25 | |
| *** fawadkhaliq has joined #openstack-infra | 07:25 | |
| *** jlanoux has joined #openstack-infra | 07:25 | |
| *** LaT0rtue has quit IRC | 07:27 | |
| *** esikachev has joined #openstack-infra | 07:27 | |
| *** fawadkhaliq has quit IRC | 07:31 | |
| *** dizquierdo has joined #openstack-infra | 07:34 | |
| *** tkelsey has joined #openstack-infra | 07:36 | |
| *** javeriak has quit IRC | 07:37 | |
| *** javeriak has joined #openstack-infra | 07:37 | |
| *** mohankumar_ has quit IRC | 07:38 | |
| *** jtomasek_ has joined #openstack-infra | 07:39 | |
| *** dtardivel has joined #openstack-infra | 07:40 | |
| *** salv-orlando has joined #openstack-infra | 07:41 | |
| *** ifarkas has joined #openstack-infra | 07:42 | |
| *** salv-orl_ has joined #openstack-infra | 07:42 | |
| *** Na3iL has joined #openstack-infra | 07:43 | |
| *** rossella_s has joined #openstack-infra | 07:44 | |
| *** salv-orlando has quit IRC | 07:46 | |
| *** fawadkhaliq has joined #openstack-infra | 07:47 | |
| *** abregman has joined #openstack-infra | 07:47 | |
| *** dizquierdo has quit IRC | 07:47 | |
| *** ajmiller has quit IRC | 07:47 | |
| *** ajmiller has joined #openstack-infra | 07:48 | |
| *** tzn has joined #openstack-infra | 07:48 | |
| *** asettle has quit IRC | 07:48 | |
| *** amitgandhinz has joined #openstack-infra | 07:49 | |
| *** yujunz has joined #openstack-infra | 07:49 | |
| *** amitgandhinz has quit IRC | 07:54 | |
| *** nadya has joined #openstack-infra | 07:54 | |
| *** yujunz has quit IRC | 07:55 | |
| *** tesseract has joined #openstack-infra | 07:56 | |
| *** sree has joined #openstack-infra | 07:57 | |
| *** sflanigan_ has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** nadya has quit IRC | 08:01 | |
| *** asettle has joined #openstack-infra | 08:01 | |
| *** asettle has quit IRC | 08:01 | |
| *** openstackgerrit has quit IRC | 08:03 | |
| *** openstackgerrit has joined #openstack-infra | 08:03 | |
| *** thorst has joined #openstack-infra | 08:04 | |
| *** jtomasek_ has quit IRC | 08:05 | |
| *** bhavik has joined #openstack-infra | 08:06 | |
| *** chem has joined #openstack-infra | 08:07 | |
| *** grue_pm has joined #openstack-infra | 08:08 | |
| *** grue_pm is now known as grue | 08:08 | |
| *** grue has quit IRC | 08:09 | |
| *** grue_pm has joined #openstack-infra | 08:09 | |
| *** tkelsey has quit IRC | 08:10 | |
| *** thorst has quit IRC | 08:12 | |
| *** _shaps_ has quit IRC | 08:15 | |
| *** electrofelix has joined #openstack-infra | 08:15 | |
| *** Swami has joined #openstack-infra | 08:15 | |
| *** Swami_ has joined #openstack-infra | 08:15 | |
| *** Na3iL has quit IRC | 08:16 | |
| openstackgerrit | Antoine Cabot proposed openstack-infra/project-config: Add watcher-dashboard doc to docs.openstack.org https://review.openstack.org/327507 | 08:17 |
| *** dtantsur is now known as dtantsur|lunch | 08:20 | |
| *** lucas-dinner is now known as lucasagomes | 08:20 | |
| *** permalac has joined #openstack-infra | 08:21 | |
| *** hdaniel has joined #openstack-infra | 08:21 | |
| *** rossella_s has quit IRC | 08:22 | |
| *** alex_xu has quit IRC | 08:22 | |
| *** Na3iL has joined #openstack-infra | 08:22 | |
| *** salv-orl_ has quit IRC | 08:25 | |
| *** zeih has quit IRC | 08:25 | |
| *** alex_xu has joined #openstack-infra | 08:25 | |
| *** daemontool has joined #openstack-infra | 08:25 | |
| *** zeih has joined #openstack-infra | 08:26 | |
| *** Douhet has quit IRC | 08:26 | |
| *** sree has quit IRC | 08:27 | |
| *** rossella_s has joined #openstack-infra | 08:27 | |
| *** jaosorior has quit IRC | 08:28 | |
| *** jaosorior has joined #openstack-infra | 08:28 | |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config: Simplify gate tests for 'intel-nfv-ci-tests' https://review.openstack.org/326989 | 08:29 |
| *** garyk1 has joined #openstack-infra | 08:30 | |
| garyk1 | hi, anyone know why the patch 324374 is not in the gate? | 08:30 |
| garyk1 | recheck does not help | 08:31 |
| *** _degorenko|afk is now known as degorenko | 08:32 | |
| *** grassass has joined #openstack-infra | 08:34 | |
| *** xarses has joined #openstack-infra | 08:35 | |
| *** Swami_ has quit IRC | 08:35 | |
| *** Swami has quit IRC | 08:35 | |
| *** xarses has quit IRC | 08:35 | |
| *** xarses has joined #openstack-infra | 08:36 | |
| *** mhickey has quit IRC | 08:36 | |
| *** dizquierdo has joined #openstack-infra | 08:36 | |
| *** jtomasek_ has joined #openstack-infra | 08:38 | |
| *** salv-orlando has joined #openstack-infra | 08:39 | |
| openstackgerrit | Merged openstack-infra/zuul: Python 3 Fix: use bytesIO instead of cStringIO https://review.openstack.org/323016 | 08:40 |
| *** markusry has joined #openstack-infra | 08:40 | |
| openstackgerrit | Merged openstack-infra/zuul: Python 3 Fix: use proper octal notation https://review.openstack.org/323017 | 08:41 |
| *** zeih has quit IRC | 08:41 | |
| *** dingyichen has quit IRC | 08:45 | |
| *** savihou has quit IRC | 08:48 | |
| *** savihou has joined #openstack-infra | 08:49 | |
| *** asettle has joined #openstack-infra | 08:49 | |
| *** amitgandhinz has joined #openstack-infra | 08:50 | |
| openstackgerrit | Anusha Ramineni proposed openstack-infra/project-config: congress: Make new arch jobs voting https://review.openstack.org/327514 | 08:51 |
| *** gema has quit IRC | 08:52 | |
| *** maishsk has quit IRC | 08:53 | |
| *** gema has joined #openstack-infra | 08:53 | |
| *** amitgandhinz has quit IRC | 08:54 | |
| *** zeih has joined #openstack-infra | 08:55 | |
| *** salv-orlando has quit IRC | 08:56 | |
| *** maishsk has joined #openstack-infra | 08:56 | |
| *** _shaps_ has joined #openstack-infra | 08:57 | |
| openstackgerrit | Anusha Ramineni proposed openstack-infra/project-config: congress: Make new arch jobs voting https://review.openstack.org/327514 | 08:58 |
| *** maishsk has quit IRC | 09:00 | |
| *** maishsk has joined #openstack-infra | 09:01 | |
| *** mohankumar_ has joined #openstack-infra | 09:01 | |
| *** jtomasek_ has quit IRC | 09:03 | |
| *** e0ne has joined #openstack-infra | 09:04 | |
| *** yamahata has quit IRC | 09:08 | |
| *** savihou has quit IRC | 09:08 | |
| *** savihou has joined #openstack-infra | 09:09 | |
| *** thorst has joined #openstack-infra | 09:10 | |
| *** kzaitsev_mb has joined #openstack-infra | 09:12 | |
| *** maishsk has quit IRC | 09:12 | |
| *** sambetts|afk is now known as sambetts | 09:12 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Bring baremetal00.vanilla.ic.openstack.org online https://review.openstack.org/327314 | 09:13 |
| rcarrillocruz | pabelanger: ^ | 09:14 |
| *** grassass has quit IRC | 09:15 | |
| rcarrillocruz | added aditional meta to baremetal00 for better tracking of hardware | 09:15 |
| *** maishsk has joined #openstack-infra | 09:15 | |
| openstackgerrit | Niall Bunting proposed openstack-infra/project-config: Create new glance_store test for swift https://review.openstack.org/327526 | 09:15 |
| rcarrillocruz | i also added a route to the ilo network via mgmt network gw, and we can now access iLOs from baremetal00 | 09:16 |
| rcarrillocruz | \o/ | 09:16 |
| *** Ravikiran_K has joined #openstack-infra | 09:16 | |
| *** thorst has quit IRC | 09:16 | |
| openstackgerrit | Akihiro Motoki proposed openstack-infra/project-config: Add api-ref jobs for neutron https://review.openstack.org/314977 | 09:16 |
| openstackgerrit | Niall Bunting proposed openstack-infra/project-config: Create new functional job for glance_store-swift https://review.openstack.org/327526 | 09:16 |
| yolanda | rcarrillocruz, that's great | 09:16 |
| rcarrillocruz | yeah, no more vpn | 09:16 |
| rcarrillocruz | :D | 09:17 |
| yolanda | that was really a blocker | 09:17 |
| rcarrillocruz | and yeah, we better put as much info on hiera as possible | 09:17 |
| rcarrillocruz | to have it as the source of truth | 09:17 |
| rcarrillocruz | tired of collating spreadsheets *and* hiera | 09:17 |
| rcarrillocruz | Zara: helo helo | 09:18 |
| rcarrillocruz | how are you | 09:18 |
| rcarrillocruz | anteaya told me you need help on gerrit-storyboard integration? | 09:18 |
| Zara | hi, racarrillocruz! yeah, some of the logs for context are here: http://eavesdrop.openstack.org/irclogs/%23openstack-meeting/%23openstack-meeting.2016-06-08.log.html#t2016-06-08T15:10:06 | 09:20 |
| Zara | so there exists my proof-of-concept gerrit bot, and also an old half-finished gerrit plugin that zaro started. the bot is horrible code and demonstrates what we'd like to do. the plugin is probably the 'right' way to do things, but may be cobweb-y | 09:21 |
| aspiers | can anyone point me to some documentation on how data arrives at logs.openstack.org? is it basically just the Jenkins build artefacts? | 09:21 |
| Zara | rcarrillocruz: and as the team is small, we have a 'I review by default, and code when I get time', meaning that progress will be slow while I'm looking at it, if we want to fix other things in storyboard. | 09:23 |
| rcarrillocruz | heh | 09:23 |
| rcarrillocruz | so | 09:23 |
| rcarrillocruz | i can help on automating a gerrit deploy with storyboard | 09:23 |
| rcarrillocruz | whatever versions are needed | 09:23 |
| rcarrillocruz | if that helps | 09:23 |
| rcarrillocruz | ? | 09:23 |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/project-config: Run api-ref gate job for api sample files changes https://review.openstack.org/327529 | 09:24 |
| Zara | rcarrillocruz: I'm not sure what that means, so... maybe? :D in this case we wanted to automatically update storyboard tasks based on their status in gerrit. does that sound like a similar sort of a thing? am I way off? | 09:25 |
| rcarrillocruz | not sure where you at... if the issue is that you don't have a gerrit/storyboard to test your code | 09:26 |
| rcarrillocruz | or if you already sorted that out and you just need help on writing code for the integration | 09:26 |
| rcarrillocruz | i have a tool called infra-ansible | 09:26 |
| rcarrillocruz | that is in progress | 09:26 |
| rcarrillocruz | that automates deployment of a mini openstack-infra | 09:26 |
| rcarrillocruz | puppetmaster, gerrit, zuul, jenkins, etc | 09:27 |
| rcarrillocruz | so i could just add storyboard | 09:27 |
| rcarrillocruz | and you could have dev environments that you can bring up and destroy at will | 09:27 |
| Zara | rcarrillocruz: ahhh, right, so until now I've been testing with production gerrit (I don't post anything *to* it) and a local instance, but that would be useful, yes, since it means things are more public and not just on my vm. | 09:28 |
| *** dmk0202 has joined #openstack-infra | 09:28 | |
| Zara | which makes it much easier to say, post changes to storyboard-dev and say 'see it here' to people | 09:28 |
| rcarrillocruz | ++ | 09:28 |
| Zara | so that would be handy, yes :D | 09:28 |
| clarkb | you could also possibly use review-dev | 09:28 |
| rcarrillocruz | k, i'll start working on it | 09:28 |
| clarkb | thatay be good for talking to storyboard dev | 09:29 |
| clarkb | aspiers: the jobs copy the files to the server | 09:29 |
| *** Ravikiran_K has quit IRC | 09:29 | |
| *** jianghuaw has quit IRC | 09:30 | |
| *** hferenc has quit IRC | 09:30 | |
| * SotK would like to use review-dev if possible | 09:30 | |
| *** derekh has joined #openstack-infra | 09:30 | |
| *** tosky has joined #openstack-infra | 09:30 | |
| Zara | oh yeah, I forgot about review dev! though it seems like it's still useful to have storyboard on infra-ansible more generally? | 09:31 |
| *** shashank_hegde has quit IRC | 09:31 | |
| SotK | yeah, that would also be sensible I think | 09:31 |
| *** sarob has joined #openstack-infra | 09:32 | |
| *** sshnaidm|afk has joined #openstack-infra | 09:32 | |
| Zara | there's already an ansible role for storyboard, if that's helpful at all: https://github.com/palvarez89/ansible-role-storyboard | 09:32 |
| aspiers | clarkb: I'm interested in setting up the same mechanism for our local Jenkins; is there any doc or job code I can look at? | 09:32 |
| openstackgerrit | Alexander Evseev proposed openstack-infra/system-config: [WIP] Move most openstack-specified settings to Hiera https://review.openstack.org/327534 | 09:32 |
| clarkb | aspiers: yes all of our job config is in openstack-infra/project-config/jenkins/jobs | 09:33 |
| clarkb | aspiers: it uses the scp plugin to copy the files | 09:33 |
| openstackgerrit | Akihiro Motoki proposed openstack-infra/project-config: Add api-ref jobs for neutron https://review.openstack.org/314977 | 09:33 |
| rcarrillocruz | Zara: but there's no puppet-storyboard ? | 09:33 |
| rcarrillocruz | haven't looked yet | 09:33 |
| SotK | rcarrillocruz: there is puppet-storyboard too | 09:34 |
| *** kzaitsev_mb has quit IRC | 09:34 | |
| rcarrillocruz | cool thx | 09:35 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute035 metadata for better tracking on hiera https://review.openstack.org/327537 | 09:36 |
| clarkb | garyk1: I am not entirely aure and on a phone in an airport but it is going to check first will +1 then fail to gate because the jenkins vote hasnt changed. you can try reapproving it to see if that gets it in the gate | 09:36 |
| *** sarob has quit IRC | 09:36 | |
| aspiers | clarkb: thanks! | 09:36 |
| *** maishsk has quit IRC | 09:36 | |
| openstackgerrit | Sebastian Schuberth proposed openstack-infra/jenkins-job-builder: Catch OSError when renaming and try to remove the destination file first https://review.openstack.org/327539 | 09:37 |
| *** maishsk has joined #openstack-infra | 09:39 | |
| *** zeih has quit IRC | 09:44 | |
| *** maishsk has quit IRC | 09:44 | |
| *** maishsk has joined #openstack-infra | 09:46 | |
| *** amotoki has quit IRC | 09:46 | |
| *** salv-orlando has joined #openstack-infra | 09:48 | |
| openstackgerrit | Sebastian Schuberth proposed openstack-infra/jenkins-job-builder: Catch OSError when renaming and try to remove the destination file first https://review.openstack.org/327539 | 09:49 |
| *** tzn has quit IRC | 09:50 | |
| *** amitgandhinz has joined #openstack-infra | 09:51 | |
| *** amitgandhinz has quit IRC | 09:55 | |
| *** nadya has joined #openstack-infra | 09:59 | |
| openstackgerrit | Merged openstack-infra/puppet-jeepyb: Add missing packages to jeepyb install https://review.openstack.org/326885 | 10:01 |
| *** fawadkhaliq has quit IRC | 10:03 | |
| *** fawadkhaliq has joined #openstack-infra | 10:03 | |
| openstackgerrit | Rabi Mishra proposed openstack-infra/project-config: Disable convergence for jobs with legacy engine https://review.openstack.org/327560 | 10:05 |
| *** Guest93364 has quit IRC | 10:05 | |
| *** tzn has joined #openstack-infra | 10:10 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute007.uswest metadata for better tracking https://review.openstack.org/327567 | 10:10 |
| *** nadya has quit IRC | 10:11 | |
| *** thorst has joined #openstack-infra | 10:14 | |
| *** tzn has quit IRC | 10:14 | |
| *** ajafo has joined #openstack-infra | 10:14 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute035.uswest metadata for better tracking on hiera https://review.openstack.org/327537 | 10:17 |
| openstackgerrit | Niall Bunting proposed openstack-infra/project-config: Create new functional job for glance_store-swift https://review.openstack.org/327526 | 10:19 |
| *** piet has joined #openstack-infra | 10:19 | |
| *** _amrith_ is now known as amrith | 10:20 | |
| *** thorst has quit IRC | 10:21 | |
| *** Hal has joined #openstack-infra | 10:22 | |
| *** Hal is now known as Guest61670 | 10:22 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute009.uswest metadata for better tracking https://review.openstack.org/327575 | 10:25 |
| *** rossella_s has quit IRC | 10:27 | |
| *** gema has quit IRC | 10:28 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute032.uswest metadata for better tracking https://review.openstack.org/327576 | 10:30 |
| *** Guest94922 has joined #openstack-infra | 10:31 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/jenkins-job-builder: Add missing parameter ZUUL_VOTING to zuul style jobs https://review.openstack.org/327577 | 10:33 |
| *** zeih has joined #openstack-infra | 10:35 | |
| *** romcheg has joined #openstack-infra | 10:36 | |
| *** romcheg has left #openstack-infra | 10:36 | |
| *** Guest94922 has quit IRC | 10:36 | |
| *** gema_ has joined #openstack-infra | 10:37 | |
| *** kmartin has quit IRC | 10:38 | |
| *** nadya has joined #openstack-infra | 10:39 | |
| *** zeih has quit IRC | 10:41 | |
| *** salv-orlando has quit IRC | 10:41 | |
| *** zeih has joined #openstack-infra | 10:42 | |
| *** netsin has quit IRC | 10:43 | |
| *** netsin has joined #openstack-infra | 10:43 | |
| *** piet has quit IRC | 10:44 | |
| *** zeih has quit IRC | 10:47 | |
| *** zeih has joined #openstack-infra | 10:47 | |
| *** nadya has quit IRC | 10:49 | |
| *** salv-orlando has joined #openstack-infra | 10:49 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:51 | |
| *** amitgandhinz has joined #openstack-infra | 10:52 | |
| *** dtantsur|lunch is now known as dtantsur | 10:54 | |
| *** sree has joined #openstack-infra | 10:55 | |
| *** sdague has joined #openstack-infra | 10:55 | |
| *** piet has joined #openstack-infra | 10:55 | |
| *** amitgandhinz has quit IRC | 10:56 | |
| garyk1 | anyone know why https://review.openstack.org/324374 is not getting into the gate? | 10:56 |
| garyk1 | AJaeger: ^ | 10:57 |
| AJaeger | garyk1: you need to toggle the +A. recheck will not help. Zuul does not see a change in status. | 10:57 |
| AJaeger | so, ask somebody else to +A or ask to remove +A and add it again. | 10:57 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute016.uswest metadata for better tracking https://review.openstack.org/327593 | 10:58 |
| AJaeger | garyk1: if you core on the repo, just add a +A | 10:58 |
| AJaeger | garyk1: this is the usual process if Zuul does not get an event, you need to add the event again | 10:58 |
| garyk1 | AJaeger: ok, thanks! | 10:59 |
| ajafo | hi I search for little help to fully understand zuul, I've few repos in configuration with zuul/jenkins/gerrit, I've gate jobs(different for each repo) where I do some git checkout based on ZUUL_REF and in this case everythings works, but I want to have some cross repository dependencies the question is how should I achieve it? should I create some shared gate job for this repos and in this job should I use zuul cloner to get all repos? does then zuul will | 11:00 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add baremetal01 metadata for better tracking https://review.openstack.org/327597 | 11:01 |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: DNM: experimenting with local.conf https://review.openstack.org/326585 | 11:01 |
| *** aysyd has joined #openstack-infra | 11:02 | |
| *** salv-orlando has quit IRC | 11:03 | |
| *** piet has quit IRC | 11:04 | |
| *** tpsilva has joined #openstack-infra | 11:04 | |
| *** javeriak has quit IRC | 11:14 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute030.uswest metadata for better tracking https://review.openstack.org/327603 | 11:17 |
| *** zeih has quit IRC | 11:19 | |
| *** thorst has joined #openstack-infra | 11:19 | |
| *** zeih has joined #openstack-infra | 11:19 | |
| *** bhavik has quit IRC | 11:20 | |
| *** fawadk has joined #openstack-infra | 11:20 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute024.uswest metadata for better tracking https://review.openstack.org/327605 | 11:21 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: WIP: Rework timeline events into a timeline https://review.openstack.org/327231 | 11:21 |
| *** fawadkhaliq has quit IRC | 11:21 | |
| *** fawadkhaliq has joined #openstack-infra | 11:21 | |
| *** vgridnev has joined #openstack-infra | 11:21 | |
| rcarrillocruz | k , enough spreadsheets and inventory consolidation for now | 11:22 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job https://review.openstack.org/229789 | 11:23 |
| *** flip214 is now known as flip217 | 11:25 | |
| openstackgerrit | Merged openstack-infra/project-config: Run api-ref gate job for api sample files changes https://review.openstack.org/327529 | 11:25 |
| *** fawadk has quit IRC | 11:25 | |
| *** Na3iL has quit IRC | 11:26 | |
| *** thorst has quit IRC | 11:26 | |
| *** Na3iL has joined #openstack-infra | 11:27 | |
| *** javeriak has joined #openstack-infra | 11:27 | |
| *** ldnunes has joined #openstack-infra | 11:30 | |
| *** lucasagomes is now known as lucas-hungry | 11:32 | |
| openstackgerrit | Martin André proposed openstack-infra/project-config: Add pypi-jobs to tripleo-validations https://review.openstack.org/327611 | 11:33 |
| *** salv-orlando has joined #openstack-infra | 11:35 | |
| *** yamamoto_ has quit IRC | 11:37 | |
| *** vgridnev_ has joined #openstack-infra | 11:42 | |
| *** rhallisey has joined #openstack-infra | 11:43 | |
| *** vgridnev has quit IRC | 11:44 | |
| *** gordc has joined #openstack-infra | 11:45 | |
| *** baoli has joined #openstack-infra | 11:45 | |
| *** baoli_ has joined #openstack-infra | 11:46 | |
| jroll | morning infra, I have two small patches: https://review.openstack.org/#/c/316662/ is a one-liner and the last patch to get grenade passing on ironic, and https://review.openstack.org/#/c/319336/ moves our grenade job to non-voting in the check queue, would love a couple reviews here if someone has time :) | 11:48 |
| *** baoli has quit IRC | 11:49 | |
| *** thorst has joined #openstack-infra | 11:52 | |
| *** amitgandhinz has joined #openstack-infra | 11:52 | |
| *** thorst_ has joined #openstack-infra | 11:53 | |
| *** sdake has joined #openstack-infra | 11:56 | |
| *** thorst has quit IRC | 11:57 | |
| *** amitgandhinz has quit IRC | 11:57 | |
| openstackgerrit | Michal Dulko proposed openstack-infra/devstack-gate: WIP, DNM: Enabling Cinder multinode testing https://review.openstack.org/327619 | 11:58 |
| *** sarob has joined #openstack-infra | 11:59 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Set resolv.conf on nodes before bootstrap https://review.openstack.org/326959 | 11:59 |
| *** scottda has joined #openstack-infra | 12:01 | |
| *** LaT0rtue has joined #openstack-infra | 12:01 | |
| lifeless | ttx: around ? | 12:01 |
| electrofelix | zxiiro zaro waynr: or anyone else with Core on JJB would you mind reviewing https://review.openstack.org/327539 for a bug that got added with the 1.6.0 release. | 12:02 |
| *** psilvad has joined #openstack-infra | 12:02 | |
| *** sarob has quit IRC | 12:03 | |
| *** psilvad has quit IRC | 12:04 | |
| *** markvoelker has joined #openstack-infra | 12:04 | |
| *** psilvad has joined #openstack-infra | 12:04 | |
| *** bhavik1 has joined #openstack-infra | 12:04 | |
| *** javeriak has quit IRC | 12:05 | |
| *** salv-orlando has quit IRC | 12:06 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Start fleshing out ansible-launcher tests https://review.openstack.org/317482 | 12:07 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Clean up results returned from ansible-launcher https://review.openstack.org/317481 | 12:07 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add jjb and zmq to requirements https://review.openstack.org/316752 | 12:07 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Prepare files for testing zuulv25 https://review.openstack.org/316753 | 12:07 |
| *** Na3iL has quit IRC | 12:08 | |
| *** trown|outtypewww is now known as trown | 12:08 | |
| openstackgerrit | Merged openstack-infra/zuul: Python 3 Fix: Absolute Imports https://review.openstack.org/323018 | 12:08 |
| *** garyk1 has quit IRC | 12:09 | |
| *** rossella_s has joined #openstack-infra | 12:10 | |
| *** psilvad has quit IRC | 12:10 | |
| *** psilvad has joined #openstack-infra | 12:12 | |
| *** Daisy has joined #openstack-infra | 12:12 | |
| *** fawadk has joined #openstack-infra | 12:14 | |
| *** kengo_sakai has joined #openstack-infra | 12:14 | |
| *** lucas-hungry is now known as lucasagomes | 12:14 | |
| *** sshnaidm|afk is now known as sshnaidm | 12:16 | |
| *** Daisy has quit IRC | 12:17 | |
| *** zz_dimtruck is now known as dimtruck | 12:17 | |
| *** fawadkhaliq has quit IRC | 12:18 | |
| *** psilvad has quit IRC | 12:19 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Set resolv.conf on nodes before bootstrap https://review.openstack.org/326959 | 12:19 |
| *** Goneri has joined #openstack-infra | 12:19 | |
| *** rossella_s has quit IRC | 12:19 | |
| *** kengo_sakai has quit IRC | 12:20 | |
| *** liusheng has quit IRC | 12:20 | |
| *** psilvad has joined #openstack-infra | 12:20 | |
| *** liusheng has joined #openstack-infra | 12:20 | |
| *** hashar has joined #openstack-infra | 12:21 | |
| *** rossella_s has joined #openstack-infra | 12:22 | |
| dtantsur | folks, please review this simple patch: https://review.openstack.org/#/c/316662/ this is the last step for ironic grenade job to pass finally | 12:22 |
| *** yamamoto has joined #openstack-infra | 12:23 | |
| *** dimtruck is now known as zz_dimtruck | 12:23 | |
| *** markusry has quit IRC | 12:25 | |
| openstackgerrit | Merged openstack-infra/infra-ansible: Reuse the puppet-openstackci zuul scheduler class https://review.openstack.org/327129 | 12:25 |
| *** tlian has joined #openstack-infra | 12:26 | |
| *** sree has quit IRC | 12:28 | |
| *** mixos has joined #openstack-infra | 12:28 | |
| *** sree has joined #openstack-infra | 12:30 | |
| *** fawadk has quit IRC | 12:31 | |
| *** Na3iL has joined #openstack-infra | 12:31 | |
| *** nwkarsten has joined #openstack-infra | 12:31 | |
| *** tlian2 has joined #openstack-infra | 12:36 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Provide more realistic resources.yml.sample https://review.openstack.org/327634 | 12:36 |
| *** markusry has joined #openstack-infra | 12:36 | |
| *** thorst_ is now known as thorst | 12:37 | |
| *** amoralej is now known as amoralej|lunch | 12:37 | |
| *** tlian has quit IRC | 12:39 | |
| *** markusry has quit IRC | 12:40 | |
| *** markusry has joined #openstack-infra | 12:40 | |
| *** rlandy has joined #openstack-infra | 12:40 | |
| *** amitgandhinz has joined #openstack-infra | 12:43 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Add a cancel button to the Add Tag text box https://review.openstack.org/325904 | 12:43 |
| *** darvon has quit IRC | 12:43 | |
| *** vgridnev_ has quit IRC | 12:45 | |
| *** tzn has joined #openstack-infra | 12:47 | |
| *** sdake has quit IRC | 12:48 | |
| *** ccamacho is now known as ccamacho|lunch | 12:49 | |
| *** gema_ is now known as gema | 12:49 | |
| *** salv-orlando has joined #openstack-infra | 12:49 | |
| *** amitgandhinz has quit IRC | 12:50 | |
| *** amitgandhinz has joined #openstack-infra | 12:51 | |
| *** julim has joined #openstack-infra | 12:51 | |
| *** darvon has joined #openstack-infra | 12:51 | |
| openstackgerrit | Beth Elwell proposed openstack-infra/storyboard-webclient: New Favicon https://review.openstack.org/327639 | 12:52 |
| *** akshai has joined #openstack-infra | 12:53 | |
| *** rkukura_ has joined #openstack-infra | 12:55 | |
| *** links has quit IRC | 12:57 | |
| *** rkukura has quit IRC | 12:57 | |
| *** esikachev has quit IRC | 12:58 | |
| *** rkukura_ is now known as rkukura | 12:59 | |
| sdague | did zuul lock up again? | 12:59 |
| *** kzaitsev_mb has quit IRC | 13:00 | |
| *** esikachev has joined #openstack-infra | 13:00 | |
| *** _ari_ has joined #openstack-infra | 13:03 | |
| *** salv-orlando has quit IRC | 13:03 | |
| *** salv-orlando has joined #openstack-infra | 13:04 | |
| AJaeger | sdague: seems that it's still counting events | 13:04 |
| sdague | it seems like the merging process really locks zuul more now than it used to. | 13:04 |
| AJaeger | right now 1262 jobs in merge-check queue... | 13:05 |
| *** kgiusti has joined #openstack-infra | 13:05 | |
| zxiiro | electrofelix: looks good to me | 13:06 |
| *** deadnull_ has joined #openstack-infra | 13:06 | |
| zxiiro | electrofelix: should be safe to release 1.6.1 since the only other patch merged was a bug fix | 13:07 |
| *** LaT0rtue has quit IRC | 13:07 | |
| *** claudiub has joined #openstack-infra | 13:09 | |
| *** mdrabe has joined #openstack-infra | 13:09 | |
| *** jcoufal has joined #openstack-infra | 13:09 | |
| *** mixos has quit IRC | 13:10 | |
| *** weshay is now known as weshay_brb | 13:10 | |
| *** ayoung has joined #openstack-infra | 13:11 | |
| *** esberglu has quit IRC | 13:11 | |
| *** ccamacho|lunch is now known as ccamacho | 13:12 | |
| *** amrith is now known as _amrith_ | 13:12 | |
| *** esberglu has joined #openstack-infra | 13:12 | |
| *** Goneri has quit IRC | 13:14 | |
| *** salv-orlando has quit IRC | 13:15 | |
| *** kzaitsev_mb has joined #openstack-infra | 13:16 | |
| *** erlon has joined #openstack-infra | 13:17 | |
| *** links has joined #openstack-infra | 13:17 | |
| *** weshay_brb has quit IRC | 13:18 | |
| *** matt-borland has joined #openstack-infra | 13:20 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Catch OSError when renaming and try to remove the destination file first https://review.openstack.org/327539 | 13:20 |
| *** rossella_s has quit IRC | 13:22 | |
| *** edmondsw has joined #openstack-infra | 13:22 | |
| *** links has quit IRC | 13:23 | |
| *** amoralej|lunch is now known as amoralej | 13:23 | |
| pabelanger | sdague: We think were pushing gearman past its limit now. jeblair has patch up (untested) to see if it will address that: https://review.openstack.org/#/c/327422/ | 13:23 |
| pabelanger | think we are going to try it out today | 13:23 |
| *** gouthamr has joined #openstack-infra | 13:24 | |
| sdague | pabelanger: gotcha | 13:24 |
| openstackgerrit | Merged openstack-infra/infra-ansible: Set resolv.conf on nodes before bootstrap https://review.openstack.org/326959 | 13:25 |
| openstackgerrit | Merged openstack-infra/infra-ansible: Provide more realistic resources.yml.sample https://review.openstack.org/327634 | 13:25 |
| *** yujunz has joined #openstack-infra | 13:25 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Fix package name on RedHat https://review.openstack.org/327659 | 13:27 |
| *** cody-somerville has joined #openstack-infra | 13:27 | |
| yolanda | pabelanger, can you review that? quick fix, package name was still wrong ^ | 13:28 |
| *** dprince has joined #openstack-infra | 13:29 | |
| *** mriedem has joined #openstack-infra | 13:29 | |
| pabelanger | yolanda: +A | 13:30 |
| yolanda | thx | 13:30 |
| *** cody-somerville has quit IRC | 13:35 | |
| *** mixos has joined #openstack-infra | 13:36 | |
| *** Goneri has joined #openstack-infra | 13:36 | |
| *** nadya has joined #openstack-infra | 13:37 | |
| *** esikachev has quit IRC | 13:37 | |
| *** abregman has quit IRC | 13:38 | |
| *** abregman has joined #openstack-infra | 13:39 | |
| *** banix has joined #openstack-infra | 13:39 | |
| *** fergal has joined #openstack-infra | 13:39 | |
| fergal | hi guys, is this the place to talk about jenkins job builder? | 13:39 |
| *** dizquierdo has quit IRC | 13:39 | |
| *** sdake has joined #openstack-infra | 13:39 | |
| jroll | fergal: go ahead and ask your question | 13:39 |
| *** rossella_s has joined #openstack-infra | 13:40 | |
| *** mixos has quit IRC | 13:40 | |
| *** esikachev has joined #openstack-infra | 13:42 | |
| fergal | sooo i guess, i’m just curious if there are any graphical front ends for viewing what components make up a jenkins job? so, in my workplace, we use jjb for hundreds of jenkins jobs. maintaining the jobs manually, and trying to match up what projects/job groups/job templates are used to generate jobs, can be a bit tough. i’ve been thinking about trying to write my own graphical client, but wanted to do some research before i tried, because i think | 13:42 |
| fergal | would be a pretty big undertaking | 13:42 |
| *** jaosorior has quit IRC | 13:42 | |
| *** eharney has joined #openstack-infra | 13:43 | |
| *** rbrndt has joined #openstack-infra | 13:43 | |
| fergal | so ideally, the gui would list jobs and list out the attributes that make up that job, and then, every attribute would list the file from which its value was derived. so like path/to/yaml/xyz.yaml or another/path/to/yaml/123.yaml | 13:44 |
| *** itisha has joined #openstack-infra | 13:44 | |
| *** nadya has quit IRC | 13:44 | |
| fergal | and beyond that, it would let you see what attributes are common among jobs, and let you consolidate common configurations, higher up the chain | 13:45 |
| *** vhosakot has joined #openstack-infra | 13:45 | |
| *** hdaniel has quit IRC | 13:45 | |
| *** fawadkhaliq has joined #openstack-infra | 13:46 | |
| rcarrillocruz | folks, what is the exact version of puppetdb that is installed in infra | 13:47 |
| rcarrillocruz | i see in modules.env 3.0.1, but per https://storyboard.openstack.org/#!/story/2000634 it seems we may not? | 13:47 |
| rcarrillocruz | or wait | 13:48 |
| rcarrillocruz | that's the version of puppet-puppetdb | 13:49 |
| * rcarrillocruz goes check site.pp | 13:49 | |
| rcarrillocruz | 2.3.8 | 13:50 |
| rcarrillocruz | ++ | 13:50 |
| *** piet has joined #openstack-infra | 13:50 | |
| *** abregman has quit IRC | 13:51 | |
| *** ihrachys has quit IRC | 13:52 | |
| *** esikachev has quit IRC | 13:53 | |
| *** weshay has joined #openstack-infra | 13:54 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Set nodepool.manager.MANAGER.nodes.STATE to zero https://review.openstack.org/327681 | 13:55 |
| openstackgerrit | Thomas Goirand proposed openstack-dev/pbr: Add --sysconfdir option to install command https://review.openstack.org/274077 | 13:55 |
| electrofelix | zxiiro: my thoughts exactly | 13:56 |
| pabelanger | jeblair: notmorgan: ^ I'm not keen on that approach, since it seems ineffective to me for tracking out manager_names. We need to initialize all key states to 0 to start however. If you don't mind reviewing and possible making a suggestion if you don't like | 13:57 |
| AJaeger | pabelanger: could you put https://review.openstack.org/327021 and https://review.openstack.org/324548 on your review queue for later, please? | 13:57 |
| pabelanger | AJaeger: sure, will check now | 13:57 |
| zxiiro | electrofelix: cool, well it's merged so we're good anytime | 13:57 |
| AJaeger | thanks, pabelanger | 13:58 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: WIP: Remove pagination for timeline events https://review.openstack.org/327682 | 14:00 |
| *** ddieterly has joined #openstack-infra | 14:01 | |
| *** esikachev has joined #openstack-infra | 14:01 | |
| *** PollyZ has joined #openstack-infra | 14:03 | |
| *** Zara has quit IRC | 14:04 | |
| *** flepied1 has quit IRC | 14:04 | |
| *** zz_dimtruck is now known as dimtruck | 14:05 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: A newer version of virtualenv is already installed https://review.openstack.org/325330 | 14:06 |
| openstackgerrit | Merged openstack-infra/project-config: translations: Handle early exit https://review.openstack.org/327021 | 14:06 |
| *** _amrith_ is now known as amrith | 14:06 | |
| *** weshay is now known as weshay_mtg | 14:06 | |
| *** rodrigods has quit IRC | 14:06 | |
| *** rodrigods has joined #openstack-infra | 14:07 | |
| *** sree has quit IRC | 14:08 | |
| *** sree has joined #openstack-infra | 14:08 | |
| *** fawadkhaliq has quit IRC | 14:08 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Enable StoryBoard for the ARA projects https://review.openstack.org/327693 | 14:09 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Enable StoryBoard for the ARA project https://review.openstack.org/327693 | 14:09 |
| *** Zara has joined #openstack-infra | 14:10 | |
| *** sree has quit IRC | 14:12 | |
| mordred | fergal: we do not have any such visualization - I can imagine you are not the only person who would find such a thing useful though | 14:14 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Normalize maillist indentation https://review.openstack.org/327698 | 14:15 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Private embargoed vulnerability announcement list https://review.openstack.org/327699 | 14:15 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Move ARA integration jobs to use ./run_tests.sh instead https://review.openstack.org/327700 | 14:15 |
| fergal | mordred: that’s what i thought, no worries! i’ll have a go at putting something together and see how i get on. would i be able to query jjb and have it tell me what parameters come from where? or would it be a case of parsing it manually? what files in particular should i be focusing on for the info i need? https://github.com/openstack-infra/jenkins-job-builder/tree/master/jenkins_jobs parser.py? builder.py? | 14:17 |
| openstackgerrit | Kirill Zaitsev proposed openstack-infra/project-config: Clean-up kilo and juno configs for murano jobs https://review.openstack.org/327702 | 14:17 |
| *** bhavik1 has quit IRC | 14:18 | |
| *** flepied has joined #openstack-infra | 14:21 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Require packages before installing jeepyb https://review.openstack.org/327707 | 14:23 |
| *** kushal has joined #openstack-infra | 14:24 | |
| yolanda | pabelanger, i sent a fix for windmill and deps, was failing with zuul install: https://review.openstack.org/324317 | 14:25 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Simplify translation logic https://review.openstack.org/324548 | 14:25 |
| *** hongbin has joined #openstack-infra | 14:26 | |
| AJaeger | yolanda, pabelanger: Suddenly bashate complaint about my change, here's it again ^. Please review again whenever conveninent | 14:26 |
| yolanda | sure | 14:27 |
| pabelanger | yolanda: nice. I need to also update requirements.txt to use zuul from git, not pip to address PBR issue | 14:27 |
| *** sarob has joined #openstack-infra | 14:27 | |
| *** tongli has joined #openstack-infra | 14:28 | |
| *** sarob has quit IRC | 14:28 | |
| *** fergal_ has joined #openstack-infra | 14:28 | |
| *** fergal has quit IRC | 14:28 | |
| *** fergal_ is now known as fergal | 14:28 | |
| *** astupnikov-tmp has joined #openstack-infra | 14:32 | |
| *** fergal has quit IRC | 14:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Revert "Migrate tripleo to centos-7"" https://review.openstack.org/327425 | 14:32 |
| *** oanson has quit IRC | 14:32 | |
| astupnikov-tmp | Hi there infra team. No gate jobs were started for https://review.openstack.org/323359 after WF+1. Can you help me with it? | 14:33 |
| pabelanger | dprince: derekh: ^ | 14:33 |
| pabelanger | FYI | 14:33 |
| AJaeger | astupnikov-tmp: let'S wait until Zuul's queue is empty - see status.openstack.org/zuul | 14:34 |
| dprince | pabelanger: ack | 14:34 |
| dprince | pabelanger: thanks | 14:34 |
| pabelanger | astupnikov-tmp: recheck your patch. I suspect it was because your Depends-On patch had not merged yet | 14:35 |
| fungi | astupnikov-tmp: remove and re-add your workflow +1. that change depended on a stable backport which didn't merge until after you approved this | 14:35 |
| pabelanger | fungi: solution is better :) | 14:35 |
| fungi | recheck won't help in that situation | 14:35 |
| astupnikov-tmp | oh, my bad :( | 14:35 |
| astupnikov-tmp | thank you guys, I will ask to re-add WF+1 | 14:36 |
| AJaeger | fungi: thanks for figuring that out | 14:36 |
| *** PollyZ has quit IRC | 14:37 | |
| openstackgerrit | Lukas Bednar proposed openstack-infra/jenkins-job-builder: Builders: Add ansible-playbook builder https://review.openstack.org/322243 | 14:38 |
| fungi | since this change and its dependencies don't share any jobs in common, zuul can't enqueue the depending change into the gate until the dependency is merged | 14:38 |
| AJaeger | mordred: when will you take https://review.openstack.org/166547 again? That's the oldest change we have in project-config right now... | 14:38 |
| AJaeger | pabelanger: what do you think about https://review.openstack.org/315033 ? Should we add grafana boards for each project? | 14:39 |
| *** markusry has quit IRC | 14:39 | |
| fungi | and due to a quirk of how gerrit doesn't emit a new vote when a verify +1 gets "replaced" by the same, zuul won't get triggered to enqueue it into the gate after a successful recheck either | 14:39 |
| *** markusry has joined #openstack-infra | 14:39 | |
| *** Guest61670 has quit IRC | 14:39 | |
| pabelanger | Jun 9 14:38:30 logstash-worker05 puppet-user[9969]: (/Stage[main]/Log_processor/Package[statsd]/ensure) change from 3.2.1 to latest failed: Could not get latest version: Timeout while contacting pypi.python.org: Timeout::Error | 14:39 |
| pabelanger | clarkb: ^ I wonder if we should be using our pypi mirrors over pypi.python.org for puppet | 14:40 |
| *** vgridnev has joined #openstack-infra | 14:40 | |
| *** gema has quit IRC | 14:40 | |
| AJaeger | jroll: is https://review.openstack.org/314966 needed? | 14:40 |
| *** gema has joined #openstack-infra | 14:41 | |
| pabelanger | AJaeger: I'm fine with projects adding them. We might want to call the dashboard openstack-midonet.yaml however | 14:41 |
| *** pcaruana has quit IRC | 14:41 | |
| pabelanger | AJaeger: or even a sub directory call openstack | 14:41 |
| fungi | pabelanger: is that happening continually, or was it just once? | 14:41 |
| *** piet has quit IRC | 14:42 | |
| pabelanger | fungi: I just noticed it this pass | 14:42 |
| pabelanger | Will monitor for the next few passes | 14:42 |
| jroll | AJaeger: not sure, you might ask vsaienko | 14:42 |
| *** dstufft is now known as nedbat2 | 14:42 | |
| *** PollyZ has joined #openstack-infra | 14:42 | |
| *** nedbat2 is now known as dstufft | 14:43 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: make all sahara-dashboard jobs voting https://review.openstack.org/326474 | 14:43 |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: setup release notes job for sahara-dashboard https://review.openstack.org/327729 | 14:43 |
| fungi | pabelanger: i'm on the fence about having our servers consume our mirrors of things. feels dangerously like we could deadlock ourselves updating things, but i don't have any concrete scenarios to justify that paranoia | 14:43 |
| *** stajkowski has quit IRC | 14:43 | |
| pabelanger | fungi: I can see that | 14:44 |
| vsaienko | AJaeger, jroll the Idea of that job was to launch smoke tests on Ironic changes without upgrade. It will allow to exclude errors in smoke tests itself. | 14:44 |
| vsaienko | since at the moment there is no job that launch smoke tests on ironic, only with upgrades in grenade. | 14:45 |
| AJaeger | pabelanger: please comment what you think is best, I have no strong opoinon | 14:45 |
| AJaeger | vsaienko: to me that change looks abandoned - no action for a month, does not pass test suite... | 14:45 |
| pabelanger | AJaeger: just did :) | 14:45 |
| *** jlanoux has quit IRC | 14:45 | |
| AJaeger | thanks! | 14:45 |
| *** banix has quit IRC | 14:45 | |
| jroll | vsaienko: hrm, not sure if we need that | 14:46 |
| jeblair | grafana project dashboards could be automatically generated | 14:46 |
| fungi | electrofelix: zxiiro: zaro: waynr: do i need to tag jjb 1.6.1 now that 327539 has merged? | 14:46 |
| fungi | i know the regression is windows-only, but a regression is a regression | 14:46 |
| fungi | happy to whip up a brown-bag release for that fix asap | 14:47 |
| electrofelix | fungi: df1ab0f01a695aeedea8a8c050636f80b61b1157 would be the correct commit, zxiiro can you confirm that's the one you see as well? | 14:48 |
| pabelanger | jeblair: that is an interesting idea. A based set of panels to default too. Or even a templated dashboard where people can select the project they care about | 14:48 |
| zxiiro | fungi: electrofelix : yep that's it, we're good to go! | 14:49 |
| fungi | electrofelix: yep, that's the current master branch tip | 14:49 |
| *** banix has joined #openstack-infra | 14:49 | |
| fungi | tag pushed | 14:50 |
| jeblair | pabelanger: ah, yeah, template may work if they all have the same graphs, but if not (like graph-per-job or something) then you could generate it by parsing the yaml | 14:50 |
| fungi | should show up on pypi shortly, release gods willing | 14:50 |
| electrofelix | fungi: thanks! | 14:50 |
| pabelanger | jeblair: right. A fun friday afternoon project to hack on | 14:51 |
| jeblair | friday is coming up! | 14:51 |
| pabelanger | indeed | 14:52 |
| jeblair | (in only 9 hours!) | 14:52 |
| *** vhosakot has quit IRC | 14:52 | |
| *** EricGonczer_ has joined #openstack-infra | 14:52 | |
| *** pradk has joined #openstack-infra | 14:52 | |
| *** mhickey has joined #openstack-infra | 14:54 | |
| vgridnev | is zuul picking up new changes for testing? | 14:54 |
| vgridnev | I've approved a change 10 mins ago, still seems to be not in queue | 14:55 |
| vgridnev | https://review.openstack.org/#/c/327080/ | 14:55 |
| mordred | jeblair: you're a friday | 14:56 |
| *** akshai has quit IRC | 14:56 | |
| pabelanger | vgridnev: yes, where hitting some performance issues this week with zuul.o.o, you can see the queue length at the top of status.o.o/zuul, it grows when zuul is doing calculations | 14:56 |
| AJaeger | vgridnev: "Queue lengths: 7524 events, 1420 results. " | 14:56 |
| AJaeger | As long as that is not to zero, zuul is processing... | 14:56 |
| mordred | AJaeger: oh! I can totally make that a thing | 14:57 |
| *** ddieterly is now known as ddieterly[away] | 14:57 | |
| *** thiagop has joined #openstack-infra | 14:57 | |
| mordred | rcarrillocruz: ^^ have you seen this very old patch - and if not, would you look. and then maybe we should chat about how/if it interacts with cloud-launcher | 14:57 |
| AJaeger | go for it mordred ! ;) | 14:57 |
| mordred | rcarrillocruz: it's possible that there is no needed interaction | 14:58 |
| rcarrillocruz | checking | 14:58 |
| AJaeger | team, ara project wants to use storyboard - see https://review.openstack.org/#/c/327693/ - is that ok? | 14:58 |
| thiagop | Folks, how often do you have to delete bad images generated by nodepool? I'm asking because this is the second image I delete in our third party ci in three weeks. | 14:58 |
| mordred | rcarrillocruz: or, rather, we could have this role just deal with building rather than building-and-uploading | 14:58 |
| thiagop | Seems not to be very reliable | 14:58 |
| *** vhosakot has joined #openstack-infra | 14:58 | |
| mordred | rcarrillocruz: and a playbook could focus on using it for building and launcher for uploading | 14:58 |
| zaro | fungi, pabelanger not sure if you are aware but that event problem is fixed in next version of gerrit | 14:58 |
| vgridnev | AJaeger, pabelanger, ok, thanks for explanations | 14:58 |
| rcarrillocruz | mordred: which patch? | 14:59 |
| rcarrillocruz | the sahara one from vgridnev ? | 14:59 |
| jeblair | thiagop: almost never, unless we change something and it turns out we broke something with the change. | 14:59 |
| rcarrillocruz | sorry, lacking context | 14:59 |
| thiagop | jeblair: but the images on infra are generated every day? | 14:59 |
| jeblair | thiagop: yes | 14:59 |
| *** maishsk has quit IRC | 14:59 | |
| jeblair | thiagop: what's going wrong with yours? | 15:00 |
| *** fawadkhaliq has joined #openstack-infra | 15:00 | |
| vgridnev | rcarrillocruz, sahara-dashboard | 15:00 |
| thiagop | jeblair: so, what the hell am I doing wrong? lol | 15:00 |
| AJaeger | rcarrillocruz: https://review.openstack.org/166547 | 15:00 |
| *** mtanino has joined #openstack-infra | 15:00 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-infracloud: Copy hpuswest to vanilla https://review.openstack.org/327332 | 15:00 |
| thiagop | jeblair: forgot to get the logs, but I'll get it next time. It was some kind of kernel error mounting the fs | 15:00 |
| mordred | rcarrillocruz: sorry, what AJaeger said | 15:01 |
| *** javeriak has joined #openstack-infra | 15:01 | |
| * thiagop fixes things too fast | 15:01 | |
| *** ddieterly[away] is now known as ddieterly | 15:01 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Bring baremetal00.vanilla.ic.openstack.org online https://review.openstack.org/327314 | 15:01 |
| *** akshai has joined #openstack-infra | 15:01 | |
| jeblair | thiagop: if there's a problem with dib or a fundamental dib element, i'm sure other folks would be interested, so you could probably post that to the openstack-infra and openstack-dev lists | 15:02 |
| rcarrillocruz | interesting, let me check ansible-build-image | 15:02 |
| pabelanger | zaro: which event is that? | 15:02 |
| thiagop | jeblair: sure, but I don't think so since my image from yesterday is OK... | 15:02 |
| thiagop | jeblair: I swear I'll get the logs next time, my bad | 15:02 |
| fungi | zaro: in 2.12? | 15:03 |
| *** dizquierdo has joined #openstack-infra | 15:03 | |
| pabelanger | zaro: ref-replicated ? | 15:03 |
| rcarrillocruz | mordred: yep, i see aht you're saying. I think it would be better to just build, since the launcher can upload images just fine | 15:03 |
| *** Hal has joined #openstack-infra | 15:04 | |
| rcarrillocruz | so wiring a playbook that calls your rule and then the launcher would do it | 15:04 |
| *** javeriak_ has joined #openstack-infra | 15:04 | |
| fungi | pabelanger: comment-added including a vote when the comment replaces a vote with the same vote | 15:04 |
| *** Hal is now known as Guest77604 | 15:04 | |
| mordred | rcarrillocruz: ++ | 15:04 |
| mordred | rcarrillocruz: I'll rework it | 15:04 |
| *** sorantis has quit IRC | 15:04 | |
| pabelanger | fungi: Oh, didn't know that | 15:04 |
| fungi | pabelanger: yeah, right now (in gerrit 2.11) if zuul updates an existing verify +1 with another verify +1, then the comment-added event doesn't indicate there was a vote included with that comment | 15:05 |
| *** rpodolyaka is now known as rpodolyaka_pto | 15:05 | |
| rcarrillocruz | mordred: that'd be awesome btw... i see this thing: ansible-build-image building images just like infra + infra ansible (which uses cloud launcher for openstaack resources creation) | 15:06 |
| rcarrillocruz | that would give people a dev box to test things as in the gate | 15:06 |
| rcarrillocruz | and we could even do cool things like testing stuff on our own images prior to publishing etc | 15:06 |
| *** javeriak has quit IRC | 15:06 | |
| rcarrillocruz | so ++ | 15:06 |
| pabelanger | fungi: good to know | 15:06 |
| fungi | pabelanger: and so things like enqueuing into the gate pipeline that trigger off a verify +1 in a comment won't trigger if there was previously a verify +1 present on the patchset (e.g., after a recheck) | 15:07 |
| pabelanger | fungi: was just looking my open reviews, do you mind looking at https://review.openstack.org/#/c/303047/. Its removing the thick_slave.pp from wheel mirrors, which is needed for xenail too | 15:07 |
| *** akshai_ has joined #openstack-infra | 15:07 | |
| fungi | pabelanger: that's why i was saying that a recheck "wouldn't help" in the situation described earlier | 15:08 |
| pabelanger | also 1 step closing to removing some jenkins bits from puppet | 15:08 |
| *** andrewbogott has quit IRC | 15:08 | |
| *** andrewbogott has joined #openstack-infra | 15:08 | |
| *** thiagop is now known as thiagop-planning | 15:08 | |
| pabelanger | fungi: Yup, make sense now | 15:08 |
| *** zeih has quit IRC | 15:08 | |
| AJaeger | fungi, so https://review.openstack.org/#/c/310093/ is wrong and can be abandoned, correct? | 15:08 |
| pabelanger | fungi: and explains some weirdness I was seeing with recheck a few weeks ago | 15:08 |
| pabelanger | TIL | 15:08 |
| * rcarrillocruz adds a watch on that change | 15:08 | |
| *** nmagnezi has quit IRC | 15:09 | |
| EmilienM | I noticed a lot of Puppet jobs don't have /logs | 15:09 |
| EmilienM | is there any issues I'm not aware today? | 15:09 |
| pabelanger | EmilienM: have review hand? | 15:09 |
| EmilienM | example http://logs.openstack.org/78/327678/1/check/gate-puppet-openstack-integration-3-scenario001-tempest-ubuntu-xenial/a5c0541/ | 15:09 |
| *** julim has quit IRC | 15:09 | |
| *** astupnikov-tmp has quit IRC | 15:10 | |
| pabelanger | EmilienM: that was on zl01, let me check the debug logs | 15:10 |
| fungi | AJaeger: well, it's not necessarily wrong, but it's additional complexity which will be come unneeded once we upgrade to 2.12 (according to zaro) | 15:10 |
| AJaeger | fungi: then let's not do it IMHO - or do you feel strongly it should go in? | 15:10 |
| *** akshai has quit IRC | 15:11 | |
| fungi | AJaeger: i don't feel strongly that it should go in, no (but i also don't feel too strongly that it shouldn't either) | 15:11 |
| rcarrillocruz | nibalizer: re: ansible-puppet just outputting on stdout or remote syslog i have something in local | 15:12 |
| rcarrillocruz | hope to push soon | 15:12 |
| pabelanger | EmilienM: http://paste.openstack.org/show/509265/ is debug log from zuul-launcher | 15:12 |
| nibalizer | rcarrillocruz: cool | 15:12 |
| pabelanger | EmilienM: checking JJB to see how your logs are copied off | 15:12 |
| *** kengo_sakai has joined #openstack-infra | 15:12 | |
| fungi | pabelanger: so the mirror builders don't need any of the packages installed by the openstack_project::thick_slave class? | 15:14 |
| AJaeger | pabelanger: yesterday I saw some log failures as well, let me get the reference again... | 15:14 |
| *** ihrachys has joined #openstack-infra | 15:14 | |
| pabelanger | fungi: Right, in my testing they didn't. However, moving forward I'd like to add wheel dependencies either into wheel_mirror.pp, or setup bindep to run on that server | 15:15 |
| AJaeger | team, I saw earler failing jobs in the post queue - but no log files. Links are http://logs.openstack.org/f2/f20de1013003414a35abd22a31516921a9d8d202/post/openstack-manuals-releasenotes/ff790b9/ and http://logs.openstack.org/39/392bdecc981ce1f07afb91f4927938d570692340/post/openstack-manuals-releasenotes/7e4bb6d/ . Any ideas? | 15:15 |
| AJaeger | pabelanger: that's what I said yesterday ^ | 15:15 |
| *** sorantis has joined #openstack-infra | 15:15 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Fix timeout debug print in tests https://review.openstack.org/327768 | 15:15 |
| * AJaeger will be back later | 15:16 | |
| *** jlanoux has joined #openstack-infra | 15:17 | |
| fungi | AJaeger: i thought jeblair mentioned certain failure modes under zuul-launcher were resulting in logs not getting uploaded, but i don't recall the details or whether that got addressed already | 15:17 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: add private gearman function https://review.openstack.org/327422 | 15:17 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Fix timeout debug print in tests https://review.openstack.org/327768 | 15:17 |
| jeblair | i can see if those are zl releated in a bit | 15:17 |
| jeblair | fungi, AJaeger: ^ | 15:17 |
| pabelanger | EmilienM: jeblair: looks like we found an issue how we create publishers. Specicially, we don't support postbuildscript which copy-puppet-logs uses | 15:17 |
| pabelanger | EmilienM: jeblair: shouldn't be too hard to add support for it | 15:18 |
| pabelanger | I think we'd use the makebuildertask | 15:18 |
| *** annegentle has joined #openstack-infra | 15:18 | |
| pabelanger | or create _makePostBuildScriptTask | 15:19 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Update scripts to look for centos job names https://review.openstack.org/327771 | 15:19 |
| *** Na3iL has quit IRC | 15:21 | |
| *** kengo_sakai has quit IRC | 15:21 | |
| EmilienM | pabelanger: I don't get it, what changed in infra? | 15:22 |
| pabelanger | EmilienM: we removed jenkins01 and replaced it with zl01 | 15:23 |
| EmilienM | ah cool ! | 15:23 |
| EmilienM | maybe something we need to change in copy-puppet-logs script? | 15:23 |
| pabelanger | EmilienM: we basically lack support for the postbuildscript publisher | 15:24 |
| pabelanger | and need to add it | 15:24 |
| fungi | pabelanger: EmilienM: well, we've been discovering corner cases in our existing jenkins job configurations which lacked feature parity in zuul-launcher, and either adjusting jobs to use more typical patterns or adding support in zl depending on what makes the most sense | 15:24 |
| *** mikelk has quit IRC | 15:25 | |
| fungi | so if there's an "easy" way to do the same thing without using postbuildscript, then we might be able to avoid implementing support for it in zl | 15:25 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Only update changed fields when editing tasks https://review.openstack.org/325247 | 15:25 |
| EmilienM | ok, let me know if you need help on something or if I need to change something in our scripts | 15:26 |
| *** esikachev has quit IRC | 15:26 | |
| pabelanger | fungi: EmilienM: jeblair: we could move postbuildpublishers simply to builders but we lose the ability to control onfailure / onsuccess | 15:26 |
| *** ramishra has quit IRC | 15:26 | |
| *** sorantis has quit IRC | 15:27 | |
| fungi | the desire is to keep zl as minimal as possible to support our existing body of job configs, since we don't want to really continue development on zuul 2.x (e.g. add support for more complex jenkins job configuration paradigms), in favor of zuul v3's job configuration model | 15:27 |
| pabelanger | I don't think it will be much work to add support for postbuildscript publisher, working on something now | 15:28 |
| *** tongli has quit IRC | 15:28 | |
| *** ramishra has joined #openstack-infra | 15:28 | |
| *** yujunz has quit IRC | 15:29 | |
| *** tesseract has quit IRC | 15:30 | |
| fungi | looks like we use it in gate-irc-meetings-tox-ical (to make the logs browsable rather than hiding them with the draft site index), in the cleanup publisher for jobs that run on the proposal slave, for the puppet job logs EmilienM pointed out, and also for swift log uploads which we aren't really using at this point | 15:31 |
| jeblair | pabelanger: hrm -- the only use of postbuildscript was supposed to be for the swift logs | 15:32 |
| *** rkukura has quit IRC | 15:32 | |
| *** rcernin has quit IRC | 15:32 | |
| jeblair | we made an exception to our rule of no new jenkins plugins for that because we knew that when we migrated off of jenkins, it would not be a problem | 15:32 |
| jeblair | more or less by definition | 15:32 |
| pabelanger | jeblair: I see proposal-slave-cleanup and copy-puppet-logs | 15:32 |
| *** asselin_ has quit IRC | 15:33 | |
| jeblair | i suppose we were not as disciplined as we should be | 15:33 |
| fungi | so yes, the number of places we have it in use is fairly small and in places which we could try to rework without impacting sensitive jobs (e.g. not used in the release process, et cetera) | 15:33 |
| EmilienM | I would be happy to follow how other projects do | 15:33 |
| jeblair | at any rate, i believe the right solution is to eliminate its use | 15:33 |
| EmilienM | +1 | 15:33 |
| pabelanger | sure, that works | 15:33 |
| EmilienM | what needs to be done for that? I'm a bit confused | 15:33 |
| *** vincentll has quit IRC | 15:33 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/tripleo-ci: Create overcloud images for liberty using EXT4 https://review.openstack.org/327773 | 15:34 |
| pabelanger | we'll need to make then straight builders | 15:34 |
| pabelanger | trick is to make sure they run if there are previous failures | 15:34 |
| *** woodster_ has joined #openstack-infra | 15:34 | |
| pabelanger | I think breaking the copy-logs into a new builders section will make that happen | 15:34 |
| pabelanger | but need to read up on it | 15:34 |
| fungi | the only reason i was hesitant to try to drop support for keep-hierarchy:false in the scp publisher replacement was that it was used in places which would be very disruptive if we didn't get it rewritten exactly correctly in the jobs, and also in important jobs which were run infrequently (so we might not notice we'd broken things for a while until it bit us at an unfortunate moment) | 15:34 |
| fungi | but yes, postbuildscript seems like a good candidate to eliminate in our configs instead | 15:35 |
| *** PollyZ_ has joined #openstack-infra | 15:35 | |
| *** dprince has quit IRC | 15:36 | |
| *** vgridnev has quit IRC | 15:36 | |
| jeblair | pabelanger: i think the first builder that fails stops the whole process | 15:36 |
| jeblair | pabelanger: so in devstack gate, we do: run devstack; save $?; run cleanup; exit $? | 15:37 |
| *** mdrabe has quit IRC | 15:37 | |
| EmilienM | let's do the same for puppet jobs ^ | 15:37 |
| *** PollyZ has quit IRC | 15:38 | |
| pabelanger | jeblair: Ya, we'll have to move the collection of jobs back into puppet, and not jenkins | 15:38 |
| *** bpokorny has joined #openstack-infra | 15:38 | |
| *** PollyZ_ has quit IRC | 15:38 | |
| *** yujunz has joined #openstack-infra | 15:38 | |
| *** ihrachys has quit IRC | 15:39 | |
| *** nadya has joined #openstack-infra | 15:39 | |
| *** ihrachys has joined #openstack-infra | 15:39 | |
| *** Goneri has quit IRC | 15:39 | |
| EmilienM | pabelanger: into our scripts you mean? | 15:40 |
| EmilienM | I can do it | 15:40 |
| *** nadya has quit IRC | 15:40 | |
| *** yujunz has quit IRC | 15:40 | |
| fungi | dstufft: got a "503 Server Error: Service Unavailable" from twine upload to warehouse, yet it appeared on pypi anyway. so switching to warehouse doesn't seem to have eliminated the issues we've been experiencing? http://logs.openstack.org/d3/d3b92fc94d49b20c3582d7d8423a2eb7416fcb6a/release/jenkins-job-builder-pypi-both-upload/a4360f3/console.html https://pypi.python.org/pypi/jenkins-job-builder | 15:40 |
| *** dmk0202 has quit IRC | 15:41 | |
| pabelanger | EmilienM: yup | 15:41 |
| pabelanger | EmilienM: bascially, you need an exit handler to setup logs for zuul | 15:41 |
| dstufft | fungi: 503s are a bit different than 500 in terms of where they stem from-- in this case Fastly is 503ing things because PyPI is being DoS'd atm | 15:41 |
| pabelanger | EmilienM: and we remove copy-puppet-logs | 15:41 |
| dstufft | (I think it's an inadvertent DoS though) | 15:42 |
| fungi | dstufft: noted. should we avoid trying to upload things for now? | 15:42 |
| *** yujunz has joined #openstack-infra | 15:42 | |
| fungi | if so, i can give the release team a heads up | 15:42 |
| *** weshay_mtg is now known as weshay | 15:42 | |
| dstufft | fungi: in any case, with Warehouse that upload should be abslolutely intact, versus the PyPI behavior where it may or may not be and you'd have to verify it (IOW, Warehouse upload: atomic, PyPI upload: Not atomic) | 15:42 |
| fungi | dstufft: so we should assume that a 503 response from twine uploading to warehouse is a success, basically equivalent to 200 ok? | 15:43 |
| annegentle | AJaeger: or yolanda: can you tell why this patchset didn't merge? https://review.openstack.org/#/c/313015/ | 15:43 |
| dstufft | fungi: Not exactly, but I can talk more on it later, for now -- https://status.python.org/incidents/2bgrnrd7nyl6 and whether or not you want to hold off on uploading, depends on if you'd rather hold off, or upload now but maybe need to retry an upload | 15:44 |
| mtreinish | jeblair: on gertty is there a config flag for showing email addresses along with the names? Or is that something I'll have to write a patch up for | 15:44 |
| *** mpaolino has joined #openstack-infra | 15:44 | |
| fungi | dstufft: noted--thanks, and please don't let me distract you from the fires at hand :/ | 15:44 |
| jeblair | mtreinish: patch | 15:44 |
| jeblair | mtreinish: also ++ | 15:45 |
| mtreinish | jeblair: ok, I'll try to start on that soon | 15:45 |
| jeblair | mtreinish: (should already be in the db, so mostly a matter of formatting) | 15:45 |
| EmilienM | pabelanger: does $WORKSPACE still exist? | 15:45 |
| EmilienM | what is the new place? | 15:46 |
| jeblair | EmilienM: yes -- everything should be exactly the same | 15:46 |
| *** Guest_84756 has joined #openstack-infra | 15:46 | |
| EmilienM | ok cool | 15:46 |
| EmilienM | thx | 15:46 |
| *** banix has quit IRC | 15:46 | |
| Guest_84756 | Allah is doing | 15:46 |
| Guest_84756 | sun is not doing Allah is doing | 15:46 |
| Guest_84756 | moon is not doing Allah is doing | 15:47 |
| *** ChanServ sets mode: +o mordred | 15:47 | |
| *** esikachev has joined #openstack-infra | 15:47 | |
| *** LaT0rtue has joined #openstack-infra | 15:47 | |
| *** Guest_84756 was kicked by mordred (Guest_84756) | 15:47 | |
| *** banix has joined #openstack-infra | 15:47 | |
| *** ddieterly is now known as ddieterly[away] | 15:47 | |
| *** chem has quit IRC | 15:48 | |
| openstackgerrit | Merged openstack-infra/project-config: Simplify translation logic https://review.openstack.org/324548 | 15:48 |
| EmilienM | jeblair, pabelanger: https://review.openstack.org/327783 | 15:49 |
| *** PollyZ has joined #openstack-infra | 15:49 | |
| *** psachin has quit IRC | 15:50 | |
| pabelanger | EmilienM: you'll have to trap exit, otherwise copy_logs.sh isn't guaranteed to run on failures | 15:50 |
| *** jogo has quit IRC | 15:51 | |
| *** mixos has joined #openstack-infra | 15:51 | |
| *** jogo has joined #openstack-infra | 15:51 | |
| EmilienM | pabelanger: I think I just need to move RESULT=$? no? | 15:51 |
| EmilienM | from L224 to L231 | 15:52 |
| jeblair | there are some set -e / set +e lines in there... so parts of the script are exit on failure, parts are not | 15:52 |
| *** mdrabe has joined #openstack-infra | 15:54 | |
| jeblair | i think the way it's currently written is basically: if it gets to tox, it will run regardless of what tox does, as long as the subunit and selinux bits don't have an error | 15:54 |
| *** deadnull_ has quit IRC | 15:54 | |
| jeblair | so it will probably run in most cases we care about, but may not run in absolutely all cases | 15:54 |
| *** bnemec has quit IRC | 15:55 | |
| jeblair | trapping exit would do that. depends on what you want. | 15:55 |
| jeblair | i'm going to propose a change to the proposal cleanup thing. | 15:55 |
| EmilienM | jeblair: see https://review.openstack.org/#/c/327783/2/run_tests.sh | 15:55 |
| pabelanger | adding it to tox.ini is a good idea. tox by default will run all scripts in order, regardless of exit code | 15:55 |
| jeblair | EmilienM: yeah, that's what i was talking about | 15:55 |
| EmilienM | jeblair: is it better? | 15:55 |
| jeblair | oh you're saying see ps2 | 15:56 |
| *** rossella_s has quit IRC | 15:56 | |
| openstackgerrit | Matthew Treinish proposed openstack/gertty: Add user email addresses to the change view https://review.openstack.org/327785 | 15:56 |
| jeblair | EmilienM: no i think that's worse | 15:56 |
| mtreinish | jeblair: ^^^ I didn't add a config option | 15:56 |
| mtreinish | but if you think making it optional is worth it I can add that | 15:56 |
| *** Apoorva has joined #openstack-infra | 15:56 | |
| jeblair | mtreinish: i'll run with that and let you know :) thanks! | 15:57 |
| *** Apoorva has quit IRC | 15:57 | |
| *** Apoorva has joined #openstack-infra | 15:58 | |
| *** furlongm_ has joined #openstack-infra | 15:59 | |
| jeblair | mtreinish: i worry it may be too wide for the reviewer table... maybe we could omit it there but add it to the review messages section instead? | 15:59 |
| *** oanson has joined #openstack-infra | 15:59 | |
| mtreinish | jeblair: ok, yeah I just was looking at review with a long email addr cutting off the workflow column | 15:59 |
| mtreinish | I'll respin it | 15:59 |
| *** esikachev has quit IRC | 16:00 | |
| EmilienM | jeblair: what do you suggest? | 16:00 |
| *** arxcruz has quit IRC | 16:01 | |
| jeblair | EmilienM: i like ps1 | 16:01 |
| EmilienM | jeblair: ok | 16:01 |
| jeblair | EmilienM: or, if there is a problem with it, what pabelanger suggests with trap exit | 16:01 |
| *** furlongm has quit IRC | 16:02 | |
| *** amotoki has joined #openstack-infra | 16:03 | |
| *** flepied has quit IRC | 16:03 | |
| EmilienM | jeblair: yeah the problem in this case is that I want logs if catch_selinux_alerts fails | 16:04 |
| *** bnemec has joined #openstack-infra | 16:04 | |
| EmilienM | jeblair: that's why I moved the set -e | 16:04 |
| jeblair | EmilienM: the probably an exit trap is best | 16:04 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/ansible-role-puppet: Syslog puppet output on Ansible control machine https://review.openstack.org/327789 | 16:04 |
| EmilienM | jeblair: right, I'll do that, thx | 16:04 |
| openstackgerrit | Matthew Treinish proposed openstack/gertty: Add user email addresses to the change view https://review.openstack.org/327785 | 16:06 |
| mtreinish | jeblair: ^^^ | 16:06 |
| *** abregman has joined #openstack-infra | 16:06 | |
| *** LaT0rtue has quit IRC | 16:08 | |
| *** anteaya has joined #openstack-infra | 16:09 | |
| *** ddieterly[away] is now known as ddieterly | 16:10 | |
| *** aviau has quit IRC | 16:10 | |
| *** aviau has joined #openstack-infra | 16:10 | |
| *** Guest77604 has quit IRC | 16:10 | |
| *** PollyZ has quit IRC | 16:11 | |
| jeblair | mtreinish: cool | 16:11 |
| jeblair | remote: https://review.openstack.org/327795 Clean up proposal slave in a builder | 16:14 |
| jeblair | pabelanger, fungi: ^ | 16:14 |
| *** Na3iL has joined #openstack-infra | 16:14 | |
| EmilienM | pabelanger: is it what you thought ? https://review.openstack.org/#/c/327783/3/run_tests.sh | 16:14 |
| jeblair | mordred: can you +3 https://review.openstack.org/326699 ? | 16:15 |
| *** vhosakot has quit IRC | 16:15 | |
| jeblair | pabelanger, mordred: can you +3 https://review.openstack.org/327333 | 16:16 |
| *** PollyZ has joined #openstack-infra | 16:16 | |
| *** andreas_s has quit IRC | 16:17 | |
| mordred | jeblair: looking | 16:17 |
| *** degorenko is now known as _degorenko|afk | 16:18 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Use underscore for node_assign job https://review.openstack.org/327799 | 16:18 |
| *** cody-somerville has joined #openstack-infra | 16:18 | |
| *** gyee has joined #openstack-infra | 16:19 | |
| jeblair | pabelanger, mordred: also https://review.openstack.org/327346 | 16:19 |
| mordred | jeblair: one note | 16:19 |
| jeblair | oh pabelanger has an old +2 on it. i'll carry it over. | 16:19 |
| mordred | jeblair: on 326699 | 16:20 |
| *** Swami has joined #openstack-infra | 16:20 | |
| *** savihou has quit IRC | 16:20 | |
| jeblair | mordred: expand groups uses the inventory cache? | 16:20 |
| jeblair | i guess it does, it just calls ansible --list-hosts | 16:21 |
| mordred | jeblair: yes. | 16:21 |
| mordred | yup. that's right. | 16:21 |
| jeblair | i ran it without having cleared the inventory cache, but maybe the cache had been updated externally... | 16:21 |
| mordred | remind me at some point to chat with the ansible core team about the hack we're doing there with expand-groups to see if we can get a better answer for that upstream | 16:21 |
| *** pvaneck has joined #openstack-infra | 16:22 | |
| fungi | jeblair: problem with 327795. see inline comments | 16:22 |
| *** nmagnezi has joined #openstack-infra | 16:23 | |
| *** Na3iL has quit IRC | 16:23 | |
| *** yamamoto has quit IRC | 16:23 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Use global generated groups in launch-node https://review.openstack.org/326699 | 16:23 |
| jeblair | mordred: ^ | 16:23 |
| jeblair | fungi: i thought the dirname would get me to "parent of $WORKSPACE" | 16:24 |
| mordred | jeblair: looks great oe me now | 16:24 |
| pabelanger | EmilienM: left comment. Might be better to move the trap sooner in your script | 16:24 |
| pabelanger | EmilienM: aside from that, it should work | 16:25 |
| *** sdake has quit IRC | 16:25 | |
| EmilienM | pabelanger: ack | 16:26 |
| jeblair | fungi: so "delete all children of workspace's parent" is what i thought i wrote | 16:26 |
| *** tkelsey has joined #openstack-infra | 16:26 | |
| jeblair | fungi: though, perhaps a -p is still warrented... also, i think cwd will not exist, so we need to cd again | 16:27 |
| *** nadya has joined #openstack-infra | 16:27 | |
| jeblair | though, we aren't running that with -e so should be fine | 16:27 |
| EmilienM | pabelanger: https://review.openstack.org/#/c/327783/4/run_tests.sh is soon enough? | 16:28 |
| pabelanger | EmilienM: Bascially, you want to move trap above anything you expect logs to be copied for | 16:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Clean up proposal slave in a builder https://review.openstack.org/327795 | 16:29 |
| jeblair | fungi, pabelanger: ^ that adds the cd | 16:30 |
| EmilienM | pabelanger: in JJB? :) | 16:30 |
| EmilienM | pabelanger: it's really above now | 16:31 |
| *** vgridnev has joined #openstack-infra | 16:31 | |
| *** vgridnev has quit IRC | 16:31 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Add support for launcher accept_nodes parameter https://review.openstack.org/327335 | 16:31 |
| jeblair | mordred, pabelanger: can you +2 https://review.openstack.org/327799 i will +3 when ready | 16:31 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: use underscores where appropriate https://review.openstack.org/327333 | 16:32 |
| EmilienM | pabelanger: we'll have to patch project-config for beaker jobs, so we also run this script | 16:32 |
| pabelanger | jeblair: just looked at the existing slave-cleanup.sh again. Is it safe to blow away the WORKSPACE directory? I thought there was some sutff cached in it. | 16:32 |
| EmilienM | jeblair: why devstack-logs still does exist in project-config? does it still work then? | 16:34 |
| *** yujunz has quit IRC | 16:35 | |
| *** tphummel has joined #openstack-infra | 16:35 | |
| jeblair | pabelanger: it's safe. it starts empty the first time a job runs. | 16:35 |
| pabelanger | jeblair: okay, +2 from me | 16:35 |
| jeblair | EmilienM: devstack-logs is a publisher -- it publishes the logs that devstack-gate collects | 16:36 |
| *** jlanoux has quit IRC | 16:36 | |
| EmilienM | jeblair: so does copy-puppet-logs | 16:36 |
| EmilienM | ah it uses postbuildscript | 16:37 |
| EmilienM | ok I see, sorry | 16:37 |
| jeblair | EmilienM: copy-puppet-logs copies logs from whereever to $WORKSPACE/logs. devstack-logs publishes logs in $WORKSPACE/logs to logs.openstack.org/job/ | 16:38 |
| *** sdake has joined #openstack-infra | 16:38 | |
| EmilienM | ok I'm working on it for our integration & beaker jobs that require logs. | 16:38 |
| *** cody-somerville has quit IRC | 16:38 | |
| EmilienM | in the meantime, should we restrict jenkins nodes for those jobs? | 16:38 |
| jeblair | EmilienM: so you'll still want to keep devstack-logs as a publisher so that it publishes the logs that the script you're moving will copy | 16:39 |
| EmilienM | ok | 16:39 |
| *** gongysh has quit IRC | 16:39 | |
| jeblair | EmilienM: we do not have the ability to restrict nodes like that, so i think we need to land this change asap | 16:39 |
| fungi | jeblair: oh! sorry, i see why i was confused now... i totally overlooked the dirname there, but shouldn't that be a shell expansion? | 16:40 |
| *** chem has joined #openstack-infra | 16:40 | |
| mordred | jeblair: done | 16:40 |
| *** dtantsur is now known as dtantsur|afk | 16:40 | |
| *** ihrachys_ has joined #openstack-infra | 16:40 | |
| *** ihrachys has quit IRC | 16:41 | |
| *** thorst has quit IRC | 16:41 | |
| jeblair | fungi: yes | 16:41 |
| *** daemontool has quit IRC | 16:41 | |
| EmilienM | jeblair: ack, working on it as a priority | 16:41 |
| jeblair | EmilienM: i can enqueue/promote that into the gate queue when you're ready | 16:42 |
| *** abregman is now known as abregman|afk | 16:42 | |
| *** trown is now known as trown|lunch | 16:43 | |
| *** d0ugal has quit IRC | 16:43 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Clean up proposal slave in a builder https://review.openstack.org/327795 | 16:44 |
| jeblair | fungi: ^ | 16:44 |
| pabelanger | jeblair: going to try and node a beaker node run by zl01. See if we can figure out the SSH issue | 16:44 |
| *** d0ugal has joined #openstack-infra | 16:44 | |
| jeblair | pabelanger: ++ there are 5 builds pending, one running, 0 on zl01 | 16:45 |
| *** sdake_ has joined #openstack-infra | 16:45 | |
| jeblair | it only happens when i'm not looknig | 16:45 |
| *** sdake has quit IRC | 16:45 | |
| *** eharney has quit IRC | 16:45 | |
| *** johnny_ has joined #openstack-infra | 16:45 | |
| *** thorst has joined #openstack-infra | 16:46 | |
| *** nadya has quit IRC | 16:46 | |
| jeblair | fungi, pabelanger, mordred: can you +3 https://review.openstack.org/327346 despite the openstackci failure (it's the zl01 problem pabelanger and i are working on) | 16:46 |
| mordred | jeblair: yup | 16:47 |
| *** _shaps_ has quit IRC | 16:47 | |
| *** dimtruck is now known as zz_dimtruck | 16:48 | |
| *** thorst_ has joined #openstack-infra | 16:48 | |
| *** tonytan4ever has joined #openstack-infra | 16:48 | |
| *** e0ne has quit IRC | 16:48 | |
| *** ddieterly is now known as ddieterly[away] | 16:49 | |
| *** ihrachys_ has quit IRC | 16:49 | |
| *** eharney has joined #openstack-infra | 16:49 | |
| *** thorst has quit IRC | 16:50 | |
| *** thorst has joined #openstack-infra | 16:51 | |
| *** hashar has quit IRC | 16:52 | |
| *** ddieterly[away] is now known as ddieterly | 16:52 | |
| *** sdake_ has quit IRC | 16:52 | |
| *** hashar has joined #openstack-infra | 16:53 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Remove copy_puppet_logs publisher https://review.openstack.org/327820 | 16:53 |
| EmilienM | pabelanger, jeblair: this is probably really WIP but any review is highly welcome ^ | 16:53 |
| *** thorst_ has quit IRC | 16:53 | |
| *** sarob has joined #openstack-infra | 16:53 | |
| *** cody-somerville has joined #openstack-infra | 16:54 | |
| *** ifarkas has quit IRC | 16:55 | |
| *** hashar has quit IRC | 16:55 | |
| *** thorst is now known as thorst_away | 16:56 | |
| *** rkukura has joined #openstack-infra | 16:57 | |
| jeblair | EmilienM: lgtm. any reason you didn't Depends-On: the other change? | 16:57 |
| *** amotoki has quit IRC | 16:57 | |
| EmilienM | jeblair: because I don't think the patch in our repo will work | 16:57 |
| notmorgan | jeblair: when you've finished putting out the current fire (and have a few moments), I'd appreciate a quick review + release [when it lands] of gear for https://review.openstack.org/#/c/327310/ :) that should open the rest of the py3 chain | 16:57 |
| *** yamamoto has joined #openstack-infra | 16:58 | |
| EmilienM | jeblair: or maybe? | 16:58 |
| EmilienM | well I can see result first and tell | 16:58 |
| pabelanger | EmilienM: left comment about your clone | 16:58 |
| jeblair | EmilienM: well, i'm pretty sure you don't want to land the project-config change until the p-o-i change lands | 16:58 |
| *** shardy has quit IRC | 16:58 | |
| *** regXboi has joined #openstack-infra | 16:58 | |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: The Non-ATC Recognition WG has changed its name to the AUC Recognition WG. This change updates the meeting information to reflect the new name and agenda location. https://review.openstack.org/327824 | 16:58 |
| *** bpokorny has quit IRC | 16:59 | |
| EmilienM | jeblair: I commented your question. Also, please double check my -checkout thing :-) | 16:59 |
| *** bpokorny has joined #openstack-infra | 16:59 | |
| jeblair | notmorgan: i love string handling in py3. b'foo'.decode('utf-8').encode('utf-8') | 16:59 |
| EmilienM | jeblair: err, sorry it was pabelanger's question | 17:00 |
| notmorgan | jeblair: right? just wait till you see the massive encode/decode patch in zuul... :( | 17:00 |
| pabelanger | jeblair: 1550198 is our node for beaker | 17:00 |
| jeblair | pabelanger: you held it? | 17:00 |
| pabelanger | jeblair: yup | 17:00 |
| notmorgan | jeblair: want to see something fun, in py3: b'abcd'[0] => 97 | 17:00 |
| zaro | fungi, pabelanger : yes, i believe it is in 2.12. it’s a change to add vote info to every ssh event. https://gerrit-review.googlesource.com/#/c/71051/ | 17:01 |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Renaming the Non-ATC Recognition WG The Non-ATC Recognition WG has changed its name to the AUC Recognition WG. This change updates the meeting information to reflect the new name and agenda location. https://review.openstack.org/327824 | 17:01 |
| *** sputnik13 has joined #openstack-infra | 17:01 | |
| zaro | fungi, pabelanger the votes will appear on every comment-added event | 17:02 |
| *** yamahata has joined #openstack-infra | 17:02 | |
| jeblair | notmorgan: if you think we should alter gear to be more friendly, we can talk about it. the idea is that gear should not make it hard to work with the low-level gearman protocol, including with binary data. but if that means constant encode/decodes that shouldn't be necessary, we can think about making the library smarter. | 17:03 |
| jeblair | pabelanger: i'm nc 158.69.71.206 8088 | 17:03 |
| *** cody-somerville has quit IRC | 17:03 | |
| *** asettle has quit IRC | 17:04 | |
| notmorgan | jeblair: possibly | 17:04 |
| pabelanger | jeblair: as am I | 17:04 |
| jeblair | notmorgan: i mean on the gear user side, of course | 17:04 |
| *** bpokorny has quit IRC | 17:04 | |
| *** vhosakot has joined #openstack-infra | 17:04 | |
| jeblair | notmorgan: anyway, let me know what you think | 17:04 |
| jeblair | notmorgan: 310 +3, will release when lands | 17:05 |
| *** yamamoto has quit IRC | 17:05 | |
| jeblair | pabelanger: i also have a root shell open on that host -- just in case :) | 17:06 |
| zaro | jeblair: wow, setting up cgit was more difficult than i anticipated. learned a few things along the way so that’s good. still grapling with SElinux though | 17:07 |
| pabelanger | jeblair: currently tailing auth.log to see the SSH failure | 17:07 |
| jeblair | zaro: you could probably setenforce 0 for this test | 17:07 |
| notmorgan | jeblair: i think its something we should consider (making gear more friendly), but it is also something i want to be careful about. i think i see where it can be done. | 17:08 |
| pabelanger | jeblair: I think I see the issue | 17:08 |
| pabelanger | 2016-06-09 17:07:48.229297 Notice: /Stage[main]/Jenkins::Jenkinsuser/File[/home/jenkins/.ssh/authorized_keys]/content: content changed '{md5}b56bfea7722db96523ead1c242bd8411' to '{md5}8fea1f3c7a91379e612957d683eaa6ad' | 17:09 |
| pabelanger | jeblair: the puppet jobs is blowing away our ssh key | 17:09 |
| zaro | jeblair: well, just wanted to clear up what we are doing. it seems to me what we are actually testing is hosting git repos with apache, correct? is that what we are doing on git.o.o? | 17:09 |
| mordred | pabelanger: that seems like a bad idea | 17:09 |
| jeblair | yep, that would do it. | 17:09 |
| pabelanger | jeblair: mordred: so, we should update beaker to use a different user then jenkins | 17:09 |
| fungi | pabelanger: are we applying, say, the jenkins slave class as part of the test, and using dummy hiera data for the key or something? | 17:09 |
| fungi | pabelanger: have you looked at /home/jenkins/.ssh/authorized_keys afterward to see what it did in there? | 17:10 |
| jeblair | fungi: i suspect that's the case | 17:10 |
| pabelanger | jeblair: mordred: which, if we moved to zuul user for zl, I don't think we'd have this issue | 17:10 |
| pabelanger | fungi: right, I think that is the problem | 17:10 |
| pabelanger | checking beaker now | 17:10 |
| jeblair | pabelanger: i thought you said you held the node? | 17:10 |
| pabelanger | jeblair: I did | 17:11 |
| pabelanger | in nodepool | 17:11 |
| jeblair | pabelanger: i don't see it | 17:11 |
| jeblair | and i've lost access | 17:11 |
| jeblair | pabelanger: i don't think changing the user this runs as will fix it. if we do that, we're just going to overwrite zuul's authorized_keys file | 17:11 |
| jeblair | (because we will have a puppet manifest that says "write garbage to zuul's authorized_keys file") | 17:12 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: [WIP] Add mulitnode CI job support https://review.openstack.org/324777 | 17:13 |
| *** julim has joined #openstack-infra | 17:13 | |
| pabelanger | jeblair: http://paste.openstack.org/show/509281/ | 17:13 |
| pabelanger | that's from the hold | 17:14 |
| jeblair | pabelanger: that node ran gate-puppet-zuul-puppet-beaker-rspec-ubuntu-trusty. 1550292 ran gate-openstackci-beaker-ubuntu-trusty | 17:14 |
| pabelanger | jeblair: we had the wrong IP | 17:14 |
| jeblair | pabelanger: we had the right ip, that was the wrong node | 17:15 |
| pabelanger | Hmm | 17:15 |
| pabelanger | well | 17:15 |
| openstackgerrit | Joe Talerico proposed openstack-infra/project-config: Adding Browbeat https://review.openstack.org/327255 | 17:15 |
| pabelanger | copy paste fail on my behalf | 17:15 |
| pabelanger | let me find another | 17:15 |
| jeblair | gate-openstackci-beaker-ubuntu-trusty telnet://158.69.71.206:8088 | 17:15 |
| *** megm has quit IRC | 17:16 | |
| openstackgerrit | James Slagle proposed openstack-infra/project-config: Add TripleO 3-node job https://review.openstack.org/326095 | 17:16 |
| jeblair | oh that's the same one | 17:16 |
| jeblair | pabelanger: before we go further -- gate-openstackci-beaker-ubuntu-trusty is the job we want to debug, right? :) | 17:16 |
| *** dizquierdo has quit IRC | 17:16 | |
| *** dprince has joined #openstack-infra | 17:17 | |
| *** ajmiller_ has joined #openstack-infra | 17:17 | |
| *** megm has joined #openstack-infra | 17:17 | |
| jeblair | pabelanger, fungi: here's a copy/paste from my nc: http://paste.openstack.org/show/509282/ | 17:17 |
| *** kushal has quit IRC | 17:18 | |
| pabelanger | jeblair: yes, I incorrectly held a puppet-zuul-beaker job | 17:18 |
| jeblair | ok cool | 17:18 |
| pabelanger | scanning again | 17:18 |
| pabelanger | jeblair: 1551029 | 17:20 |
| pabelanger | held, ip 63.251.114.175 | 17:20 |
| *** mpaolino has quit IRC | 17:20 | |
| *** lucasagomes is now known as lucas-afk | 17:20 | |
| *** ajmiller has quit IRC | 17:21 | |
| *** derekh has quit IRC | 17:21 | |
| *** PollyZ has quit IRC | 17:23 | |
| pabelanger | Ya, our puppet-jenkins manifests are hard coded to the jenkins user. | 17:23 |
| *** ajmiller__ has joined #openstack-infra | 17:23 | |
| pabelanger | Which our puppet-openstackci acceptance test is setting up as SSH key | 17:24 |
| pabelanger | http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/spec/acceptance/fixtures/default.pp#n4 | 17:24 |
| *** jerryz has joined #openstack-infra | 17:24 | |
| jeblair | that explains why centos7 isn't failing | 17:25 |
| pabelanger | A potential fix would be to set http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/manifests/logserver.pp#n20 to undef and not pass jenkins_ssh_key on our acceptanence testing | 17:25 |
| pabelanger | then revert once we move to zuul user | 17:25 |
| *** ajmiller_ has quit IRC | 17:26 | |
| dstufft | fungi: ok fires are out | 17:26 |
| openstackgerrit | Merged openstack-infra/project-config: Create new functional job for glance_store-swift https://review.openstack.org/327526 | 17:27 |
| *** Goneri has joined #openstack-infra | 17:28 | |
| *** ddieterly is now known as ddieterly[away] | 17:28 | |
| pabelanger | jeblair: actually, it might be faster just to add out public key into the acceptance testing. Even if puppet run, it wouldn't change our authorized_keys file | 17:28 |
| dstufft | fungi: so key thing with legacy PyPI-- uploads were not atomic so any interuption in an upload (500 or 503 error) could cause an upload to be partially recorded. This was made worse by the fact that legacy PyPI is slowly bitrotting to death and is consistently returning 500 errors for some percentage of requests | 17:28 |
| jeblair | nibalizer: ping ^ | 17:28 |
| dstufft | fungi: in Warehouse they're atomic, and we're not consistently returning 500 errors for some percentage of requests | 17:28 |
| zaro | rcarrillocruz sorry missed your discussion earlier but what were you planning to do with gerrit SB integration? | 17:28 |
| openstackgerrit | Merged openstack-infra/gear: Do not encode the type (b) in the job name https://review.openstack.org/327310 | 17:29 |
| dstufft | that still means that you can get 503 errors, or 500 errors, but they'll generally mean that something new is broken (in this case, it meant that we had someone spinning up a huge puppet cluster slamming our xmlrpc API and causing lock contention in the database making upload take too long and making it time out with a 503 error-- but it was already buffered by Warehouse it was just waiting on locks to commit it) | 17:29 |
| nibalizer | jeblair: im on a thing but will be around in about half an hour | 17:30 |
| *** bpokorny has joined #openstack-infra | 17:30 | |
| *** markusry has quit IRC | 17:30 | |
| *** HeOS has quit IRC | 17:31 | |
| *** tobiash has quit IRC | 17:31 | |
| *** lmiccini has quit IRC | 17:31 | |
| *** permalac has quit IRC | 17:31 | |
| *** sdake has joined #openstack-infra | 17:31 | |
| *** ajmiller__ has quit IRC | 17:31 | |
| *** ajmiller has joined #openstack-infra | 17:32 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openstackci: Use openstack-infra jenkins users public ssh key https://review.openstack.org/327834 | 17:32 |
| *** salv-orlando has joined #openstack-infra | 17:32 | |
| *** tobiash has joined #openstack-infra | 17:32 | |
| jeblair | pabelanger: is it possible to do that without having a second copy of the key? | 17:32 |
| *** mat128 is now known as mat128|away | 17:32 | |
| pabelanger | jeblair: it is, but will require a lot more code churn | 17:32 |
| jeblair | we *used* to have that in the puppet-jenkins module | 17:33 |
| jeblair | which is the wrong place for it | 17:33 |
| jeblair | but would have made this easy | 17:33 |
| openstackgerrit | Merged openstack-infra/system-config: Do not accept nodes on zlstatic01 https://review.openstack.org/327346 | 17:33 |
| pabelanger | jeblair: right, the right fix would be to make the static jenkins user in jenkinsuser.pp configurable, then we'd create a foobar account for acceptance testing | 17:34 |
| *** sdake_ has joined #openstack-infra | 17:35 | |
| *** salv-orlando has quit IRC | 17:35 | |
| jeblair | pabelanger: honestly, i'm not sure 'jenkinsuser' is the right way to think about this... it's more that 'openstackci::logserver' needs an 'upload user'. | 17:35 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openstackci: Use openstack-infra jenkins users public ssh key https://review.openstack.org/327834 | 17:35 |
| *** yamamoto has joined #openstack-infra | 17:36 | |
| *** salv-orlando has joined #openstack-infra | 17:36 | |
| jeblair | pabelanger: then we can stop relying on 'jenkinsuser' for that | 17:36 |
| pabelanger | jeblair: Ya, we could do that too | 17:36 |
| fungi | dstufft: makes sense. so unlike pypi, this error from warehouse is something we shouldn't expect to be commonplace and therefore don't need to bake extra considerations for into our tooling (we can just handle occasions where it crops up as unique and probably unrelated events)? | 17:36 |
| pleia2 | so many meetings this morning | 17:36 |
| dstufft | fungi: correct | 17:37 |
| jeblair | pabelanger: +3 | 17:37 |
| fungi | dstufft: perfect--thanks again! | 17:37 |
| *** sdake has quit IRC | 17:37 | |
| dstufft | fungi: at least, that's the intention-- I certainly don't want people to have to be working around this sort of thing as a commonplace problem, so if it *is* common place, I want to fix it. | 17:37 |
| *** nwkarsten has quit IRC | 17:37 | |
| dstufft | and as it stands today, it is my expectation that it won't be common place. | 17:38 |
| *** lmiccini has joined #openstack-infra | 17:38 | |
| pleia2 | fungi, pabelanger: yesterday I started writing the spec for the wiki upgrade, but I could use a few minutes of your time when you have a chance to discuss some draft text for our plan | 17:38 |
| pabelanger | jeblair: I'll poke around in logserver.pp next week, and see how to remove the dependency | 17:38 |
| pleia2 | like, using packages vs source | 17:38 |
| *** mtanino has quit IRC | 17:39 | |
| jeblair | pabelanger: cool. that'll give nibalizer time to weigh in as well. | 17:40 |
| fungi | pleia2: thanks, also i was thinking we need to incorporate a wiki-dev server into the plan | 17:40 |
| *** yamamoto has quit IRC | 17:41 | |
| pabelanger | pleia2: sure, I have some bandwidth now | 17:41 |
| pleia2 | fungi: yeah | 17:41 |
| fungi | pleia2: which would give us an opportunity to collaborate on working through the upgrade without lots of unnecessary production impact | 17:41 |
| * pleia2 nods | 17:41 | |
| fungi | and also somewhere to try out spam mitigation extensions we can point people at to test-drive | 17:41 |
| *** maishsk has joined #openstack-infra | 17:42 | |
| pleia2 | keeping it around would be good, so we can also do things like test our auth cutover, should we do that in the future | 17:42 |
| jeblair | #status log deleted jenkins01.openstack.org | 17:42 |
| openstackstatus | jeblair: finished logging | 17:42 |
| pabelanger | \o/ | 17:42 |
| fungi | pleia2: agreed | 17:42 |
| pleia2 | jeblair: ^5 | 17:42 |
| fungi | jeblair: major milestone!!! | 17:43 |
| pleia2 | fungi: so, your research into packages, do you think we should be looking at going the install from source route? | 17:43 |
| *** Sukhdev has joined #openstack-infra | 17:43 | |
| fungi | pleia2: honestly, it looks like the only actual route open to us | 17:43 |
| pleia2 | there are a couple puppet modules around on the internet for installing mediawiki too, but I haven't looked at them | 17:43 |
| pabelanger | fungi: I'm currently dancing in my chair with excitement | 17:43 |
| fungi | the distro packaging landscape for mediawiki is bleak | 17:43 |
| mordred | jeblair: \o/ | 17:43 |
| pleia2 | fungi: ok, that's in sync with my expectations | 17:43 |
| pleia2 | https://forge.puppet.com/tags/mediawiki | 17:44 |
| fungi | the puppet modules i found on the puppetforge are quite stale. most active one hasn't seen any updates for two years | 17:44 |
| jeblair | :) i'm going to spin up zl02 now, mostly so that i can gracefully switch between them for zuul launcher upgrades. we may not want to run both at once for long until the puppet fixes land. | 17:44 |
| pleia2 | fungi: yeah :\ | 17:44 |
| fungi | so likely we need to either update one of them or the one we have | 17:44 |
| *** rbrndt has quit IRC | 17:44 | |
| pabelanger | pleia2: fungi: Right, I think that shows the difficulties puppetizing mediawiki | 17:45 |
| pleia2 | is hashar still at wikimedia? | 17:45 |
| *** shashank_hegde has joined #openstack-infra | 17:45 | |
| pleia2 | I imagine they do it by hand, but might be worth asking | 17:45 |
| fungi | like, i ignored the release info for those modules and dug into the actual master branches for their git repos. deadsville | 17:45 |
| *** nwkarsten has joined #openstack-infra | 17:45 | |
| pleia2 | that's unfortunate | 17:45 |
| *** mriedem1 has joined #openstack-infra | 17:47 | |
| *** mriedem has quit IRC | 17:47 | |
| *** bpokorny has quit IRC | 17:47 | |
| *** mriedem1 has quit IRC | 17:48 | |
| pleia2 | ok, this is easy enough then, I'll finish my first pass at this spec today so we can get to more concrete collaboration in gerrit | 17:48 |
| fungi | pleia2: thanks! | 17:48 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Add zl02 https://review.openstack.org/327839 | 17:49 |
| *** zz_dimtruck is now known as dimtruck | 17:49 | |
| *** mriedem has joined #openstack-infra | 17:49 | |
| *** nwkarsten has quit IRC | 17:49 | |
| fungi | jeblair: do you have any feel for scaling from the load on zl01 to guess at how many we'll need to replace jenkins01-07? or are you just going to go for a 1:1 replacement with each of them? | 17:50 |
| *** vgridnev has joined #openstack-infra | 17:50 | |
| *** trown|lunch is now known as trown | 17:51 | |
| jeblair | fungi: i believe zl01 is right where we want it utilization wise... | 17:51 |
| fungi | cool beans | 17:51 |
| jeblair | so i was planning on a 1:1 replacement -- HOWEVER | 17:51 |
| jeblair | fungi: zl01 is an 8GB node, unlike the 30GB node that jenkins01 was :) | 17:51 |
| mordred | jeblair: excellent | 17:52 |
| fungi | right, some serious performance gains (4x) | 17:52 |
| jeblair | yep | 17:52 |
| *** tonytan4ever has quit IRC | 17:52 | |
| fungi | at least as far as ram is concerned | 17:52 |
| fungi | that's going to free up a good chunk of quota | 17:52 |
| jeblair | yeah, i think they both have 8 vcpus | 17:52 |
| jeblair | here's the cacti graph for zl01: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=291 | 17:53 |
| mordred | jeblair: yes, they both have 8 vcpus | 17:53 |
| fungi | looking pretty healthy indeed | 17:53 |
| fungi | at peak it's coming close to using all its cpu, but not quite | 17:54 |
| mordred | although amusingly enough the processor types between the two are different | 17:54 |
| *** ramishra has quit IRC | 17:54 | |
| jeblair | looking at load average though, i think we may be somewhat oversubscribed | 17:54 |
| fungi | yep | 17:54 |
| mordred | jenkins masters: AMD Opteron(tm) Processor 4170 HE ... zuul launchers: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz | 17:54 |
| fungi | er, oversubscribed? | 17:55 |
| fungi | overpowered from the looks of it | 17:55 |
| jeblair | probably all that ssh activity is keeping it busy -- probably accounts for the system time | 17:55 |
| fungi | oh, wait, i was misreading teh units | 17:55 |
| fungi | yes, high load, but lots of idle processes | 17:56 |
| jeblair | so i expect that the ansible interaction with the workers is probably a bit laggy. | 17:56 |
| *** amoralej is now known as amoralej|off | 17:56 | |
| *** ramishra has joined #openstack-infra | 17:56 | |
| *** gtmanfred has quit IRC | 17:56 | |
| fungi | so one of those situations like an mta where high load doesn't necessarily translate to lots of cpu time | 17:56 |
| jeblair | so we could stand to throw some more cpu at it, otoh, there are no interactive users on the other end of this, so it's probably fine the way it is. | 17:56 |
| jeblair | fungi: right | 17:56 |
| mordred | ++ | 17:56 |
| mordred | fungi: wait. you're applying judgement to a NUMBER? | 17:57 |
| fungi | still a lot of free cycles | 17:57 |
| jeblair | we are keeping the *kernel* busy :) | 17:57 |
| fungi | mordred: i judge numbers all the time. i'm just judgemental like that | 17:57 |
| *** tpeoples has quit IRC | 17:57 | |
| *** bpokorny has joined #openstack-infra | 17:57 | |
| *** tpsilva has quit IRC | 17:57 | |
| *** monester has quit IRC | 17:57 | |
| anteaya | letter 3 | 17:57 |
| anteaya | you seem to be okay with the letter 3 | 17:57 |
| *** esikachev has joined #openstack-infra | 17:58 | |
| fungi | best letter ever | 17:58 |
| mordred | anteaya: it's a friendly letter | 17:58 |
| openstackgerrit | C.J. Collier proposed openstack-infra/gerritbot: Improve gerritbot error handling https://review.openstack.org/287056 | 17:58 |
| *** serverascode has quit IRC | 17:58 | |
| anteaya | ha ha ha | 17:58 |
| *** flip217 has quit IRC | 17:58 | |
| jeblair | it's l33t | 17:58 |
| *** flip217 has joined #openstack-infra | 17:58 | |
| anteaya | hehe | 17:58 |
| * mordred hands jeblair an indeterminate amount of cats | 17:58 | |
| fungi | mordred measures cats by volume | 17:59 |
| *** _sarob has joined #openstack-infra | 17:59 | |
| *** maximov has quit IRC | 17:59 | |
| *** electrofelix has quit IRC | 17:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Clean up proposal slave in a builder https://review.openstack.org/327795 | 17:59 |
| *** zz_ja has quit IRC | 17:59 | |
| notmorgan | mordred: are they in boxes? and are they alive or dead as a superposition? | 17:59 |
| mordred | fungi: I measure them by total aggregate amount of being ignored that's happening from the set of cats in my direction | 17:59 |
| fungi | water displacement is the most effective measure for cat volume | 17:59 |
| *** monester has joined #openstack-infra | 17:59 | |
| *** mgkwill has quit IRC | 18:00 | |
| *** serverascode has joined #openstack-infra | 18:00 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Set nodepool.manager.MANAGER.nodes.STATE to zero https://review.openstack.org/327681 | 18:00 |
| *** timburke has quit IRC | 18:00 | |
| *** tpeoples has joined #openstack-infra | 18:00 | |
| jeblair | fungi: also most painful | 18:00 |
| *** gema has quit IRC | 18:00 | |
| *** electrofelix has joined #openstack-infra | 18:00 | |
| *** tpsilva has joined #openstack-infra | 18:01 | |
| *** zz_ja has joined #openstack-infra | 18:01 | |
| *** gema has joined #openstack-infra | 18:01 | |
| *** electrofelix has quit IRC | 18:01 | |
| *** mgkwill has joined #openstack-infra | 18:01 | |
| *** maximov has joined #openstack-infra | 18:01 | |
| fungi | likely to result in personal blood loss, based on my prior experience at least | 18:02 |
| *** itisha has quit IRC | 18:02 | |
| *** yolanda has quit IRC | 18:02 | |
| *** timburke has joined #openstack-infra | 18:02 | |
| *** gtmanfred has joined #openstack-infra | 18:02 | |
| *** Odd_Bloke has quit IRC | 18:03 | |
| *** itisha has joined #openstack-infra | 18:03 | |
| *** sarob has quit IRC | 18:03 | |
| *** tkelsey has quit IRC | 18:03 | |
| anteaya | mordred: you cat is ignoring you? how unusual | 18:03 |
| *** mtreinish has quit IRC | 18:03 | |
| anteaya | your | 18:03 |
| *** mtreinish has joined #openstack-infra | 18:03 | |
| *** jbryce has quit IRC | 18:05 | |
| *** kzaitsev_mb has quit IRC | 18:05 | |
| pabelanger | fungi: we migrated tripleo to centos-7 again this morning. Things look real good | 18:06 |
| fungi | pabelanger: excellent news, thanks | 18:06 |
| pabelanger | fungi: I figured my monday we can remove tripleo-f22 bits from nodepool | 18:06 |
| pabelanger | by* | 18:06 |
| fungi | wfm | 18:06 |
| pabelanger | update spec, and start to remove snapshot code from nodepool | 18:07 |
| *** scottda has quit IRC | 18:07 | |
| *** jbryce has joined #openstack-infra | 18:07 | |
| *** PollyZ has joined #openstack-infra | 18:07 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: Remove copy_puppet_logs publisher https://review.openstack.org/327820 | 18:07 |
| EmilienM | pabelanger: I had to rebase it and I tried to improve it, can you tell me if you like it? ^ | 18:08 |
| *** Sukhdev has quit IRC | 18:08 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Update scripts to look for centos job names https://review.openstack.org/327771 | 18:08 |
| *** gtmanfred has quit IRC | 18:09 | |
| *** tpsilva has quit IRC | 18:09 | |
| mordred | anteaya: well, my cat is in another country right now | 18:09 |
| *** scottda has joined #openstack-infra | 18:09 | |
| mordred | anteaya: but when he desires, he is VERY good at actively ignoring me | 18:09 |
| *** mtreinish has quit IRC | 18:10 | |
| anteaya | mordred: ah, I had thought the subtext to your statement was "I am home" which was what I was querying | 18:10 |
| mordred | he, of course, makes it crystal clear that he wants nothing to do with me and does not regard my presence as valuable to him | 18:10 |
| anteaya | mordred: gives lessons on it, does he? | 18:10 |
| *** tpsilva has joined #openstack-infra | 18:10 | |
| mordred | anteaya: he's very good | 18:10 |
| anteaya | mordred: have you found his slide presentations yet? | 18:10 |
| mordred | of course, his other state is indicating very clearly thatI should be paying rapt attention to him | 18:10 |
| anteaya | what is his name, we can google him | 18:10 |
| *** gtmanfred has joined #openstack-infra | 18:10 | |
| anteaya | mordred: there are no other choices on the spectrum | 18:11 |
| mordred | while communicating that he doesn't _actually_ care if I do pay attention, but I'd clearly be deficient if I didn't | 18:11 |
| anteaya | he's good | 18:11 |
| mordred | yah | 18:11 |
| mordred | he can cat very well | 18:11 |
| anteaya | glad to know you have one of the best at catting | 18:12 |
| notmorgan | jeblair: i'm going to figure out the no-error zuul gate failure today. this is bugging me. | 18:12 |
| notmorgan | s/gate/unit test | 18:12 |
| *** mtreinish has joined #openstack-infra | 18:13 | |
| mordred | notmorgan: you probably have not fed the cat | 18:13 |
| mordred | notmorgan: so the cat has probably eaten the unit test | 18:13 |
| nibalizer | pabelanger: jeblair so our logserver.pp code is wiping out the key we use to hop into the box? | 18:13 |
| nibalizer | if i understand correctly? | 18:13 |
| notmorgan | mordred: probably | 18:13 |
| openstackgerrit | Steven Dake proposed openstack-infra/irc-meetings: Move kolla meeting to 16:00 UTC https://review.openstack.org/327845 | 18:13 |
| pabelanger | nibalizer: exactly | 18:14 |
| *** mat128|away is now known as mat128 | 18:14 | |
| notmorgan | mordred: on the plus side. i think we can have py3 zuul soon | 18:14 |
| pabelanger | nibalizer: and wasn't an issue with jenkins, because we only SSH 1 time into the node | 18:14 |
| openstackgerrit | Merged openstack-infra/system-config: Use global generated groups in launch-node https://review.openstack.org/326699 | 18:15 |
| *** mtanino has joined #openstack-infra | 18:15 | |
| *** mgagne has quit IRC | 18:16 | |
| openstackgerrit | Merged openstack-infra/puppet-jeepyb: Fix package name on RedHat https://review.openstack.org/327659 | 18:16 |
| *** jbryce has quit IRC | 18:16 | |
| notmyname | I'd like to have a feature/crypto-review branch made in swift, please. rooted off of current HEAD of master (11c5ef7d22) | 18:16 |
| *** erlon has quit IRC | 18:17 | |
| *** clayton has quit IRC | 18:17 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-jenkins: Add ssh key to jenkinuser non destructively https://review.openstack.org/327847 | 18:17 |
| *** mgagne has joined #openstack-infra | 18:17 | |
| nibalizer | pabelanger: i think that'll fix it | 18:17 |
| *** clayton has joined #openstack-infra | 18:17 | |
| pabelanger | nibalizer: ssh_authorized_key appends right? | 18:18 |
| nibalizer | yah | 18:18 |
| *** erlon has joined #openstack-infra | 18:18 | |
| nibalizer | so it wont wipe out what is already there | 18:18 |
| nibalizer | it might acutally stick em at the top, but the point is it doesn't wipe the others out | 18:19 |
| pabelanger | nibalizer: okay, so I think we need to remove line 65 too | 18:19 |
| pabelanger | but, I like that idea | 18:19 |
| *** hongbin has quit IRC | 18:19 | |
| jeblair | i'm pretty sure when we rotate the key, we're going to want it to go away | 18:19 |
| nibalizer | oh yea ill update | 18:19 |
| *** jbryce has joined #openstack-infra | 18:19 | |
| *** hongbin_ has joined #openstack-infra | 18:19 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/puppet-jenkins: Add ssh key to jenkinuser non destructively https://review.openstack.org/327847 | 18:21 |
| mordred | notmyname: done | 18:21 |
| *** salv-orlando has quit IRC | 18:21 | |
| notmyname | mordred: thanks | 18:21 |
| *** rbrndt has joined #openstack-infra | 18:21 | |
| nibalizer | jeblair: if you rotate the key by changing the variable in $ssh_key it will do the right thing | 18:21 |
| *** salv-orlando has joined #openstack-infra | 18:22 | |
| jeblair | nibalizer: pretty sure there's a flaw in that module where you have to change the name too | 18:22 |
| pabelanger | I am hoping most of our jenkins puppet code goes away :p | 18:22 |
| mordred | pabelanger: ++ | 18:22 |
| jeblair | nibalizer: which is why we use dates in the key name when we use that module | 18:22 |
| mordred | yup | 18:22 |
| *** mohankumar_ has quit IRC | 18:23 | |
| nibalizer | well let me test it | 18:23 |
| nibalizer | but it should work | 18:23 |
| openstackgerrit | Anita Kuno proposed openstack-infra/storyboard: Create a StoryBoard gui manual https://review.openstack.org/325474 | 18:23 |
| *** asselin has quit IRC | 18:23 | |
| *** tqtran has joined #openstack-infra | 18:24 | |
| *** fawadkhaliq has quit IRC | 18:24 | |
| *** thorst_away has quit IRC | 18:24 | |
| *** asselin_ has joined #openstack-infra | 18:25 | |
| *** salv-orlando has quit IRC | 18:26 | |
| *** salv-orlando has joined #openstack-infra | 18:26 | |
| *** ddieterly[away] has quit IRC | 18:28 | |
| anteaya | rcarrillocruz: reading backscroll, the ideal scenario is to have the gerrit storyboard plugin on the review-dev server | 18:29 |
| nibalizer | jeblair: mordred totally works | 18:29 |
| nibalizer | http://paste.openstack.org/show/509292/ | 18:29 |
| nibalizer | all i did in the 'vim' secion was change the ssh_key value | 18:29 |
| anteaya | rcarrillocruz: I understand your enthusasium for adding gerrit deployment to the list of things you can deploy with ansible | 18:29 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci: Use openstack-infra jenkins users public ssh key https://review.openstack.org/327834 | 18:29 |
| anteaya | rcarrillocruz: but the goal here is for the gerrit storyboard plugin to be on our review-dev server, so that folks can access it and hopefully start to test with it | 18:30 |
| *** ihrachys has joined #openstack-infra | 18:30 | |
| *** Odd_Bloke has joined #openstack-infra | 18:30 | |
| anteaya | rcarrillocruz: folks being storyboard developers with support from infra-root where applicalbe | 18:30 |
| *** ddieterly has joined #openstack-infra | 18:30 | |
| anteaya | applicable | 18:30 |
| *** thorst_away has joined #openstack-infra | 18:30 | |
| *** elmiko has joined #openstack-infra | 18:31 | |
| *** Ng has quit IRC | 18:31 | |
| elmiko | hi, did something change with the openid backend for review.openstack.org? (i am getting an unstyled login page that is unfamiliar and doesn't recognize my login id) | 18:31 |
| *** ramishra has quit IRC | 18:32 | |
| jeblair | nibalizer: do you have an explanation for how it could know to do that and yet know not to remove an existing key (the problem we're trying to solve)? | 18:32 |
| *** sc68cal has quit IRC | 18:32 | |
| *** Ng has joined #openstack-infra | 18:32 | |
| jeblair | nibalizer: oh! | 18:32 |
| jeblair | nibalizer: i know what it is | 18:32 |
| anteaya | elmiko: no, though we did have problems with ubuntuone our single sign on provider the other day | 18:32 |
| jeblair | nibalizer: the problem isn't actually that -- it's *rolling* | 18:33 |
| anteaya | elmiko: have you a url you can share? | 18:33 |
| *** sc68cal has joined #openstack-infra | 18:33 | |
| elmiko | anteaya: i wonder if that's whay i'm hitting | 18:33 |
| jeblair | nibalizer: becase we need two keys at once. | 18:33 |
| *** ayoung has quit IRC | 18:33 | |
| nibalizer | jeblair: i assume it keys on the comment | 18:33 |
| jeblair | nibalizer: add key 1. add key 2. delete key 1. | 18:33 |
| *** hashar has joined #openstack-infra | 18:33 | |
| elmiko | anteaya: https://review.openstack.org/login/%23%2Fq%2Fstatus%3Aopen | 18:33 |
| jeblair | nibalizer: yeah, and rolling is why we need the comments to be different | 18:33 |
| elmiko | when i hit that, i get an unstyled page with broken image links and it says "Provider is not supported, or was incorrectly entered." | 18:34 |
| *** ramishra has joined #openstack-infra | 18:34 | |
| nibalizer | at any rate, i believe this will resolve the issue with testing | 18:34 |
| anteaya | elmiko: I see it and that was not the page we were getting before | 18:34 |
| elmiko | anteaya: this is where the "sign in" link is redirecting me to | 18:35 |
| *** thorst_away has quit IRC | 18:35 | |
| anteaya | elmiko: this is what I see: http://imgur.com/2z6yhgU | 18:36 |
| elmiko | anteaya: just tried a different browser, same effect | 18:36 |
| anteaya | yes, me too that screenshot was in firefox, I see the same in chrome | 18:36 |
| elmiko | yup, same screen that i see | 18:36 |
| *** thorst_away has joined #openstack-infra | 18:36 | |
| *** cody-somerville has joined #openstack-infra | 18:36 | |
| anteaya | can you refresh review.openstack.org and try again? | 18:37 |
| elmiko | yup, 1sec | 18:37 |
| anteaya | elmiko: I was logged in in firefox and just tried to log in with chrome and get the same issue | 18:37 |
| fungi | maybe login.launchpad.net is broken at the moment? | 18:38 |
| *** dimtruck is now known as zz_dimtruck | 18:38 | |
| anteaya | I just joined the ubuntuone channel and was about to share my screenshot | 18:38 |
| *** banix has quit IRC | 18:38 | |
| *** sdake_ has quit IRC | 18:38 | |
| elmiko | anteaya: refresh produces the same result | 18:38 |
| elmiko | fungi: i am getting a good page back from login.lp.net | 18:38 |
| *** mdrabe has quit IRC | 18:39 | |
| *** savihou has joined #openstack-infra | 18:39 | |
| elmiko | weird thing is my launchpad page is showing a login from review.os.org today | 18:39 |
| fungi | same here, but it looks like maybe gerrit is not getting the response it expects when trying to call it. i'm seeing the same behavior testing with review-dev.openstack.org (didn't want to risk getting stuck logged out of review.o.o at the moment) | 18:39 |
| elmiko | fungi: smart move ;) | 18:40 |
| anteaya | posted my screenshot in ubuntuone | 18:40 |
| elmiko | i hadn't been logged in for a few days | 18:40 |
| *** nwkarsten has joined #openstack-infra | 18:40 | |
| *** mdrabe has joined #openstack-infra | 18:40 | |
| fungi | now it's working for me again | 18:40 |
| *** sarob has joined #openstack-infra | 18:41 | |
| EmilienM | jeblair: I had to rebase https://review.openstack.org/#/c/327820/ and I also addressed pabelanger's comment - is it fine now? | 18:41 |
| *** thorst_away has quit IRC | 18:41 | |
| anteaya | fungi: elmiko refreshing works for me too | 18:41 |
| anteaya | elmiko: are you good? | 18:41 |
| *** ihrachys_ has joined #openstack-infra | 18:41 | |
| fungi | hrm, now it's back to broken for me again | 18:41 |
| anteaya | :( | 18:41 |
| elmiko | anteaya: trying again | 18:42 |
| TheJulia | weirdness here as well, getting 401's back on the review sign-in page | 18:42 |
| anteaya | broken for me again too | 18:42 |
| elmiko | anteaya: still failure | 18:42 |
| fungi | maybe the lp sso is under a denial of service attack or something | 18:42 |
| anteaya | I can't find the logs in the ubuntuone channel topic to check backscroll | 18:42 |
| elmiko | fungi: i could believe that, as i'm seeing strange slowness when hitting that url | 18:43 |
| fungi | as am i | 18:43 |
| *** ihrachys has quit IRC | 18:43 | |
| pabelanger | jeblair: success: http://logs.openstack.org/47/327847/2/check/gate-openstackci-beaker-ubuntu-trusty/a07c9cc/console.html | 18:43 |
| *** banix has joined #openstack-infra | 18:43 | |
| tqtran | AJaeger: is there documentation where a new developer can go to for learning how to make his/her code translatable? I could have sworn a saw a really good one, but havent been able to locate it. Would love to point it as a source of reference from the horizon docs. | 18:44 |
| pabelanger | jeblair: going to keep an eye on it for the next few runs | 18:44 |
| *** _sarob has quit IRC | 18:44 | |
| *** jbryce has quit IRC | 18:45 | |
| *** jbryce has joined #openstack-infra | 18:45 | |
| openstackgerrit | Merged openstack-infra/system-config: Remove dependency on think_slave for wheel mirrors https://review.openstack.org/303047 | 18:46 |
| *** ayoung has joined #openstack-infra | 18:47 | |
| *** cody-somerville has quit IRC | 18:47 | |
| *** thorst_away has joined #openstack-infra | 18:48 | |
| *** Sukhdev has joined #openstack-infra | 18:50 | |
| *** thorst_away has quit IRC | 18:52 | |
| nibalizer | jeblair: so what we could do re: rolling I think is pass in an array of keys | 18:53 |
| nibalizer | then sometimes we pass 1 key and sometimes we pass 2 keys maybe | 18:53 |
| nibalizer | so we're just manipulating data not scratching code needlessly | 18:53 |
| *** baoli_ has quit IRC | 18:53 | |
| openstackgerrit | Merged openstack-infra/system-config: Add zl02 https://review.openstack.org/327839 | 18:54 |
| *** zz_dimtruck is now known as dimtruck | 18:54 | |
| *** stevebaker has quit IRC | 18:55 | |
| *** stevebaker has joined #openstack-infra | 18:55 | |
| anteaya | and login to gerrit works for me again, fungi elmiko TheJulia | 18:55 |
| *** esikachev has quit IRC | 18:55 | |
| elmiko | anteaya: trying again | 18:55 |
| elmiko | \o/ | 18:56 |
| anteaya | elmiko: yay | 18:56 |
| elmiko | thanks for the hand-holding anteaya ;) | 18:56 |
| TheJulia | awesome, thank you! | 18:56 |
| anteaya | elmiko: thanks for reporting | 18:56 |
| anteaya | TheJulia: you in now too? | 18:56 |
| TheJulia | yup | 18:57 |
| *** ihrachys has joined #openstack-infra | 18:57 | |
| anteaya | oh good | 18:57 |
| anteaya | yay intermittent issues | 18:57 |
| anteaya | like jumping though the chomping things in galaxy quest | 18:57 |
| elmiko | lol, yes! | 18:57 |
| *** gordc has quit IRC | 18:57 | |
| anteaya | "This episode was badly written. We shouldn't have to do this!" | 18:58 |
| *** kzaitsev_mb has joined #openstack-infra | 18:58 | |
| *** ihrachys_ has quit IRC | 18:58 | |
| *** sdake has joined #openstack-infra | 18:59 | |
| *** e0ne has joined #openstack-infra | 18:59 | |
| *** shamail has joined #openstack-infra | 18:59 | |
| elmiko | <3 that movie | 18:59 |
| anteaya | me too | 19:00 |
| *** thorst_away has joined #openstack-infra | 19:00 | |
| *** weshay is now known as weshay_mtg | 19:00 | |
| *** dimtruck is now known as zz_dimtruck | 19:01 | |
| jeblair | nibalizer: maybe we should go with what you have for now, and explore the idea pabelanger and i were talking about next week -- which is stop using jenkinsuser inside of openstackci::logserver | 19:01 |
| jeblair | nibalizer: we can put the nice rollable stuff in there | 19:01 |
| *** zz_dimtruck is now known as dimtruck | 19:02 | |
| *** e0ne has quit IRC | 19:02 | |
| nibalizer | cool | 19:03 |
| *** tonytan4ever has joined #openstack-infra | 19:04 | |
| *** thorst_away has quit IRC | 19:05 | |
| jeblair | oops, i needed to delete the ansible inventory cache after deleting jenkins01 | 19:05 |
| jeblair | ansible is stuck sshing to it | 19:05 |
| jeblair | killed jenkins01 ansible, puppetmaster ansible is continuning now | 19:05 |
| *** tzn has quit IRC | 19:06 | |
| *** thorst_away has joined #openstack-infra | 19:06 | |
| *** shamail has left #openstack-infra | 19:07 | |
| *** e0ne has joined #openstack-infra | 19:07 | |
| dmsimard | hello -infra, I would appreciate a final +2+W on https://review.openstack.org/#/c/327700/ to help us move forward, thanks :) | 19:10 |
| *** netsin has quit IRC | 19:10 | |
| *** thorst_away has quit IRC | 19:10 | |
| *** pvaneck has quit IRC | 19:10 | |
| *** thorst_away has joined #openstack-infra | 19:10 | |
| *** thorst_away is now known as thorst | 19:10 | |
| *** stajkowski has joined #openstack-infra | 19:12 | |
| *** harlowja has quit IRC | 19:12 | |
| stajkowski | AJaeger: hey, was able to meet all of the requirements you've stated and addressed the -1s, possible we can get a few reviews on this and potentially get it merged? https://review.openstack.org/#/c/319368/4 | 19:13 |
| *** erikmwilson has quit IRC | 19:14 | |
| *** bpokorny has quit IRC | 19:15 | |
| *** erikmwilson has joined #openstack-infra | 19:16 | |
| *** pvaneck has joined #openstack-infra | 19:16 | |
| *** nwkarsten has quit IRC | 19:18 | |
| *** sdake has quit IRC | 19:19 | |
| *** nwkarsten has joined #openstack-infra | 19:19 | |
| *** nwkarsten has quit IRC | 19:19 | |
| *** nwkarsten has joined #openstack-infra | 19:21 | |
| pabelanger | jeblair: nibalizer: sure, works for me | 19:21 |
| *** erikmwilson has quit IRC | 19:21 | |
| *** erikmwilson has joined #openstack-infra | 19:23 | |
| *** cloudtrainme has joined #openstack-infra | 19:24 | |
| notmorgan | jeblair: hehe oops! | 19:26 |
| *** bdemers has quit IRC | 19:27 | |
| pabelanger | dmsimard: keep in mind you are duplicating the running for python-jobs now inside run_test.sh | 19:27 |
| pabelanger | dmsimard: Oh, I see 327888 | 19:28 |
| *** coreyob has quit IRC | 19:30 | |
| pabelanger | jeblair: notmorgan: if you have bandwidth, I've updated 327681 for nodepool again | 19:31 |
| *** regXboi has quit IRC | 19:31 | |
| openstackgerrit | Joe Talerico proposed openstack-infra/project-config: Adding Browbeat https://review.openstack.org/327255 | 19:31 |
| notmorgan | pabelanger: going to eat food and sit outside when reviewing it. but def. on my list for that | 19:31 |
| notmorgan | pabelanger: :) | 19:31 |
| *** Jeffrey4l_ has joined #openstack-infra | 19:31 | |
| pabelanger | notmorgan: good plan, a little chilly here today | 19:32 |
| notmorgan | pabelanger: wahts the temp there? | 19:32 |
| pabelanger | notmorgan: rbot | [Ottawa, ON, CA] Current: Scattered Clouds 63F, 17C (Feels like: 63F, 17C), Humidity: 34%, Wind: From the WNW at 21 MPH Gusting to 30 MPH | 19:33 |
| *** Jeffrey4l has quit IRC | 19:34 | |
| *** xarses has quit IRC | 19:34 | |
| notmorgan | a little chilly | 19:34 |
| notmorgan | not bad though | 19:34 |
| *** Apoorva has quit IRC | 19:34 | |
| anteaya | getting warmer | 19:35 |
| anteaya | was colder yesterday, had a fire on last night | 19:35 |
| notmorgan | it's maybe 60F here atm | 19:35 |
| notmorgan | so not too far off your temps. | 19:35 |
| openstackgerrit | Mathieu Gagné proposed openstack-infra/zuul: Add ability to configure Zuul web interface with zuul.config.js https://review.openstack.org/327892 | 19:35 |
| anteaya | not far | 19:35 |
| pabelanger | little cloudy here, having access to the sun makes it better | 19:35 |
| anteaya | sun is nice | 19:35 |
| anteaya | blue sky here | 19:36 |
| notmorgan | jeblair: i'm planning on doing a bunch of rebase/deconflicting for the webhook things soon. | 19:36 |
| notmorgan | jeblair: once i chase down this annoying unit test bug that reports no logs. | 19:36 |
| AJaeger | tqtran: the horizon docu already points t o | 19:36 |
| AJaeger | tqtran: to http://docs.openstack.org/infra/manual/creators.html#enabling-translation-infrastructure | 19:37 |
| *** esikachev has joined #openstack-infra | 19:38 | |
| *** thorst has quit IRC | 19:39 | |
| *** thorst has joined #openstack-infra | 19:40 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove keystone-propose-config-updates https://review.openstack.org/315130 | 19:40 |
| notmorgan | AJaeger: probably also want to drop the sample config from keystone's git tree with that change landing ^ :) | 19:41 |
| openstackgerrit | Merged openstack-infra/project-config: Move ARA integration jobs to use ./run_tests.sh instead https://review.openstack.org/327700 | 19:41 |
| rcarrillocruz | zaro: got no plans for integration | 19:43 |
| openstackgerrit | Tim Buckley proposed openstack-infra/system-config: Collect 'karma.subunit' files https://review.openstack.org/325369 | 19:43 |
| EmilienM | jeblair: ok https://review.openstack.org/#/c/327820/ is ready to land, let's give a try | 19:44 |
| *** thorst has quit IRC | 19:44 | |
| rcarrillocruz | anteaya read i was looking for streams of work, and she suggested to help out with SB-Gerrit integration, i offered i could automate SB deployment with ansible tied with Gerrit and she agreed to it, so did Zara this morning. But if they can do it with sb-dev that's good | 19:44 |
| rcarrillocruz | whatever is , i'll add to infra-ansible eventually | 19:44 |
| *** mhickey has quit IRC | 19:46 | |
| *** thorst has joined #openstack-infra | 19:46 | |
| *** dtardivel has quit IRC | 19:47 | |
| *** javeriak_ has quit IRC | 19:48 | |
| *** savihou has quit IRC | 19:48 | |
| *** harlowja has joined #openstack-infra | 19:49 | |
| *** e0ne has quit IRC | 19:49 | |
| jeblair | AJaeger, fungi: those openstack-manuals-releasenotes jobs did run on zl01, and the fact that they did not publish logs is related to that: http://paste.openstack.org/show/509300/ | 19:50 |
| *** thorst has quit IRC | 19:50 | |
| dmsimard | pabelanger: oh no, you found my secret review :) | 19:50 |
| jeblair | AJaeger, fungi, pabelanger: that error is baffling (the workspace doesn't exist?) so i'd like us to hold a node when one of those jobs is running. | 19:51 |
| *** javeriak has joined #openstack-infra | 19:51 | |
| dhellmann | fungi , AJaeger : can one of you help me track down what happened to the jobs for the python-tackerclient 0.3.1 release? I don't see any logs at all where I'm expecting them, and it doesn't look like the artifact was built | 19:51 |
| pabelanger | jeblair: ack | 19:51 |
| jeblair | EmilienM: don't you need copy-logs to land in the puppet repo first? | 19:52 |
| *** thorst has joined #openstack-infra | 19:52 | |
| fungi | jeblair: yeah, nothing is immediately coming to mind as to what could have caused that | 19:52 |
| jeblair | fungi: here's a working run i dug up for comparison: http://logs.openstack.org/98/985e4a51ff377156879dff588480e75776b6a2b0/post/openstack-manuals-releasenotes/893f000/console.html | 19:52 |
| jeblair | <shrug> | 19:52 |
| EmilienM | jeblair: right, but it's ok, we're landing it within the hour | 19:52 |
| openstackgerrit | Merged openstack-infra/system-config: Update third party recheck syntax recommendations https://review.openstack.org/327065 | 19:53 |
| fungi | dhellmann: agreed, i find no logs associated with that tag. checking zuul's debug log next | 19:53 |
| dhellmann | fungi : thanks | 19:53 |
| anteaya | rcarrillocruz: zaro what I had said and still say is needed it the storyboard gerrit plugin on the review-dev server | 19:54 |
| fungi | dhellmann: this'll take a few minutes since the logs are enormous and this was in yesterday's log so it's also going to take some time to decompress | 19:54 |
| phschwartz | Does the launch_node.py script usually take a very long time to run? Seems like there is 120+ sec delays between ssh actions with it. | 19:55 |
| dhellmann | fungi : ok | 19:55 |
| jeblair | phschwartz: ansible output is buffered until completion | 19:55 |
| jeblair | fungi: we had a zuul outage yesterday and lost maybe .5 hour of events | 19:56 |
| *** thorst has quit IRC | 19:56 | |
| jeblair | fungi: should be in status wikipage if you want to check timestamps | 19:56 |
| phschwartz | jeblair: ok, good to know, but I am talking about after I know an asible playbook run finished. | 19:56 |
| phschwartz | Output is already there and nothing else is happening till ssh timesout | 19:57 |
| jeblair | phschwartz: unknown problem then | 19:57 |
| * sridhar_ram is here in case to answer any tacker / tackerclient questions | 19:57 | |
| *** javeriak has quit IRC | 19:57 | |
| rcarrillocruz | i didn't say anything about review-dev server and you didn't mention yesterday, you said it was needed to test the plugin with the gerrit version we use. What I said is that i have a tool that automates deployments, that I have gerrit automatically deployed and that it could be 'easy' to just add storyboard | 19:57 |
| dhellmann | jeblair : where is the "status wiki page"? | 19:58 |
| anteaya | rcarrillocruz: okay great, that doesn't not solve the problem I am looking to solve | 19:58 |
| *** tonytan4ever has quit IRC | 19:58 | |
| rcarrillocruz | but is all good | 19:58 |
| jeblair | dhellmann: https://wiki.openstack.org/wiki/Infrastructure_Status | 19:58 |
| dhellmann | jeblair : thanks | 19:58 |
| anteaya | rcarrillocruz: and if I failed to say we need the storyboard gerrit plugin on the review-dev server yesterday than that is my fault and I apologize | 19:58 |
| rcarrillocruz | if others suggested that review-dev is ready to use for that and they're cool with it, wfm | 19:58 |
| EmilienM | jeblair: ok once it's close to land, I'll ping you to +A the infra patch so we're synced | 19:59 |
| anteaya | what storyboard needs is to have the storyboard gerrit plugin on the review-dev server | 19:59 |
| jeblair | EmilienM: cool | 19:59 |
| rcarrillocruz | anteaya: it's all good, no need to apologize :-) | 19:59 |
| rcarrillocruz | just happy to help with wahtever | 19:59 |
| anteaya | rcarrillocruz: well that is the thing i need help with | 19:59 |
| rcarrillocruz | i'm not core, so my hands are a bit tied to mess with sb-dev | 19:59 |
| anteaya | I'm not asking you to mess with the storyboard-dev server | 19:59 |
| anteaya | what I need help with is to have the storyboard gerrit plugin on the review-dev server | 20:00 |
| *** rbrndt has quit IRC | 20:00 | |
| *** xyang1 has joined #openstack-infra | 20:00 | |
| *** szaher has quit IRC | 20:00 | |
| dhellmann | jeblair , fungi : this was at 20:54 UTC so I think it was after the zuul restart | 20:00 |
| zaro | anteaya, rcarrillocruz : i think i may see some daylight soon. i think the first step is to validate what works and what doesn’t before we event attempt to install on review-dev. | 20:00 |
| anteaya | zaro: wonderful thank you | 20:01 |
| zaro | which can be done on local machine. | 20:01 |
| anteaya | zaro: have you any instructions for how to test that on a local machine? | 20:01 |
| zaro | yes, it’s documented. https://gerrit.googlesource.com/plugins/its-storyboard/+/master/src/main/resources/Documentation/quick-install-guide.md | 20:02 |
| dmsimard | A zuul layout/jenkins jobs change merged around 20 minutes ago but the changes don't seem to be effective yet. The "POST" queue seems rather clean, is there something going on ? | 20:02 |
| jeblair | dhellmann, fungi: i'm going to put jenkins.o.o into prepare for shutdown and start zlstatic01. shouldn't affect release team, but wanted to give a heads up that if something goes wrong with proposal/release/wheel-mirror jobs, that's probably where we should look first, and also, that we should keep an eye on it to verify jobs run correctly. | 20:02 |
| *** szaher has joined #openstack-infra | 20:02 | |
| anteaya | zaro: thank you, I'll look at the docs | 20:02 |
| dhellmann | jeblair : thanks for the heads up. dims ^^ | 20:02 |
| *** Apoorva has joined #openstack-infra | 20:03 | |
| zaro | anteaya, rcarrillocruz : lots of those might have change, like the SB rest apis | 20:03 |
| pabelanger | dmsimard: once merged, it takes another 15mins for our puppet wheel to fire off | 20:03 |
| rcarrillocruz | yay | 20:03 |
| anteaya | dmsimard: jenkins job changes take about an hour to propogate out to all the jenkins/zuul launchers running the jobs | 20:03 |
| zaro | if SB rest apis have changed then it require more work obviously | 20:03 |
| dmsimard | anteaya, pabelanger: ahhhh, okay, I thought it was fairly instantaneous after the merge. I'll be patient then, thanks. | 20:03 |
| pabelanger | dmsimard: so, depending on when the merge happens, you could wait up to 29mins for it to apply | 20:03 |
| anteaya | zaro: what makes you think the SB rest apis have changed? | 20:03 |
| *** thorst has joined #openstack-infra | 20:03 | |
| dmsimard | pabelanger: right | 20:03 |
| dmsimard | pabelanger: I thought the jjb/zuul update would run in a post job, not from puppet | 20:04 |
| pabelanger | dmsimard: puppetboard.o.o should show the last time zuul.o.o was updated | 20:04 |
| dmsimard | makes sense | 20:04 |
| zaro | anteaya: i’m not saying i know i’m just saying that if anything between SB gerrit interfaces have changed then the plugin will no longer work | 20:04 |
| *** kushal has joined #openstack-infra | 20:05 | |
| openstackgerrit | Joe Talerico proposed openstack-infra/project-config: Adding Browbeat https://review.openstack.org/327255 | 20:05 |
| zaro | best scenario is that the interface hasn’t changed and plugin still works | 20:05 |
| anteaya | zaro: I don't know of any such changes or any discussion about such changes | 20:05 |
| *** amrith is now known as _amrith_ | 20:05 | |
| anteaya | zaro: I thinnk that is the scenario to assume | 20:05 |
| rcarrillocruz | zaro: it's ok, you can always extend jeepyb | 20:05 |
| * rcarrillocruz crouches | 20:05 | |
| anteaya | zaro: https://storyboard.openstack.org/api/v1/systeminfo | 20:05 |
| anteaya | well I've never successfully got a gerrit running locally | 20:06 |
| *** gordc has joined #openstack-infra | 20:06 | |
| jeblair | #status log jenkins.o.o is in prepare-for-shutdown in preparation for decomissioning. zlstatic01.openstack.org is running and attached to its workers instead. | 20:07 |
| anteaya | rcarrillocruz: so if you want to do that and allow me access I will go through the documentation at https://gerrit.googlesource.com/plugins/its-storyboard/+/master/src/main/resources/Documentation/quick-install-guide.md if that is what zaro thinks is the first step | 20:07 |
| openstackstatus | jeblair: finished logging | 20:07 |
| pabelanger | dmsimard: moving to zuul-launcher will remove the need to run JJB and update jenkins, since zuul-launcher reads the files each time a job launches. It would be interesting to see if we could move those bits into AFS, so the changes are atomic in some POST pipeline | 20:07 |
| anteaya | zaro: unless you have documentation I can follow about getting a gerrit running locally | 20:07 |
| pabelanger | dmsimard: but zuulv3 makes that almost a moot point anyways. Since we are moving configuration into project repos | 20:08 |
| *** thorst has quit IRC | 20:08 | |
| dmsimard | zuulv3 will be awesome :D | 20:08 |
| *** dprince has quit IRC | 20:08 | |
| pabelanger | krotscheck: first problem with stackviz building: http://nodepool.openstack.org/dib.ubuntu-xenial.log | 20:08 |
| pabelanger | krotscheck: one of my fears by putting it into a DIB element | 20:09 |
| *** xarses has joined #openstack-infra | 20:09 | |
| *** bpokorny has joined #openstack-infra | 20:09 | |
| anteaya | zaro: since storyboard uses ubuntuone for login this would be the same username and password used for ubuntuone logging in? https://gerrit.googlesource.com/plugins/its-storyboard/+/master/src/main/resources/Documentation/quick-install-guide.md#its_connection_its_connectionConnection-Configuration | 20:09 |
| krotscheck | pabelanger: ... stackviz? Context? | 20:09 |
| pabelanger | krotscheck: doh, wrong person | 20:09 |
| krotscheck | pabelanger: kk. I'm guessing the dib build failed again? | 20:10 |
| mtreinish | pabelanger: that just looks like it couldn't talk to npmjs.org | 20:10 |
| pabelanger | krotscheck: only on xenial, which you care about | 20:10 |
| timothyb89 | pabelanger: odd... it looks like a combination of several proxy fails and then: ccache: error: Failed to create file /tmp/ccache/tmp/tmp.cpp_stderr.nodepool.4481.dSaml6: Permission denied | 20:10 |
| mtreinish | or whatever the addr for the npm repo is | 20:11 |
| krotscheck | pabelanger: \o. | 20:11 |
| zaro | you’ll need to install both gerrit and storyboard locally. | 20:11 |
| fungi | dhellmann: jeblair: according to this the tarball job failed. for some reason it was assigned to zlstatic01? http://paste.openstack.org/show/509308/ according to nodepool it did indeed run on a single-use node assigned there http://paste.openstack.org/show/509309/ | 20:11 |
| anteaya | zaro: this bit would need to be modified as we would want to associate a gerrit change with a task id not a story id: https://gerrit.googlesource.com/plugins/its-storyboard/+/master/src/main/resources/Documentation/quick-install-guide.md#its_associate_change_its_associate_changeAssociating-Gerrit-Changes https://storyboard.openstack.org/#!/story/2000584 | 20:11 |
| jeblair | pabelanger: (the zuul-launchers don't read it each time -- they still only re-read jobs when told to. but they take 30s to do it) | 20:11 |
| fungi | dhellmann: jeblair: i'm looking next in the zuul logs on zlstatic01 for further enlightenment | 20:11 |
| *** coreyob has joined #openstack-infra | 20:11 | |
| zaro | anteaya: yeah, it’s wasn’t clear when i wrote the plugin what the mapping would be. | 20:12 |
| fungi | dhellmann: jeblair: presumably the zlstatic01 assignment was teh known issue later fixed by https://review.openstack.org/327346 | 20:12 |
| *** ddieterly is now known as ddieterly[away] | 20:12 | |
| anteaya | zaro: fair enough, we have since clarified it better | 20:12 |
| jeblair | fungi: oh, yes there was a brief time where zlstatic01 was accepting nodes because we forgot to tell it not to. | 20:12 |
| pabelanger | mtreinish: timothyb89: so, what if we did this. Any way we could skip failures for stackviz, so we don't break nodepool. And then update JJB to do rebuilds if the stackviz bits are missing for the day? | 20:12 |
| zaro | anteaya: and i think that was just decided at last storyboard mid cycle? | 20:12 |
| anteaya | zaro: when you have time to go through that story if that doesn't answer all your questions do ask | 20:12 |
| anteaya | zaro: correct, at least it was whiteboarded then and the story created | 20:12 |
| jeblair | fungi: i stopped it, though i would have expected it to get relaunched... maybe the way i stopped it, it sent a failure result? | 20:13 |
| anteaya | zaro: individuals may have decided prior to then but we converged and documented at the mid-cycle in May | 20:13 |
| *** esikachev has quit IRC | 20:13 | |
| pabelanger | mtreinish: timothyb89: another option could be to use our AFS mirror for npm. cc fungi | 20:13 |
| mtreinish | pabelanger: which jjb? the point is we don't want any running jobs to go out to the network to pull in the js deps just to build static content | 20:13 |
| zaro | anteaya: yeah, and Zara’s change for the bot thing will help | 20:14 |
| anteaya | zaro: oh it will? she'll be happy to hear you think so | 20:14 |
| pabelanger | mtreinish: right, but right now we are breaking nodepool DIBs if stackviz fails. Which is a bigger issue for me | 20:14 |
| timothyb89 | pabelanger: I'd be in favor of ignoring failures | 20:14 |
| anteaya | zaro: she keeps refering to it as the horrible bot script | 20:14 |
| zaro | yeah, the both script has the mapping she wants | 20:15 |
| *** oanson has quit IRC | 20:15 | |
| pabelanger | mtreinish: timothyb89: is there any way to cache the files to the HDD? We can map it outside the chroot in an effort to minimize what we download | 20:15 |
| *** Sukhdev has quit IRC | 20:15 | |
| zaro | anteaya: which seems more definative than anything else i’ve see | 20:15 |
| zaro | n | 20:15 |
| mtreinish | pabelanger: sure, but this isn't any different than any other package install we do, except it's for js (which makes it much more annoying) | 20:15 |
| *** thorst_ has joined #openstack-infra | 20:15 | |
| jeblair | mtreinish: well, more frequent :) | 20:15 |
| anteaya | zaro: yay | 20:15 |
| mtreinish | pabelanger: to make your life easier I'm fine disabling it on centos :) | 20:15 |
| timothyb89 | mtreinish: fwiw this issue is probably similarly likely to occur on any image | 20:16 |
| zaro | anteaya: anways i think it’s probably going to be quiet a process to validate that the plugin works since you’ll need to install gerrit and SB and get them both to talk to each other. | 20:16 |
| anteaya | zaro: well we already have review-dev and storyboard-dev installed and ready to go | 20:17 |
| zaro | if rcarrillocruz can setup both and working then it would help | 20:17 |
| anteaya | zaro: so installing test instances of both is done | 20:17 |
| anteaya | zaro: why can't we use the storyboard-dev and review-dev instances? | 20:17 |
| *** oanson has joined #openstack-infra | 20:17 | |
| anteaya | why do we have to have one person set up local instances of both? | 20:17 |
| anteaya | I see that direction slowing us down | 20:18 |
| pabelanger | mtreinish: timothyb89: using our npm mirror should be more stable, but will wait for krotscheck or fungi to comment | 20:18 |
| anteaya | zaro: personally I would like to see the current gerrit storyboard plugin on review-dev | 20:18 |
| fungi | dhellmann: jeblair: i think this means it lost connectivity with the node when trying to rsync the logs off? http://paste.openstack.org/show/509310/ | 20:18 |
| mtreinish | pabelanger: except our npm mirror was unreliable and we had to stop using it for jobs (unless things got better and I didn't notice) | 20:18 |
| anteaya | that would speed things up considerably | 20:18 |
| anteaya | plus many folks have access to that instance and could lend dev support | 20:19 |
| zaro | anteaya: fair point. | 20:19 |
| pabelanger | mtreinish: as for package installs, we do cache a lot of things today on nodepool.o.o in an effort to only go to the web as little as possible. In this case, we are hitting npm.org 6 times for the same files | 20:19 |
| anteaya | zaro: so can we agree to put the storyboard gerrit plugin on the review-dev server as a first step? even if you don't have time to do that? | 20:19 |
| *** kgiusti has left #openstack-infra | 20:19 | |
| pabelanger | mtreinish: right, I think were still working on making it stable. But I haven't been involved in that too much | 20:19 |
| dhellmann | fungi : broken pipe sounds like you're right, though "no such file or directory" makes it seem like the target (or source?) weren't there | 20:19 |
| zaro | anteaya: sure i we can agree | 20:19 |
| openstackgerrit | Joe Talerico proposed openstack-infra/project-config: Adding Browbeat https://review.openstack.org/327255 | 20:19 |
| mtreinish | pabelanger: I don't know enough about npm to say how to cache things. But, I expect we should be able to do that too | 20:20 |
| anteaya | zaro: thank you, I appreciate it | 20:20 |
| pabelanger | mtreinish: right, I would expect too | 20:20 |
| zaro | anteaya: although doing with puppet is going to be awful | 20:20 |
| anteaya | zaro: well it is going to be aweful with puppet if we put it into production too | 20:20 |
| anteaya | not puppet in production on review is not an option | 20:20 |
| *** jcoufal has quit IRC | 20:20 | |
| anteaya | so if it is aweful with puppet, we best address that now | 20:21 |
| zaro | anteaya: no i’m not even talking about prod | 20:21 |
| timothyb89 | pabelanger: I'm not sure what can be cached in advance, we could do 100% of the download+build externally if we really wanted | 20:21 |
| pabelanger | timothyb89: mtreinish: in the interm, if we can disable error handing for stackviz to finish our DIBs, I would be most grateful. Then work to adress the caching issue | 20:21 |
| zaro | anteaya: i mean how do you plan to install and configure on review-dev? | 20:21 |
| *** rbrndt has joined #openstack-infra | 20:21 | |
| pabelanger | timothyb89: so, we could do a periodic job and then download the bits with nodepool? | 20:21 |
| anteaya | zaro: let me worry about that, you said you don't have time to help and I respect that | 20:21 |
| *** oanson has quit IRC | 20:22 | |
| anteaya | zaro: it took me almost a week to get agreement on step one | 20:22 |
| timothyb89 | pabelanger: that could be an option, though at that point I'd maybe prefer to just use https://etherpad.openstack.org/p/stackviz-deployment (in terms of benefit vs added complexity) | 20:22 |
| anteaya | zaro: let me relish the moment | 20:22 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: make get_env safer https://review.openstack.org/327921 | 20:23 |
| zaro | anteaya: ok, agreed. i’m not opposed to others installing the plugin on review-dev if they so wish | 20:23 |
| anteaya | zaro: great thank you | 20:24 |
| jeblair | fungi, pabelanger: we need to land 327921 for zlstatic01 -- the locale file on proposal.slave has a comment in it | 20:24 |
| pabelanger | timothyb89: interesting idea. How far along are you before stackviz.o.o could be launched? | 20:24 |
| anteaya | zaro: also something interesting the first time I tried to log into review-dev I got a page that said I didn't have permissions to access that page, second time I got logged in | 20:24 |
| pabelanger | jeblair: looking | 20:24 |
| anteaya | it was odd, first time I've seen that, I wonder if it is a remnant of the login hiccup we had earlier today | 20:25 |
| jeblair | fungi: yeah, sounds like it. i'll see what i can do about making sure that a hard stop causes jobs to be relaunched. | 20:25 |
| timothyb89 | pabelanger: I think it's done, to be honest, we'd just need to port my docker stuff over to puppet configs | 20:25 |
| zaro | anteaya: yes, that is correct. you need to submit a change to give you access then it get’s voted on like anything else. | 20:25 |
| anteaya | zaro: to log in? | 20:26 |
| fungi | jeblair: one other thing worth noting, nodepool didn't delete that node until 2016-06-09 02:49:00 (5.75 hours after the job failed) | 20:26 |
| zaro | anteaya: ohh you mean to gerrit not review-dev? | 20:26 |
| timothyb89 | pabelanger: if we wanted we could replace the devstack-gate stuff with a little html redirect and call it good | 20:26 |
| jeblair | fungi: maybe we killed the rsync | 20:26 |
| anteaya | zaro: I mean to log in as a gui user on review-dev | 20:26 |
| zaro | anteaya: gerrit times out on review-dev due to lack of activity. you just need to refresh | 20:26 |
| anteaya | zaro: I wasn't talking about adding my ssh key to the review-dev server | 20:26 |
| anteaya | zaro: ah, thank you | 20:27 |
| jeblair | anteaya: there's a database timeout that usually causes the first 4 requests to review-dev every day to fail. we never see it in production because we're too busy. | 20:27 |
| anteaya | zaro: yes, refresh did it | 20:27 |
| anteaya | jeblair: wonderful thank you | 20:27 |
| pabelanger | timothyb89: So standing up a new server will likely take a few days, if not week(s). Which could be the end game. I do like the idea of a periodic job to deal with the download+build external process. That would be uploaded to logs.o.o and we simply fetch it in our stackviz elements. | 20:27 |
| *** cody-somerville has joined #openstack-infra | 20:27 | |
| fungi | jeblair: doesn't look like it. z-l logged the onFinalized event immediately anyway | 20:27 |
| pabelanger | timothyb89: we can start with an experimental job first, then move it to periodic | 20:28 |
| fungi | jeblair: within a second of the rsync error being logged | 20:28 |
| *** _ari_ has quit IRC | 20:28 | |
| jeblair | fungi: no, i think we reported our own abort as a failure: | 20:29 |
| jeblair | 2016-06-08 21:05:08,745 DEBUG zuul.NodeWorker.ubuntu-trusty-ovh-bhs1-1521940: Abort: sending kill signal to job process group | 20:29 |
| anteaya | fungi: zaro has provided this helpful guide to installing the storyboard gerrit plugin or a gerrit server: https://gerrit.googlesource.com/plugins/its-storyboard/+/master/src/main/resources/Documentation/quick-install-guide.md#its_associate_change_its_associate_changeAssociating-Gerrit-Changes I would like to follow this guide, what would be your suggestions for my best way to proceed (subtext, may I have access to the revi | 20:29 |
| anteaya | ew-dev server?) | 20:29 |
| fungi | jeblair: oh, i wondered what that meant | 20:29 |
| jeblair | fungi: though, er... that should affect the main job, not the post job | 20:29 |
| timothyb89 | pabelanger: I'm unfamiliar with periodic jobs ... what all would making one entail? | 20:29 |
| jeblair | fungi: oh... | 20:29 |
| jeblair | fungi: i wonder if it's because the post playbook reused the ssh controlmaster started by the main playbook | 20:30 |
| fungi | anteaya: what do you mean by access? are you saying we can't use configuration management to install it? | 20:30 |
| pabelanger | timothyb89: we'd basically add a job into the periodic pipeline in zuul/layout.yaml. The jobs would function the same as check / gate but run once a day | 20:30 |
| *** kzaitsev_mb has quit IRC | 20:30 | |
| *** aeng has joined #openstack-infra | 20:30 | |
| anteaya | fungi: I don't know if we can use configuration management | 20:30 |
| jeblair | fungi: but the kill -9 to the process group took that out with everything else | 20:30 |
| fungi | anteaya: any solution which we can't install via some sort of automation or configuration management is a no-go for us anyway | 20:30 |
| anteaya | fungi: I can certainly go that route if that is your preference, which it sounds like it is, since zaro is uncertain if the plugin even works in the first place | 20:31 |
| pabelanger | timothyb89: periodic-package-afs-centos7 is an example of how we do daily RPM builds for the openafs client for centos-7. I would see stackviz working almost the same | 20:32 |
| fungi | anteaya: if you want to try and figure out whether the plugin is even working, then that would be the time to do it locally on your own gerrit deployment where you can play around with it and figure out what needs to be puppeted. rolling it out on review-dev would be to demo it with storyboard-dev longer term | 20:32 |
| fungi | and would need proper configuration management | 20:32 |
| anteaya | fungi: do we use the same puppet module for both review review-dev? | 20:32 |
| fungi | anteaya: yes | 20:32 |
| timothyb89 | pabelanger: that would be a nice improvement... I'll take a look | 20:32 |
| anteaya | fungi: okay thanks | 20:32 |
| fungi | anteaya: but we have separate classes for them in system-config | 20:32 |
| anteaya | fungi: ah ha, thank you that was what I was looking for | 20:33 |
| fungi | jeblair: that sounds likely based on the logs (and unfortunate) | 20:33 |
| *** cody-somerville has quit IRC | 20:33 | |
| jeblair | fungi: i think it will be easy to fix, hacking that up now | 20:33 |
| *** baoli has joined #openstack-infra | 20:34 | |
| *** mixos has quit IRC | 20:34 | |
| fungi | dhellmann: sridhar_ram: in the meantime, i'm going to reenqueue that tag in zuul. you're reasonably confident branch tarballs are being correctly generated for python-tackerclient in post? | 20:34 |
| *** ddieterly[away] is now known as ddieterly | 20:35 | |
| dhellmann | fungi : let me double check that we've done one before | 20:35 |
| *** rcernin has joined #openstack-infra | 20:36 | |
| dhellmann | fungi : yes, 0.3.0 has a tarball | 20:36 |
| sridhar_ram | fungi: not sure if this answers, we had no issues in getting tacker tarballs in tarballs.o.o. | 20:36 |
| *** elmiko has left #openstack-infra | 20:36 | |
| dhellmann | so unless something broken between 0.3.0 and 0.3.1 it should be fine | 20:36 |
| fungi | yep, i see a master branch tarball from yesterday too, so presumably it's fine | 20:36 |
| dhellmann | and there's -- yeah, I was just checking that | 20:37 |
| fungi | punted it back to zuul again | 20:37 |
| fungi | fingers crossed | 20:37 |
| anteaya | zaro: is review-dev running on trusty? | 20:38 |
| jeblair | fungi: i have stopped zlstatic01, so that's being queued | 20:38 |
| jeblair | fungi: i'm going to land 327921 and restart zlstatic01 with it. | 20:39 |
| jeblair | fungi: if you are in a hurry, i can unpause jenkins and let it run that one. | 20:39 |
| dhellmann | fungi : thanks! | 20:39 |
| jeblair | fungi: but if we can wait 10-20 mins, id like to run it on zlstatic | 20:39 |
| *** gouthamr has quit IRC | 20:40 | |
| jeblair | dhellmann: ^ | 20:40 |
| fungi | jeblair: yeah, it can presumably wait. the tarball job needs to complete first anyway and that won't use zlstatic01 | 20:40 |
| jeblair | good point | 20:40 |
| *** kushal has quit IRC | 20:40 | |
| fungi | it just happened to use zlstatic01 last night because we hadn't yet told it not to put single-use nodes there | 20:40 |
| zaro | anteaya: yes | 20:42 |
| *** dimtruck is now known as zz_dimtruck | 20:42 | |
| anteaya | zaro: thank you, thought so, just didn't want to assume | 20:43 |
| *** abregman|afk has quit IRC | 20:43 | |
| pabelanger | I see zl02 in graphite.o.o :D | 20:43 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: don't report result on abort https://review.openstack.org/327938 | 20:44 |
| jeblair | fungi: ^ that should take care of that | 20:44 |
| jeblair | so a 'zuul-launcher graceful' will let all jobs complete before stopping, so no user-visible impact. 'zuul-launcher stop' will kill all of them immediately, so there will be a delay as jobs are re-launched, but no random failures for users. | 20:45 |
| jeblair | notmorgan: are you ready for a gear release? | 20:46 |
| fungi | jeblair: that's awesome. so we have less-wasteful vs faster as the tradeoff, not impacting vs non-impacting? | 20:47 |
| jeblair | ya | 20:48 |
| jeblair | except for folks who watch the status page and notice their jobs get rescheduled | 20:48 |
| pabelanger | very cool indeed | 20:48 |
| pabelanger | also make rolling updates awesome! | 20:48 |
| pabelanger | zuul-launcher stop, kill server, launch new server, profit | 20:49 |
| *** thiagop-planning is now known as thiagop | 20:49 | |
| notmorgan | jeblair: i am | 20:50 |
| notmorgan | jeblair: :) | 20:50 |
| jeblair | oh neat, zl02 has been running for a while :) | 20:50 |
| jeblair | seems to be working | 20:50 |
| jeblair | i will gracefully stop zl01 now | 20:50 |
| pabelanger | jeblair: I was just going to say, the nodepool is not updating the state to used | 20:50 |
| pabelanger | for zl02 | 20:51 |
| *** esikachev has joined #openstack-infra | 20:51 | |
| pabelanger | http://grafana.openstack.org/dashboard/db/nodepool | 20:51 |
| jeblair | pabelanger: oh, probably needs a restart | 20:51 |
| jeblair | pabelanger: are we all caught up on nodepool patches? | 20:51 |
| *** banix has quit IRC | 20:51 | |
| *** HeOS has joined #openstack-infra | 20:51 | |
| pabelanger | 1 sec | 20:51 |
| pabelanger | would like to land 327681 if possible | 20:51 |
| *** Goneri has quit IRC | 20:51 | |
| pabelanger | otherwise, happy to wait | 20:51 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: make get_env safer https://review.openstack.org/327921 | 20:52 |
| phschwartz | jeblair: for the launch node, what is the purpose of the reboot? It appears that is what causes the issue with the run and the instnace | 20:52 |
| phschwartz | *instance | 20:52 |
| notmorgan | pabelanger: that patch looks reasonable to me. i have been looking at zuul code so swapping context is a bit hard. | 20:52 |
| notmorgan | pabelanger: but it doesn't look insane or doing anything weird we wouldn't want anyway. | 20:53 |
| jeblair | phschwartz: aww, i didn't -1 ps1, but i have to ps2 :( | 20:53 |
| jeblair | pabelanger: ^ | 20:53 |
| jeblair | sorry tab fail | 20:53 |
| phschwartz | lol | 20:54 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config: Extend regexp to really include https://review.openstack.org/327942 | 20:54 |
| jeblair | phschwartz: because we've done an apt-get upgrade, and all kinds of configuration that might not take effect until boot | 20:54 |
| pabelanger | jeblair: Oh, neat. TIL | 20:54 |
| phschwartz | jeblair: ah, so I need to find out why the reboot is dying | 20:54 |
| phschwartz | check | 20:54 |
| notmorgan | pabelanger: yeah sets are awesome | 20:54 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Add the devstack-logs publisher to the ARA jobs https://review.openstack.org/327943 | 20:55 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Set nodepool.manager.MANAGER.nodes.STATE to zero https://review.openstack.org/327681 | 20:55 |
| pabelanger | notmorgan: indeed, I'll start using them more | 20:55 |
| notmorgan | pabelanger: it's even cooler when you do supersets, subsets, intersections, difference, etc | 20:56 |
| pabelanger | Ya, I can see that | 20:56 |
| notmorgan | pabelanger: and i am also guilty of using frozensets for order-inspecific collections of constants | 20:57 |
| openstackgerrit | Tim Buckley proposed openstack-infra/project-config: Create new project karma-subunit-reporter https://review.openstack.org/327945 | 20:57 |
| *** kzaitsev_mb has joined #openstack-infra | 20:58 | |
| *** annegentle has quit IRC | 20:59 | |
| *** cody-somerville has joined #openstack-infra | 20:59 | |
| openstackgerrit | Tim Buckley proposed openstack-infra/project-config: Create new project karma-subunit-reporter https://review.openstack.org/327945 | 21:00 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: add tool for watching release jobs https://review.openstack.org/327948 | 21:01 |
| timothyb89 | mtreinish, krotscheck: ^^ | 21:01 |
| *** eharney has quit IRC | 21:02 | |
| openstackgerrit | Merged openstack-infra/nodepool: Set nodepool.manager.MANAGER.nodes.STATE to zero https://review.openstack.org/327681 | 21:02 |
| *** maishsk_ has joined #openstack-infra | 21:02 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: initialize pid variables https://review.openstack.org/327950 | 21:02 |
| jeblair | pabelanger: can you +3 https://review.openstack.org/327950 ? | 21:03 |
| jeblair | i just noticed that when i tried out 'release' on zl01 | 21:03 |
| pabelanger | jeblair: done | 21:04 |
| jeblair | anyone else want to look at or +3 https://review.openstack.org/327938 ? | 21:04 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openstackci: Expose gearman_check_job_registration setting https://review.openstack.org/327952 | 21:04 |
| dims | jeblair : are we done with "put jenkins.o.o into prepare for shutdown and start zlstatic01"? is it safe now? (cc @dhellmann) | 21:04 |
| *** psilvad has quit IRC | 21:05 | |
| *** maishsk has quit IRC | 21:05 | |
| *** maishsk_ is now known as maishsk | 21:05 | |
| jeblair | dims, dhellmann, fungi: i'm about to start zlstatic01 again, we'll see how it goes | 21:06 |
| dims | ack. i don't mind waiting | 21:06 |
| jeblair | looks like a proposal job just succeded | 21:07 |
| jeblair | hrm, do we need to update the firewall on the static slaves? | 21:07 |
| *** kzaitsev_mb has quit IRC | 21:08 | |
| jeblair | (for console streaming) | 21:08 |
| jeblair | a mirror build job just failed strangely | 21:08 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: DNM: Trying to extract frame info on failure https://review.openstack.org/327956 | 21:08 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Stop zuul from checking if jobs are registered https://review.openstack.org/327957 | 21:08 |
| *** kzaitsev_mb has joined #openstack-infra | 21:09 | |
| *** PollyZ has quit IRC | 21:09 | |
| dhellmann | jeblair : is the source for http://status.openstack.org/zuul/ in the zuul repo, or somewhere else? | 21:10 |
| *** PollyZ has joined #openstack-infra | 21:10 | |
| pabelanger | watching zlstatic01.o.o now | 21:10 |
| jeblair | ah, see the problem with the wheel thing | 21:10 |
| jeblair | dhellmann: system-config | 21:10 |
| jeblair | dhellmann: zuul.openstack.org is in zuul | 21:10 |
| dhellmann | jeblair : cool. if I can figure out how to make that javascript page filter on queue, in addition to the pattern-based filter it has now, would that be a feature you'd accept? | 21:11 |
| dhellmann | I'd like to make it show all release and tag jobs, and ignore everything else, for example | 21:11 |
| *** vgridnev has quit IRC | 21:11 | |
| jeblair | dhellmann: sadly, since we seem to be permanently mid-transition, we would need to update both sites | 21:12 |
| jeblair | dhellmann: however, since we are *mostly* through the transition, they should be *very* similar at this point :) | 21:12 |
| dhellmann | so should I only update the zuul repo? | 21:12 |
| jeblair | so hopefully just a dull copy/paste exercise | 21:12 |
| * dhellmann didn't know we had zuul.openstack.org as a UI | 21:12 | |
| jeblair | dhellmann: i don't know that anyone is actively working on the transition anymore | 21:12 |
| *** baoli has quit IRC | 21:13 | |
| dhellmann | ok | 21:13 |
| jeblair | but i really don't want to regress, since i'm certain it will get picked up again | 21:13 |
| *** baoli has joined #openstack-infra | 21:13 | |
| jeblair | dhellmann: so my recommendation would be to do both, but start with zuul.o.o | 21:14 |
| dhellmann | ok, wfm | 21:14 |
| * dhellmann adds another project to his backlog | 21:14 | |
| jeblair | (and hopefully just copy/paste into system-config) | 21:14 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: don't report result on abort https://review.openstack.org/327938 | 21:14 |
| notmorgan | jeblair: cause i'm watching this thing, it isn't going to fail for me now :( | 21:15 |
| *** rcernin has quit IRC | 21:15 | |
| *** burgerk has quit IRC | 21:15 | |
| *** ldnunes has quit IRC | 21:15 | |
| jeblair | dhellmann: it would be nice if at the same time, you also added a filter in the json endpoint for just a given pipeline. there's already such a thing there to get only the json for a given change, so the framework should be easy to extend | 21:15 |
| jeblair | dhellmann: that would mean far less sending of 5MB json files if you only want to watch a pipeline | 21:16 |
| jeblair | dhellmann: having said that, the javascript stuff isn't set up for that | 21:16 |
| jeblair | dhellmann: so maybe that would be a big change. :| | 21:16 |
| jeblair | dhellmann: something to think about :) | 21:16 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Fix typo in centos-7 wheel mirror worker name https://review.openstack.org/327963 | 21:17 |
| jeblair | pabelanger: ^ whoops | 21:17 |
| jeblair | dhellmann, dims, fungi: zlstatic seems to be running jobs okay, aside from centos-7 wheel builds | 21:17 |
| pabelanger | jeblair: +2 | 21:17 |
| fungi | dhellmann: sridhar_ram: i see a release announcement for it now, so i guess it worked when i reenqueued the tag | 21:17 |
| fungi | jeblair: ^ | 21:17 |
| dims | thanks jeblair | 21:17 |
| dhellmann | jeblair : ok, I'll think about those other changes at the same time | 21:17 |
| *** Sukhdev has joined #openstack-infra | 21:17 | |
| dhellmann | fungi, sridhar_ram, dims : great! thanks for your help with that | 21:18 |
| sridhar_ram | fungi: great thanks! | 21:18 |
| sridhar_ram | dhellmann: thanks for your help ! | 21:18 |
| *** ajmiller has quit IRC | 21:18 | |
| jeblair | fungi: http://logs.openstack.org/16/1665ad79cc479f0c9d744076cc7ad4abe2eca808/release/python-tackerclient-pypi-both-upload/f902134/console.html | 21:18 |
| sridhar_ram | infra-team - i need help in changing the description in github for tacker... https://github.com/openstack/tacker. It is awfully wrong | 21:19 |
| pabelanger | jeblair: I wonder if we should reuse UNSTABLE for v25 to indicate there is a problem with publisher tasks? As an easy way to identify upload problems | 21:19 |
| jeblair | pabelanger: well, i'd prefer "POST_PLAYBOOK_FAILURE" myself :) | 21:20 |
| pabelanger | jeblair: I would not object to that | 21:20 |
| *** kzaitsev_mb has quit IRC | 21:20 | |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: initialize pid variables https://review.openstack.org/327950 | 21:20 |
| fungi | jeblair: yay, logs and everything! | 21:21 |
| fungi | did anybody ever get a review up for osla to fix the multi-line matching issue there? | 21:21 |
| jeblair | pabelanger: i'm going to restart nodepool now that the change has landed | 21:21 |
| jeblair | fungi: nak | 21:21 |
| *** csomerville has joined #openstack-infra | 21:22 | |
| *** nemish has quit IRC | 21:22 | |
| *** mixos has joined #openstack-infra | 21:23 | |
| *** trown is now known as trown|outtypewww | 21:24 | |
| anteaya | zaro: what kind of value is gerrit expecting from lp_sync_consumer_key in puppet? a string? an int? | 21:24 |
| anteaya | zaro: I gave it a madeup string and it doesn't seem to like that | 21:24 |
| notmorgan | ooh i might have a wedged worker | 21:25 |
| jeblair | dhellmann: you're about to be able to have a terminal window up where you can constantly see the console output from release.slave.openstack.org all the time | 21:25 |
| notmorgan | what's the job running hard-timeout? 120m? | 21:25 |
| jeblair | dhellmann: is that interesting to you? | 21:25 |
| dhellmann | jeblair : what is that server? | 21:25 |
| *** cody-somerville has quit IRC | 21:25 | |
| jeblair | dhellmann: it's the 'jenkins' slave that uploads releases to pypi | 21:26 |
| jeblair | dhellmann: it's this output: http://logs.openstack.org/16/1665ad79cc479f0c9d744076cc7ad4abe2eca808/release/python-tackerclient-pypi-both-upload/f902134/console.html | 21:26 |
| dhellmann | Ah. I don't really want to watch these things run. What I want is some sort of notification if they fail. | 21:26 |
| jeblair | dhellmann: you will be able to pipe it to grep. | 21:26 |
| zaro | anteaya: are you setting up locally? | 21:26 |
| zaro | anteaya: you don’t need LP for testing with SB | 21:27 |
| jeblair | dhellmann: you want email when those jobs fail? | 21:27 |
| anteaya | zaro: i'm trying, I have limited hope of success but trying to get the puppet module to work | 21:27 |
| dhellmann | jeblair : I'm trying to anticipate a world where I do releases by approving patches in gerrit. In that situation, a push notification like an irc bot or email would be better. | 21:27 |
| anteaya | zaro: so I can just remove those parameters from the .pp file? | 21:27 |
| anteaya | zaro: I'll do that | 21:27 |
| dhellmann | jeblair : is it easy to add email or IRC? either is good, and we can set up a release manager list to receive the email if that's simplest. | 21:28 |
| jeblair | dhellmann: email is easy. irc is not written yet. | 21:28 |
| *** PollyZ has quit IRC | 21:28 | |
| dhellmann | email is great | 21:28 |
| zaro | anteaya: yes, it’s not needed in gerrit.conf | 21:28 |
| anteaya | zaro: awesome thank you | 21:28 |
| dhellmann | jeblair : who do I talk to to get a new mailman list set up, and what can I do to help set up the email notifications? | 21:28 |
| dhellmann | jeblair : I didn't think this was easy, or I would have brought it up before. I'm highly motivated to do it now. :-) | 21:29 |
| jeblair | dhellmann: mailman is a system-config patch, and notifications are a zuul layout change in project-config | 21:29 |
| dhellmann | cool, I'll look at both of those things tomorrow and drop in here if I can't figure it out on my own | 21:29 |
| *** edmondsw has quit IRC | 21:29 | |
| jeblair | dhellmann: ++ | 21:30 |
| dhellmann | jeblair: thanks, I think this will make it much easier to track issues with release failures | 21:30 |
| anteaya | zaro: it was my fault I had a syntax error, anyway good to know those parameters aren't needed | 21:31 |
| *** annegentle has joined #openstack-infra | 21:31 | |
| *** _amrith_ is now known as amrith | 21:31 | |
| *** matt-borland has quit IRC | 21:31 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Open port 8088 on long-lived slaves https://review.openstack.org/327969 | 21:32 |
| jeblair | pabelanger: ^ | 21:32 |
| *** ayoung has quit IRC | 21:34 | |
| *** piet has joined #openstack-infra | 21:34 | |
| *** ayoung has joined #openstack-infra | 21:35 | |
| *** kzaitsev_mb has joined #openstack-infra | 21:35 | |
| *** nmagnezi has quit IRC | 21:36 | |
| pabelanger | jeblair: ack to nodepool restart | 21:36 |
| pabelanger | jeblair: and looking at 327969 | 21:36 |
| *** NM has joined #openstack-infra | 21:36 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: add private gearman function https://review.openstack.org/327422 | 21:37 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: remove deprecated hyphens https://review.openstack.org/327347 | 21:37 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: Add graceful stop command https://review.openstack.org/325523 | 21:37 |
| pabelanger | neat, patch is working now for http://grafana.openstack.org/dashboard/db/nodepool | 21:37 |
| jeblair | hrm. i'm still not seeing any zl02 zmq activity on nodepool | 21:38 |
| openstackgerrit | Merged openstack-infra/zuul: Make isJobRegistered for gearman optional https://review.openstack.org/327323 | 21:38 |
| EmilienM | jeblair: go | 21:39 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openstackci: Expose gearman_check_job_registration setting https://review.openstack.org/327952 | 21:39 |
| jeblair | EmilienM: done | 21:39 |
| EmilienM | thx! | 21:39 |
| jeblair | EmilienM: thank you :) | 21:39 |
| *** zul has quit IRC | 21:39 | |
| timothyb89 | could any infra cores take a look at https://review.openstack.org/#/c/325369/ ? thanks! | 21:40 |
| jeblair | oh, zl02 is not in nodepool.yaml | 21:40 |
| jeblair | forgot the project-config change | 21:40 |
| pabelanger | woops | 21:40 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Add zl02 to nodepool https://review.openstack.org/327972 | 21:41 |
| jeblair | pabelanger: instapprove ^? | 21:41 |
| pabelanger | yes please | 21:41 |
| jeblair | enqueued | 21:41 |
| jeblair | pabelanger: are you done with the held nodes from earlier? | 21:43 |
| clarkb | jroll: vsaienko I will ask again. does grenade support os test timeout? | 21:43 |
| jeblair | pabelanger: i can delete them if so | 21:43 |
| pabelanger | jeblair: Yup, we can delete it | 21:45 |
| clarkb | jroll: vsaienko my readi g of grenade is that it wont honor that value | 21:46 |
| clarkb | sdague: ^ you +2'd the change do you k ow? | 21:46 |
| JayF | clarkb: look at the depends-on change, if you're talking about the patch I think you are | 21:47 |
| clarkb | JayF: 316662 there is no depends on and that change does not work if grenadr doesnt explicitly pass that var through I dont think | 21:48 |
| clarkb | but I am not a greande expert so was hoping someone else cpuld provide more info | 21:48 |
| JayF | I know that we have grenade passing with a dummy change with that in depends-on, and it doesn't pass without it | 21:49 |
| JayF | and we've gotten about a dozen various things merged in to make this work | 21:49 |
| *** njohnston is now known as njohnston|afk | 21:49 | |
| JayF | I can probably find the commit in question that added support though | 21:49 |
| clarkb | and when I responded to the please review here in irc this question response went unanswered | 21:49 |
| EmilienM | jeblair: I'll monitor puppet CI tonight and make some adjustments if needed in project-config | 21:49 |
| JayF | clarkb: yeah, I want that merged, and I know jroll is gone and vsaienko is not here yet so I'll find the proof you need to get that landed | 21:50 |
| openstackgerrit | Merged openstack-infra/zuul: Python 3 Fixes: use six.moves.configparser https://review.openstack.org/323020 | 21:50 |
| jeblair | EmilienM: cool, there's still a 1/8 chance of the job running on a zuul launcher vs jenkins | 21:50 |
| *** nwkarsten has quit IRC | 21:50 | |
| JayF | clarkb: had I seen your question before i would've tried to answer then :( | 21:50 |
| jeblair | EmilienM: you can tell the difference in the console logs for the job; zuul says "[Zuul]" in a couple of places | 21:50 |
| EmilienM | jeblair: we have enough jobs/patches to hit it | 21:50 |
| clarkb | JayF: its ok I shpuldve responded on the change but thought that I would get quicker response in irc since it was right after review request | 21:51 |
| EmilienM | jeblair: ok good to know | 21:51 |
| clarkb | JayF: basically the question is will that var get sanitized out by sudo and or tox like it does with normal devstack runa | 21:51 |
| anteaya | zaro: this is my log from my puppet run, if you see a line repeated that is my error in my copy/pasting: http://paste.openstack.org/show/509316/ | 21:51 |
| clarkb | if not then yay else :( you can probably look in your test logs if you have a change that deps on 3162 | 21:52 |
| JayF | clarkb: yeah, I don't know these changes very well so it'll take me a sec to get context; I do know that it works with and does not work without that change | 21:52 |
| anteaya | zaro: if puppet was successful, gerrit would be running and I would get something returned from ps aux | grep gerrit, would i not? | 21:52 |
| JayF | clarkb: so I'm certain it's effective but I can prove it in code, and I should probably know where it is too | 21:52 |
| *** yamahata has quit IRC | 21:52 | |
| pabelanger | jeblair: I was also thinking we could swap out the [Zuul] tag with [self.hostname], to better indicate which zuul-launcher fire off the job. Or, add some blob to the top of the log files with launcher details | 21:52 |
| *** gordc has quit IRC | 21:53 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove copy_puppet_logs publisher https://review.openstack.org/327820 | 21:53 |
| *** zul has joined #openstack-infra | 21:54 | |
| *** amitgandhinz has quit IRC | 21:54 | |
| *** amitgandhinz has joined #openstack-infra | 21:54 | |
| openstackgerrit | Merged openstack-infra/project-config: Add zl02 to nodepool https://review.openstack.org/327972 | 21:55 |
| notmorgan | pabelanger: i like that | 21:55 |
| notmorgan | pabelanger: knowing which launcher would be fantastic | 21:55 |
| *** esikachev has quit IRC | 21:56 | |
| *** kzaitsev_mb has quit IRC | 21:56 | |
| anteaya | zaro: top also does not show gerrit running | 21:56 |
| openstackgerrit | Merged openstack-infra/system-config: Fix typo in centos-7 wheel mirror worker name https://review.openstack.org/327963 | 21:56 |
| jlk | Hey all | 21:57 |
| notmorgan | pabelanger: i'm inclined to say it should be at the top of the log, i worry changing the [Zuul] tag would make it harder to do matching unless a new tag specifically for the launcher host was added? | 21:57 |
| jlk | so the blue box cloud is all rebuilt on Kilo, and we're ready for y'all to have it back. | 21:57 |
| zaro | anteaya: it looks like there are some failures creating the db | 21:57 |
| jlk | I could turn it on in nodepool, just not sure if you needed to do any more prep work before its usable for ya | 21:57 |
| anteaya | jlk: maybe turn one node on in nodepool to start? | 21:57 |
| anteaya | zaro: thanks | 21:58 |
| clarkb | jlk: did the upgrade drop all existing tenants and images and networks? if not then should be fine to turn on | 21:58 |
| zaro | anteaya: install assumes connection to mysql db somewhere. | 21:58 |
| *** weshay_mtg has quit IRC | 21:58 | |
| *** ddieterly is now known as ddieterly[away] | 21:58 | |
| *** amotoki has joined #openstack-infra | 21:58 | |
| jlk | clarkb: yeah it dropped everything. unfortunately it was a very early install in softlayer, and it was from a release we have no upgrade path from. I had previously discussed with mordred and others that we would do a full rebuild and you'd get a fresh new cloud. | 21:58 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Add POST_PLAYBOOK_FAILURE status https://review.openstack.org/327978 | 21:59 |
| clarkb | ok in that case wewill need to rwbuild all the user level stuff | 21:59 |
| clarkb | tenants, images, networks | 21:59 |
| pabelanger | notmorgan: jeblair: ^ that should provide better debug information | 21:59 |
| *** amotoki has quit IRC | 21:59 | |
| jeblair | notmorgan, pabelanger: yeah, let's leave "[Zuul]" as meaning "zuul says this" and add identity info to the top of the job if we want it | 21:59 |
| pabelanger | in the case of publisher failures | 21:59 |
| *** amotoki has joined #openstack-infra | 21:59 | |
| pabelanger | jeblair: notmorgan: WFM | 21:59 |
| zaro | anteaya: you’ll want to replace [database] block with type = h2 and database = db/ReviewDB in gerri.config | 21:59 |
| anteaya | zaro: thank | 22:00 |
| notmorgan | pabelanger: yeah playbook error is much more descriptive than UNSTABLE | 22:00 |
| notmorgan | PST_PLAYBOOK_ERROR * | 22:00 |
| *** kzaitsev_mb has joined #openstack-infra | 22:00 | |
| jeblair | yeah. though now that i see it "POST_FAILURE" would probably work just as well. | 22:01 |
| jeblair | bikeshed | 22:01 |
| notmorgan | jeblair: BLUE! | 22:01 |
| jeblair | notmorgan: that's jenkins for "green" :) | 22:01 |
| notmorgan | jeblair: heheh :) | 22:01 |
| jeblair | omg i made a jenkins joke | 22:01 |
| notmorgan | jeblair: was just going to say | 22:01 |
| *** tlian2 has quit IRC | 22:02 | |
| JayF | clarkb: from a patch taht depends-on the review in question: http://logs.openstack.org/09/311109/66/experimental/gate-grenade-dsvm-ironic/e2b7fed/console.html#_2016-06-07_16_17_30_223 it's definately effective; but it doesn't look like there was any explicit change in grenade to enable it to work | 22:02 |
| openstackgerrit | Jesse Keating proposed openstack-infra/project-config: Revert "Turn off bluebox while the cluster is rebuilt" https://review.openstack.org/327979 | 22:02 |
| notmorgan | jeblair: solution to the random zuul timeouts.. "watch the gate like a kettle boiling water" *sigh* | 22:02 |
| notmorgan | i so want streaming logs again. | 22:02 |
| clarkb | jeblair did you stop using the ansible block error catching? looks like two playbooks now? | 22:03 |
| anteaya | zaro: where does gerrit.conf reside? | 22:03 |
| jeblair | notmorgan: if a job runs on zl* you can get the telnet url from the json | 22:03 |
| jeblair | clarkb: always been 2 playbooks -- the error catching was just to print "success" or "failure" | 22:03 |
| clarkb | JayF: ok that probably means that grenade isnt filtering stuff out like devstack gate does via sudo | 22:03 |
| pabelanger | jeblair: notmorgan: Indeed, more errors codes the better | 22:03 |
| notmorgan | jeblair: yeh. | 22:03 |
| clarkb | jeblair: I thought you had it in one playbook using blocks | 22:04 |
| jeblair | clarkb: but, now that you say that, heh, we could have implemented that as a step in the post playbook :) | 22:04 |
| clarkb | so that failed build would then enter the block that uploaded logs | 22:04 |
| thiagop | jeblair: new image is with the same issue | 22:04 |
| thiagop | jeblair: http://paste.openstack.org/show/509319/ | 22:04 |
| anteaya | zaro: found it | 22:04 |
| pabelanger | jeblair: clarkb: that would also fix our SSH issue we had with beaker, since it is a single SSH command | 22:04 |
| pabelanger | I like it | 22:05 |
| *** ayoung has quit IRC | 22:05 | |
| JayF | clarkb: if I'm reading this properly, it doesn't even look like it "sudoes" the ./stack.sh call directly : https://github.com/openstack-dev/grenade/blob/master/grenade.sh#L215 | 22:05 |
| *** mriedem has quit IRC | 22:05 | |
| jeblair | clarkb: https://review.openstack.org/321135 is where we added the block construct | 22:05 |
| openstackgerrit | Merged openstack-infra/system-config: Open port 8088 on long-lived slaves https://review.openstack.org/327969 | 22:05 |
| *** NM has quit IRC | 22:05 | |
| jeblair | clarkb: if you look at the bottom of that diff, you can see we already had the post_playbook for publishers | 22:06 |
| clarkb | JayF: aha I bet the sudo happens in d-g which is why d-g breaks but shoving it doen a level to grenade works | 22:06 |
| jeblair | pabelanger: i am not certain it would fix ssh issues | 22:06 |
| *** mriedem has joined #openstack-infra | 22:06 | |
| *** mriedem has quit IRC | 22:06 | |
| pabelanger | jeblair: Oh, ya. async | 22:06 |
| clarkb | jeblair: gotcha (also not I feel like I have been awake for 24 hours though it hasnt been quite that long) | 22:06 |
| pabelanger | for got about that | 22:06 |
| asselin_ | rcarrillocruz, or anyone else know about the cloud config cache settings? https://pypi.python.org/pypi/os-client-config | 22:06 |
| notmorgan | jeblair: soon everything will be zl* so we'll have telnet for them all | 22:06 |
| *** esberglu has quit IRC | 22:07 | |
| anteaya | zaro: http://paste.openstack.org/show/509320/ | 22:07 |
| notmyname | dims: I want to get started on the TC follow-up about swift and golang. can you join me in -swift or email me@not.mn? | 22:07 |
| notmorgan | notmyname: lol nice email. :) | 22:07 |
| *** tpsilva has quit IRC | 22:07 | |
| notmorgan | jeblair: i'm good for a gear release at any point (dunno if i answered you earlier) was reading scroll back. | 22:07 |
| notmyname | notmorgan: the release one just now? or a different one? | 22:08 |
| JayF | clarkb: so are we good for 316662 then or was there any other question? | 22:08 |
| notmorgan | notmyname: your addr me@not.mn | 22:08 |
| *** jerryz has quit IRC | 22:08 | |
| notmyname | notmorgan: ah :-) | 22:08 |
| clarkb | JayF: yup I approved | 22:08 |
| JayF | thanks! | 22:08 |
| openstackgerrit | Merged openstack-infra/zuul: Python 3 Fixes: Replace missing builtins https://review.openstack.org/324205 | 22:09 |
| * clarkb goes back to travel related exhaustion | 22:09 | |
| *** jamielennox is now known as jamielennox|away | 22:09 | |
| jeblair | notmorgan: yeah, i was holding off until some zl stuff landed then i'll push it through to help test it :) | 22:09 |
| zaro | anteaya: you can comment out the username and connectionpool | 22:09 |
| *** hashar has quit IRC | 22:09 | |
| notmorgan | jeblair: ok cool w00t. also yay zl! | 22:09 |
| anteaya | zaro: thanks | 22:09 |
| JayF | clarkb: it's not showing up as approved? https://review.openstack.org/#/c/316662 unless my browser cache decieves me | 22:10 |
| *** tlian has joined #openstack-infra | 22:10 | |
| JayF | clarkb: sorry for being watchful, I just know a lot of folks who will be thrilled when this goes in for a landing :) | 22:10 |
| JayF | ...and as usual as soon as I say something it updates | 22:10 |
| clarkb | JayF: there it was my mobile browser failing at gerrit | 22:10 |
| JayF | wheee | 22:10 |
| *** ihrachys has quit IRC | 22:11 | |
| asselin_ | rcarrillocruz, nevermind...found the default locations. very slow the first time. | 22:12 |
| *** ihrachys has joined #openstack-infra | 22:12 | |
| *** pahuang has joined #openstack-infra | 22:14 | |
| anteaya | zaro: this is the log from my latest puppet run: http://paste.openstack.org/show/509321/ ps aux | grep gerrit and top both show nothing | 22:14 |
| *** kzaitsev_mb has quit IRC | 22:15 | |
| openstackgerrit | Jay Faulkner proposed openstack-infra/project-config: Make grenade-dsvm-ironic non voting https://review.openstack.org/319336 | 22:15 |
| openstackgerrit | Jay Faulkner proposed openstack-infra/project-config: Make grenade-dsvm-ironic -nv on grenade https://review.openstack.org/327985 | 22:15 |
| *** piet has quit IRC | 22:16 | |
| JayF | AJaeger: when you get a chance, would love you to renew your +2 on https://review.openstack.org/#/c/319336 | 22:17 |
| * jeblair suspects the jenkins masters will be gone before jjb configs on them are fully synchronized again | 22:17 | |
| anteaya | yay? | 22:17 |
| JayF | mtreinish: I split the Ironic grenade patch into two; one to add to Ironic check pipeline and one to add to grenade check pipeline, so if you wouldn't mind re-reviewing at least the ironic side: https://review.openstack.org/#/c/319336 I'd appreciate it. Thanks!!! | 22:17 |
| pabelanger | woah, first time I see this on zuul.o.o | 22:17 |
| pabelanger | InternalError: (pymysql.err.InternalError) (1205, u'Lock wait timeout exceeded; try restarting transaction') | 22:17 |
| jeblair | pabelanger: zuul does not use mysql | 22:17 |
| pabelanger | doh, nodepool.o.o | 22:18 |
| jeblair | pabelanger: nodepool? | 22:18 |
| jeblair | ya | 22:18 |
| pabelanger | jeblair: Ya, was looking at zuul when I was typing | 22:18 |
| anteaya | zaro: am I installing an h2 database manually before I run puppet? | 22:18 |
| anteaya | zaro: I have mysql installed on this server | 22:19 |
| *** banix has joined #openstack-infra | 22:19 | |
| *** thorst_ has quit IRC | 22:21 | |
| *** rhallisey has quit IRC | 22:22 | |
| *** banix has quit IRC | 22:22 | |
| pabelanger | jeblair: just to confirm, you did zuul-launcher graceful on zl01 right? | 22:23 |
| *** thorst has joined #openstack-infra | 22:23 | |
| jeblair | pabelanger: i don't think that had landed, so i did 'pause' then 'release' | 22:24 |
| *** vhosakot has quit IRC | 22:25 | |
| jeblair | it looks like we leaked one used node, probably when we restarted nodepool | 22:25 |
| pabelanger | jeblair: okay, cool. Was trying to see if something showed up in the logs, but didn't see anything specific | 22:25 |
| jeblair | pabelanger: 'pause' and 'release' should have shown up, but not graceful, yeah | 22:25 |
| jeblair | zl01 is idle so i'm stopping it now | 22:25 |
| pabelanger | Yup see pause from socket | 22:26 |
| pabelanger | followed by Pausing | 22:26 |
| *** erlon has quit IRC | 22:26 | |
| *** zz_dimtruck is now known as dimtruck | 22:26 | |
| *** thorst has quit IRC | 22:27 | |
| jeblair | pabelanger: indeed https://review.openstack.org/325523 still hasn't landed :) | 22:27 |
| *** amitgandhinz has quit IRC | 22:28 | |
| pabelanger | looking | 22:28 |
| *** ddieterly[away] is now known as ddieterly | 22:29 | |
| pabelanger | jeblair: +2 | 22:30 |
| pabelanger | jeblair: think it would be neat to expose some of that info to statsd. Quickly checking graphite.o.o for the status of the launcher | 22:32 |
| pabelanger | I'm sure there would be away to do that in zookeeper too | 22:33 |
| *** EricGonczer_ has quit IRC | 22:33 | |
| notmorgan | pabelanger: should be doable via ZK. | 22:33 |
| notmorgan | pabelanger: as long as the data is exposed there. | 22:33 |
| anteaya | zaro: h2 needs a browser to complete set-up, I don't have a browser on my vm | 22:33 |
| openstackgerrit | Merged openstack-infra/nodepool: Use underscore for node_assign job https://review.openstack.org/327799 | 22:33 |
| pabelanger | notmorgan: I would think so too | 22:34 |
| anteaya | zaro: do you have any instructions for how to get mysql working? I have that installed | 22:34 |
| *** ddieterly is now known as ddieterly[away] | 22:34 | |
| notmorgan | pabelanger: it also wouldn't be hard to just directly send that data into statsd. | 22:35 |
| notmorgan | pabelanger: oh you said that :P | 22:35 |
| fungi | updated my sh10151.py script to also delete non-file pages created by the same list of spammer accounts (currently running and logging now, deletion count to be provided when complete) http://paste.openstack.org/show/509432 | 22:35 |
| * notmorgan learns to read better. | 22:35 | |
| notmorgan | fungi: i like your password;) | 22:37 |
| *** kzaitsev_mb has joined #openstack-infra | 22:37 | |
| fungi | notmorgan: me too, that's why i didn't include it in that script ;) | 22:37 |
| notmorgan | REDACTED is like the best password ever | 22:37 |
| clarkb | fungi: can that go in puppet-mediwiki? | 22:37 |
| notmorgan | fungi: ;) | 22:37 |
| fungi | clarkb: sure, might could stand some additional generalization | 22:37 |
| notmorgan | fungi: at least is isn't hunter2, right? | 22:38 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: Add graceful stop command https://review.openstack.org/325523 | 22:38 |
| openstackgerrit | Sridhar Ramaswamy proposed openstack-infra/project-config: Add stable/mitaka to tacker gerritbot https://review.openstack.org/327991 | 22:38 |
| clarkb | notmorgan: you mean ******** | 22:38 |
| fungi | okay, that pass finished successfully removing another 289 pages | 22:38 |
| notmorgan | clarkb: haha | 22:38 |
| pabelanger | notmorgan: I am thinking something like zuul.service.laucher.zl01.state as a namespace | 22:38 |
| notmorgan | pabelanger: it could also be part of the zl01 | 22:38 |
| pabelanger | launcher* | 22:38 |
| notmorgan | pabelanger: node, the data for the node update would be atomic anyway | 22:39 |
| notmorgan | so you could query the node directly. having a lot of subnodes is ok, but becomes unweildy overtime | 22:39 |
| fungi | i need to go knock out some yardwork before it gets dark, and will search teh wiki for other spam i might find ways to programmatically mass clean once i'm done | 22:39 |
| notmorgan | pabelanger: it just depends on how much data we stash in the node too ;) deserializing can be expensive too :P | 22:39 |
| anteaya | fungi: yay | 22:40 |
| pabelanger | notmorgan: ya, we don't track node data like that today. Could be interesting | 22:40 |
| anteaya | fungi: enjoy the sun | 22:40 |
| *** Sukhdev has quit IRC | 22:40 | |
| notmorgan | pabelanger: sorry subnode = zk thing should have said child :) | 22:40 |
| notmorgan | pabelanger: but that also would be interesting for the actual "subnode" things we have. | 22:41 |
| pabelanger | notmorgan: Moving forward with zuulv3, I could see us expanding the data we collect to include job runtime, # launched, etc | 22:42 |
| pabelanger | since jenkins provides some of that today in the UI | 22:42 |
| notmorgan | pabelanger: aye | 22:42 |
| anteaya | when we create a gerrit instance do we install mysql-server on it manually or with puppet? | 22:43 |
| jeblair | pabelanger: zuul logs the runtime of ever job it launches | 22:44 |
| *** rbrndt has quit IRC | 22:44 | |
| jeblair | and counts too | 22:44 |
| *** sarob has quit IRC | 22:44 | |
| jeblair | restarting nodepool | 22:45 |
| pabelanger | ack | 22:45 |
| jeblair | restarting zlstatic01 | 22:47 |
| *** csomerville has quit IRC | 22:47 | |
| anteaya | also do I have to run puppet apply as the gerrit2 user to get a gerrit running? | 22:47 |
| pleia2 | you'll need to use sudo, puppet itself will define what user to run gerrit as | 22:48 |
| jeblair | nc release.slave.openstack.org 8088 | 22:48 |
| jeblair | works | 22:48 |
| pleia2 | jeblair: that is so cool \o/ | 22:49 |
| *** jamesmcarthur has joined #openstack-infra | 22:49 | |
| anteaya | pleia2: thank you I had not been using sudo, I will do so now | 22:49 |
| anteaya | pleia2: sorry I have been using sudo | 22:49 |
| jeblair | pleia2: keep watching, i'm going to tag gear now :) | 22:50 |
| jeblair | pushed | 22:50 |
| jeblair | oh, proposal.slave just ran its rtfd job | 22:51 |
| zaro | anteaya: if you want to setup mysql then you can do that but using h2 is much easier | 22:51 |
| zaro | you need browser to setup 1st user with either h2 or mysql | 22:52 |
| anteaya | zaro: well h2 needs a browser to complete setup, and I don't have an ability to use a browser on a vm I'm ssh'd into | 22:52 |
| jeblair | yay, nodepool is seeing zmq events from zl02 now | 22:52 |
| *** thiagop has quit IRC | 22:53 | |
| notmorgan | jeblair: woo | 22:53 |
| *** jamesmcarthur has quit IRC | 22:53 | |
| jeblair | deleting stale ready nodes | 22:54 |
| zaro | anteaya: that is odd, i think all you need is the ip address, http://ipaddress:8080 | 22:54 |
| *** aysyd has quit IRC | 22:54 | |
| jeblair | notmorgan: geat 0.6.2 is uploaded | 22:54 |
| notmorgan | jeblair: tyvm! | 22:54 |
| jeblair | np! | 22:54 |
| pabelanger | jeblair: with nc release.slave.openstack.org 8088 will the buffer continue to grow as more jobs running when we first connect? | 22:54 |
| jeblair | pabelanger: yes! forever! | 22:54 |
| jeblair | we should probably do something about that | 22:54 |
| pabelanger | NICE | 22:55 |
| notmorgan | jeblair: once that syncs out, i should be able to tell if all the py3 things are passing for zuul. [going to need to take a very close look at the ansible launcher code still, before we can run it] | 22:55 |
| jeblair | pabelanger: oh wait, maybe not | 22:55 |
| jeblair | pabelanger: nope, we're covered -- it will be truncated at the start of each new job | 22:55 |
| jeblair | apparently i already thought of that :) | 22:55 |
| *** annegentle has quit IRC | 22:55 | |
| pabelanger | coolio | 22:55 |
| zaro | anteaya: it will be much simpler if you just skip the puppet stuff and follow install gerrit install instructions here https://review.openstack.org/Documentation/install-quick.html#initialization | 22:56 |
| craige | o/ | 22:56 |
| zaro | once you have your site you can install and configure the pluign | 22:57 |
| anteaya | zaro: well the thing of it is that eventually I need to puppet the installation of the plugin | 22:57 |
| anteaya | zaro: but thanks for these docs | 22:57 |
| zaro | anteaya: no, it’s already done for you. all you need to do is make a change like this.. https://review.openstack.org/#/c/319400/ | 22:58 |
| *** mtanino has quit IRC | 22:58 | |
| *** ddieterly[away] is now known as ddieterly | 22:58 | |
| *** kzaitsev_mb has quit IRC | 22:58 | |
| anteaya | zaro: okay thank you | 22:58 |
| zaro | anteaya: you just need to setup gerrit and SB locally | 22:58 |
| jeblair | pabelanger, notmorgan, clarkb: this could use some review https://review.openstack.org/327422 | 22:58 |
| zaro | anteaya: when everything works, we just add a little change to existing puppet | 22:59 |
| *** jamesmcarthur has joined #openstack-infra | 22:59 | |
| pabelanger | jeblair: going to defer to clarkb on that one. Don't know gear that much | 22:59 |
| pabelanger | gearman* | 22:59 |
| *** ddieterly has quit IRC | 22:59 | |
| anteaya | zaro: okay thank you | 23:00 |
| *** ddieterly has joined #openstack-infra | 23:00 | |
| anteaya | zaro: I appreciate the patch example | 23:00 |
| openstackgerrit | Merged openstack-infra/zuul: Python 3 Fix: dict().iteritems no longer exists https://review.openstack.org/325620 | 23:00 |
| *** ddieterly has quit IRC | 23:00 | |
| jeblair | it's not urgent; i'd like to get it running tomorrow, so that geard gets better as zuul-launchers come online | 23:00 |
| notmorgan | jeblair: i'm less familiar still with the packet communications in gear (still learning) | 23:01 |
| notmorgan | but this looks reasonable. | 23:01 |
| jhesketh | Morning | 23:01 |
| zaro | anteaya: i know this is hard and will take a bit of time | 23:01 |
| notmorgan | jhesketh: o/ :) | 23:01 |
| *** asettle has joined #openstack-infra | 23:02 | |
| anteaya | zaro: thank you for your patience with me, I appreciate it :) | 23:02 |
| anteaya | morning jhesketh | 23:02 |
| notmorgan | jeblair: i have only one minor nit, you're defining MASS_DO in multiple locations [total bikeshed/nit] | 23:02 |
| *** baoli has quit IRC | 23:03 | |
| notmorgan | jeblair: also, is there any concern the mass_do packet would exceed limits of some sort? | 23:03 |
| *** kzaitsev_mb has joined #openstack-infra | 23:03 | |
| notmorgan | jeblair: as in too much in the packet itself? | 23:03 |
| *** davideagnello has joined #openstack-infra | 23:03 | |
| openstackgerrit | Jay Faulkner proposed openstack-infra/project-config: Experimental jobs to test IPA w/diff ram https://review.openstack.org/327994 | 23:03 |
| *** ams__ has quit IRC | 23:03 | |
| jeblair | notmorgan: yeah. but i don't want to import the server from the launcher, or vice versa. and a file for a single constant seems :| | 23:03 |
| *** xyang1 has quit IRC | 23:04 | |
| notmorgan | jeblair: "const.py => MASS_DO = 101 #bikeshed" | 23:04 |
| notmorgan | jeblair: :P | 23:04 |
| *** davideagnello has quit IRC | 23:04 | |
| notmorgan | jeblair: so my only real thought is how much data ends up in the packet. | 23:04 |
| notmorgan | you said this was pushing ~20k elements in cases? | 23:05 |
| zaro | anteaya: np, BTW h2 db is created by gerri so it doesn’t require any additional configuring. if you go mysql route you would need to run addtional sql against the db before hooking up gerrit to it. | 23:05 |
| notmorgan | that is *a lot* (having been looking at the function caches), so ... like... a few 100k there? | 23:05 |
| notmorgan | s/k/kb | 23:05 |
| jeblair | notmorgan: gearman max packet length is a 32bit integer, so 4294967296 bytes assuming it's unsigned | 23:05 |
| jeblair | (which the spec doc is typically vague about) | 23:06 |
| *** sflanigan_ has joined #openstack-infra | 23:06 | |
| *** sflanigan_ has joined #openstack-infra | 23:06 | |
| notmorgan | jeblair: yeah | 23:06 |
| *** pradk has quit IRC | 23:06 | |
| jeblair | we unpack it as '!4sii' | 23:06 |
| notmorgan | i'm just wondering if pushing that data is going to cause serialize/deserialize headaches | 23:06 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: remove deprecated hyphens https://review.openstack.org/327347 | 23:06 |
| jeblair | notmorgan: oh, in practice? no. it's super fast | 23:07 |
| jeblair | notmorgan: (i wrote a test script) | 23:07 |
| notmorgan | jeblair: yeah just wasn't sure because i haven't poked at that type of serialize/deserialize in a while | 23:07 |
| notmorgan | unpack o f!4sii should be find. | 23:07 |
| notmorgan | fine* | 23:07 |
| *** tkelsey has joined #openstack-infra | 23:08 | |
| *** tphummel has quit IRC | 23:08 | |
| jeblair | notmorgan: we will definitely be creating a 640k string in python, 4k at a time. | 23:08 |
| notmorgan | yeah. | 23:08 |
| *** nwkarsten has joined #openstack-infra | 23:08 | |
| anteaya | zaro: okay thank you | 23:08 |
| jeblair | notmorgan: we could probably optimize that more, but it's already way faster than current | 23:08 |
| jeblair | cstringio or something maybe | 23:08 |
| notmorgan | jeblair: eh lets play optimise after this lands | 23:08 |
| *** jamesmcarthur has quit IRC | 23:08 | |
| notmorgan | yeah was thinking cstringio might be the way to go | 23:09 |
| jeblair | notmorgan: yeah. or better yet, zuulv3 and delete it. | 23:09 |
| *** rlandy has quit IRC | 23:09 | |
| notmorgan | LOL | 23:09 |
| notmorgan | depending on timelines and when we hit issues. | 23:09 |
| notmorgan | but yeah | 23:09 |
| *** asettle has quit IRC | 23:09 | |
| jeblair | we've been past the human and cpu scaling limits of zuulv2 for months. we pretty much need to v3 asap. | 23:09 |
| jeblair | before either AJaeger or zuul.openstack.org burn out :) | 23:10 |
| notmorgan | ok so commented, basically, LGTM as long gear-smarter-people agree | 23:10 |
| *** sdague has quit IRC | 23:10 | |
| notmorgan | nothing says "OMG NO BAD IDEA" to me. | 23:11 |
| anteaya | zaro: I'm going to get some dinner and try the no puppet instructions on another vm, thanks for answering my questions | 23:11 |
| *** tphummel has joined #openstack-infra | 23:11 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Update gear requirement https://review.openstack.org/326173 | 23:11 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Update gear requirement https://review.openstack.org/326173 | 23:12 |
| *** tkelsey has quit IRC | 23:12 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Fix timeout debug print in tests https://review.openstack.org/327768 | 23:12 |
| *** ihrachys has quit IRC | 23:13 | |
| *** nwkarsten has quit IRC | 23:13 | |
| notmorgan | jeblair: LOL I can drop one of my patches now because of that | 23:13 |
| jeblair | oh heh, sorry | 23:13 |
| notmorgan | hehe i had https://review.openstack.org/#/c/327304/ no yours is better | 23:13 |
| *** kzaitsev_mb has quit IRC | 23:15 | |
| *** kzaitsev_mb has joined #openstack-infra | 23:16 | |
| *** kengo_sakai has joined #openstack-infra | 23:17 | |
| *** jamesmcarthur has joined #openstack-infra | 23:17 | |
| *** kengo_sakai has quit IRC | 23:18 | |
| *** kengo_sakai has joined #openstack-infra | 23:19 | |
| jeblair | i'm deleteing ready zl02 nodes as they hit the 2 hour mark | 23:21 |
| *** jamesmcarthur has quit IRC | 23:22 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python3 Fix: tools use six.moves.urllib https://review.openstack.org/327459 | 23:23 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: Bundled Fixes https://review.openstack.org/325661 | 23:23 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 fix: GC final state inspection https://review.openstack.org/327436 | 23:23 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fix: Divide operator returns float https://review.openstack.org/327303 | 23:23 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: More encode/decode fixes https://review.openstack.org/327435 | 23:23 |
| *** gouthamr has joined #openstack-infra | 23:23 | |
| *** hongbin_ has quit IRC | 23:23 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: WIP: Add web-based console log streaming https://review.openstack.org/320563 | 23:24 |
| *** salv-orl_ has joined #openstack-infra | 23:27 | |
| *** salv-orl_ has quit IRC | 23:28 | |
| *** kzaitsev_mb has quit IRC | 23:28 | |
| *** salv-orl_ has joined #openstack-infra | 23:29 | |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add more oslo libs to the run-tox-with-oslo-master script https://review.openstack.org/327999 | 23:29 |
| *** amitgandhinz has joined #openstack-infra | 23:29 | |
| *** Apoorva has quit IRC | 23:30 | |
| *** salv-orlando has quit IRC | 23:31 | |
| *** pradk has joined #openstack-infra | 23:32 | |
| *** xarses has quit IRC | 23:33 | |
| *** johnny_ has quit IRC | 23:33 | |
| *** xarses has joined #openstack-infra | 23:33 | |
| *** amitgandhinz has quit IRC | 23:33 | |
| openstackgerrit | Merged openstack-infra/devstack-gate: Allow to pass OS_TEST_TIMEOUT for grenade job https://review.openstack.org/316662 | 23:34 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/nodepool: Update gear requirement https://review.openstack.org/326172 | 23:35 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Fix timeout debug print in tests https://review.openstack.org/327768 | 23:36 |
| *** kzaitsev_mb has joined #openstack-infra | 23:36 | |
| *** jamielennox|away is now known as jamielennox | 23:37 | |
| anteaya | zaro: is this the version of gerrit we are running on -dev? gerrit-v2.11.4.13.cb9800e.war | 23:38 |
| dims | notmyname : i had carved out time tomorrow to work on it. will ping tomorrow | 23:39 |
| *** cloudtrainme has quit IRC | 23:40 | |
| *** Apoorva has joined #openstack-infra | 23:43 | |
| *** tosky has quit IRC | 23:43 | |
| *** mdrabe has quit IRC | 23:43 | |
| *** chlong has quit IRC | 23:45 | |
| *** xarses has quit IRC | 23:46 | |
| *** kzaitsev_mb has quit IRC | 23:47 | |
| *** NM has joined #openstack-infra | 23:50 | |
| *** bpokorny_ has joined #openstack-infra | 23:56 | |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Remove deprecated hyphens from zuul launcher https://review.openstack.org/327349 | 23:56 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Add zuul.service.HOSTNAME.command.COMMAND for statsd https://review.openstack.org/328006 | 23:57 |
| pabelanger | jeblair: notmorgan: first attempt to track commands in statsd^. No actually testing yet | 23:58 |
| notmorgan | pabelanger: nice. | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!