| notmorgan | jeblair: so.. your patch is to not include project-config? but is for? | 00:00 |
|---|---|---|
| *** IlyaG has quit IRC | 00:01 | |
| jeblair | notmorgan: that patch implements the insane promise that we made to support having zuul alter its running config in response to a proposed change to its config | 00:01 |
| *** karthikp_ has quit IRC | 00:01 | |
| notmorgan | right. | 00:01 |
| notmorgan | i get that | 00:01 |
| jeblair | oh ok, sorry | 00:01 |
| notmorgan | what i am not getting is... what repos should see that vs what shouldn't | 00:02 |
| * notmorgan is trying to parse your commit message. | 00:02 | |
| ianw | pabelanger / greghaynes : so i "fixed" the locale thing ... and now the build fails in puppet as it tries to install emacs with yum. | 00:02 |
| notmorgan | and compare it to code. | 00:02 |
| *** zhurong has quit IRC | 00:02 | |
| notmorgan | so i have some idea of what is intended and what is implemented. | 00:02 |
| *** weshay_afk has quit IRC | 00:03 | |
| *** amotoki has joined #openstack-infra | 00:04 | |
| openstackgerrit | Merged openstack-infra/nodepool: Add a simple status webapp https://review.openstack.org/293706 | 00:04 |
| openstackgerrit | Merged openstack-infra/nodepool: Add some tracking for image sizes https://review.openstack.org/289678 | 00:04 |
| *** rajinir has quit IRC | 00:05 | |
| *** Jeffrey4l has joined #openstack-infra | 00:05 | |
| jeblair | notmorgan: basically, any repo that has a .zuul.yaml file should be able to change the configuration on the fly -- the change should take effect for that very change as well as any changes enqueued after it -- even if those changes are in other repos. | 00:05 |
| *** bhunter71 has joined #openstack-infra | 00:05 | |
| *** baoli has quit IRC | 00:06 | |
| *** rfolco has joined #openstack-infra | 00:06 | |
| jeblair | notmorgan: (now, there's all kinds of things it *shouldn't* be able to do, like change the project-pipeline config for a different project -- those restrictions are not yet implemented -- i think that's what needs to be done next in this area) | 00:06 |
| *** cardeois_ has joined #openstack-infra | 00:06 | |
| *** Apoorva_ has quit IRC | 00:06 | |
| *** sflanigan has joined #openstack-infra | 00:06 | |
| jeblair | notmorgan: if you look at test_v3.py in that change, and take a look at test_dynamic config -- that sort of walks through what this does | 00:07 |
| notmorgan | ok | 00:07 |
| *** EricGonczer_ has joined #openstack-infra | 00:07 | |
| *** Apoorva has joined #openstack-infra | 00:07 | |
| *** tkelsey has joined #openstack-infra | 00:07 | |
| jeblair | notmorgan: that's a change that starts with 'org-project' running 'project-test1' but the test changes that to run 'project-test2' (by proposing a commit to the .zuul.yaml) instead | 00:07 |
| notmorgan | ah ok i think i see it now. | 00:08 |
| jeblair | notmorgan: cool... i have to eod now, but we can pick this up tomorrow | 00:09 |
| *** mika has joined #openstack-infra | 00:10 | |
| notmorgan | sounds good. i'll be around for a chunk of tomorrow, but will be out a little early | 00:10 |
| *** cardeois_ has quit IRC | 00:11 | |
| *** tkelsey has quit IRC | 00:12 | |
| *** Sukhdev has quit IRC | 00:12 | |
| *** sarob has joined #openstack-infra | 00:16 | |
| *** rfolco has quit IRC | 00:16 | |
| *** sarob has quit IRC | 00:16 | |
| fungi | clarkb: thinking about it, all we really add is stable branch histories not overlapping master, which ultimately is probably a very small added number of commits after all | 00:22 |
| *** yamamoto has quit IRC | 00:24 | |
| *** rdx_matt has quit IRC | 00:24 | |
| *** piet has joined #openstack-infra | 00:27 | |
| *** adrian_otto has quit IRC | 00:27 | |
| *** coolsvap has joined #openstack-infra | 00:28 | |
| *** Julien-zte has joined #openstack-infra | 00:28 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 00:29 |
| *** Ravikiran_K has quit IRC | 00:32 | |
| *** amitgandhinz has joined #openstack-infra | 00:32 | |
| *** simondodsley has quit IRC | 00:33 | |
| *** shashank_hegde has quit IRC | 00:35 | |
| openstackgerrit | Julien proposed openstack-infra/git-review: Add project-name support in topic https://review.openstack.org/310092 | 00:35 |
| greghaynes | ianw: nice! (well, not nice about emacs, but ya) | 00:37 |
| Shrews | jeblair: perhaps it's just my newness with the zuul merger code, but i keep looking at 340597 and seeing these new 'file' variables and keep wondering... what the heck are these files? | 00:37 |
| *** amitgandhinz has quit IRC | 00:37 | |
| *** OSNEXUS has quit IRC | 00:38 | |
| *** clenimar_ has quit IRC | 00:38 | |
| ianw | greghaynes: actually it's just emacs that is broken | 00:39 |
| craige | Same as it ever was ;-) | 00:41 |
| * craige hides his VIm sword. | 00:41 | |
| *** Sukhdev has joined #openstack-infra | 00:43 | |
| *** amotoki has quit IRC | 00:46 | |
| *** markvoelker has quit IRC | 00:48 | |
| *** roxanagh_ has joined #openstack-infra | 00:48 | |
| *** amotoki has joined #openstack-infra | 00:48 | |
| *** thorst_ has joined #openstack-infra | 00:49 | |
| *** tonytan4ever has joined #openstack-infra | 00:49 | |
| *** roxanagh_ has quit IRC | 00:52 | |
| *** tonytan4ever has quit IRC | 00:53 | |
| *** tonytan4ever has joined #openstack-infra | 00:53 | |
| *** Jeffrey4l has quit IRC | 00:53 | |
| *** thorst_ has quit IRC | 00:56 | |
| *** clenimar_ has joined #openstack-infra | 00:56 | |
| *** jianghuaw has quit IRC | 00:57 | |
| *** kzaitsev_mb has quit IRC | 00:59 | |
| *** kzaitsev_mb has joined #openstack-infra | 01:00 | |
| *** ajmiller has quit IRC | 01:02 | |
| *** yamamoto has joined #openstack-infra | 01:03 | |
| *** Apoorva_ has joined #openstack-infra | 01:03 | |
| *** dimtruck is now known as zz_dimtruck | 01:04 | |
| *** zz_dimtruck is now known as dimtruck | 01:06 | |
| *** Apoorva has quit IRC | 01:06 | |
| *** clenimar_ has quit IRC | 01:07 | |
| *** cardeois_ has joined #openstack-infra | 01:07 | |
| *** Apoorva_ has quit IRC | 01:08 | |
| *** aspiers has quit IRC | 01:11 | |
| *** cardeois_ has quit IRC | 01:12 | |
| *** zhurong has joined #openstack-infra | 01:12 | |
| *** yanyanhu has joined #openstack-infra | 01:13 | |
| *** kzaitsev_mb has quit IRC | 01:17 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix error in Neutron VPNaaS API CI job. https://review.openstack.org/342068 | 01:17 |
| *** Sukhdev has quit IRC | 01:18 | |
| *** clenimar_ has joined #openstack-infra | 01:19 | |
| *** esberglu has quit IRC | 01:22 | |
| *** dimtruck is now known as zz_dimtruck | 01:22 | |
| *** rdx_matt has joined #openstack-infra | 01:22 | |
| *** sputnik13 has quit IRC | 01:24 | |
| *** sarob has joined #openstack-infra | 01:25 | |
| *** cody-somerville has quit IRC | 01:26 | |
| *** amotoki has quit IRC | 01:26 | |
| *** sputnik13 has joined #openstack-infra | 01:27 | |
| *** tonytan4ever has quit IRC | 01:30 | |
| *** thorst_ has joined #openstack-infra | 01:30 | |
| *** rderose has quit IRC | 01:31 | |
| *** amotoki has joined #openstack-infra | 01:31 | |
| *** thorst_ has quit IRC | 01:31 | |
| *** rkukura has quit IRC | 01:31 | |
| *** thorst_ has joined #openstack-infra | 01:32 | |
| *** amotoki has quit IRC | 01:32 | |
| *** gongysh has joined #openstack-infra | 01:32 | |
| *** amitgandhinz has joined #openstack-infra | 01:33 | |
| *** tphummel has quit IRC | 01:34 | |
| *** Julien-zte has quit IRC | 01:36 | |
| *** rajinir has joined #openstack-infra | 01:37 | |
| *** amitgandhinz has quit IRC | 01:37 | |
| *** flepied has joined #openstack-infra | 01:38 | |
| *** thorst_ has quit IRC | 01:40 | |
| *** jcoufal has joined #openstack-infra | 01:40 | |
| openstackgerrit | Julien proposed openstack-infra/git-review: Add project-name support in topic https://review.openstack.org/310092 | 01:43 |
| *** yamahata has quit IRC | 01:43 | |
| *** annegentle has quit IRC | 01:44 | |
| openstackgerrit | Julien proposed openstack-infra/git-review: Add project-name support in topic https://review.openstack.org/310092 | 01:46 |
| *** aspiers has joined #openstack-infra | 01:48 | |
| *** rdx_matt has quit IRC | 01:56 | |
| *** zshuo has joined #openstack-infra | 01:58 | |
| openstackgerrit | Henry Gessau proposed openstack-infra/project-config: Switch py35 to voting for neutron stadium projects https://review.openstack.org/337946 | 02:01 |
| *** mriedem has left #openstack-infra | 02:01 | |
| *** yuanying has joined #openstack-infra | 02:05 | |
| *** zz_dimtruck is now known as dimtruck | 02:06 | |
| *** gongysh has quit IRC | 02:06 | |
| openstackgerrit | Henry Gessau proposed openstack-infra/project-config: Switch py35 to voting for neutron stadium projects https://review.openstack.org/337946 | 02:06 |
| *** thorst_ has joined #openstack-infra | 02:07 | |
| *** thorst_ has quit IRC | 02:08 | |
| *** cardeois_ has joined #openstack-infra | 02:08 | |
| *** thorst_ has joined #openstack-infra | 02:09 | |
| *** tkelsey has joined #openstack-infra | 02:09 | |
| *** flepied has quit IRC | 02:10 | |
| *** sputnik13 has quit IRC | 02:11 | |
| *** shashank_hegde has joined #openstack-infra | 02:11 | |
| openstackgerrit | Julien proposed openstack-infra/git-review: Add project-name support in topic https://review.openstack.org/310092 | 02:13 |
| *** tkelsey has quit IRC | 02:13 | |
| *** cardeois_ has quit IRC | 02:14 | |
| *** links has joined #openstack-infra | 02:16 | |
| *** sdake_ has quit IRC | 02:16 | |
| *** sflanigan is now known as sflanigan_afk | 02:17 | |
| *** thorst_ has quit IRC | 02:17 | |
| *** shashank_hegde has quit IRC | 02:19 | |
| *** woodster_ has quit IRC | 02:19 | |
| *** asettle has joined #openstack-infra | 02:19 | |
| openstackgerrit | James Polley proposed openstack-dev/pbr: Fix handling of old git log output https://review.openstack.org/339392 | 02:21 |
| *** asettle has quit IRC | 02:25 | |
| *** piet has quit IRC | 02:28 | |
| *** sarob has quit IRC | 02:29 | |
| *** yamahata has joined #openstack-infra | 02:31 | |
| *** tonytan4ever has joined #openstack-infra | 02:31 | |
| *** sdake has joined #openstack-infra | 02:31 | |
| openstackgerrit | Liping Mao proposed openstack-infra/project-config: Add python35-nv jobs in Kuryr https://review.openstack.org/342531 | 02:32 |
| *** amitgandhinz has joined #openstack-infra | 02:34 | |
| *** tonytan4ever has quit IRC | 02:36 | |
| *** roxanagh_ has joined #openstack-infra | 02:36 | |
| *** ddieterly has joined #openstack-infra | 02:37 | |
| *** sflanigan_afk is now known as sflanigan | 02:38 | |
| *** amitgandhinz has quit IRC | 02:39 | |
| *** gongysh has joined #openstack-infra | 02:39 | |
| *** roxanagh_ has quit IRC | 02:41 | |
| *** amotoki has joined #openstack-infra | 02:44 | |
| *** amotoki has quit IRC | 02:44 | |
| *** ddieterly has quit IRC | 02:46 | |
| jeblair | Shrews: ah, variables named 'files' are generally either lists of filenames, or dictionaries of {filename:content}. in the first case, we might be asking a merger to get us the content of those files. in the second, it's the filename and then the actual content of the file in the git repo. when zuul starts, it enumerates all the git repos that might contain config files, then asks the mergers to retrieve the contents of those ... | 02:46 |
| jeblair | ... files. in this change, we're extending that to also happen, though for a subset of affected repos, each time a change shows up where one of those files is modified. | 02:46 |
| *** ddieterly has joined #openstack-infra | 02:47 | |
| jeblair | Shrews: put another way -- the main zuul server only reads the very initial config files off of its local disk -- the ones that say what gerrit to talk to and what git repos where the rest of the config can be found | 02:47 |
| jeblair | Shrews: after that, it is wholly dependent on the distributed merge workers to go out and actually collect its configuration for it before it can even finish starting | 02:48 |
| *** markvoelker has joined #openstack-infra | 02:48 | |
| jeblair | (i think that's pretty cool actually -- nothing says distributed application like "i don't even read my own config file") | 02:49 |
| *** ddieterly has quit IRC | 02:50 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: Add an experimental grenade job for networking-midonet https://review.openstack.org/342535 | 02:50 |
| *** amotoki has joined #openstack-infra | 02:50 | |
| *** gongysh has quit IRC | 02:50 | |
| *** rdx_matt has joined #openstack-infra | 02:53 | |
| *** zul has quit IRC | 02:54 | |
| *** chlong has joined #openstack-infra | 02:56 | |
| *** DaiDV has joined #openstack-infra | 02:58 | |
| *** daidv_ has joined #openstack-infra | 02:58 | |
| *** sdake_ has joined #openstack-infra | 02:58 | |
| *** yuanying has quit IRC | 03:01 | |
| *** sdake has quit IRC | 03:02 | |
| *** daidv_ has quit IRC | 03:03 | |
| *** EricGonc_ has joined #openstack-infra | 03:03 | |
| openstackgerrit | Merged openstack-infra/project-config: Cache all branches in dib repo cache https://review.openstack.org/342461 | 03:03 |
| *** SumitNaiksatam has joined #openstack-infra | 03:04 | |
| openstackgerrit | James Polley proposed openstack-dev/pbr: Fix handling of old git log output https://review.openstack.org/339392 | 03:05 |
| *** sdake_ has quit IRC | 03:05 | |
| *** sarob has joined #openstack-infra | 03:05 | |
| *** Julien-zte has joined #openstack-infra | 03:05 | |
| *** EricGonczer_ has quit IRC | 03:05 | |
| *** julim has quit IRC | 03:06 | |
| *** EricGonc_ has quit IRC | 03:07 | |
| *** sdake has joined #openstack-infra | 03:08 | |
| *** sarob has quit IRC | 03:10 | |
| *** yamahata has quit IRC | 03:10 | |
| *** cardeois_ has joined #openstack-infra | 03:10 | |
| *** tonytan4ever has joined #openstack-infra | 03:11 | |
| *** xarses has quit IRC | 03:13 | |
| *** clenimar_ has quit IRC | 03:13 | |
| *** xarses has joined #openstack-infra | 03:13 | |
| *** piet has joined #openstack-infra | 03:14 | |
| *** cardeois_ has quit IRC | 03:15 | |
| *** thorst_ has joined #openstack-infra | 03:15 | |
| *** bhunter71 has quit IRC | 03:16 | |
| *** xarses has quit IRC | 03:16 | |
| *** Srinu has quit IRC | 03:16 | |
| *** xarses has joined #openstack-infra | 03:16 | |
| *** shashank_hegde has joined #openstack-infra | 03:19 | |
| *** xarses has quit IRC | 03:21 | |
| *** xarses has joined #openstack-infra | 03:21 | |
| *** sarob has joined #openstack-infra | 03:21 | |
| *** thorst_ has quit IRC | 03:22 | |
| *** yanyanhu has quit IRC | 03:25 | |
| *** xarses has quit IRC | 03:25 | |
| *** xarses has joined #openstack-infra | 03:25 | |
| *** amotoki_ has joined #openstack-infra | 03:25 | |
| *** rdx_matt has quit IRC | 03:26 | |
| *** sarob has quit IRC | 03:26 | |
| *** xarses has quit IRC | 03:26 | |
| *** xarses has joined #openstack-infra | 03:27 | |
| *** treaki__ has joined #openstack-infra | 03:27 | |
| *** sputnik13 has joined #openstack-infra | 03:27 | |
| *** amotoki has quit IRC | 03:29 | |
| *** treaki_ has quit IRC | 03:31 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: yum-minimal: set locale.conf and tz in chroot https://review.openstack.org/342544 | 03:31 |
| *** amotoki_ has quit IRC | 03:31 | |
| *** shashank_hegde has quit IRC | 03:31 | |
| ianw | pabelanger / greghaynes / jeblair : ^ this *might* explain the ansible issues? forgotten locale? | 03:31 |
| *** pahuang has quit IRC | 03:33 | |
| *** links has quit IRC | 03:33 | |
| *** xarses has quit IRC | 03:34 | |
| *** xarses has joined #openstack-infra | 03:34 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: yum-minimal: set locale.conf and tz in chroot https://review.openstack.org/342544 | 03:35 |
| *** armax has quit IRC | 03:35 | |
| *** sarob has joined #openstack-infra | 03:35 | |
| *** amitgandhinz has joined #openstack-infra | 03:36 | |
| *** sputnik13 has quit IRC | 03:39 | |
| *** sarob has quit IRC | 03:40 | |
| *** sputnik13 has joined #openstack-infra | 03:40 | |
| *** armax has joined #openstack-infra | 03:40 | |
| *** amitgandhinz has quit IRC | 03:40 | |
| *** cardeois_ has joined #openstack-infra | 03:41 | |
| *** piet has quit IRC | 03:45 | |
| *** pahuang has joined #openstack-infra | 03:45 | |
| *** cardeois_ has quit IRC | 03:46 | |
| *** Srinu has joined #openstack-infra | 03:50 | |
| Srinu | Hi | 03:50 |
| *** sputnik13 has quit IRC | 03:51 | |
| *** sputnik13 has joined #openstack-infra | 03:51 | |
| Srinu | For jenkins ci in zuul configuration file i am using 2 jobs under check pipeline and if one job success then it is terminating the other job and vice versa. did any one face this issue | 03:52 |
| Srinu | slave went offline | 03:52 |
| *** yuanying has joined #openstack-infra | 03:53 | |
| *** PollyZ has joined #openstack-infra | 03:55 | |
| *** clenimar_ has joined #openstack-infra | 03:55 | |
| *** gyee has quit IRC | 03:55 | |
| openstackgerrit | Liping Mao proposed openstack-infra/project-config: Add python35-nv jobs in Kuryr https://review.openstack.org/342531 | 03:59 |
| *** shashank_hegde has joined #openstack-infra | 04:00 | |
| *** karthikp_ has joined #openstack-infra | 04:01 | |
| *** sputnik13 has quit IRC | 04:02 | |
| *** [1]Thelo has joined #openstack-infra | 04:03 | |
| *** bhavik has joined #openstack-infra | 04:05 | |
| *** sdake has quit IRC | 04:05 | |
| *** amotoki has joined #openstack-infra | 04:05 | |
| *** Thelo has quit IRC | 04:05 | |
| *** [1]Thelo is now known as Thelo | 04:05 | |
| *** jcoufal has quit IRC | 04:06 | |
| *** adrian_otto has joined #openstack-infra | 04:11 | |
| *** sputnik13 has joined #openstack-infra | 04:12 | |
| *** amotoki has quit IRC | 04:12 | |
| *** PollyZ has quit IRC | 04:13 | |
| *** PollyZ has joined #openstack-infra | 04:16 | |
| *** thorst_ has joined #openstack-infra | 04:20 | |
| * craige wonders if there is any good doco about mocking up a VM to think it's infra-managed to test puppet modules? | 04:21 | |
| *** amotoki has joined #openstack-infra | 04:22 | |
| *** amotoki has quit IRC | 04:22 | |
| *** karthikp_ has quit IRC | 04:22 | |
| nibalizer | craige: just applying openstack_project::server is about what you have to do | 04:22 |
| *** amotoki has joined #openstack-infra | 04:22 | |
| *** rdx_matt has joined #openstack-infra | 04:22 | |
| * craige follows nibalizer's lead | 04:23 | |
| *** adrian_otto has quit IRC | 04:23 | |
| *** asettle has joined #openstack-infra | 04:24 | |
| *** thorst_ has quit IRC | 04:27 | |
| *** maishsk has quit IRC | 04:27 | |
| *** maishsk has joined #openstack-infra | 04:29 | |
| *** asettle has quit IRC | 04:29 | |
| *** dimtruck is now known as zz_dimtruck | 04:32 | |
| *** PollyZ has quit IRC | 04:34 | |
| *** amitgandhinz has joined #openstack-infra | 04:36 | |
| *** eric_lopez has joined #openstack-infra | 04:37 | |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Enable py35 voting on neutron-fwaas https://review.openstack.org/342596 | 04:37 |
| *** links has joined #openstack-infra | 04:39 | |
| *** DaiDV has quit IRC | 04:39 | |
| *** elo has quit IRC | 04:40 | |
| *** isaacb has quit IRC | 04:41 | |
| *** amitgandhinz has quit IRC | 04:41 | |
| *** coolsvap_ has joined #openstack-infra | 04:41 | |
| *** maishsk has quit IRC | 04:44 | |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Enable py35 voting on neutron-vpnaas https://review.openstack.org/342599 | 04:44 |
| *** maishsk has joined #openstack-infra | 04:46 | |
| *** coolsvap has quit IRC | 04:48 | |
| *** coolsvap_ has quit IRC | 04:48 | |
| *** coolsvap_ has joined #openstack-infra | 04:49 | |
| *** coolsvap_ is now known as coolsvap | 04:49 | |
| *** maishsk has quit IRC | 04:50 | |
| *** maishsk has joined #openstack-infra | 04:52 | |
| openstackgerrit | Merged openstack-infra/project-config: Add python35-nv jobs in Kuryr https://review.openstack.org/342531 | 04:52 |
| openstackgerrit | Merged openstack-infra/project-config: Turn zmq gates to voting https://review.openstack.org/340439 | 04:53 |
| openstackgerrit | Merged openstack-infra/project-config: Install default-jre version for searchlight jobs https://review.openstack.org/339636 | 04:54 |
| *** rdx_matt has quit IRC | 04:55 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.concurrency https://review.openstack.org/341277 | 04:56 |
| *** sputnik13 has quit IRC | 04:56 | |
| *** zhurong has quit IRC | 04:58 | |
| AJaeger | ianw, jhesketh, could either of you review two of my changes for project-config, please? https://review.openstack.org/335808 and https://review.openstack.org/341307 | 04:59 |
| jhesketh | sure, looking | 05:00 |
| AJaeger | thanks | 05:00 |
| *** zhurong has joined #openstack-infra | 05:00 | |
| jhesketh | AJaeger: re 335808 did you have a way of checking which projects might have needed it? | 05:01 |
| AJaeger | jhesketh: I searched for all projects with those requirements via codesearch - and merged chnages to all of them | 05:01 |
| jhesketh | great, thanks | 05:02 |
| jhesketh | there's a small chance some project pull it in through a sub-dep, but I figure we can deal with those when it comes up | 05:02 |
| AJaeger | it were 5 or 6 projects, so not many... | 05:03 |
| openstackgerrit | Renat Akhmerov proposed openstack-infra/project-config: Add new "mistral-lib" project https://review.openstack.org/340844 | 05:03 |
| AJaeger | but yes, if one is missed, we can quickly add an other-requirements file there | 05:03 |
| *** rajinir has quit IRC | 05:05 | |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Enable py35 voting on neutron-vpnaas https://review.openstack.org/342606 | 05:09 |
| *** gongysh has joined #openstack-infra | 05:09 | |
| *** psachin has joined #openstack-infra | 05:10 | |
| *** tkelsey has joined #openstack-infra | 05:11 | |
| openstackgerrit | Merged openstack-infra/project-config: Make py35 voting for reno https://review.openstack.org/341307 | 05:11 |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Enable py35 voting on swift https://review.openstack.org/342606 | 05:12 |
| *** armax_ has joined #openstack-infra | 05:13 | |
| *** armax has quit IRC | 05:14 | |
| *** armax_ is now known as armax | 05:14 | |
| *** tkelsey has quit IRC | 05:15 | |
| *** netsinn has joined #openstack-infra | 05:16 | |
| *** sdake has joined #openstack-infra | 05:17 | |
| *** gouthamr_ has quit IRC | 05:17 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Add dbus-1 development libraries" https://review.openstack.org/335808 | 05:17 |
| *** sputnik13 has joined #openstack-infra | 05:19 | |
| *** netsin has quit IRC | 05:20 | |
| ianw | AJaeger: cool, thanks for cleaning up the keyring thing | 05:20 |
| *** itisha has quit IRC | 05:20 | |
| *** armax_ has joined #openstack-infra | 05:22 | |
| *** armax has quit IRC | 05:23 | |
| *** armax_ is now known as armax | 05:23 | |
| *** armax has quit IRC | 05:25 | |
| *** thorst_ has joined #openstack-infra | 05:25 | |
| *** pt_15 has quit IRC | 05:26 | |
| AJaeger | ;) | 05:26 |
| AJaeger | ianw: yeah - was only a few repos and I'd like to shrink, not grow, the default list... | 05:27 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update clamav plugin to use convert xml https://review.openstack.org/341249 | 05:28 |
| AJaeger | if anybody has time for reviews, I'd like to help land https://review.openstack.org/340844 and https://review.openstack.org/#/c/341671/ for project-config, please | 05:28 |
| AJaeger | Wait, 341671 has a non-merged dependency ;( | 05:29 |
| openstackgerrit | Tony Breeds proposed openstack-infra/irc-meetings: Add IRC nick for nova-notification-meeting https://review.openstack.org/341950 | 05:31 |
| openstackgerrit | Tony Breeds proposed openstack-infra/irc-meetings: Set meeting chairs to the canonical form https://review.openstack.org/251195 | 05:31 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.context https://review.openstack.org/341285 | 05:32 |
| *** thorst_ has quit IRC | 05:32 | |
| *** Sukhdev has joined #openstack-infra | 05:33 | |
| *** pgadiya has joined #openstack-infra | 05:33 | |
| *** aeng has quit IRC | 05:33 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update scoverage to use convert xml https://review.openstack.org/340576 | 05:34 |
| ianw | AJaeger: is 340844 ready? governance not merged? | 05:34 |
| AJaeger | ianw: we first merge project-config, then governance merges - for official projects. But only if there's PTL+1 on the project-config. | 05:36 |
| AJaeger | And Renat is PTL | 05:36 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.reports https://review.openstack.org/341290 | 05:37 |
| AJaeger | ianw: that way we don't need to wait for governance to merge first - and if the repo gets renamed, it would give another review... | 05:37 |
| *** amitgandhinz has joined #openstack-infra | 05:37 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.serialization https://review.openstack.org/341292 | 05:37 |
| *** piet has joined #openstack-infra | 05:37 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslotest https://review.openstack.org/341294 | 05:37 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.versionedobjects https://review.openstack.org/341295 | 05:37 |
| AJaeger | ianw: But we need these depends-on/needed-by for proper linking. ttx approves the governance change quickly a spure housekeeping, there's no vote on them | 05:37 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on stevedore https://review.openstack.org/341297 | 05:38 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.log https://review.openstack.org/341288 | 05:38 |
| *** pahuang has quit IRC | 05:38 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.messaging https://review.openstack.org/341272 | 05:38 |
| *** coolsvap has quit IRC | 05:42 | |
| *** amitgandhinz has quit IRC | 05:42 | |
| *** coolsvap has joined #openstack-infra | 05:42 | |
| *** markvoelker has quit IRC | 05:43 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/irc-meetings: Proposing OpenStack SWG Meeting https://review.openstack.org/338134 | 05:43 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update WhiteSource plugin https://review.openstack.org/341256 | 05:44 |
| *** amotoki has quit IRC | 05:44 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update WhiteSource plugin https://review.openstack.org/341256 | 05:47 |
| *** nadya has joined #openstack-infra | 05:47 | |
| *** asettle has joined #openstack-infra | 05:48 | |
| *** pahuang has joined #openstack-infra | 05:50 | |
| *** amotoki has joined #openstack-infra | 05:51 | |
| *** sdake has quit IRC | 05:51 | |
| *** asettle has quit IRC | 05:53 | |
| *** rdx_matt has joined #openstack-infra | 05:53 | |
| *** camunoz has quit IRC | 05:54 | |
| *** tonytan4ever has quit IRC | 05:54 | |
| *** piet has quit IRC | 05:55 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.rootwrap https://review.openstack.org/341264 | 05:55 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on futurist https://review.openstack.org/341276 | 06:00 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on debtcollector https://review.openstack.org/341274 | 06:00 |
| *** nadya has quit IRC | 06:00 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on taskflow https://review.openstack.org/341269 | 06:00 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on automaton https://review.openstack.org/341271 | 06:01 |
| openstackgerrit | Merged openstack-infra/project-config: Add new "mistral-lib" project https://review.openstack.org/340844 | 06:01 |
| *** Na3iL has joined #openstack-infra | 06:04 | |
| *** timello has quit IRC | 06:04 | |
| *** tonytan4ever has joined #openstack-infra | 06:04 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Test constraints with zuul-cloner in post queue https://review.openstack.org/342623 | 06:09 |
| *** ramishra has quit IRC | 06:10 | |
| *** timello has joined #openstack-infra | 06:10 | |
| *** ramishra has joined #openstack-infra | 06:11 | |
| *** nadya has joined #openstack-infra | 06:12 | |
| *** sdake has joined #openstack-infra | 06:13 | |
| *** Dragon535 has joined #openstack-infra | 06:15 | |
| *** gongysh has quit IRC | 06:17 | |
| *** rcernin has joined #openstack-infra | 06:17 | |
| *** zhurong has quit IRC | 06:19 | |
| *** aviau has quit IRC | 06:20 | |
| *** zhurong has joined #openstack-infra | 06:21 | |
| *** aviau has joined #openstack-infra | 06:21 | |
| *** gongysh has joined #openstack-infra | 06:23 | |
| *** amotoki has quit IRC | 06:23 | |
| *** asettle has joined #openstack-infra | 06:24 | |
| *** amotoki has joined #openstack-infra | 06:24 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Test constraints with zuul-cloner in post queue https://review.openstack.org/342623 | 06:24 |
| *** rdx_matt has quit IRC | 06:26 | |
| *** asettle has quit IRC | 06:29 | |
| *** markvoelker has joined #openstack-infra | 06:30 | |
| *** nadya has quit IRC | 06:31 | |
| *** thorst_ has joined #openstack-infra | 06:31 | |
| *** openstackgerrit has quit IRC | 06:33 | |
| *** openstackgerrit has joined #openstack-infra | 06:33 | |
| *** puiterwijk|mnt is now known as puiterwijk | 06:34 | |
| *** thorst_ has quit IRC | 06:37 | |
| *** chlong has quit IRC | 06:38 | |
| *** amitgandhinz has joined #openstack-infra | 06:38 | |
| *** markvoelker has quit IRC | 06:39 | |
| *** Sukhdev has quit IRC | 06:40 | |
| *** amitgandhinz has quit IRC | 06:43 | |
| *** gongysh has quit IRC | 06:43 | |
| *** amotoki has quit IRC | 06:44 | |
| *** ccamacho|out has quit IRC | 06:47 | |
| *** sdake_ has joined #openstack-infra | 06:49 | |
| *** coolsvap_ has joined #openstack-infra | 06:51 | |
| *** coolsvap_ has quit IRC | 06:52 | |
| *** sdake has quit IRC | 06:52 | |
| *** xarses has quit IRC | 06:54 | |
| *** hashar has joined #openstack-infra | 06:56 | |
| *** andreas_s has joined #openstack-infra | 06:56 | |
| openstackgerrit | Merged openstack/diskimage-builder: package-installs: add list to arch and "not-arch" list https://review.openstack.org/335748 | 06:58 |
| Srinu | i configured a zuul with check pipeline. in check pipeline i used two jobs and if one jobs success it is killing other job and vice versa. did anyone face | 06:58 |
| *** binbincong has quit IRC | 06:59 | |
| *** binbincong has joined #openstack-infra | 07:00 | |
| *** amotoki has joined #openstack-infra | 07:03 | |
| *** gongysh has joined #openstack-infra | 07:03 | |
| *** jlanoux has joined #openstack-infra | 07:04 | |
| *** javeriak has joined #openstack-infra | 07:05 | |
| *** mrmartin has joined #openstack-infra | 07:07 | |
| *** arxcruz has joined #openstack-infra | 07:10 | |
| openstackgerrit | Markus Zoeller (markus_z) proposed openstack-infra/release-tools: release_notes.py: search your feelings https://review.openstack.org/342053 | 07:12 |
| *** asettle has joined #openstack-infra | 07:13 | |
| *** tesseract- has joined #openstack-infra | 07:15 | |
| *** sdake_ has quit IRC | 07:16 | |
| *** yamahata has joined #openstack-infra | 07:19 | |
| *** jlanoux has quit IRC | 07:19 | |
| *** mikelk has joined #openstack-infra | 07:21 | |
| *** jlanoux has joined #openstack-infra | 07:21 | |
| *** shardy has joined #openstack-infra | 07:22 | |
| *** rdx_matt has joined #openstack-infra | 07:23 | |
| *** mikelk has quit IRC | 07:23 | |
| *** sputnik13 has quit IRC | 07:23 | |
| *** rossella_s has joined #openstack-infra | 07:24 | |
| *** mrmartin has quit IRC | 07:25 | |
| *** rdx_matt has quit IRC | 07:25 | |
| *** rdx_matt has joined #openstack-infra | 07:26 | |
| *** pcaruana has joined #openstack-infra | 07:27 | |
| *** tonytan4ever has quit IRC | 07:30 | |
| *** kong has quit IRC | 07:30 | |
| *** d0ugal has joined #openstack-infra | 07:31 | |
| *** d0ugal has quit IRC | 07:31 | |
| *** coolsvap has quit IRC | 07:31 | |
| *** coolsvap has joined #openstack-infra | 07:31 | |
| *** d0ugal has joined #openstack-infra | 07:31 | |
| *** d0ugal has quit IRC | 07:31 | |
| *** d0ugal has joined #openstack-infra | 07:31 | |
| *** mikelk has joined #openstack-infra | 07:32 | |
| *** esikachev has joined #openstack-infra | 07:32 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:34 | |
| *** thorst_ has joined #openstack-infra | 07:35 | |
| *** jaosorior has joined #openstack-infra | 07:35 | |
| *** coolsvap has quit IRC | 07:37 | |
| *** coolsvap has joined #openstack-infra | 07:37 | |
| *** tkelsey has joined #openstack-infra | 07:38 | |
| *** amitgandhinz has joined #openstack-infra | 07:39 | |
| *** jlanoux has quit IRC | 07:42 | |
| *** ifarkas has joined #openstack-infra | 07:42 | |
| *** daemontool has joined #openstack-infra | 07:43 | |
| *** thorst_ has quit IRC | 07:43 | |
| *** amitgandhinz has quit IRC | 07:43 | |
| *** Ravikiran_K has joined #openstack-infra | 07:44 | |
| *** matthewbodkin has joined #openstack-infra | 07:45 | |
| *** nadya has joined #openstack-infra | 07:45 | |
| *** asettle has quit IRC | 07:46 | |
| *** Dragon535 has left #openstack-infra | 07:47 | |
| *** amotoki has quit IRC | 07:48 | |
| *** asettle has joined #openstack-infra | 07:51 | |
| *** rwsu has quit IRC | 07:51 | |
| *** gongysh has quit IRC | 07:52 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/342657 | 07:56 |
| *** sflanigan has quit IRC | 07:59 | |
| openstackgerrit | James Polley proposed openstack-dev/pbr: Fix handling of old git log output https://review.openstack.org/339392 | 07:59 |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** tchaypo is now known as tchaypo_vacay | 08:00 | |
| openstackgerrit | Merged openstack-infra/zuul: Do not contest locks in PythonGit https://review.openstack.org/342462 | 08:02 |
| *** derekh has joined #openstack-infra | 08:04 | |
| *** dstufft has quit IRC | 08:05 | |
| *** dstufft has joined #openstack-infra | 08:06 | |
| jd__ | is everyone aware of the sudo thing that make tests fail like http://logs.openstack.org/36/342636/1/check/gate-ceilometer-python27/1d93ea1/console.html#_2016-07-15_07_04_07_055369 ? | 08:06 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Update to use quintupleo https://review.openstack.org/336309 | 08:07 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Deploy full net-iso environment https://review.openstack.org/341161 | 08:08 |
| *** mrmartin has joined #openstack-infra | 08:12 | |
| *** eezhova has joined #openstack-infra | 08:14 | |
| *** e0ne has joined #openstack-infra | 08:21 | |
| *** mrmartin has quit IRC | 08:22 | |
| *** markvoelker has joined #openstack-infra | 08:23 | |
| *** lucas|afk is now known as lucasagomes | 08:23 | |
| *** mrmartin has joined #openstack-infra | 08:26 | |
| *** markvoelker has quit IRC | 08:27 | |
| *** yamamoto has quit IRC | 08:29 | |
| *** tonytan4ever has joined #openstack-infra | 08:30 | |
| *** Na3iL has quit IRC | 08:33 | |
| *** Na3iL has joined #openstack-infra | 08:33 | |
| *** sorantis has joined #openstack-infra | 08:35 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: TEST IRONIC SSL - DO NOT MERGE https://review.openstack.org/342669 | 08:35 |
| *** tonytan4ever has quit IRC | 08:36 | |
| d0ugal | I can't be added as a reviewer by other users, I think this is because I have two gerrit accounts somehow. Is there a way to fix this? http://i.imgur.com/FJmKxY3.png | 08:38 |
| d0ugal | As far as I can remember I have only ever signed up once... | 08:38 |
| openstackgerrit | Jon-Paul Sullivan (j3p0uk) proposed openstack-infra/zuul: Make the cloner strategy clearer https://review.openstack.org/177201 | 08:39 |
| *** amitgandhinz has joined #openstack-infra | 08:39 | |
| *** Na3iL has quit IRC | 08:40 | |
| *** Na3iL has joined #openstack-infra | 08:40 | |
| *** thorst_ has joined #openstack-infra | 08:41 | |
| *** gongysh has joined #openstack-infra | 08:41 | |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Make py35 voting for Trove https://review.openstack.org/342671 | 08:41 |
| *** shashank_hegde has quit IRC | 08:42 | |
| *** amitgandhinz has quit IRC | 08:44 | |
| *** Na3iL has quit IRC | 08:44 | |
| *** dtantsur|afk is now known as dtantsur | 08:44 | |
| *** Na3iL has joined #openstack-infra | 08:45 | |
| mordred | jd__: I have not seen that, but that does look weird to me - maybe AJaeger has seen something? | 08:45 |
| *** gongysh has quit IRC | 08:45 | |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Make py35 voting for Magnum https://review.openstack.org/342677 | 08:46 |
| *** gibi_ is now known as gibi | 08:47 | |
| *** thorst_ has quit IRC | 08:47 | |
| rcarrillocruz | mordred: if you remember the 500 hundred on my role test when creating nova instances, the fact that i put an artificial delay at the beginning of the test fixed it was purely coincidental | 08:48 |
| rcarrillocruz | i held a node last night | 08:48 |
| rcarrillocruz | the issue seems on oslo-config package | 08:48 |
| rcarrillocruz | http://paste.openstack.org/show/532918/ | 08:49 |
| rcarrillocruz | :/ | 08:49 |
| mordred | rcarrillocruz: that does not look happy | 08:49 |
| openstackgerrit | Tony Breeds proposed openstack-infra/irc-meetings: Set meeting chairs to the canonical form https://review.openstack.org/251195 | 08:49 |
| mordred | johnthetubaguy: ^^ you seen anything like that before? | 08:49 |
| rcarrillocruz | https://review.openstack.org/#/c/341542/ is pending | 08:50 |
| rcarrillocruz | who usually approves those changes on requirements? are there liaisons that do that ? | 08:50 |
| mordred | rcarrillocruz: not sure - we physically can, but I'm not sure of flow. any chance you feel like re-checking a a shade test with a depends-on that requirements patch? I think that'll get us an updated devstack | 08:51 |
| rcarrillocruz | oh, good idea | 08:52 |
| rcarrillocruz | sec | 08:52 |
| *** binbincong has quit IRC | 08:53 | |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Make py35 voting for Sahara https://review.openstack.org/342681 | 08:53 |
| * Zara notices https://review.openstack.org/#/c/341901/ has been merged, yayyyy! | 08:55 | |
| Zara | (I'd +1'd the patch that was abandoned in its favour) | 08:55 |
| Zara | thanks, everyone! | 08:56 |
| *** javeriak has quit IRC | 08:57 | |
| Zara | I guess now I can do another shameless plug for zaro's its-storyboard integration patches. These are to intergrate gerrit with storyboad, so that task-statuses can change automatically to match those in review: https://review.openstack.org/#/q/topic:its-storyboard | 08:57 |
| Zara | *integrate | 08:57 |
| AJaeger | rcarrillocruz: for requirements there's a core team now - they're on #openstack-requirements | 08:58 |
| rcarrillocruz | ah | 08:58 |
| rcarrillocruz | thx AJaeger | 08:58 |
| AJaeger | rcarrillocruz: you should have permissions to do so as infra-admin - intention is to use that in emergencies... | 08:58 |
| AJaeger | d0ugal: an admin needs to do some gerrit magic for you, perhaps rcarrillocruz and mordred know what to do - otherwise ask again later, please | 08:59 |
| rcarrillocruz | yeah, i will wait a bit for someone else to look, otherwise i can just approve cos the package 3.12 looks broken | 09:00 |
| AJaeger | mordred, jd__ There was no change in python27, was there a change in ceilometer? | 09:00 |
| d0ugal | AJaeger: Will do, thanks. I thought it might be a bit early in the day :) | 09:00 |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/342657 | 09:01 |
| AJaeger | rcarrillocruz: I suggest to go to #openstack-requirements and ask ;) | 09:01 |
| rcarrillocruz | done that yeah | 09:01 |
| *** kzaitsev_mb has quit IRC | 09:03 | |
| johnthetubaguy | mordred: thats a new one on me I am afraid | 09:03 |
| openstackgerrit | Evgeny Sikachev proposed openstack-infra/project-config: [Sahara] Added transient jobs in gate https://review.openstack.org/342688 | 09:06 |
| *** jaosorior is now known as jaosorior_brb | 09:11 | |
| *** yamamoto has joined #openstack-infra | 09:12 | |
| *** yamamoto_ has joined #openstack-infra | 09:13 | |
| *** yamamoto_ has quit IRC | 09:13 | |
| AJaeger | project-config cores, here's a list of 11 reviews that have already a +2 and could use another +2A - if any of you have time, please review: http://paste.openstack.org/show/533027/ | 09:13 |
| *** zhenguo_ is now known as zhenguo | 09:14 | |
| *** DaiDV has joined #openstack-infra | 09:15 | |
| *** yamahata has quit IRC | 09:16 | |
| *** yamamoto has quit IRC | 09:17 | |
| *** markvoelker has joined #openstack-infra | 09:17 | |
| *** DaiDV has quit IRC | 09:18 | |
| *** jaosorior_brb has quit IRC | 09:19 | |
| *** binbincong has joined #openstack-infra | 09:20 | |
| rcarrillocruz | eugh | 09:21 |
| rcarrillocruz | now got past oslo-config issue, fails due to oslo-utils package :/ | 09:21 |
| rcarrillocruz | http://paste.openstack.org/show/533028/ | 09:21 |
| *** markvoelker has quit IRC | 09:21 | |
| AJaeger | rcarrillocruz: is that just your change - or is that seen everywhere else? | 09:22 |
| rcarrillocruz | i've asked around and just aware of my change | 09:23 |
| rcarrillocruz | however | 09:23 |
| rcarrillocruz | it's similar | 09:23 |
| rcarrillocruz | the job installs something on the tox venv | 09:23 |
| *** yamamoto has joined #openstack-infra | 09:23 | |
| rcarrillocruz | that is newer to what the oslo-utils error message displays | 09:23 |
| rcarrillocruz | the job installs oslo.utils 3.16.0 | 09:23 |
| rcarrillocruz | but the error complains about 3.15 | 09:24 |
| rcarrillocruz | i'll see if there's a bump change for oslo.utils requirements | 09:24 |
| rcarrillocruz | https://review.openstack.org/#/c/342155/1 | 09:25 |
| *** yamamoto has quit IRC | 09:25 | |
| *** pahuang has quit IRC | 09:25 | |
| *** john-davidge has joined #openstack-infra | 09:27 | |
| AJaeger | rcarrillocruz: I guess 3.15 is out for some time. Why are you hitting this today - and nobody else? What's different? | 09:27 |
| rcarrillocruz | very odd... | 09:27 |
| *** john-dav_ has joined #openstack-infra | 09:28 | |
| AJaeger | rcarrillocruz: exactly. Do we have broken files on our mirror - that were only updated this morning? (the pip wheels)? | 09:28 |
| coolsvap | AJaeger, we are observing this across multiple projects | 09:29 |
| AJaeger | coolsvap: Since when? | 09:30 |
| coolsvap | nova for example its not broken since we have not released u-c update | 09:30 |
| coolsvap | i think since yesterday evening | 09:30 |
| *** jpich has joined #openstack-infra | 09:30 | |
| openstackgerrit | Witold Bedyk proposed openstack-infra/project-config: Add gate-tempest for monasca-log-api https://review.openstack.org/342700 | 09:31 |
| *** john-davidge has quit IRC | 09:31 | |
| *** yamamoto has joined #openstack-infra | 09:33 | |
| *** Julien-zte has quit IRC | 09:33 | |
| *** dizquierdo has joined #openstack-infra | 09:38 | |
| *** YorikSar_ has quit IRC | 09:39 | |
| *** YorikSar has joined #openstack-infra | 09:40 | |
| *** amitgandhinz has joined #openstack-infra | 09:40 | |
| *** mrmartin has quit IRC | 09:40 | |
| *** javeriak has joined #openstack-infra | 09:41 | |
| *** zshuo has quit IRC | 09:42 | |
| *** yuanying is now known as yuanying-afk | 09:43 | |
| *** tosky has joined #openstack-infra | 09:44 | |
| openstackgerrit | Evgeny Sikachev proposed openstack-infra/project-config: [Sahara] Added transient jobs in gate https://review.openstack.org/342688 | 09:44 |
| *** amitgandhinz has quit IRC | 09:45 | |
| *** thorst_ has joined #openstack-infra | 09:46 | |
| *** eezhova has quit IRC | 09:46 | |
| *** piet has joined #openstack-infra | 09:46 | |
| openstackgerrit | Evgeny Sikachev proposed openstack-infra/project-config: [Sahara] Added transient jobs in gate https://review.openstack.org/342688 | 09:46 |
| *** pfallenop has joined #openstack-infra | 09:47 | |
| *** pfallenop has joined #openstack-infra | 09:47 | |
| *** pfallenop has quit IRC | 09:48 | |
| *** eezhova has joined #openstack-infra | 09:49 | |
| mordred | rcarrillocruz: so perhaps the problem currently is that we're not honoring upper-constraints somehow? Also, why would that matter for shade ... when did we start pulling in oslo-utils? | 09:50 |
| * mordred is confused | 09:50 | |
| *** berendt has joined #openstack-infra | 09:50 | |
| *** pfallenop has joined #openstack-infra | 09:51 | |
| *** zul has joined #openstack-infra | 09:52 | |
| *** thorst_ has quit IRC | 09:52 | |
| rcarrillocruz | mordred: i just created a venv and installed requirements, oslo-utils is pulled via python-novaclient | 09:52 |
| rcarrillocruz | http://paste.openstack.org/show/533033/ | 09:52 |
| *** yamamoto has quit IRC | 09:53 | |
| coolsvap | rcarrillocruz, makes sense | 09:54 |
| mordred | oh. right | 09:54 |
| mordred | well - I'm actually going to suggest that we do not make shade honor upper-constraints, since end users are likely not going to either | 09:55 |
| mordred | and if there are problems just pip installing shade we should consider them general bugs | 09:55 |
| *** YorikSar has quit IRC | 09:55 | |
| *** mhickey has joined #openstack-infra | 09:55 | |
| *** YorikSar has joined #openstack-infra | 09:57 | |
| *** john-dav_ is now known as john-davidge | 10:02 | |
| *** Liuqing has joined #openstack-infra | 10:03 | |
| *** zhurong has quit IRC | 10:04 | |
| *** sarob has joined #openstack-infra | 10:05 | |
| *** jlanoux has joined #openstack-infra | 10:05 | |
| *** zhurong has joined #openstack-infra | 10:06 | |
| openstackgerrit | Alex Kavanagh proposed openstack-infra/project-config: Add charm-barbican, and related https://review.openstack.org/340945 | 10:09 |
| *** mrmartin has joined #openstack-infra | 10:09 | |
| *** sarob has quit IRC | 10:10 | |
| *** markvoelker has joined #openstack-infra | 10:11 | |
| *** piet has quit IRC | 10:13 | |
| *** nadya has quit IRC | 10:14 | |
| *** zhurong has quit IRC | 10:15 | |
| *** Liuqing has quit IRC | 10:15 | |
| *** thorst_ has joined #openstack-infra | 10:15 | |
| *** markvoelker has quit IRC | 10:16 | |
| *** Douhet has quit IRC | 10:16 | |
| *** Douhet has joined #openstack-infra | 10:17 | |
| *** piet has joined #openstack-infra | 10:18 | |
| *** mika has quit IRC | 10:18 | |
| *** mika has joined #openstack-infra | 10:18 | |
| *** mika has joined #openstack-infra | 10:18 | |
| *** eezhova has quit IRC | 10:19 | |
| *** baoli has joined #openstack-infra | 10:19 | |
| *** baoli_ has joined #openstack-infra | 10:20 | |
| *** thorst__ has joined #openstack-infra | 10:20 | |
| *** _degorenko|afk is now known as degorenko | 10:21 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:23 | |
| *** thorst_ has quit IRC | 10:23 | |
| *** baoli has quit IRC | 10:24 | |
| *** baoli_ has quit IRC | 10:24 | |
| *** rcernin has quit IRC | 10:26 | |
| *** yamamoto has joined #openstack-infra | 10:27 | |
| *** yamamoto has quit IRC | 10:28 | |
| *** ccamacho has joined #openstack-infra | 10:30 | |
| *** ccamacho1 has joined #openstack-infra | 10:30 | |
| *** sandanar has joined #openstack-infra | 10:30 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Make stack deletion work on all branches https://review.openstack.org/317730 | 10:31 |
| *** tonytan4ever has joined #openstack-infra | 10:32 | |
| *** piet has quit IRC | 10:35 | |
| *** mrmartin has quit IRC | 10:35 | |
| *** tonytan4ever has quit IRC | 10:36 | |
| *** sorantis has quit IRC | 10:38 | |
| *** rodrigods has quit IRC | 10:38 | |
| *** rodrigods has joined #openstack-infra | 10:39 | |
| *** DaiDV has joined #openstack-infra | 10:40 | |
| *** kushal has quit IRC | 10:41 | |
| *** amitgandhinz has joined #openstack-infra | 10:41 | |
| *** DaiDV has quit IRC | 10:42 | |
| *** DaiDV has joined #openstack-infra | 10:43 | |
| *** flepied has joined #openstack-infra | 10:45 | |
| *** amitgandhinz has quit IRC | 10:45 | |
| *** rcernin has joined #openstack-infra | 10:45 | |
| *** rcernin is now known as rcernin|lunch | 10:46 | |
| openstackgerrit | Evgeny Sikachev proposed openstack-infra/project-config: [Sahara] Added transient jobs in gate https://review.openstack.org/342688 | 10:47 |
| *** flepied has quit IRC | 10:50 | |
| *** flepied has joined #openstack-infra | 10:51 | |
| openstackgerrit | Evgeny Sikachev proposed openstack-infra/project-config: [Sahara] Added transient jobs in gate https://review.openstack.org/342688 | 10:53 |
| *** DaiDV has quit IRC | 10:56 | |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Make py35 voting for Ceilometer https://review.openstack.org/342740 | 11:02 |
| *** sorantis has joined #openstack-infra | 11:04 | |
| *** markvoelker has joined #openstack-infra | 11:05 | |
| *** Goneri has joined #openstack-infra | 11:05 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Always run linters for OpenStack-Ansible jobs https://review.openstack.org/342742 | 11:05 |
| openstackgerrit | Evgeny Sikachev proposed openstack-infra/project-config: [Sahara] Added transient jobs in gate https://review.openstack.org/342688 | 11:07 |
| *** abregman has joined #openstack-infra | 11:09 | |
| *** yamamoto has joined #openstack-infra | 11:09 | |
| *** rossella_s has quit IRC | 11:09 | |
| *** markvoelker has quit IRC | 11:10 | |
| *** lucasagomes is now known as lucas-hungry | 11:10 | |
| openstackgerrit | PanFengyun proposed openstack-infra/project-config: Make py35 voting for Keystone https://review.openstack.org/342745 | 11:13 |
| openstackgerrit | Evgeny Sikachev proposed openstack-infra/project-config: [Sahara] Added transient jobs in gate https://review.openstack.org/342688 | 11:14 |
| *** yamamoto has quit IRC | 11:15 | |
| *** kdas__ has joined #openstack-infra | 11:19 | |
| *** dtantsur is now known as dtantsur|brb | 11:20 | |
| *** tpsilva has joined #openstack-infra | 11:21 | |
| *** abregman has quit IRC | 11:21 | |
| *** zhurong has joined #openstack-infra | 11:22 | |
| *** kdas__ is now known as kushal | 11:25 | |
| *** kushal has quit IRC | 11:25 | |
| *** kushal has joined #openstack-infra | 11:25 | |
| *** zul has quit IRC | 11:26 | |
| *** shardy is now known as shardy_lunch | 11:28 | |
| *** yamamoto has joined #openstack-infra | 11:30 | |
| *** eezhova has joined #openstack-infra | 11:32 | |
| *** yamamoto has quit IRC | 11:36 | |
| *** psachin has quit IRC | 11:37 | |
| *** Na3iL has quit IRC | 11:37 | |
| *** ldnunes has joined #openstack-infra | 11:41 | |
| *** eezhova has quit IRC | 11:42 | |
| *** amitgandhinz has joined #openstack-infra | 11:42 | |
| *** flepied has quit IRC | 11:42 | |
| *** Na3iL has joined #openstack-infra | 11:42 | |
| *** rfolco has joined #openstack-infra | 11:45 | |
| *** eezhova has joined #openstack-infra | 11:45 | |
| openstackgerrit | Carlos Camacho proposed openstack-infra/tripleo-ci: Update deprecated references to heat in tripleo.sh https://review.openstack.org/312350 | 11:46 |
| *** amitgandhinz has quit IRC | 11:47 | |
| *** rhallisey has joined #openstack-infra | 11:47 | |
| *** claudiub has joined #openstack-infra | 11:48 | |
| *** derekh has quit IRC | 11:49 | |
| *** julim has joined #openstack-infra | 11:50 | |
| *** Julien-zte has joined #openstack-infra | 11:51 | |
| *** flepied has joined #openstack-infra | 11:54 | |
| *** dtantsur|brb is now known as dtantsur | 11:55 | |
| openstackgerrit | zheng yin proposed openstack-infra/project-config: Enable py35 voting on swift https://review.openstack.org/342758 | 11:55 |
| *** piet has joined #openstack-infra | 11:57 | |
| *** yamamoto has joined #openstack-infra | 11:57 | |
| *** weshay has joined #openstack-infra | 11:58 | |
| *** markvoelker has joined #openstack-infra | 11:59 | |
| *** ccamacho1 is now known as ccamacho_ | 12:01 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Allow comments to be edited by their owner https://review.openstack.org/333418 | 12:01 |
| *** _ari_ has joined #openstack-infra | 12:01 | |
| *** yamamoto has quit IRC | 12:03 | |
| *** lucas-hungry is now known as lucasagomes | 12:03 | |
| *** markvoelker has quit IRC | 12:03 | |
| *** shardy_lunch is now known as shardy | 12:04 | |
| *** dprince has joined #openstack-infra | 12:04 | |
| *** sarob has joined #openstack-infra | 12:05 | |
| *** ddieterly has joined #openstack-infra | 12:07 | |
| coolsvap | AJaeger, I want to update the documentation for addition/removal of jobs to projects, can you point me to the project which I need to update | 12:09 |
| *** sarob has quit IRC | 12:10 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Add a subscribe button to worklists https://review.openstack.org/324550 | 12:14 |
| *** yamamoto has joined #openstack-infra | 12:16 | |
| *** eric_lopez has quit IRC | 12:17 | |
| *** clenimar_ has quit IRC | 12:17 | |
| *** raildo has joined #openstack-infra | 12:18 | |
| *** M-docaedo_vector has quit IRC | 12:18 | |
| *** pradk has joined #openstack-infra | 12:20 | |
| *** ddieterly has quit IRC | 12:20 | |
| *** javeriak has quit IRC | 12:21 | |
| *** yamamoto has quit IRC | 12:21 | |
| *** yamamoto has joined #openstack-infra | 12:22 | |
| *** gordc has joined #openstack-infra | 12:24 | |
| *** javeriak has joined #openstack-infra | 12:24 | |
| *** ddieterly has joined #openstack-infra | 12:25 | |
| *** deadnull has joined #openstack-infra | 12:25 | |
| *** ddieterly has quit IRC | 12:27 | |
| *** yamamoto has quit IRC | 12:28 | |
| *** gouthamr has joined #openstack-infra | 12:28 | |
| sileht | dansmith, dtroyer Hi, can I got some eyes on https://review.openstack.org/#/c/335363/, thanks in advance. | 12:28 |
| *** trown|outtypewww is now known as trown | 12:29 | |
| *** yolanda has joined #openstack-infra | 12:30 | |
| AJaeger | coolsvap: what exactly do you want to describe? | 12:32 |
| AJaeger | coolsvap: do you talk aboutr docs.openstack.org/infra/manual/ ? that's in openstack-infra/infra-manual | 12:32 |
| coolsvap | AJaeger, the requirements jobs were broken yesterday since mistral-extra check-requirements job was removed but project was not removed from requirements | 12:33 |
| coolsvap | I want to add that to documentation | 12:34 |
| openstackgerrit | amrith proposed openstack-infra/irc-meetings: Proposing OpenStack SWG Meeting https://review.openstack.org/338134 | 12:36 |
| amrith | tonyb, please let me know if the description ^^ is better. The review is still a WIP because there's a doodle poll out to find a better time. | 12:36 |
| openstackgerrit | amrith proposed openstack-infra/irc-meetings: Proposing OpenStack SWG Meeting https://review.openstack.org/338134 | 12:37 |
| *** M-docaedo_vector has joined #openstack-infra | 12:38 | |
| openstackgerrit | amrith proposed openstack-infra/irc-meetings: Proposing OpenStack SWG Meeting https://review.openstack.org/338134 | 12:38 |
| *** mescanef has joined #openstack-infra | 12:38 | |
| *** eezhova has quit IRC | 12:38 | |
| mescanef | hi! please review https://review.openstack.org/#/c/340432/ | 12:38 |
| AJaeger | coolsvap: I don't believe that. I agree that it shouldn't have been done but could you tell me how it was broken? | 12:39 |
| coolsvap | AJaeger, there is a job to validate-projects in requirements | 12:40 |
| AJaeger | coolsvap: Ah, forgot about that one - that is newish ;) | 12:40 |
| coolsvap | it checks whether all projects in projects.txt have check-requirements job | 12:40 |
| AJaeger | a good job ;) | 12:40 |
| coolsvap | and its voting | 12:40 |
| coolsvap | so it failed all changes after check-requirement job was removed from project-config | 12:41 |
| AJaeger | coolsvap: I gave a +2 on the mistral-extra - and missed during my review to check that it was removed from requirements. | 12:41 |
| AJaeger | coolsvap: I normally ask to order these two - so for removal, check that requirements is removed. | 12:41 |
| AJaeger | coolsvap: Document it best in REVIEWING.rst in project-config. This is too detailed for the infra manual IMHO | 12:42 |
| *** amitgandhinz has joined #openstack-infra | 12:43 | |
| coolsvap | AJaeger, sure I think I will add a small note in infra manual and a change in REVIEWING.rst | 12:43 |
| coolsvap | of project-config | 12:43 |
| *** eezhova has joined #openstack-infra | 12:43 | |
| coolsvap | AJaeger, thanks for the info | 12:43 |
| *** coolsvap is now known as coolsvap|afk | 12:43 | |
| AJaeger | Thanks, coolsvap|afk ! | 12:44 |
| *** amitgandhinz has quit IRC | 12:47 | |
| *** yolanda has quit IRC | 12:48 | |
| *** Liuqing has joined #openstack-infra | 12:49 | |
| *** thorst__ has quit IRC | 12:49 | |
| phschwartz | asselin__: did you get a chance to take a look at that spec review? I want to round it out and fixed for all so we can get it merged. I am setting time asside this week to knock out all the code and would prefer the spec being inplace before putting the reviews up. | 12:50 |
| phschwartz | mordred: your eyes on it would be appreciated too. https://review.openstack.org/#/c/239810/ | 12:51 |
| *** kushal has quit IRC | 12:51 | |
| *** ddieterly has joined #openstack-infra | 12:52 | |
| *** ccamacho has quit IRC | 12:52 | |
| *** ccamacho_ has quit IRC | 12:53 | |
| *** rlandy has joined #openstack-infra | 12:55 | |
| *** sorantis has quit IRC | 12:55 | |
| *** binbincong has quit IRC | 13:00 | |
| *** ddieterly has quit IRC | 13:01 | |
| Zara | nibalizer: fungi: bit out of the blue, but thanks again for all your help setting up storyboard-dev. it has been absolutely invaluable to us demonstating things to people so far! :) I was just reading through test stories today and thinking about that, so might as well let you know. :) | 13:02 |
| *** xyang1 has joined #openstack-infra | 13:03 | |
| *** markvoelker has joined #openstack-infra | 13:03 | |
| *** javeriak has quit IRC | 13:03 | |
| Zara | I feel like really I should ping everyone involved in the patches, which I think is almost all of infra, but I don't know how to ping all infra and that might also be annoying. so, er, if you've been involved, thanks! :) | 13:05 |
| *** jpich has quit IRC | 13:06 | |
| *** amcguign has joined #openstack-infra | 13:07 | |
| *** amcguign has quit IRC | 13:08 | |
| *** kdas__ has joined #openstack-infra | 13:08 | |
| *** arxcruz has quit IRC | 13:10 | |
| *** mdrabe has joined #openstack-infra | 13:10 | |
| *** kdas__ has quit IRC | 13:11 | |
| *** pgadiya has quit IRC | 13:11 | |
| *** thorst_ has joined #openstack-infra | 13:11 | |
| *** thorst_ has quit IRC | 13:12 | |
| *** thorst_ has joined #openstack-infra | 13:12 | |
| *** javeriak has joined #openstack-infra | 13:12 | |
| *** thiagop has joined #openstack-infra | 13:13 | |
| *** weshay is now known as weshay_mtg | 13:14 | |
| *** kdas__ has joined #openstack-infra | 13:16 | |
| *** berendt has quit IRC | 13:17 | |
| *** yaume has joined #openstack-infra | 13:19 | |
| openstackgerrit | Vladimir Kozhukalov proposed openstack-infra/project-config: Revert "Disable all gate jobs for fuel-mirror master" https://review.openstack.org/342843 | 13:19 |
| *** javeriak has quit IRC | 13:20 | |
| *** Na3iL has quit IRC | 13:21 | |
| *** maishsk has quit IRC | 13:22 | |
| *** esberglu has joined #openstack-infra | 13:24 | |
| *** kdas__ is now known as kushal | 13:25 | |
| *** kushal has joined #openstack-infra | 13:25 | |
| *** bhavik has quit IRC | 13:25 | |
| *** yamamoto has joined #openstack-infra | 13:29 | |
| *** ccamacho has joined #openstack-infra | 13:31 | |
| *** ccamacho1 has joined #openstack-infra | 13:31 | |
| *** Na3iL has joined #openstack-infra | 13:31 | |
| *** cardeois_ has joined #openstack-infra | 13:36 | |
| *** tonytan4ever has joined #openstack-infra | 13:36 | |
| *** yamamoto has quit IRC | 13:36 | |
| *** ddieterly has joined #openstack-infra | 13:36 | |
| *** ccamacho2 has joined #openstack-infra | 13:37 | |
| *** zhurong has quit IRC | 13:38 | |
| *** ccamacho2 has left #openstack-infra | 13:41 | |
| *** ccamacho has quit IRC | 13:41 | |
| *** ccamacho1 has quit IRC | 13:41 | |
| *** ccamacho has joined #openstack-infra | 13:42 | |
| *** amitgandhinz has joined #openstack-infra | 13:43 | |
| nibalizer | Zara: dawwww | 13:45 |
| *** maishsk has joined #openstack-infra | 13:46 | |
| *** bhunter71 has joined #openstack-infra | 13:46 | |
| *** sdake has joined #openstack-infra | 13:47 | |
| Zara | nibalizer: :) | 13:48 |
| *** ociuhandu has quit IRC | 13:49 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-jenkins: Make the Gerrit SSH key configurable https://review.openstack.org/342863 | 13:50 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Use the Release Bot account on the signing node https://review.openstack.org/342864 | 13:50 |
| *** sdake_ has joined #openstack-infra | 13:50 | |
| *** sdake has quit IRC | 13:52 | |
| *** sdake_ has quit IRC | 13:52 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: add milestone-checkup command https://review.openstack.org/342867 | 13:53 |
| *** pt_15 has joined #openstack-infra | 13:53 | |
| *** slaweq has quit IRC | 13:54 | |
| *** r-mibu has quit IRC | 13:54 | |
| *** r-mibu has joined #openstack-infra | 13:55 | |
| *** PollyZ has joined #openstack-infra | 13:56 | |
| *** rderose has joined #openstack-infra | 13:57 | |
| *** itisha has joined #openstack-infra | 13:57 | |
| *** shardy is now known as shardy_mtg | 13:58 | |
| openstackgerrit | Merged openstack-infra/release-tools: release_notes.py: search your feelings https://review.openstack.org/342053 | 13:58 |
| *** rbrndt has joined #openstack-infra | 13:59 | |
| *** mriedem has joined #openstack-infra | 13:59 | |
| *** amitgandhinz has quit IRC | 14:00 | |
| fungi | nice commit message | 14:00 |
| * fungi knows it to be true | 14:00 | |
| *** amitgandhinz has joined #openstack-infra | 14:00 | |
| *** eharney has joined #openstack-infra | 14:01 | |
| *** akshai has joined #openstack-infra | 14:01 | |
| anteaya | ha ha ha | 14:02 |
| *** edmondsw has joined #openstack-infra | 14:02 | |
| Zara | :D | 14:02 |
| *** [1]Thelo has joined #openstack-infra | 14:03 | |
| TheJulia | So does gerrit know it is Friday? | 14:03 |
| anteaya | seem so | 14:05 |
| anteaya | memory usage seems very high to me: http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=27&rra_id=all | 14:05 |
| TheJulia | wow, ouch | 14:05 |
| *** Thelo has quit IRC | 14:05 | |
| *** [1]Thelo is now known as Thelo | 14:05 | |
| TheJulia | which means it is going to spend most of it's time GC'ing | 14:05 |
| anteaya | seems so | 14:05 |
| TheJulia | s/it's/its/ | 14:05 |
| anteaya | oh I give you apostrophe opps as freebies | 14:06 |
| TheJulia | are the GC settings visible someplace, I'm curious how it's running | 14:06 |
| TheJulia | s/running/configured to run/ | 14:06 |
| anteaya | I don't know to be honest | 14:06 |
| anteaya | I know cacti is available, but gc settings aren't shown that I can see, I don't know if they can be or not in cacti | 14:07 |
| TheJulia | I'm thinking more of the jvm command line | 14:07 |
| *** hashar is now known as hasharAway | 14:07 | |
| anteaya | I do know gerrit admins have access to a dashboard called javamelody which is gerrit's internal monitoring tool | 14:07 |
| anteaya | javamelody isn't public | 14:08 |
| anteaya | yeah, I think javamelody is the tool that would display what you are after but only admins can see it | 14:08 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Use the Release Bot account on the signing node https://review.openstack.org/342864 | 14:09 |
| *** jpich has joined #openstack-infra | 14:09 | |
| *** andreas_s has quit IRC | 14:09 | |
| TheJulia | Oh well, at that amount of memory even parallel GC'ing would likely be a multi-second JVM pause | 14:09 |
| anteaya | what do you think fungi, can we consider a gerrit restart? | 14:09 |
| fungi | probably. i'm trying to keep up with the release team meeting at the moment though | 14:09 |
| anteaya | fungi: very good, thanks | 14:09 |
| fungi | any other infra-root around and wants to do an emergency gerrit restart to address its current performance slump? | 14:10 |
| jeblair | i just woke up what could go wrong | 14:10 |
| TheJulia | jeblair: everything? | 14:10 |
| *** jlanoux has quit IRC | 14:10 | |
| anteaya | jeblair: ha ha ha | 14:10 |
| TheJulia | jeblair: perhaps coffee first? | 14:10 |
| jeblair | TheJulia: i don't drink coffee, so that could make things worse! :) | 14:11 |
| TheJulia | For some reason I thought you did | 14:11 |
| jeblair | no, i think that's *everyone else in the world* (afaict) | 14:12 |
| anteaya | well I don't either | 14:12 |
| *** rderose_ has joined #openstack-infra | 14:12 | |
| *** markvoelker has quit IRC | 14:13 | |
| *** sdague has joined #openstack-infra | 14:13 | |
| jeblair | oh, yeah, the gc time is spiking | 14:13 |
| jeblair | i'd show you the graph, but i just got a proxy error | 14:14 |
| anteaya | :( | 14:14 |
| *** rkukura has joined #openstack-infra | 14:14 | |
| jeblair | it was up to 5 seconds for the past several hours at least | 14:14 |
| anteaya | oh my | 14:14 |
| anteaya | what is it usually? | 14:15 |
| *** rderose has quit IRC | 14:15 | |
| jeblair | sub-second | 14:15 |
| anteaya | ah, this would be larger then | 14:15 |
| jeblair | anteaya, TheJulia: http://imgur.com/cqZ3eId | 14:15 |
| anteaya | thanks | 14:15 |
| *** markvoelker has joined #openstack-infra | 14:16 | |
| *** markvoelker has quit IRC | 14:16 | |
| jeblair | status notice Gerrit is restarting to correct memory/performance issues. | 14:16 |
| jeblair | that look good? | 14:16 |
| anteaya | yes | 14:16 |
| anteaya | thanks for the image | 14:16 |
| jeblair | #status notice Gerrit is restarting to correct memory/performance issues. | 14:16 |
| openstackstatus | jeblair: sending notice | 14:16 |
| -openstackstatus- NOTICE: Gerrit is restarting to correct memory/performance issues. | 14:18 | |
| *** ccamacho has quit IRC | 14:18 | |
| *** ihrachys has joined #openstack-infra | 14:18 | |
| fungi | thanks, jeblair! | 14:19 |
| jeblair | should be back up now | 14:19 |
| anteaya | back up for me | 14:19 |
| openstackstatus | jeblair: finished sending notice | 14:19 |
| pabelanger | thanks, I had stepped away from my desktop | 14:19 |
| anteaya | thanks jeblair | 14:20 |
| TheJulia | thank you jeblair | 14:20 |
| jeblair | np | 14:22 |
| *** ociuhandu has joined #openstack-infra | 14:22 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool: Remove dead code from nodepoolcmd https://review.openstack.org/342450 | 14:23 |
| *** rderose_ has quit IRC | 14:24 | |
| AJaeger | project-config cores, here's a list of 11 reviews that have already a +2 and could use another +2A - if any of you have time, please review: http://paste.openstack.org/show/533027/ | 14:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Add dynamic reconfiguration https://review.openstack.org/340597 | 14:26 |
| *** cbader has joined #openstack-infra | 14:27 | |
| *** links has quit IRC | 14:27 | |
| *** jaosorior has joined #openstack-infra | 14:28 | |
| AJaeger | fungi, jeblair: Is https://review.openstack.org/#/c/342623/ a good test for zuul-cloner in post queue with constraints? The related openstack-manuals change has merged already but I can update if you have suggestions. | 14:28 |
| *** hasharAway has quit IRC | 14:29 | |
| fungi | AJaeger: thanks, that was on my to do list to look at once the review team meeting wrapped up | 14:29 |
| AJaeger | fungi: thanks! | 14:30 |
| jeblair | AJaeger: i don't know enough about constraints to say -- but if you have questions like 'what did zuul-cloner do' or 'what will zuul-cloner do' i can answer those. | 14:30 |
| AJaeger | anteaya: https://review.openstack.org/#/c/341058/3/jenkins/scripts/release-tools/release_from_yaml.sh limits it to release-test repo. | 14:30 |
| fungi | gertty needs a moment to sync the manuals change | 14:30 |
| clarkb | TheJulia: the init script and associated configure the jvm is all shipped by gerrit itself. You can poke at its choices by grabbing the war yourself and unpacking/installing | 14:31 |
| *** ccamacho has joined #openstack-infra | 14:31 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: force a fuller update of the repo being cloned for tagging https://review.openstack.org/342890 | 14:31 |
| AJaeger | jeblair: my question is: Will that job work as is with checking out correct branch/change and requirements? | 14:31 |
| AJaeger | jeblair: as a post job. | 14:31 |
| TheJulia | clarkb: Awesome, thank you | 14:31 |
| *** zz_dimtruck is now known as dimtruck | 14:32 | |
| *** mriedem has quit IRC | 14:32 | |
| AJaeger | jeblair: the constraints is just another repo to check out in this picture | 14:32 |
| fungi | AJaeger: looks sufficient to me (in conjunction with the manuals changes), though you could have done it all with just a job definition and not needed to call out to tox at all | 14:33 |
| *** mriedem has joined #openstack-infra | 14:33 | |
| *** matthewbodkin has quit IRC | 14:33 | |
| fungi | so you ended up creating a lot more changes you'll have to revert than you needed to, but i guess it exercises pip with constraints in the context of tox configuration | 14:33 |
| AJaeger | fungi: yeah, was thinking about that as well (that was my first iteration) but wanted to test with constraints completely | 14:34 |
| fungi | so slightly more thorough than doing it in a job config alone | 14:34 |
| AJaeger | fungi: yes. | 14:34 |
| *** jlanoux has joined #openstack-infra | 14:34 | |
| AJaeger | anteaya: thanks for carefully reviewing! | 14:34 |
| anteaya | AJaeger: one of your 11 patches is a duplicate of a patch already in the paste | 14:35 |
| *** daemontool has quit IRC | 14:35 | |
| anteaya | AJaeger: welcome, thanks for doing the heavy lifting with them beforehand | 14:35 |
| AJaeger | sorry about the duplicate | 14:35 |
| anteaya | no worries | 14:36 |
| anteaya | fungi: I'd like you to weigh in on a project-config patch that deals with adding packages: https://review.openstack.org/#/c/341856/1 | 14:36 |
| anteaya | I'm not sure on the current policy or if it is even settled yet | 14:37 |
| anteaya | fungi: when you can add it to your list, and thank you | 14:37 |
| *** markvoelker has joined #openstack-infra | 14:37 | |
| fungi | pabelanger: can we stack 341856 with the one extracting the ceph repo from the default list (in either direction, i don't really care which)? | 14:38 |
| *** maishsk_ has joined #openstack-infra | 14:39 | |
| fungi | pabelanger: nevermind, i see how it's broken up now, there's no need since it shouldn't conflict | 14:39 |
| fungi | i missed that was for a different platform | 14:39 |
| pabelanger | fungi: okay, I'm still looking for the ceph one atm | 14:40 |
| fungi | though we likely need to refactor a lot of the logic in configure_mirror.sh before we start making it too repetitive | 14:40 |
| openstackgerrit | Merged openstack-infra/project-config: telemetry: enable integration test on panko https://review.openstack.org/341711 | 14:40 |
| pabelanger | agreed | 14:40 |
| *** maishsk has quit IRC | 14:40 | |
| *** maishsk_ is now known as maishsk | 14:40 | |
| *** jamesmcarthur has joined #openstack-infra | 14:41 | |
| fungi | the heredocs interspersed in conditional stanzas is already pretty hard to follow | 14:41 |
| fungi | we could probably group them as string definitions and keep them out of the conditional logic | 14:42 |
| *** shardy_mtg is now known as shardy | 14:42 | |
| anteaya | AJaeger: I have completed your list, I think there were two -1's, the release patch and one py35 patch that added a py34 job into check | 14:43 |
| fungi | pabelanger: i'm okay with the mechanism and end result of 341856. do you have time to work on making that script a little easier to follow? i'm worried that continuing to add the slew of coming optional repos will get out of hand very quickly if we don't reorganize it to support them more cleanly | 14:43 |
| *** jamesmcarthur_ has joined #openstack-infra | 14:44 | |
| AJaeger | anteaya: the py35 was a great spot, I disagree with the release one - see my reply to it. might be that the commit message is not clear enough, but if you look at the shell script, it limits it. | 14:44 |
| AJaeger | anteaya, thanks for going through it! | 14:44 |
| anteaya | okay I'll look again at the release one, I had thought the commit message had said the test was to go on the release-test repo in this patch though | 14:45 |
| pabelanger | fungi: sure, I can refactor configure_mirror.sh | 14:45 |
| fungi | pabelanger: thanks! | 14:45 |
| clarkb | fungi: pabelanger maybe a add line to sources fioe function | 14:45 |
| clarkb | or function per heredoc | 14:46 |
| openstackgerrit | Merged openstack-infra/project-config: telemetry: add panko in telemetry integration job https://review.openstack.org/341712 | 14:46 |
| fungi | pabelanger: i appreciate you working on that. it's something we probably should have cleaned up before it ever got to this point | 14:47 |
| *** jaosorior_ has joined #openstack-infra | 14:47 | |
| anteaya | AJaeger: oh okay I get it now, yeah I mis-read the commit message, thanks for the clarification | 14:47 |
| pabelanger | ya, np | 14:47 |
| *** jaosorior has quit IRC | 14:47 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Disable all gate jobs for fuel-mirror master" https://review.openstack.org/342843 | 14:48 |
| AJaeger | anteaya: no worries. I had some trouble understanding it myself initially... | 14:48 |
| *** jaosorior has joined #openstack-infra | 14:48 | |
| jamesmcarthur_ | hi all - i'm working with the PWG to try to help them transition their User Story Tracker from demo to dev. We'd like to get it in gerrit and set up a dev server with infra team (currently hosted at Rackspace with o.o). | 14:49 |
| anteaya | AJaeger: thanks for your patience with me | 14:49 |
| jamesmcarthur_ | we've discussed the current dash with the storyboard team and PWG have agreed to help them where we can when they start working on epics | 14:49 |
| fungi | jamesmcarthur_: sounds great | 14:50 |
| jamesmcarthur_ | so the idea this would be a bridging service. I believe we've discussed in here before, but here's link to etherpad to refresh: https://etherpad.openstack.org/p/User_Story_Tracker | 14:50 |
| openstackgerrit | Merged openstack-infra/project-config: [Monasca-agent] Remove check-requirements check job for monasca-agent https://review.openstack.org/341091 | 14:51 |
| jamesmcarthur_ | fungi: one thing they do not have is a puppet person. I was wondering if you all knew anyone that might be able to guide them in that respect. | 14:51 |
| *** edtubill has joined #openstack-infra | 14:51 | |
| *** jamesmcarthur has quit IRC | 14:51 | |
| *** jamesmcarthur_ is now known as jamesmcarthur | 14:51 | |
| *** rderose has joined #openstack-infra | 14:52 | |
| *** jaosorior_ has quit IRC | 14:52 | |
| *** kien-ha has joined #openstack-infra | 14:52 | |
| fungi | jamesmcarthur: i am willing to, but we have others here who can too so i'm happy with whoever wants to tackle that part | 14:52 |
| openstackgerrit | Merged openstack-infra/project-config: Override install_command in neutron unit tests for periodic queue https://review.openstack.org/340612 | 14:53 |
| jamesmcarthur | fungi: cool, thanks | 14:54 |
| fungi | jamesmcarthur: feel free to list me as the primary point of contact initially and i can drive until/unless others step forward to help there | 14:54 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: add email notifications for release-post queue https://review.openstack.org/342906 | 14:54 |
| fungi | getting them better integrated with our developer community and infrastructure management is a high priority for me | 14:54 |
| jamesmcarthur | fungi: they're checking in with folks at Intel as well to see if there is anyone with bandwidth | 14:55 |
| jamesmcarthur | fungi: yes sir, agreed. I think that's high on everyone's list :) | 14:55 |
| fungi | jamesmcarthur: thanks! | 14:55 |
| openstackgerrit | Merged openstack-infra/project-config: Clone grenade for multinode linuxbridge job https://review.openstack.org/340962 | 14:55 |
| jamesmcarthur | fungi: the major players are out of town for the rest of the week, but I wanted to check in with you all directly. I'd expect some traction in the next week or so. | 14:56 |
| jamesmcarthur | fungi: the current development hasn't gone beyond the demo they did in Austin. | 14:56 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on oslo.config https://review.openstack.org/340657 | 14:56 |
| fungi | jamesmcarthur: there's a "rest of the week"? i thought today was just the buffer for weekend drinking | 14:56 |
| *** mtanino has joined #openstack-infra | 14:56 | |
| jamesmcarthur | fungi: oh yeah. it's friday. | 14:56 |
| *** amotoki has joined #openstack-infra | 14:56 | |
| *** weshay_mtg is now known as weshay_bbiab | 14:56 | |
| jamesmcarthur | fungi: should I already be drinking? | 14:56 |
| openstackgerrit | Merged openstack-infra/project-config: Make gate-grenade-dsvm-neutron-dvr-multinode voting https://review.openstack.org/336116 | 14:56 |
| fungi | jamesmcarthur: you mean you aren't | 14:57 |
| fungi | ? | 14:57 |
| Shrews | fungi: you say "weekend drinking" as if one should at some point stop drinking | 14:57 |
| fungi | Shrews: it's all in how you define "weekend" | 14:57 |
| jamesmcarthur | fungi: <feeling like i'm not trying hard enough> | 14:57 |
| jamesmcarthur | fungi: thanks for the time. i'll herd some cats and send folks your way. | 14:58 |
| *** akshai has quit IRC | 14:58 | |
| jamesmcarthur | fungi: i think we are also ready to pull the trigger on o.o move to gerrit | 14:58 |
| asselin__ | phschwartz, i read it briefly yesterday and you took out some of the main 'new' parts. So now it looks more like doing the same as the first spec, but add 3 new tools to puppet-openstackci. Is that accurate statement? | 14:58 |
| jamesmcarthur | fungi: clarkb has kindly offered to help wes with gerrit skillz | 14:58 |
| openstackgerrit | Merged openstack-infra/project-config: add job to apply tags when based on changes in openstack/releases https://review.openstack.org/341058 | 14:58 |
| AJaeger | pabelanger, anteaya : Could either of you review my zuul-cloner test change as well, please? https://review.openstack.org/#/c/342623/ Like to have it in soon so that we can move forward... | 14:59 |
| *** xarses has joined #openstack-infra | 14:59 | |
| fungi | jamesmcarthur: excellent. i think i heard hogepodge on our conference call offer to help too, but i may have misheard voices | 14:59 |
| jamesmcarthur | fungi: yeah, i think that's right | 14:59 |
| jamesmcarthur | fungi: jeblair: if you could let us know next steps, we're ready to take the nestea plunge | 15:00 |
| jeblair | you might even have imagined you heard my voice too :) | 15:00 |
| jamesmcarthur | jeblair: the ghost of gerrit future | 15:00 |
| jeblair | ~woooooh~ | 15:00 |
| jeblair | jamesmcarthur: basically we need to schedule a time when you'll stop pushing to github, we'll update the import, then you can start using gerrit | 15:01 |
| jeblair | should only take a few minutes | 15:01 |
| *** akshai has joined #openstack-infra | 15:02 | |
| fungi | yeah, basically one of our gerrit admins just needs to git clone from the current github repo, then git push --force into gerrit | 15:02 |
| jeblair | oh, and did we decide on whether we wanted to use the existing import, or a different repo name? | 15:02 |
| jamesmcarthur | jeblair: ok - let me coordinate with my team and the o.o design guys | 15:02 |
| openstackgerrit | Merged openstack-infra/project-config: force a fuller update of the repo being cloned for tagging https://review.openstack.org/342396 | 15:02 |
| openstackgerrit | Balazs Gibizer proposed openstack-infra/irc-meetings: Reschedule nova-notification meeting https://review.openstack.org/342916 | 15:03 |
| phschwartz | Asselin__: yes. It was expressed by most that they don't want server and user templating as part of openstackci | 15:03 |
| jeblair | jamesmcarthur, fungi: we have this ancient import: http://git.openstack.org/cgit/openstack-infra/openstackweb/ we can just update that if we're happy with the name, or if we wanted a new name, we can make a new repo and retire that one | 15:04 |
| asselin__ | phschwartz, so in this case, I suggest greatly simplifying the spec to just add those 2 nodes. There's still a lot of discussion now about 'interfaces', but seems most of that is not needed. | 15:04 |
| *** armax has joined #openstack-infra | 15:04 | |
| pabelanger | AJaeger: +3 | 15:04 |
| AJaeger | thanks, pabelanger | 15:04 |
| fungi | yep, right now it's openstack-infra/openstackweb in gerrit (imported from the "openstackweb/openstack-org" repo in github). i'm fine with that if everyone else is | 15:04 |
| AJaeger | armax: could you review https://review.openstack.org/337946 - to enable python35 on all of neutron -, please? | 15:04 |
| jamesmcarthur | jeblair: yeah, we're fine with whatever | 15:04 |
| jeblair | cool, that will be easiest then | 15:05 |
| asselin__ | phschwartz, I'm also not even sure we really need a spec on this....if we're not doing anything substantial other than continuing the current spec's work to other tools.... | 15:05 |
| * AJaeger will be back later | 15:05 | |
| jamesmcarthur | jeblair: ok - let me round everyone up on our side and i'll get back to you all here as soon as we're ready | 15:05 |
| jeblair | ++ | 15:05 |
| jamesmcarthur | jeblair: fungi: thanks for the info and assist on the PWG stuff | 15:06 |
| jamesmcarthur | jeblair: fungi: looking forward to next steps | 15:06 |
| jeblair | jamesmcarthur: thank you! | 15:06 |
| fungi | thanks jamesmcarthur! | 15:06 |
| *** Julien-zte has quit IRC | 15:07 | |
| *** SumitNaiksatam has quit IRC | 15:07 | |
| *** sputnik13 has joined #openstack-infra | 15:07 | |
| phschwartz | asselin__: I will make changes to that affect. I personally would love the abbility to control users, server settings, and iptables in openstackci, but as usual, we bow to the communities wishes. | 15:11 |
| AJaeger | thanks for the review, armax ! | 15:11 |
| crinkle | good morning infra | 15:12 |
| *** armax_ has joined #openstack-infra | 15:12 | |
| *** rajinir has joined #openstack-infra | 15:13 | |
| openstackgerrit | Merged openstack-infra/project-config: Test constraints with zuul-cloner in post queue https://review.openstack.org/342623 | 15:13 |
| *** maishsk has quit IRC | 15:13 | |
| *** armax has quit IRC | 15:14 | |
| *** armax_ is now known as armax | 15:14 | |
| *** kushal has quit IRC | 15:14 | |
| Zara | btw, if anyone is interested in task-statuses updating automatically in a storyboard instance from a gerrit instance, zaro worked hard on these: https://review.openstack.org/#/q/status:open+topic:its-storyboard . It gets especially exciting with automatic worklists! And now I'm off before I somehow infect you all with my cold over irc. :) | 15:14 |
| *** abregman has joined #openstack-infra | 15:14 | |
| Zara | please take a look, and thank you! :) | 15:15 |
| *** kushal has joined #openstack-infra | 15:15 | |
| *** maishsk has joined #openstack-infra | 15:16 | |
| anteaya | Zara: take care of your cold, have a good weekend, and thanks for all your hard work | 15:16 |
| anteaya | crinkle: good morning | 15:16 |
| asselin__ | phschwartz, honestly, that seems to really be what the spec you're writing is trying to solve...but you took all of that out, so I question the value/need of the spec now altogether. | 15:17 |
| phschwartz | asselin__: I don't mind scrapping it and adding these servers to the old spec, but I thought the old spec was marked as completed | 15:18 |
| phschwartz | crinkle: good morning | 15:18 |
| *** kushal has quit IRC | 15:19 | |
| *** nadya has joined #openstack-infra | 15:20 | |
| asselin__ | phschwartz, it's not 'complete' yet...one more tool (elk stack) left. Would be good to get opinions from others. nibalizer? | 15:20 |
| *** burgerk has joined #openstack-infra | 15:20 | |
| phschwartz | asselin__: not sure if he is on and looking, he is at a conference right now | 15:21 |
| openstackgerrit | Merged openstack-infra/project-config: Switch py35 to voting for neutron stadium projects https://review.openstack.org/337946 | 15:21 |
| phschwartz | asselin__: I will be working on the implementation of the stuff on the spec no matter what this week. So doesn't matter to me if we keep it separate or merge it into the old spec. | 15:21 |
| *** mdrabe has quit IRC | 15:22 | |
| phschwartz | asselin__: and if I am adding gerrit, I wonder if it would be worth it to anyone to adding a setup for jeepyb around project creation/management. | 15:22 |
| *** kushal has joined #openstack-infra | 15:23 | |
| asselin__ | phschwartz, i would expect the gerrit & jeepyb part to be a single class in puppet-openstackci | 15:23 |
| anteaya | zaro: what are you lacking in this patch that you still have it marked -1 workflow? https://review.openstack.org/#/c/340605/ | 15:23 |
| phschwartz | asselin__: I would too, But worth noting in the spec so it is not a surpirse to anyone | 15:23 |
| asselin__ | phschwartz, since that is how openstack runs gerrit and all downstream users. | 15:24 |
| rcarrillocruz | i'd like at least the jeepy thing is optional | 15:24 |
| rcarrillocruz | like a manage_projects flag or something | 15:24 |
| *** sputnik13 has quit IRC | 15:24 | |
| rcarrillocruz | so some people have an option to opt-out, in case they want to manage that in some other fashion | 15:24 |
| phschwartz | rcarrillocruz: that is what I was thinking as there are people that would prefer to do it by hand | 15:24 |
| asselin__ | no objections to optional | 15:24 |
| rcarrillocruz | ++ | 15:24 |
| clarkb | just dont provide a projects.yaml | 15:25 |
| phschwartz | clarkb: my thought would be to provide one in an example, but not used by default | 15:26 |
| hogepodge | fungi: you rang? | 15:26 |
| *** EricGonczer_ has joined #openstack-infra | 15:26 | |
| rcarrillocruz | yeah, ala hiera.yaml that is in contrib | 15:26 |
| rcarrillocruz | that makes sense to me phschwartz | 15:26 |
| fungi | hogepodge: oh, just saying i thought i'd overheard you on the staff call volunteering to help wes with gerrit/git-review workflow | 15:26 |
| asselin__ | ++ sample in contrib | 15:27 |
| openstackgerrit | Merged openstack-infra/project-config: add email notifications for release-post queue https://review.openstack.org/342906 | 15:28 |
| hogepodge | fungi: definitely, I'd be happy to help out with that | 15:28 |
| *** Goneri has quit IRC | 15:28 | |
| *** reed_ has joined #openstack-infra | 15:28 | |
| *** mdrabe has joined #openstack-infra | 15:29 | |
| hogepodge | jamesmcarthur: fly up to Potland for a visit! We can all meet in person. :-D | 15:29 |
| *** reed_ has quit IRC | 15:29 | |
| anteaya | crinkle: could I trouble you for your puppety-type review skills on the its-storyboard patches? https://review.openstack.org/#/q/topic:its-storyboard+status:open | 15:31 |
| pabelanger | fungi: now that signing01.ci.o.o is online, are we able to start using it? For example, just looking at zigo comments about gpg signing debian packages | 15:32 |
| fungi | pabelanger: yeah, believe it or not i'm in the middle of writing an initial tarball/wheel signing job | 15:33 |
| fungi | pabelanger: gimme another ~30 minutes and i'll have it up for review | 15:33 |
| jamesmcarthur | hogepodge: i've never been, actually. Have been wanting to for some time. | 15:33 |
| pabelanger | fungi: nice! | 15:33 |
| pabelanger | fungi: okay, ya, I'll see how that looks and then see what would be needed to move reprepro to signing01.ci.o.o | 15:33 |
| *** kushal has quit IRC | 15:34 | |
| *** EricGonc_ has joined #openstack-infra | 15:35 | |
| crinkle | anteaya: sure | 15:36 |
| anteaya | if any channel ops need a distraction, meeting-3 could do with a topic refresh | 15:36 |
| anteaya | crinkle: thank you :) | 15:36 |
| *** PollyZ has quit IRC | 15:36 | |
| anteaya | crinkle: I'm going with the patch that is -1 workflow is waiting for all the other patches to merge first | 15:37 |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder: Refactor: block-device handling (local loop) https://review.openstack.org/319591 | 15:37 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Add dynamic reconfiguration https://review.openstack.org/340597 | 15:37 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Add nodepool request framework https://review.openstack.org/304302 | 15:37 |
| *** ddieterly is now known as ddieterly[away] | 15:38 | |
| *** EricGonczer_ has quit IRC | 15:38 | |
| *** kushal has joined #openstack-infra | 15:38 | |
| *** kushal has joined #openstack-infra | 15:38 | |
| *** TheodoreBe has joined #openstack-infra | 15:40 | |
| *** kushal has quit IRC | 15:40 | |
| *** deadnull has quit IRC | 15:40 | |
| jeblair | pabelanger, fungi, clarkb: about nodepool -- i assume it has not been restarted? do we care about image uploads today, or can we abort them for a restart? | 15:40 |
| *** esikachev has quit IRC | 15:40 | |
| fungi | there's not anything i've personally been waiting on in an image update | 15:41 |
| *** maishsk has quit IRC | 15:41 | |
| jeblair | (yeah, june 22 -- has not been restarted) | 15:41 |
| clarkb | jeblair: it has not been restarted. I think we want the image builds to finish due to the fails yesterday on charm-aodh not being a repo yet. But once the builds are done we can drop the uploads then manually queue them up after if we want | 15:41 |
| *** maishsk has joined #openstack-infra | 15:41 | |
| jeblair | clarkb: ack. will stand down for now then. thanks. :) | 15:41 |
| *** PollyZ_ has joined #openstack-infra | 15:42 | |
| clarkb | oh and todays builds should get stable branches in the git cache too so thats another reason to let the builds complete | 15:42 |
| jeblair | ++ | 15:42 |
| *** sputnik13 has joined #openstack-infra | 15:43 | |
| *** ddieterly[away] is now known as ddieterly | 15:44 | |
| *** links has joined #openstack-infra | 15:45 | |
| *** coolsvap|afk has quit IRC | 15:50 | |
| *** gyee has joined #openstack-infra | 15:50 | |
| *** coolsvap|afk has joined #openstack-infra | 15:51 | |
| *** coolsvap|afk is now known as coolsvap | 15:51 | |
| *** sarob has joined #openstack-infra | 15:51 | |
| openstackgerrit | Merged openstack-infra/project-config: Always run linters for OpenStack-Ansible jobs https://review.openstack.org/342742 | 15:51 |
| *** ociuhandu has quit IRC | 15:52 | |
| *** PollyZ__ has joined #openstack-infra | 15:54 | |
| *** sarob has quit IRC | 15:55 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Debian Jessie to AFS mirrors https://review.openstack.org/342934 | 15:56 |
| *** asettle has quit IRC | 15:57 | |
| *** PollyZ_ has quit IRC | 15:57 | |
| *** weshay_bbiab is now known as weshay | 15:57 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config: Add networking-generic-switch-dsvm-temepst job. https://review.openstack.org/342936 | 15:58 |
| pabelanger | zigo: fungi: Added debian mirror to AFS ^ | 15:58 |
| *** cbader has quit IRC | 15:58 | |
| pabelanger | I haven't done the afs volumes yet | 15:58 |
| *** mescanef has quit IRC | 15:58 | |
| zigo | pabelanger: Meaning we'll have a Debian mirror available? Super nice! :) | 15:58 |
| pabelanger | zigo: yes, debian-jessie for the moment | 15:58 |
| armax | sc68cal clarkb: we spotted bug 1603268 recently in the grenade gate, the issue seems MTU related and it occurs after upgrade | 15:59 |
| openstack | bug 1603268 in neutron "Unstable grenade multinode job" [Critical,Confirmed] https://launchpad.net/bugs/1603268 | 15:59 |
| zigo | pabelanger: FYI, I fixed stuff, and it works now. | 15:59 |
| pabelanger | zigo: great, what is the plan moving forward now? | 15:59 |
| armax | I was scouring anywhere for clues and smoking guns but kevinbenton and I ran out of ideas | 15:59 |
| openstackgerrit | Eric Brown proposed openstack-infra/project-config: Enable py35 voting on vmware-nsx https://review.openstack.org/340708 | 15:59 |
| zigo | pabelanger: I need to put this a bit on hold, so I can finish packaging Newton b2 first. I hope to have this finished early next week. | 15:59 |
| armax | do you have by any change any tip? | 15:59 |
| * sc68cal looks | 16:00 | |
| pabelanger | zigo: okay, I am going to look at signed packaging today, with help from fungi | 16:00 |
| armax | from what I can tell this mostly happens on OVH nodes, but it happened on a RAX node at least once | 16:00 |
| zigo | pabelanger: Then the next thing is to make a new project which will contain a list of packages to backport from Sid to Jessie, and have the logic to automatically backport them. | 16:00 |
| clarkb | armax: we haven't changed anything in the infra side of things for multinode in a while | 16:00 |
| armax | I don’t believe it is infra related | 16:00 |
| zigo | pabelanger: Once that is done, we'll have everything... | 16:00 |
| clarkb | armax: so not immediate items come to mind | 16:00 |
| armax | clarkb: ok, thanks for confirming | 16:00 |
| armax | I have a potential culprit https://review.openstack.org/#/c/342538/ | 16:00 |
| clarkb | armax: one thing to double check is that metadata service is functioning properly there | 16:01 |
| zigo | pabelanger: The only thing I don't know, is how to add the built packages repo correctly in sources.list, so that it uses the closest mirror in the AFS. | 16:01 |
| *** Liuqing has quit IRC | 16:01 | |
| zigo | pabelanger: I can use the RAX one first, and then you may help to fix... | 16:01 |
| armax | clarkb: I remember that in the past metadata was the issue | 16:01 |
| clarkb | armax: we can't do live migrations with cofnig drive so all of the multinode jobs use metadata service. SSH key setup depends on that working properly | 16:01 |
| zigo | Not a big deal. | 16:01 |
| armax | clarkb: but it does not look to be this time | 16:01 |
| armax | clarkb: the job fails during the verify phase | 16:01 |
| zigo | pabelanger: Once you figure out the signing part, we should add that public key to the Jessie image. | 16:02 |
| armax | clarkb: where cinder tries to ssh into its vm and it goes in time out during the key exchange | 16:02 |
| zigo | (apt-key add pubkey.asc) | 16:02 |
| pabelanger | zigo: this is an example of how we distribute the source.list: https://review.openstack.org/#/c/341856/ | 16:02 |
| pabelanger | but we need to refactor it for a bit | 16:02 |
| sc68cal | armax: and it appears to be dropping packets outbound from the VM | 16:02 |
| armax | clarkb: ok, I recall that was still the case | 16:02 |
| sc68cal | ? | 16:02 |
| armax | clarkb: that is not going to change any time soon is it? | 16:03 |
| sc68cal | since it's expecting a reply from the instance, that never comes | 16:03 |
| zigo | pabelanger: Oh, so it's already done? :) | 16:03 |
| clarkb | armax: I think nova is trying to add support for config drive and live migrations this cycle. So possibly next cycle we can switch from metadata server on thse jobs. But yes looking at that log its trying to do diffie hellman init | 16:03 |
| armax | sc68cal: it looks that way, I have been scouring but nothing stands out yet | 16:03 |
| *** ifarkas has quit IRC | 16:03 | |
| *** adrian_otto has joined #openstack-infra | 16:03 | |
| pabelanger | zigo: it is done, but I need to refactor the code. | 16:04 |
| armax | clarkb: ok, thanks | 16:04 |
| jpmaxman | Hey fungi - just trying to follow https://review.openstack.org/#/c/341800/ | 16:04 |
| zigo | pabelanger: We will need to wait tomorror so that the image is rebuilt as well, no? | 16:04 |
| *** amotoki has quit IRC | 16:04 | |
| zigo | Also, it's not super nice if we have "-newton" hardcoded ... | 16:04 |
| clarkb | armax: and you double checked that neutron set MTUs properly? | 16:04 |
| *** elo has joined #openstack-infra | 16:04 | |
| armax | it looks like it has | 16:04 |
| *** javeriak has joined #openstack-infra | 16:04 | |
| jpmaxman | those variables couldn't be removed because they still exist in hiera? | 16:04 |
| sc68cal | hmmm. I see path_mtu is set to 1500 and network_device_mtu is set to 1450 | 16:04 |
| *** mikelk has quit IRC | 16:04 | |
| pabelanger | zigo: we'll have separate source.list files for each release, once we add more | 16:05 |
| armax | that’s how it is supposed to be though | 16:05 |
| *** sputnik13 has quit IRC | 16:05 | |
| clarkb | armax: it is? | 16:05 |
| armax | sc68cal: these values do not change prior or after the upgrade | 16:05 |
| clarkb | armax: I don't think thats correct | 16:05 |
| sc68cal | armax: right. so maybe your suspicion about that patch for ipv6 mtu is well founded | 16:06 |
| clarkb | well to clarify I don't know what path_mtu is supposed to reflect | 16:06 |
| fungi | jpmaxman: our entrypoint for puppeting servers is the system-config repo, so it's still passing in a value for wg_captchaquestions to the mediawiki module's main class. we test that applying that works, and puppet will asplode if you try to pass an undeclared class variable | 16:06 |
| *** PollyZ__ has quit IRC | 16:06 | |
| armax | clarkb, sc68cal: the computation from haleyb’s patch seems to preserve the old values | 16:06 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config: Restructure Sahara CLI test job https://review.openstack.org/336118 | 16:06 |
| armax | but kevinbenton did confirm that the mtu was set on network creation so even after an L3 agent reboot things should stay the same | 16:06 |
| *** akshai has quit IRC | 16:07 | |
| fungi | jpmaxman: following from https://review.openstack.org/342372 may make it slightly clearer to you | 16:07 |
| jpmaxman | ok right so it needs to be removed there first | 16:07 |
| jpmaxman | yeah was looking at that | 16:07 |
| *** sdake has joined #openstack-infra | 16:07 | |
| sc68cal | armax: agree - but the original MTU problem, we were facing the MTU issue during the initial resource creation | 16:07 |
| armax | sc68cal, clarkb but I must admit a fresh look at this might help spot the real problem | 16:07 |
| sc68cal | armax: this appears to be post-upgrade | 16:07 |
| armax | sc68cal: correct | 16:07 |
| fungi | jpmaxman: the direct issue is the fact that it's being passed to class { 'mediawiki': in modules/openstack_project/manifests/wiki.pp | 16:07 |
| *** asettle has joined #openstack-infra | 16:08 | |
| *** sandanar has quit IRC | 16:08 | |
| jpmaxman | makes sense, but what about the answers? $wg_captchaquestions_answer | 16:08 |
| sc68cal | so something must be broke between mitaka and HEAD that done goof'd it | 16:08 |
| armax | sc68cal: so it seems something is screwy when we get to Newton | 16:08 |
| clarkb | armax: sc68cal: looking more carefully at the log this is hitting the floating ip address. This means traffic will go over the infra created vxlan overlay. eth0 has mtu of 1500 so our overlay should have mtu of 1450. As long as the instance is also told to use mtu 1450 or less then this should work | 16:08 |
| armax | sc68cal: that part is clear :) | 16:08 |
| armax | sc68cal: why is the unknown bit | 16:08 |
| *** jlanoux has quit IRC | 16:08 | |
| *** kushal has joined #openstack-infra | 16:08 | |
| armax | clarkb: ack | 16:08 |
| sc68cal | armax: ack. I'm now sync'd | 16:08 |
| clarkb | armax: actually no sorry thats true in the nova net and neutron dvr cases | 16:09 |
| fungi | jpmaxman: $wg_captchaquestions_answer was removed similarly in https://review.openstack.org/284418 some months ago per the old todo comment we're removing from pabelanger | 16:09 |
| sc68cal | clarkb: although we do SSH into the floating IP during the initial resource creation phase with mitaka code too right? | 16:09 |
| clarkb | too many different options here | 16:09 |
| *** adrian_otto1 has joined #openstack-infra | 16:09 | |
| *** eharney has quit IRC | 16:09 | |
| *** edtubill has quit IRC | 16:09 | |
| armax | sc68cal: mitaka is good | 16:09 |
| jpmaxman | ahhh | 16:09 |
| jpmaxman | ok | 16:09 |
| armax | sc68cal: all tests pass | 16:09 |
| clarkb | in the non dvr neutron case the fip is terminated on the controller node (primary), NATed there and then traffic to the VM on the subnode/compute will go over the neutron created overlay | 16:09 |
| armax | clarkb: the failure mode shows up 1 in 3 | 16:09 |
| armax | and it always fails during the verify phase (post upgrade) when we attempt to ssh into a cinder vm using the FIP | 16:10 |
| *** e0ne has quit IRC | 16:10 | |
| *** TheodoreBe has quit IRC | 16:10 | |
| *** roxanagh_ has joined #openstack-infra | 16:10 | |
| armax | clarkb: dvr is no worse than non-dvr | 16:10 |
| jpmaxman | so once these go through we'd do another patch to remove wg_captchaquestions | 16:10 |
| *** pblaho has quit IRC | 16:10 | |
| jpmaxman | from the media wiki repo | 16:10 |
| armax | actually the dvr flavor of grenade multi never failed in the gate | 16:10 |
| clarkb | tempest has address 172.24.4.1 and will try to ssh to 172.24.5.51 which is terminate on the same host via NAT. After NATing to the internel net addr it will go over br-not-ex to the compute node | 16:10 |
| armax | but I think that’s just coincidence | 16:11 |
| clarkb | armax: the dvr flavor will use the infra overlay for this :) | 16:11 |
| armax | clarkb: and so the non-dvr, right? | 16:11 |
| fungi | jpmaxman: yep, which i already proposed as https://review.openstack.org/342374 (with a cross-repo dependency specified accordingly in its commit message) | 16:11 |
| clarkb | armax: no non-dvr uses the neutron overlay because everything is terminated on the controller | 16:11 |
| *** roxanagh_ has quit IRC | 16:11 | |
| *** woodster_ has joined #openstack-infra | 16:11 | |
| armax | clarkb: um that’s a good data point | 16:11 |
| jpmaxman | ok so where do we stand, there seems to be a desire to remove the new user restriction removal from this patch | 16:12 |
| *** eharney has joined #openstack-infra | 16:12 | |
| clarkb | armax: the reason we have an infra overlay in the dvr/nova-net case is each compute host terminates the fips for its VMs. Which means we need to ahve some way in the infra to route traffic to the fips terminated on the compute hosts | 16:12 |
| jpmaxman | should I make that adjustment amend the patch to leave the restriction in place and submit a new patch to remove it? | 16:12 |
| armax | clarkb: right | 16:13 |
| *** adrian_otto has quit IRC | 16:13 | |
| armax | clarkb: totally forgot | 16:13 |
| clarkb | armax: with neutron non dvr this is not needed because neutron backhauls everything over its br-int to the controller host and the controllers br-ex is where we terminate the fips | 16:13 |
| clarkb | armax: me too, had to refresh my memory :) | 16:13 |
| *** bhavik has joined #openstack-infra | 16:13 | |
| armax | I am with you, now of course dvr the FIP is on the compute node | 16:13 |
| * armax duh! | 16:13 | |
| fungi | jpmaxman: yeah, that'd be easiest i think. and that leaves us with a clear commit we can revert/unrevert as needed to disable/reenable account creation if needed | 16:13 |
| armax | so the fact that is happens only on non-dvr helps narrow down things quite a bit | 16:14 |
| anteaya | fungi: jpmaxman thank you | 16:14 |
| *** javeriak_ has joined #openstack-infra | 16:14 | |
| *** Teresita-Warrior has joined #openstack-infra | 16:14 | |
| clarkb | armax: ya I think that points at the br-int overlay that neutron sets up as a likely source of the problem | 16:14 |
| *** hemanthm|afk is now known as hemanthm | 16:14 | |
| d0ugal | I can't be added as a reviewer by other users, I think this is because I have two gerrit accounts somehow. Is there a way to fix this? http://i.imgur.com/FJmKxY3.png | 16:15 |
| d0ugal | As far as I can remember I have only ever signed up once... | 16:15 |
| *** shardy has quit IRC | 16:15 | |
| *** javeriak has quit IRC | 16:16 | |
| *** _ari_ has quit IRC | 16:16 | |
| *** tesseract- has quit IRC | 16:16 | |
| armax | clarkb: ok, thanks for the feedback, let me go and look with a fresh pair of eyes | 16:16 |
| anteaya | d0ugal: did you change your email address in launchpad? | 16:17 |
| anteaya | d0ugal: launchpad thinks every email address is unique when signing in with ubuntuone (the single sign on provider) | 16:17 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: gitlab trigger: Support new "trigger-open-merge-request-push" options https://review.openstack.org/312121 | 16:17 |
| clarkb | armax: http://logs.openstack.org/46/321146/2/gate/gate-grenade-dsvm-neutron-multinode/988f952/logs/worlddump-2016-07-14-175426-nova_resources_created.txt.gz look at br-ex in the ip link listing | 16:17 |
| clarkb | armax: its mtu is 1500, but should be 1450 I think | 16:17 |
| d0ugal | anteaya: oh, I probably did | 16:18 |
| *** rcernin|lunch has quit IRC | 16:18 | |
| d0ugal | anteaya: as I had a weird email until I was able to get the dougal@ alias in the imgur | 16:18 |
| anteaya | d0ugal: okay when you are logged into gerrit what gerrit id do you have for your account? | 16:18 |
| armax | clarkb: let me double check that | 16:18 |
| clarkb | armax: depends if NAT happens before or after entering that interface. I am not sure how linux orders that but l2 -> l3 (br-ex -> NAT) seems likely? | 16:18 |
| d0ugal | anteaya: Account ID 9712 | 16:19 |
| d0ugal | anteaya: and username is "dougal" | 16:19 |
| clarkb | er if NAT happens first it should go straight onto br-int and never touch br-ex | 16:19 |
| anteaya | is there an infra root willing to spend time fixing the gerrit db to help d0ugal become one again? | 16:19 |
| *** pcaruana has quit IRC | 16:20 | |
| d0ugal | FWIW, there is no rush for this. It has been like this for some time. | 16:20 |
| d0ugal | So happy to ask again later if folks are busy (which seems likely) | 16:20 |
| anteaya | d0ugal: well one account ends up getting disabled, as we can't merge them together | 16:20 |
| clarkb | armax: I also notice br-tun is that way too. I forget what its relationship is to br-int | 16:20 |
| anteaya | so the longer you have two the more likely it is you are losing work you can't recover | 16:20 |
| dougwig | mordred: if any release didn't need a separate name, it's 'Q'. that release names itself! | 16:21 |
| anteaya | d0ugal: folks are always busy, stick around | 16:21 |
| *** vjamezcu has joined #openstack-infra | 16:21 | |
| anteaya | dougwig: ho ho ho | 16:21 |
| armax | clarkb: br-ex is set to 1500 even stable/mitaka | 16:21 |
| armax | http://logs.openstack.org/71/327671/1/check/gate-grenade-dsvm-neutron-multinode/6193a3c/logs/worlddump-2016-06-09-142947-cinder_resources_created.txt.gz | 16:21 |
| armax | clarkb: this is what we run for stable/mitaka changes… | 16:21 |
| armax | I am confused | 16:21 |
| sc68cal | OVS bridges don't honor MTUs anyway IIRC | 16:21 |
| d0ugal | anteaya: okay, thanks. I'll be here :) | 16:21 |
| clarkb | armax: 172.24.5.0/24 dev br-ex proto kernel scope link src 172.24.5.1 implies to be it should hit br-ex with an mtu of 1500, then get NAT'd, go into br-int and fragment but since there is no l3 transition on br-ex it has no way of negotiating down? | 16:22 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Add a job to sign tarballs and wheels https://review.openstack.org/342946 | 16:23 |
| fungi | pabelanger: ^ | 16:23 |
| clarkb | and even if there was default neutron security group rules are to drop icmp anyways iirc | 16:23 |
| anteaya | d0ugal: thank you | 16:23 |
| clarkb | sc68cal: even if they don't eventually we hit the VMs interface or the tap or something along the way that should | 16:24 |
| sc68cal | clarkb: true | 16:24 |
| *** sdake has quit IRC | 16:25 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Add a job to sign tarballs and wheels https://review.openstack.org/342946 | 16:25 |
| fungi | (minor clarification in the commit message) | 16:25 |
| *** maishsk has quit IRC | 16:26 | |
| *** karthikp_ has joined #openstack-infra | 16:26 | |
| *** tphummel has joined #openstack-infra | 16:26 | |
| clarkb | armax: I do know that in the nova-net/dvr case br-ex is set to have an mtu of 1450 by the devstack-gate bridge setup | 16:26 |
| armax | clarkb: so you’re seeing it overridden? | 16:27 |
| *** edtubill has joined #openstack-infra | 16:27 | |
| *** shashank_hegde has joined #openstack-infra | 16:28 | |
| pabelanger | clarkb: re: heredoc not sure I understand your last comment. "Add line to sources file function?" | 16:28 |
| pabelanger | fungi: looking | 16:28 |
| sc68cal | armax: do we know what the spike in failures was for grenade-neutron-multinode from 7/9 to 7/10 was? | 16:29 |
| sc68cal | very strange. | 16:29 |
| clarkb | armax: we don't touch br-ex for neutron unless dvr is enabled | 16:29 |
| armax | clarkb: ok | 16:29 |
| armax | sc68cal: there was a transient last week | 16:30 |
| armax | sc68cal: things went quiet and then we spiked at 30% | 16:30 |
| clarkb | armax: https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/devstack-vm-gate.sh#n129 | 16:30 |
| anteaya | crinkle: thank you for your reviews | 16:30 |
| *** Apoorva has joined #openstack-infra | 16:30 | |
| clarkb | pabelanger: fungi was saying the control flow is hard to follow due to heredoc indentation and all that. I was thinking if we replaced large heredocs with a function that could add line(s) to the files then it might be easier to follow the control flow | 16:31 |
| clarkb | pabelanger: alternatively you could have a function per heredoc so that the flow confusion is confined to a function with no branches | 16:31 |
| crinkle | zaro: i wonder if maybe we could start putting an example for every new parameter into the beaker tests in http://git.openstack.org/cgit/openstack-infra/puppet-gerrit/tree/spec/acceptance/fixtures/default.pp ? that way the templates get tested and neither i nor you have to manually test them to make sure they compile | 16:31 |
| crinkle | anteaya: you're welcome | 16:31 |
| anteaya | :) | 16:31 |
| *** jheroux has joined #openstack-infra | 16:32 | |
| sc68cal | armax: from 6/17 all the way up till 7/4 this job behaved | 16:32 |
| sc68cal | armax: then it started misbehaving | 16:32 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: gitlab trigger: Support new "trigger-open-merge-request-push" options https://review.openstack.org/312121 | 16:32 |
| armax | sc68cal: if I look that the commit hiistory nothing strikes as the clear smoking gun | 16:33 |
| pabelanger | fungi: left question | 16:33 |
| sc68cal | armax: yeah I looked back through till like 7/7 and nothing obvious - which makes me worry something had an unintended effect | 16:33 |
| *** hemnafk has quit IRC | 16:33 | |
| armax | sc68cal: perhaps this one? | 16:34 |
| armax | https://github.com/openstack/neutron/commit/f9d0dd39e6b3b14244e0a185c4a1f7f35f48f3e7 | 16:34 |
| clarkb | sc68cal: armax you could fairly quickly check if br-ex changed mtus in that time period | 16:34 |
| clarkb | by looking at job logs I think | 16:34 |
| armax | clarkb: when I looked at a grenade job run for a stable mitaka change I didn’t spot differences | 16:35 |
| odyssey4me | hmm, I've seen a bit of odd behaviour lately with some of our long running jobs - for example https://review.openstack.org/340525 dropped into the gate queue nearly 7 hours ago, and the timeout for the job is 90 mins... it appears to have restarted several time | 16:35 |
| *** ociuhandu has joined #openstack-infra | 16:35 | |
| odyssey4me | any thoughts regarding why this would happen | 16:35 |
| *** Ravikiran_K has quit IRC | 16:35 | |
| sc68cal | armax: I saw that one, but I don't think so | 16:36 |
| anteaya | odyssey4me: if something happened to the host node, sometimes the job can be held and restarted on a new node | 16:36 |
| odyssey4me | typically this would happen if a job had another one ahead of it that failed, but this one has restarted within the last hour after being nearly complete, and nothing else has been ahead of it during that time | 16:36 |
| anteaya | odyssey4me: which job is restarting? | 16:36 |
| armax | sc68cal: agreed | 16:36 |
| odyssey4me | oh, silly me - forgot to paste the review https://review.openstack.org/340525 | 16:36 |
| jpmaxman | fungi anteaya no problem will do it momentarily | 16:36 |
| sc68cal | armax: I think you were on to something with 51a6978 (the ipv6 mtu patch) | 16:37 |
| anteaya | odyssey4me: no you pasted the review, I just asked which job is restarting? | 16:37 |
| anteaya | jpmaxman: thank you | 16:37 |
| sc68cal | that merged on 7/7, so still in that period of noise of that transient failure | 16:37 |
| armax | sc68cal: true | 16:37 |
| armax | but if you look at the patch, I can’t seem to fault it | 16:37 |
| odyssey4me | anteaya uh, I have no way of telling the job number any more - how do I find that? all I have is the link to the telnet session: https://review.openstack.org/340525 | 16:37 |
| *** asettle has quit IRC | 16:37 | |
| odyssey4me | urgh | 16:37 |
| odyssey4me | telnet://158.69.67.116:8088 | 16:38 |
| sc68cal | armax: right. But I wonder if OVH vs. RAX triggers something | 16:38 |
| *** akshai has joined #openstack-infra | 16:38 | |
| fungi | clarkb: pabelanger: i don't even know that it needs a function for that. something as simple as converting the heredocs to consistently-named shell variables grouped at the top of the script, and echoing those out to the files instead would be plenty readable for me | 16:38 |
| armax | sc68cal: the had a failure on RAX too | 16:38 |
| sc68cal | armax: I forget which has native IPv6 and which didn't. I think it was HP cloud that didn't and RAX that did | 16:38 |
| odyssey4me | anteaya how do I find the job number? | 16:38 |
| anteaya | odyssey4me: | 16:38 |
| anteaya | gate-openstack-ansible-dsvm-commit: | 16:38 |
| armax | sc68cal: though it was 1 in 13 gate failures | 16:38 |
| anteaya | odyssey4me: was just looking for the name of it | 16:38 |
| *** Kaiyan has joined #openstack-infra | 16:38 | |
| odyssey4me | anteaya oh the job name? :) yeah, sorry - I should have thought of that | 16:39 |
| clarkb | sc68cal: armax rax and vexxhost have ipv6 and thats it | 16:39 |
| clarkb | sc68cal: armax but this is all over ipv4 so not sure why that would matter | 16:39 |
| sc68cal | dunno, just thinking out loud | 16:39 |
| anteaya | odyssey4me: this says these jobs use an in repo script: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n8935 did you change your in repo script lately? | 16:40 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Add a job to sign tarballs and wheels https://review.openstack.org/342946 | 16:40 |
| fungi | pabelanger: thanks for spotting, fixed ^ | 16:40 |
| *** shashank_hegde has quit IRC | 16:40 | |
| *** asettle has joined #openstack-infra | 16:41 | |
| anteaya | fungi: can I impose on you to fix d0ugal's duplicated gerrit db account when you have a moment? | 16:41 |
| anteaya | fungi: or teach pabelanger how to do it when you both have moments? | 16:41 |
| armax | sc68cal: I think that code path touched in that patch only runs during network creation | 16:41 |
| odyssey4me | anteaya nope, although I'm not certain how that relates to the way the job executes | 16:41 |
| *** david-lyle has joined #openstack-infra | 16:42 | |
| odyssey4me | anteaya the job itself appears to restart without failing | 16:42 |
| anteaya | neither am I, I'm trying to figure out the landscape | 16:42 |
| armax | sc68cal: I am increasingly convinced it’s not the culprit | 16:42 |
| *** asettle has quit IRC | 16:42 | |
| anteaya | odyssey4me: right something is telling it to run again | 16:42 |
| jeblair | odyssey4me: pabelanger may have been looking into this yesterday | 16:42 |
| fungi | anteaya: pabelanger: d0ugal: taking a look in the db now to see what the state of accounts is there | 16:42 |
| sc68cal | yeah let me see if anything in d-g or devstack changed during that time period | 16:42 |
| * d0ugal is very grateful | 16:42 | |
| odyssey4me | anteaya jeblair pabelanger don't get me wrong - I love a good retry, but it's different behaviour and I'm wondering what's up | 16:42 |
| jpmaxman | fungi anteaya one thing to keep in mind with all of this though is there's another effort to upgrade all things wiki.o.o - just saying I appreciate the attention to detail but it all could be somewhat ephemeral | 16:43 |
| sc68cal | nope | 16:43 |
| anteaya | jpmaxman: I wish | 16:44 |
| fungi | jpmaxman: yeah, i'm leading the upgrade effort at this point, so definitely keeping that in mind | 16:44 |
| jeblair | odyssey4me: yeah, i know pabelanger and i got as far as agreeing that zuul needs to set a limit on the number of retries, so at least we will bottom out eventually and you'd get logs | 16:44 |
| pabelanger | odyssey4me: jeblair: anteaya: Yes, the job is running longer then expected, zuul-launcher aborts the job and automatically requeues the patch again for testing. As a result, there is an infinite loop until the job passes or fails in the allocated time. | 16:44 |
| *** SumitNaiksatam has joined #openstack-infra | 16:44 | |
| jpmaxman | :D | 16:44 |
| jeblair | pabelanger: oh, it's hitting a timeout but retrying? | 16:44 |
| anteaya | pabelanger: what is the allocated time? | 16:44 |
| pabelanger | zuul-launcher watchdog does that aborting | 16:44 |
| *** lucasagomes is now known as lucas-afk | 16:45 | |
| pabelanger | jeblair: yes, watchdog | 16:45 |
| pabelanger | anteaya: sorry, job timeout | 16:45 |
| odyssey4me | pabelanger jeblair ah, that makes sense | 16:45 |
| jeblair | pabelanger: oh wow, so the ansible timeout is not firing, watchdog is, but we're mistaking a watchdog timeout for something that should retry? | 16:45 |
| anteaya | pabelanger: oh, how many times does it loop before calling it a day? | 16:45 |
| odyssey4me | that saves on rechecks, but does hold up the queues somewhat | 16:45 |
| jeblair | anteaya: infinite | 16:45 |
| anteaya | fun | 16:45 |
| pabelanger | jeblair: right | 16:45 |
| anteaya | odyssey4me: thanks for speaking up | 16:45 |
| odyssey4me | anteaya :) | 16:45 |
| pabelanger | I haven't debugged why ansible is not timing out yet | 16:45 |
| * anteaya takes a seat at milliway's | 16:46 | |
| fungi | pabelanger: on review.openstack.org you can `sudo -H mysql reviewdb` to get a mysql> prompt for gerrit's backend db. from there `select * from accounts where account_id=9712;` lets me see the details on that account, and then i can do a similar select on preferred_email= what it does there to see the other duplicate account (9553) | 16:46 |
| jeblair | pabelanger: so we have 3 issues? 1) ansible is not timing out [low priority] 2) watchdog timeout is not reported as a failure [high priority] 3) cap on number of retries [med priority] | 16:47 |
| fungi | pabelanger: after that, `select * from account_external_ids where account_id in (9553, 9712);` lets me see the openids and ssh usernames for those | 16:48 |
| fungi | d0ugal: it looks like you have username:d0ugal for 9553 and username:dougal for 9712. i assume you're only using the latter for git push and api access? | 16:49 |
| *** dtantsur is now known as dtantsur|afk | 16:49 | |
| *** asettle has joined #openstack-infra | 16:49 | |
| *** IlyaG has joined #openstack-infra | 16:49 | |
| fungi | d0ugal: also it looks like 9553 was created by logging into launchpad sso with your dmatthew@ address, then you logged in with your dougal@ some time later which caused it to create account 9712 | 16:50 |
| openstackgerrit | Jp Maxwell proposed openstack-infra/puppet-mediawiki: Setup ReCaptcha in Wiki to Prevent Spam https://review.openstack.org/341800 | 16:50 |
| d0ugal | fungi: Right, that makes sense - I later got the dougal@ alias | 16:52 |
| d0ugal | fungi: I am only using 9712, I don't think I can even log into the other now? | 16:52 |
| fungi | jpmaxman: looks like that latest patchset reverted my readding the $wg_captchaquestions class variable. you probably didn't pull the new version of that change from gerrit before you amended it | 16:52 |
| jpmaxman | was just noticing that | 16:52 |
| jpmaxman | :( | 16:52 |
| pabelanger | jeblair: yes to all three | 16:52 |
| fungi | jpmaxman: if you're using git-review, easiest way to do that would have been `git review -d 341800` before you made your latest edit | 16:53 |
| fungi | jpmaxman: though at this point it's probably simpler to just readd the missing comment and variable | 16:53 |
| fungi | and commit -a --amend again | 16:54 |
| jpmaxman | aha! ok - good to know - yup I'll add it back in thanks for the tips | 16:54 |
| pabelanger | jeblair: for number 2 we abort the job using -9, which is falling into our requeue job logic | 16:54 |
| *** asettle has quit IRC | 16:54 | |
| pabelanger | fungi: sure, let me do that now | 16:54 |
| fungi | jpmaxman: yeah, sorry about that. it's one thing we get used to pretty quickly when collaborating on changes... the possibility that someone else has amended your change since the last time you pushed it | 16:54 |
| AJaeger | wow, we have jobs in the release queue for 48 hours! They're queued on wheel-release-trustyx64 etc. | 16:54 |
| AJaeger | could somebody check this, please ? ^ | 16:55 |
| pabelanger | AJaeger: yup, that is correct | 16:55 |
| pabelanger | we've just overloaded our wheel-build servers | 16:55 |
| fungi | jpmaxman: in many cases we will just correct a minor issue by pushing a new patchset rather than leaving a -1 and waiting for the author, so that we can iterate more quickly on getting things in shape to merge | 16:55 |
| AJaeger | pabelanger: ;( | 16:55 |
| *** roxanagh_ has joined #openstack-infra | 16:56 | |
| *** jpich has quit IRC | 16:56 | |
| pabelanger | AJaeger: the proper fix won't be until zuulv3, but the short of it, we don't need to rebuild wheels for each release | 16:56 |
| *** ddieterly is now known as ddieterly[away] | 16:56 | |
| clarkb | pabelanger: why don't we remove those jobs and make it a daily periodic job? | 16:57 |
| fungi | AJaeger: pabelanger: yeah, it might make more sense to just make it periodic for now | 16:57 |
| clarkb | as an interim solution | 16:57 |
| pabelanger | well, periodic already exists | 16:57 |
| pabelanger | so, just removing the other part | 16:57 |
| fungi | oh right, this is the "and also on every release" | 16:57 |
| pabelanger | I'm okay with that | 16:57 |
| clarkb | None of the wheels we produce actaully have long compiles so installing from sdist should be fine too | 16:57 |
| pabelanger | fungi: yes | 16:58 |
| clarkb | its lxml and numpy that we care about in the wheel mirror | 16:58 |
| *** sputnik13 has joined #openstack-infra | 16:58 | |
| *** hurgleburgler has joined #openstack-infra | 16:58 | |
| fungi | the main thing it's there for is not actually generating wheels of our packages (since those are already available in the pypi mirror and so don't need to be recompiled) | 16:58 |
| fungi | it's in case, say, a new release of something drags in a newer lxml because of updated requirements or something | 16:59 |
| *** ociuhandu has quit IRC | 16:59 | |
| clarkb | in fact if the only two things we had in the wheel mirror were lxml and numpy we would probably not notice a large change in job runtimes :) | 16:59 |
| fungi | but regardless, that's not going to have a significant negative impact if it's delayed by up to 24 hours | 16:59 |
| *** trown is now known as trown|lunch | 16:59 | |
| pabelanger | fungi: any issues using pastebin on the select query? | 17:00 |
| jpmaxman | fungi makes sense - and yeah sounds like good practice. | 17:00 |
| armax | clarkb: we’ll look into the br-ex issue and why it’s 1500 rather than 1450 | 17:00 |
| armax | clarkb: we’ll get it to honor what d-g gate | 17:00 |
| armax | s/gate/does | 17:00 |
| pabelanger | is it possible to run a periodic job more then 1 a day? | 17:00 |
| *** berendt has joined #openstack-infra | 17:00 | |
| clarkb | armax: ok, let me know if I can do anything else to help | 17:01 |
| pabelanger | I've never looked into that before | 17:01 |
| clarkb | armax: also check if br-tun needs tweaking too | 17:01 |
| armax | clarkb: thanks, much appreciated it | 17:01 |
| AJaeger | pabelanger: that would need a different queue AFAIK | 17:01 |
| fungi | pabelanger: only that d0ugal might be wary of having his additional e-mail addresses mentioned in public (which is why i didn't paste them in irc). though any used to push commits in git are obviously already public | 17:01 |
| d0ugal | fungi: I don't mind it being publin | 17:01 |
| d0ugal | public | 17:01 |
| pabelanger | fungi: http://paste.openstack.org/show/533074/ | 17:02 |
| fungi | pabelanger: http://paste.openstack.org/show/533075 | 17:02 |
| pabelanger | AJaeger: that would make sense | 17:02 |
| *** javeriak_ has quit IRC | 17:02 | |
| anteaya | d0ugal: thanks, it helps to have more root folks know how to fix the db in these cases, I appreciate your support here | 17:03 |
| d0ugal | fungi: btw, that paste does contain API password. | 17:03 |
| d0ugal | I think, or maybe that is another password but it looks familiar | 17:03 |
| pabelanger | fungi: odd, both were registered at the same datetime | 17:03 |
| fungi | d0ugal: d'oh, yes sorry about that | 17:03 |
| *** cody-somerville has joined #openstack-infra | 17:03 | |
| AJaeger | pabelanger: check http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n174 for current definition of the queue. Duplicate that, rename, ... | 17:03 |
| d0ugal | fungi: np, I have used it but not for some time. I'll reset it | 17:03 |
| anteaya | time for a new password | 17:04 |
| fungi | d0ugal: https://review.openstack.org/#/settings/http-password and click the generate button | 17:04 |
| openstackgerrit | Jp Maxwell proposed openstack-infra/puppet-mediawiki: Setup ReCaptcha in Wiki to Prevent Spam https://review.openstack.org/341800 | 17:04 |
| d0ugal | fungi: New password generated, so don't be surprised if something changes in your output :) | 17:04 |
| d0ugal | fungi: Yup, done, thanks | 17:04 |
| anteaya | d0ugal: thank you | 17:04 |
| fungi | d0ugal: totally wasn't paying attention. that's a huge oops on my part :/ | 17:04 |
| pabelanger | AJaeger: perfect, thanks | 17:04 |
| *** adrian_otto1 has quit IRC | 17:04 | |
| fungi | pabelanger: d0ugal: so anyway, easiest thing to do at this point is `ssh review.openstack.org -p 29418 gerrit set-account --inactive 9553` | 17:05 |
| jpmaxman | ok - hopefully that should be in good shape now fungi | 17:05 |
| jpmaxman | I'm going to grab a bite to eat back in a few | 17:05 |
| AJaeger | pabelanger: if you do add a new pipeline, please discsus it here - there have been some concerns about it in the past. So, let's fungi and jeblair ack it, please | 17:05 |
| pabelanger | fungi: clarkb: So, based on the AJaeger comments, we could setup a new wheel specific pipeline and run it every 6 or 12 hours | 17:05 |
| d0ugal | I need to run out for a bit, I'll try and get back soon but if I'm not here in time - thanks very much for the help fungi, anteaya, pabelanger | 17:05 |
| anteaya | d0ugal: thanks for your patience and understanding | 17:06 |
| pabelanger | fungi: should I do that or are you? | 17:06 |
| fungi | d0ugal: yeah, it should work here fine once we deactivate the old account | 17:06 |
| fungi | pabelanger: go for it | 17:06 |
| pabelanger | command run | 17:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Remove scheduler.getProject https://review.openstack.org/342965 | 17:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Prevent jobs from overriding jobs in other repos https://review.openstack.org/342966 | 17:07 |
| *** david-lyle has quit IRC | 17:08 | |
| AJaeger | fungi, I just had two jobs merged in openstack-manuals. The zuul-cloner post job log files are http://logs.openstack.org/ee/ee213608a68f6d8ce8f6299a45c1355eb599f548/post/zuul-cloner-test/5ec5f40/console.html and http://logs.openstack.org/28/28954692be5c9e963f41d7dbc5962fb5c85a95d3/post/zuul-cloner-test/47b1e82/console.html . One is stable/mitaka, the other master. Could you review those later, | 17:08 |
| AJaeger | please? | 17:08 |
| anteaya | jpmaxman: looks good once you fix the errant whitespace | 17:08 |
| fungi | pabelanger: you can try adding that e-mail address as a reviewer on a change and see if it still breaks | 17:08 |
| pabelanger | fungi: sure | 17:08 |
| fungi | AJaeger: sure, taking a look now | 17:08 |
| jeblair | pabelanger: i will start hacking on #2 now -- the watchdog error | 17:09 |
| *** Na3iL has quit IRC | 17:09 | |
| pabelanger | fungi: no errors adding email address to 342934 | 17:09 |
| fungi | AJaeger: that looks like it worked! | 17:09 |
| AJaeger | fungi, wait, both might be mitaka - let me double check... | 17:10 |
| fungi | pabelanger: d0ugal: should be all set then | 17:10 |
| AJaeger | fungi: yes, both mitaka... | 17:10 |
| armax | clarkb: I think I might be onto something | 17:10 |
| *** thiagop is now known as thiagop-afk | 17:10 | |
| pabelanger | excellent | 17:10 |
| * AJaeger searches for a master change to merge and test ;) | 17:10 | |
| armax | clarkb: I see no sudo ip link set mtu 1450 dev br-ex for gate-grenade-dsvm-neutron-multinode | 17:10 |
| jpmaxman | anteaya was just trying to copy fungi ;) in mine last line is 17 and starts again on 19 | 17:10 |
| fungi | AJaeger: INFO:zuul.Cloner:Prepared openstack/requirements repo with branch stable/mitaka at commit e9149002eab0e1fa0991f71d8076c1a46935aba0 | 17:11 |
| pabelanger | jeblair: lets land 341819 before nodepool restarts too please. Updates our logging.conf file with latest images | 17:11 |
| fungi | AJaeger: lgtm | 17:11 |
| armax | clarkb: but only on gate-tempest-dsvm-neutron-dvr-multinode-full and the gate-grenade-dsvm-neutron-dvr-multinode | 17:11 |
| jeblair | pabelanger: ack | 17:11 |
| clarkb | armax: yes becuase on the dvr jobs devsatck-gate manages br-ex directly | 17:11 |
| AJaeger | fungi: yes, indeed. I was surprised that both at stable/mitaka - until I realized that both were mitaka ;) | 17:11 |
| jpmaxman | will remove the empty line | 17:11 |
| clarkb | armax: but otherwise its neutrons thing I think | 17:11 |
| clarkb | armax: since br-ex is where we put the vxlan overlay we precreate it on the dvr jobs | 17:12 |
| armax | clarkb: but then, Neutron itself doesn’t bring the mtu to 1450 | 17:12 |
| AJaeger | fungi: both changes have a POST_FAILURE for the publish job, any idea what failed on those? | 17:12 |
| openstackgerrit | Jp Maxwell proposed openstack-infra/puppet-mediawiki: Setup ReCaptcha in Wiki to Prevent Spam https://review.openstack.org/341800 | 17:12 |
| clarkb | armax: devstack may create it? | 17:13 |
| anteaya | jpmaxman: sure and you have a whitespace error | 17:13 |
| anteaya | jpmaxman: see that red square? | 17:13 |
| *** degorenko is now known as _degorenko|afk | 17:13 | |
| armax | clarkb: well, I am thinking that this might be what is hurting us | 17:13 |
| armax | clarkb: perhaps that’s what you’re trying to tell me all along :) | 17:13 |
| armax | clarkb: I must admit I have only had a single cup of coffee so far | 17:14 |
| *** karthikp_ has quit IRC | 17:14 | |
| jpmaxman | should be fixed now anteaya | 17:14 |
| *** roxanagh_ has quit IRC | 17:14 | |
| *** karthikp_ has joined #openstack-infra | 17:14 | |
| clarkb | armax: basically infra is hands off br-ex in the non dvr jobs. So whatever happens in devstack and/or neutron needs to make that mtu be correct | 17:14 |
| *** hurgleburgler has quit IRC | 17:14 | |
| AJaeger | fungi, and the commit for requirements is head of mitaka - so that looks fine as well. | 17:15 |
| fungi | AJaeger: normally i would look at http://logs.openstack.org/ee/ee213608a68f6d8ce8f6299a45c1355eb599f548/post/zuul-cloner-test/5ec5f40/_zuul_ansible/ansible_log.txt to see what caused the POST_FAILURE but i'm not immediately spotting it | 17:15 |
| armax | clarkb: ack | 17:15 |
| *** john-davidge has quit IRC | 17:15 | |
| clarkb | armax: infra does configure br-ex in the dvr jobs because it is where we put the vxlan tunnel so we make sure it is configured properly for that tunnel | 17:15 |
| armax | clarkb: and that’s what I expect to happen | 17:15 |
| anteaya | jpmaxman: thank you | 17:15 |
| openstackgerrit | Eric Brown proposed openstack-infra/project-config: Enable py35 voting on vmware-nsx https://review.openstack.org/340708 | 17:15 |
| armax | clarkb: so right now my best shot is to get that br-ex mtu to match the other jobs and pray | 17:15 |
| fungi | AJaeger: though i will admit to finding the ansible log a little hard to follow so i'm probably overlooking something there | 17:16 |
| armax | clarkb: that the issue goes away | 17:16 |
| clarkb | armax: ya I think thats a good first thing to try | 17:16 |
| armax | clarkb: not very scientific, but nonetheless unharmful | 17:16 |
| armax | kevinbenton: do you copy? | 17:16 |
| armax | sc68cal: ^ | 17:16 |
| *** yamahata has joined #openstack-infra | 17:17 | |
| AJaeger | fungi: the zuul-cloner-test is fine, the failure was for the publish job | 17:17 |
| openstackgerrit | Merged openstack-infra/system-config: Refresh nodepool-builder logging https://review.openstack.org/341819 | 17:17 |
| AJaeger | fungi, let me check the next change merging and see what exactly fails... | 17:18 |
| pabelanger | fungi: AJaeger: the problem is related to the publisher logic. Because of the way we do logs in zuul-launcher atm, we truncate the logs for publisher taks | 17:18 |
| pabelanger | task* | 17:18 |
| pabelanger | fungi: if you look in zl02.o.o zuul debug.log you can see the issue | 17:19 |
| AJaeger | And so, if they're too large, we have a problem?Or when? | 17:19 |
| fungi | no, he's saying if we hit an issue with another publisher we end up also not publishing the console log | 17:20 |
| AJaeger | pabelanger, fungi: Please check last line of http://logs.openstack.org/ee/ee213608a68f6d8ce8f6299a45c1355eb599f548/post/openstack-manuals-tox-doc-publishdocs/372ada8/_zuul_ansible/ansible_log.txt - any ideas? | 17:20 |
| *** karthikp_ has quit IRC | 17:20 | |
| AJaeger | fungi: Ah... | 17:20 |
| AJaeger | fungi, the hint to look at ansible_log was good, thanks | 17:21 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/devstack-gate: Configure infrastructure MTU with global_physnet_mtu https://review.openstack.org/342975 | 17:22 |
| fungi | AJaeger: so we're getting 550 auth failures from teh ftp site? | 17:22 |
| *** tkelsey has quit IRC | 17:22 | |
| AJaeger | "stale file handle" ? | 17:22 |
| * AJaeger logs now into the ftp server and investigates... | 17:22 | |
| pabelanger | I have to step away for a few hours to run an errand. But I'll catch up on the discussions when I return. | 17:23 |
| openstackgerrit | Tim Burke proposed openstack-infra/project-config: Make py35 jobs voting for swiftclient https://review.openstack.org/342978 | 17:23 |
| fungi | AJaeger: sounds like there might be an issue with their backend fs (nfs server problems?) | 17:23 |
| *** tosky has quit IRC | 17:25 | |
| *** Kaiyan has quit IRC | 17:25 | |
| anteaya | pabelanger: happy errands | 17:25 |
| fungi | AJaeger: also the error could be misleading, maybe something like the parent directory for the target doesn't actually exist? | 17:25 |
| *** ddieterly[away] is now known as ddieterly | 17:26 | |
| AJaeger | fungi, might be - but how to figure out which one? | 17:27 |
| fungi | AJaeger: each ftp error seems to be about a specific path | 17:27 |
| fungi | AJaeger: so i would check the one it's mentioning | 17:27 |
| *** Ravikiran_K has joined #openstack-infra | 17:28 | |
| AJaeger | "environment-networking-storage-cinder.html: 550..." - that file exists 3 times and all occurences look fine | 17:29 |
| rcarrillocruz | pabelanger: fwiw, you can also do 'ssh pabelanger@review.openstack.org -p 29418 gsql', that should drop you in a mysql interactive session | 17:29 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-mediawiki: Clean up unused questy captcha variable https://review.openstack.org/342374 | 17:29 |
| clarkb | rcarrillocruz: pabelanger but it uses gsql which has far fewer features than mysql's client. Like a lack of readline | 17:29 |
| AJaeger | fungi, http://paste.openstack.org/show/533077/ | 17:29 |
| *** shashank_hegde has joined #openstack-infra | 17:30 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: fail on watchdog timeout https://review.openstack.org/342985 | 17:30 |
| jeblair | pabelanger, odyssey4me: ^ | 17:30 |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Revert "Allow comments to be edited by their owner" https://review.openstack.org/342987 | 17:30 |
| clarkb | it takes 9 minutes to make a qcow2 from our raw images on nodepool server | 17:30 |
| clarkb | :/ | 17:30 |
| jeblair | clarkb: we can move the builder to a new host now | 17:32 |
| clarkb | jeblair: ya, or two new hosts and make things happen at the same time | 17:32 |
| jeblair | yup | 17:32 |
| *** TheodoreBe has joined #openstack-infra | 17:32 | |
| AJaeger | fungi, http://logs.openstack.org/d9/d929bf03ad18bb4f3897fa4b4af13cb5fef401ea/post/zuul-cloner-test/5f1c79e/ is from master - looks fine as well. | 17:33 |
| AJaeger | woot, I think that zuul-cloner is working fine now in the post queue. Thanks a lot Nakato - and all those that reviewed! | 17:34 |
| *** ddieterly is now known as ddieterly[away] | 17:34 | |
| odyssey4me | jeblair thanks, to my untrained eye that seems to be right | 17:34 |
| *** TheodoreBe has quit IRC | 17:34 | |
| *** berendt has quit IRC | 17:38 | |
| *** roxanagh_ has joined #openstack-infra | 17:38 | |
| zaro | clarkb: RE: beaker example test for puppet-gerrit. i'm not sure how to do what you are suggesting. could you point me to an example? | 17:39 |
| *** harlowja has quit IRC | 17:39 | |
| Zara | ugh, it may take a few hours before storyboard stories are viewable again. :( that'll teach me to merge anything when I'm ill- especially on a Friday. :) on the plus side, I doubt many people want to read stories at the weekend. | 17:39 |
| *** TheodoreBe has joined #openstack-infra | 17:40 | |
| clarkb | zaro: context? sorry not sure exactly what is being referred to | 17:40 |
| *** tkelsey has joined #openstack-infra | 17:41 | |
| zaro | clarkb: ohh sorry, that was meant for crinkle | 17:41 |
| *** harlowja has joined #openstack-infra | 17:41 | |
| fungi | AJaeger: oh, here's a possibility... our ftp data session may be dying. wonder if something is killing the data socket and then we're getting those errors back over the control socket | 17:41 |
| *** Goneri has joined #openstack-infra | 17:41 | |
| *** piet has quit IRC | 17:41 | |
| AJaeger | btw. the last job succeeded. But yeah, a timeout/kill of that would explain it. | 17:42 |
| zaro | crinkle: i'm not sure what you are suggesting with http://git.openstack.org/cgit/openstack-infra/puppet-gerrit/tree/spec/acceptance/fixtures/default.pp | 17:42 |
| AJaeger | We might have had *two* ftp sessions running at the same time, for both of the merged changes. Could that cause problems? | 17:42 |
| anteaya | Zara: thanks for fixing it and for letting us know | 17:42 |
| *** patricku is now known as puiterwijk|mob | 17:43 | |
| crinkle | zaro: that fixture gets run by beaker, so my suggestion is to add new parameters to that gerrit class definition as though it were in system-config | 17:43 |
| fungi | AJaeger: ooh, uploads fighting over the same files _could_ possibly explain it, yes | 17:43 |
| anteaya | Zara: feel better! | 17:44 |
| fungi | AJaeger: ftp uploading there is non-atomic after all, and so racy with other jobs trying to do the same | 17:44 |
| crinkle | zaro: i'm not saying to block the its-storyboard patches on it, but it would be a timesaver in the future | 17:44 |
| Zara | anteaya: thank you. :) | 17:44 |
| anteaya | :) | 17:44 |
| Zara | I'll keep an eye in case the gate doesn't like it for whatever reason | 17:45 |
| zaro | crinkle: gotcha, will do. | 17:45 |
| *** pfallenop has quit IRC | 17:45 | |
| *** pfallenop has joined #openstack-infra | 17:45 | |
| anteaya | Zara: k | 17:45 |
| AJaeger | fungi, so, anything we can do now - or will zuul v3 fix this magically? | 17:45 |
| fungi | AJaeger: i think the alternative docs publishing spec should fix the atomicity, though zuul v3 may fix the concurrency and ordering races we see for publication as well | 17:46 |
| AJaeger | fungi, yeah - ok, then let's not dig further for now... | 17:46 |
| AJaeger | fungi, thanks! | 17:46 |
| fungi | or if not, fixes for those will likely need to wait for zuul v3 at any rate | 17:46 |
| AJaeger | so, what's the next step with zuul-cloner and post jobs? The test output looked fine to me... | 17:47 |
| *** Na3iL has joined #openstack-infra | 17:49 | |
| *** asettle has joined #openstack-infra | 17:50 | |
| AJaeger | btw. do we have zuul-cloner on all nodes - also on the proposal slave? | 17:52 |
| *** links has quit IRC | 17:52 | |
| fungi | AJaeger: checking... | 17:52 |
| *** karthikp_ has joined #openstack-infra | 17:52 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Revert "Allow comments to be edited by their owner" https://review.openstack.org/342987 | 17:53 |
| fungi | fungi@proposal:~$ /usr/zuul-env/bin/zuul-cloner --version | 17:53 |
| fungi | Zuul version: 2.1.1.dev15 | 17:53 |
| fungi | AJaeger: looks like yes | 17:53 |
| anteaya | Zara: ^^ | 17:53 |
| Zara | \o/ | 17:53 |
| Zara | now to wait for everything to build and deploy | 17:53 |
| AJaeger | fungi, great. | 17:53 |
| anteaya | yay! | 17:53 |
| Zara | I hope this is one of those things that somehow builds in an hour and not 3 weeks, but we'll see! | 17:53 |
| anteaya | :) | 17:54 |
| fungi | AJaeger: i think next step is to cautiously try using z-c in real post jobs, and then if we still don't see any new issues announce it's ready | 17:54 |
| AJaeger | Which jobs are best for these? Coverage ones? ;) | 17:55 |
| fungi | AJaeger: yeah, i think coverage is the best candidate | 17:55 |
| * AJaeger prepares a change... | 17:55 | |
| fungi | pick a project or two that already use constraints and adjust their tox.ini to do so with the cover env | 17:55 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 voting on vmware-nsx https://review.openstack.org/340708 | 17:55 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Always set br-ex mtu including for grenade runs https://review.openstack.org/343011 | 17:55 |
| *** Goneri has quit IRC | 17:56 | |
| *** asettle has quit IRC | 17:57 | |
| clarkb | jeblair: sdague: fungi other devstack-gaters ^ is thought to be the temporary fix for neutrons mtu issues. Then they will address the todo in there to have devsatck do it properly and we can remove the d-g code | 17:58 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use zuul-cloner for coverage job https://review.openstack.org/343014 | 17:58 |
| *** rcernin has joined #openstack-infra | 17:59 | |
| *** EricGonc_ has quit IRC | 18:00 | |
| *** hashar has joined #openstack-infra | 18:00 | |
| *** nadya has quit IRC | 18:00 | |
| *** IlyaG has quit IRC | 18:01 | |
| AJaeger | fungi, stevemar - and keystone change related to 343014: https://review.openstack.org/343016 | 18:01 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: WIP: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 18:03 |
| AJaeger | fungi, stevemar is up for it ;) So, 343014 needs reviews | 18:03 |
| *** EricGonczer_ has joined #openstack-infra | 18:04 | |
| *** sdague has quit IRC | 18:05 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Add configurations for gerrit its plugins https://review.openstack.org/330922 | 18:05 |
| *** jaosorior has quit IRC | 18:05 | |
| *** jsavak has joined #openstack-infra | 18:05 | |
| sc68cal | armax: sounds like a plan | 18:05 |
| *** cardeois__ has joined #openstack-infra | 18:06 | |
| prometheanfire | is irtermite in today? | 18:08 |
| *** jmccrory_away is now known as jmccrory | 18:08 | |
| * fungi has no idea what an irtermite is | 18:09 | |
| *** cardeois_ has quit IRC | 18:09 | |
| anteaya | iterative termite was my go to phrase | 18:09 |
| anteaya | in which case I do hope the answer is no | 18:09 |
| *** bhavik has quit IRC | 18:10 | |
| *** cardeois__ has quit IRC | 18:10 | |
| fungi | could be an infra-red termite i suppose | 18:10 |
| fungi | er, infrared (i guess that's usually not hyphenated after all) | 18:10 |
| anteaya | could be, could be one of those | 18:11 |
| anteaya | I'm still going with I hope the answer is no | 18:11 |
| *** xyang1 has quit IRC | 18:11 | |
| *** pvaneck has joined #openstack-infra | 18:14 | |
| *** dizquierdo has quit IRC | 18:15 | |
| *** rwsu has joined #openstack-infra | 18:16 | |
| anteaya | fungi: the more I watch the news the more I find cartoons appealing | 18:16 |
| *** kzaitsev_mb has quit IRC | 18:17 | |
| fungi | anteaya: i can't remember the last time i "watched the news" | 18:17 |
| *** eezhova has quit IRC | 18:17 | |
| fungi | it's been many years, at least | 18:17 |
| clarkb | fungi: you don't enjoy the 24/7 election coverage? | 18:17 |
| clarkb | :P | 18:17 |
| anteaya | you make and continue to make a good choice | 18:17 |
| *** Na3iL has quit IRC | 18:17 | |
| anteaya | made and continue to make* | 18:18 |
| jeblair | i do remember the last time i watched futurama. it was last night. | 18:18 |
| *** Sukhdev has joined #openstack-infra | 18:18 | |
| pleia2 | heh | 18:18 |
| fungi | it was a nightly ritual for my parents, but long ago i realized that it's just another television program... no more or less true than the gameshow that came on before it or the sitcom after it | 18:18 |
| jeblair | (it's on netflix streaming!) | 18:18 |
| fungi | jeblair: indeed! i watched quite a bit of futurama last night | 18:19 |
| timrc | Is thisjeblair camp fire story time? | 18:19 |
| fungi | my younger brother got me the full boxed set for my birthday, so christine and i have been watching it in sequence | 18:19 |
| timrc | this jeblair* so excited I don't even want to take the time to put spaces between words | 18:19 |
| *** trown|lunch is now known as trown | 18:20 | |
| timrc | Rick and Morty is my version of Futurama. Kids have really messed things up for me though. I have to watch it at odd hours while they're asleep or away. | 18:21 |
| fungi | oh yes, r&m is a favorite of mine as well | 18:21 |
| fungi | timrc: did you make it to the summit in atlanta? there was a grampa rick statue sitting on one of teh benches in an alcove in the turner building next to the conference center | 18:22 |
| timrc | I feel like we're living in a golden age of cartoons and not just adult cartoons.. I'm envious of my kids. | 18:22 |
| timrc | I had he-man... which I loved, but was literally 5 frames of animation over 20 minutes. | 18:22 |
| clarkb | timrc: what is crazy for me is they won't have to wait for saturday mornings | 18:22 |
| *** davidlenwell has quit IRC | 18:23 | |
| *** Na3iL has joined #openstack-infra | 18:23 | |
| timrc | fungi: No I was not there :/ Wife was about to have our son. | 18:23 |
| fungi | timrc: thundar was way better than he-man. pure post-apocalyptia | 18:23 |
| fungi | er, thundarr | 18:23 |
| timrc | clarkb: Nope! Which is sort of a shame they don't have to put in the work for their cartoons like we did | 18:23 |
| anteaya | fungi: futurama was a nightly ritual for your parents? | 18:24 |
| fungi | anteaya: no, the nightly news | 18:24 |
| anteaya | ah | 18:24 |
| clarkb | timrc: or trading around vhs tapes | 18:24 |
| timrc | fungi: I've not seen thundarr. | 18:24 |
| timrc | I'll have to look it up. | 18:24 |
| fungi | we sort of switched topics quickly there | 18:24 |
| clarkb | we had dragon ball z all 20 minutes in 5 hours in a group | 18:25 |
| timrc | clarkb: Yeah... it's weird. My daughter has mastered Netflix and its strange to me to watch a 4 year old curate a watch list and use voice search to get content. | 18:25 |
| clarkb | fungi: hey now, still talking cartoons | 18:25 |
| anteaya | timrc: congratulations on the birth of your son | 18:26 |
| timrc | anteaya: Oh that happened 2 years ago :) | 18:26 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Refactor to accept java_home from parent modules https://review.openstack.org/331523 | 18:26 |
| clarkb | as an adult watching dragon ball z is painful. I can't believe how little content they stretched out over a season | 18:26 |
| anteaya | oh sorry, at that was the atlanta summit, I misread austin | 18:26 |
| clarkb | but as a kid that stuff was gold | 18:26 |
| timrc | But thanks! It's been a good 2 years :) | 18:26 |
| fungi | clarkb: the original dragonball was much better imho | 18:27 |
| timrc | clarkb: Yeah I made the mistake of trying to watch he-man recently and regretted it. | 18:27 |
| anteaya | timrc: oh I'm glad | 18:27 |
| anteaya | he-man sounds like hercules | 18:27 |
| *** ddieterly[away] is now known as ddieterly | 18:27 | |
| anteaya | all 5 frames of it | 18:27 |
| clarkb | timrc: they are also starting to remake a bunch of shows | 18:27 |
| clarkb | thundercats for example | 18:28 |
| fungi | clarkb: dbz was so terribly dragged out, though the dbz "kai" rework where they distilled out a lot of the repetition is more watchable | 18:28 |
| fungi | oh, i've seen some of the thundercats reboot. not impressed | 18:28 |
| *** xyang1 has joined #openstack-infra | 18:28 | |
| clarkb | inspector gadget and danger mouse too | 18:29 |
| clarkb | the originals are much better imo | 18:29 |
| fungi | the sailor moon reboot (sailor moon crystal) isn't too bad, as someone who sat through all of the original for reasons not of my choosing | 18:29 |
| timrc | clarkb: I'd be super sad if they touched Count Duckula. | 18:29 |
| timrc | Literally my favorite cartoon of all time. | 18:29 |
| *** Teresita-Warrior has quit IRC | 18:29 | |
| anteaya | fungi: I'm impressed you have critque to offer on sailor moon | 18:30 |
| timrc | My daughter really enjoys Ranma 1/2 which makes me a really bad father considering some of the more mature story lines. | 18:30 |
| zaro | crinkle: question for you in https://review.openstack.org/#/c/331523 | 18:30 |
| * AJaeger signs off and wishes everybody a great weekend | 18:31 | |
| clarkb | AJaeger: enjoy | 18:31 |
| jroll | prometheanfire: didn't realize irtermite hung out here? or wrong channel I guess :) | 18:31 |
| * AJaeger looks forward to some great red wine now ;) | 18:31 | |
| fungi | anteaya: well, the subtitled japanese versions anyway. i've caught bits of the english-dubbed edits they ran in the usa in years past and they're entirely unwatchable | 18:31 |
| fungi | AJaeger: have a good weekend! | 18:31 |
| AJaeger | thanks, fungi | 18:31 |
| timrc | AJaeger: Alcohol is a good way to end the week :) Enjoy | 18:31 |
| AJaeger | timrc: especially if the kids are sleeping ;) | 18:32 |
| *** Na3iL has quit IRC | 18:32 | |
| AJaeger | thanks, timrc | 18:32 |
| timrc | AJaeger: Indeed! | 18:32 |
| prometheanfire | did I ask that here | 18:33 |
| prometheanfire | lol, oops | 18:33 |
| anteaya | fungi: ha ha ha | 18:33 |
| anteaya | AJaeger: enjoy your weekend | 18:33 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/devstack-gate: Configure br-ex MTU using PUBLIC_DEVICE_MTU variable https://review.openstack.org/343024 | 18:34 |
| clarkb | ubuntu-trusty is almost done, then ubuntu-xenial will start and when thats done we can restart nodepool | 18:34 |
| anteaya | fungi: okay I have to ask, what circumstances forced you to watch the subtitled japanese sailor moon episodes? | 18:34 |
| fungi | anteaya: years ago, accommodating a girlfriend who was into a lot of majokko genre anime | 18:35 |
| pleia2 | I don't mind the english dubbing, the content edits are what drive me batty | 18:36 |
| anteaya | fungi: ah okay, credit given then | 18:36 |
| fungi | pleia2: yes, that's more what made it unwatchable | 18:36 |
| fungi | pleia2: cutting out significant scenes and changing storylines in ways that made them even more inane | 18:36 |
| pleia2 | fungi: yep | 18:36 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/devstack-gate: Configure br-ex MTU using PUBLIC_DEVICE_MTU variable https://review.openstack.org/343024 | 18:37 |
| jeblair | wow: https://www.reddit.com/r/videos/comments/3dgbg1/the_futurama_theme_was_actually_based_on_a_1967/ | 18:37 |
| pleia2 | I recently started rewatching subtitled, unedited Tenchi Muyo! several episodes make a ton more sense than now when I watched them on the cartoon network decades ago | 18:37 |
| jeblair | i love both the pierre henry original as well as the fatboy slim remix | 18:37 |
| *** tkelsey has quit IRC | 18:38 | |
| fungi | oh yes. i' have it flagged to watch soon (either on crunchyroll or funimation, i don't recall which) | 18:38 |
| pleia2 | funimation | 18:38 |
| openstackgerrit | Merged openstack-infra/glean: Added support for setting bond-lacp-rate and bond-xmit_hash_policy https://review.openstack.org/338607 | 18:38 |
| *** karthikp_ has quit IRC | 18:38 | |
| fungi | crunchyroll has started to slack off. not getting much good new content as in years past | 18:39 |
| anteaya | jeblair: I'm listening now | 18:39 |
| *** karthikp_ has joined #openstack-infra | 18:40 | |
| jeblair | (also i love that the video of the original is a video of a 33rpm spinning) | 18:40 |
| anteaya | he he he | 18:40 |
| jeblair | now i have a challenge for my local hipster record joint | 18:40 |
| fungi | hah | 18:41 |
| *** ihrachys has quit IRC | 18:42 | |
| jeblair | and if they don't have it -- well, there's a place down the street that is full of reel-to-reels and moogs. i have no idea what they do in there -- sell? repair? just hang out and groove? i just assume i'm not cool enough to know. maybe i can ask them. | 18:42 |
| *** mhickey has quit IRC | 18:42 | |
| crinkle | zaro: responded | 18:43 |
| anteaya | moogs? | 18:44 |
| * anteaya looks it up | 18:44 | |
| * anteaya is not cool | 18:44 | |
| timrc | fungi: So you're endorsing crunchyroll and funimation? I've been on the fence... | 18:44 |
| jeblair | anteaya: https://en.wikipedia.org/wiki/Moog_synthesizer | 18:44 |
| openstackgerrit | C.J. Collier proposed openstack-infra/gerritbot: DO NOT MERGE - Improve gerritbot error handling https://review.openstack.org/287056 | 18:44 |
| timrc | fungi: I've been watching a lot of anime on Hulu. | 18:44 |
| anteaya | ah thanks jeblair | 18:45 |
| anteaya | I was thinking this was not it: http://www.moogparts.com/ | 18:45 |
| fungi | timrc: i mostly just go to whoever has the shows i'm looking for | 18:45 |
| fungi | timrc: also hulu wants me to pay for a lot of other unrelated stuff | 18:45 |
| timrc | fungi: Fair enough. I watch some of that unrelated stuff :) | 18:46 |
| timrc | Still these services are so cheap. I got rid of capable and even paying for HBO now, hulu, netflix, and amazon prime, I'm still not close to what my cable bill was. | 18:47 |
| timrc | got rid of cable* | 18:47 |
| pleia2 | timrc: same | 18:47 |
| pleia2 | (though I haven't had cable since 2001, not actually a big tv person) | 18:47 |
| anteaya | I found this too: http://artsites.ucsc.edu/EMS/music/equipment/synthesizers/analog/moog/Moog.html which actually is really funny | 18:47 |
| *** sdake has joined #openstack-infra | 18:47 | |
| clarkb | I cancelled hulu and put down the reason for cancelling as "Tired of paying to watch ads" then a few months later found out you could now pay them more for no ads | 18:48 |
| clarkb | its like they listend to me but too little too late | 18:48 |
| pleia2 | clarkb: yeah, we sprung for hulu plus | 18:48 |
| timrc | clarkb: Yeah just a couple $/mo more. | 18:48 |
| fungi | yep, i don't watch streaming services that serve me ads | 18:48 |
| pleia2 | it wasn't just the ads, they were super repetitive | 18:48 |
| pleia2 | repetitive was annoying | 18:49 |
| clarkb | pleia2: ya I had plus too, and had to watch the ads | 18:49 |
| timrc | Now they ask you to choose an ad category. | 18:49 |
| clarkb | its only very recently they fixed it and I like to think as a direct result of my feedback t othem | 18:49 |
| timrc | Do I want to see outdoor ads, cooking ads, sports ads, etc | 18:49 |
| pleia2 | clarkb: er yeah, plus + without ads :) | 18:49 |
| fungi | nibalizer: should i add openstack-infra/vinz to governance? i realize we never put it in there because it was empty and looked like it was dead (until recently!) | 18:49 |
| *** timello has quit IRC | 18:49 | |
| pleia2 | vinz++ | 18:49 |
| rcarrillocruz | haha ^ | 18:49 |
| timrc | clarkb: Obviously your feedback had everything to do with that change | 18:49 |
| clarkb | timrc: I am sure of it :) | 18:49 |
| clarkb | I use netflix, amazon prime (paid for anyways due to all that lovely free shipping), and slingtv with hbo | 18:50 |
| *** timello has joined #openstack-infra | 18:50 | |
| anteaya | fungi: vinz contributors should be able to vote in infra and tc elections, I feel | 18:50 |
| clarkb | I should probably drop netflix and only resubscribe when house of cards ahppens | 18:50 |
| fungi | anteaya: that's the way i'm feeling about it too | 18:50 |
| anteaya | cool | 18:50 |
| timrc | Now I just need a single interface to all these services and one that I enjoy using. | 18:50 |
| fungi | anteaya: but i don't want to add them without confirming with them | 18:51 |
| * anteaya moves onto listening/watching other fatboy slim selections | 18:51 | |
| timrc | I hate the pattern of checking all the services for a moving before renting it on Prime. | 18:51 |
| clarkb | timrc: thats what the wiiu was supposed to be then they screwed it up so badly | 18:51 |
| clarkb | timrc: but roku has grown a fairly decent search feature that I use now | 18:51 |
| timrc | blargh movie* | 18:51 |
| anteaya | I never get tired of watching christopher walken dance | 18:51 |
| *** akshai has quit IRC | 18:51 | |
| timrc | clarkb: Ah, I might have to look at roku again. | 18:51 |
| anteaya | fungi: sounds reasonable | 18:51 |
| *** akshai has joined #openstack-infra | 18:51 | |
| jeblair | timrc, clarkb: i think the roku has a oh yeah | 18:51 |
| fungi | anteaya: weapon of choice? yes, excellent video. some of his best work | 18:51 |
| jeblair | clarkb just said that :) | 18:51 |
| fungi | yes, roku's search is not too bad | 18:52 |
| fungi | though i wish there was a free software platform to run the streaming apps on | 18:52 |
| clarkb | also if you do digital rentals google play tends to do 48hours and amazon and others only 24 hours | 18:52 |
| clarkb | at the same price | 18:52 |
| timrc | clarkb: I also want to check out Kodi. | 18:53 |
| anteaya | fungi: yes and agreed | 18:53 |
| clarkb | fungi: I take consolation in the fact that roku seems to try to be good about publishing the sources they are required to | 18:53 |
| fungi | clarkb: this is true at least | 18:53 |
| jeblair | our local physical dvd rental is generally cheaper than the streaming rentals. | 18:53 |
| fungi | they don't want to be on the wrong end of a gpl violation suit | 18:53 |
| clarkb | https://roku.app.box.com/v/rokuopensourcesoftware | 18:53 |
| timrc | jeblair: I miss that browsing experience as inefficient and sometimes upsetting (if what I wanted was unavailable) as it was | 18:54 |
| *** EricGonc_ has joined #openstack-infra | 18:54 | |
| jeblair | timrc: and the movie buffs at the counter who might judge your rental, or suggest something really cool -- depending on your and their mood. :) | 18:54 |
| pleia2 | jeblair: local physical dvd rental still exists?! | 18:54 |
| fungi | clarkb: also i was able to get their shoutcast app pointed at my local icecast2 stream at the house (though it required using a remote debug interface over the network to inject configuration into the roku) | 18:54 |
| timrc | pleia2: We have it here... there's a really cool place called ILuvVideo.. http://www.iluvvideo.com/ | 18:55 |
| anteaya | jeblair: you still have a physical dvd rental place? I'm envious | 18:55 |
| timrc | jeblair: Totally | 18:55 |
| jeblair | pleia2: at least one? -- http://www.yelp.com/biz/five-star-video-berkeley-4 | 18:56 |
| *** esberglu has quit IRC | 18:56 | |
| pleia2 | timrc: how retro :) | 18:56 |
| clarkb | timrc: jeblair back on the island every rental place had bootlegs adn you couldn't get originals so in addition to that portion of the browsing experience we also had to add in "is this a good rip? was it recordedin a theater? do the foregin language subtitles work?" | 18:56 |
| pleia2 | jeblair: aha | 18:56 |
| clarkb | timrc: jeblair now apparently they don't do rentals anymore you just pay them $3 or somethign for whatever bootleg you want | 18:56 |
| anteaya | clarkb: ha ha ha | 18:56 |
| clarkb | pleia2: we have videorama in portland | 18:57 |
| *** alaski is now known as lascii | 18:57 | |
| pleia2 | clarkb: you would in portland | 18:57 |
| *** EricGonczer_ has quit IRC | 18:57 | |
| pleia2 | the last non-adult one that I knew about in SF turned into a burger shop | 18:58 |
| *** cody-somerville has quit IRC | 18:58 | |
| *** cody-somerville has joined #openstack-infra | 18:58 | |
| *** cody-somerville has quit IRC | 18:58 | |
| *** cody-somerville has joined #openstack-infra | 18:58 | |
| *** cody-somerville has quit IRC | 18:59 | |
| jeblair | locally sourced grass-fed wagyu burgers, i assume | 18:59 |
| timrc | Looks like Kodi + PlayOn can sub Roku if you want something mostly free / open source (have to pay for playon) | 18:59 |
| *** cody-somerville has joined #openstack-infra | 18:59 | |
| openstackgerrit | Merged openstack-infra/glean: workaround for ubuntu 14.04 bonding issue https://review.openstack.org/339067 | 18:59 |
| *** yamahata has quit IRC | 18:59 | |
| clarkb | jeblair: my local butcher makes burgers from their painted hills ground chuck. So good | 19:00 |
| *** cardeois__ has joined #openstack-infra | 19:01 | |
| timrc | jeblair, clarkb: lol | 19:01 |
| fungi | i need to go run some friday errands while tourists are still out on the beach frying themselves. back in a while | 19:02 |
| *** cardeois__ has quit IRC | 19:02 | |
| *** jsavak has quit IRC | 19:02 | |
| *** esberglu has joined #openstack-infra | 19:02 | |
| timrc | I'm sure the tables and chairs at the burger joint are made from recycles vhs tapes and rental vcrs. | 19:02 |
| timrc | recycled* | 19:02 |
| anteaya | fungi: enjoy tourist free errands | 19:02 |
| *** cardeois__ has joined #openstack-infra | 19:02 | |
| timrc | To get access to the secret menu you need to whisper "Be kind, rewind" | 19:02 |
| nibalizer | fungi: sure! | 19:02 |
| anteaya | 2 for 1 every Wednesday | 19:03 |
| *** cardeois__ has quit IRC | 19:03 | |
| nibalizer | hurgleburgler and i submitted a talk on it to the sunmit as well | 19:03 |
| *** TheodoreBe has quit IRC | 19:03 | |
| fungi | nibalizer: thanks for confirming. i'll whip up that governance change forthwith | 19:03 |
| *** cardeois has joined #openstack-infra | 19:03 | |
| nibalizer | and were actually suoer close to me writing the poopit to run vinz on review-dev | 19:03 |
| nibalizer | the hype is real | 19:03 |
| fungi | nibalizer: we do have a repo for puppet-vinz already in governance at least | 19:04 |
| fungi | anyway, bbl | 19:04 |
| nibalizer | im glad we got that one :p | 19:04 |
| clarkb | jeblair: almost down to the last image build. ubuntu-trusty is gettings its vhd conversion done | 19:05 |
| *** tonytan4ever has quit IRC | 19:05 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/system-config: Ignore ansible .retry files https://review.openstack.org/343040 | 19:05 |
| d0ugal | fungi, anteaya, pabelanger: yay, I only see one Dougal (just how I like it!). Thanks again! :) | 19:05 |
| jeblair | clarkb: cool, i'm about to head for lunch, but will be happy to restart nodepool if it's done when i get back and you haven't already | 19:05 |
| clarkb | jeblair: I expect that it will be done about 1:30 PDT | 19:06 |
| clarkb | since it has to do xenial next | 19:06 |
| clarkb | I am about to find food too | 19:06 |
| jeblair | cool | 19:06 |
| openstackgerrit | Brian Haley proposed openstack-infra/devstack-gate: [WIP] Support an IPv6 underlay network https://review.openstack.org/343041 | 19:07 |
| anteaya | yay one d0ugal | 19:07 |
| anteaya | d0ugal: thanks for the feedback | 19:08 |
| *** ccamacho is now known as ccamacho|out | 19:08 | |
| * d0ugal is now signing off for the week, have a good 'un | 19:08 | |
| *** d0ugal has quit IRC | 19:09 | |
| anteaya | thanks you too | 19:09 |
| *** piet has joined #openstack-infra | 19:10 | |
| stevemar | AJaeger: fwiw, i pulled down the cover change and it was fine, i'm still happy to have keystone be the guinea pig :) | 19:10 |
| anteaya | I understand parts of 70's music I never understood before: https://www.youtube.com/watch?v=n3K_fZDvINs | 19:11 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: WIP: Enable net-iso for ovb jobs https://review.openstack.org/343017 | 19:14 |
| rcarrillocruz | asselin__: just in case you are not aware of it, it's possible to disable the creation of .retry ansible files | 19:17 |
| rcarrillocruz | re: your system change about adding .retry to .gitignore | 19:17 |
| *** xyang1 has quit IRC | 19:17 | |
| rcarrillocruz | just add retry_files_enabled = False to ansible.cfg defaults section iirc | 19:17 |
| *** xyang1 has joined #openstack-infra | 19:17 | |
| *** yamahata has joined #openstack-infra | 19:19 | |
| *** karthikp_ has quit IRC | 19:20 | |
| *** e0ne has joined #openstack-infra | 19:20 | |
| rcarrillocruz | folks, anyone aware of pypi mirroring issues? | 19:21 |
| rcarrillocruz | i've had since a couple of days issues on my launcher tests | 19:21 |
| rcarrillocruz | due to package being broken | 19:21 |
| rcarrillocruz | METADATA file missing | 19:21 |
| rcarrillocruz | latest one: | 19:21 |
| rcarrillocruz | http://paste.openstack.org/show/533092/ | 19:21 |
| * anteaya has moved to supertramp | 19:21 | |
| *** xarses_ has joined #openstack-infra | 19:22 | |
| *** xarses has quit IRC | 19:22 | |
| *** xarses_ has quit IRC | 19:23 | |
| *** xarses_ has joined #openstack-infra | 19:23 | |
| *** xarses_ has quit IRC | 19:23 | |
| *** akshai has quit IRC | 19:23 | |
| *** xarses_ has joined #openstack-infra | 19:24 | |
| asselin__ | rcarrillocruz, ok I wasn't aware of that. | 19:25 |
| rcarrillocruz | fungi: any hint ^ ? i've had since yesterday issues with oslo.config, oslo.utils.... | 19:25 |
| rcarrillocruz | same symptom, Nova 500 due to missing METADATA file on pypi package | 19:26 |
| *** akshai has joined #openstack-infra | 19:26 | |
| anteaya | rcarrillocruz: he is running errands at the moment | 19:26 |
| rcarrillocruz | ah k, thx | 19:26 |
| anteaya | you're welcome | 19:27 |
| rcarrillocruz | asselin__: and yeah, they're very irritating | 19:27 |
| rcarrillocruz | i wonder why they're not disabled by default :/ | 19:27 |
| *** akshai has quit IRC | 19:27 | |
| clarkb | rcarrillocruz: thats a package not mirror issue aiui | 19:28 |
| clarkb | you will want to investigate the install and packages themselves | 19:28 |
| rcarrillocruz | well, as i have had the same problem with multiple packages i wonder if there's something systemic in the mirroring | 19:28 |
| clarkb | I really doubt it | 19:28 |
| clarkb | probably systemic in our packaging | 19:29 |
| clarkb | all the mirroring does is copy the sdists and wherls | 19:29 |
| clarkb | which domt do anything to their contents | 19:29 |
| rcarrillocruz | i must be superunlucky then, 3 different broken packages :/ | 19:29 |
| rcarrillocruz | i got past a few of them as they got versions bumped on requirements | 19:30 |
| clarkb | would be easy for you to test by pointing at pypi proper | 19:30 |
| anteaya | wherls very descriptive | 19:30 |
| rcarrillocruz | i'm now stuck on oslo.context, but that's on hold since it seems the bump affects some projects | 19:30 |
| rcarrillocruz | :/ | 19:30 |
| rcarrillocruz | phschwartz: +1'd the puppet-openstackci extension spec | 19:32 |
| armax | clarkb: this is upsetting: | 19:34 |
| armax | http://logs.openstack.org/11/343011/1/check/gate-grenade-dsvm-neutron-multinode/7ee8f94/logs/worlddump-2016-07-15-185030-cinder_resources_created.txt.gz | 19:34 |
| clarkb | armax: its possibly br-tun also needs it? | 19:35 |
| *** e0ne has quit IRC | 19:35 | |
| armax | clarkb: let’s see | 19:35 |
| clarkb | I just pulled lunch out of the oven but can help afyer | 19:37 |
| armax | clarkb: no worries | 19:37 |
| *** xarses_ is now known as xarses | 19:37 | |
| *** e0ne has joined #openstack-infra | 19:38 | |
| rcarrillocruz | abregman: i rechecked your shade change | 19:42 |
| rcarrillocruz | the fix that caused neutron test failures merged a couple days ago | 19:42 |
| rcarrillocruz | your change now passed tests | 19:42 |
| *** nadya has joined #openstack-infra | 19:43 | |
| *** karthikp_ has joined #openstack-infra | 19:45 | |
| *** jsavak has joined #openstack-infra | 19:47 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add mulitnode CI job support to tripleo-ci https://review.openstack.org/324777 | 19:47 |
| *** woodster_ has quit IRC | 19:49 | |
| *** ihrachys has joined #openstack-infra | 19:50 | |
| *** jheroux has quit IRC | 19:52 | |
| *** TheodoreBe has joined #openstack-infra | 19:53 | |
| *** ddieterly is now known as ddieterly[away] | 19:58 | |
| *** nadya has quit IRC | 19:58 | |
| *** piet has quit IRC | 19:59 | |
| *** Apoorva has quit IRC | 20:00 | |
| *** piet has joined #openstack-infra | 20:02 | |
| phschwartz | rcarrillocruz: I have a good bit of changes to it that I will be pushing out soon | 20:03 |
| phschwartz | fungi: as the PTL I think you would be the best to answer this. There is the openstackci spec that is not 100% done. ELK is still being worked on, then there is the extenstion spec for it that I was working on. Due to community discussion I am abandoning a large portion of the spec. Would there be an issue updating the original spec to include the new work or should I leave it in a separate spec? | 20:05 |
| clarkb | rcarrillocruz: can you point me at a log where that fails again? | 20:05 |
| *** tonytan4ever has joined #openstack-infra | 20:05 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul: Support swift logserver without Send-Temp-Url-Key https://review.openstack.org/270338 | 20:06 |
| *** eezhova has joined #openstack-infra | 20:07 | |
| *** sdake has quit IRC | 20:08 | |
| *** kien-ha has quit IRC | 20:09 | |
| *** hashar has quit IRC | 20:09 | |
| pabelanger | and back | 20:09 |
| *** cardeois has quit IRC | 20:10 | |
| *** tonytan4ever has quit IRC | 20:11 | |
| *** TheodoreBe has quit IRC | 20:11 | |
| *** amitgandhinz has quit IRC | 20:13 | |
| *** burgerk has quit IRC | 20:15 | |
| *** Apoorva has joined #openstack-infra | 20:16 | |
| *** ddieterly[away] is now known as ddieterly | 20:20 | |
| Zara | storyboard.openstack.org and storyboard-dev.openstack.org are back up and running as usual! | 20:22 |
| Zara | thanks for your patience, everyone :) | 20:22 |
| *** roxanagh_ has quit IRC | 20:23 | |
| *** weshay has quit IRC | 20:25 | |
| clarkb | jeblair: qcow2 conversion just started so ~15 minutes away from completion? | 20:27 |
| jeblair | clarkb: cool | 20:27 |
| *** hurgleburgler has joined #openstack-infra | 20:28 | |
| *** adrian_otto has joined #openstack-infra | 20:31 | |
| *** adrian_otto has quit IRC | 20:31 | |
| *** adrian_otto has joined #openstack-infra | 20:33 | |
| openstackgerrit | Merged openstack-infra/system-config: Ignore ansible .retry files https://review.openstack.org/343040 | 20:34 |
| *** davidlenwell has joined #openstack-infra | 20:34 | |
| *** rajinir has quit IRC | 20:35 | |
| *** EricGonc_ has quit IRC | 20:35 | |
| *** tkelsey has joined #openstack-infra | 20:35 | |
| *** thorst_ has quit IRC | 20:35 | |
| *** EricGonczer_ has joined #openstack-infra | 20:37 | |
| *** piet has quit IRC | 20:39 | |
| *** tkelsey has quit IRC | 20:39 | |
| *** e0ne has quit IRC | 20:41 | |
| *** thorst_ has joined #openstack-infra | 20:41 | |
| *** gyee has quit IRC | 20:42 | |
| notmorgan | jeblair: sorry for the delay today, had an emergency (had a bottle explode and drench desk, computer, monitor, floor, couch, coffee table, etc) | 20:43 |
| *** gordc has quit IRC | 20:43 | |
| notmorgan | jeblair: can we follow up on the config stuff monday? this has been a complete mess to deal with (everything is sticky) | 20:43 |
| *** Ravikiran_K has quit IRC | 20:43 | |
| jeblair | notmorgan: of course! also, the config stuff is kinda sticky too. | 20:44 |
| *** EricGonczer_ has quit IRC | 20:44 | |
| clarkb | jeblair: ok image builds are done now. Should I go ahead and restart the daemon? | 20:44 |
| notmorgan | yah, i noticed, just not "make me want to take a shower" sticky | 20:44 |
| jeblair | notmorgan: also, i have started some actual reasonably sized patches with a real narrative. hopefully zuul things will start to get less sticky. | 20:44 |
| notmorgan | jeblair: cool. | 20:44 |
| *** esberglu has quit IRC | 20:44 | |
| *** armax_ has joined #openstack-infra | 20:45 | |
| notmorgan | jeblair: stupid champagne bottle (I wasn't even opening it for drinking... it just went POP) | 20:45 |
| notmorgan | on the plus side my desk no longer has clutter on it. | 20:45 |
| *** EricGonczer_ has joined #openstack-infra | 20:46 | |
| timrc | notmorgan: Wouldn't that be cool if the producer of that champagne had a bug tracker and you could file the defect? | 20:46 |
| *** Teresita-Warrior has joined #openstack-infra | 20:46 | |
| *** armax has quit IRC | 20:46 | |
| *** armax_ is now known as armax | 20:46 | |
| *** thorst_ has quit IRC | 20:46 | |
| notmorgan | timrc: dude... that would be awesome | 20:46 |
| notmorgan | except... who knows who produced it, it was something that was relabled after being found in an estate | 20:47 |
| *** TheodoreBe has joined #openstack-infra | 20:47 | |
| notmorgan | it was a cool bottle though =/ | 20:47 |
| clarkb | notmorgan: often times bottle bombs are a result of subsequent fermentation | 20:48 |
| notmorgan | clarkb: it might have been | 20:48 |
| *** esberglu has joined #openstack-infra | 20:48 | |
| *** thorst_ has joined #openstack-infra | 20:48 | |
| notmorgan | clarkb: or flawed glass. | 20:48 |
| *** armax_ has joined #openstack-infra | 20:48 | |
| jeblair | or carelessness with your sword | 20:49 |
| *** jsavak has quit IRC | 20:49 | |
| notmorgan | jeblair: if only i had a sword! | 20:49 |
| clarkb | jeblair: looks like we have the up to date nodepool installed so its ready for restart. I will do that now unless you have a reason to wait | 20:49 |
| notmorgan | jeblair: i'd totally take the blame for that :) | 20:49 |
| *** armax has quit IRC | 20:49 | |
| *** armax_ is now known as armax | 20:49 | |
| phschwartz | notmorgan: hey at least it wasn't your loop again. lol | 20:49 |
| jeblair | clarkb: go for it | 20:49 |
| *** dimtruck is now known as zz_dimtruck | 20:49 | |
| notmorgan | phschwartz: you know what the fault with open frame cases is? ... when champagne explodes... it gets into the case | 20:49 |
| clarkb | jeblair: done | 20:50 |
| phschwartz | notmorgan: yup, that is why I don't use my wet test bench any more. lol | 20:50 |
| timrc | notmorgan: Champagne-cooled hardware. | 20:50 |
| notmorgan | timrc: i totally want to do that now | 20:50 |
| *** dprince has quit IRC | 20:50 | |
| phschwartz | notmorgan: There have been people to do it. I saw a guy testing his loop with gatoraide and milk not too long ago. Gotta love youtube. | 20:50 |
| notmorgan | timrc: maybe cram a computer into one of the Jeroboam sized bottle | 20:51 |
| *** gouthamr has quit IRC | 20:51 | |
| notmorgan | timrc: then again, would be easier to convert a keg [and if done right, could even still have beer in it, with plexi to see beer+computer] | 20:51 |
| phschwartz | so there is an issue that is concerning, the ubuntu forums were hacked and looks like the hacker got 2m users/creds off of the Ubuntu one auth system | 20:52 |
| timrc | notmorgan: Hey just pot a rasberry pi in silicone and drop it in :) | 20:52 |
| timrc | What could go wrong? | 20:52 |
| notmorgan | timrc: but... that is less "cool" | 20:52 |
| phschwartz | notmorgan: NO NO NO NO, repeat after me. I WILL NOT WARM BEER!!! | 20:52 |
| *** thorst_ has quit IRC | 20:52 | |
| notmorgan | phschwartz: same would go for sparkling wine | 20:53 |
| clarkb | phschwartz: have a link for that? | 20:53 |
| notmorgan | phschwartz: but.. in a Keg you have a LOT of liquid to do thermal transfer... | 20:53 |
| phschwartz | I can go back and grab it. It was on the front page of tomshardware and was on slashdot | 20:54 |
| notmorgan | clarkb: i was hesitant to ask cause... i dunnoe... does my computer want electrolytes? | 20:54 |
| phschwartz | clarkb: ^^ | 20:54 |
| *** cbader has joined #openstack-infra | 20:54 | |
| clarkb | https://insights.ubuntu.com/2016/07/15/notice-of-security-breach-on-ubuntu-forums/ | 20:54 |
| notmorgan | clarkb: https://www.youtube.com/watch?v=sS6uSk8hfxE | 20:54 |
| phschwartz | clarkb: also looking at it again, they mis spoken when they said creds. They didn't get passwords or ssh keys | 20:55 |
| clarkb | jeblair: http://paste.openstack.org/show/533101/ we may need to revert the demand calculations if we can't get that sorted | 20:55 |
| phschwartz | notmorgan: yeah, but still. I don't mind heating damn sparkling wine, but bear is a party foul | 20:55 |
| clarkb | phschwartz: ya thats why I asked for source :) devil is in the details with these things | 20:55 |
| phschwartz | clarkb: yup | 20:55 |
| jeblair | clarkb: ack | 20:56 |
| notmorgan | phschwartz: depends on how cold your beer is. if you're drinking crappy enough beer that it needs to be cold (frigid) to enjoy... | 20:56 |
| phschwartz | notmorgan: I like all my beer at no more then 33f!!! | 20:56 |
| *** Goneri has joined #openstack-infra | 20:56 | |
| phschwartz | I don't go for the room temp chimay craft crap | 20:57 |
| clarkb | jeblair: hrm looks like gearman server disconnected too | 20:57 |
| notmorgan | phschwartz: LOL H24 Orange Wheat Beer in that video. | 20:57 |
| jeblair | clarkb: i think we have nodepool disconnect/reconnect on that timeout | 20:57 |
| clarkb | jeblair: gotcha | 20:58 |
| clarkb | jeblair: I did check and its still running on zuul so didn't crash | 20:58 |
| *** edmondsw has quit IRC | 20:58 | |
| phschwartz | notmorgan: and that vid you posted is the one I was talking about | 20:58 |
| notmorgan | phschwartz: yah | 20:58 |
| clarkb | phschwartz: ya looking at this its not great but its also not world shattering | 20:59 |
| phschwartz | clarkb: yeah, without passwords and keys obtained, it is meh. Yay for spam | 20:59 |
| *** asettle has joined #openstack-infra | 20:59 | |
| jeblair | clarkb: when did you restart? | 21:01 |
| jeblair | 20:49:48? | 21:01 |
| clarkb | jeblair: right around then yes | 21:01 |
| clarkb | I said "done" at 20:50:00 according to my irc logs | 21:01 |
| clarkb | process timestamp for nodepoold matches that too | 21:02 |
| jeblair | clarkb: a 'workers' command i manually issue takes 15 seconds... that should be ok. | 21:02 |
| *** rfolco has quit IRC | 21:02 | |
| clarkb | jeblair: perhaps the nodepoold isn't giving the gear thread enough cpu cycles to fully fetch the data? | 21:02 |
| *** edmondsw has joined #openstack-infra | 21:02 | |
| *** edmondsw has quit IRC | 21:02 | |
| jeblair | clarkb: could be :| | 21:03 |
| *** edmondsw has joined #openstack-infra | 21:03 | |
| *** sdake has joined #openstack-infra | 21:03 | |
| *** edmondsw has quit IRC | 21:04 | |
| *** ddieterly is now known as ddieterly[away] | 21:04 | |
| *** Goneri has quit IRC | 21:04 | |
| *** jsavak has joined #openstack-infra | 21:04 | |
| jeblair | clarkb: i'm timing it again using gear to do it | 21:05 |
| clarkb | jeblair: ok | 21:05 |
| clarkb | pabelanger: I am also noticing that tripleo's rh1 cloud seems to be unworking | 21:05 |
| *** amitgandhinz has joined #openstack-infra | 21:05 | |
| clarkb | pabelanger: do you know if it should be up? | 21:05 |
| *** Goneri has joined #openstack-infra | 21:08 | |
| *** thorst has joined #openstack-infra | 21:08 | |
| jeblair | clarkb: it hasn't returned yet. i think we should manually install the previous commit and restart while i debug | 21:08 |
| clarkb | jeblair: ok I can work on doing that. WIll start by disabling puppet/ansible with the mergency file | 21:09 |
| *** ddieterly[away] is now known as ddieterly | 21:09 | |
| *** karthikp_ has quit IRC | 21:09 | |
| clarkb | thats done, will checkout and install older version then restart now | 21:09 |
| clarkb | jeblair: or would you prefer I do a local revert to keep the other changes you wanted? | 21:10 |
| *** rajinir has joined #openstack-infra | 21:10 | |
| jeblair | clarkb: doesn't matter | 21:10 |
| *** mdrabe_ has joined #openstack-infra | 21:12 | |
| clarkb | ok done | 21:12 |
| *** thorst has quit IRC | 21:12 | |
| clarkb | I went with local revert | 21:12 |
| *** cody-somerville has quit IRC | 21:13 | |
| *** esberglu has quit IRC | 21:13 | |
| *** cody-somerville has joined #openstack-infra | 21:13 | |
| *** cody-somerville has joined #openstack-infra | 21:13 | |
| *** mdrabe has quit IRC | 21:15 | |
| clarkb | still no building instances, I wonder if the rh1 situation is preventing other things from happening | 21:16 |
| *** roxanagh_ has joined #openstack-infra | 21:16 | |
| *** ldnunes has quit IRC | 21:16 | |
| clarkb | I am going to comment out rh1 too | 21:16 |
| jeblair | i found something interesting.... | 21:17 |
| jeblair | geard is continually trying to send data to fd 84 but getting EAGAIN -- fd 84 is in close_wait | 21:17 |
| jeblair | (fd 84 is to nodepool.o.o) | 21:18 |
| clarkb | jeblair: and nodepool is claiming that it isn't connected to a gearman server right now | 21:19 |
| *** yaume has quit IRC | 21:20 | |
| *** gyee has joined #openstack-infra | 21:20 | |
| jeblair | clarkb: yeah, it's probably either the old process or one of my aborted 'workers' calls | 21:20 |
| *** pradk has quit IRC | 21:21 | |
| *** amitgandhinz has quit IRC | 21:21 | |
| *** snarwade has quit IRC | 21:21 | |
| *** trown is now known as trown|outtypewww | 21:21 | |
| *** amitgandhinz has joined #openstack-infra | 21:21 | |
| *** EricGonc_ has joined #openstack-infra | 21:22 | |
| *** flepied has quit IRC | 21:22 | |
| *** armax has quit IRC | 21:23 | |
| *** armax has joined #openstack-infra | 21:23 | |
| *** tonytan4ever has joined #openstack-infra | 21:23 | |
| *** amitgandhinz has quit IRC | 21:24 | |
| *** EricGonczer_ has quit IRC | 21:25 | |
| *** edtubill has quit IRC | 21:25 | |
| *** asettle has quit IRC | 21:25 | |
| *** nixpenguin has joined #openstack-infra | 21:26 | |
| jeblair | clarkb: i think geard may be out of commission because of this | 21:26 |
| clarkb | :( | 21:26 |
| clarkb | do we save queues and restart it? | 21:26 |
| jeblair | clarkb: yeah, i think so. i will do that. | 21:26 |
| clarkb | ok | 21:26 |
| jeblair | i also think i know how to fix it. but first -- restart zuul/gear, then see if i can repro, then fix gear, then try new nodepool again. | 21:27 |
| clarkb | sounds good | 21:27 |
| fungi | phschwartz: i would leave the separate spec separate. we've had poor luck in the past with specs whose scopes increase without bound. easier to be able to mark the original off as done and have the separate addition spec outstanding since they may be prioritized differently | 21:27 |
| *** tonytan4ever has quit IRC | 21:28 | |
| jeblair | restarting zuul now | 21:28 |
| *** clenimar_ has joined #openstack-infra | 21:29 | |
| fungi | clarkb: champagne and similar sparkling wines are by nature a result of subsequent fermentation... but exploding ones have miscalculated the specific gravity at bottling time | 21:29 |
| *** esberglu has joined #openstack-infra | 21:29 | |
| *** seldenr has joined #openstack-infra | 21:29 | |
| clarkb | fungi: yes they fermentin the bottle but in addition to miscalculating the gravity if you get bacteria in there then it can continue to chew on other parts of the liquid and make more co2 | 21:30 |
| *** piet has joined #openstack-infra | 21:30 | |
| phschwartz | fungi: ok, sounds good. I will push out the final changes tonight to the current spec | 21:30 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/devstack-gate: Configure br-ex MTU using PUBLIC_DEVICE_MTU variable https://review.openstack.org/343024 | 21:33 |
| jeblair | restarted and re-enqueued | 21:33 |
| clarkb | nodepool is looking more happy but was mad at me for unconfiguring rh1 so it has been put back. Its doing stuff now | 21:34 |
| *** sarob has joined #openstack-infra | 21:34 | |
| clarkb | there is one new building host so tahts good | 21:35 |
| clarkb | I expected more but its a start | 21:35 |
| fungi | our air conditioning seems to be able to maintain roughly a 11c/19f differential from outside. which is great except when it's 38c/101f outside with 100% relative humidity and the indoor humidity won't drop below 70% | 21:36 |
| *** tkelsey has joined #openstack-infra | 21:36 | |
| timrc | fungi: That's some TX weather right there. | 21:37 |
| phschwartz | timrc: sounds like south fl weather | 21:37 |
| timrc | fungi: I am truly surprised people lived here before AC was a thing. | 21:37 |
| fungi | timrc: phschwartz: tell me about it. my own little slice of fl/tx right here | 21:37 |
| phschwartz | fungi: lol | 21:37 |
| timrc | I guess houses were designed better for that sort of weather and front porches were a thing. Still | 21:37 |
| phschwartz | fungi: My son never wants to come inside and it has had a heat index ranging from 106f-111f here the past 2 weeks | 21:38 |
| phschwartz | fungi: not sure how kids do it | 21:38 |
| *** sarob has quit IRC | 21:38 | |
| *** csomerville has joined #openstack-infra | 21:39 | |
| clarkb | jeblair: looking at our demand calcs I wonder if we need to cleanup some of these older used instances | 21:39 |
| clarkb | jeblair: do you know if zuul launcher will properly relinquish those in the event of a gearman server restart or loss of connection? | 21:39 |
| *** tkelsey has quit IRC | 21:41 | |
| jeblair | clarkb: in a gearman server restart, it will (correctly, i think) not relinquish them. however, since we had a nonfunctioning geard for a while, there may be errors | 21:41 |
| jeblair | clarkb: (once a node is assigned, it's assigned until revoked or used) | 21:41 |
| *** timello has quit IRC | 21:41 | |
| clarkb | jeblair: ok, that means any hour old used servers can now be deleted because those jobs are all restarted at this point right? | 21:41 |
| jeblair | clarkb: (but if nodepool revoked nodes without geard working, launchers would not have seen that) | 21:41 |
| clarkb | I will work on marking those delete | 21:42 |
| jeblair | clarkb: yes | 21:42 |
| *** cody-somerville has quit IRC | 21:42 | |
| *** timello has joined #openstack-infra | 21:42 | |
| jeblair | clarkb: oh, it is possible that zuul-launcher does not abort jobs on gearman disconnect.. maybe that's what you were asking | 21:42 |
| *** sarob has joined #openstack-infra | 21:42 | |
| jeblair | (which it probably should since there's no way the result will make it back) | 21:43 |
| clarkb | that was part of what i was askign too but since it doesn't report killing the instances under them won't affect results | 21:44 |
| clarkb | so I am going ahead and remove all of the used isntances that are older than ~10 minutes | 21:44 |
| jeblair | clarkb: woo! i am able to reproduce | 21:45 |
| clarkb | jeblair: fun fun | 21:45 |
| jeblair | clarkb: (with the help of an iptables rule added in the middle of a large data transfer) | 21:45 |
| clarkb | and now we have a large amount of building instances. I think we are back to normal again | 21:46 |
| *** flepied has joined #openstack-infra | 21:46 | |
| *** julim has quit IRC | 21:48 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Add userstory-dashboard repos https://review.openstack.org/343095 | 21:48 |
| *** ddieterly is now known as ddieterly[away] | 21:48 | |
| clarkb | we should figure out what tripleo's cloud situation is | 21:49 |
| clarkb | but other than that things look happy to me again | 21:49 |
| abregman | rcarrillocruz: thanks! now I need reviews :) | 21:50 |
| *** ddieterly[away] is now known as ddieterly | 21:51 | |
| *** csomerville has quit IRC | 21:52 | |
| *** Goneri has quit IRC | 21:52 | |
| *** cody-somerville has joined #openstack-infra | 21:53 | |
| *** cody-somerville has joined #openstack-infra | 21:53 | |
| *** gentletheme has joined #openstack-infra | 21:55 | |
| openstackgerrit | James E. Blair proposed openstack-infra/gear: Geard: Handle connections closed while sending https://review.openstack.org/343099 | 21:58 |
| jeblair | clarkb: i think that's the fix ^ | 21:58 |
| clarkb | looking | 21:59 |
| jeblair | clarkb: make sure you see my inline comment i just added | 22:00 |
| *** Goneri has joined #openstack-infra | 22:00 | |
| clarkb | ok | 22:00 |
| *** gentletheme has quit IRC | 22:01 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:01 | |
| *** mdrabe_ has quit IRC | 22:02 | |
| clarkb | jeblair: rather than tightly looping trying to send around eagains this bubbles up to main loop which in a non closed connection will find out when ti can write again and try again. But in the case of a clsoed connection will just stop it | 22:03 |
| clarkb | seems to make sense | 22:04 |
| jeblair | clarkb: yeah. the previous behavior was an infinite loop as eagain is suppressed and the data pushed back onto the stack | 22:04 |
| clarkb | jeblair: this won't fix nodepools ability to read witin its timeout but will keep the server alive right? | 22:04 |
| *** mriedem has quit IRC | 22:04 | |
| jeblair | clarkb: yep. i don't know when the infinite loop started, but it may have affected my attempt to find out how long it should take, and if it started early enough, possibly nodepool itself. | 22:05 |
| jeblair | clarkb: we *could* try the workers command again from nodepool, but i am slightly hesitant to do so without this patch running on geard. | 22:06 |
| clarkb | jeblair: ya I can see that | 22:06 |
| jeblair | clarkb: should we go ahead and land that and restart zuul? | 22:06 |
| clarkb | jeblair: up to you. I will be around for another couple hours so can help | 22:06 |
| *** rhallisey has quit IRC | 22:06 | |
| jeblair | ok. i will land that patch, install gear from source, restart zuul. if it looks good, i will tag gear. | 22:07 |
| clarkb | ok | 22:07 |
| *** Goneri has quit IRC | 22:08 | |
| fungi | nibalizer: hurgleburgler: are there any plans to use the vinz-webclient repo, or should we formally retire that? | 22:09 |
| fungi | http://git.openstack.org/cgit/openstack-infra/vinz-webclient | 22:09 |
| hurgleburgler | fungi: don't think we plan on using it | 22:09 |
| *** abregman has quit IRC | 22:09 | |
| hurgleburgler | it can probably be retired | 22:10 |
| *** abregman has joined #openstack-infra | 22:10 | |
| fungi | hurgleburgler: cool, i won't propose it to governance and will just stick to vinz itself then. thanks! | 22:10 |
| hurgleburgler | np! | 22:10 |
| *** eezhova has quit IRC | 22:12 | |
| *** cody-somerville has quit IRC | 22:12 | |
| *** cody-somerville has joined #openstack-infra | 22:12 | |
| fungi | hurgleburgler: nibalizer: https://review.openstack.org/343101 "Add vinz as an Infra deliverable" | 22:12 |
| fungi | thanks for hacking on that! | 22:13 |
| *** eezhova has joined #openstack-infra | 22:13 | |
| hurgleburgler | woo hoo! | 22:13 |
| *** clenimar_ has quit IRC | 22:14 | |
| *** ramishra has quit IRC | 22:14 | |
| *** bknudson_ has left #openstack-infra | 22:15 | |
| *** ramishra has joined #openstack-infra | 22:16 | |
| *** seldenr has quit IRC | 22:16 | |
| *** clenimar_ has joined #openstack-infra | 22:16 | |
| *** tonytan4ever has joined #openstack-infra | 22:17 | |
| *** abregman has quit IRC | 22:18 | |
| *** raildo has quit IRC | 22:19 | |
| *** sdake has quit IRC | 22:19 | |
| openstackgerrit | Merged openstack-infra/gear: Geard: Handle connections closed while sending https://review.openstack.org/343099 | 22:20 |
| jeblair | restarting zuul | 22:21 |
| *** vjamezcu has quit IRC | 22:23 | |
| jeblair | deleting used nodes | 22:24 |
| *** amitgandhinz has joined #openstack-infra | 22:25 | |
| clarkb | jeblair: are you wanting to try the demand calculation change again? or wait for next week to sort that out? | 22:27 |
| jeblair | clarkb: i just ran a manual test -- it took 1m for gear to get the worker list | 22:27 |
| jeblair | that's not great, but it should work | 22:28 |
| jeblair | (should probably use cstringio or something for that) | 22:29 |
| *** ddieterly is now known as ddieterly[away] | 22:29 | |
| *** amitgandhinz has quit IRC | 22:30 | |
| jeblair | clarkb: so i say we give it another try | 22:30 |
| clarkb | ok | 22:31 |
| clarkb | I will check master back out, reinstall then restart the daemon | 22:31 |
| jeblair | clarkb: sounds good | 22:31 |
| *** ddieterly[away] has quit IRC | 22:32 | |
| clarkb | thats done | 22:32 |
| *** eezhova has quit IRC | 22:33 | |
| *** armax has quit IRC | 22:34 | |
| *** rbergeron has quit IRC | 22:34 | |
| jeblair | i'm watching geard strace | 22:34 |
| *** eezhova has joined #openstack-infra | 22:35 | |
| jeblair | i can see it sending the megadata to nodepool, but also actually handling other requests | 22:35 |
| jeblair | that could end up helping zuul a lot | 22:35 |
| clarkb | I am watching nodepool debug log, haven't seen it report or fail yet on demand | 22:35 |
| openstackgerrit | Kevin Benton proposed openstack-infra/devstack-gate: Set Q_USE_PROVIDERNET_FOR_PUBLIC=True for Neutron https://review.openstack.org/343104 | 22:35 |
| pleia2 | aww, yesterday was the five year birthday of system-config :) http://git.openstack.org/cgit/openstack-infra/system-config/commit/?id=5d85125bf7739a38c788256ccfba7877b46d2132 | 22:35 |
| *** ramishra has quit IRC | 22:35 | |
| pleia2 | (I'm writing an article about our use of revision control) | 22:35 |
| *** nixpenguin has quit IRC | 22:36 | |
| *** ramishra has joined #openstack-infra | 22:37 | |
| jeblair | cake for everyone! | 22:37 |
| mordred | yay cake! | 22:37 |
| clarkb | pleia2: but we called it something else back then | 22:38 |
| clarkb | I don't even remember anymore | 22:38 |
| clarkb | jeblair: it timed out | 22:38 |
| mgagne | clarkb: just openstack-infra/config? =) | 22:38 |
| jeblair | clarkb: i think we're into the territory of your cpu starvation theory now | 22:38 |
| mgagne | oh damn, 2011 :O | 22:39 |
| *** tkelsey has joined #openstack-infra | 22:39 | |
| clarkb | jeblair: should I go ahead and reinstall my revert and restart the service? | 22:39 |
| clarkb | we might let it go a few more cycles to see if it manages to work occasionally but that seems unlikely | 22:40 |
| *** edmondsw has joined #openstack-infra | 22:40 | |
| jeblair | clarkb: let's give it another chance | 22:40 |
| jeblair | clarkb: right that | 22:40 |
| *** rbergeron has joined #openstack-infra | 22:40 | |
| *** sdake has joined #openstack-infra | 22:40 | |
| clarkb | jeblair: its complaining bout no connected gearman servers which is what it did last time too | 22:42 |
| openstackgerrit | Alex Schultz proposed openstack-infra/project-config: Add LANG for puppet-cookiecutter https://review.openstack.org/343106 | 22:42 |
| *** tkelsey has quit IRC | 22:44 | |
| jeblair | clarkb: yeah, it should reconnect | 22:44 |
| jeblair | and i think it just did | 22:44 |
| *** rlandy has quit IRC | 22:44 | |
| *** jsavak has quit IRC | 22:45 | |
| *** edmondsw has quit IRC | 22:45 | |
| *** sdake has quit IRC | 22:46 | |
| *** EricGonc_ has quit IRC | 22:47 | |
| mgagne | openstack/openstack-ci-puppet and openstack/openstack-ci maybe? found those names in a presentation | 22:47 |
| *** tpsilva has quit IRC | 22:48 | |
| clarkb | mgagne: infra-config maybe | 22:48 |
| mgagne | depends on how far back you want to go I guess | 22:48 |
| pleia2 | it was config when I started | 22:48 |
| mgagne | http://eavesdrop.openstack.org/meetings/openstack-meeting/2011/openstack-meeting.2011-07-26-19.04.log.html | 22:48 |
| clarkb | pleia2: was it just config? then that | 22:49 |
| mgagne | found this meeting where repo was moved to github | 22:49 |
| clarkb | jeblair: it timed out again | 22:50 |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config: Update citycloud to list new regions https://review.openstack.org/339990 | 22:50 |
| mordred | GheRivero, Shrews: ^^ updated commit message to address concerns | 22:51 |
| *** cryptarium has joined #openstack-infra | 22:51 | |
| mordred | mgagne: reading old meeting logs is pretty amazing | 22:51 |
| jeblair | clarkb: okay, want to pin again while i try to make this less awful? | 22:52 |
| clarkb | jeblair: sure will do that now | 22:52 |
| *** armax has joined #openstack-infra | 22:53 | |
| clarkb | and done | 22:53 |
| *** Teresita-Warrior has quit IRC | 22:54 | |
| pabelanger | clarkb: rh1 is still moving IIRC | 22:56 |
| clarkb | pabelanger: ok well we have it enabled in our config and we can't connect to its keystone | 22:57 |
| clarkb | pabelanger: seems to not actually be causing issues | 22:57 |
| clarkb | but want to make usre its not something we need to debug further | 22:57 |
| clarkb | jeblair: allocations and demand calculation look good so we didn't kill the server this time around | 22:57 |
| pabelanger | clarkb: okay, if it is affecting us, we should remove it from configuration. Its been online for a few weeks | 22:58 |
| *** eharney has quit IRC | 22:58 | |
| *** rbrndt has quit IRC | 22:59 | |
| clarkb | pabelanger: I thought it might have been related to nodepool not getting demand but we tracked that down to an unhappy gearman server. So now I think its just noise in logs | 23:00 |
| pabelanger | clarkb: okay | 23:01 |
| *** sdake has joined #openstack-infra | 23:02 | |
| *** armax has quit IRC | 23:02 | |
| *** esberglu has quit IRC | 23:03 | |
| Shrews | mordred: does citycloud have BOTH a v1 and v2 endpoint? if both are available, changing the default could break folks depending on v1 behavior | 23:06 |
| *** armax has joined #openstack-infra | 23:06 | |
| *** gongysh has joined #openstack-infra | 23:11 | |
| *** clenimar_ has quit IRC | 23:11 | |
| *** mriedem has joined #openstack-infra | 23:13 | |
| mordred | Shrews: it does - hrm, that's a good point. shade users should be fine, but yeah, I suppose if other people are consuming that api version that could be iffy | 23:13 |
| mordred | how about I pull that bit out for now | 23:15 |
| Shrews | mordred: ++ | 23:16 |
| *** armax_ has joined #openstack-infra | 23:18 | |
| *** armax has quit IRC | 23:19 | |
| *** armax_ is now known as armax | 23:19 | |
| *** xyang1 has quit IRC | 23:21 | |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config: Update citycloud to list new regions https://review.openstack.org/339990 | 23:21 |
| *** sdake has quit IRC | 23:22 | |
| *** mdrabe has joined #openstack-infra | 23:23 | |
| *** clenimar_ has joined #openstack-infra | 23:23 | |
| *** berendt has joined #openstack-infra | 23:24 | |
| *** mdrabe_ has joined #openstack-infra | 23:24 | |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config: Add the new DreamCompute cloud https://review.openstack.org/339989 | 23:24 |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config: Add support for listing a cloud as shut down https://review.openstack.org/343112 | 23:25 |
| *** mtanino has quit IRC | 23:25 | |
| *** amitgandhinz has joined #openstack-infra | 23:26 | |
| jeblair | clarkb: it looks like geard can send the whole thing in a few seconds, but the client is really slow at processing large admin requests | 23:28 |
| jeblair | it's going to need some more thought | 23:28 |
| *** mdrabe has quit IRC | 23:28 | |
| *** ddieterly has joined #openstack-infra | 23:28 | |
| clarkb | ok | 23:28 |
| clarkb | we can get to it nxt week | 23:28 |
| clarkb | overall I am pretty happy with the progress made this week. No major issues job side we just need to run them on xenial now | 23:29 |
| *** mdrabe__ has joined #openstack-infra | 23:30 | |
| *** rcernin has quit IRC | 23:30 | |
| *** amitgandhinz has quit IRC | 23:31 | |
| *** ihrachys has quit IRC | 23:31 | |
| *** mdrabe_ has quit IRC | 23:33 | |
| *** sarob has quit IRC | 23:34 | |
| jeblair | ++ | 23:34 |
| *** tonytan4ever has quit IRC | 23:35 | |
| *** mdrabe__ has quit IRC | 23:35 | |
| *** Apoorva_ has joined #openstack-infra | 23:39 | |
| *** tkelsey has joined #openstack-infra | 23:41 | |
| *** Apoorva has quit IRC | 23:42 | |
| *** Apoorva_ has quit IRC | 23:43 | |
| *** tkelsey has quit IRC | 23:45 | |
| *** xarses has quit IRC | 23:46 | |
| *** gyee has quit IRC | 23:49 | |
| *** piet has quit IRC | 23:52 | |
| *** pvaneck has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
