| *** mmalik4 has quit IRC | 00:01 | |
| fungi | yay gerrit daily database backup time | 00:01 |
|---|---|---|
| *** esikachev has joined #openstack-infra | 00:01 | |
| logan- | cool. will take a look at that. | 00:03 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Allow disto-specific mirror settings https://review.openstack.org/408850 | 00:04 |
| *** wolverineav has quit IRC | 00:05 | |
| *** ihrachys has quit IRC | 00:05 | |
| *** esikachev has quit IRC | 00:05 | |
| logan- | seems like I could use zuul-cloner to sync a local mirror of all the repos | 00:05 |
| *** saibarspeis has quit IRC | 00:06 | |
| *** jamielennox|away is now known as jamielennox | 00:08 | |
| *** piet has joined #openstack-infra | 00:12 | |
| *** askb has joined #openstack-infra | 00:14 | |
| *** spzala has joined #openstack-infra | 00:15 | |
| *** esikachev has joined #openstack-infra | 00:15 | |
| *** esikachev has quit IRC | 00:19 | |
| *** priteau has quit IRC | 00:21 | |
| *** yamahata has joined #openstack-infra | 00:22 | |
| *** ggillies has quit IRC | 00:25 | |
| *** ggillies has joined #openstack-infra | 00:25 | |
| *** adrian_otto has joined #openstack-infra | 00:29 | |
| armax | hello | 00:29 |
| armax | I wonder if patch https://review.openstack.org/#/c/408020/ caused a bit of havoc in the neutron gate | 00:30 |
| armax | I see that rally jobs no longer deploys neutron | 00:30 |
| armax | this is a recent run on master | 00:30 |
| armax | http://logs.openstack.org/25/408825/1/check/gate-rally-dsvm-neutron-neutron-ubuntu-xenial/2d1aa00/ | 00:30 |
| armax | and I see the job fail on stable branches too | 00:30 |
| armax | would anyone have any insight? | 00:31 |
| armax | thanks | 00:31 |
| openstackgerrit | xiaodongwang991481 proposed openstack-infra/project-config: change networking-sfc tempest and functional tests to voting https://review.openstack.org/406396 | 00:32 |
| clarkb | looks like the lack of DEVSTACK_GATE_NEUTRON did brrak it afterall | 00:32 |
| *** baoli has joined #openstack-infra | 00:32 | |
| *** harlowja_ has quit IRC | 00:32 | |
| clarkb | armax: its a rally bug, their config was weird and it was called out at review time but yhey were insisting it would fix stuff | 00:32 |
| fungi | yeah, we were mostly taking the rally devs at their word on that one | 00:33 |
| armax | clarkb: I am not sure about the timing though | 00:33 |
| armax | I see a successful run for rully a couple of hours ago | 00:33 |
| armax | http://logs.openstack.org/52/400852/4/check/gate-rally-dsvm-neutron-neutron-ubuntu-xenial/4b96cef/console.html | 00:33 |
| armax | so it might not be it | 00:33 |
| *** Julien-zte has joined #openstack-infra | 00:34 | |
| fungi | that change probably took effect ~4 hours ago | 00:34 |
| armax | but what I can certainly see now, is that the run nova-net instead of neutron | 00:34 |
| armax | fungi: a | 00:34 |
| armax | h | 00:34 |
| armax | ok that would explain it | 00:35 |
| fungi | should be able to look at the zuul log data collected alongside the rest of those logs to see if it matches the current job definition | 00:35 |
| fungi | oh, wait, not 408020 but the other change which just went in earlier today | 00:35 |
| fungi | digging up the right change now | 00:35 |
| fungi | no, i guess that was the change after all. was that really a day ago? | 00:37 |
| fungi | this week has turned into a blur | 00:37 |
| *** maeker has quit IRC | 00:37 | |
| *** adrian_otto has quit IRC | 00:37 | |
| *** spzala has quit IRC | 00:38 | |
| *** maeker has joined #openstack-infra | 00:38 | |
| *** baoli has quit IRC | 00:38 | |
| armax | from what I can tell | 00:39 |
| armax | the job gate-rally-dsvm-neutron-rally-ubuntu-xenial runs fine | 00:39 |
| armax | but what’s broken is the one called gate-rally-dsvm-neutron-neutron-ubuntu-xenial and its trusty variant | 00:39 |
| *** dkehn has quit IRC | 00:40 | |
| *** dkehn has joined #openstack-infra | 00:40 | |
| *** adrian_otto has joined #openstack-infra | 00:40 | |
| fungi | armax: http://logs.openstack.org/52/400852/4/check/gate-rally-dsvm-neutron-neutron-ubuntu-xenial/4b96cef/_zuul_ansible/scripts/03-dd6bcdc783584c6ea75208a2c7735827.sh from your passing job example looks like it matches the result of the https://review.openstack.org/408020 change | 00:41 |
| armax | fungi: so you concur that’s the smoking gun | 00:41 |
| armax | fungi: thanks for enlighten me on this btw | 00:42 |
| fungi | no, saying your example from earlier today passed with the change from 408020 in place | 00:42 |
| armax | oh | 00:42 |
| armax | but I look at ENABLED_SERVICES | 00:43 |
| armax | and I see no neutron in there | 00:43 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Replace swift capabilities call with REST https://review.openstack.org/408865 | 00:43 |
| fungi | double-checking to make sure i'm looking at the right builder definition in the job config. they have so many | 00:44 |
| *** Julien-zte has quit IRC | 00:44 | |
| *** Julien-zte has joined #openstack-infra | 00:45 | |
| fungi | the gate-rally-dsvm-neutron-{name}-{node}{job-suffix} template from that change would run their devstack-rally-gate builder, no longer passing neutron: '1' | 00:46 |
| fungi | and they dropped use of the $DEVSTACK_GATE_NEUTRON envvar from that builder, given that it's now a default assumption in devstack | 00:47 |
| armax | fungi: I think the change is incomplete | 00:47 |
| armax | by removing DEVSTACK_GATE_NEUTRON | 00:47 |
| armax | and by not adding the list of neutron services to ENABLED_SERVICES | 00:47 |
| armax | does devstack falls back to nova-net by default still? | 00:47 |
| fungi | good question. checking devstack to find out | 00:48 |
| * armax too | 00:48 | |
| clarkb | devstack gate should do it | 00:48 |
| clarkb | but that may only work on mastet | 00:48 |
| *** Goneri has joined #openstack-infra | 00:48 | |
| armax | clarkb: but if ENABLED_SERVICES is defined | 00:48 |
| armax | does d-g do anything? | 00:48 |
| armax | I would be surprised if it did | 00:48 |
| clarkb | oh if yhey define their own services it may not | 00:49 |
| fungi | the passing example ran against master at least | 00:49 |
| armax | https://github.com/openstack-dev/devstack/blob/master/stackrc#L52 | 00:49 |
| armax | I think if ENABLED_SERVICES is set | 00:49 |
| armax | devstack humbly honors what the user wants | 00:49 |
| *** jgriffith is now known as jgriffith_AutoAw | 00:49 | |
| armax | hence the snafu? | 00:50 |
| fungi | oh, indeed, their ENABLED_SERVICES list in that job is not purely additive | 00:50 |
| armax | if we don’t add https://github.com/openstack-dev/devstack/blob/master/stackrc#L62 | 00:50 |
| armax | to the rally definition | 00:50 |
| armax | though I am not clearly how nova-net went into localrc | 00:50 |
| *** harlowja has joined #openstack-infra | 00:51 | |
| fungi | i can't see any difference between the builder script zuul recorded for the passing and failing run, at least | 00:52 |
| fungi | oh, the failing example was stable/newton so would have used a different devstack branch | 00:53 |
| armax | I have a failed build on master too | 00:53 |
| fungi | did you have a failing example from a master branch handy? | 00:53 |
| armax | no wait | 00:54 |
| fungi | just want to be sure i'm making a direct comparison | 00:54 |
| armax | I am getting confused | 00:54 |
| armax | this might be broken on stable branches on | 00:54 |
| armax | so yeah, newton and mitaka for sure | 00:54 |
| fungi | if it's passing on master and broken on stable, then yes i can see where this is probably wrong | 00:54 |
| armax | which would explain why | 00:54 |
| clarkb | if its stable only then likely bcause it only defaults to neutron on master | 00:54 |
| armax | n-net sneaks back in | 00:54 |
| clarkb | so they need to hard set devstack gate neutron | 00:55 |
| fungi | agreed | 00:55 |
| armax | ok | 00:55 |
| armax | good | 00:55 |
| fungi | they probably didn't check this against a stable environment and failed to consider devstack's behavior changed between branches | 00:55 |
| armax | yeah | 00:55 |
| *** Julien-zte has quit IRC | 00:55 | |
| armax | that’s what happened | 00:55 |
| armax | so shall I tweak rally.yaml | 00:55 |
| armax | ? | 00:55 |
| *** Julien-zte has joined #openstack-infra | 00:56 | |
| armax | to add EVSTACK_GATE_NEUTRON if BRANCH is prior to masteR? | 00:56 |
| clarkb | sure you can just add that line back amd hard set it to 1 | 00:56 |
| armax | clarkb: that too | 00:56 |
| clarkb | its safe to set on master too | 00:56 |
| armax | clarkb: coming right up! | 00:56 |
| fungi | right, it's a no-op but that way it's consistent | 00:56 |
| fungi | and avoids messy branching logic | 00:57 |
| armax | um | 01:00 |
| *** Julien-zte has quit IRC | 01:00 | |
| armax | but what about their nova job? | 01:00 |
| *** Julien-zte has joined #openstack-infra | 01:00 | |
| armax | let me check something real quick | 01:00 |
| *** Julien-zte has joined #openstack-infra | 01:00 | |
| *** hurgleburgler has quit IRC | 01:02 | |
| *** jascott1 has quit IRC | 01:02 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config: Hard-code neutron setup for rally jobs https://review.openstack.org/408870 | 01:04 |
| armax | clarkb, fungi ^ please see if it makes sense | 01:04 |
| armax | and thanks for helping the troubleshooting process | 01:04 |
| *** Julien-zte has quit IRC | 01:04 | |
| *** jascott1 has joined #openstack-infra | 01:04 | |
| *** jamielennox is now known as jamielennox|away | 01:04 | |
| *** Julien-zte has joined #openstack-infra | 01:05 | |
| *** Julien-zte has quit IRC | 01:06 | |
| *** spzala has joined #openstack-infra | 01:07 | |
| *** Julien-zte has joined #openstack-infra | 01:07 | |
| *** Julien-zte has quit IRC | 01:07 | |
| *** Julien-zte has joined #openstack-infra | 01:08 | |
| *** Julien-zte has quit IRC | 01:08 | |
| *** Julien-zte has joined #openstack-infra | 01:09 | |
| *** Julien-zte has quit IRC | 01:10 | |
| *** esikachev has joined #openstack-infra | 01:11 | |
| openstackgerrit | dongwenjuan proposed openstack-infra/nodepool: add log input para to ssh_connect https://review.openstack.org/408873 | 01:11 |
| *** Julien-zte has joined #openstack-infra | 01:12 | |
| *** Julien-zte has joined #openstack-infra | 01:12 | |
| *** zhurong has joined #openstack-infra | 01:13 | |
| *** Julien-zte has quit IRC | 01:14 | |
| *** Julien-zte has joined #openstack-infra | 01:15 | |
| *** adrian_otto has quit IRC | 01:15 | |
| *** Julien-zte has quit IRC | 01:16 | |
| *** Julien-zte has joined #openstack-infra | 01:17 | |
| *** esikachev has quit IRC | 01:17 | |
| *** Julien-zte has quit IRC | 01:18 | |
| *** Julien-zte has joined #openstack-infra | 01:18 | |
| *** Julien-zte has quit IRC | 01:18 | |
| *** Julien-zte has joined #openstack-infra | 01:19 | |
| *** jamielennox|away is now known as jamielennox | 01:20 | |
| *** Julien-zte has quit IRC | 01:20 | |
| *** mmandel has joined #openstack-infra | 01:20 | |
| *** esikachev has joined #openstack-infra | 01:21 | |
| *** Julien-zte has joined #openstack-infra | 01:21 | |
| *** markvoelker has quit IRC | 01:22 | |
| *** Julien-zte has quit IRC | 01:23 | |
| *** Julien-zte has joined #openstack-infra | 01:24 | |
| *** mmandel has quit IRC | 01:25 | |
| *** esikachev has quit IRC | 01:25 | |
| *** Julien-zte has quit IRC | 01:26 | |
| *** Julien-zte has joined #openstack-infra | 01:27 | |
| *** Julien-zte has quit IRC | 01:27 | |
| *** markvoelker has joined #openstack-infra | 01:27 | |
| openstackgerrit | Merged openstack-infra/project-config: Hard-code neutron setup for rally jobs https://review.openstack.org/408870 | 01:29 |
| *** tqtran has quit IRC | 01:33 | |
| *** Julien-zte has joined #openstack-infra | 01:37 | |
| *** Julien-zte has quit IRC | 01:37 | |
| *** Julien-zte has joined #openstack-infra | 01:38 | |
| *** mtanino has quit IRC | 01:39 | |
| *** Sukhdev has quit IRC | 01:39 | |
| *** jamesdenton has quit IRC | 01:39 | |
| *** yanyanhu has joined #openstack-infra | 01:42 | |
| *** viks has quit IRC | 01:42 | |
| *** tphummel has quit IRC | 01:42 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add support for using infra's mirrors in tests https://review.openstack.org/408851 | 01:51 |
| *** esikachev has joined #openstack-infra | 02:21 | |
| *** john-davidge has joined #openstack-infra | 02:24 | |
| *** hurgleburgler has joined #openstack-infra | 02:26 | |
| *** esikachev has quit IRC | 02:27 | |
| *** john-davidge has quit IRC | 02:28 | |
| *** esikachev has joined #openstack-infra | 02:31 | |
| *** rbuzatu has joined #openstack-infra | 02:31 | |
| *** esberglu has joined #openstack-infra | 02:32 | |
| *** rbuzatu has quit IRC | 02:35 | |
| wznoinsk | hi, anyone still around could weigh in on https://review.openstack.org/#/c/407424/ ? | 02:36 |
| *** esikachev has quit IRC | 02:36 | |
| *** esberglu has quit IRC | 02:39 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add support for using infra's mirrors in tests https://review.openstack.org/408851 | 02:43 |
| *** sflanigan is now known as sflanigan_afk | 02:48 | |
| *** PramodJayathirth has quit IRC | 02:50 | |
| *** yuanying has quit IRC | 02:50 | |
| *** yamamot__ has joined #openstack-infra | 02:51 | |
| *** jascott1 has quit IRC | 02:55 | |
| *** harlowja has quit IRC | 02:55 | |
| *** jascott1 has joined #openstack-infra | 03:00 | |
| *** esikachev has joined #openstack-infra | 03:01 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add support for using infra's mirrors in tests https://review.openstack.org/408851 | 03:01 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add support for using infra's mirrors in tests https://review.openstack.org/408851 | 03:05 |
| ianw | hmm, no, that's not true ... http://logs.openstack.org/50/408850/3/check/gate-dib-dsvm-functests-ubuntu-xenial/cc96285/console.html was osic and worked | 03:05 |
| *** esikachev has quit IRC | 03:05 | |
| ianw | $ sudo dnf update | 03:06 |
| ianw | Error: Failed to synchronize cache for repo 'updates' | 03:06 |
| ianw | i just got that *locally* | 03:06 |
| *** thorst_ has joined #openstack-infra | 03:09 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add support for using infra's mirrors in tests https://review.openstack.org/408851 | 03:12 |
| *** esikachev has joined #openstack-infra | 03:13 | |
| *** Kiall has quit IRC | 03:18 | |
| *** Kiall has joined #openstack-infra | 03:20 | |
| *** esikachev has quit IRC | 03:22 | |
| *** mmandel has joined #openstack-infra | 03:22 | |
| *** gomarivera has joined #openstack-infra | 03:22 | |
| *** tuanluong has joined #openstack-infra | 03:25 | |
| *** maeker has quit IRC | 03:25 | |
| *** dave-mccowan has quit IRC | 03:25 | |
| *** mmandel has quit IRC | 03:26 | |
| *** gomarivera has quit IRC | 03:26 | |
| openstackgerrit | Eric K proposed openstack-infra/project-config: Enable heat plugin in congress dsvm jobs https://review.openstack.org/408930 | 03:27 |
| *** cschwede has joined #openstack-infra | 03:28 | |
| *** adarazs has joined #openstack-infra | 03:28 | |
| *** links has joined #openstack-infra | 03:29 | |
| *** slagle has joined #openstack-infra | 03:29 | |
| *** stevebaker has joined #openstack-infra | 03:30 | |
| *** Goneri has quit IRC | 03:31 | |
| *** megm has quit IRC | 03:40 | |
| *** furlongm has joined #openstack-infra | 03:40 | |
| *** sflanigan_afk is now known as sflanigan | 03:40 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Disable centos6 testing https://review.openstack.org/408936 | 03:41 |
| *** vikrant has joined #openstack-infra | 03:41 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add support for using infra's mirrors in tests https://review.openstack.org/408851 | 03:41 |
| *** megm has joined #openstack-infra | 03:43 | |
| *** fguillot has joined #openstack-infra | 03:44 | |
| *** knangia has quit IRC | 03:44 | |
| *** stakeda has joined #openstack-infra | 03:50 | |
| *** yuanying has joined #openstack-infra | 03:50 | |
| *** yuanying has quit IRC | 03:50 | |
| *** yuanying has joined #openstack-infra | 03:51 | |
| *** yamahata has quit IRC | 03:54 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add support for using infra's mirrors in tests https://review.openstack.org/408851 | 03:55 |
| *** tovin07 has quit IRC | 03:55 | |
| *** tovin07 has joined #openstack-infra | 03:56 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Remove yum chroot caching https://review.openstack.org/408942 | 03:56 |
| *** adriant has quit IRC | 03:56 | |
| *** claudiub has joined #openstack-infra | 03:57 | |
| *** maeker has joined #openstack-infra | 04:01 | |
| *** Julien-zte has quit IRC | 04:02 | |
| *** Julien-zte has joined #openstack-infra | 04:05 | |
| openstackgerrit | fumihiko kakuma proposed openstack-infra/project-config: Add scenario test for neutron-dynamic-routing https://review.openstack.org/407778 | 04:06 |
| *** viks has joined #openstack-infra | 04:06 | |
| *** nicolasbock has quit IRC | 04:08 | |
| *** esikachev has joined #openstack-infra | 04:08 | |
| *** Julien-zte has quit IRC | 04:09 | |
| *** esikachev has quit IRC | 04:12 | |
| *** drifterza has joined #openstack-infra | 04:13 | |
| *** mdrabe has quit IRC | 04:13 | |
| *** psachin has joined #openstack-infra | 04:14 | |
| *** fguillot has quit IRC | 04:15 | |
| *** claudiub has quit IRC | 04:19 | |
| *** ijw has quit IRC | 04:19 | |
| *** ijw has joined #openstack-infra | 04:20 | |
| *** rbuzatu has joined #openstack-infra | 04:21 | |
| *** askb has quit IRC | 04:21 | |
| *** esberglu has joined #openstack-infra | 04:21 | |
| *** spzala has quit IRC | 04:23 | |
| *** yuanying has quit IRC | 04:23 | |
| *** yuanying has joined #openstack-infra | 04:24 | |
| *** yuanying has quit IRC | 04:24 | |
| *** ijw has quit IRC | 04:24 | |
| *** yuanying has joined #openstack-infra | 04:25 | |
| *** yuanying has quit IRC | 04:25 | |
| *** jerryz has quit IRC | 04:25 | |
| *** rbuzatu has quit IRC | 04:26 | |
| *** esberglu has quit IRC | 04:26 | |
| *** maeker has quit IRC | 04:26 | |
| *** viks has quit IRC | 04:28 | |
| *** tovin07 has quit IRC | 04:31 | |
| *** tovin07 has joined #openstack-infra | 04:32 | |
| *** ijw has joined #openstack-infra | 04:35 | |
| openstackgerrit | Sampath Priyankara proposed openstack-infra/irc-meetings: Add Masakari team meeting https://review.openstack.org/408948 | 04:39 |
| *** ijw has quit IRC | 04:40 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add note about redundant file https://review.openstack.org/408949 | 04:42 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Don't try to execute _stop commands https://review.openstack.org/408950 | 04:42 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add command list into server.py https://review.openstack.org/408951 | 04:42 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add zmq back into launcher server https://review.openstack.org/408952 | 04:42 |
| *** snarwade has joined #openstack-infra | 04:44 | |
| *** tojuvone has joined #openstack-infra | 04:47 | |
| *** john-davidge has joined #openstack-infra | 04:51 | |
| *** mmalik4 has joined #openstack-infra | 04:52 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Remove yum chroot caching https://review.openstack.org/408942 | 04:53 |
| *** links has quit IRC | 04:54 | |
| *** john-davidge has quit IRC | 04:56 | |
| *** jamespage has quit IRC | 05:00 | |
| *** jamespag` has joined #openstack-infra | 05:01 | |
| *** hamzy has joined #openstack-infra | 05:05 | |
| *** ijw_ has joined #openstack-infra | 05:05 | |
| *** Julien-zte has joined #openstack-infra | 05:06 | |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config: Enable Py35-NV jobs for monasca https://review.openstack.org/398774 | 05:07 |
| *** links has joined #openstack-infra | 05:09 | |
| *** Julien-zte has quit IRC | 05:10 | |
| *** askb has joined #openstack-infra | 05:11 | |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Restrict nova-network jobs for ec2 and gce gating jobs https://review.openstack.org/406344 | 05:11 |
| *** sc68cal has quit IRC | 05:12 | |
| *** nadya has joined #openstack-infra | 05:12 | |
| *** nadya has quit IRC | 05:12 | |
| openstackgerrit | Andrey Pavlov proposed openstack-infra/project-config: Simplify template and its usage for gce and ec2 jobs https://review.openstack.org/408956 | 05:15 |
| *** esberglu has joined #openstack-infra | 05:16 | |
| *** furlongm has quit IRC | 05:20 | |
| *** esberglu has quit IRC | 05:21 | |
| *** esberglu has joined #openstack-infra | 05:21 | |
| *** mmandel has joined #openstack-infra | 05:22 | |
| *** sc68cal has joined #openstack-infra | 05:24 | |
| *** sdake has quit IRC | 05:24 | |
| *** esberglu has quit IRC | 05:25 | |
| *** yuanying has joined #openstack-infra | 05:26 | |
| *** mmandel has quit IRC | 05:26 | |
| *** nadya has joined #openstack-infra | 05:28 | |
| *** vikrant has quit IRC | 05:29 | |
| *** tqtran has joined #openstack-infra | 05:31 | |
| *** vikrant has joined #openstack-infra | 05:32 | |
| *** tqtran has quit IRC | 05:35 | |
| *** yuanying has quit IRC | 05:35 | |
| *** yuanying has joined #openstack-infra | 05:36 | |
| *** mmalik4 has quit IRC | 05:42 | |
| *** _ody has quit IRC | 05:42 | |
| *** yamahata has joined #openstack-infra | 05:45 | |
| *** _ody has joined #openstack-infra | 05:45 | |
| *** thorst_ has quit IRC | 05:45 | |
| *** thorst_ has joined #openstack-infra | 05:46 | |
| *** yuanying has quit IRC | 05:49 | |
| *** sandanar has joined #openstack-infra | 05:51 | |
| *** ramishra has quit IRC | 05:51 | |
| *** ramishra has joined #openstack-infra | 05:53 | |
| *** thorst_ has quit IRC | 05:54 | |
| *** gildub has quit IRC | 06:04 | |
| *** hurgleburgler has quit IRC | 06:10 | |
| *** esberglu has joined #openstack-infra | 06:11 | |
| *** nadya has quit IRC | 06:15 | |
| *** esberglu has quit IRC | 06:16 | |
| *** esberglu has joined #openstack-infra | 06:16 | |
| *** esberglu has quit IRC | 06:21 | |
| *** Julien-zte has joined #openstack-infra | 06:22 | |
| *** spzala has joined #openstack-infra | 06:24 | |
| *** esikachev has joined #openstack-infra | 06:24 | |
| *** maeker has joined #openstack-infra | 06:26 | |
| *** camunoz has quit IRC | 06:27 | |
| *** aeng has quit IRC | 06:27 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: networking-midonet: Add "full" variants for ml2 and v2 jobs https://review.openstack.org/408457 | 06:28 |
| *** yuanying has joined #openstack-infra | 06:28 | |
| *** spzala has quit IRC | 06:29 | |
| *** sdake has joined #openstack-infra | 06:29 | |
| *** maeker has quit IRC | 06:33 | |
| *** rbuzatu has joined #openstack-infra | 06:33 | |
| *** rbuzatu has quit IRC | 06:34 | |
| *** rbuzatu has joined #openstack-infra | 06:34 | |
| *** isaacb has joined #openstack-infra | 06:34 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add Masakari team meeting https://review.openstack.org/408948 | 06:36 |
| *** rbuzatu_ has joined #openstack-infra | 06:39 | |
| *** rbuzatu_ has quit IRC | 06:40 | |
| *** rbuzatu_ has joined #openstack-infra | 06:40 | |
| *** rbuzatu has quit IRC | 06:43 | |
| *** oanson has joined #openstack-infra | 06:45 | |
| *** isaacb has quit IRC | 06:49 | |
| *** ijw_ has quit IRC | 06:51 | |
| *** ijw has joined #openstack-infra | 06:51 | |
| *** thorst_ has joined #openstack-infra | 06:53 | |
| *** yuanying_ has joined #openstack-infra | 06:55 | |
| *** ijw has quit IRC | 06:56 | |
| *** zhurong has quit IRC | 06:57 | |
| *** zhurong has joined #openstack-infra | 06:58 | |
| *** yuanying has quit IRC | 06:58 | |
| *** mkoderer has joined #openstack-infra | 06:58 | |
| *** sdake has quit IRC | 06:59 | |
| *** thorst_ has quit IRC | 06:59 | |
| *** jaosorior has joined #openstack-infra | 07:00 | |
| *** e0ne has joined #openstack-infra | 07:04 | |
| *** esberglu has joined #openstack-infra | 07:06 | |
| *** tesseract has joined #openstack-infra | 07:07 | |
| *** tesseract is now known as Guest99997 | 07:07 | |
| *** rbuzatu_ has quit IRC | 07:10 | |
| *** florianf has joined #openstack-infra | 07:11 | |
| *** rbuzatu has joined #openstack-infra | 07:12 | |
| *** ihrachys has joined #openstack-infra | 07:13 | |
| *** Guest99997 is now known as tesseract- | 07:13 | |
| *** pgadiya has joined #openstack-infra | 07:13 | |
| *** yuanying_ has quit IRC | 07:14 | |
| *** yamamot__ has quit IRC | 07:14 | |
| *** armax has quit IRC | 07:14 | |
| *** esberglu has quit IRC | 07:15 | |
| *** knangia has joined #openstack-infra | 07:18 | |
| *** john-davidge has joined #openstack-infra | 07:18 | |
| *** ramineni has joined #openstack-infra | 07:19 | |
| ramineni | hi, can anyone review https://review.openstack.org/#/c/408930/1 | 07:20 |
| *** nadya has joined #openstack-infra | 07:21 | |
| *** markus_z has joined #openstack-infra | 07:21 | |
| ramineni | needed to unblock the gate | 07:21 |
| *** john-davidge has quit IRC | 07:22 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:23 | |
| *** gomarivera has joined #openstack-infra | 07:23 | |
| *** nherciu has joined #openstack-infra | 07:25 | |
| *** gomarivera has quit IRC | 07:27 | |
| *** rcernin has joined #openstack-infra | 07:28 | |
| *** Julien-zte has quit IRC | 07:31 | |
| *** Julien-zte has joined #openstack-infra | 07:31 | |
| *** oanson has quit IRC | 07:33 | |
| *** tqtran has joined #openstack-infra | 07:34 | |
| openstackgerrit | Rabi Mishra proposed openstack-infra/project-config: Cleanup services enabled for heat jobs https://review.openstack.org/398757 | 07:37 |
| *** andreas_s has joined #openstack-infra | 07:38 | |
| *** tqtran has quit IRC | 07:39 | |
| AJaeger | ramineni: we have 70+ repos covered here and not everybody reviews everything - would be nice if you explain what your change is for... | 07:41 |
| *** katkapilatova has joined #openstack-infra | 07:45 | |
| *** Julien-zte has quit IRC | 07:46 | |
| *** Julien-zte has joined #openstack-infra | 07:46 | |
| *** katkapilatova has left #openstack-infra | 07:47 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: transform tripleo upgrade in 2-nodes job https://review.openstack.org/409000 | 07:48 |
| EmilienM | AJaeger: do you mind looking at ^ when you can? | 07:54 |
| *** jtomasek has joined #openstack-infra | 07:54 | |
| *** jaosorior has quit IRC | 07:55 | |
| *** Hal has joined #openstack-infra | 07:55 | |
| *** jaosorior has joined #openstack-infra | 07:55 | |
| *** Hal is now known as Guest76524 | 07:55 | |
| AJaeger | EmilienM: LGTM | 07:57 |
| *** thorst_ has joined #openstack-infra | 07:57 | |
| *** yamahata has quit IRC | 08:00 | |
| *** jaosorior has quit IRC | 08:00 | |
| *** jaosorior has joined #openstack-infra | 08:02 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable keystone-saml2-federation service https://review.openstack.org/400750 | 08:02 |
| *** yamamot__ has joined #openstack-infra | 08:02 | |
| *** ihrachys has quit IRC | 08:03 | |
| *** thorst_ has quit IRC | 08:05 | |
| *** esberglu has joined #openstack-infra | 08:06 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: oslo.messaging: use non-dsvm for functional job https://review.openstack.org/405142 | 08:06 |
| *** dtardivel has joined #openstack-infra | 08:09 | |
| *** esberglu has quit IRC | 08:11 | |
| *** pcaruana has joined #openstack-infra | 08:16 | |
| *** e0ne has quit IRC | 08:19 | |
| *** hashar has joined #openstack-infra | 08:21 | |
| *** Julien-zte has quit IRC | 08:22 | |
| *** Julien-zte has joined #openstack-infra | 08:22 | |
| *** jascott1 has quit IRC | 08:23 | |
| openstackgerrit | Chaoyi Huang proposed openstack-infra/project-config: Add non-voting function test job to Trio2o https://review.openstack.org/407821 | 08:25 |
| *** dizquierdo has joined #openstack-infra | 08:26 | |
| openstackgerrit | Rabi Mishra proposed openstack-infra/project-config: Cleanup services enabled for heat jobs https://review.openstack.org/398757 | 08:27 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use test-setup for monasca-analytics https://review.openstack.org/409015 | 08:28 |
| openstackgerrit | Chaoyi Huang proposed openstack-infra/project-config: Add non-voting function test job to Trio2o https://review.openstack.org/407821 | 08:32 |
| *** ramineni has left #openstack-infra | 08:33 | |
| *** jpena|off is now known as jpena | 08:34 | |
| *** sdake has joined #openstack-infra | 08:34 | |
| *** s-shiono has quit IRC | 08:37 | |
| *** matrohon has joined #openstack-infra | 08:37 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use test-setup for monasca-analytics https://review.openstack.org/409015 | 08:37 |
| *** Jeffrey4l has quit IRC | 08:38 | |
| *** yamamot__ has quit IRC | 08:41 | |
| *** rbuzatu_ has joined #openstack-infra | 08:44 | |
| *** priteau has joined #openstack-infra | 08:45 | |
| *** jascott1 has joined #openstack-infra | 08:47 | |
| *** rbuzatu has quit IRC | 08:47 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Fix some node lists https://review.openstack.org/409028 | 08:48 |
| AJaeger | infra-root, anybody around to +2A this to unblock nova and neutron gates, please? ^ | 08:49 |
| *** kzaitsev_mb has quit IRC | 08:49 | |
| AJaeger | ianw, jhesketh, yolanda ^ | 08:50 |
| *** rbuzatu has joined #openstack-infra | 08:51 | |
| *** sflanigan has quit IRC | 08:52 | |
| *** rbuzatu_ has quit IRC | 08:52 | |
| *** rbuzatu_ has joined #openstack-infra | 08:54 | |
| *** rbuzatu__ has joined #openstack-infra | 08:55 | |
| *** dizquierdo has quit IRC | 08:56 | |
| *** martinkopec has joined #openstack-infra | 08:56 | |
| *** rbuzatu has quit IRC | 08:56 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:58 | |
| * AJaeger self approves now | 08:59 | |
| *** rbuzatu_ has quit IRC | 08:59 | |
| *** zzzeek has quit IRC | 09:00 | |
| *** zzzeek has joined #openstack-infra | 09:00 | |
| *** ihrachys has joined #openstack-infra | 09:00 | |
| *** ralonsoh has joined #openstack-infra | 09:00 | |
| *** flepied has joined #openstack-infra | 09:01 | |
| *** esberglu has joined #openstack-infra | 09:01 | |
| *** thorst_ has joined #openstack-infra | 09:02 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix some node lists https://review.openstack.org/409028 | 09:05 |
| jaosorior | AJaeger: if I would like to get a project to start running tests with py35, is it a matter of that being enabled in tox.ini or do I have to modify something in project-config? | 09:06 |
| *** esberglu has quit IRC | 09:06 | |
| AJaeger | jaosorior: you need to add the python35 jobs, there are open requests in project-config, see how others did it. | 09:07 |
| AJaeger | you don't need any changes in your tox.ini - in most cases it just works. | 09:07 |
| AJaeger | but if you need specific settings, you might need to add them. Just run "tox -e py35" locally and check it... | 09:08 |
| openstackgerrit | xiaodongwang991481 proposed openstack-infra/project-config: change networking-sfc tempest and functional tests to voting https://review.openstack.org/406396 | 09:08 |
| jaosorior | AJaeger: thanks | 09:08 |
| *** thorst_ has quit IRC | 09:10 | |
| *** strigazi_AFK is now known as strigazi | 09:10 | |
| *** sdake_ has joined #openstack-infra | 09:10 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/project-config: Enable non-voting python35 job for novajoin https://review.openstack.org/409038 | 09:11 |
| openstackgerrit | Merged openstack-infra/project-config: transform tripleo upgrade in 2-nodes job https://review.openstack.org/409000 | 09:11 |
| openstackgerrit | Merged openstack-infra/project-config: Add pep8 gate for rpm-packaging-tools https://review.openstack.org/408182 | 09:12 |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting function test job to Trio2o https://review.openstack.org/407821 | 09:13 |
| *** sdake has quit IRC | 09:13 | |
| *** kzaitsev_mb has quit IRC | 09:13 | |
| openstackgerrit | Merged openstack-infra/project-config: ironic: multinode VM count per node increased to 7 https://review.openstack.org/407284 | 09:14 |
| openstackgerrit | Merged openstack-infra/project-config: [Zun] Add a pipeline for testing new driver https://review.openstack.org/408356 | 09:14 |
| openstackgerrit | Merged openstack-infra/project-config: Enable heat plugin in congress dsvm jobs https://review.openstack.org/408930 | 09:14 |
| openstackgerrit | Merged openstack-infra/project-config: [Rally]Enable lbaas and fwaas services https://review.openstack.org/407315 | 09:15 |
| openstackgerrit | Merged openstack-infra/project-config: New project proposal: alexandria https://review.openstack.org/403300 | 09:16 |
| openstackgerrit | Merged openstack-infra/project-config: Create repo for Enterprise Work Group Workload Reference Architectures. https://review.openstack.org/405042 | 09:16 |
| openstackgerrit | Merged openstack-infra/project-config: Switch to the new DevStack plugin for Blazar https://review.openstack.org/406875 | 09:18 |
| openstackgerrit | Merged openstack-infra/project-config: Publish branch tarball for tacker-horizon project https://review.openstack.org/407887 | 09:18 |
| openstackgerrit | Merged openstack-infra/project-config: [rally] rename DEVSTACK_GATE_USE_KEYSTONE_V2API var https://review.openstack.org/408037 | 09:18 |
| openstackgerrit | Merged openstack-infra/project-config: networking-odl: graphs for grenade/fullstack/multinode https://review.openstack.org/408399 | 09:18 |
| openstackgerrit | Merged openstack-infra/project-config: Enable python35-nv jobs for VirtualBMC https://review.openstack.org/408697 | 09:20 |
| openstackgerrit | Merged openstack-infra/project-config: Add test-results to networking-odl jobs https://review.openstack.org/408026 | 09:20 |
| openstackgerrit | Merged openstack-infra/project-config: Make pxe_snmp jobs official and non-voting https://review.openstack.org/407604 | 09:21 |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting s3-tests to Swift3 https://review.openstack.org/406409 | 09:21 |
| openstackgerrit | Merged openstack-infra/project-config: Restrict nova-network jobs for ec2 and gce gating jobs https://review.openstack.org/406344 | 09:22 |
| openstackgerrit | Merged openstack-infra/project-config: Simplify template and its usage for gce and ec2 jobs https://review.openstack.org/408956 | 09:22 |
| *** yolanda has quit IRC | 09:22 | |
| *** saibarspeis has joined #openstack-infra | 09:23 | |
| *** Douhet has quit IRC | 09:23 | |
| *** mmandel has joined #openstack-infra | 09:24 | |
| *** Douhet has joined #openstack-infra | 09:24 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: oslo.messaging: use non-dsvm for functional job https://review.openstack.org/405142 | 09:25 |
| *** mmandel has quit IRC | 09:28 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Switch to Xenial: Zuul https://review.openstack.org/408819 | 09:28 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Switch to Xenial: storyboard https://review.openstack.org/408821 | 09:29 |
| *** Julien-zte has quit IRC | 09:29 | |
| *** john-davidge has joined #openstack-infra | 09:29 | |
| openstackgerrit | xiaodongwang991481 proposed openstack-infra/project-config: add multinode tempest test coverage in networking-sfc https://review.openstack.org/391979 | 09:30 |
| *** e0ne has joined #openstack-infra | 09:31 | |
| openstackgerrit | Yanyan Hu proposed openstack-infra/project-config: Enable heat plugin in senlin gate job https://review.openstack.org/409043 | 09:32 |
| *** vponomaryov1 has joined #openstack-infra | 09:34 | |
| openstackgerrit | Michal Dulko proposed openstack-infra/project-config: Remove multinode grenade in Cinder's stable/mitaka https://review.openstack.org/407463 | 09:35 |
| *** tqtran has joined #openstack-infra | 09:36 | |
| *** yanyanhu has quit IRC | 09:39 | |
| *** tqtran has quit IRC | 09:40 | |
| *** jascott1 has quit IRC | 09:40 | |
| *** zhurong has quit IRC | 09:46 | |
| *** dtantsur|afk is now known as dtantsur | 09:46 | |
| *** tosky has joined #openstack-infra | 09:50 | |
| *** claudiub has joined #openstack-infra | 09:51 | |
| *** esberglu has joined #openstack-infra | 09:51 | |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config: Enable Py35-NV jobs for monasca https://review.openstack.org/398774 | 09:54 |
| openstackgerrit | Thierry Carrez proposed openstack-infra/system-config: Enable #openstack-meeting-5 https://review.openstack.org/409053 | 09:57 |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config: Enable #openstack-meeting-5 https://review.openstack.org/409054 | 09:57 |
| *** askb has quit IRC | 09:58 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata https://review.openstack.org/404831 | 09:58 |
| *** esberglu has quit IRC | 10:01 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/irc-meetings: Allow meetings to be scheduled in meeting-5 https://review.openstack.org/409055 | 10:02 |
| *** nadya has quit IRC | 10:03 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/irc-meetings: Allow meetings to be scheduled in meeting-5 https://review.openstack.org/409055 | 10:05 |
| *** thorst_ has joined #openstack-infra | 10:09 | |
| *** cody-somerville has joined #openstack-infra | 10:13 | |
| *** cody-somerville has joined #openstack-infra | 10:13 | |
| *** thorst_ has quit IRC | 10:15 | |
| *** csomerville has quit IRC | 10:16 | |
| AJaeger | infra-root, do we need to restart nodepool due to the large amount of node changes in jobs? | 10:18 |
| AJaeger | Looking at grafana I see no jobs ready for xenial - and 30 for trusty. That looks odd | 10:18 |
| AJaeger | and 14 ubuntu-trusty-2-node which is odd as well | 10:18 |
| *** kzaitsev_mb has joined #openstack-infra | 10:20 | |
| *** tovin07 has quit IRC | 10:22 | |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Move lvm-multibackend to check queue https://review.openstack.org/408228 | 10:22 |
| *** gomarivera has joined #openstack-infra | 10:23 | |
| *** gomarivera has quit IRC | 10:27 | |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Adds new experimental Cinder migration job https://review.openstack.org/381737 | 10:28 |
| *** erlon-airlong has joined #openstack-infra | 10:28 | |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Move lvm-multibackend to check queue https://review.openstack.org/408228 | 10:30 |
| *** nadya has joined #openstack-infra | 10:33 | |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Move lvm-multibackend to check queue https://review.openstack.org/408228 | 10:33 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Adds new experimental Cinder migration job https://review.openstack.org/381737 | 10:33 |
| *** nherciu_ has joined #openstack-infra | 10:34 | |
| openstackgerrit | Dirk Mueller proposed openstack/diskimage-builder: (DNM) Enable build-succeeds functests for openSUSE https://review.openstack.org/409071 | 10:34 |
| *** nadya has quit IRC | 10:37 | |
| *** apuimedo is now known as apuimedo|away | 10:37 | |
| *** nherciu has quit IRC | 10:38 | |
| *** jascott1 has joined #openstack-infra | 10:40 | |
| *** jascott1 has quit IRC | 10:45 | |
| *** Jeffrey4l has joined #openstack-infra | 10:47 | |
| EmilienM | AJaeger: thanks | 10:48 |
| *** nadya has joined #openstack-infra | 10:49 | |
| *** timello has joined #openstack-infra | 10:51 | |
| *** esberglu has joined #openstack-infra | 10:51 | |
| *** dizquierdo has joined #openstack-infra | 10:55 | |
| *** esberglu has quit IRC | 10:55 | |
| *** jaosorior is now known as jaosorior_lunch | 11:00 | |
| *** tpsilva has joined #openstack-infra | 11:01 | |
| *** derekh has joined #openstack-infra | 11:06 | |
| *** claudiub has quit IRC | 11:08 | |
| *** sdague_ has joined #openstack-infra | 11:09 | |
| openstackgerrit | Julia Varlamova proposed openstack-infra/project-config: Add Keystone multinode Grenade job https://review.openstack.org/409088 | 11:09 |
| openstackgerrit | Julia Varlamova proposed openstack-infra/project-config: Add Keystone multinode Grenade job https://review.openstack.org/407428 | 11:11 |
| *** thorst_ has joined #openstack-infra | 11:13 | |
| *** john-davidge has quit IRC | 11:17 | |
| *** john-dav_ has joined #openstack-infra | 11:17 | |
| *** thorst_ has quit IRC | 11:20 | |
| *** jordanP has joined #openstack-infra | 11:20 | |
| *** knangia has quit IRC | 11:24 | |
| *** spzala has joined #openstack-infra | 11:24 | |
| *** claudiub has joined #openstack-infra | 11:24 | |
| *** spzala has quit IRC | 11:29 | |
| *** maeker has joined #openstack-infra | 11:30 | |
| *** bhavik1 has joined #openstack-infra | 11:31 | |
| *** tuanluong has quit IRC | 11:34 | |
| *** mhickey has joined #openstack-infra | 11:34 | |
| *** nicolasbock has joined #openstack-infra | 11:35 | |
| *** david-lyle_ has joined #openstack-infra | 11:35 | |
| *** maeker has quit IRC | 11:35 | |
| *** tqtran has joined #openstack-infra | 11:37 | |
| *** david-lyle has quit IRC | 11:37 | |
| *** tqtran has quit IRC | 11:41 | |
| *** pgadiya has quit IRC | 11:45 | |
| *** deadnull has joined #openstack-infra | 11:45 | |
| *** deadnull has quit IRC | 11:45 | |
| *** saibarspeis has quit IRC | 11:46 | |
| *** saibarspeis has joined #openstack-infra | 11:46 | |
| *** deadnull has joined #openstack-infra | 11:47 | |
| *** Goneri has joined #openstack-infra | 11:52 | |
| *** drifterza has quit IRC | 11:54 | |
| *** rbuzatu__ has quit IRC | 11:55 | |
| *** jaosorior_lunch is now known as jaosorior | 11:55 | |
| *** nherciu__ has joined #openstack-infra | 11:57 | |
| *** jkilpatr has quit IRC | 11:58 | |
| *** bauzas is now known as bauwseer | 12:00 | |
| *** bauwseer is now known as bauwser | 12:00 | |
| openstackgerrit | Julia Varlamova proposed openstack-infra/project-config: Add Keystone multinode Grenade job https://review.openstack.org/407428 | 12:00 |
| *** nherciu_ has quit IRC | 12:00 | |
| *** edmondsw_ has joined #openstack-infra | 12:02 | |
| *** nherciu_ has joined #openstack-infra | 12:02 | |
| *** jascott1 has joined #openstack-infra | 12:03 | |
| Cibo | hello everybody, what should I do to forbid in devstack-gate using of localrc? | 12:03 |
| *** edmondsw_ has quit IRC | 12:03 | |
| *** nherciu__ has quit IRC | 12:05 | |
| *** ram______ has joined #openstack-infra | 12:07 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata https://review.openstack.org/404831 | 12:07 |
| *** ram______ has left #openstack-infra | 12:07 | |
| *** jascott1 has quit IRC | 12:08 | |
| *** dave-mccowan has joined #openstack-infra | 12:09 | |
| wznoinsk | sdague_, fungi any chance you could revise comments on https://review.openstack.org/#/c/407424/ ? | 12:10 |
| AJaeger | sdague_: could you review https://review.openstack.org/398757 ? I'm not sure about the useage of ENABLED_SERVICES there | 12:13 |
| *** oanson has joined #openstack-infra | 12:13 | |
| *** raildo has joined #openstack-infra | 12:17 | |
| *** thorst_ has joined #openstack-infra | 12:18 | |
| *** hashar is now known as hasharLunch | 12:18 | |
| *** jkilpatr has joined #openstack-infra | 12:24 | |
| sdague_ | AJaeger: I think that's right, because OVERRIDE kills any other definition, and doing that will just add to them | 12:24 |
| *** thorst_ has quit IRC | 12:24 | |
| sdague_ | wznoinsk: it's really hard to get behind a basically undocumented line in devstack-gate that isn't used in our CI system anywhere so we know if we are going to break it. It's very fragile. Plus it's in the middle of the multinode stuff, which is already fragile. | 12:25 |
| sdague_ | wznoinsk: is there an email on the list somewhere, or a bug, with context about what is not working | 12:26 |
| *** nherciu_ has quit IRC | 12:27 | |
| AJaeger | sdague_: thanks. | 12:27 |
| wznoinsk | sdague_, there's this https://storyboard.openstack.org/#!/story/2000824 (instead of lp bug) | 12:27 |
| *** nherciu has joined #openstack-infra | 12:27 | |
| wznoinsk | I guess we need to rely on reviews from infra for this change as this line is not documented yet, I'm happy to answer any questions about the problem we're trying to solve in here or on the story board | 12:29 |
| openstackgerrit | Kyrylo Romanenko proposed openstack-infra/project-config: Add flat network driver for ironicclient functional tests https://review.openstack.org/409122 | 12:30 |
| sdague_ | wznoinsk: so what if we fix devstack to instead not throw an error in that case but just warn? | 12:30 |
| *** jkilpatr has quit IRC | 12:30 | |
| *** jamielennox is now known as jamielennox|away | 12:33 | |
| *** rhallisey has joined #openstack-infra | 12:33 | |
| openstackgerrit | Merged openstack-infra/project-config: Cleanup services enabled for heat jobs https://review.openstack.org/398757 | 12:34 |
| sdague_ | wznoinsk: like this - https://review.openstack.org/409123 ? | 12:34 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Revert "Rate limit testenv creation" https://review.openstack.org/409127 | 12:40 |
| *** rbuzatu has joined #openstack-infra | 12:42 | |
| *** jpena is now known as jpena|lunch | 12:43 | |
| *** jkilpatr has joined #openstack-infra | 12:44 | |
| *** nherciu_ has joined #openstack-infra | 12:45 | |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/devstack-gate: [DO NOT MERGE] Testing gate job https://review.openstack.org/396592 | 12:48 |
| *** vikrant has quit IRC | 12:48 | |
| electrofelix | should zuul automatically reload it's conf on changes to layout.yaml or is it manual reload only? | 12:48 |
| electrofelix | experimenting with zuul in a container | 12:48 |
| *** nherciu has quit IRC | 12:48 | |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/devstack-gate: [DO NOT MERGE] Testing gate job https://review.openstack.org/396592 | 12:53 |
| *** firenka_ has left #openstack-infra | 12:53 | |
| pabelanger | electrofelix: sighup1 should trigger a reload for zuul | 12:54 |
| *** psachin has quit IRC | 12:55 | |
| pabelanger | AJaeger: not nodepool, but zuul likely needs a reboot. We can wait until clarkb is online to see what he would like to do | 12:55 |
| AJaeger | pabelanger: ok - whatever fixes it ;) | 13:00 |
| *** pradk has joined #openstack-infra | 13:01 | |
| openstackgerrit | Oksana Voshchana proposed openstack-infra/devstack-gate: TEST for multinode grenade devstack https://review.openstack.org/396679 | 13:01 |
| *** deadnull is now known as _deadnull | 13:02 | |
| *** _deadnull is now known as deadnull | 13:02 | |
| *** jascott1 has joined #openstack-infra | 13:05 | |
| *** electrical has quit IRC | 13:05 | |
| *** pcaruana has quit IRC | 13:05 | |
| *** electrical has joined #openstack-infra | 13:06 | |
| *** hasharLunch is now known as hashar | 13:07 | |
| *** kzaitsev_mb has quit IRC | 13:07 | |
| *** pcaruana has joined #openstack-infra | 13:08 | |
| *** thorst_ has joined #openstack-infra | 13:08 | |
| *** jascott1 has quit IRC | 13:10 | |
| *** chem has quit IRC | 13:12 | |
| *** nherciu_ has quit IRC | 13:14 | |
| *** nherciu has joined #openstack-infra | 13:15 | |
| *** abregman has joined #openstack-infra | 13:17 | |
| *** abregman has quit IRC | 13:17 | |
| *** chem has joined #openstack-infra | 13:18 | |
| *** asettle has quit IRC | 13:18 | |
| *** asettle has joined #openstack-infra | 13:19 | |
| *** jpena|lunch is now known as jpena | 13:22 | |
| *** edwarnicke has quit IRC | 13:22 | |
| *** edwarnicke has joined #openstack-infra | 13:23 | |
| *** trown|outtypewww is now known as trown | 13:25 | |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/devstack-gate: [DO NOT MERGE] Testing gate job https://review.openstack.org/396592 | 13:25 |
| *** kzaitsev_mb has joined #openstack-infra | 13:28 | |
| *** fkautz has quit IRC | 13:28 | |
| *** fkautz has joined #openstack-infra | 13:29 | |
| *** mdrabe has joined #openstack-infra | 13:30 | |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config: Make fwaas multinode job nonvoting https://review.openstack.org/409147 | 13:31 |
| *** warpc has quit IRC | 13:34 | |
| *** warpc has joined #openstack-infra | 13:34 | |
| *** dougwig has quit IRC | 13:34 | |
| *** dougwig has joined #openstack-infra | 13:35 | |
| *** nmathew- has joined #openstack-infra | 13:35 | |
| smcginnis | clarkb: Hi, you around? Wondering if you could explain what you suggest here? http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-11-30.log.html#t2016-11-30T19:14:31 | 13:35 |
| smcginnis | clarkb: If you could point me at a similar setup in the file, that would help. Still a zuul noob. | 13:36 |
| *** ewindisch has quit IRC | 13:36 | |
| *** ewindisch has joined #openstack-infra | 13:36 | |
| *** jpena is now known as jpena|brb | 13:39 | |
| *** sweston has quit IRC | 13:39 | |
| *** tqtran has joined #openstack-infra | 13:39 | |
| *** sweston has joined #openstack-infra | 13:39 | |
| *** EricGonczer_ has joined #openstack-infra | 13:40 | |
| *** gordc has joined #openstack-infra | 13:40 | |
| *** andreas_s has quit IRC | 13:41 | |
| *** davidlenwell has quit IRC | 13:41 | |
| *** davidlenwell has joined #openstack-infra | 13:41 | |
| *** tqtran has quit IRC | 13:43 | |
| *** nherciu_ has joined #openstack-infra | 13:46 | |
| *** nherciu has quit IRC | 13:47 | |
| *** csomerville has joined #openstack-infra | 13:50 | |
| *** boris-42 has quit IRC | 13:51 | |
| *** spzala has joined #openstack-infra | 13:51 | |
| *** boris-42 has joined #openstack-infra | 13:51 | |
| wznoinsk | sdague_, we still want to die if TEMPEST_CONFIG is not in main node's local.conf tho | 13:51 |
| *** andymaier has joined #openstack-infra | 13:53 | |
| *** cody-somerville has quit IRC | 13:54 | |
| sdague_ | wznoinsk: maybe, I'm honestly not convinced that supporting multiple local.conf for a single deploy is a sensible way to go. I think knowing that we want to do more multinode devstack, we should actually make the devstack config aware of that fact | 13:55 |
| sdague_ | so this softens that so it can work in your case, and we can figure out the next steps here | 13:55 |
| *** adarazs has left #openstack-infra | 13:56 | |
| *** fguillot has joined #openstack-infra | 13:57 | |
| *** nherciu has joined #openstack-infra | 13:58 | |
| *** kgiusti has joined #openstack-infra | 13:58 | |
| *** nherciu_ has quit IRC | 13:58 | |
| openstackgerrit | Merged openstack-infra/shade: Move image tasks to REST https://review.openstack.org/385190 | 13:59 |
| openstackgerrit | Merged openstack-infra/shade: Update image downloads to use direct REST https://review.openstack.org/385192 | 14:00 |
| *** esikachev has quit IRC | 14:00 | |
| *** daemontool has joined #openstack-infra | 14:00 | |
| wznoinsk | sdague_, sounds fair to have it in devstack long term, for the time being, to get our multinode job going, it would be good to do it in d-g, as other part of the devstack's configuration is already handled there, i.e.: https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L196:L197 | 14:01 |
| *** lunarlamp is now known as mariusv | 14:02 | |
| *** xyang1 has joined #openstack-infra | 14:03 | |
| *** jpena|brb is now known as jpena | 14:03 | |
| *** mgkwill has quit IRC | 14:03 | |
| wznoinsk | sdague_, if we could workaround it temporarily in our ci ourselves we'd do it, but it's not possible without _hacking_ the d-g code currently | 14:03 |
| *** tlian has joined #openstack-infra | 14:03 | |
| *** mgkwill has joined #openstack-infra | 14:03 | |
| sdague_ | wznoinsk: well, I don't think it's really maintainable there. I proposed an alternative solution which you don't seem to be accepting. | 14:03 |
| wznoinsk | sdague_, I can accept your alternative - will sort me out for sure, it will make devstack itself more vulnerable though - it will progress ignoring(warning only) if for some reason essential parts of the configuration are missing | 14:05 |
| wznoinsk | + http://logs.openstack.org/23/409123/1/check/gate-devstack-unit-tests/b811893/console.html#_2016-12-09_12_36_31_356202 | 14:06 |
| *** arxcruz has quit IRC | 14:07 | |
| *** jamesdenton has joined #openstack-infra | 14:08 | |
| *** hrybacki has quit IRC | 14:08 | |
| *** hrybacki has joined #openstack-infra | 14:08 | |
| *** arxcruz has joined #openstack-infra | 14:10 | |
| *** rbuzatu has quit IRC | 14:10 | |
| *** rbuzatu has joined #openstack-infra | 14:11 | |
| openstackgerrit | Merged openstack-infra/gerritlib: Specify testenv command for tox>=2.5 https://review.openstack.org/399795 | 14:12 |
| openstackgerrit | Merged openstack-infra/gerritlib: Allow creating projects with spaces https://review.openstack.org/399310 | 14:13 |
| openstackgerrit | Merged openstack-infra/system-config: Correct signing key export instructions https://review.openstack.org/400469 | 14:13 |
| *** jtomasek_ has joined #openstack-infra | 14:13 | |
| openstackgerrit | Merged openstack-infra/gerritlib: Remove deprecated create-project '--name' option https://review.openstack.org/399308 | 14:14 |
| *** baoli has joined #openstack-infra | 14:15 | |
| *** rbuzatu has quit IRC | 14:15 | |
| *** arxcruz has quit IRC | 14:15 | |
| *** Guest76524 has quit IRC | 14:16 | |
| *** srobert has joined #openstack-infra | 14:16 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: TLS everywhere job introduction https://review.openstack.org/391738 | 14:17 |
| *** deadnull has quit IRC | 14:17 | |
| *** esberglu has joined #openstack-infra | 14:17 | |
| *** arxcruz has joined #openstack-infra | 14:18 | |
| openstackgerrit | Sean McGinnis proposed openstack-infra/bugdaystats: Remove use of simplejson library https://review.openstack.org/409166 | 14:18 |
| *** mattfarina has quit IRC | 14:19 | |
| *** mattfarina has joined #openstack-infra | 14:19 | |
| *** mtanino has joined #openstack-infra | 14:19 | |
| *** lindycoder has joined #openstack-infra | 14:19 | |
| *** esikachev has joined #openstack-infra | 14:20 | |
| *** srobert has quit IRC | 14:21 | |
| *** srobert has joined #openstack-infra | 14:21 | |
| *** dimtruck has quit IRC | 14:22 | |
| *** andrewbogott has quit IRC | 14:23 | |
| *** andrewbogott has joined #openstack-infra | 14:23 | |
| *** andrewbogott has quit IRC | 14:23 | |
| *** andrewbogott has joined #openstack-infra | 14:23 | |
| *** saibarspeis has joined #openstack-infra | 14:24 | |
| *** rbuzatu has joined #openstack-infra | 14:25 | |
| *** yamamoto has joined #openstack-infra | 14:25 | |
| *** nherciu_ has joined #openstack-infra | 14:25 | |
| *** hurgleburgler has joined #openstack-infra | 14:26 | |
| jroll | AJaeger: thanks for the +W :) | 14:27 |
| *** nherciu__ has joined #openstack-infra | 14:28 | |
| *** nherciu has quit IRC | 14:28 | |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: lvm-multibackend https://review.openstack.org/409168 | 14:29 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: devstack-plugin-nfs-nv https://review.openstack.org/409169 | 14:29 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: full-bdd https://review.openstack.org/409170 | 14:29 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: zeromq-multibackend https://review.openstack.org/409171 | 14:29 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: drbd-devstack https://review.openstack.org/409172 | 14:29 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: devstack-plugin-ceph https://review.openstack.org/409173 | 14:29 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: neutron-identity-v3-only https://review.openstack.org/409174 | 14:29 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: sheepdog https://review.openstack.org/409175 | 14:29 |
| openstackgerrit | Erlon R. Cruz proposed openstack-infra/project-config: Run Cinder in-tree tests: full-lio https://review.openstack.org/409176 | 14:29 |
| openstackgerrit | Merged openstack-infra/project-config: Enable ironic tempest plugin in grenade job https://review.openstack.org/407488 | 14:29 |
| *** nherciu_ has quit IRC | 14:30 | |
| *** jeckersb_gone is now known as jeckersb | 14:34 | |
| *** nherciu_ has joined #openstack-infra | 14:36 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata https://review.openstack.org/404831 | 14:37 |
| *** nherciu__ has quit IRC | 14:37 | |
| *** patrickeast has quit IRC | 14:38 | |
| *** jcoufal has joined #openstack-infra | 14:38 | |
| *** patrickeast has joined #openstack-infra | 14:38 | |
| trown | is there a known issue with sphinx unable to process the "Team and repository tags" rst? | 14:39 |
| *** yamamoto has quit IRC | 14:39 | |
| *** piet has quit IRC | 14:40 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Revert "Rate limit testenv creation" https://review.openstack.org/409127 | 14:40 |
| *** nadya has quit IRC | 14:41 | |
| *** maeker has joined #openstack-infra | 14:42 | |
| *** pcaruana has quit IRC | 14:42 | |
| *** maeker has quit IRC | 14:43 | |
| *** yamamoto has joined #openstack-infra | 14:44 | |
| *** markus_z has quit IRC | 14:45 | |
| *** nmathew- has quit IRC | 14:45 | |
| *** iremizov has quit IRC | 14:45 | |
| *** maeker has joined #openstack-infra | 14:45 | |
| *** iremizov has joined #openstack-infra | 14:45 | |
| trown | seeing the docs job fail on multiple repos: https://review.openstack.org/#/c/406165/ https://review.openstack.org/#/c/408773/ ... If I comment out the image tag in the "Team and repository tags" section at the top of README.rst sphinx builds fine | 14:47 |
| trown | since docs job is in the gate, nothing can merge | 14:47 |
| mnaser | github slowly transforming to a similar version of gerrit :p | 14:47 |
| *** cargonza has quit IRC | 14:50 | |
| *** maeker has quit IRC | 14:50 | |
| *** cargonza has joined #openstack-infra | 14:50 | |
| openstackgerrit | Oksana Voshchana proposed openstack-infra/devstack-gate: TEST for multinode grenade devstack https://review.openstack.org/396679 | 14:52 |
| *** HeOS has quit IRC | 14:52 | |
| *** raddaoui has quit IRC | 14:53 | |
| *** raddaoui has joined #openstack-infra | 14:53 | |
| *** jeckersb is now known as jeckersb_gone | 14:54 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Set noop dhcp provider for ironicclient func tests https://review.openstack.org/409188 | 14:54 |
| *** ediardo has quit IRC | 14:56 | |
| *** ediardo has joined #openstack-infra | 14:56 | |
| *** pcaruana has joined #openstack-infra | 14:56 | |
| *** jtomasek has quit IRC | 14:57 | |
| *** EricGonczer_ has quit IRC | 14:58 | |
| *** jeckersb_gone is now known as jeckersb | 14:58 | |
| *** mriedem has joined #openstack-infra | 14:58 | |
| *** tphummel has joined #openstack-infra | 14:59 | |
| *** links has quit IRC | 14:59 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Set noop dhcp provider for ironicclient func tests https://review.openstack.org/409188 | 15:00 |
| *** sdake_ has quit IRC | 15:01 | |
| *** zaro has quit IRC | 15:02 | |
| *** zaro has joined #openstack-infra | 15:03 | |
| *** gouthamr has joined #openstack-infra | 15:04 | |
| *** dansmith is now known as superdan | 15:04 | |
| *** csatari has quit IRC | 15:05 | |
| *** csatari has joined #openstack-infra | 15:05 | |
| *** Hal has joined #openstack-infra | 15:08 | |
| *** jascott1 has joined #openstack-infra | 15:08 | |
| *** Hal is now known as Guest92587 | 15:08 | |
| *** florianf has quit IRC | 15:09 | |
| *** yamamoto has quit IRC | 15:10 | |
| *** tomhambleton has quit IRC | 15:10 | |
| *** tomhambleton has joined #openstack-infra | 15:10 | |
| AJaeger | fungi, clarkb, could you check https://review.openstack.org/407434 - for project-config. What do you think about this? | 15:12 |
| AJaeger | trown: best ask flaper87 | 15:12 |
| *** rbrndt has joined #openstack-infra | 15:12 | |
| *** jascott1 has quit IRC | 15:12 | |
| esberglu | asselin: I have a problem that I thought you might be able to help with | 15:13 |
| *** derekh has quit IRC | 15:13 | |
| *** yamamoto has joined #openstack-infra | 15:14 | |
| *** derekh has joined #openstack-infra | 15:15 | |
| *** yamamoto has quit IRC | 15:16 | |
| *** florianf has joined #openstack-infra | 15:17 | |
| *** yamamoto has joined #openstack-infra | 15:18 | |
| *** mwhahaha has quit IRC | 15:18 | |
| *** mwhahaha has joined #openstack-infra | 15:18 | |
| Cibo | hello, devstack-gate: when I am adding changes in my localrc in pre test hook they get wiped out by new... configs | 15:19 |
| *** yamamoto has quit IRC | 15:21 | |
| Cibo | it looks like I missing something in configuration... | 15:22 |
| tosky | Cibo: export KEEP_LOCALRC=1 | 15:22 |
| AJaeger | sdague_: is 409168 really a good solution? Using a regex looks counter-intuitive to me. Are there no better alternatives? | 15:22 |
| Cibo | =( | 15:22 |
| tosky | Cibo: for example http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/sahara.yaml#n158 | 15:23 |
| AJaeger | fungi: reading your comment on the success/failure message: That was exactly my initial reaction... | 15:23 |
| tosky | Cibo: if I this is the issue that you are hitting, feel free to vote on this :) https://review.openstack.org/#/c/362696/ | 15:23 |
| JayF | Hey, if anyone has a sec, AJaeger already has a +2 on https://review.openstack.org/#/c/409188/ and we need it to fix the python-ironicclient gate -- I just landed the ironic-half of that patch | 15:24 |
| Cibo | tosky: if you ever get in Belarus, ---> I owe you beer | 15:24 |
| *** robcresswell has quit IRC | 15:24 | |
| AJaeger | JayF: the depends-on has not merged yet ;( | 15:25 |
| *** robcresswell has joined #openstack-infra | 15:25 | |
| sdague_ | AJaeger: honestly I don't know what the current preferred approach is by the tempest team for enabling tests from plugin | 15:25 |
| AJaeger | JayF: once it has merged we need to +W again | 15:25 |
| *** yamahata has joined #openstack-infra | 15:25 | |
| JayF | AJaeger: I approved it about 5m ago -- it won't go in automaticaly once that one lands? | 15:25 |
| *** jtomasek_ is now known as jtomasek | 15:25 | |
| AJaeger | JayF: no - they don't share a queue and thus this will not work. | 15:26 |
| jordanP | guys, Tempest seems to have a problem with its doc job | 15:26 |
| AJaeger | Since we cannot enqueue them one after another, the second one is just blocked. | 15:26 |
| JayF | AJaeger: Ah, sad, do you mind a ping when the Ironic side fully merges so you can push it back into the queue? | 15:26 |
| AJaeger | JayF: just ping here and somebody will help - if I'm around I'll +A | 15:26 |
| JayF | Awesome, thanks! And TIL about the depends-on | 15:27 |
| *** jtomasek has quit IRC | 15:27 | |
| AJaeger | sdague_: any idea who can help with tempest? | 15:27 |
| AJaeger | JayF: TIL? | 15:27 |
| *** jtomasek has joined #openstack-infra | 15:27 | |
| JayF | AJaeger: today I learned | 15:27 |
| Cibo | tosky: +1 | 15:29 |
| *** mriedem is now known as mriedem_meeting | 15:29 | |
| AJaeger | JayF: ah, thanks. If you want to learn some more, read http://docs.openstack.org/infra/manual/developers.html#limitations-and-caveats | 15:30 |
| jordanP | the docs issue could be related to docutils | 15:31 |
| fungi | yeah, i or someone else can reapprove it when that happens. i didn't check whether it had merged and assumed that's what "landed" meant | 15:31 |
| jordanP | 0.13 went out recently | 15:31 |
| AJaeger | fungi, is 409168 really a good solution? Using a regex looks counter-intuitive to me. Are there no better alternatives? Or do you know who from tempest could give guideance on this, please? | 15:33 |
| *** daemontool has quit IRC | 15:33 | |
| JayF | AJaeger: thanks, I just read over that :) | 15:34 |
| *** jaosorior has quit IRC | 15:35 | |
| fungi | AJaeger: mtreinish is enjoying some time off, but maybe sdague_ or oomichi could give us a thumbs-up? or just ask in #openstack-qa | 15:35 |
| fungi | i really don't know, and treat most of the devstack and tempest envvars as a black box | 15:35 |
| AJaeger | yeah, that's my challenge here as well. | 15:35 |
| AJaeger | Ok, will ask on #openstack-qa - thanks | 15:35 |
| *** Julien-zte has joined #openstack-infra | 15:37 | |
| *** knangia has joined #openstack-infra | 15:37 | |
| *** yamamoto has joined #openstack-infra | 15:43 | |
| AJaeger | fungi, change looks fine according to openstack-qa ;) | 15:43 |
| fungi | wfm then | 15:44 |
| *** eharney has joined #openstack-infra | 15:44 | |
| AJaeger | smcginnis: do you wnat to review the stack starting at https://review.openstack.org/#/c/409168 ? | 15:44 |
| fungi | i guess the only concern there is that it's duplicating information from tempest, which could change there and diverge over time | 15:44 |
| tosky | fungi: duplicating? | 15:45 |
| AJaeger | fungi: to me it sounds like they have no better interface. | 15:45 |
| smcginnis | AJaeger: Thanks, I'll take a look. | 15:45 |
| fungi | tosky: "same as a tempest -efull would run" presupposes that tempest -efull behavior won't change in the future | 15:46 |
| AJaeger | tosky: it takes the full regex from teempest and enhances it. | 15:46 |
| fungi | but as long as the cinder devs are going to keep up with that i don't really mind | 15:46 |
| *** ccamacho1 has joined #openstack-infra | 15:46 | |
| AJaeger | tosky: I'm missing variable to extend the default - whatever the default is | 15:46 |
| tosky | not sure that it exists, it could be something to check | 15:46 |
| *** krtaylor has joined #openstack-infra | 15:46 | |
| tosky | on the other side, I would find extremely strange that tempest.(api|scenario) would break in the future | 15:47 |
| *** ccamacho has quit IRC | 15:47 | |
| *** yamamoto has quit IRC | 15:47 | |
| *** yamamoto has joined #openstack-infra | 15:48 | |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config: Enable plugin for fwaas multinode tests https://review.openstack.org/409202 | 15:48 |
| *** pradk- has joined #openstack-infra | 15:49 | |
| *** pradk has quit IRC | 15:49 | |
| *** mmandel_ has joined #openstack-infra | 15:51 | |
| flaper87 | trown: I was afk (and likely will be afk soon). How can I help? | 15:52 |
| *** rbuzatu has quit IRC | 15:52 | |
| *** rbuzatu has joined #openstack-infra | 15:52 | |
| AJaeger | flaper87: https://review.openstack.org/#/c/406165/ - the docs job fail with the badges - and we haven't pinpoint it yet | 15:53 |
| openstackgerrit | Rabi Mishra proposed openstack-infra/project-config: Remove n-novnc from enabled servcies https://review.openstack.org/409204 | 15:53 |
| flaper87 | looking at the patch now | 15:53 |
| flaper87 | I think stevemar hit something like this for keystone, but I might be wrong | 15:53 |
| *** chandankumar|off is now known as chandankumar | 15:54 | |
| flaper87 | some projects include the readme as part of the docs | 15:54 |
| *** mriedem_meeting is now known as mriedem | 15:54 | |
| * flaper87 gets the email | 15:54 | |
| flaper87 | https://review.openstack.org/#/c/402878/ | 15:55 |
| * flaper87 tries this for tripleo-common | 15:55 | |
| AJaeger | but removing the README from the docs looks like the wrong thing to do in general | 15:56 |
| *** tesseract- has quit IRC | 15:56 | |
| AJaeger | flaper87: might be broken sincd docutils 0.13 was released | 15:56 |
| * flaper87 spoke too fast... he had not noticed keystone removed it from the readme | 15:56 | |
| *** rbuzatu has quit IRC | 15:56 | |
| *** Julien-zte has quit IRC | 15:56 | |
| flaper87 | could be | 15:56 |
| * flaper87 tries to replicate locally | 15:57 | |
| *** pradk- is now known as pradk | 15:58 | |
| *** nadya has joined #openstack-infra | 15:59 | |
| *** hongbin has joined #openstack-infra | 15:59 | |
| openstackgerrit | Merged openstack-infra/project-config: Run Cinder in-tree tests: lvm-multibackend https://review.openstack.org/409168 | 15:59 |
| flaper87 | mmh, worked locally | 16:00 |
| flaper87 | ok, nvm, replicated | 16:00 |
| *** Guest92587 has quit IRC | 16:01 | |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config: Add Fuxi job to run fullstack tests https://review.openstack.org/408330 | 16:02 |
| clarkb | smcginnis: basically zuul's config lets you specify branch names or regexes to match branch names to run tests. Because it is a python regex you can use negative lookahead to do negativem atching too. ALl this means you can set your cinder jobs to run on not branch foo | 16:03 |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Change meeting room for interop challenge https://review.openstack.org/409207 | 16:04 |
| openstackgerrit | Shamail Tahir proposed openstack-infra/irc-meetings: Change meeting room for interop challenge https://review.openstack.org/409207 | 16:04 |
| clarkb | AJaeger: pabelanger I think its specifically the gearman server that needs to be restarted so that it loses all memory of jobs. But since zuul daemon runs the geard we can restart the whole thing to keep it simple | 16:05 |
| *** Julien-zte has joined #openstack-infra | 16:06 | |
| *** rcernin has quit IRC | 16:07 | |
| smcginnis | clarkb: Do you have any pointer to a similar scenario as I'm trying to cover? | 16:07 |
| flaper87 | AJaeger: think I fixed it | 16:08 |
| clarkb | smcginnis: https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n1922 its pretty commonly used to exclude older stable branches from running new jobs they have no hope of passing | 16:08 |
| smcginnis | clarkb: Awesome, thanks. I'll take a look. | 16:09 |
| *** ralonsoh has quit IRC | 16:09 | |
| smcginnis | clarkb: So I would need to call out each unique test name pattern the not run "^(?!...)" on driver fixes branches. | 16:10 |
| flaper87 | AJaeger: trown https://review.openstack.org/#/c/409209/ | 16:10 |
| AJaeger | flaper87: several repos are affected - if you have a fix, best tell the world. Great! | 16:10 |
| smcginnis | s/the not run/to not run/ | 16:10 |
| *** armax has joined #openstack-infra | 16:11 | |
| flaper87 | AJaeger: I know, I'm on that | 16:11 |
| clarkb | smcginnis: the names can be regexes too, so you can match .*-cinder-.* and get a good chunk of them but for the others yea | 16:11 |
| *** chlong has joined #openstack-infra | 16:11 | |
| AJaeger | thanks a lot, flaper87 | 16:11 |
| *** chlong has quit IRC | 16:12 | |
| smcginnis | clarkb: And keep up on any non .*-cinder-.* added down the road. But works for me! | 16:12 |
| smcginnis | Thanks! | 16:12 |
| flaper87 | np, my pleasure. I'll go with an email this time instead of a massive patch submission. I'm about to board a plane | 16:12 |
| trown | flaper87: nice, I reverted the original patch on tripleo-quickstart because we had some patches we needed to merge, but I will put up an unrevert with that patch on top | 16:12 |
| *** chlong has joined #openstack-infra | 16:12 | |
| *** ihrachys has quit IRC | 16:12 | |
| flaper87 | trown: thank you :) | 16:12 |
| *** sdake has joined #openstack-infra | 16:13 | |
| flaper87 | trown: can you help me verify the fix in your local env ? | 16:13 |
| trown | flaper87: sure | 16:14 |
| flaper87 | I'd like to send the email out and waiting for the gate is not an option since I've a flight to catch | 16:14 |
| AJaeger | armax: are you around for 408846 ? | 16:14 |
| *** annegentle has joined #openstack-infra | 16:14 | |
| armax | yes | 16:14 |
| *** ijw has joined #openstack-infra | 16:14 | |
| * AJaeger checks out the tree to double check | 16:15 | |
| *** thorst__ has joined #openstack-infra | 16:15 | |
| armax | AJaeger: so how is that job landing on the neutron-lib queue? | 16:15 |
| AJaeger | tempest-dsvm-neutron-src-neutron-lib is not in zuul/layout.yaml anymore after your change | 16:15 |
| AJaeger | armax: it might have been in the past... | 16:15 |
| armax | what do you mean? | 16:15 |
| armax | the link I sent you is from a recent run | 16:16 |
| AJaeger | that's 5 days ago, lets check... | 16:16 |
| armax | I can pull a newer one | 16:16 |
| armax | hang on | 16:16 |
| *** Julien-zte has quit IRC | 16:17 | |
| armax | AJaeger: https://review.openstack.org/#/c/407975/ | 16:17 |
| *** thorst_ has quit IRC | 16:17 | |
| AJaeger | armax, perhaps I screwed up in https://review.openstack.org/#/c/407168/ | 16:17 |
| armax | AJaeger: um | 16:17 |
| armax | or maybe you killed a duplication | 16:18 |
| * AJaeger is confused now ;) | 16:18 | |
| AJaeger | armax: let me test something... | 16:18 |
| *** martinkopec has quit IRC | 16:18 | |
| armax | your change landed a few days ago | 16:19 |
| AJaeger | let me check layout logfile... | 16:19 |
| armax | that run I posted is from yesterday | 16:19 |
| armax | AJaeger: I rely on you for this, I am only 50% sure of what I do here :) | 16:19 |
| AJaeger | armax: let me check, I might have been confused due to templates... | 16:20 |
| trown | flaper87: confirmed that fix locally | 16:20 |
| AJaeger | armax: indeed job exists and is part of neutron-lib. | 16:20 |
| AJaeger | armax: I'll change my -1 to a +2 now | 16:20 |
| armax | AJaeger: phew | 16:20 |
| armax | AJaeger: you scared me a little tehre | 16:20 |
| AJaeger | armax: thanks for beeing insistent | 16:20 |
| * AJaeger is only human... | 16:20 | |
| armax | AJaeger: oh, don’t worry about | 16:20 |
| armax | that | 16:21 |
| armax | AJaeger: you should know by now I am pretty stubborn :) | 16:21 |
| *** matrohon has quit IRC | 16:21 | |
| AJaeger | ;) | 16:21 |
| armax | AJaeger: thank you for helping out | 16:21 |
| *** ralonsoh has joined #openstack-infra | 16:21 | |
| AJaeger | armax, you're welcome! | 16:22 |
| stevemar | flaper87: i could have just added ":remote:" :O | 16:22 |
| flaper87 | stevemar: :) | 16:22 |
| flaper87 | I wonder why it didn't fail for these projects before the patch landed | 16:22 |
| stevemar | flaper87: not all projects fail docs job on warnings | 16:23 |
| stevemar | the keystone-specs job will fail on a warning | 16:23 |
| flaper87 | stevemar: gotcha | 16:23 |
| flaper87 | and the new release is broken on this | 16:23 |
| flaper87 | I mean, the failure is a docutils assertion failure so it's not entirely my fault | 16:24 |
| flaper87 | :P | 16:24 |
| flaper87 | the new release broke the gate | 16:24 |
| flaper87 | that being said, I could've noticed the warning but I didn't till now | 16:24 |
| flaper87 | I've sent the email out | 16:24 |
| *** rbuzatu has joined #openstack-infra | 16:27 | |
| *** adrian_otto has joined #openstack-infra | 16:27 | |
| *** jordanP has quit IRC | 16:27 | |
| *** slaweq has quit IRC | 16:28 | |
| *** rbuzatu has quit IRC | 16:29 | |
| *** rbuzatu has joined #openstack-infra | 16:29 | |
| *** andrey-mp has joined #openstack-infra | 16:29 | |
| *** andrey-mp has left #openstack-infra | 16:29 | |
| *** jascott1 has joined #openstack-infra | 16:29 | |
| *** john-dav_ is now known as john-davidge | 16:30 | |
| *** john-davidge has quit IRC | 16:32 | |
| openstackgerrit | Merged openstack-infra/project-config: change networking-sfc tempest and functional tests to voting https://review.openstack.org/406396 | 16:32 |
| openstackgerrit | Merged openstack-infra/project-config: networking-midonet: Add "full" variants for ml2 and v2 jobs https://review.openstack.org/408457 | 16:32 |
| *** john-davidge has joined #openstack-infra | 16:32 | |
| *** kragniz has quit IRC | 16:32 | |
| *** sandanar has quit IRC | 16:33 | |
| *** yamamoto has quit IRC | 16:33 | |
| *** kragniz has joined #openstack-infra | 16:33 | |
| *** kzaitsev_mb has quit IRC | 16:34 | |
| *** jascott1 has quit IRC | 16:36 | |
| *** yamamoto has joined #openstack-infra | 16:37 | |
| jeblair | pabelanger, Shrews: are you up for switching nodepool.o.o to zuulv3? | 16:39 |
| *** esikachev has quit IRC | 16:40 | |
| jeblair | i'll have that system-config doc update done by the end of the day | 16:40 |
| jeblair | but otherwise i think we're ready, and it'd be a shame to waste a good friday :) | 16:40 |
| AJaeger | jeblair: could you review this zuul change for xenial in project-config, please? https://review.openstack.org/408819 | 16:40 |
| *** slaweq has joined #openstack-infra | 16:40 | |
| *** e0ne has quit IRC | 16:41 | |
| *** yamamoto has quit IRC | 16:41 | |
| AJaeger | dinner time - will be briefly back later... | 16:42 |
| jeblair | AJaeger: i had no idea there was a periodic test | 16:42 |
| AJaeger | jeblair: I'm happy to drop it instead - just need guideance from the zuul team ;) | 16:42 |
| jeblair | AJaeger: why the branch matchefr for %zuul-coverage.*$? | 16:43 |
| *** Vivek has quit IRC | 16:43 | |
| jeblair | AJaeger: why the branch matcher for ^zuul-coverage.*$? | 16:43 |
| *** yamamoto has joined #openstack-infra | 16:43 | |
| AJaeger | jeblair: default regex runs trusty jobs on stable/liberty and mitaka - and you want it on all branches | 16:43 |
| *** yamamoto has quit IRC | 16:43 | |
| *** ijw has quit IRC | 16:44 | |
| AJaeger | bbl, will answer questions or review comments then... | 16:44 |
| *** florianf has quit IRC | 16:45 | |
| *** yamamoto has joined #openstack-infra | 16:46 | |
| sc` | would it be possible to land https://review.openstack.org/#/c/253616/ ? it's been in limbo for a while | 16:47 |
| *** clenimar has joined #openstack-infra | 16:49 | |
| jeblair | AJaeger: haha! the zuul periodic test was the *first* periodic job - | 16:49 |
| jeblair | AJaeger: we added it to test that periodic jobs worked at all: https://review.openstack.org/39805 | 16:49 |
| jeblair | AJaeger: i think we can get rid of it now :) | 16:50 |
| *** electrofelix has quit IRC | 16:50 | |
| *** yamamoto has quit IRC | 16:51 | |
| *** esikachev has joined #openstack-infra | 16:51 | |
| openstackgerrit | DavidPurcell proposed openstack-infra/project-config: Add project Patrole to the list of projects. https://review.openstack.org/409254 | 16:52 |
| *** dtantsur is now known as dtantsur|afk | 16:52 | |
| *** cody-somerville has joined #openstack-infra | 16:53 | |
| *** cody-somerville has joined #openstack-infra | 16:53 | |
| *** achevychalov has quit IRC | 16:53 | |
| openstackgerrit | Sean McGinnis proposed openstack-infra/project-config: Limit jobs for Cinder driverfixes branch https://review.openstack.org/393481 | 16:55 |
| *** tqtran has joined #openstack-infra | 16:55 | |
| *** csomerville has quit IRC | 16:56 | |
| *** maeker has joined #openstack-infra | 16:59 | |
| *** rcernin has joined #openstack-infra | 16:59 | |
| fungi | jeblair: agreed, periodic jobs seem to be working fine now ;) | 17:01 |
| *** claudiub|2 has joined #openstack-infra | 17:01 | |
| *** asettle has quit IRC | 17:02 | |
| *** esikachev has quit IRC | 17:02 | |
| *** xarses has joined #openstack-infra | 17:03 | |
| *** asettle has joined #openstack-infra | 17:03 | |
| *** pcaruana has quit IRC | 17:03 | |
| *** xarses_ has joined #openstack-infra | 17:03 | |
| *** claudiub has quit IRC | 17:04 | |
| *** _Cyclone_ has joined #openstack-infra | 17:05 | |
| *** dizquierdo has quit IRC | 17:05 | |
| *** asettle__ has joined #openstack-infra | 17:05 | |
| openstackgerrit | Miles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package https://review.openstack.org/344214 | 17:05 |
| *** xarses has quit IRC | 17:07 | |
| *** asettle has quit IRC | 17:07 | |
| *** xarses_ is now known as xarses | 17:08 | |
| *** esikachev has joined #openstack-infra | 17:09 | |
| *** derekh has quit IRC | 17:09 | |
| *** asettle__ has quit IRC | 17:09 | |
| *** esikachev has quit IRC | 17:13 | |
| *** esikachev has joined #openstack-infra | 17:14 | |
| *** yamamoto has joined #openstack-infra | 17:15 | |
| *** mhickey has quit IRC | 17:15 | |
| *** kzaitsev_mb has joined #openstack-infra | 17:15 | |
| *** sdake_ has joined #openstack-infra | 17:18 | |
| *** yamamoto has quit IRC | 17:18 | |
| *** esikachev has quit IRC | 17:19 | |
| JayF | the dependant patch for https://review.openstack.org/#/c/409188/ just merged, if someone doesn't mind rekicking it into the gate | 17:19 |
| JayF | Thanks! | 17:19 |
| *** esikachev has joined #openstack-infra | 17:20 | |
| *** kzaitsev_mb has quit IRC | 17:20 | |
| * clarkb looks | 17:20 | |
| clarkb | sc`: I think pabelanger was volunteering to work on that? I would ping him and see what needs doing (I am guessing AFS things and code review) | 17:21 |
| *** sdake has quit IRC | 17:21 | |
| *** yamahata has quit IRC | 17:21 | |
| *** ihrachys has joined #openstack-infra | 17:22 | |
| *** nicolasbock has quit IRC | 17:23 | |
| *** nicolasbock has joined #openstack-infra | 17:23 | |
| *** esikachev has quit IRC | 17:24 | |
| dhellmann | fungi : should I put this spec "completed" patch on the infra meeting agenda for review? I'm not clear on the process for marking it "done" - https://review.openstack.org/404917 | 17:25 |
| *** nherciu_ has quit IRC | 17:25 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Make dib-python use the default python for distro https://review.openstack.org/408288 | 17:25 |
| *** jascott1 has joined #openstack-infra | 17:25 | |
| clarkb | speaking of marking specs completed | 17:26 |
| clarkb | fungi: ^ you ready to mark the xenialification spec/priority effort done? | 17:26 |
| *** chlong has quit IRC | 17:27 | |
| *** jascott1 has quit IRC | 17:28 | |
| sc` | clarkb: ack. i'll follow up with him | 17:28 |
| clarkb | sc`: I am doing a quick review of the change now but likely won't approve as I don't think I have time to do the afs bits today (have family visiting town) | 17:29 |
| openstackgerrit | Merged openstack-infra/project-config: Set noop dhcp provider for ironicclient func tests https://review.openstack.org/409188 | 17:29 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Replace swift capabilities call with REST https://review.openstack.org/408865 | 17:29 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Honor image_api_version when doing version discovery https://review.openstack.org/409270 | 17:29 |
| *** Swami has joined #openstack-infra | 17:29 | |
| *** jascott1 has joined #openstack-infra | 17:29 | |
| sc` | quite alright. we've been waiting a while for this one, a little bit longer won't hurt | 17:29 |
| jeblair | mordred: are you around to help me with nodepool? | 17:31 |
| *** nherciu has joined #openstack-infra | 17:31 | |
| *** apuimedo|away is now known as apuimedo | 17:31 | |
| *** hashar has quit IRC | 17:31 | |
| *** nherciu_ has joined #openstack-infra | 17:32 | |
| *** bhavik1 has quit IRC | 17:32 | |
| fungi | dhellmann: i just approved it as administrivia--thanks for the heads up | 17:33 |
| dhellmann | fungi : ack, thanks | 17:33 |
| fungi | clarkb: yep, let's put a nail in it. remember to remove from priority effort queries as well | 17:34 |
| fungi | or want me to propose it? | 17:34 |
| clarkb | I can propose it | 17:34 |
| clarkb | just wanted to make sure there weren't any last items you wanted related to it | 17:34 |
| mordred | jeblair: yes! | 17:34 |
| *** yamamoto has joined #openstack-infra | 17:34 | |
| fungi | clarkb: nope, seems done to me | 17:34 |
| jeblair | mordred: yay! | 17:34 |
| mordred | jeblair: gimme just a sec to finish pushing up the patch I'm working on | 17:35 |
| mordred | (so I don't lose my context) | 17:35 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Switch nodepool to zuulv3 https://review.openstack.org/409272 | 17:35 |
| jeblair | mordred: ok | 17:35 |
| *** kzaitsev_mb has joined #openstack-infra | 17:35 | |
| jeblair | mordred: just let me know when you're ready | 17:35 |
| *** nherciu has quit IRC | 17:36 | |
| *** yamamoto has quit IRC | 17:36 | |
| *** tphummel has quit IRC | 17:36 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/infra-specs: Mark the newton on xenial spec as done https://review.openstack.org/409273 | 17:38 |
| clarkb | fungi: ^ there it is | 17:38 |
| fungi | thanks! | 17:38 |
| clarkb | jeblair: mordred does that mean that hard shutdown and restart of zookeeper was acceptable? | 17:38 |
| openstackgerrit | Merged openstack-infra/infra-specs: mark the branch-automation spec as implemented https://review.openstack.org/404917 | 17:39 |
| clarkb | curious how long that took with our 10k transaction count | 17:39 |
| *** yamamoto has joined #openstack-infra | 17:39 | |
| *** tosky has quit IRC | 17:39 | |
| jeblair | clarkb: i think pabelanger said it was down for ... 10 seconds? or something of that order of magnitude. it's possible most of that was him typing. would need him to confirm. | 17:40 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Honor image_api_version when doing version discovery https://review.openstack.org/409270 | 17:40 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Change fixtures to use https https://review.openstack.org/409275 | 17:40 |
| jeblair | clarkb: i don't think that tells us much about what will happen with nodepool in the future. we should synthetic-test that. | 17:40 |
| *** kzaitsev_mb has quit IRC | 17:40 | |
| jeblair | clarkb: but he shut it down a few times during critical phases and it recovered as expected. | 17:40 |
| *** arxcruz has quit IRC | 17:40 | |
| mordred | jeblair: okie. patch done / context flushed - ready for new context injection | 17:41 |
| *** esikachev has joined #openstack-infra | 17:42 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Switch to Xenial: storyboard https://review.openstack.org/408821 | 17:43 |
| *** strigazi is now known as strigazi_AFK | 17:43 | |
| *** pblaho has quit IRC | 17:44 | |
| jeblair | mordred: okay roll up your sleeve... and review https://review.openstack.org/409272 | 17:44 |
| jeblair | mordred: i think we just need to approve that and restart nodepool after it lands | 17:44 |
| mordred | jeblair: i dunno man. I'm not sure I've got enough headspace for that one | 17:44 |
| jeblair | mordred: oh 1 sec... | 17:45 |
| *** Jeffrey4l has quit IRC | 17:45 | |
| jeblair | it's possible we may need to make some changes to nodepool.yaml | 17:45 |
| jeblair | let's see about switching the project-config test for that file over to the zuulv3 branch first | 17:45 |
| clarkb | sc`: pabelanger EmilienM can you see comment on 253616, I don't think puppet will automagic that | 17:46 |
| mordred | jeblair: ++ | 17:47 |
| openstackgerrit | Merged openstack-infra/infra-specs: Mark the newton on xenial spec as done https://review.openstack.org/409273 | 17:47 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Fixed entity events timezone query https://review.openstack.org/409276 | 17:48 |
| jeblair | i'm blanking -- how does nodepool get installed for the tox 'nodepool' target in project-config? | 17:48 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Switch to Xenial: Zuul https://review.openstack.org/408819 | 17:48 |
| *** nadya has quit IRC | 17:48 | |
| jeblair | oh i see it | 17:48 |
| AJaeger | jeblair: updated the zuul change ^ | 17:48 |
| mordred | jeblair: wait - which nodepool.yaml changes are you thinking we need to make? we have zk related things in there already | 17:49 |
| clarkb | jeblair: via test-requirements.txt | 17:49 |
| jeblair | clarkb: ya | 17:49 |
| *** esikachev has quit IRC | 17:49 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources: Fixed entity events timezone query https://review.openstack.org/409276 | 17:49 |
| jeblair | mordred: well, we dropped some unecessary things from nodepool.yaml; i'm not positive our current nodepool.yaml will config validate | 17:49 |
| fungi | did something happen recently to our private hiera git repo on the puppetmaster? git log only has one commit from november 23 adding the nodepool hiera group and then complains about a loose object | 17:49 |
| mordred | jeblair: gotcha | 17:50 |
| fungi | fatal: loose object 7565d354711a8f2ad771cb7b7641d481e158565e (stored in .git/objects/75/65d354711a8f2ad771cb7b7641d481e158565e) is corrupt | 17:50 |
| clarkb | I haven't touched it since I copied the translate-dev file a couple weeks ago | 17:50 |
| clarkb | it seemed fine then | 17:50 |
| mordred | fungi: I haven't touched it in a while - but that certainly makes me think of quickly making a backup :) | 17:50 |
| jeblair | ugh | 17:50 |
| fungi | nevermind! :/ | 17:50 |
| fungi | i forgot my root perms | 17:50 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement major upgrade for Newton to Ocata https://review.openstack.org/404831 | 17:50 |
| jeblair | whew | 17:51 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Removed cache from summit list endpoint https://review.openstack.org/409277 | 17:51 |
| * fungi calls off red alert siren | 17:51 | |
| mordred | fungi: it still makes me want to ensure a solid backup due to panic | 17:51 |
| fungi | please do! | 17:51 |
| *** sdake_ has quit IRC | 17:51 | |
| jeblair | so it's currently "-e git://git.openstack.org/openstack-infra/nodepool#egg=nodepool | 17:51 |
| AJaeger | clarkb: https://review.openstack.org/408821 and https://review.openstack.org/408819 are the only xenial leftovers - nothing to hold off marking the spec as done | 17:51 |
| jeblair | in test-requirements.txt | 17:51 |
| jeblair | mordred, clarkb: can we convince that to use the zuulv3 branch? | 17:51 |
| clarkb | I believe so let me look at pip doc really quickly | 17:52 |
| mordred | yah - one sec | 17:52 |
| openstackgerrit | Merged openstack-infra/openstackid-resources: Removed cache from summit list endpoint https://review.openstack.org/409277 | 17:52 |
| jeblair | @zuulv3 ? | 17:52 |
| *** tphummel has joined #openstack-infra | 17:52 | |
| clarkb | nodepool@$refname#egg=nodepool | 17:52 |
| clarkb | ya | 17:52 |
| *** rcernin has quit IRC | 17:52 | |
| *** rcernin has joined #openstack-infra | 17:52 | |
| *** yamamoto has quit IRC | 17:52 | |
| mordred | @feature/zuulv3 no? | 17:52 |
| jeblair | oh yeah :) | 17:52 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Switch nodepool validation to zuulv3 branch https://review.openstack.org/409280 | 17:53 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Switch nodepool validation to zuulv3 branch https://review.openstack.org/409280 | 17:54 |
| jeblair | ps2 adds a minor change to nodepool.yaml to make it self testing. | 17:54 |
| pabelanger | jeblair: Shrews: yes, I am ready now. | 17:54 |
| mordred | jeblair: woot | 17:54 |
| jeblair | pabelanger: we're looking at 409280 (especially the test results) and then 409272 | 17:55 |
| pabelanger | just catching up on backscroll | 17:55 |
| *** ijw has joined #openstack-infra | 17:55 | |
| pabelanger | clarkb: sc`: yes, I think we can work on that for monday | 17:55 |
| *** larainema has quit IRC | 17:56 | |
| pabelanger | jeblair: clarkb: yes, zookeeper was stopped for about 10 seconds, I didn't notice any delay on stop / start | 17:56 |
| *** yamamoto has joined #openstack-infra | 17:56 | |
| pabelanger | jeblair: ack, looking now | 17:57 |
| Shrews | jeblair: if you and pabelanger are comfortable with it, sure | 17:57 |
| *** yamamoto has quit IRC | 17:58 | |
| *** e0ne has joined #openstack-infra | 17:58 | |
| *** ijw has quit IRC | 17:59 | |
| pabelanger | jeblair: mordred: clarkb: we also need to land 405549, puppet-nodepool change | 17:59 |
| jeblair | pabelanger: ++ | 18:00 |
| mordred | pabelanger: safe to land land now? | 18:00 |
| *** yamamoto has joined #openstack-infra | 18:00 | |
| AJaeger | clarkb: smcginnis change https://review.openstack.org/#/c/393481 is still way to aggressive, it overrides far too many jobs ;( | 18:00 |
| *** chlong has joined #openstack-infra | 18:01 | |
| pabelanger | mordred: think so, yes | 18:01 |
| jeblair | mordred, pabelanger: http://logs.openstack.org/80/409280/2/check/gate-project-config-nodepool/a12ff86/console.html seems happy | 18:02 |
| jeblair | i'll go ahead and approve that | 18:02 |
| mordred | jeblair: ++ | 18:02 |
| jeblair | and i guess 409272 as well | 18:02 |
| pabelanger | agreed | 18:02 |
| mordred | \o/ | 18:03 |
| *** knangia has quit IRC | 18:04 | |
| *** yamahata has joined #openstack-infra | 18:04 | |
| *** mmandel has joined #openstack-infra | 18:05 | |
| AJaeger | mriedem, sdague_: Are you happy to have python35 voting on nova? https://review.openstack.org/408416 | 18:05 |
| openstackgerrit | Merged openstack-infra/puppet-nodepool: Remove --no-images from nodepool https://review.openstack.org/405549 | 18:05 |
| *** rbrndt has quit IRC | 18:06 | |
| mriedem | AJaeger: i think it's still racy | 18:06 |
| mriedem | b/c of mox | 18:06 |
| mriedem | so i think i'd rather see it non-voting to start | 18:07 |
| *** mmandel_ has quit IRC | 18:07 | |
| AJaeger | mriedem: It's non-voting, so, please -1 408416 which makes it voting | 18:07 |
| mriedem | since it hasn't been running in the gate queue i don't have a good idea as to how stable it is | 18:08 |
| mriedem | so i haven't -1ed | 18:08 |
| mriedem | check queue results on it wouldn't be representative simply due to busted patches | 18:09 |
| Shrews | jeblair: pabelanger: i don't suppose those remaining nodepool patches are critical, but would make ops things nicer | 18:09 |
| *** andymaier has quit IRC | 18:09 | |
| mriedem | AJaeger: how about we hold off until monday? | 18:09 |
| mriedem | i don't like making a job voting on a friday | 18:09 |
| mriedem | especially when i plan on sitting on my lazy butt most of this afternoon | 18:09 |
| openstackgerrit | Merged openstack-infra/project-config: Switch nodepool validation to zuulv3 branch https://review.openstack.org/409280 | 18:10 |
| AJaeger | mriedem: I'm fine waiting for your explicit +1 on it... | 18:11 |
| pabelanger | Shrews: agree, nice to have | 18:11 |
| *** ijw has joined #openstack-infra | 18:11 | |
| pabelanger | everything I reported has been merged at this point | 18:11 |
| jeblair | i'll review them now! :) | 18:11 |
| openstackgerrit | Merged openstack-infra/system-config: Switch nodepool to zuulv3 https://review.openstack.org/409272 | 18:11 |
| jeblair | and that's the actual ^ | 18:11 |
| mordred | woot! | 18:12 |
| sc` | pabelanger: awesome, thanks | 18:12 |
| *** maeker has quit IRC | 18:13 | |
| *** esikachev has joined #openstack-infra | 18:13 | |
| AJaeger | smcginnis, tonyb : could you +1 https://review.openstack.org/#/c/407463/ , please? | 18:13 |
| AJaeger | (or -1 as appropriate ;) | 18:14 |
| *** jgriffith_AutoAw is now known as jgriffith | 18:14 | |
| jeblair | now we wait for puppet | 18:16 |
| *** maeker has joined #openstack-infra | 18:16 | |
| mordred | jeblair: it's my favorite activity! | 18:17 |
| *** rcernin has quit IRC | 18:17 | |
| *** jerryz has joined #openstack-infra | 18:18 | |
| openstackgerrit | Merged openstack-infra/nodepool: Add __repr__ to ConfigValue objects https://review.openstack.org/408776 | 18:19 |
| openstackgerrit | Merged openstack-infra/nodepool: Check for in progress build/upload in CLI https://review.openstack.org/408794 | 18:19 |
| AJaeger | while you're waiting, do you have time to review https://review.openstack.org/#/c/408819/ and https://review.openstack.org/#/c/408821, please? That finishes the xenial changes... | 18:20 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool: Re-use build data when we set for DELETING https://review.openstack.org/408808 | 18:21 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Stop telling users to check logs https://review.openstack.org/409290 | 18:21 |
| mordred | saw that ^^ while looking in to a test failure | 18:21 |
| *** yamamoto has quit IRC | 18:22 | |
| clarkb | pabelanger: sc` EmilienM monday sounds great. I have a comment on the change though that someone more puppety might want to double check | 18:22 |
| *** asettle has joined #openstack-infra | 18:24 | |
| *** dtardivel has quit IRC | 18:24 | |
| *** asettle has quit IRC | 18:26 | |
| mordred | AJaeger, clarkb: did something change in devstack defaults wrt swift? | 18:26 |
| *** asettle has joined #openstack-infra | 18:26 | |
| clarkb | mordred: not that I am aware of | 18:26 |
| mordred | hrm. shade ansible tests just started failing on unable to create object-store client - on patches that didn't touch that | 18:27 |
| notmyname | mordred: I'm not sure, but maybe the default size of the swift storage area? but that would have been 2-3 months ago | 18:27 |
| pabelanger | clarkb: ack, I'll poke around shortly | 18:28 |
| mordred | notmyname: yah - this just started this morning | 18:28 |
| mordred | notmyname: I'll look in further and let you know if it's anything related to swift or devstack itself | 18:28 |
| AJaeger | mordred: I'm not watching devstack itself - but I'm not aware of anything in project-config for it | 18:28 |
| notmyname | mordred: ok. no idea on my side. FWIW there were some ec library releases yesterday | 18:28 |
| mordred | notmyname: https://review.openstack.org/#/c/407767/ is an example of a patch where the ansible jobs started failing | 18:29 |
| mordred | there sure does not seem to be swift running there | 18:30 |
| notmyname | mordred: nothing's obvious to me there. but i've got a meeting right now. let me know if you find something | 18:30 |
| clarkb | mordred: there should still be one for glance in a default setup as far as I know | 18:31 |
| mordred | yah - the swift services are not running in that devstack | 18:31 |
| mordred | but they were yesterday | 18:31 |
| mordred | sdague_: ^^ you aware of any changes to devstack defaults WRT swift? | 18:31 |
| *** harlowja has joined #openstack-infra | 18:32 | |
| clarkb | mordred: http://logs.openstack.org/67/407767/2/check/gate-shade-dsvm-ansible/6262a99/logs/localrc.txt.gz they are enabled | 18:33 |
| mordred | clarkb: 5ede934 Merge "Disable horizon, swift and ceilometer" | 18:34 |
| *** harlowja_ has joined #openstack-infra | 18:34 | |
| clarkb | what project? git show on devstack with that doesn't give me anything | 18:34 |
| *** harlowja has quit IRC | 18:35 | |
| mordred | http://logs.openstack.org/67/407767/2/check/gate-shade-dsvm-ansible/6262a99/logs/devstacklog.txt.gz#_2016-12-09_15_53_59_792 | 18:35 |
| mordred | me either | 18:35 |
| clarkb | oh that could be a zuul ref | 18:35 |
| mordred | this is check queue - why would there be an unmerged devstack ref in the history? | 18:36 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add fedora-24 voting jobs https://review.openstack.org/409295 | 18:36 |
| clarkb | mordred: depends on? but ya less likely in shade | 18:36 |
| jeblair | 2016-12-09 15:56:16.006 | ++ functions-common:git_clone:565 : cd /opt/stack/new/magnum | 18:36 |
| jeblair | http://git.openstack.org/cgit/openstack/magnum/log/ | 18:36 |
| jeblair | http://git.openstack.org/cgit/openstack/magnum/commit/?id=5ede934aafa1629149168af376f8ada1ae2d7145 | 18:36 |
| jeblair | it's the current magnum tip | 18:36 |
| mordred | OH - that's in _magnum_ ??? | 18:36 |
| mordred | adrian_otto: heya | 18:37 |
| clarkb | ah yes swift is not used (except it is...) | 18:37 |
| clarkb | http://git.openstack.org/cgit/openstack/magnum/commit/?id=67d71104bbd5122187232460026f654c4edd9228 is the change side of the merge commit | 18:37 |
| mordred | https://review.openstack.org/409296 is the revert I just pushed up | 18:39 |
| clarkb | ok now that xenialing is done, I want ot do a complete restart of the elasticsearch cluster to try and get the cluster to recover and go green | 18:40 |
| clarkb | if anyone is interested in how that works I am happy to walk you through it elsewise I will be doing that momentarily | 18:41 |
| JayF | ES clusters are pretty fun to maintain :) such self-healing | 18:42 |
| clarkb | JayF: except for when they don't self heal like now | 18:42 |
| clarkb | pretty sure its some index allocation lock that is deadlocked | 18:42 |
| *** ihrachys has quit IRC | 18:43 | |
| JayF | The cluster I worked on was hooked up to graylog, and we typically fixed the indexes that wouldn't self heal with a curl -X DELETE and starting over :x | 18:43 |
| jeblair | mordred, pabelanger, Shrews: nodepool.o.o git is updated and the init scripts. puppet has finished running there. i don't see any errors. | 18:43 |
| clarkb | and basically causes certain replica shards to refuse to find a home, they round robin across all the valid hosts and failed in a loop | 18:43 |
| pabelanger | \o/ | 18:43 |
| clarkb | JayF: last time this happened a full cluster restart seemed to correct it (which is why I think a lock is involved) | 18:43 |
| JayF | clarkb: yeah, there weren't many issues a full cluster restart couldn't fix ime, that more what I meant by self-healing | 18:44 |
| clarkb | ah, I want it to self heal while running :) | 18:44 |
| jeblair | mordred, pabelanger, Shrews: i think we're ready to restart nodepool there. shall i? | 18:44 |
| pabelanger | ready here | 18:45 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Remove magnum from shade's gate https://review.openstack.org/409302 | 18:45 |
| mordred | jeblair, clarkb: ^^ | 18:45 |
| mordred | jeblair: yes! | 18:45 |
| mordred | jeblair: restart away! | 18:45 |
| * mordred is excited | 18:45 | |
| *** jpena is now known as jpena|off | 18:46 | |
| *** PramodJayathirth has joined #openstack-infra | 18:46 | |
| Shrews | yup | 18:46 |
| Shrews | jeblair: oh, that last change had the db failure | 18:47 |
| Shrews | just rechecked, but it isn't 100% necessary | 18:47 |
| jeblair | ya, and only affects builders | 18:47 |
| jeblair | so we can restart them later after it lands | 18:48 |
| jeblair | okay, i'm stopping nodepool-builder on nodepool.o.o | 18:49 |
| clarkb | ok cluster restarted and recovering | 18:49 |
| jeblair | (it's idle currently ftr) | 18:49 |
| mordred | notmyname: tl;dr - it wasn't you - it was the magnum devstack plugin | 18:49 |
| jeblair | oh, hah, it wasn't idle. oh well. :) | 18:50 |
| clarkb | jeblair: its safe to kill those things | 18:50 |
| jeblair | yep | 18:50 |
| mordred | jeblair: whoopsie :) | 18:50 |
| clarkb | jeblair: sometimes the dib process will hang around if it isn the middle of the big file mv it does but once the mv is done it catches the signal and goes away | 18:50 |
| *** pramodrj07 has joined #openstack-infra | 18:51 | |
| jeblair | yeah, i'm going to leave the dib running to clean up itself | 18:51 |
| *** HeOS has joined #openstack-infra | 18:51 | |
| jeblair | i'm going to restart the deleter now (while the launcher is still running). this should be a noop, but just want to make sure, and it's the low hanging fruit. | 18:52 |
| jeblair | okay, it's restarted | 18:52 |
| jeblair | restarting launcher now | 18:52 |
| * mordred does a little dance | 18:53 | |
| *** gordc has quit IRC | 18:53 | |
| pabelanger | get down tonight | 18:53 |
| jeblair | it doesn't seem to be seeing any demand | 18:54 |
| pabelanger | hmm | 18:55 |
| mordred | jeblair: it seems lke it just saw some? | 18:56 |
| jeblair | well, that's not true, it does see demand | 18:56 |
| jeblair | okay, maybe it's just actually keeping up really well right now? :) | 18:57 |
| mordred | yah. I'm not sure I've seen it this healthy in a while :) | 18:57 |
| jeblair | yeah, there go a bunch of nodes | 18:57 |
| clarkb | #status log performed full elasticsearch cluster restart in an attempt to get it to fully recover and go green. Previously was yellow for days unable to initialize some replica shards. Recovery of shards in progress now. | 18:57 |
| openstackstatus | clarkb: finished logging | 18:57 |
| jeblair | i'm going to poke at a ready node and make sure it was launched from a new image | 18:59 |
| notmyname | mordred: ack | 19:00 |
| clarkb | jeblair: openstack server show on the instance will give you image uuid | 19:00 |
| jeblair | ya, halfway there :) | 19:01 |
| *** jcoufal_ has joined #openstack-infra | 19:01 | |
| jeblair | | image | template-centos-7-1481234326 (72532bf3-f575-43f9-8694-148d377edab7) | | 19:01 |
| jeblair | | 0000000001 | 0000000001 | osic-cloud1-s3500 | centos-7 | template-centos-7-1481234326 | 72532bf3-f575-43f9-8694-148d377edab7 | ready | 00:20:33:18 | | 19:01 |
| jeblair | that looks good :) | 19:01 |
| mordred | \o/ | 19:01 |
| clarkb | neat | 19:02 |
| pabelanger | Yay | 19:02 |
| *** mriedem has quit IRC | 19:03 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable #openstack-meeting-5 https://review.openstack.org/409054 | 19:03 |
| jeblair | #status log launcher/deleter on nodepool.o.o are now running the zuulv3 branch. zookeeper based nodepool builders (nb01, nb02) are in production | 19:03 |
| openstackstatus | jeblair: finished logging | 19:03 |
| *** jcoufal has quit IRC | 19:03 | |
| *** ijw has quit IRC | 19:04 | |
| *** eharney has quit IRC | 19:04 | |
| fungi | and there was much rejoicing! | 19:05 |
| Shrews | phase 1 complete | 19:05 |
| *** ijw has joined #openstack-infra | 19:05 | |
| Shrews | phase 2 is get drunk in a celebratory manner, fwiw | 19:06 |
| jeblair | :) | 19:06 |
| fungi | it's friday, so i'm already feeling relatively celebratory | 19:06 |
| pabelanger | indeed, awesome work everybody | 19:06 |
| *** rbrndt has joined #openstack-infra | 19:07 | |
| mordred | Shrews: ++ | 19:07 |
| mordred | pabelanger: ++ | 19:07 |
| jeblair | pabelanger, Shrews: thank you :) | 19:07 |
| fungi | zuul v3 draws closer every day | 19:08 |
| asselin | congrat! :) | 19:09 |
| *** Sukhdev has joined #openstack-infra | 19:09 | |
| *** tnovacik has joined #openstack-infra | 19:09 | |
| *** trown is now known as trown|lunch | 19:09 | |
| *** ijw has quit IRC | 19:10 | |
| * Shrews waits for the spec to move from "Priority Effort" to "Implemented" | 19:10 | |
| pabelanger | ++ | 19:10 |
| fungi | at which point we'll probably have the zuul v4 spec all ready to review ;) | 19:11 |
| *** david-lyle_ is now known as david-lyle | 19:13 | |
| jeblair | i'm going to afk for a little bit | 19:15 |
| mordred | jeblair: enjoy afking | 19:15 |
| fungi | jeblair: Shrews: pabelanger: thanks for getting our initial use of zk into place! | 19:15 |
| *** asettle has quit IRC | 19:16 | |
| *** asettle has joined #openstack-infra | 19:16 | |
| *** ralonsoh has quit IRC | 19:16 | |
| *** john-davidge has quit IRC | 19:17 | |
| *** john-davidge has joined #openstack-infra | 19:17 | |
| pabelanger | fungi: big thanks to jeblair and Shrews for making it easy to use! | 19:18 |
| amrith | hi infra, just cleaning up my laundry list and noticed https://review.openstack.org/#/c/401459/. any chance someone can review and approve a minor infra manual change. thx | 19:18 |
| *** kzaitsev_mb has joined #openstack-infra | 19:21 | |
| *** asettle has quit IRC | 19:21 | |
| *** john-davidge has quit IRC | 19:22 | |
| openstackgerrit | Merged openstack-infra/system-config: Enable #openstack-meeting-5 https://review.openstack.org/409053 | 19:22 |
| *** ldnunes has joined #openstack-infra | 19:23 | |
| *** jgriffith is now known as jgriffith_AutoAw | 19:24 | |
| fungi | amrith: lgtm. thanks for the correction! | 19:26 |
| *** esikachev has quit IRC | 19:27 | |
| openstackgerrit | Merged openstack-infra/nodepool: Re-use build data when we set for DELETING https://review.openstack.org/408808 | 19:28 |
| *** gordc has joined #openstack-infra | 19:28 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Allow package-installs to parse python_version https://review.openstack.org/408293 | 19:29 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Move pip-and-virtualenv source install to 10- https://review.openstack.org/408277 | 19:29 |
| openstackgerrit | xiaodongwang991481 proposed openstack-infra/project-config: add multinode tempest test coverage in networking-sfc https://review.openstack.org/391979 | 19:29 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add install-types as pip-and-virtualenv dep https://review.openstack.org/408248 | 19:29 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Fix pip-and-virtualenv to work with python3 https://review.openstack.org/407723 | 19:29 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Create ubuntu/fedora test for pip-and-virtualenv https://review.openstack.org/407769 | 19:29 |
| openstackgerrit | Merged openstack-infra/infra-manual: update the drivers.rst for project retirement steps https://review.openstack.org/401459 | 19:30 |
| *** esikachev has joined #openstack-infra | 19:31 | |
| adrian_otto | mordred: ?? | 19:31 |
| mordred | adrian_otto: I word-vomitted in #openstack-containers - but tl;dr magnum devstack plugin patch broke shade gate | 19:31 |
| *** ijw has joined #openstack-infra | 19:32 | |
| amrith | thx fungi | 19:33 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Remove magnum from shade's gate https://review.openstack.org/409302 | 19:33 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Remove python-glanceclient from shade git libs https://review.openstack.org/408154 | 19:33 |
| mordred | AJaeger: ^^ fixed - thanks | 19:34 |
| adrian_otto | mordred: I see https://review.openstack.org/409302 | 19:34 |
| adrian_otto | mordred: is there a way for the magnum tests to run without swift, and other projects to run with it? | 19:35 |
| adrian_otto | the problem is that we can't run our own tests because of resource constraints in the gate | 19:36 |
| mordred | adrian_otto: yes - we just need to turn off swift in the job configs | 19:36 |
| mordred | adrian_otto: I can make you a patch to do that if you like | 19:36 |
| notmyname | adrian_otto: just curious, what are the constraints and why is swift causing you to blow through them? | 19:37 |
| mordred | limited disk space, yeah? | 19:37 |
| adrian_otto | notmyname: disk use is part of it, and we are really tight on memory overhead | 19:37 |
| notmyname | oh, interesting. is swift the abuser of memory there? or is it "oh here's something that's not needed so every little bit helps"? | 19:38 |
| adrian_otto | we also intermittently have slow test runs that we don't understand. Things that shoudl take 30 minutes take ~2 hours and the jobs get killed off and fail from timeout | 19:38 |
| adrian_otto | notmyname: it's part of a strategy to identify all non-essentialy components. We have not singled out swift as an abuser at all. | 19:39 |
| notmyname | adrian_otto: ok. like I said, I'm just curious. if you do find something, of course I'd be happy to take a look :-) | 19:39 |
| adrian_otto | mordred: I'd be thrilled if you could make that patch | 19:39 |
| adrian_otto | notmyname: thanks!! | 19:39 |
| clarkb | adrian_otto: fwiw swift is typically used to back glance so in that manner its "required" ish | 19:40 |
| *** jgriffith_AutoAw is now known as jgriffith | 19:40 | |
| clarkb | for compute workloads | 19:40 |
| adrian_otto | we might have received enough relief from disabling the other bits to see about putting swift back in. We do have a use case for it, which is why it was added to begin with. We use it as a backing store for local docker container repositories (so you don't have to use docker hub if you don't want to). | 19:41 |
| *** pvaneck has joined #openstack-infra | 19:42 | |
| *** openstack has joined #openstack-infra | 19:45 | |
| *** rbrndt has quit IRC | 19:46 | |
| mordred | adrian_otto: openstackgerrit | Monty Taylor proposed openstack/magnum: Disable horizon, ceilomter and swift in gate hook https://review.openstack.org/409296 | 19:46 |
| adrian_otto | mordred: thanks | 19:47 |
| *** jcoufal_ has quit IRC | 19:47 | |
| *** ijw has quit IRC | 19:48 | |
| *** rbuzatu has quit IRC | 19:49 | |
| *** ijw has joined #openstack-infra | 19:49 | |
| *** ijw has quit IRC | 19:49 | |
| *** ihrachys has joined #openstack-infra | 19:54 | |
| *** dkehn__ has quit IRC | 19:54 | |
| *** esikachev has quit IRC | 19:55 | |
| *** dkehn__ has joined #openstack-infra | 19:55 | |
| AJaeger | fungi, thanks - adding to one of the reviews.... | 19:55 |
| *** sdague__ has joined #openstack-infra | 19:56 | |
| openstackgerrit | DavidPurcell proposed openstack-infra/project-config: Add patrole projects. https://review.openstack.org/409317 | 19:56 |
| *** sdake has joined #openstack-infra | 19:57 | |
| *** esberglu has quit IRC | 19:59 | |
| *** sdague__ has quit IRC | 19:59 | |
| *** esberglu has joined #openstack-infra | 19:59 | |
| clarkb | 7 more shards to recover | 20:01 |
| *** esberglu has quit IRC | 20:04 | |
| *** masber has quit IRC | 20:07 | |
| *** rbuzatu has joined #openstack-infra | 20:07 | |
| *** rbuzatu has quit IRC | 20:07 | |
| *** rbuzatu has joined #openstack-infra | 20:07 | |
| *** masber has joined #openstack-infra | 20:07 | |
| *** esberglu_ has joined #openstack-infra | 20:09 | |
| *** erlon-airlong has quit IRC | 20:10 | |
| *** ssbarnea has quit IRC | 20:14 | |
| clarkb | zaro: reading gerrit mailing list there appear to be auth issues with 2.13 and we may need to update the config on openstack/openstack to have it properly update on submodule merges? | 20:15 |
| clarkb | zxiiro: do you know if those auth issues affect openid? | 20:15 |
| clarkb | er zaro ^, sorry zxiiro | 20:15 |
| *** eharney has joined #openstack-infra | 20:16 | |
| *** john-davidge has joined #openstack-infra | 20:18 | |
| *** spzala has quit IRC | 20:18 | |
| *** thorst__ has quit IRC | 20:18 | |
| *** hwoarang has quit IRC | 20:18 | |
| *** thorst has joined #openstack-infra | 20:19 | |
| *** ldnunes has quit IRC | 20:20 | |
| *** jkilpatr has quit IRC | 20:21 | |
| *** john-davidge has quit IRC | 20:23 | |
| *** hwoarang has joined #openstack-infra | 20:24 | |
| *** ssbarnea has joined #openstack-infra | 20:25 | |
| *** e0ne has quit IRC | 20:26 | |
| *** thorst has quit IRC | 20:27 | |
| *** rbuzatu has quit IRC | 20:28 | |
| *** mriedem has joined #openstack-infra | 20:29 | |
| *** john-davidge has joined #openstack-infra | 20:30 | |
| *** raildo has quit IRC | 20:30 | |
| sc` | are the ipv6 links from status.o.o/zuul supposed to work? i can telnet -6 to the address just fine, but chrome won't spawn a terminal | 20:35 |
| clarkb | sc`: you have to set up your browser to process them properly but if you do that yes they work | 20:35 |
| fungi | sc`: do the v4 telnet:// urls work for you in chrome? | 20:35 |
| sc` | fungi: they do | 20:35 |
| sc` | clarkb: ah. some reading ahead then | 20:35 |
| fungi | we tried to set correct v6 address quoting with [] | 20:35 |
| greghaynes | I just got a really odd test failure I dont know what to think of - http://logs.openstack.org/93/408293/8/check/gate-diskimage-builder-python27-ubuntu-xenial/483c3b1/console.html | 20:36 |
| greghaynes | failed right after zuul cloner | 20:36 |
| greghaynes | with no debug info AFAICT | 20:36 |
| fungi | greghaynes: "think i'll have a drink" maybe? | 20:36 |
| greghaynes | wow, thats a much better thought than I had | 20:36 |
| fungi | greghaynes: look at the timestamps | 20:37 |
| greghaynes | aha | 20:37 |
| greghaynes | wierd | 20:37 |
| fungi | i'd go digging in the zuul ansible log next | 20:38 |
| *** rcernin has joined #openstack-infra | 20:38 | |
| *** rcernin has quit IRC | 20:38 | |
| *** trown|lunch is now known as trown|brb | 20:38 | |
| greghaynes | awesome, ty | 20:38 |
| *** trown|brb is now known as trown | 20:38 | |
| * greghaynes goes spelunking | 20:38 | |
| jeblair | yeah, that's about 20 minutes after we all decided to have celebratory drinks after the nodepool rollout | 20:38 |
| fungi | nothing else much exciting in http://logs.openstack.org/93/408293/8/check/gate-diskimage-builder-python27-ubuntu-xenial/483c3b1/_zuul_ansible/ansible_log.txt | 20:38 |
| *** rcernin has joined #openstack-infra | 20:38 | |
| jeblair | so i assume that's zuul just tagging along | 20:38 |
| fungi | but looks like it probably hung on a git remote operation and then the job timed out | 20:39 |
| AJaeger | jeblair, fungi, we have some miscalculation of needed nodes; let me repeat my earlier message: | 20:39 |
| *** esikachev has joined #openstack-infra | 20:39 | |
| greghaynes | fungi: ah, seems plausible | 20:40 |
| fungi | AJaeger: yep, you were asking whether we should restart zuul/nodepool following all the job changes? | 20:40 |
| AJaeger | too long ago ;( | 20:40 |
| AJaeger | yep | 20:40 |
| jeblair | AJaeger: yeah, i saw people talking about restarting zuul -- no one did? | 20:40 |
| fungi | saw the question, forgot to follow up | 20:40 |
| AJaeger | I just checked and still same sitution | 20:40 |
| AJaeger | seems nobody did - looking at current grafana graphs | 20:40 |
| AJaeger | when I asked earlier you were asleep ;) | 20:40 |
| jeblair | nov 22 | 20:41 |
| fungi | luckily we don't seem to be behind on the job queue | 20:41 |
| jeblair | yeah, just assumed with discussion about it, someone would have done it :) | 20:41 |
| jeblair | i will do it now | 20:41 |
| AJaeger | thanks | 20:41 |
| fungi | thanks jeblair! | 20:41 |
| * clarkb is still catching up on the service restart I was saying needed to be done yesterday :P | 20:41 | |
| clarkb | but its almost done now. service is green, shards recovered, and are reloacting | 20:42 |
| *** spzala has joined #openstack-infra | 20:42 | |
| AJaeger | btw. we also have two hold nodes - are those still needed? | 20:42 |
| AJaeger | clarkb: great, thanks! | 20:42 |
| fungi | AJaeger: looking, possibly mine | 20:42 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Revert "Remove magnum from shade's gate" https://review.openstack.org/409326 | 20:43 |
| fungi | nope, mine are all cleaned up | 20:43 |
| fungi | one says "pabelanger" and the other has no comment | 20:43 |
| *** rbrndt has joined #openstack-infra | 20:43 | |
| fungi | centos-7-internap-mtl01-5794819 | 20:43 |
| mordred | I'm going to go ahead and do the shade magnum removal - but there's the revert patch for once the magnum patch lands | 20:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Make diskimage-builder command configurable for testing https://review.openstack.org/404976 | 20:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Properly cleanup failed diskimage builds https://review.openstack.org/409327 | 20:43 |
| *** jkilpatr has joined #openstack-infra | 20:43 | |
| fungi | held for ~11 and ~15 days, respectively | 20:43 |
| AJaeger | pabelanger: do you still need that node? | 20:44 |
| clarkb | I don't think I have any but if my name is on them they can be deleted | 20:44 |
| *** esikachev has quit IRC | 20:44 | |
| pabelanger | AJaeger: fungi: yes, it can be removed | 20:44 |
| fungi | pabelanger: is the other one with no comment yours as well? | 20:44 |
| *** thorst has joined #openstack-infra | 20:44 | |
| pabelanger | fungi: possible, I might have not applied the reason correctly | 20:45 |
| jeblair | the zuul startup delay is a bunch of can_do packets being sent to geard | 20:46 |
| AJaeger | team, I'm approving in ten mins jhesketh's change https://review.openstack.org/#/c/407434/ to remove the custom success/failure messages - we tried to fix the regexes several times and failed. This seems to be the best way forward IMHO. If you disagree, please speak up now. | 20:46 |
| *** spzala has quit IRC | 20:46 | |
| jeblair | AJaeger: absolutely not | 20:46 |
| AJaeger | ;) | 20:47 |
| jeblair | i have -2d that. i will follow up with a reason when i finish restarting zuul | 20:47 |
| *** thorst has quit IRC | 20:47 | |
| AJaeger | Oh, misunderstood your not - ok, will wait... | 20:47 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for ceph vol busy snapshot delete fail bug 1648885 https://review.openstack.org/409330 | 20:48 |
| openstack | bug 1648885 in Cinder "ceph: TestVolumeBootPattern.test_create_ebs_image_and_check_boot times out failing to delete volume due to snapshot is busy" [Undecided,New] https://launchpad.net/bugs/1648885 | 20:48 |
| *** rbrndt has quit IRC | 20:48 | |
| jeblair | AJaeger: those messages are a pretty fundamental zuul concept, and they *far* predate the hideci stuff | 20:49 |
| jeblair | AJaeger: if no one is able to maintain the hideci javascript, *that's* the problem. not those messages. | 20:49 |
| jeblair | zuul is restarted and i'm re-loading the queues | 20:50 |
| *** ijw has joined #openstack-infra | 20:51 | |
| AJaeger | jeblair: I agree that the hideci is the problem and nobody seems able/willing to fix it. | 20:51 |
| jeblair | then let's remove it | 20:51 |
| AJaeger | So, this is the best workaround for now if we keep it. | 20:51 |
| jeblair | no, this is not a workaround | 20:52 |
| *** rbrndt has joined #openstack-infra | 20:52 | |
| greghaynes | can we stick a span or something around the zuul message so hideci can parse an attribute of it? | 20:53 |
| greghaynes | or does gerrit not allow you post html in a comment like that? | 20:53 |
| jeblair | greghaynes: they shouldn't be html, but using the gerrit commentlink, you absolutely can translate it to html (that's what happens in fact) | 20:54 |
| *** EricGonczer_ has joined #openstack-infra | 20:54 | |
| mordred | AJaeger: do you have links to the previous attemps to fix the regex? | 20:55 |
| greghaynes | hrm, maybe im misunderstanding. What I mean is - the issue seems like hideci needs some indicator of 'this is a fail / success', if we can drop in an html attribute in the response message like <span response="success">message</span> then hideci should be able to DTRT | 20:56 |
| *** spzala has joined #openstack-infra | 20:56 | |
| greghaynes | or maybe im missing some other obvious fix | 20:56 |
| bkero | Anybody familiar with zuul-swift-upload have a clue why it would be returning HTTP 401s? | 20:56 |
| AJaeger | mordred: let me grab... | 20:56 |
| jeblair | greghaynes: yeah, i'm saying that's exactly what's already happening with gerrit commentlinks | 20:57 |
| greghaynes | oh | 20:57 |
| AJaeger | mordred: last revert was I99896753342a9082f922df645ece82c5a8af78e7 | 20:57 |
| mordred | AJaeger: thanks! | 20:57 |
| greghaynes | well then | 20:57 |
| * greghaynes does some rtfsing | 20:57 | |
| jeblair | greghaynes: i mean, in principle, obviously not working for the exact thing in question | 20:57 |
| *** masuberu has joined #openstack-infra | 20:58 | |
| greghaynes | jeblair: gotcha | 20:58 |
| greghaynes | I dont mind poking at this a bit, I like to practice some js every now and then | 20:58 |
| *** jamielennox|away is now known as jamielennox | 20:58 | |
| *** kzaitsev_mb has quit IRC | 20:58 | |
| *** zz_ja has quit IRC | 20:58 | |
| *** ekhugen has quit IRC | 20:58 | |
| jeblair | greghaynes: i think the actual thing that needs practicing is regex :) | 20:58 |
| jerryz | jeblair: is zuul fully supporting multiple gerrits now? i am having an issue the change triggered by one gerrit entering wrong pipeline of another gerrit. | 20:59 |
| AJaeger | mordred: not sure whether there were more... | 20:59 |
| jeblair | greghaynes, mordred: that change AJaeger linked is where i think the solution should be | 20:59 |
| *** masber has quit IRC | 20:59 | |
| fungi | yeah, unclear if the previous hideci.js regex adjustments simply ceased matching the right things, or actually resulted in javascript-level errors | 20:59 |
| jeblair | other solutions might be: | 20:59 |
| jeblair | have the custom messages always lead with a SUCCESS or FAILURE | 21:00 |
| jeblair | actually, that might be the easiest :) | 21:00 |
| jeblair | it would fit right in to the pattern which assumes a one word response | 21:00 |
| jeblair | but why doesn't the current regex just highlight the first word as the result? | 21:01 |
| greghaynes | I must be missing something, if were already adding in some html tag we shouldnt really need some crazy regexing - if you add a class to a span around the response you can just have jq grab all instances of that class out of the DOM | 21:02 |
| greghaynes | then grab another attr on those objects to see if they are response/fail messages | 21:02 |
| mordred | greghaynes: it's the mechanism | 21:02 |
| mordred | greghaynes: we add text | 21:02 |
| *** ijw has quit IRC | 21:02 | |
| jeblair | greghaynes: zuul doesn't emit html -- the comment is not html, it's text. gerrit translates those text comments to html using commentlinks (see the change AJaeger linked) | 21:02 |
| *** ekhugen has joined #openstack-infra | 21:02 | |
| mordred | yah, that | 21:03 |
| jeblair | hideci recognizes the resulting html and modifies it further | 21:03 |
| *** ijw has joined #openstack-infra | 21:03 | |
| *** zz_ja has joined #openstack-infra | 21:03 | |
| greghaynes | aha, got it. So we dont actually have the ability to put anything in to the dob | 21:03 |
| greghaynes | er, dom | 21:03 |
| mordred | not directly from zuul, no | 21:03 |
| mordred | our mechanism for doing that is the commentlinks construct | 21:03 |
| mordred | but it's the comment links construct that's breaking down | 21:04 |
| greghaynes | Yea, then I think its what jeblair said re: lead with some known text | 21:04 |
| jeblair | jerryz: it should, can you paste the pipeline section of your layout? | 21:04 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for ceph vol busy snapshot delete fail bug 1648885 https://review.openstack.org/409330 | 21:04 |
| openstack | bug 1648885 in Cinder "ceph: TestVolumeBootPattern.test_create_ebs_image_and_check_boot times out failing to delete volume due to snapshot is busy" [Undecided,New] https://launchpad.net/bugs/1648885 | 21:04 |
| jerryz | jebliar: http://paste.openstack.org/show/591973/ | 21:04 |
| clarkb | '<li>([^ ]+) <a href=\"[^\"]+\" target=\"_blank\" rel=\"nofollow\">([^<]+)</a> : ([^ ]+)([^<]*)</li>' is the comment link match btw | 21:05 |
| jerryz | jebliar: if the trigger criteria is the same as a different pipeline, it may go into that. so i modified the regex for comment-add in my-check, and it can go to my-check nw | 21:05 |
| mordred | anybody happen to have a link to a change that has one of the custom messages in it? | 21:06 |
| clarkb | mordred: look at requirements updates I think those are an example | 21:06 |
| jerryz | jebliar: but patch created received by my_gerrit would still go to check pipeline instead of my-check | 21:06 |
| *** knangia has joined #openstack-infra | 21:07 | |
| AJaeger | current version at http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/review.pp#n180 | 21:08 |
| *** maeker has quit IRC | 21:08 | |
| *** adrian_otto has quit IRC | 21:08 | |
| pabelanger | jerryz: yes, zuul.rdoproject.org is connected to 2 gerrit servers too | 21:08 |
| *** maeker has joined #openstack-infra | 21:08 | |
| jeblair | jerryz: are those two different gerrit servers, or two connections to the same server? | 21:09 |
| *** rlandy has quit IRC | 21:09 | |
| mordred | AJaeger: I'm looking through patches trying to find one with the custom message showing and the hideci not working right - if you happen to have one of those or see one, I'd love to look at one | 21:09 |
| jerryz | jeblair: two different servers | 21:09 |
| *** cody-somerville has quit IRC | 21:09 | |
| jeblair | jerryz: hrm, i can't think of why that would not work :( | 21:10 |
| jerryz | jeblair: here is my zuul conf http://paste.openstack.org/show/591974/ | 21:11 |
| AJaeger | mordred: I'll search... | 21:11 |
| jerryz | jeblair: i am using zuul 2.5.0 | 21:11 |
| AJaeger | mordred: I'm just building one ;) | 21:12 |
| jerryz | jeblair: the trigger events can be the same for different pipelines is that correct? | 21:12 |
| AJaeger | mordred: https://review.openstack.org/#/c/409334/ should be ready in 5 mins if we have nodes - let's see whether that shows the problem | 21:12 |
| mordred | AJaeger: woot! that's a much easier way that what I was doing | 21:13 |
| mordred | :) | 21:13 |
| *** ijw has quit IRC | 21:13 | |
| *** mmalik4 has joined #openstack-infra | 21:13 | |
| jeblair | jerryz: yes, it should -- part of the trigger match should verify that it came from the right connection | 21:14 |
| jeblair | jerryz: we have a test for multiple gerrits: http://git.openstack.org/cgit/openstack-infra/zuul/tree/tests/test_connection.py#n62 | 21:14 |
| jeblair | jerryz: but it looks like it only has one change... | 21:14 |
| jeblair | jerryz: so we may not be testing as much as we need to there | 21:14 |
| *** thorst_ has joined #openstack-infra | 21:14 | |
| jeblair | (maybe it works one way, but not the other way) | 21:15 |
| *** cody-somerville has joined #openstack-infra | 21:16 | |
| *** cody-somerville has quit IRC | 21:16 | |
| *** cody-somerville has joined #openstack-infra | 21:16 | |
| AJaeger | mordred: from backscroll https://review.openstack.org/#/c/388870/ | 21:17 |
| *** Hal has joined #openstack-infra | 21:18 | |
| AJaeger | mordred: patch set 4 | 21:18 |
| *** ijw has joined #openstack-infra | 21:18 | |
| *** Hal is now known as Guest95801 | 21:18 | |
| AJaeger | mordred: and change 400254 | 21:19 |
| *** john-davidge has quit IRC | 21:19 | |
| *** john-dav_ has joined #openstack-infra | 21:19 | |
| jerryz | jeblair: this is what i got http://paste.openstack.org/show/591977/ when patchset created event was received | 21:19 |
| AJaeger | btw. zaro said on 22nd of November that "Toggle CI" like change will be in gerrit 2.14 | 21:19 |
| jerryz | jeblair: it got a wrong independent pipeline manager instance not having the project | 21:20 |
| *** esberglu_ has quit IRC | 21:20 | |
| jeblair | AJaeger: i think in 2.13 we'll be able to add this as structured data | 21:20 |
| *** esberglu has joined #openstack-infra | 21:21 | |
| *** ijw has quit IRC | 21:21 | |
| jeblair | jerryz: right, it shouldn't have said "Adding..." right before the exception -- that happens after it matches the pipeline which it should not have | 21:23 |
| *** ijw has joined #openstack-infra | 21:23 | |
| mordred | AJaeger: awesome. thanks! | 21:23 |
| *** john-dav_ has quit IRC | 21:23 | |
| AJaeger | mordred: I remembered having answered these questions in the past ;) | 21:24 |
| mordred | :) | 21:24 |
| greghaynes | mordred: the fail for me is the rel="nofollow" | 21:24 |
| greghaynes | in the a tag | 21:24 |
| greghaynes | otherwise I think it matches | 21:24 |
| AJaeger | mordred: check https://review.openstack.org/#/c/409334 now | 21:25 |
| jeblair | jerryz: it looks like the eventfilter (which is created by the gerrit trigger) does not actuall match against the trigger source: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/model.py#n1232 | 21:25 |
| *** esberglu has quit IRC | 21:25 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Properly cleanup failed diskimage builds https://review.openstack.org/409327 | 21:25 |
| greghaynes | http://paste.openstack.org/show/591978/ being my little test case | 21:25 |
| *** ssbarnea has quit IRC | 21:26 | |
| jeblair | jerryz: so i think that's a bug, and would explain gerrit events matching every gerrit trigger. and it would slip past that test which doesn't verify that the separate pipelines don't see each others changes | 21:26 |
| *** maeker has quit IRC | 21:27 | |
| greghaynes | also, TIL about gerrit comment matches | 21:27 |
| greghaynes | which was sort of my mental missing link here | 21:27 |
| AJaeger | so, this has a nofollow but others don't have? | 21:28 |
| *** esberglu has joined #openstack-infra | 21:28 | |
| mordred | greghaynes: http://paste.openstack.org/show/591979/ is the text I got from that, yeah? | 21:28 |
| greghaynes | AJaeger: yea it seems odd, itd be really helpful if we could also see the pre-match-replaced version | 21:29 |
| *** chlong has quit IRC | 21:29 | |
| *** rbuzatu has joined #openstack-infra | 21:29 | |
| jeblair | greghaynes: yeah, it's also slightly more complex since gerrit translates the original text a little before the commentlinks | 21:29 |
| greghaynes | mordred: yes, I think that looks the same as what I put in line 4 of my regex match | 21:29 |
| *** rbrndt has quit IRC | 21:29 | |
| greghaynes | jeblair: right, theres I think 3 layers of compilation here, makes is super fun to reason about | 21:30 |
| jeblair | so the full process is zuul_text -> gerrit translation to <ul> -> gerrit commentlink translation -> hideci dom manipulation | 21:30 |
| jeblair | you can see the zuul text with "gerrit query" | 21:30 |
| jeblair | - gate-networking-bgpvpn-releasenotes http://docs-draft.openstack.org/70/388870/8/check/gate-networking-bgpvpn-releasenotes/a23137c//releasenotes/build/html/ : SUCCESS in 3m 00s | 21:30 |
| jeblair | - gate-networking-bgpvpn-requirements http://logs.openstack.org/70/388870/8/check/gate-networking-bgpvpn-requirements/8c0b0ed/ : Incompatible requirement found; see http://docs.openstack.org/developer/requirements/ in 2m 02s | 21:31 |
| *** esberglu_ has joined #openstack-infra | 21:31 | |
| jeblair | but commentlink operates on the initial gerrit html translation of that, which will wrap it in <li> and add <a href> | 21:31 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Properly cleanup failed diskimage builds https://review.openstack.org/409327 | 21:32 |
| greghaynes | ok. It seems really odd to me that gerrit would decide to put a rel="nofollow" only on this one link but AFAICT thats the only thing breaking our regex | 21:32 |
| *** esberglu has quit IRC | 21:33 | |
| greghaynes | and also - it looks like the other links that were matched had a rel="nofollow | 21:33 |
| greghaynes | when they got output | 21:33 |
| jeblair | i think gerrit will add rel=nofollow to all of them | 21:33 |
| jeblair | yeah | 21:33 |
| greghaynes | I dont see how our RE should match that though since it comes after the target=blank | 21:34 |
| jeblair | it's just that when you look at gerrit and you see a list item that did not match the commentlink, that's when you'll see gerrit's manipulation, but not the commentlink manipulation | 21:34 |
| mordred | yah - which is what I put into that paste link above | 21:34 |
| jeblair | i don't know of a way to see gerrit-manipulated versions of the results that *do* match the commentlink, other than turning it off or going to review-dev | 21:34 |
| mordred | so the question is - why does the regex in question not match the text that's in the pastebin | 21:34 |
| jerryz | jeblair: thanks for the debugging. i will see if i can fix that. | 21:34 |
| AJaeger | team, I'll sign off now and wish you a great weekend. Would be nice to have the final xenial changes in: https://review.openstack.org/408819 and https://review.openstack.org/408821 - please review if you still have time after fixing Toggle CI;) | 21:35 |
| greghaynes | mordred: for me its purely the rel=nofollow - re fails until I change the regex to have that | 21:35 |
| jeblair | jerryz: that would be awesome, thanks! jhesketh has some expertise in this area too, btw. he's in australia. | 21:35 |
| greghaynes | I just agree that it seems the other links have it so #confused | 21:35 |
| jlvillal | Is it unusual that if I did a recheck on a patch over 2 hours ago, that http://status.openstack.org/zuul/ would say the job has been running for 44 minutes? This is patch set 397775 | 21:36 |
| ihrachys | gerrit seems very slow, hanging on "Working..." for minutes here. is it just me, or the buddy needs a reset? | 21:36 |
| *** esberglu_ has quit IRC | 21:36 | |
| jeblair | jlvillal: i restarted zuul | 21:36 |
| jlvillal | jeblair: Ah. Thanks! | 21:36 |
| *** esberglu has joined #openstack-infra | 21:37 | |
| jeblair | jlvillal: ... yeah, about 45 minutes ago. np :) | 21:37 |
| * jlvillal was confused :) | 21:37 | |
| *** adrian_otto has joined #openstack-infra | 21:37 | |
| mordred | greghaynes: I'm confused though - because the regex has rel='nofollow' | 21:38 |
| mordred | greghaynes: where do you mean? | 21:38 |
| greghaynes | mordred: on https://review.openstack.org/#/c/313125/1/modules/openstack_project/manifests/gerrit.pp it doesnt | 21:38 |
| greghaynes | did we add that? | 21:38 |
| jeblair | greghaynes: it does in current prod: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/review.pp#n180 | 21:38 |
| greghaynes | oh derp | 21:39 |
| greghaynes | so yea, it matches for me locally then :/ | 21:39 |
| jeblair | greghaynes: i think rel=nofollow came with a recent gerrit upgrade | 21:39 |
| *** adrianofr_ has quit IRC | 21:39 | |
| mordred | greghaynes: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/review.pp#n180 <-- I'm looking at that | 21:39 |
| greghaynes | oh hey, so that RE doesnt match in my little test | 21:40 |
| * greghaynes pokes | 21:40 | |
| greghaynes | just doing http://paste.openstack.org/show/591980/ | 21:41 |
| *** esberglu has quit IRC | 21:42 | |
| jeblair | is it because the requiremnts message has another url? | 21:42 |
| *** rbuzatu has quit IRC | 21:42 | |
| jeblair | i'd like to see one of the other messages -- like the docs link warning | 21:42 |
| jeblair | (it doesn't have a url in the custom message) | 21:42 |
| mordred | greghaynes, jeblair: my theory is the other url | 21:42 |
| mordred | but I'm not to proof yet :) | 21:43 |
| jeblair | >>> print r.match('<li>gate-networking-bgpvpn-requirements <a href="http://logs.openstack.org/70/388870/4/check/gate-networking-bgpvpn-requirements/0b14b82/" target="_blank" rel="nofollow">http://logs.openstack.org/70/388870/4/check/gate-networking-bgpvpn-requirements/0b14b82/</a> : Incompatible requirement found; see in 2m 32s</li>').groups() | 21:43 |
| jeblair | ('gate-networking-bgpvpn-requirements', 'http://logs.openstack.org/70/388870/4/check/gate-networking-bgpvpn-requirements/0b14b82/', 'Incompatible', ' requirement found; see in 2m 32s') | 21:43 |
| greghaynes | I removed the other url and it matched | 21:43 |
| jeblair | yeah that's what happens if i just cut out the other url | 21:43 |
| mordred | greghaynes: I'm doing this: http://paste.openstack.org/show/591981/ | 21:43 |
| mordred | ++ | 21:43 |
| jeblair | (and i think that would do what i expected, which is highlight the first word as the test result: "Incompatible", which is weird, but something. :) | 21:43 |
| AJaeger | yeah, should be the other URL. We have "([^<]*)" - so that does not match a second URL | 21:44 |
| *** ihrachys has quit IRC | 21:44 | |
| AJaeger | I mean: It fails because of the other URL due to ([^<]*) in the match | 21:44 |
| jeblair | i *can* get behind us changing the messages not to include urls :) | 21:44 |
| greghaynes | ah! and thats probably so we dont get too greedy? | 21:44 |
| mordred | well... | 21:45 |
| mordred | yah | 21:45 |
| AJaeger | we want to match until the first </li> | 21:45 |
| mordred | the first regex is too greedy | 21:45 |
| *** jeckersb is now known as jeckersb_gone | 21:45 | |
| jeblair | i mean, we could try to fix the regex to support 2+ urls, but if we just wanted to say "custom messages should start with a one word test result and can not have urls, i would +2 that. | 21:45 |
| mordred | it's matching all the way to the end of the second url | 21:45 |
| *** catherineD has joined #openstack-infra | 21:45 | |
| greghaynes | hah | 21:46 |
| *** mdrabe has quit IRC | 21:46 | |
| pvaneck | fungi: Hey, after updating refstack to the latest release, https://refstack.openstack.org seems to be having SSL issues. Can you see what's up in the logs? | 21:46 |
| greghaynes | mordred: thats odd because we have a ^< | 21:46 |
| mordred | greghaynes: yah | 21:47 |
| * AJaeger really signs off - bye | 21:47 | |
| greghaynes | mordred: oh is the href matching all of it? | 21:49 |
| *** claudiub|2 has quit IRC | 21:49 | |
| mordred | greghaynes: yah - lemme show you my ... | 21:49 |
| mordred | greghaynes: so if I just do this: | 21:49 |
| mordred | regex='<li>([^ ]+) <a href="[^"]+" target="_blank" rel="nofollow">([^<]+)</a> : ' | 21:49 |
| mordred | replace="""'<li class=\"comment_test\"><span class=\"comment_test_name\"><a href=\"$2\" rel=\"nofollow\">$1</a></span>'""" | 21:49 |
| mordred | I get this: | 21:49 |
| mordred | '<li class="comment_test"><span class="comment_test_name"><a href="$2" rel="nofollow">$1</a></span>'Incompatible requirement found; see <a href="http://docs.openstack.org/developer/requirements/" target="_blank" rel="nofollow">http://docs.openstack.org/developer/requirements/</a> in 2m 32s</li> | 21:49 |
| *** dave-mccowan has quit IRC | 21:50 | |
| mordred | greghaynes: nevermind- I needed more things in the first regex | 21:50 |
| *** esberglu has joined #openstack-infra | 21:52 | |
| greghaynes | so, how do we feel about negative lookarounds :p | 21:56 |
| clarkb | does the java regex engine support them? | 21:56 |
| mordred | it's a combo of the two | 21:56 |
| mordred | things | 21:56 |
| mordred | I've got a working regex | 21:56 |
| mordred | I mean "working" regex | 21:56 |
| greghaynes | coolio | 21:56 |
| *** ijw has quit IRC | 21:56 | |
| mordred | that does the thing jeblair expected this one _should_ do | 21:57 |
| *** dingyichen has quit IRC | 21:57 | |
| mordred | so I think we should do it - and then add FAILURE to the custom message | 21:57 |
| greghaynes | I replaced the last ^< with ((?!</li>).)* and it works for my super small case, but you seem to be actually testing things | 21:57 |
| mordred | '<li>([^ ]+) <a href="[^"]+" target="_blank" rel="nofollow">([^<]+)</a> : ([^ ]+)(.*)</li>' | 21:57 |
| *** spzala has quit IRC | 21:57 | |
| mordred | produces | 21:57 |
| mordred | '<li class="comment_test"><span class="comment_test_name"><a href="http://logs.openstack.org/70/388870/4/check/gate-networking-bgpvpn-requirements/0b14b82/" rel="nofollow">gate-networking-bgpvpn-requirements</a></span> <span class=\"comment_test_result\"><span class=\"result_Incompatible\">Incompatible</span> requirement found; see <a href="http://docs.openstack.org/developer/requirements/" | 21:57 |
| mordred | target="_blank" rel="nofollow">http://docs.openstack.org/developer/requirements/</a> in 2m 32s</span></li>' | 21:57 |
| *** rbrndt has joined #openstack-infra | 21:58 | |
| *** ihrachys has joined #openstack-infra | 21:58 | |
| greghaynes | sweet | 21:58 |
| *** ijw has joined #openstack-infra | 21:58 | |
| mordred | jeblair: ^^ that output work for you? | 21:58 |
| greghaynes | will that .* not get too greedy because newlines? | 21:59 |
| mordred | it's not too greedy because the </li> anchors the regex | 21:59 |
| mordred | I think | 21:59 |
| mordred | man it's been WAY too long since I was a perl dev :) | 21:59 |
| clarkb | #status log elasticsearch has finished shard recovery and relocation. Cluster is now green | 22:00 |
| openstackstatus | clarkb: finished logging | 22:00 |
| jeblair | mordred: yeah, i think that will work, and then we can add an extra word to the custom message if we don't like "Incompatible" being highlighted as the result | 22:00 |
| greghaynes | yea but if you have two of them separated between newlines | 22:00 |
| mordred | yah - I think it's line by line | 22:00 |
| mordred | rather than multi-line | 22:01 |
| greghaynes | ok, good deal | 22:01 |
| *** lindycoder has quit IRC | 22:01 | |
| *** chem has quit IRC | 22:02 | |
| *** chlong has joined #openstack-infra | 22:02 | |
| *** spzala has joined #openstack-infra | 22:03 | |
| *** rcernin has quit IRC | 22:04 | |
| *** thorst_ has quit IRC | 22:04 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Allow links in the message https://review.openstack.org/409344 | 22:04 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Update Nodepool docs with ZK info https://review.openstack.org/409345 | 22:04 |
| *** jamielennox is now known as jamielennox|away | 22:04 | |
| *** thorst_ has joined #openstack-infra | 22:04 | |
| jeblair | clarkb, pabelanger, fungi: ^ does 409345 lgty? | 22:05 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Use os-net-config in multinode jobs https://review.openstack.org/409346 | 22:05 |
| *** gordc has quit IRC | 22:05 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Allow package-installs to parse python_version https://review.openstack.org/408293 | 22:06 |
| jeblair | greghaynes, mordred: thank you :) | 22:06 |
| *** srobert_ has joined #openstack-infra | 22:06 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add FAILURE to requirements failure messages https://review.openstack.org/409347 | 22:06 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config: Add FAILURE to requirements failure messages https://review.openstack.org/409347 | 22:06 |
| mordred | jeblair: yay regex fun! | 22:06 |
| *** spzala has quit IRC | 22:07 | |
| greghaynes | many mental cobwebs were dusted | 22:07 |
| *** tnovacik has quit IRC | 22:07 | |
| *** vponomaryov1 has left #openstack-infra | 22:07 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Move pip-and-virtualenv source install to 10- https://review.openstack.org/408277 | 22:07 |
| mordred | greghaynes: dude. where did all this dust come from? | 22:08 |
| jeblair | mordred: did i mention that somewhere between the all-bash afs zuul-launcher publisher, and our new afs module, i had a perl script doing a bunch of stream parsing? | 22:08 |
| mordred | jeblair: it is the right thing to use perl for :) | 22:08 |
| *** trown is now known as trown|outtypewww | 22:09 | |
| mordred | that could sound negative - what I mean is "there are no tools better for that job than perl" | 22:09 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add install-types as pip-and-virtualenv dep https://review.openstack.org/408248 | 22:09 |
| jeblair | mordred: yeah, but sadly, even that started to look ridiculous so it didn't make the cut :| | 22:10 |
| *** srobert has quit IRC | 22:10 | |
| mordred | booo | 22:10 |
| clarkb | jeblair: yes those docs look good | 22:10 |
| *** srobert_ has quit IRC | 22:10 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:10 | |
| *** jamielennox|away is now known as jamielennox | 22:11 | |
| jeblair | urns | 22:11 |
| *** thorst_ has quit IRC | 22:13 | |
| *** ijw has quit IRC | 22:13 | |
| *** jesusaur has quit IRC | 22:15 | |
| openstackgerrit | Merged openstack-infra/system-config: Update Nodepool docs with ZK info https://review.openstack.org/409345 | 22:17 |
| catherineD | Hi, https://refstack.openstack.org site is down at the moment after updating refstack to the latest release. Could some one help to check the log on SSL issue? Thanks! | 22:17 |
| jeblair | catherineD: i'll take a look | 22:18 |
| catherineD | jeblair: Thank you so much! | 22:18 |
| *** asettle has joined #openstack-infra | 22:19 | |
| *** john-davidge has joined #openstack-infra | 22:20 | |
| jeblair | catherineD: do you know then this started? | 22:21 |
| jeblair | [Fri Dec 09 20:41:24.933961 2016] [ssl:warn] [pid 4571:tid 140085832283904] AH01990: PRNG still contains insufficient entropy! | 22:21 |
| jeblair | there are a lot of those errors | 22:21 |
| *** gouthamr has quit IRC | 22:23 | |
| *** Guest95801 has quit IRC | 22:23 | |
| *** fguillot has quit IRC | 22:23 | |
| jeblair | yeah, every time i make a request another of those shows up | 22:23 |
| jeblair | so i assume that's related | 22:23 |
| *** ihrachys has quit IRC | 22:23 | |
| catherineD | jeblair: we updated refstack about 2 hours ago ... we expect a reboot after that | 22:24 |
| *** asettle has quit IRC | 22:24 | |
| *** john-davidge has quit IRC | 22:24 | |
| jeblair | catherineD: that lines up -- the host hasn't rebooted, but apache was restarted at 20:13 and the errors appear shortly after that. | 22:25 |
| jeblair | catherineD: did anything about the apache configuration change? | 22:25 |
| pvaneck | jeblair: nothing apache related changed as far as we know | 22:26 |
| catherineD | jeblair: I do not think so | 22:26 |
| pvaneck | jeblair: it has been a while since the last apache restart before today though | 22:26 |
| *** pradk has quit IRC | 22:28 | |
| *** jamesdenton has quit IRC | 22:28 | |
| *** annegentle has quit IRC | 22:29 | |
| jeblair | catherineD, pvaneck: erm. okay, i restarted apache and it seems to work now. i have no explanation, which is not very satisfying. but the service is up, so yay? :) | 22:32 |
| jeblair | oh wait! now the errors have returned | 22:32 |
| jeblair | that makes even less sense | 22:32 |
| mordred | jeblair: is the server missing that package we have to install for randomness? | 22:33 |
| *** devkulkarni has joined #openstack-infra | 22:33 | |
| jeblair | mordred: haveged? i don't think we install it everywhere, and i think apache should use urandom anyway... | 22:34 |
| jeblair | catherineD, pvaneck: oh i just found that this is showing up in the error log: | 22:35 |
| jeblair | extern "Python": function Cryptography_rand_status() called, but @ffi.def_extern() was not called in the current subinterpreter. Returning 0. | 22:36 |
| jeblair | extern "Python": function Cryptography_rand_bytes() called, but @ffi.def_extern() was not called in the current subinterpreter. Returning 0. | 22:36 |
| jeblair | it was being buffered and it doesn't have a timestamp, so i didn't realize it was current | 22:36 |
| jeblair | catherineD, pvaneck: i wonder if that could provide a causal link between updating to the latest refstack and getting this error from apache (somehow) | 22:37 |
| mordred | oh - yah - that does sound like a potential python cryptography thing | 22:38 |
| jeblair | ./mods-available/ssl.conf:SSLRandomSeed connect builtin | 22:38 |
| jeblair | ./mods-available/ssl.conf:SSLRandomSeed connect file:/dev/urandom 512 | 22:38 |
| jeblair | mordred: ^ that's what makes me think apache should be reading from urandom, which isn't blocking (i double checked) | 22:38 |
| mordred | yah. agree | 22:38 |
| *** rbuzatu has joined #openstack-infra | 22:39 | |
| jeblair | catherineD, pvaneck: can we roll refstack back to the previous version? | 22:39 |
| devkulkarni | clarkb, mriedem: is it a good to chat? .. solum's dsvm-ubuntu-xenial gate (gate is currently failing with "No hosts found to map to cell, exiting" error arising from lib/nova:create_cell) .. if you have some time, would like to discuss how to resolve it.. here is the failure log: http://logs.openstack.org/09/408409/4/check/gate-solum-devstack-dsvm-ubuntu-xenial/a41ada3/logs/devstacklog.txt.gz#_2016-12-09_07_04_07_953 | 22:40 |
| devkulkarni | s/good/good time/ | 22:40 |
| catherineD | jeblair: having a discussion with pvaneck: on how to do that | 22:40 |
| pvaneck | jeblair, hmm could be something to do with pyopenssl and the libffi-dev package that it depends on | 22:41 |
| jeblair | catherineD, pvaneck, mordred: https://etherpad.openstack.org/p/PLi6Jjl5Rg | 22:41 |
| mordred | pvaneck: mind if I ask what you're using pyopenssl for? (I'm grepping the source but can't find it) | 22:41 |
| clarkb | devkulkarni: sure? to me that sounds like a nova cells issue, nto sure how much help I can be | 22:42 |
| jeblair | catherineD, pvaneck: do we deploy refstack from tags? so did we just upgrade from 1.1.4 to 1.2.0? | 22:42 |
| mordred | oh my. I see instructions in your docs that should change ... | 22:43 |
| catherineD | yes deployment is from tags... the update is from 1.1.4 to 1.2.0 | 22:43 |
| devkulkarni | clarkb: ok, I just wanted to make sure that it is not related to the xenial gate change that you had mentioned will be turned on for all projects on Dec 6th (per http://lists.openstack.org/pipermail/openstack-dev/2016-November/106906.html) | 22:43 |
| *** armax has quit IRC | 22:44 | |
| *** tqtran has quit IRC | 22:44 | |
| *** jogo has quit IRC | 22:44 | |
| *** Swami has quit IRC | 22:45 | |
| openstackgerrit | Adam Harwell proposed openstack/diskimage-builder: Fix pip-and-virtualenv to work with python3 https://review.openstack.org/407723 | 22:45 |
| openstackgerrit | Adam Harwell proposed openstack/diskimage-builder: Create ubuntu/fedora test for pip-and-virtualenv https://review.openstack.org/407769 | 22:45 |
| *** hurgleburgler has quit IRC | 22:45 | |
| *** armax has joined #openstack-infra | 22:46 | |
| clarkb | devkulkarni: I mean it may be | 22:46 |
| clarkb | devkulkarni: but if it is nova likely needs to be fixed | 22:46 |
| pvaneck | jeblair, right we upgrade from tags, and did just upgrade to 1.20. and not sure about pyopenssl even being necessary | 22:46 |
| devkulkarni | clarkb: looks like others ran into this issue as well today http://stackoverflow.com/questions/41064739/openstack-no-hosts-found-to-map-to-cell-exiting | 22:47 |
| mordred | pvaneck, jeblair: I do not see anywhere in the code where pyopenssl is being used | 22:47 |
| *** cinerama has quit IRC | 22:47 | |
| *** tqtran has joined #openstack-infra | 22:47 | |
| clarkb | devkulkarni: the trivial fix would be to stop using cells | 22:47 |
| *** cinerama has joined #openstack-infra | 22:47 | |
| clarkb | devkulkarni: is there a reason solum needs to be tested with cells? if not I would just diasble them | 22:48 |
| mordred | pvaneck: oh - you make outbound requests calls it seems? | 22:48 |
| mordred | to talk to openstackid | 22:49 |
| devkulkarni | clarkb: yes, that can be done.. but this commit makes cells default option https://github.com/openstack-dev/devstack/commit/f166081d0e4a3b04b157a25e94f0bb2a1769f813 | 22:49 |
| mordred | so it would need ssl for that. k. I grok now | 22:49 |
| clarkb | devkulkarni: ah in that case ya I would talk to nova | 22:50 |
| *** EricGonczer_ has quit IRC | 22:51 | |
| *** jogo has joined #openstack-infra | 22:51 | |
| *** jogo has joined #openstack-infra | 22:51 | |
| devkulkarni | clarkb: I guess so.. thanks clarkb | 22:52 |
| *** rbuzatu has quit IRC | 22:53 | |
| *** xyang1 has quit IRC | 22:55 | |
| jeblair | pvaneck, catherineD, mordred: perhaps not directly related but similar: https://github.com/pyca/cryptography/issues/2299 | 22:56 |
| *** baoli has quit IRC | 22:56 | |
| *** mtanino has quit IRC | 22:59 | |
| *** sdake has quit IRC | 22:59 | |
| jeblair | also https://github.com/pyca/pyopenssl/issues/542 | 23:00 |
| *** sdake has joined #openstack-infra | 23:01 | |
| jeblair | pvaneck, catherineD: i'd like to restart apache with the wsgi application commented out just to verify that this is a wsgi app issue and not an apache one | 23:01 |
| *** tpsilva has quit IRC | 23:01 | |
| catherineD | jeblair: please go ahead | 23:02 |
| pvaneck | jeblair: sure go right ahead | 23:02 |
| jeblair | okay, that's returning the default apache page over ssl successfully | 23:03 |
| jeblair | including after several reload attempts | 23:03 |
| jeblair | restarting again with the wsgi app back in place | 23:04 |
| mordred | jeblair: think running | 23:05 |
| mordred | from cryptography.hazmat.backends.openssl.backend import backend | 23:05 |
| mordred | backend.activate_builtin_random() | 23:05 |
| mordred | is worth trying? | 23:05 |
| jeblair | refstack seemed to work, and then i tried logging in, and now the error has returned. that exercises the requests/crypto path, so i think that points in the direction we've been heading | 23:06 |
| jeblair | mordred: yeah | 23:06 |
| jeblair | mordred: i think maybe there's a way we can put that in the apache config? | 23:06 |
| wznoinsk | hi infra | 23:06 |
| wznoinsk | what do you use (and is it public) to feed data from zuul to graphite? | 23:07 |
| jeblair | mordred: (or am i thinking of mod_perl) | 23:07 |
| jeblair | wznoinsk: zuul has statsd support built in; we configure our zuul to emit statsd to a statsd server which stores info in graphite | 23:07 |
| clarkb | wznoinsk: zuul has built in statsd support, you just have to configure it and zuul does the rest | 23:07 |
| pvaneck | jeblair: wondering what would happen if you just pip uninstall pyopenssl | 23:07 |
| wznoinsk | jeblair, clarkb great stuff, thanks | 23:08 |
| jeblair | pvaneck: i can try that | 23:08 |
| jeblair | pvaneck, mordred: i uninstalled pyopennsl, restarted, authenticated, and it still workes | 23:09 |
| pvaneck | jeblair, yea working fine for me now | 23:10 |
| mordred | neat! | 23:10 |
| jeblair | i don't know what that implies for what requests is using | 23:10 |
| mordred | I think cryptography has several different backends | 23:10 |
| mordred | and if you have pyopenssl it'll use that, but uses something else if you don't | 23:11 |
| mordred | jeblair: didn't we have this problem on another server recently? | 23:11 |
| clarkb | yes nodepool builder? | 23:11 |
| jeblair | #status log manually ran "pip uninstall pyopenssl" on refstack.openstack.org to resolve a problem with requests/cryptography/pyopenssl/mod_wsgi | 23:11 |
| *** thorst has joined #openstack-infra | 23:11 | |
| openstackstatus | jeblair: finished logging | 23:11 |
| clarkb | except that was about an import path | 23:11 |
| *** piet has joined #openstack-infra | 23:13 | |
| jeblair | pvaneck: i believe the server will stay in that state until refstack is installed again, but you might consider removing pyopenssl from requirements.txt until it works. | 23:13 |
| pvaneck | jeblair, yea, looking into the reasoning why we had it in the first place. Likely it will be removed. | 23:14 |
| pvaneck | jeblair, thanks a bunch for the assistance | 23:14 |
| jeblair | pvaneck: no problem. maybe someday we'll all understand this. or maybe we'll all just start writing c++. :) | 23:15 |
| pvaneck | haha maybe | 23:16 |
| catherineD | jeblair: mordred: thank you sooo much! you have a nice weekend! (and we will have a nice one too :-) ) | 23:17 |
| *** thorst has quit IRC | 23:18 | |
| openstackgerrit | Ivan Udovichenko proposed openstack-infra/puppet-apps_site: Ensure that memcached service is installed and running https://review.openstack.org/407100 | 23:18 |
| *** mmalik4 has quit IRC | 23:18 | |
| *** priteau has quit IRC | 23:20 | |
| *** priteau has joined #openstack-infra | 23:20 | |
| *** priteau has quit IRC | 23:21 | |
| openstackgerrit | Mike Dorman proposed openstack-infra/jenkins-job-builder: Add support for Saltstack plugin builder https://review.openstack.org/391567 | 23:25 |
| openstackgerrit | Ivan Udovichenko proposed openstack-infra/puppet-apps_site: Ensure that memcached service is installed and running https://review.openstack.org/407100 | 23:26 |
| *** chlong has quit IRC | 23:28 | |
| openstackgerrit | Mike Dorman proposed openstack-infra/jenkins-job-builder: Add support for Saltstack plugin builder https://review.openstack.org/391567 | 23:32 |
| *** adrian_otto has quit IRC | 23:33 | |
| *** rbrndt has quit IRC | 23:35 | |
| *** rbrndt has joined #openstack-infra | 23:35 | |
| *** rbrndt has quit IRC | 23:36 | |
| *** esikachev has joined #openstack-infra | 23:37 | |
| *** xarses has quit IRC | 23:40 | |
| *** esikachev has quit IRC | 23:42 | |
| *** esikachev has joined #openstack-infra | 23:42 | |
| openstackgerrit | Mike Dorman proposed openstack-infra/jenkins-job-builder: Add support for Saltstack plugin builder https://review.openstack.org/391567 | 23:42 |
| *** esikachev has quit IRC | 23:47 | |
| openstackgerrit | Adam Gandelman proposed openstack-infra/zuul: Re-enable test_build_configuration_conflict https://review.openstack.org/409376 | 23:48 |
| *** esberglu has quit IRC | 23:49 | |
| *** rbuzatu has joined #openstack-infra | 23:50 | |
| *** cj_ has joined #openstack-infra | 23:52 | |
| *** jgriffith is now known as jgriffith_AutoAw | 23:52 | |
| *** cj_ has quit IRC | 23:54 | |
| openstackgerrit | Mike Dorman proposed openstack-infra/jenkins-job-builder: Add support for Saltstack plugin builder https://review.openstack.org/391567 | 23:56 |
| *** catherineD has left #openstack-infra | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!