| *** salv-orlando has quit IRC | 00:02 | |
| *** hongbin has quit IRC | 00:17 | |
| *** yamahata has quit IRC | 00:19 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Add help links to /var/lib/mistral https://review.openstack.org/521671 | 00:25 |
|---|---|---|
| *** ijw has joined #openstack-infra | 00:28 | |
| *** efoley_ is now known as efoley | 00:31 | |
| *** Apoorva has joined #openstack-infra | 00:32 | |
| *** sree has joined #openstack-infra | 00:33 | |
| *** ijw has quit IRC | 00:35 | |
| *** efoley has quit IRC | 00:35 | |
| *** Apoorva has quit IRC | 00:36 | |
| *** sree has quit IRC | 00:37 | |
| *** tosky has quit IRC | 00:46 | |
| *** zhurong has joined #openstack-infra | 00:46 | |
| *** masber has quit IRC | 00:47 | |
| *** camunoz has quit IRC | 00:53 | |
| *** salv-orlando has joined #openstack-infra | 00:59 | |
| *** yamahata has joined #openstack-infra | 01:00 | |
| *** jascott1 has joined #openstack-infra | 01:06 | |
| *** xingchao has joined #openstack-infra | 01:08 | |
| *** salv-orlando has quit IRC | 01:10 | |
| *** xingchao has quit IRC | 01:18 | |
| *** jascott1 has quit IRC | 01:18 | |
| *** jascott1 has joined #openstack-infra | 01:18 | |
| *** Daisy_ has joined #openstack-infra | 01:19 | |
| *** jascott1 has quit IRC | 01:23 | |
| *** ijw has joined #openstack-infra | 01:31 | |
| *** liujiong has joined #openstack-infra | 01:35 | |
| *** ijw has quit IRC | 01:36 | |
| *** Goneri has quit IRC | 01:38 | |
| *** ihrachys has quit IRC | 01:40 | |
| *** yamamoto has quit IRC | 01:52 | |
| *** puiterwijk|mnt is now known as puiterwijk | 01:52 | |
| *** cuongnv has joined #openstack-infra | 01:54 | |
| *** cuongnv_ has joined #openstack-infra | 01:55 | |
| *** cuongnv has quit IRC | 01:55 | |
| *** yamamoto has joined #openstack-infra | 01:57 | |
| EmilienM | pabelanger: any idea why https://review.openstack.org/#/c/521967 doesn't work? I don't want puppet3 jobs running on stable/ocata, does zuul use layout from master? | 01:57 |
| *** yamamoto has quit IRC | 01:57 | |
| *** xingchao has joined #openstack-infra | 02:01 | |
| *** xingchao has quit IRC | 02:02 | |
| *** salv-orlando has joined #openstack-infra | 02:07 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: github: add integration documentation https://review.openstack.org/522420 | 02:10 |
| *** salv-orlando has quit IRC | 02:13 | |
| *** liujiong has quit IRC | 02:16 | |
| *** Qiming has quit IRC | 02:16 | |
| *** Qiming has joined #openstack-infra | 02:17 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Consume collect-logs directly from zuul/post playbook https://review.openstack.org/522400 | 02:20 |
| *** mriedem has quit IRC | 02:23 | |
| *** yamamoto has joined #openstack-infra | 02:25 | |
| *** yamamoto has quit IRC | 02:28 | |
| *** udesale has joined #openstack-infra | 02:29 | |
| *** masber has joined #openstack-infra | 02:38 | |
| *** Wei_Liu has joined #openstack-infra | 02:39 | |
| *** ykarel|away has joined #openstack-infra | 02:40 | |
| *** masuberu has joined #openstack-infra | 02:42 | |
| *** masber has quit IRC | 02:47 | |
| *** ramishra has joined #openstack-infra | 02:54 | |
| *** reed_ has joined #openstack-infra | 02:59 | |
| *** sree has joined #openstack-infra | 03:02 | |
| *** salv-orlando has joined #openstack-infra | 03:09 | |
| *** ykarel|away has quit IRC | 03:12 | |
| *** ricolin has joined #openstack-infra | 03:12 | |
| *** Qiming has quit IRC | 03:12 | |
| *** salv-orlando has quit IRC | 03:14 | |
| *** Qiming has joined #openstack-infra | 03:14 | |
| *** hongbin has joined #openstack-infra | 03:21 | |
| *** zhurong has quit IRC | 03:24 | |
| *** iyamahat has joined #openstack-infra | 03:28 | |
| *** zhurong has joined #openstack-infra | 03:28 | |
| *** yamamoto has joined #openstack-infra | 03:29 | |
| *** yamamoto has quit IRC | 03:32 | |
| *** yamamoto has joined #openstack-infra | 03:32 | |
| *** dhinesh_ has quit IRC | 03:42 | |
| *** bobh has quit IRC | 03:45 | |
| *** lbragstad has joined #openstack-infra | 03:46 | |
| *** Daisy_ has quit IRC | 03:47 | |
| *** Daisy_ has joined #openstack-infra | 03:47 | |
| *** yamamoto has quit IRC | 03:47 | |
| *** zhurong has quit IRC | 03:47 | |
| *** Daisy_ has quit IRC | 03:48 | |
| *** yamamoto has joined #openstack-infra | 03:50 | |
| *** bobh has joined #openstack-infra | 03:51 | |
| *** lbragstad has quit IRC | 03:52 | |
| *** tinwood has quit IRC | 03:55 | |
| *** erlon has quit IRC | 03:56 | |
| *** tinwood has joined #openstack-infra | 03:56 | |
| *** bobh has quit IRC | 03:57 | |
| *** Dinesh_Bhor has joined #openstack-infra | 04:03 | |
| *** lbragstad has joined #openstack-infra | 04:04 | |
| *** bobh has joined #openstack-infra | 04:06 | |
| *** salv-orlando has joined #openstack-infra | 04:10 | |
| *** coolsvap has joined #openstack-infra | 04:10 | |
| *** bobh has quit IRC | 04:11 | |
| *** rlandy|rover|bbl is now known as rlandy|rover | 04:13 | |
| *** salv-orlando has quit IRC | 04:16 | |
| *** Daisy_ has joined #openstack-infra | 04:16 | |
| openstackgerrit | liusheng proposed openstack-infra/project-config master: Switch to pep8 jobs for mogan-specs https://review.openstack.org/522436 | 04:18 |
| *** Daisy_ has quit IRC | 04:20 | |
| openstackgerrit | liusheng proposed openstack-infra/project-config master: Switch to pep8 jobs for mogan-specs https://review.openstack.org/522436 | 04:21 |
| openstackgerrit | Rajat Sharma proposed openstack-infra/project-config master: Remove legacy jobs from Blazar https://review.openstack.org/522200 | 04:21 |
| pabelanger | EmilienM: could be because of: http://git.openstack.org/cgit/openstack/puppet-openstack-integration/tree/.zuul.yaml#n362 | 04:25 |
| *** ijw has joined #openstack-infra | 04:28 | |
| pabelanger | EmilienM: but you can look at the inventory file and see how zuul loaded the job | 04:31 |
| pabelanger | http://logs.openstack.org/67/521967/6/check/puppet-openstack-unit-3.6-centos-7/91428f1/zuul-info/inventory.yaml | 04:32 |
| pabelanger | EmilienM: yah, looks like p-o-i is where it is sourcing job from, since jobs are global | 04:32 |
| *** bobh has joined #openstack-infra | 04:33 | |
| *** lbragstad has quit IRC | 04:34 | |
| *** Wei_Liu1 has joined #openstack-infra | 04:34 | |
| *** Wei_Liu has quit IRC | 04:35 | |
| *** Wei_Liu1 is now known as Wei_Liu | 04:35 | |
| *** iyamahat has quit IRC | 04:36 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: DNM: negative test https://review.openstack.org/522438 | 04:37 |
| *** bobh has quit IRC | 04:38 | |
| *** bobh has joined #openstack-infra | 04:42 | |
| *** hemna_ has quit IRC | 04:45 | |
| *** bobh has quit IRC | 04:46 | |
| *** dhajare has joined #openstack-infra | 04:47 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: zuul: Run OVB jobs only when it matters https://review.openstack.org/522321 | 04:50 |
| *** andreas_s has joined #openstack-infra | 04:53 | |
| *** hongbin has quit IRC | 04:56 | |
| *** andreas_s has quit IRC | 04:58 | |
| *** bobh has joined #openstack-infra | 04:58 | |
| *** Daisy_ has joined #openstack-infra | 04:59 | |
| *** Daisy_ has quit IRC | 05:01 | |
| *** Daisy_ has joined #openstack-infra | 05:02 | |
| *** bobh has quit IRC | 05:02 | |
| *** psachin has joined #openstack-infra | 05:06 | |
| *** claudiub|2 has joined #openstack-infra | 05:07 | |
| *** ykarel has joined #openstack-infra | 05:08 | |
| EmilienM | pabelanger: ok thx | 05:10 |
| EmilienM | pabelanger: but it's weird the job layout in my patch doesn't override it | 05:10 |
| EmilienM | pabelanger: I'll need your help tomorrow, to get https://review.openstack.org/#/c/521967 - I can't figure why the layout isn't applied. | 05:12 |
| *** salv-orlando has joined #openstack-infra | 05:12 | |
| *** salv-orlando has quit IRC | 05:18 | |
| *** jesusaur has quit IRC | 05:20 | |
| *** zhurong has joined #openstack-infra | 05:24 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Consume collect-logs directly from zuul/post playbook https://review.openstack.org/522400 | 05:28 |
| openstackgerrit | Rajat Sharma proposed openstack-infra/openstack-zuul-jobs master: Remove Blazar legacy jobs https://review.openstack.org/522197 | 05:28 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Consume collect-logs directly from zuul/post playbook https://review.openstack.org/522400 | 05:28 |
| *** bobh has joined #openstack-infra | 05:30 | |
| *** jamesmcarthur has joined #openstack-infra | 05:33 | |
| EmilienM | AJaeger: could you please help me to get https://review.openstack.org/#/c/521967/ ? I don't understand why the layout isn't applied | 05:33 |
| EmilienM | AJaeger: pabelanger commented on p-o-i ^ but I don't get why job layout in my patch doesn't override it | 05:34 |
| *** bobh has quit IRC | 05:34 | |
| *** jesusaur has joined #openstack-infra | 05:35 | |
| EmilienM | AJaeger: I'm off now, if you have time to comment in the review, thanks a lot! | 05:38 |
| *** bobh has joined #openstack-infra | 05:42 | |
| *** armax has quit IRC | 05:43 | |
| *** udesale has quit IRC | 05:44 | |
| *** armax has joined #openstack-infra | 05:44 | |
| *** armax has quit IRC | 05:44 | |
| *** armax has joined #openstack-infra | 05:44 | |
| *** armax has quit IRC | 05:45 | |
| *** rlandy|rover has quit IRC | 05:45 | |
| *** armax has joined #openstack-infra | 05:45 | |
| *** armax has quit IRC | 05:46 | |
| *** bobh has quit IRC | 05:47 | |
| jianghuaw_ | seems most of CIs are failing with the same error: | 05:52 |
| jianghuaw_ | 2017-11-22 23:44:28.200 | Could not find a version that satisfies the requirement protobuf===3.5.0 (from -c /opt/stack/new/requirements/upper-constraints.txt (line 338)) (from versions: 2.0.0b0, 2.0.3, 2.3.0, 2.4.1, 2.5.0, 2.6.0, 2.6.1, 3.0.0a2, 3.0.0a3, 3.0.0b1, 3.0.0b1.post1, 3.0.0b1.post2, 3.0.0b2, 3.0.0b2.post1, 3.0.0b2.post2, 3.0.0b3, 3.0.0b4, 3.0.0, 3.1.0, 3.1.0.post1, 3.2.0rc1, 3.2.0rc1.post1, 3.2.0rc2, 3.2.0, 3.3.0, 3.4.0, | 05:52 |
| jianghuaw_ | 3.5.0.post1) | 05:52 |
| *** jamesmcarthur has quit IRC | 05:53 | |
| jianghuaw_ | Can't see 3.5.0 from https://pypi.python.org/pypi/protobuf | 05:53 |
| *** jamesmcarthur has joined #openstack-infra | 05:53 | |
| *** udesale has joined #openstack-infra | 05:53 | |
| AJaeger | jianghuaw_: file a bug with protobuf and ask why they removed it | 05:53 |
| AJaeger | And then bump the constraints version down to 3.4.0 in requirements repo, please | 05:54 |
| jianghuaw_ | AJaeger, sure. thanks. | 05:54 |
| *** udesale has joined #openstack-infra | 05:54 | |
| *** salv-orlando has joined #openstack-infra | 05:57 | |
| *** wolverineav has joined #openstack-infra | 05:57 | |
| *** salv-orlando has quit IRC | 05:57 | |
| *** jamesmcarthur has quit IRC | 05:57 | |
| *** salv-orlando has joined #openstack-infra | 05:57 | |
| *** yamamoto has quit IRC | 05:59 | |
| AJaeger | EmilienM: why not remove the lines completely? Restricting to stable/newton on an ocata branch looks very wrong. you might need jeblair to look closer at this | 05:59 |
| *** bobh has joined #openstack-infra | 06:02 | |
| jianghuaw_ | AJaeger, the requirements repo change is at here: https://review.openstack.org/#/c/522447/ | 06:04 |
| *** bhavik1 has joined #openstack-infra | 06:05 | |
| *** yamamoto has joined #openstack-infra | 06:05 | |
| *** bobh has quit IRC | 06:07 | |
| jianghuaw_ | Here is the tracking ticket in protobuf: https://github.com/google/protobuf/issues/3942 | 06:08 |
| *** ijw has quit IRC | 06:09 | |
| *** yamamoto has quit IRC | 06:09 | |
| AJaeger | jianghuaw_: thanks. If you want to ask for requirements reviews, best ask on #openstack-requirements channel | 06:10 |
| *** threestrands has quit IRC | 06:10 | |
| jianghuaw_ | AJaeger, sure. I will. thanks. | 06:10 |
| *** bobh has joined #openstack-infra | 06:16 | |
| *** bobh has quit IRC | 06:20 | |
| AJaeger | config-core, please review https://review.openstack.org/522377 to help with aodh releasenotes and https://review.openstack.org/521497 as cleanup | 06:21 |
| *** reed_ has quit IRC | 06:31 | |
| *** qba73 has joined #openstack-infra | 06:32 | |
| *** bobh has joined #openstack-infra | 06:37 | |
| *** bobh has quit IRC | 06:42 | |
| openstackgerrit | liusheng proposed openstack-infra/project-config master: Switch to pep8 jobs for mogan-specs https://review.openstack.org/522436 | 06:43 |
| *** bobh has joined #openstack-infra | 06:47 | |
| *** cshastri has joined #openstack-infra | 06:48 | |
| *** florianf has joined #openstack-infra | 06:50 | |
| *** bobh has quit IRC | 06:52 | |
| *** kiennt26 has joined #openstack-infra | 06:55 | |
| *** jesusaur has quit IRC | 07:00 | |
| *** jesusaur has joined #openstack-infra | 07:02 | |
| openstackgerrit | Merged openstack-infra/project-config master: Temporarily disable releasenotes for aodh https://review.openstack.org/522377 | 07:02 |
| *** ijw has joined #openstack-infra | 07:09 | |
| *** yamamoto has joined #openstack-infra | 07:09 | |
| *** spectr has joined #openstack-infra | 07:12 | |
| *** wolverineav has quit IRC | 07:16 | |
| *** ijw has quit IRC | 07:17 | |
| *** sree has quit IRC | 07:17 | |
| *** yamamoto has quit IRC | 07:18 | |
| *** kjackal has quit IRC | 07:25 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy-murano-firstapp-docs https://review.openstack.org/521497 | 07:28 |
| AJaeger | frickler: thanks for catching that - removed ^ | 07:29 |
| *** yamamoto has joined #openstack-infra | 07:32 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove ruby from bindep-fallback.txt https://review.openstack.org/519534 | 07:33 |
| *** andreas_s has joined #openstack-infra | 07:34 | |
| *** annp has joined #openstack-infra | 07:34 | |
| *** bobh has joined #openstack-infra | 07:35 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Remove some of legacy jobs https://review.openstack.org/522465 | 07:36 |
| *** jtomasek has joined #openstack-infra | 07:38 | |
| *** bobh has quit IRC | 07:39 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/openstack-zuul-jobs master: networking-midonet: Remove some of legacy jobs https://review.openstack.org/522471 | 07:43 |
| *** yamamoto has quit IRC | 07:43 | |
| *** rcernin has quit IRC | 07:44 | |
| *** psachin has quit IRC | 07:44 | |
| *** bobh has joined #openstack-infra | 07:45 | |
| *** pcaruana has joined #openstack-infra | 07:45 | |
| *** qba73 has quit IRC | 07:46 | |
| *** wolverineav has joined #openstack-infra | 07:47 | |
| *** alexchadin has joined #openstack-infra | 07:48 | |
| *** martinkopec has joined #openstack-infra | 07:50 | |
| *** e0ne has joined #openstack-infra | 07:51 | |
| *** gmann_afk is now known as gmann | 07:51 | |
| *** wolverineav has quit IRC | 07:52 | |
| *** makowals has joined #openstack-infra | 07:54 | |
| *** bobh has quit IRC | 07:54 | |
| *** wolverineav has joined #openstack-infra | 07:55 | |
| *** bobh has joined #openstack-infra | 07:56 | |
| *** wolverin_ has joined #openstack-infra | 07:57 | |
| *** bobh has quit IRC | 08:01 | |
| *** wolverineav has quit IRC | 08:02 | |
| *** shardy has joined #openstack-infra | 08:02 | |
| *** armaan has joined #openstack-infra | 08:05 | |
| *** wolverin_ has quit IRC | 08:05 | |
| *** bobh has joined #openstack-infra | 08:06 | |
| *** kjackal has joined #openstack-infra | 08:10 | |
| *** bobh has quit IRC | 08:10 | |
| *** apetrich has quit IRC | 08:11 | |
| *** ijw has joined #openstack-infra | 08:13 | |
| *** psachin has joined #openstack-infra | 08:14 | |
| *** gmann is now known as gmann_afk | 08:15 | |
| *** LindaWang has joined #openstack-infra | 08:15 | |
| *** ijw has quit IRC | 08:17 | |
| *** ykarel is now known as ykarel|lunch | 08:19 | |
| *** ralonsoh has joined #openstack-infra | 08:20 | |
| *** spectr has quit IRC | 08:26 | |
| *** bobh has joined #openstack-infra | 08:26 | |
| *** jistr|off|trng is now known as jistr | 08:28 | |
| *** dbecker has joined #openstack-infra | 08:28 | |
| *** tesseract has joined #openstack-infra | 08:29 | |
| *** Hal has joined #openstack-infra | 08:29 | |
| *** mtreinish has quit IRC | 08:29 | |
| *** bobh has quit IRC | 08:30 | |
| *** jpena|off is now known as jpena | 08:35 | |
| *** wolverineav has joined #openstack-infra | 08:36 | |
| *** jascott1 has joined #openstack-infra | 08:36 | |
| *** amoralej|off is now known as amoralej | 08:37 | |
| *** bobh has joined #openstack-infra | 08:39 | |
| *** jascott1 has quit IRC | 08:40 | |
| *** slaweq has joined #openstack-infra | 08:40 | |
| *** wolverineav has quit IRC | 08:40 | |
| *** bobh_ has joined #openstack-infra | 08:42 | |
| *** bobh has quit IRC | 08:43 | |
| *** yamamoto has joined #openstack-infra | 08:43 | |
| *** mtreinish has joined #openstack-infra | 08:46 | |
| *** bobh_ has quit IRC | 08:47 | |
| *** e0ne has quit IRC | 08:49 | |
| *** e0ne has joined #openstack-infra | 08:50 | |
| *** e0ne has quit IRC | 08:51 | |
| *** e0ne has joined #openstack-infra | 08:51 | |
| *** yamamoto has quit IRC | 08:51 | |
| *** e0ne has quit IRC | 08:52 | |
| *** e0ne has joined #openstack-infra | 08:53 | |
| *** e0ne has quit IRC | 08:53 | |
| *** e0ne has joined #openstack-infra | 08:53 | |
| *** e0ne has quit IRC | 08:54 | |
| *** e0ne has joined #openstack-infra | 08:54 | |
| *** e0ne has quit IRC | 08:55 | |
| *** wolverineav has joined #openstack-infra | 08:55 | |
| *** bhavik1 has quit IRC | 08:57 | |
| *** dingyichen has quit IRC | 08:59 | |
| *** ykarel|lunch is now known as ykarel | 09:03 | |
| *** Daisy_ has quit IRC | 09:03 | |
| *** jpich has joined #openstack-infra | 09:07 | |
| *** lucas-afk is now known as lucasagomes | 09:08 | |
| *** apetrich has joined #openstack-infra | 09:13 | |
| *** hashar has joined #openstack-infra | 09:13 | |
| *** IRCNickname has quit IRC | 09:14 | |
| *** ijw has joined #openstack-infra | 09:14 | |
| *** bobh has joined #openstack-infra | 09:15 | |
| *** efoley has joined #openstack-infra | 09:17 | |
| *** yamamoto has joined #openstack-infra | 09:17 | |
| *** ijw has quit IRC | 09:19 | |
| *** bobh has quit IRC | 09:20 | |
| *** matbu_ has quit IRC | 09:24 | |
| *** wolverineav has quit IRC | 09:24 | |
| *** sree has joined #openstack-infra | 09:24 | |
| *** yamamoto has quit IRC | 09:25 | |
| *** wolverineav has joined #openstack-infra | 09:25 | |
| *** sree has quit IRC | 09:25 | |
| *** sree has joined #openstack-infra | 09:26 | |
| openstackgerrit | Rui Chen proposed openstack-infra/nodepool feature/zuulv3: Fix nodepool alien-list issue https://review.openstack.org/522495 | 09:27 |
| *** sree has quit IRC | 09:28 | |
| *** wolverineav has quit IRC | 09:29 | |
| *** dsariel has joined #openstack-infra | 09:30 | |
| *** zhurong has quit IRC | 09:32 | |
| *** qba73 has joined #openstack-infra | 09:36 | |
| *** yamamoto has joined #openstack-infra | 09:37 | |
| *** rcernin has joined #openstack-infra | 09:38 | |
| *** rossella_s has quit IRC | 09:38 | |
| *** bobh has joined #openstack-infra | 09:40 | |
| *** bobh has quit IRC | 09:41 | |
| *** bobh has joined #openstack-infra | 09:42 | |
| *** yamamoto has quit IRC | 09:43 | |
| *** jascott1 has joined #openstack-infra | 09:45 | |
| openstackgerrit | Rajat Sharma proposed openstack-infra/project-config master: Remove legacy jobs from Bandit https://review.openstack.org/522504 | 09:46 |
| *** bobh has quit IRC | 09:47 | |
| *** rfolco has quit IRC | 09:47 | |
| openstackgerrit | Rajat Sharma proposed openstack-infra/openstack-zuul-jobs master: Remove Bandit legacy jobs https://review.openstack.org/522507 | 09:50 |
| *** LindaWang has quit IRC | 09:52 | |
| *** jascott1 has quit IRC | 09:56 | |
| openstackgerrit | Rajat Sharma proposed openstack-infra/project-config master: Remove legacy jobs from Bandit https://review.openstack.org/522504 | 09:56 |
| *** qba73 has quit IRC | 09:56 | |
| *** jascott1 has joined #openstack-infra | 09:57 | |
| *** rfolco has joined #openstack-infra | 09:59 | |
| openstackgerrit | Rajat Sharma proposed openstack-infra/openstack-zuul-jobs master: Remove Bandit legacy jobs https://review.openstack.org/522507 | 09:59 |
| *** dbecker has quit IRC | 09:59 | |
| andreas_s | Hi. There seems to be an issue with some pip packages - this affects several Third Party CIs. Is anybody already working on this? No matching distribution found for protobuf===3.5.0 | 10:00 |
| andreas_s | https://bugs.launchpad.net/openstack-requirements/+bug/1734009 | 10:00 |
| openstack | Launchpad bug 1734009 in OpenStack Global Requirements "No matching distribution found for protobuf===3.5.0" [Undecided,New] | 10:00 |
| *** erlon has joined #openstack-infra | 10:00 | |
| *** electrofelix has joined #openstack-infra | 10:05 | |
| *** tosky has joined #openstack-infra | 10:12 | |
| *** alexchadin has quit IRC | 10:12 | |
| *** alexchadin has joined #openstack-infra | 10:13 | |
| *** ijw has joined #openstack-infra | 10:16 | |
| *** armaan has quit IRC | 10:16 | |
| *** armaan has joined #openstack-infra | 10:17 | |
| odyssey4me | hmm, it would seem that we aren't seeing documentation publishing for our pike docs - I'm not seeing https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?h=stable/pike&id=156e34e88a9fb509ae4deb34e001487c7c9eadd6 published... any idea why that might be? | 10:20 |
| *** ijw has quit IRC | 10:21 | |
| *** annp has quit IRC | 10:23 | |
| *** sshnaidm|off is now known as sshnaidm | 10:24 | |
| *** cuongnv_ has quit IRC | 10:27 | |
| *** dizquierdo has joined #openstack-infra | 10:29 | |
| *** armaan has quit IRC | 10:31 | |
| *** khappone has joined #openstack-infra | 10:31 | |
| *** armaan has joined #openstack-infra | 10:31 | |
| *** pblaho has quit IRC | 10:33 | |
| *** e0ne has joined #openstack-infra | 10:34 | |
| *** yamamoto has joined #openstack-infra | 10:37 | |
| *** umbarkar has joined #openstack-infra | 10:41 | |
| *** bobh has joined #openstack-infra | 10:42 | |
| *** kiennt26 has quit IRC | 10:43 | |
| *** andreas_s has quit IRC | 10:45 | |
| *** andreas_s has joined #openstack-infra | 10:45 | |
| *** [HeOS] has quit IRC | 10:46 | |
| *** bobh has quit IRC | 10:46 | |
| *** pblaho has joined #openstack-infra | 10:47 | |
| *** daidv has quit IRC | 10:52 | |
| tosky | AJaeger: hi, re https://review.openstack.org/#/c/522358/ , is there some plan to get rid of tox_install.sh? Every time I see a file copied in 1525235 OpenStack repositories I shivers | 10:53 |
| *** jamesdenton has quit IRC | 10:53 | |
| *** andreas_s has quit IRC | 10:54 | |
| *** andreas_s has joined #openstack-infra | 10:55 | |
| tosky | AJaeger: also: if merged, should we backport the change to the older branches? | 10:56 |
| *** ldnunes has joined #openstack-infra | 10:56 | |
| zhongjun | hi, anyone could help me to find what happened about centos jate http://logs.openstack.org/94/520994/3/check/manila-tempest-minimal-dsvm-lvm-centos-7/d8bd03e/job-output.txt.gz#_2017-11-23_03_44_16_485802 | 10:57 |
| *** bobh has joined #openstack-infra | 10:59 | |
| zhongjun | it's only affecting centos | 11:00 |
| *** wolverineav has joined #openstack-infra | 11:01 | |
| *** udesale__ has joined #openstack-infra | 11:03 | |
| *** alexchadin has quit IRC | 11:03 | |
| *** alexchadin has joined #openstack-infra | 11:03 | |
| *** bobh has quit IRC | 11:04 | |
| AJaeger | tosky: best to discuss with mordred, he has some ideas on that. | 11:04 |
| tosky | AJaeger: and about backporting the change to the previous branches? | 11:05 |
| AJaeger | zhongjun: you need to look at the main log file. Best ask on #openstack-dev on how to debug these | 11:05 |
| *** bobh has joined #openstack-infra | 11:05 | |
| AJaeger | tosky: yes, I have to backport that one... | 11:05 |
| *** udesale has quit IRC | 11:05 | |
| tosky | AJaeger: ack | 11:06 |
| AJaeger | andreas_s: https://review.openstack.org/522447 and https://review.openstack.org/522412 are two alternative solutions for protobuf | 11:06 |
| AJaeger | andreas_s: best ask on #openstack-requirements | 11:06 |
| zhongjun | AJaeger: http://logs.openstack.org/94/520994/3/check/manila-tempest-minimal-dsvm-lvm-centos-7/d8bd03e/logs/devstacklog.txt.gz#_2017-11-23_03_44_15_189 | 11:06 |
| *** armaan has quit IRC | 11:06 | |
| AJaeger | zhongjun: Ah, protobuf - see above. Known issues, they removed it from pypi and that broke us ;( | 11:07 |
| zhongjun | It looks like we have the same issue | 11:07 |
| *** armaan has joined #openstack-infra | 11:07 | |
| *** dizquierdo has quit IRC | 11:08 | |
| zhongjun | AJaeger: got it, they are trying to fix it? | 11:09 |
| *** andreas_s has quit IRC | 11:09 | |
| zhongjun | AJaeger: I saw some patches for fixing protobuf, thanks | 11:09 |
| *** bobh has quit IRC | 11:10 | |
| *** armaan has quit IRC | 11:14 | |
| *** armaan has joined #openstack-infra | 11:15 | |
| *** jascott1 has quit IRC | 11:15 | |
| *** bobh has joined #openstack-infra | 11:16 | |
| *** ijw has joined #openstack-infra | 11:17 | |
| *** bobh has quit IRC | 11:20 | |
| *** ijw has quit IRC | 11:22 | |
| *** sree has joined #openstack-infra | 11:23 | |
| *** sree has quit IRC | 11:23 | |
| *** sree has joined #openstack-infra | 11:23 | |
| *** armaan has quit IRC | 11:24 | |
| *** bobh has joined #openstack-infra | 11:25 | |
| *** dizquierdo has joined #openstack-infra | 11:25 | |
| *** armaan has joined #openstack-infra | 11:25 | |
| *** apetrich has quit IRC | 11:29 | |
| *** bobh has quit IRC | 11:30 | |
| *** andreas_s has joined #openstack-infra | 11:33 | |
| AJaeger | zhongjun: exactly - work going on, nothing you can do right now here | 11:34 |
| *** bobh has joined #openstack-infra | 11:35 | |
| *** bobh has quit IRC | 11:39 | |
| sshnaidm | which ansible version is used in running zuul playbooks? | 11:40 |
| *** bobh has joined #openstack-infra | 11:42 | |
| erlon | @all, guys is anyone having problems on CIs with protobuf==3.5? Pypi only has "3.5.0.post" and devstack is asking for "3.5.0". http://logs.openstack.org/02/505202/9/check/legacy-tempest-dsvm-neutron-nova-next-full/80d9bfb/logs/devstack-early.txt.gz#_2017-11-23_10_02_06_408 | 11:44 |
| *** yamamoto_ has joined #openstack-infra | 11:45 | |
| erlon | This link is upstream CI, ours hit his: http://logs.openstack.netapp.com/ci-logs/logs/70/470/4/downstream-functional/cinder-cDOT-iSCSI/bcc60e1/logs/devstack-early.txt.gz | 11:45 |
| * erlon wonders where infra VMs are downloading that from | 11:46 | |
| *** bobh has quit IRC | 11:47 | |
| *** ajo has joined #openstack-infra | 11:47 | |
| ajo | hi folks :) | 11:48 |
| ajo | does anybody know how do we make ansible output the shell command stdout in realtime ? | 11:48 |
| *** yamamoto has quit IRC | 11:48 | |
| ajo | in the ansible scripts from zuulv3 | 11:48 |
| ajo | I was surprised to see it, and I find it particularly helpful | 11:48 |
| ajo | but I don't know how do we manage to get that | 11:48 |
| ajo | dmellado: | 11:49 |
| ajo | do you know who could I ask? :) | 11:49 |
| *** kjackal has quit IRC | 11:50 | |
| dmellado | hey ajo :) | 11:52 |
| ajo | dmellado: hi :), any idea of whom to ask? | 11:53 |
| dmellado | maybe AJaeger could know, but otherwise I'd recommend you to just ask on the zuul channel | 11:53 |
| *** ganso has joined #openstack-infra | 11:53 | |
| ajo | ping AJaeger ^ :) | 11:53 |
| *** pblaho has quit IRC | 11:53 | |
| rcarrillocruz | ajo: zuul ships a module for that | 11:53 |
| rcarrillocruz | https://github.com/openstack-infra/zuul/blob/feature/zuulv3/zuul/ansible/library/zuul_console.py | 11:53 |
| dmellado | maybe also pabelanger o rcarrillocruz | 11:53 |
| dmellado | heh, he was even faster than me | 11:53 |
| ajo | ooohh :) | 11:53 |
| ajo | rcarrillocruz++ | 11:54 |
| ajo | rcarrillocruz: can I use that in ansible standalone? | 11:54 |
| *** thiagolib has joined #openstack-infra | 11:54 | |
| *** bobh has joined #openstack-infra | 11:54 | |
| *** dbecker has joined #openstack-infra | 11:54 | |
| rcarrillocruz | i guess you could adapt it, but is not a drop-in replacement for shell, not sure how that's hooked up with ansible/ansible shell | 11:55 |
| ajo | rcarrillocruz: yeah, that's what I was thinking of now... | 11:56 |
| rcarrillocruz | iirc there's work to be done to upstream that, but stuff needs to happen in ansible upstream | 11:56 |
| rcarrillocruz | check with mordred when he gets around | 11:56 |
| dmellado | IIRC ansible folks were usually against that | 11:56 |
| rcarrillocruz | i'm sure what this lands upstream will be the number one new feature, shell task streaming has been asked since forever | 11:56 |
| rcarrillocruz | :D | 11:56 |
| dmellado | so I think it'd be an interesting discussion | 11:56 |
| *** bobh has quit IRC | 11:58 | |
| *** efoley has quit IRC | 11:59 | |
| *** yamamoto_ has quit IRC | 12:02 | |
| *** dsariel has quit IRC | 12:03 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix mixed canonical/non-canonical project merge https://review.openstack.org/513331 | 12:04 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add regex support to project stanzas https://review.openstack.org/513368 | 12:04 |
| sshnaidm | which ansible version is used in running zuul playbooks? | 12:05 |
| frickler | erlon: see https://bugs.launchpad.net/openstack-requirements/+bug/1734009 , infra CIs use a mirror of pypi and thus still may be able to get the release that was deleted upstream | 12:06 |
| openstack | Launchpad bug 1734009 in OpenStack Global Requirements "No matching distribution found for protobuf===3.5.0" [Undecided,New] | 12:06 |
| *** pblaho has joined #openstack-infra | 12:06 | |
| *** jpena is now known as jpena|lunch | 12:07 | |
| *** amoralej is now known as amoralej|lunch | 12:07 | |
| *** snapiri has joined #openstack-infra | 12:09 | |
| *** bobh has joined #openstack-infra | 12:09 | |
| rcarrillocruz | i believe 2.3 | 12:09 |
| rcarrillocruz | at least from feature/zuulv3 requirements | 12:09 |
| rcarrillocruz | sshnaidm: ^ | 12:09 |
| *** andreas_s has quit IRC | 12:10 | |
| rcarrillocruz | ansible>=2.3.0.0,<2.4 | 12:10 |
| *** andreas_s has joined #openstack-infra | 12:10 | |
| *** raissa has joined #openstack-infra | 12:13 | |
| *** bobh has quit IRC | 12:14 | |
| *** andreas_s has quit IRC | 12:14 | |
| erlon | frickler: oww, good to know thanks, so apparently infra mirror only adds new packages from pypi but not removes the ones removed, otherwise it would be broken already | 12:16 |
| *** rcernin has quit IRC | 12:17 | |
| *** ijw has joined #openstack-infra | 12:19 | |
| *** dtantsur|afk is now known as dtantsur | 12:19 | |
| *** dsariel has joined #openstack-infra | 12:19 | |
| *** bobh has joined #openstack-infra | 12:19 | |
| *** andreas_s has joined #openstack-infra | 12:21 | |
| *** kjackal has joined #openstack-infra | 12:22 | |
| *** jascott1 has joined #openstack-infra | 12:22 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add regex support to project stanzas https://review.openstack.org/513368 | 12:23 |
| *** bobh has quit IRC | 12:23 | |
| *** ijw has quit IRC | 12:24 | |
| mpeterson | hello, in the project where I contribute we get quite a lot of random failures in the jobs... I'm guessing it's not something that happens globally, otherwise more people would be aware about this, can anyone help me try figuring out? | 12:24 |
| *** bobh has joined #openstack-infra | 12:24 | |
| *** jascott1 has quit IRC | 12:27 | |
| *** jascott1 has joined #openstack-infra | 12:27 | |
| fungi | frickler: erlon: actually, our pypi mirror does perform deletions when packages are deleted on pypi... however we also build per-platform wheel mirrors and those are purely additive right now. so we're no longer mirroring the sdist for that project but likely still providing pre-built wheels for our specific test platforms | 12:27 |
| *** bobh has quit IRC | 12:28 | |
| erlon | fungi: hmm, as I guessed | 12:28 |
| erlon | mpeterson: what project is that? do you have logs? | 12:29 |
| *** andreas_s has quit IRC | 12:30 | |
| mpeterson | erlon: yes, I have. Do you want multiple links or one by one? Here is the first for example http://logs.openstack.org/49/504949/11/check/openstack-tox-py35/88366c7/job-output.txt.gz | 12:30 |
| mpeterson | erlon: it's networking-odl | 12:30 |
| *** andreas_s has joined #openstack-infra | 12:31 | |
| erlon | mpeterson: are the multiple links related to different failures? as you said they are ramdon right? | 12:31 |
| mpeterson | erlon: yes, they have different failures | 12:31 |
| *** bobh has joined #openstack-infra | 12:32 | |
| mpeterson | erlon: let me work on that list and I'll post it on a paste bin, should take me just a minute | 12:33 |
| erlon | mpeterson: send the changes links, like: https://review.openstack.org/#/c/504949/ | 12:35 |
| mpeterson | erlon: okey | 12:35 |
| *** efoley has joined #openstack-infra | 12:36 | |
| *** bobh has quit IRC | 12:36 | |
| mpeterson | erlon: http://paste.openstack.org/show/627234/ just a few, I can get more if neededd | 12:39 |
| *** andreas_s has quit IRC | 12:40 | |
| mpeterson | erlon: and you can see that even in those reviews without changing the code (just rechecks) the failed tests are not always the same | 12:41 |
| erlon | mpeterson: for this one 504949 seems that the unit tests are not mocked correctly, the py27 test is failing with a waiting error: http://logs.openstack.org/49/504949/11/check/openstack-tox-py27/6130ccc/job-output.txt.gz#_2017-11-23_07_54_21_089632 | 12:41 |
| erlon | the other one the py35 I believe that is happening something similar. The test is calling a wait function or something that is just waiting there. See that you first test is passing: http://logs.openstack.org/49/504949/11/check/openstack-tox-py35/88366c7/job-output.txt.gz#_2017-11-23_07_46_28_074248 | 12:42 |
| erlon | checking the others... | 12:42 |
| erlon | mpeterson: 492608 is a failure in the test, see "testtools.matchers._impl.MismatchError: 'D' not in ['S', 'R']" on test_handle_sighup_gracefully | 12:48 |
| mpeterson | erlon: in 492608 you can see the same test didn't failed in py35 and also not in py27. And another test just failed in one of those. | 12:50 |
| *** bobh has joined #openstack-infra | 12:50 | |
| mpeterson | erlon: I do reckon that perhaps the tests are badly written, I'll look into reviewing them | 12:51 |
| *** andreas_s has joined #openstack-infra | 12:52 | |
| *** bobh has quit IRC | 12:54 | |
| *** alexchadin has quit IRC | 12:54 | |
| *** alexchadin has joined #openstack-infra | 12:55 | |
| *** dmsimard|afk is now known as dmsimard | 12:55 | |
| erlon | mpeterson: the 492607 is a devstack failure, that some happens and the nitrogen Im not sure | 12:56 |
| frickler | AJaeger: infra-root: https://docs.openstack.org looks broken, is there a way to restore previous index.html while finding out what went wrong? | 12:56 |
| *** andreas_s has quit IRC | 12:57 | |
| erlon | mpeterson: the last one seems just a network failure: http://logs.openstack.org/06/492606/19/check/networking-odl-functional-carbon/f731837/job-output.txt.gz#_2017-11-23_08_28_31_619635 | 12:57 |
| *** derekh has joined #openstack-infra | 12:57 | |
| erlon | mpeterson: so, the bottom line is, you have some tests that should be fixed, and proper mocked. The other test failure usually can happen | 12:58 |
| mpeterson | erlon: gotcha, thanks for taking the time | 12:58 |
| *** ramishra has quit IRC | 12:58 | |
| erlon | mpeterson: what you usually need to pay attention is to tests that fails with the same reason, or similar reasons, so you can nail the problem. | 12:59 |
| erlon | mpeterson: failing devstack, connections, etc is the kind of thing that usually happen, then you need to keep rechecking to see if that persists | 12:59 |
| erlon | mpeterson: no problem! | 12:59 |
| mpeterson | awesome, will try to fix those that are incorrectly written and keep on nailing quality issues in the project | 13:00 |
| *** [HeOS] has joined #openstack-infra | 13:02 | |
| *** bobh has joined #openstack-infra | 13:02 | |
| *** yamamoto has joined #openstack-infra | 13:02 | |
| *** jpena|lunch is now known as jpena | 13:06 | |
| *** andreas_s has joined #openstack-infra | 13:06 | |
| *** bobh has quit IRC | 13:07 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Create openstack/governance-sigs repository https://review.openstack.org/522543 | 13:07 |
| *** jascott1 has quit IRC | 13:08 | |
| *** yamamoto has quit IRC | 13:09 | |
| AJaeger | frickler: on it... | 13:10 |
| *** tlaxkit has joined #openstack-infra | 13:10 | |
| *** LindaWang1 has joined #openstack-infra | 13:10 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/system-config master: Create publish space for SIGs under governance.o.o https://review.openstack.org/522545 | 13:12 |
| *** bobh has joined #openstack-infra | 13:12 | |
| *** andreas_s has quit IRC | 13:14 | |
| *** efoley has quit IRC | 13:14 | |
| *** andreas_s has joined #openstack-infra | 13:14 | |
| *** bobh has quit IRC | 13:16 | |
| sshnaidm | I have a question about including external roles with zuulv3, how to do it? I saw an example like: - zuul: openstack-infra/zuul-jobs - but what is "zuul:" there and what is "openstack-infra/zuul-jobs" ? | 13:18 |
| *** efoley has joined #openstack-infra | 13:19 | |
| *** ijw has joined #openstack-infra | 13:20 | |
| AJaeger | sshnaidm: openstack-infra/zuul-jobs is a repository | 13:21 |
| *** salv-orlando has quit IRC | 13:21 | |
| *** salv-orlando has joined #openstack-infra | 13:22 | |
| sshnaidm | AJaeger, and what is "zuul:" there? | 13:24 |
| *** ykarel is now known as ykarel|away | 13:24 | |
| *** ijw has quit IRC | 13:25 | |
| *** dizquierdo has quit IRC | 13:26 | |
| AJaeger | sshnaidm: I would need to check the manual at https://docs.openstack.org/infra/zuul/feature/zuulv3 for that | 13:26 |
| *** salv-orlando has quit IRC | 13:27 | |
| AJaeger | sorry, firefighting with docs right now | 13:27 |
| *** dsariel has quit IRC | 13:27 | |
| sshnaidm | AJaeger, thanks | 13:30 |
| *** coolsvap has quit IRC | 13:30 | |
| *** dprince has joined #openstack-infra | 13:31 | |
| *** apetrich has joined #openstack-infra | 13:33 | |
| *** annp has joined #openstack-infra | 13:35 | |
| sshnaidm | AJaeger, do you know maybe about plans to move to ansible 2.4.1? Now version 2.3.2 is used as I see. | 13:36 |
| fungi | sshnaidm: i believe the plan is to try and make zuul work with whatever newer versions of ansible come out, but there's probably going to have to be a support matrix of some kind since we won't be able to support every version | 13:37 |
| sean-k-mooney | o/ does anyone know if there is a patch in flight to fix the install of protobuf in the upper constratis file | 13:38 |
| AJaeger | sean-k-mooney: see scroll-back, two changes up for requirements repo | 13:39 |
| sean-k-mooney | cool ya just found the bug on lauchpad/the logstach link for the issue | 13:39 |
| *** ramishra has joined #openstack-infra | 13:40 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: POC: logs collection from native zuulv3 job https://review.openstack.org/522554 | 13:40 |
| *** dsariel has joined #openstack-infra | 13:40 | |
| sean-k-mooney | AJaeger: any idea why they seam to randomly add .post1 to the end of there version numbers | 13:40 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: POC: logs collection from native zuulv3 job https://review.openstack.org/522554 | 13:43 |
| fungi | sean-k-mooney: "post" revisions are defined in pep 440 as packaging-only updates to a release version | 13:43 |
| AJaeger | sean-k-mooney: there was a bug report openened, not sure what the status is - it's linked in backscroll and as comment from one of the two changes . Sorry, can't help further... | 13:43 |
| sean-k-mooney | AJaeger: ah i see that they delete 3.5.0 from pypi | 13:43 |
| *** bobh has joined #openstack-infra | 13:44 | |
| fungi | sean-k-mooney: post revisions sort after the release from which they'ree derived (vs dev, alpha, beta, rc, which sort before their version numbers) | 13:44 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: POC: logs collection from native zuulv3 job https://review.openstack.org/522554 | 13:44 |
| * fungi disappears into a car for a while | 13:46 | |
| *** bobh has quit IRC | 13:49 | |
| *** Hal has quit IRC | 13:49 | |
| sean-k-mooney | fungi: thanks thats good to know however i guess our pbr/pip install logic with the upper constriatns file does not like it hence the stack error. ill just locally modify my requirements repo to work around it for now | 13:50 |
| *** pblaho has quit IRC | 13:51 | |
| *** pblaho has joined #openstack-infra | 13:52 | |
| *** amoralej|lunch is now known as amoralej | 13:52 | |
| *** janki has joined #openstack-infra | 13:53 | |
| *** sree has quit IRC | 13:53 | |
| *** andreas_s has quit IRC | 13:54 | |
| *** sree has joined #openstack-infra | 13:54 | |
| *** andreas_s has joined #openstack-infra | 13:54 | |
| *** tobiash has quit IRC | 13:55 | |
| *** Hal has joined #openstack-infra | 13:55 | |
| *** efoley has quit IRC | 13:55 | |
| *** efoley has joined #openstack-infra | 13:56 | |
| *** alexchadin has quit IRC | 13:58 | |
| *** sree has quit IRC | 13:58 | |
| *** dprince has quit IRC | 14:00 | |
| *** ramishra has quit IRC | 14:00 | |
| *** ykarel|away has quit IRC | 14:01 | |
| *** ramishra has joined #openstack-infra | 14:02 | |
| *** andreas_s has quit IRC | 14:04 | |
| *** Goneri has joined #openstack-infra | 14:05 | |
| *** janki has quit IRC | 14:05 | |
| *** yamamoto has joined #openstack-infra | 14:05 | |
| *** janki has joined #openstack-infra | 14:06 | |
| *** mat128 has joined #openstack-infra | 14:08 | |
| *** pblaho has quit IRC | 14:08 | |
| *** andreas_s has joined #openstack-infra | 14:08 | |
| *** yamamoto has quit IRC | 14:10 | |
| *** spectr has joined #openstack-infra | 14:10 | |
| pabelanger | sshnaidm: see https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html?highlight=roles#attr-job.roles for info, but there is zuul and galaxy for role location it will search, right now we only support zuul | 14:12 |
| pabelanger | we likely should just drop galaxy, since it is github.com really | 14:12 |
| sshnaidm | pabelanger, yeah, seems like I started to understand how to configure.. | 14:13 |
| sshnaidm | pabelanger, do we have a possibility to import playbooks too? | 14:13 |
| sshnaidm | pabelanger, it's supported in 2.4, but you use 2.3.2 | 14:14 |
| pabelanger | sshnaidm: no, you'll need to have zuul(2.3.2) launch a bastion host (2.4.0.1) then run your playbooks. Today we are pinned to 2.3.2 for some reasons, basically 2.4 changes configs is a large way and we still need to do some changes to zuul to support it | 14:17 |
| *** andreas_s has quit IRC | 14:17 | |
| *** martinkopec has quit IRC | 14:17 | |
| sshnaidm | pabelanger, so no possibility to use external playbooks for now? | 14:18 |
| *** bobh has joined #openstack-infra | 14:18 | |
| *** andreas_s has joined #openstack-infra | 14:18 | |
| pabelanger | sshnaidm: what do you mean by external playbooks? | 14:18 |
| tosky | afaik no, only roles | 14:19 |
| *** armaan_ has joined #openstack-infra | 14:19 | |
| sshnaidm | pabelanger, for example from external repo, like https://github.com/openstack/tripleo-quickstart/tree/master/playbooks | 14:19 |
| *** armaan has quit IRC | 14:19 | |
| tosky | but if you wraps your code properly, your playbooks should be mostly glue around roles | 14:19 |
| sshnaidm | pabelanger, so I need just to copy them to local playbooks/ ? | 14:19 |
| *** sree has joined #openstack-infra | 14:20 | |
| pabelanger | sshnaidm: you want project A to run a playbook from project B? | 14:20 |
| sshnaidm | pabelanger, yes, even a few of them | 14:21 |
| sshnaidm | pabelanger, and from project C too :) | 14:22 |
| pabelanger | sshnaidm: you could create a parant / child jobs to do that. Parent runs playbooks from project A, and child does playbook B | 14:22 |
| *** ijw has joined #openstack-infra | 14:22 | |
| pabelanger | you'd setup jobs that inherit from each other | 14:22 |
| *** salv-orlando has joined #openstack-infra | 14:22 | |
| pabelanger | othereise, you can use setup a single playbook with includes to other plays | 14:22 |
| *** bobh has quit IRC | 14:22 | |
| pabelanger | example of include: http://git.openstack.org/cgit/openstack/windmill/tree/playbooks/site.yaml | 14:23 |
| *** sree has quit IRC | 14:24 | |
| *** bobh has joined #openstack-infra | 14:24 | |
| sshnaidm | pabelanger, yeah.. actually I have these playbooks already cloned to src/ by zuul as part of required-projects, so I can just point to them | 14:24 |
| pabelanger | I'd create parent / client job first, and see if you can achieve your goal. | 14:25 |
| *** dave-mccowan has joined #openstack-infra | 14:26 | |
| pabelanger | how do you do it today? Are you just running multiple ansible-playbook commands to different playbooks across repos? | 14:26 |
| *** ijw has quit IRC | 14:27 | |
| *** andreas_s has quit IRC | 14:27 | |
| *** smarcet has joined #openstack-infra | 14:27 | |
| *** bobh has quit IRC | 14:28 | |
| *** lbragstad has joined #openstack-infra | 14:29 | |
| *** salv-orlando has quit IRC | 14:31 | |
| *** armaan_ has quit IRC | 14:32 | |
| *** andreas_s has joined #openstack-infra | 14:33 | |
| *** dbecker has quit IRC | 14:35 | |
| *** bobh has joined #openstack-infra | 14:42 | |
| *** andreas_s has quit IRC | 14:46 | |
| *** bobh has quit IRC | 14:46 | |
| *** annp has quit IRC | 14:47 | |
| *** dbecker has joined #openstack-infra | 14:48 | |
| *** cshastri has quit IRC | 14:49 | |
| *** pbourke has quit IRC | 14:50 | |
| *** dave-mccowan has quit IRC | 14:51 | |
| *** andreas_s has joined #openstack-infra | 14:52 | |
| *** pbourke has joined #openstack-infra | 14:52 | |
| *** janki has quit IRC | 14:53 | |
| *** jamesmcarthur has joined #openstack-infra | 14:54 | |
| *** bobh has joined #openstack-infra | 14:55 | |
| *** dizquierdo has joined #openstack-infra | 14:57 | |
| *** jamesmcarthur has quit IRC | 14:59 | |
| *** bobh has quit IRC | 15:00 | |
| *** udesale__ has quit IRC | 15:02 | |
| *** dsariel has quit IRC | 15:05 | |
| *** andreas_s has quit IRC | 15:06 | |
| *** wolverineav has quit IRC | 15:06 | |
| *** yamamoto has joined #openstack-infra | 15:06 | |
| *** tobiash has joined #openstack-infra | 15:06 | |
| *** pblaho has joined #openstack-infra | 15:07 | |
| *** slaweq has quit IRC | 15:08 | |
| *** smarcet has quit IRC | 15:09 | |
| *** yamamoto has quit IRC | 15:12 | |
| *** salv-orlando has joined #openstack-infra | 15:13 | |
| *** sree has joined #openstack-infra | 15:13 | |
| tbarron | centos7 jobs appeart to be failing with this error: http://logs.openstack.org/32/515432/8/check/manila-tempest-minimal-dsvm-lvm-centos-7/f79affc/logs/devstack-early.txt.gz#_2017-11-23_13_25_02_868 | 15:15 |
| tbarron | can't find protobuf===3.5.0 | 15:15 |
| pabelanger | yah, I've seen that a few times today. Not sure if anybody is working on it | 15:15 |
| pabelanger | I can start poking at it myself here shortly | 15:16 |
| pabelanger | https://review.openstack.org/522412/ and https://review.openstack.org/522447/ are options | 15:17 |
| *** sree has quit IRC | 15:17 | |
| tbarron | pabelanger: awesome, I'm off today but we have a bunch of folks working outside the US so help is appreciated | 15:17 |
| *** pblaho has quit IRC | 15:18 | |
| tbarron | pabelanger: curiously only our centos jobs hit the issue, not the ubuntu jobs | 15:19 |
| pabelanger | tbarron: I think because we have a wheel for ubuntu, that's what fungi suggested | 15:20 |
| pabelanger | I can check that | 15:20 |
| tbarron | pabelanger: makes sense | 15:20 |
| pabelanger | yah, http://mirror.dfw.rax.openstack.org/wheel/ubuntu-16.04-x86_64/protobuf/ | 15:20 |
| pabelanger | so, if I delete it, it would break the world | 15:20 |
| pabelanger | one reason why deleting things from pypi is bad | 15:21 |
| *** bobh has joined #openstack-infra | 15:21 | |
| clarkb | this was discussed in #openstack-dev yesterday | 15:22 |
| clarkb | there should be a change to update requiremwnts from.jgriffith if he managed to push it | 15:22 |
| *** ijw has joined #openstack-infra | 15:23 | |
| *** amito-infinidat has joined #openstack-infra | 15:23 | |
| amito-infinidat | Hi, it looks like there's a new issue with the manila centos jobs: http://logs.openstack.org/32/515432/8/check/manila-tempest-minimal-dsvm-lvm-centos-7/f79affc/logs/devstack-early.txt.gz#_2017-11-23_13_25_02_868 . | 15:24 |
| pabelanger | clarkb: yah, there is, 2 patches above. 1 to roll back, another to jump forward to 3.5.0.post1 | 15:25 |
| pabelanger | clarkb: but we might be lacking requirements people due to US holiday | 15:25 |
| *** bobh has quit IRC | 15:25 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: ovb: run ovb-containers-oooq on on pike https://review.openstack.org/522579 | 15:27 |
| pabelanger | looks like requirements is missing pcre.h for a job | 15:27 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: ovb: run ovb-containers-oooq only on pike https://review.openstack.org/522579 | 15:27 |
| *** ijw has quit IRC | 15:28 | |
| AJaeger | clarkb: both changes are failing our tests ;( | 15:30 |
| AJaeger | pabelanger: yeah, fallout from the bindep cleanup from yesterday | 15:30 |
| AJaeger | pabelanger: see how I solved it elsewhere https://review.openstack.org/522533 | 15:30 |
| AJaeger | amito-infinidat: protobuf? Known issue, they deleted a version from pypi | 15:31 |
| AJaeger | speaking about bindep - ready to remove ruby from it? https://review.openstack.org/519534 | 15:32 |
| pabelanger | remote: https://review.openstack.org/522581 Add pcre headers for libvirt-python | 15:32 |
| pabelanger | AJaeger: ^just pushed that | 15:32 |
| * AJaeger can +A, just wanted to ask | 15:33 | |
| pabelanger | should help with libvirt wheel builds | 15:33 |
| AJaeger | pabelanger: make it "rpm" instead of "redhat" - suse has the same library | 15:33 |
| *** szaher has quit IRC | 15:34 | |
| AJaeger | pabelanger: otherwise fine | 15:34 |
| pabelanger | AJaeger: done | 15:34 |
| *** szaher has joined #openstack-infra | 15:34 | |
| AJaeger | pabelanger: any ideas why ZUUL_ variables are not visible in my job? See http://logs.openstack.org/49/522549/4/check/build-tox-manuals-checkbuild/03b0092/job-output.txt.gz#_2017-11-23_15_03_43_152096 and https://review.openstack.org/#/c/522549/4 | 15:36 |
| pabelanger | AJaeger: are you using base-legacy as parent? | 15:36 |
| pabelanger | or legacy-base | 15:37 |
| AJaeger | pabelanger: I'm not - and forgot to pass them in ;( Updateing .zuul.yaml now | 15:37 |
| pabelanger | AJaeger: you could start making the switch to zuulv3 variables too, is this project the only place that uses the script? | 15:38 |
| *** andreas_s has joined #openstack-infra | 15:38 | |
| AJaeger | pabelanger: yes. I can use zuul v3 variables. See my update now for https://review.openstack.org/#/c/522549 - or how would you do it? | 15:39 |
| *** sree has joined #openstack-infra | 15:40 | |
| pabelanger | AJaeger: yah, you'd need to start converting tools/build-all-rst.sh into ansible playbook or bring it into a shell task in the playbook replace variables. But what you have now is a good first step | 15:41 |
| AJaeger | pabelanger: ok, thanks | 15:42 |
| *** bobh has joined #openstack-infra | 15:42 | |
| *** rlandy has joined #openstack-infra | 15:43 | |
| *** ykarel|away has joined #openstack-infra | 15:44 | |
| *** sree has quit IRC | 15:45 | |
| *** ramishra has quit IRC | 15:45 | |
| *** bobh has quit IRC | 15:46 | |
| AJaeger | infra-root, I approved now https://review.openstack.org/519534 to remove ruby from bindep fallback list. | 15:47 |
| AJaeger | mnaser: heads up in case it affects puppet ^ | 15:47 |
| pabelanger | AJaeger: wfm | 15:47 |
| *** andreas_s has quit IRC | 15:48 | |
| mnaser | AJaeger: thanks for the heads up, ill double check | 15:48 |
| *** LindaWang1 has quit IRC | 15:50 | |
| *** andreas_s has joined #openstack-infra | 15:51 | |
| *** andreas__ has joined #openstack-infra | 15:52 | |
| *** lbragstad has quit IRC | 15:53 | |
| *** spectr has quit IRC | 15:54 | |
| *** bobh has joined #openstack-infra | 15:54 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove ruby from bindep-fallback.txt https://review.openstack.org/519534 | 15:55 |
| *** andreas_s has quit IRC | 15:56 | |
| *** ykarel|away has quit IRC | 15:56 | |
| *** armax has joined #openstack-infra | 15:57 | |
| *** bobh has quit IRC | 15:58 | |
| *** bobh has joined #openstack-infra | 15:58 | |
| *** rlandy has quit IRC | 16:01 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Revert "Temporarily disable releasenotes for aodh" https://review.openstack.org/522589 | 16:07 |
| *** yamamoto has joined #openstack-infra | 16:08 | |
| *** pcaruana has quit IRC | 16:09 | |
| EmilienM | pabelanger: when you have time, I need your help on https://review.openstack.org/#/c/521967/ - I can't figure out what's the best option | 16:10 |
| *** yamamoto has quit IRC | 16:13 | |
| sshnaidm | can I invlude vars from file in job definition? | 16:13 |
| sshnaidm | pabelanger, ^^ | 16:14 |
| *** andreas__ has quit IRC | 16:14 | |
| *** andreas_s has joined #openstack-infra | 16:15 | |
| *** ykarel|away has joined #openstack-infra | 16:15 | |
| *** andreas__ has joined #openstack-infra | 16:17 | |
| *** andrea___ has joined #openstack-infra | 16:18 | |
| *** mat128 has quit IRC | 16:18 | |
| *** mat128 has joined #openstack-infra | 16:19 | |
| *** andreas_s has quit IRC | 16:19 | |
| *** witek has quit IRC | 16:19 | |
| *** andreas__ has quit IRC | 16:22 | |
| *** andreas_s has joined #openstack-infra | 16:23 | |
| *** andrea___ has quit IRC | 16:23 | |
| pabelanger | sshnaidm: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.vars | 16:24 |
| *** dhajare has quit IRC | 16:24 | |
| pabelanger | EmilienM: looking | 16:24 |
| *** ijw has joined #openstack-infra | 16:25 | |
| sshnaidm | pabelanger, yeah, I saw this, but I'm looking for loading vars from file | 16:25 |
| pabelanger | sshnaidm: zuul doesn't directly support that, but ansible does | 16:25 |
| *** andreas_s has quit IRC | 16:27 | |
| pabelanger | EmilienM: I think the path forward is to remove branches from http://git.openstack.org/cgit/openstack/puppet-openstack-integration/tree/.zuul.yaml?h=stable/ocata#n367 and then add it to pipelines of projects you want to run the job on | 16:28 |
| pabelanger | EmilienM: eg: on the puppet-openstack-module-unit-jobs project-template jobs, then you can remove that from your projects that don't want to run the job | 16:28 |
| EmilienM | pabelanger: it's too much work, imho | 16:28 |
| pabelanger | and setup new branches | 16:28 |
| pabelanger | EmilienM: why? | 16:28 |
| EmilienM | I thought zuul did let you change the layout on a specific branch | 16:28 |
| EmilienM | because I just want to change the layout for one project on one branch | 16:29 |
| EmilienM | pabelanger: we have more than 40 modules | 16:29 |
| AJaeger | pabelanger: stable/ocata should not have branches lines in zuul.yaml AFAIU | 16:29 |
| pabelanger | EmilienM: well, the job is first getting loaded from p-o-i | 16:29 |
| EmilienM | i'm confused | 16:29 |
| EmilienM | I thought zuul was smart enough to read in-tree layout | 16:29 |
| pabelanger | and it has branch modifier applied | 16:29 |
| EmilienM | yeah but I override it | 16:29 |
| EmilienM | and I want my override taken in account | 16:30 |
| EmilienM | and it doesn't work | 16:30 |
| *** ijw has quit IRC | 16:30 | |
| EmilienM | now you're asking me to change the parent layout, which is not exactly what I'm asking | 16:30 |
| *** armax has quit IRC | 16:30 | |
| *** dave-mccowan has joined #openstack-infra | 16:31 | |
| EmilienM | the layout in-tree in the branch for the project should be high priority versus the parent | 16:31 |
| EmilienM | if you say no, then I don't get why we have in-tree layouts. | 16:31 |
| pabelanger | jobs are global, so it is getting loaded from p-o-i: http://logs.openstack.org/67/521967/6/check/puppet-openstack-unit-3.6-centos-7/91428f1/zuul-info/inventory.yaml | 16:31 |
| pabelanger | why that is, I am not sure | 16:31 |
| pabelanger | I am just giving you an option to fix it | 16:31 |
| EmilienM | and I'm saying this should not be this way for the user | 16:32 |
| EmilienM | (it's a feedback) | 16:32 |
| EmilienM | as a zuul user, I want to change the job layout for a specific project on a specific branch | 16:32 |
| EmilienM | and I can't. I need to change the parent layout for all projects | 16:32 |
| pabelanger | why all jobs? | 16:33 |
| pabelanger | I am saying | 16:33 |
| pabelanger | move http://git.openstack.org/cgit/openstack/puppet-openstack-integration/tree/.zuul.yaml?h=stable/ocata#n370 | 16:33 |
| pabelanger | to http://git.openstack.org/cgit/openstack/puppet-openstack-integration/tree/.zuul.yaml?h=stable/ocata#n402 | 16:33 |
| EmilienM | to me, https://review.openstack.org/#/c/521967/6/zuul.d/layout.yaml should work | 16:33 |
| EmilienM | I removed puppet-openstack-module-unit-jobs | 16:33 |
| EmilienM | and it's still running it | 16:33 |
| EmilienM | this is imho a bug | 16:33 |
| AJaeger | EmilienM: to me it looks like your setup is wrong, I would remove line 370 completely. | 16:34 |
| pabelanger | well, I am sure there is reasons why it does this, but I don't know them myself. It will need to be a jeblair question | 16:34 |
| EmilienM | mhh fine.. i'll do what you said. I'm just saying this is not the best user experience. | 16:34 |
| *** apetrich has quit IRC | 16:35 | |
| pabelanger | actually, AJaeger is right, why do you need like 370 in p-o-i? branch match is implied on octca | 16:35 |
| *** baoli has joined #openstack-infra | 16:35 | |
| pabelanger | and ocata branch shouldn't run anything on newton | 16:36 |
| AJaeger | EmilienM: read https://docs.openstack.org/infra/manual/zuulv3.html#how-jobs-are-selected-to-run-in-zuul-v3 | 16:37 |
| AJaeger | EmilienM: the logic is: When a job gets selected to run, it runs. You cannot unrun it in a child | 16:37 |
| *** e0ne has quit IRC | 16:38 | |
| AJaeger | EmilienM: so, in https://review.openstack.org/#/c/521967/ - just *remove* the jobs. You're on ocata branch, don't add branches selectors, just remove the job | 16:38 |
| pabelanger | I haven't done much work on zuul jobs for branches, but set a job with a branch modifier, then having another variant to modify the branch again, seems confusing | 16:38 |
| pabelanger | especically, when the jobs are running on the branch you are trying to match | 16:39 |
| AJaeger | EmilienM: mind if I edit 521967? | 16:39 |
| EmilienM | AJaeger: wait | 16:39 |
| EmilienM | AJaeger, pabelanger: https://review.openstack.org/522602 - how does it sound? | 16:39 |
| pabelanger | EmilienM: I actually think you might be able to just delete branches there | 16:40 |
| AJaeger | EmilienM, the project has branches, not sure about that... | 16:40 |
| pabelanger | EmilienM: well, atleast on stable/ocata if you made the changes there | 16:41 |
| EmilienM | AJaeger: I'm very confused at this stage. Feel free to edit anything. | 16:41 |
| AJaeger | EmilienM: you need to change the stable/ocata branch of p-o-i | 16:41 |
| EmilienM | yes I know | 16:41 |
| EmilienM | but I patch master and will backport | 16:41 |
| pabelanger | I don't think you want to change master | 16:41 |
| *** dizquierdo has quit IRC | 16:42 | |
| EmilienM | pabelanger: what does it hurt? | 16:42 |
| pabelanger | you can see in http://logs.openstack.org/67/521967/6/check/puppet-openstack-unit-3.6-centos-7/91428f1/zuul-info/inventory.yaml @ _inheritance_path job is loaded from stable/ocata | 16:42 |
| EmilienM | I'm doing the backports now. | 16:43 |
| EmilienM | and I'll depends-on | 16:43 |
| AJaeger | EmilienM: updated 521967 | 16:43 |
| pabelanger | EmilienM: well, what AJaeger and I are saying, we don't think you actually need to match on branches for stable/ocata, and if you delete it, your patch in puppet-tripleo should just work | 16:43 |
| EmilienM | AJaeger, pabelanger : no it didn't | 16:44 |
| EmilienM | I tried what AJaeger did | 16:44 |
| EmilienM | in PS5 | 16:44 |
| *** hashar is now known as hasharAway | 16:44 | |
| EmilienM | see https://review.openstack.org/#/c/521967/5/zuul.d/layout.yaml | 16:44 |
| EmilienM | I told you I tried everything it didn't work | 16:45 |
| *** dave-mccowan has quit IRC | 16:45 | |
| EmilienM | AJaeger: see PS5 jobs. puppet3 jobs still ran. | 16:45 |
| AJaeger | EmilienM: can you do a followup change on top of 521967 and check whether it runs the same jobs? | 16:45 |
| pabelanger | okay, now I am confused | 16:45 |
| AJaeger | MIght be that removing of jobs is not self-testing... | 16:45 |
| pabelanger | 1 sec | 16:45 |
| EmilienM | AJaeger: I can do that... | 16:45 |
| EmilienM | AJaeger: but that would be a bug | 16:46 |
| AJaeger | EmilienM: but then we know where the bug is ;) | 16:46 |
| pabelanger | right, what AJaeger did in 521967 is correct if you don't want puppet-openstack-unit-3.6-centos-7 to run. if it still does, then some project-template or something in project-config is adding the job still | 16:46 |
| EmilienM | AJaeger: done, 522608 | 16:46 |
| pabelanger | I don't think we actually want to load the job and use a branch modifier to stop it from running | 16:46 |
| AJaeger | bbl | 16:47 |
| EmilienM | AJaeger: doesn't work | 16:47 |
| EmilienM | AJaeger: puppet3 jobs worked | 16:47 |
| EmilienM | I think I need https://review.openstack.org/#/c/522604/ | 16:47 |
| *** ykarel|away has quit IRC | 16:47 | |
| EmilienM | pabelanger: https://review.openstack.org/#/c/521967/ didn't work | 16:49 |
| EmilienM | pabelanger: with the Depends-On | 16:49 |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config master: Create openstack/governance-sigs repository https://review.openstack.org/522543 | 16:49 |
| openstackgerrit | Daniel Alvarez proposed openstack-infra/system-config master: Index networking-ovn functional index logs in logstash https://review.openstack.org/522609 | 16:51 |
| pabelanger | EmilienM: no, I24f4d5e0c4496f9ea9ffddbfc0f75c8544304a37 isn't needed. The issue is, something is still adding your puppet3 jobs into the pipeline | 16:52 |
| pabelanger | trying to search all your configs now | 16:52 |
| *** yamamoto has joined #openstack-infra | 16:52 | |
| openstackgerrit | Merged openstack-infra/odsreg master: Update OpenStack logo https://review.openstack.org/505145 | 16:53 |
| openstackgerrit | Merged openstack-infra/odsreg master: Increase title to 60 chars https://review.openstack.org/505150 | 16:53 |
| openstackgerrit | Merged openstack-infra/odsreg master: Single topic uses admins to review https://review.openstack.org/505151 | 16:53 |
| *** sree has joined #openstack-infra | 16:54 | |
| *** sree has quit IRC | 16:59 | |
| pabelanger | EmilienM: check 522612, is that the right jobs you want running? | 17:00 |
| EmilienM | pabelanger: you're testing on master | 17:01 |
| EmilienM | but let me see | 17:01 |
| EmilienM | pabelanger: yes it is | 17:01 |
| *** smarcet has joined #openstack-infra | 17:01 | |
| pabelanger | EmilienM: okay, let me try something now | 17:02 |
| *** jpich has quit IRC | 17:07 | |
| *** yamamoto_ has joined #openstack-infra | 17:09 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job https://review.openstack.org/522618 | 17:11 |
| *** yamamoto has quit IRC | 17:13 | |
| *** lbragstad has joined #openstack-infra | 17:19 | |
| *** Goneri has quit IRC | 17:21 | |
| *** tesseract has quit IRC | 17:21 | |
| *** cshastri has joined #openstack-infra | 17:25 | |
| *** salv-orlando has quit IRC | 17:26 | |
| *** ijw has joined #openstack-infra | 17:26 | |
| *** salv-orlando has joined #openstack-infra | 17:27 | |
| *** psachin has quit IRC | 17:27 | |
| *** dizquierdo has joined #openstack-infra | 17:28 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Don't run functional tests when patching Puppetfile* https://review.openstack.org/522622 | 17:29 |
| pabelanger | EmilienM: do you still run puppet-openstack-unit-3.6-centos-7 on master branches? | 17:30 |
| EmilienM | no | 17:30 |
| EmilienM | you need to keep in mind the rules are different for p-o-i and puppet-tripleo | 17:30 |
| pabelanger | EmilienM: why is it being adding into the pipeline at http://git.openstack.org/cgit/openstack/puppet-openstack-integration/tree/.zuul.yaml#n394 for master? | 17:30 |
| pabelanger | that could be removed right? | 17:30 |
| EmilienM | pabelanger: could be | 17:31 |
| EmilienM | pabelanger: again, we did the layout on master | 17:31 |
| EmilienM | pabelanger: and backported. | 17:31 |
| * EmilienM afk 15 min | 17:31 | |
| *** ijw has quit IRC | 17:31 | |
| pabelanger | EmilienM: right, so you'll start having different layouts for branches and master now I think | 17:31 |
| *** salv-orlando has quit IRC | 17:31 | |
| pabelanger | EmilienM: lets propose a patch to remove puppet3 jobs on master p-o-i, as they will never run there again | 17:32 |
| pabelanger | EmilienM: same with syntax-3-legacy jobs | 17:32 |
| *** lucasagomes is now known as lucas-afk | 17:35 | |
| frickler | whoever is looking into dns issues, this seems to be an unsual kind of error to me, host cannot determine its own address/name? http://logs.openstack.org/20/522320/5/gate/openstack-tox-py35/26cc3c3/job-output.txt.gz#_2017-11-23_16_55_09_628051 | 17:35 |
| *** lbragstad has quit IRC | 17:36 | |
| *** salv-orlando has joined #openstack-infra | 17:37 | |
| *** salv-orlando has quit IRC | 17:37 | |
| *** salv-orlando has joined #openstack-infra | 17:37 | |
| pabelanger | frickler: the line above that doesn't look right | 17:40 |
| pabelanger | ERROR [pifpaf] Program did not print: `Available at http://' | 17:41 |
| pabelanger | or, is that because DNS failed | 17:41 |
| *** slaweq has joined #openstack-infra | 17:41 | |
| frickler | pabelanger: yeah, I'm thinking the error is the result of the dns failure | 17:43 |
| *** dtantsur is now known as dtantsur|afk | 17:44 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Create ovb-ha-ipv6 and deprecate fs024 job https://review.openstack.org/522618 | 17:44 |
| *** dbecker has quit IRC | 17:45 | |
| sshnaidm | pabelanger, can you please suggest how to debug it? https://review.openstack.org/#/c/522554/ I don't see any logs and no idea what went wrong.. | 17:49 |
| pabelanger | sshnaidm: you don't have access to localhost in untrusted-jobs, well you do but limited. That is actually the executor that we run jobs from | 17:51 |
| pabelanger | let me look at error | 17:51 |
| *** jamesmcarthur has joined #openstack-infra | 17:54 | |
| pabelanger | 2017-11-23 17:10:33,819 DEBUG zuul.AnsibleJob: [build: ea8a1908b0ec444a891bb90a4483ae95] Ansible output: b'ERROR! vars file /opt/stack/tripleo-ci/toci-quickstart/config/collect-logs.yml was not found' | 17:55 |
| pabelanger | sshnaidm: ^ | 17:55 |
| *** derekh has quit IRC | 17:55 | |
| pabelanger | we need to bubble those errors up to the user | 17:55 |
| sshnaidm | pabelanger, where do you see this error? | 17:56 |
| pabelanger | zuul-executor debug.log | 17:57 |
| pabelanger | like I said, we need to do a better job pushing that error into gerrit or ARA | 17:57 |
| *** jamesmcarthur has quit IRC | 17:58 | |
| sshnaidm | pabelanger, yeah, it's impossible to debug without logs.. | 17:59 |
| sshnaidm | pabelanger, is it possible to see this log in any way..? | 18:00 |
| pabelanger | sshnaidm: if you actually look at ARA, you'll see your playbook for post.yaml is missing, that is how I knew there was a syntax error with it | 18:01 |
| sshnaidm | pabelanger, if I don't know what's wrong, I can't know what to fix. I think it's blocker to move to zuulv3 native jobs | 18:01 |
| pabelanger | sshnaidm: sure, but you can run the post.yaml playbook locally too and see there is an issue, but agree we need to fix it | 18:02 |
| pabelanger | I wouldn't say a blocker, just harder to move to native zuulv3 | 18:03 |
| sshnaidm | pabelanger, there are no issues with post.yml, it's a real blocker | 18:03 |
| sshnaidm | pabelanger, it's not syntax error and I can't really run it locally, I need a zuul environment | 18:04 |
| *** Goneri has joined #openstack-infra | 18:04 | |
| pabelanger | Ya, i see the issue | 18:04 |
| pabelanger | zuul executor won't be able to load /opt/stack/tripleo-ci/toci-quickstart/config/collect-logs.yml | 18:04 |
| pabelanger | you need to use the relative path in from git | 18:05 |
| pabelanger | because zuul-executor doesn't have /opt | 18:05 |
| pabelanger | you are still thinking like ansible is running on the node | 18:05 |
| pabelanger | when it is not | 18:05 |
| sshnaidm | does it have it in /home/zuul/src/git.openstack.org ? | 18:05 |
| pabelanger | no, it will be {{ zuul.executor.work_root }}/src/git.o.o/ | 18:06 |
| sshnaidm | pabelanger, ok, so how can I reach playbooks from different project | 18:06 |
| pabelanger | https://docs.openstack.org/infra/zuul/feature/zuulv3/user/jobs.html#working-directory | 18:06 |
| *** jpena is now known as jpena|off | 18:07 | |
| pabelanger | what do you mean playbook from different project? | 18:07 |
| pabelanger | right now, only roles are reusable, playbooks are scoped to a project | 18:09 |
| *** shardy has quit IRC | 18:09 | |
| pabelanger | unless you did parent/child job | 18:09 |
| sshnaidm | pabelanger, nm, so I need to set it as {{ zuul.executor.work_root }}/src/git.o.o/tripleo-ci/toci-quickstart/config/collect-logs.yml ? | 18:09 |
| *** dklyle has joined #openstack-infra | 18:10 | |
| pabelanger | sshnaidm: yes, that should work for var_files | 18:10 |
| *** david-lyle has quit IRC | 18:10 | |
| sshnaidm | pabelanger, and instead of localhost - should I use "primary"? | 18:10 |
| pabelanger | sshnaidm: where do you want to playbook to run again? | 18:11 |
| pabelanger | sshnaidm: likely primary, localhost in zuulv3 context is our zuul-executor | 18:11 |
| sshnaidm | pabelanger, on primary node | 18:11 |
| pabelanger | sshnaidm: yah, if you are still using legacy nodeset, then it likley is primary | 18:12 |
| pabelanger | sshnaidm: but, it could be what ever you want to name it now | 18:12 |
| pabelanger | sshnaidm: you have that control if you want to change it | 18:12 |
| *** slaweq has quit IRC | 18:13 | |
| sshnaidm | pabelanger, I don't know how to do it, so I'd like to use what it's now.. | 18:13 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: POC: logs collection from native zuulv3 job https://review.openstack.org/522554 | 18:15 |
| *** yamamoto_ has quit IRC | 18:17 | |
| *** cshastri has quit IRC | 18:18 | |
| pabelanger | sshnaidm: another things to look into, we already have a few post log collection playbooks / roles for devstack, might just be able to start using them now | 18:20 |
| sshnaidm | pabelanger, we have a special role for this in quickstart | 18:20 |
| sshnaidm | pabelanger, it's completely different from what you can have in devstack, etc | 18:21 |
| *** ricolin has quit IRC | 18:23 | |
| *** tobiash has quit IRC | 18:26 | |
| pabelanger | sshnaidm: right, it is possible we can colapse them into a single role to support both cases | 18:27 |
| *** tosky has quit IRC | 18:28 | |
| *** ijw has joined #openstack-infra | 18:28 | |
| *** efoley has quit IRC | 18:29 | |
| *** dhajare has joined #openstack-infra | 18:29 | |
| *** tobiash has joined #openstack-infra | 18:30 | |
| *** ralonsoh has quit IRC | 18:31 | |
| *** ijw has quit IRC | 18:33 | |
| *** amoralej is now known as amoralej|off | 18:39 | |
| *** viktor_ has joined #openstack-infra | 18:43 | |
| jgriffith | pabelanger: clarkb https://review.openstack.org/#/c/522412/3 but looks like Andreas already merged a downgrade on that so either way things should be good now | 18:48 |
| AJaeger | jgriffith: wasn't me, I just linked the two options... | 18:49 |
| *** tlaxkit has quit IRC | 18:49 | |
| *** tlaxkit has joined #openstack-infra | 18:50 | |
| *** tlaxkit has quit IRC | 18:50 | |
| *** tlaxkit has joined #openstack-infra | 18:50 | |
| jgriffith | AJaeger: ahh, jianghuaw_ submitted one | 18:51 |
| jgriffith | Either way things should be all set now | 18:51 |
| pabelanger | yah, https://review.openstack.org/522447/ should be the fix for now | 18:52 |
| pabelanger | we can consider deleting the 3.5.0 wheel from xenial too if we wanted | 18:52 |
| pabelanger | since pypi doesn't exist :( | 18:52 |
| *** dizquierdo has quit IRC | 18:52 | |
| *** viktor_ has quit IRC | 18:52 | |
| jgriffith | pabelanger: sure, whatever folks want to do there. Just threw it up there if it's wanted | 18:52 |
| jgriffith | and on that not... I've got cooking to do :) | 18:53 |
| jgriffith | "note" even | 18:53 |
| *** rtjure has quit IRC | 18:54 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added get my member endpoint https://review.openstack.org/522638 | 18:54 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added get my member endpoint https://review.openstack.org/522638 | 18:56 |
| *** tosky has joined #openstack-infra | 19:01 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Updated get member endpoint https://review.openstack.org/522381 | 19:02 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Updated get member endpoint https://review.openstack.org/522381 | 19:03 |
| *** ijw has joined #openstack-infra | 19:09 | |
| *** dizquierdo has joined #openstack-infra | 19:09 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual master: Mention doc/requirements.txt https://review.openstack.org/522642 | 19:12 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual master: Mention doc/requirements.txt https://review.openstack.org/522642 | 19:14 |
| *** dizquierdo has quit IRC | 19:14 | |
| AJaeger | jeblair: ^ | 19:15 |
| *** ijw has quit IRC | 19:15 | |
| *** dhajare has quit IRC | 19:16 | |
| *** yamamoto has joined #openstack-infra | 19:17 | |
| *** Hal has quit IRC | 19:19 | |
| *** tlaxkit has quit IRC | 19:20 | |
| *** bobh has quit IRC | 19:23 | |
| pabelanger | AJaeger: ianw: i think we can +A 519535 now, happy to monitor too | 19:24 |
| *** yamamoto has quit IRC | 19:25 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Don't install centos-release-openstack-ocata https://review.openstack.org/519535 | 19:25 |
| AJaeger | pabelanger: rebased to remove WIP ^ | 19:25 |
| * AJaeger will +A now... | 19:25 | |
| pabelanger | KK | 19:26 |
| pabelanger | just did too :) | 19:26 |
| AJaeger | ;) | 19:26 |
| *** bobh has joined #openstack-infra | 19:26 | |
| pabelanger | I'll check nodepool-builders in 30mins or so after it merges, will line up well with next builds of centos-7 dib | 19:27 |
| AJaeger | ok, thanks | 19:30 |
| *** electrofelix has quit IRC | 19:30 | |
| pabelanger | EmilienM: dmsimard: ^FYI: moving forward of removal of centos-release-openstack-ocata from centos-7 DIBs | 19:31 |
| *** bobh has quit IRC | 19:32 | |
| AJaeger | pabelanger: could you review https://review.openstack.org/522392 https://review.openstack.org/522284 https://review.openstack.org/520376 , please? Some small fixes and cleanups... | 19:32 |
| AJaeger | and https://review.openstack.org/521497 is another cleanup | 19:32 |
| *** hasharAway is now known as hashar | 19:34 | |
| EmilienM | pabelanger: lgtm, we install the package ourselves when needed. | 19:35 |
| openstackgerrit | Merged openstack-infra/project-config master: Don't install centos-release-openstack-ocata https://review.openstack.org/519535 | 19:37 |
| pabelanger | AJaeger: done | 19:39 |
| *** sshnaidm is now known as sshnaidm|off | 19:40 | |
| openstackgerrit | Merged openstack-infra/project-config master: Temporarily remove requirements-check for monasca-ceilometer https://review.openstack.org/522392 | 19:42 |
| *** e0ne has joined #openstack-infra | 19:43 | |
| *** jaypipes has joined #openstack-infra | 19:43 | |
| *** slaweq has joined #openstack-infra | 19:45 | |
| AJaeger | thanks, pabelanger | 19:46 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: [rally] Remove rally-install jobs https://review.openstack.org/520376 | 19:49 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove legacy-murano-firstapp-docs https://review.openstack.org/521497 | 19:49 |
| *** ijw has joined #openstack-infra | 19:54 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Update upload-npm credentials secret https://review.openstack.org/522284 | 19:55 |
| *** wolverineav has joined #openstack-infra | 19:57 | |
| *** ijw has quit IRC | 20:00 | |
| *** ijw has joined #openstack-infra | 20:02 | |
| *** ijw has quit IRC | 20:06 | |
| *** ijw has joined #openstack-infra | 20:09 | |
| *** ldnunes has quit IRC | 20:12 | |
| *** ijw has quit IRC | 20:13 | |
| *** wolverineav has quit IRC | 20:15 | |
| *** wolverineav has joined #openstack-infra | 20:16 | |
| *** e0ne has quit IRC | 20:17 | |
| *** raissa has quit IRC | 20:18 | |
| AJaeger | pabelanger: we have a post job for openstack-manuals running since over 6 hours - should we kill that one? Or will it disappear on its own | 20:18 |
| *** armax has joined #openstack-infra | 20:18 | |
| pabelanger | AJaeger: should finish on its own, but possible there is an issue | 20:18 |
| pabelanger | AJaeger: lets see if it produces logs | 20:19 |
| *** ijw has joined #openstack-infra | 20:19 | |
| *** raissa has joined #openstack-infra | 20:19 | |
| AJaeger | it has produces logs - but is now since ages in "syncing AFS" - that's the final step | 20:19 |
| *** raissa has quit IRC | 20:19 | |
| *** e0ne has joined #openstack-infra | 20:19 | |
| *** raissa has joined #openstack-infra | 20:20 | |
| *** raissa has quit IRC | 20:20 | |
| *** wolverineav has quit IRC | 20:20 | |
| *** raissa has joined #openstack-infra | 20:20 | |
| *** wolverineav has joined #openstack-infra | 20:21 | |
| *** raissa has quit IRC | 20:21 | |
| pabelanger | AJaeger: sounds like either a lot of things to sync to AFS, or maybe just a network issue and job will eventually time out | 20:21 |
| *** raissa has joined #openstack-infra | 20:21 | |
| pabelanger | AJaeger: is running in infracloud, so leaning towards networking | 20:21 |
| *** raissa has quit IRC | 20:22 | |
| pabelanger | AJaeger: how much data will be sync'd? | 20:22 |
| AJaeger | it normally takes a few minutes, not hours | 20:22 |
| *** raissa has joined #openstack-infra | 20:22 | |
| *** raissa has quit IRC | 20:22 | |
| pabelanger | let me see if I can figure out node | 20:22 |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Fix /dev/pts mount options handling https://review.openstack.org/522654 | 20:22 |
| *** raissa has joined #openstack-infra | 20:23 | |
| *** raissa has quit IRC | 20:23 | |
| *** raissa has joined #openstack-infra | 20:24 | |
| *** raissa has quit IRC | 20:24 | |
| *** e0ne has quit IRC | 20:24 | |
| AJaeger | timeout is 1800 - and its hanging in this step for several hours | 20:24 |
| *** ijw has quit IRC | 20:25 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Fix /dev/pts mount options handling https://review.openstack.org/522654 | 20:25 |
| *** ijw has joined #openstack-infra | 20:26 | |
| pabelanger | AJaeger: actually, this would be running on the executor | 20:27 |
| pabelanger | not the node | 20:27 |
| pabelanger | hmm | 20:27 |
| AJaeger | indeed, it would | 20:28 |
| pabelanger | k, ze03 is where it is running | 20:29 |
| pabelanger | checking it now | 20:29 |
| AJaeger | jeblair: could you run your ".yaml" script again for jobs? A few changes merged and we have now repos without full path. Examples are tacker, zaqar. Looks like a dozen or so repos but I didn't check all files, just .zuul.yaml | 20:34 |
| pabelanger | AJaeger: somehow we lost the ansible-playbook for that process | 20:39 |
| pabelanger | AJaeger: err, job | 20:39 |
| AJaeger | oops ;( | 20:41 |
| AJaeger | we should be more robust in updating... | 20:41 |
| AJaeger | don't think we merged anything related here | 20:42 |
| pabelanger | http://paste.openstack.org/show/627262/ | 20:44 |
| pabelanger | so that is on the node side, the SSH connection from executor | 20:44 |
| pabelanger | 13:45 UTC was last connection attempt, and it was closed from executor | 20:45 |
| pabelanger | but, dont' see anything in executor-debug.log | 20:45 |
| AJaeger | ;( Can we kill it? | 20:48 |
| pabelanger | there is nothing to kill | 20:48 |
| pabelanger | zuul stopped running the job and didn't properly catch it | 20:48 |
| pabelanger | I think we'd need to restart the executor at this point | 20:49 |
| AJaeger | ah... | 20:49 |
| pabelanger | but want to check a few more things before I do that | 20:49 |
| *** yamamoto has joined #openstack-infra | 20:49 | |
| *** yamamoto has quit IRC | 20:49 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Stop using tox to build python tarballs https://review.openstack.org/513927 | 20:50 |
| *** baoli has quit IRC | 20:55 | |
| *** sree has joined #openstack-infra | 20:55 | |
| *** salv-orlando has quit IRC | 20:56 | |
| *** salv-orlando has joined #openstack-infra | 20:57 | |
| pabelanger | AJaeger: so, it looks like a few jobs that are over 7hours are related to ze03 | 20:57 |
| pabelanger | AJaeger: look at both check and gate pipelines | 20:57 |
| pabelanger | there must be something wrong with ze03, but I cannot see anything | 20:58 |
| pabelanger | I think at this point, we take it out of service and other infra-root can help audit it when back from holiday | 20:58 |
| *** sree has quit IRC | 20:59 | |
| *** yamamoto has joined #openstack-infra | 20:59 | |
| *** yamamoto has quit IRC | 20:59 | |
| pabelanger | however, I am not sure if this is going to abort the jobs properly or not | 21:00 |
| AJaeger | ooh, only checked post since it had openstack-manuals ;( | 21:00 |
| AJaeger | pabelanger: yes, let's take it out - and #status log that | 21:00 |
| pabelanger | yup | 21:00 |
| *** salv-orlando has quit IRC | 21:01 | |
| pabelanger | AJaeger: we cannot abort jobs via CLI yet, so we likely need to either abandon/restore in gerrit, promote, or restart scheduler | 21:01 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Stop using tox to build python tarballs https://review.openstack.org/513927 | 21:02 |
| pabelanger | 2017-11-23 21:02:21,493 DEBUG zuul.ExecutorServer: Stopped | 21:02 |
| pabelanger | let me check how many jobs we leaked | 21:02 |
| pabelanger | I see 61 directories left | 21:03 |
| pabelanger | that isn't good | 21:03 |
| AJaeger | 61? ;( | 21:04 |
| AJaeger | sounds like restart scheduler - otherwise the post job will never end ;( | 21:04 |
| pabelanger | AJaeger: ya, i think we can do a restart in a few hours, when it is slow | 21:05 |
| *** bobh has joined #openstack-infra | 21:05 | |
| pabelanger | AJaeger: for now, Iam going to abandon / restore patches to unblock gate | 21:05 |
| AJaeger | pabelanger: ah, that's what happened to tripleo... | 21:05 |
| pabelanger | yup | 21:06 |
| AJaeger | want to +2a https://review.openstack.org/#/c/522181/ - retire ceilometerclient? | 21:06 |
| pabelanger | #status log We seem to have an issue with ze03.o.o loosing ansible-playbook processes, as such we've been jobs continue to run in pipelines for 11+ hours. For now, I have stopped ze03 and will audit other servers | 21:06 |
| openstackstatus | pabelanger: finished logging | 21:06 |
| AJaeger | infra-root, we merged mordred's change 513927 to "stop using tox to build python tarballs" - in case publish-openstack-python-tarball gives problems... | 21:10 |
| AJaeger | config-core, please review https://review.openstack.org/#/c/522589/ to enable releasenotes on aodh again | 21:15 |
| *** hongbin has joined #openstack-infra | 21:16 | |
| AJaeger | config-core, one job removal for project-config that is ready now: https://review.openstack.org/#/c/519467/ | 21:18 |
| AJaeger | thanks, dmellado | 21:19 |
| AJaeger | dmellado: I meant to thank dmsimard for reviewing. But I'm grateful today, so thanks to you as well! | 21:19 |
| dmsimard | dmellado is used to be mixed up with me, it's alright | 21:20 |
| dmsimard | dmanchad too | 21:20 |
| dmsimard | we're the dm gang | 21:20 |
| AJaeger | ;) | 21:20 |
| dmsimard | AJaeger: I've been behind on reviews but I'll start ramping up again btw | 21:21 |
| AJaeger | tab completion was my problem | 21:21 |
| dmsimard | don't be mad at me | 21:21 |
| AJaeger | dmsimard: no worries | 21:21 |
| dmsimard | the 3 months long zuul v3 fire meant some other things had been neglected in the background | 21:21 |
| dmsimard | in fact I'm trying to wrap up zuul v3 things I haven't been able to get to the bottom of | 21:22 |
| dmsimard | like fixing the truncated json issue and the finger urls | 21:22 |
| AJaeger | ah, nice to see that . Thanks! | 21:23 |
| pabelanger | AJaeger: okay, I've cleared out everything i think was stuck, going to check nodepool now and see if we leaked any nodes | 21:23 |
| AJaeger | thanks, pabelanger . We still have the post job but that's cosmetic... | 21:23 |
| pabelanger | yah, I think we've leaked a fair bit of nodes too, i see seem them in-use for 9+hours | 21:24 |
| pabelanger | I suspect once we stop / start zuul they will clear out | 21:24 |
| *** ijw has joined #openstack-infra | 21:26 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy-tempest-dsvm-full-lio from Cinder https://review.openstack.org/519467 | 21:26 |
| *** Apoorva has joined #openstack-infra | 21:27 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy-tempest-dsvm-full-lio https://review.openstack.org/522661 | 21:27 |
| *** ijw has quit IRC | 21:32 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy-puppet-tripleo-puppet-unit jobs https://review.openstack.org/522662 | 21:34 |
| *** efoley has joined #openstack-infra | 21:36 | |
| AJaeger | config-core, please review https://review.openstack.org/522181 to retire ceilometer client. | 21:36 |
| *** efoley has quit IRC | 21:36 | |
| *** efoley has joined #openstack-infra | 21:36 | |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Temporarily disable releasenotes for aodh" https://review.openstack.org/522589 | 21:38 |
| *** bobh has quit IRC | 21:39 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove obsolete openstack-helm jobs https://review.openstack.org/522664 | 21:41 |
| *** bobh has joined #openstack-infra | 21:42 | |
| *** mat128 has quit IRC | 21:43 | |
| AJaeger | config-core, I pushed a couple of unused job removals in case anybody wants to review... https://review.openstack.org/522661 https://review.openstack.org/522662 https://review.openstack.org/522664 - thanks | 21:43 |
| * AJaeger waves good night now | 21:43 | |
| *** bobh has quit IRC | 21:46 | |
| openstackgerrit | Merged openstack-infra/project-config master: Retire ceilometerclient https://review.openstack.org/522181 | 21:46 |
| *** bobh has joined #openstack-infra | 21:47 | |
| *** liusheng has quit IRC | 21:47 | |
| dmsimard | I was poking fun at people who had to deal with the quantum -> neutron rename | 21:48 |
| *** liusheng has joined #openstack-infra | 21:48 | |
| dmsimard | zuul v3 is much much worse heh | 21:48 |
| *** kjackal has quit IRC | 21:54 | |
| *** ccamacho has quit IRC | 21:55 | |
| *** baoli has joined #openstack-infra | 21:55 | |
| *** salv-orlando has joined #openstack-infra | 21:58 | |
| *** yamamoto has joined #openstack-infra | 22:00 | |
| *** baoli has quit IRC | 22:00 | |
| *** slaweq has quit IRC | 22:03 | |
| *** salv-orlando has quit IRC | 22:03 | |
| *** yamamoto has quit IRC | 22:07 | |
| *** bobh has quit IRC | 22:14 | |
| *** bobh has joined #openstack-infra | 22:14 | |
| *** Apoorva has quit IRC | 22:15 | |
| *** bobh has quit IRC | 22:19 | |
| *** erlon has quit IRC | 22:20 | |
| dmsimard | pabelanger: I'm trying to understand zuul_return | 22:22 |
| *** smarcet has quit IRC | 22:23 | |
| dmsimard | pabelanger: my understanding is that all this module does is to write a json file that can be passed from a playbook to the other http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/ansible/library/zuul_return.py?h=feature/zuulv3 | 22:23 |
| dmsimard | pabelanger: "results.json", the default file, is actually loaded by the executor twice -- first here http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/executor/server.py?h=feature/zuulv3#n345 and then here: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/executor/server.py?h=feature/zuulv3#n740 | 22:24 |
| dmsimard | And we run zuul_return as the very very last task in upload-logs to provide the URL to the logs: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/upload-logs/tasks/main.yaml#n47 | 22:26 |
| *** florianf has quit IRC | 22:26 | |
| dmsimard | I'm not 100% positive but to me that means that if anything hard crashes the playbook run, we're not getting a results.json populated with the log url and it eventually escalates to a finger:// URL | 22:27 |
| dmsimard | I have this reproduced in a test deployment.. the executor on a successful run will print at the very end something like: 2017-11-23 22:00:51,088 DEBUG zuul.AnsibleJob: [build: 7933a7ab0e9a4a508cea5a3804d9e1c2] Sending result: {"data": {"zuul": {"log_url": "https://27-rc1.com/logs/7/7/6/check/break-things/7933a7a/"}}, "result": "SUCCESS"} | 22:28 |
| dmsimard | While a failed run will print something like this instead: 2017-11-23 21:58:13,928 DEBUG zuul.AnsibleJob: [build: 69114e85af144cf1be1c604e62e10c7f] Sending result: {"data": {}, "result": "POST_FAILURE"} | 22:28 |
| *** ijw has joined #openstack-infra | 22:29 | |
| pabelanger | dmsimard: as I understand it, we can move zuul_return to any place in our playbooks. So, it could be one of the first things we set in the base pre-run playbook | 22:29 |
| pabelanger | which, helps if things crash | 22:29 |
| pabelanger | specific to log_url, FWIW | 22:30 |
| dmsimard | pabelanger: well, yeah, that's a bit what I'm thinking.. we can deduce the log URL long before they are uploaded | 22:31 |
| dmsimard | pabelanger: worst case it turns up a 404 ? | 22:31 |
| dmsimard | which is probably more adequate than a finger URL | 22:31 |
| pabelanger | right, it doesn't fix the crash issue, but makes are gerrit comments a little nicer | 22:31 |
| dmsimard | pabelanger: yes and no, I'm troubleshooting the truncated json issue right now | 22:32 |
| dmsimard | pabelanger: I'm super confused because I'm able to reproduce it easily with zuul v3 but not through ansible-playbook and using zuul_stream/zuul_json | 22:33 |
| pabelanger | we only have 84GB of ram left for zuulv3 | 22:33 |
| pabelanger | then we start swapping | 22:33 |
| pabelanger | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=63979&rra_id=all | 22:33 |
| dmsimard | pabelanger: jeblair is not back until monday ? | 22:33 |
| *** ijw has quit IRC | 22:34 | |
| pabelanger | looks like 6 new .zuul.yaml files in check | 22:34 |
| pabelanger | chef repos | 22:35 |
| *** rcernin has joined #openstack-infra | 22:37 | |
| dmsimard | pabelanger: ok so I found part of the finger URL issue | 22:39 |
| pabelanger | cool | 22:39 |
| dmsimard | pabelanger: it's... a mix of things. I'll send an email to openstack-infra | 22:39 |
| pabelanger | I wish I knew how to dump zuulv3 queues, that would make scheduling a restart a little easier | 22:40 |
| *** hashar has quit IRC | 22:40 | |
| dmsimard | pabelanger: pretty sure I've seen fungi mention that it was the same thing as in v2 | 22:41 |
| *** efoley has quit IRC | 22:41 | |
| dmsimard | pabelanger: there's a python script to dump the current queues to a file, you restart and then load them back | 22:41 |
| pabelanger | almost | 22:42 |
| pabelanger | there is a typo in our docs | 22:42 |
| pabelanger | but looks like I got it | 22:42 |
| pabelanger | okay, queues dumped | 22:43 |
| pabelanger | I am going to stop scheduler | 22:43 |
| EmilienM | is there anything wrong right now with zuul? | 22:45 |
| pabelanger | and started | 22:46 |
| EmilienM | ok | 22:46 |
| EmilienM | there is something wrong | 22:46 |
| pabelanger | EmilienM: yes, we are restarting due to memory | 22:46 |
| EmilienM | arghhhhh | 22:46 |
| EmilienM | ok | 22:46 |
| pabelanger | I'll be enqueue changes in a moment | 22:46 |
| *** Goneri has quit IRC | 22:48 | |
| *** raissa has joined #openstack-infra | 22:49 | |
| pabelanger | #status notice Zuul has been restarted due to an unexpected issue. We're able to re-enqueue changes from check and gate pipelines, please check http://zuulv3.openstack.org/ for more information. | 22:53 |
| openstackstatus | pabelanger: sending notice | 22:53 |
| pabelanger | okay, check and gate restored | 22:53 |
| -openstackstatus- NOTICE: Zuul has been restarted due to an unexpected issue. We're able to re-enqueue changes from check and gate pipelines, please check http://zuulv3.openstack.org/ for more information. | 22:54 | |
| openstackstatus | pabelanger: finished sending notice | 22:56 |
| *** baoli has joined #openstack-infra | 22:56 | |
| pabelanger | #status log zuulv3.o.o restarted to address memory issues, was a few 100MB from swapping (15GB). Additionally, cleans up leaks nodepool nodes from previous ze03.o.o issues listed above (or below). | 22:58 |
| openstackstatus | pabelanger: finished logging | 22:58 |
| *** salv-orlando has joined #openstack-infra | 22:59 | |
| *** ganso has quit IRC | 23:00 | |
| *** wolverineav has quit IRC | 23:00 | |
| *** baoli has quit IRC | 23:01 | |
| *** pahuang has quit IRC | 23:01 | |
| *** jaypipes has quit IRC | 23:03 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Update zuulv3 docs for saving queues https://review.openstack.org/522678 | 23:03 |
| *** yamamoto has joined #openstack-infra | 23:03 | |
| *** dhill_ has quit IRC | 23:04 | |
| *** raissa has quit IRC | 23:04 | |
| *** raissa has joined #openstack-infra | 23:05 | |
| *** salv-orlando has quit IRC | 23:05 | |
| *** yamamoto has quit IRC | 23:08 | |
| *** slaweq has joined #openstack-infra | 23:08 | |
| *** pbourke has quit IRC | 23:13 | |
| *** slaweq has quit IRC | 23:14 | |
| *** rfolco has quit IRC | 23:15 | |
| *** pahuang has joined #openstack-infra | 23:15 | |
| *** pbourke has joined #openstack-infra | 23:19 | |
| *** raissa_ has joined #openstack-infra | 23:20 | |
| *** raissa has quit IRC | 23:21 | |
| *** linkmark has quit IRC | 23:27 | |
| *** ijw has joined #openstack-infra | 23:30 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Remove fake pingtest subunit https://review.openstack.org/522680 | 23:31 |
| *** armax has quit IRC | 23:34 | |
| *** armax has joined #openstack-infra | 23:34 | |
| *** armax has quit IRC | 23:34 | |
| *** armax has joined #openstack-infra | 23:35 | |
| *** ijw has quit IRC | 23:35 | |
| *** armax has quit IRC | 23:35 | |
| *** armax has joined #openstack-infra | 23:36 | |
| *** armax has quit IRC | 23:36 | |
| *** toabctl has quit IRC | 23:37 | |
| *** lbragstad has joined #openstack-infra | 23:56 | |
| *** baoli has joined #openstack-infra | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!