| *** eernst has quit IRC | 00:16 | |
| *** bobh has joined #openstack-infra | 00:16 | |
| *** jamesmcarthur has joined #openstack-infra | 00:17 | |
| *** markvoelker has joined #openstack-infra | 00:20 | |
| *** bobh has quit IRC | 00:20 | |
| *** bobh has joined #openstack-infra | 00:21 | |
| *** markvoelker has quit IRC | 00:25 | |
| openstackgerrit | Mohammed Naser proposed opendev/base-jobs master: logs: stop uploading to mtl1 https://review.opendev.org/678439 | 00:34 |
|---|---|---|
| mnaser | infra-core / infra-root: can we push this through right now? | 00:36 |
| fungi | mnaser: i already approved it ;) | 00:36 |
| mnaser | woah, that was quick. thanks fungi ! | 00:36 |
| fungi | thanks for the heads-up! | 00:36 |
| mnaser | i'll publish a revert with a -1 to keep it on my dashboard | 00:36 |
| ianw | ++ :) | 00:36 |
| fungi | good thinking | 00:36 |
| openstackgerrit | Mohammed Naser proposed opendev/base-jobs master: Revert "logs: stop uploading to mtl1" https://review.opendev.org/678440 | 00:37 |
| mnaser | ps it would be really good to look into finding a way that deduplicates all the ara stuff | 00:40 |
| mnaser | the static assets are duplicated by the thousands | 00:40 |
| openstackgerrit | Merged opendev/base-jobs master: logs: stop uploading to mtl1 https://review.opendev.org/678439 | 00:42 |
| *** hongbin has joined #openstack-infra | 00:51 | |
| fungi | yeah, i wonder if we could serve those somewhere centrally and then not bundle them in the archive upload | 00:53 |
| *** dchen has quit IRC | 01:30 | |
| *** bobh has quit IRC | 01:30 | |
| *** bobh_ has joined #openstack-infra | 01:30 | |
| *** dchen has joined #openstack-infra | 01:30 | |
| *** jamesmcarthur has quit IRC | 01:33 | |
| *** dchen has quit IRC | 01:43 | |
| *** dchen has joined #openstack-infra | 01:43 | |
| *** larainema has joined #openstack-infra | 01:49 | |
| *** eernst has joined #openstack-infra | 01:49 | |
| *** bobh_ has quit IRC | 01:51 | |
| *** eernst has quit IRC | 01:55 | |
| *** bobh has joined #openstack-infra | 01:57 | |
| *** bobh has quit IRC | 02:02 | |
| *** bobh has joined #openstack-infra | 02:09 | |
| *** bobh has quit IRC | 02:13 | |
| *** jamesmcarthur has joined #openstack-infra | 02:14 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: [dnm] lb idea https://review.opendev.org/678159 | 02:18 |
| *** bobh has joined #openstack-infra | 02:26 | |
| *** bobh has quit IRC | 02:30 | |
| *** jamesmcarthur has quit IRC | 02:37 | |
| *** dchen has quit IRC | 02:38 | |
| *** dchen has joined #openstack-infra | 02:39 | |
| *** masayukig has joined #openstack-infra | 02:41 | |
| *** bobh has joined #openstack-infra | 02:42 | |
| *** bobh has quit IRC | 02:46 | |
| *** markvoelker has joined #openstack-infra | 02:55 | |
| *** bobh has joined #openstack-infra | 02:58 | |
| *** markvoelker has quit IRC | 03:00 | |
| *** bobh has quit IRC | 03:02 | |
| *** jamesmcarthur has joined #openstack-infra | 03:04 | |
| *** rcernin_ has joined #openstack-infra | 03:15 | |
| *** rcernin has quit IRC | 03:15 | |
| *** bobh has joined #openstack-infra | 03:17 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: [dnm] lb idea https://review.opendev.org/678159 | 03:17 |
| *** bobh has quit IRC | 03:22 | |
| *** ijw has joined #openstack-infra | 03:27 | |
| *** jamesmcarthur has quit IRC | 03:33 | |
| *** bobh has joined #openstack-infra | 03:37 | |
| *** bobh has quit IRC | 03:41 | |
| *** bhavikdbavishi has joined #openstack-infra | 03:44 | |
| *** hamzy_ has joined #openstack-infra | 03:48 | |
| *** hamzy has quit IRC | 03:51 | |
| *** bobh has joined #openstack-infra | 03:55 | |
| *** ykarel has joined #openstack-infra | 03:59 | |
| *** bobh has quit IRC | 04:00 | |
| *** ramishra has joined #openstack-infra | 04:01 | |
| *** hongbin has quit IRC | 04:02 | |
| *** whoami-rajat has joined #openstack-infra | 04:04 | |
| AJaeger | config-core, the next round of promote jobs is ready, please review the topic promote-docs - https://review.opendev.org/#/q/status:open+++topic:promote-docs | 04:05 |
| *** udesale has joined #openstack-infra | 04:05 | |
| *** bobh has joined #openstack-infra | 04:07 | |
| *** bobh has quit IRC | 04:12 | |
| *** ykarel has quit IRC | 04:19 | |
| *** bobh has joined #openstack-infra | 04:19 | |
| *** markvoelker has joined #openstack-infra | 04:20 | |
| *** bobh has quit IRC | 04:24 | |
| *** ykarel has joined #openstack-infra | 04:24 | |
| *** markvoelker has quit IRC | 04:25 | |
| *** ijw has quit IRC | 04:29 | |
| *** ijw has joined #openstack-infra | 04:31 | |
| *** bobh has joined #openstack-infra | 04:31 | |
| *** bobh has quit IRC | 04:35 | |
| *** raukadah is now known as chkumar|rover | 04:37 | |
| *** bobh has joined #openstack-infra | 04:44 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Make haproxy role more generic https://review.opendev.org/677903 | 04:44 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: [dnm] lb idea https://review.opendev.org/678159 | 04:44 |
| *** bobh has quit IRC | 04:49 | |
| openstackgerrit | Merged openstack/os-testr master: Add Python 3 Train unit tests https://review.opendev.org/670636 | 04:55 |
| *** bobh has joined #openstack-infra | 04:56 | |
| *** bobh has quit IRC | 05:00 | |
| *** bobh has joined #openstack-infra | 05:08 | |
| *** bobh has quit IRC | 05:13 | |
| *** beekneemech has quit IRC | 05:16 | |
| *** bnemec has joined #openstack-infra | 05:20 | |
| *** ijw_ has joined #openstack-infra | 05:24 | |
| *** ijw has quit IRC | 05:28 | |
| *** bobh has joined #openstack-infra | 05:35 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Make haproxy role more generic https://review.opendev.org/677903 | 05:39 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: [dnm] lb idea https://review.opendev.org/678159 | 05:39 |
| *** odicha has joined #openstack-infra | 05:40 | |
| *** bobh has quit IRC | 05:40 | |
| *** AJaeger has quit IRC | 05:45 | |
| *** AJaeger has joined #openstack-infra | 05:48 | |
| *** bobh has joined #openstack-infra | 05:49 | |
| *** jaosorior has joined #openstack-infra | 05:51 | |
| *** ijw_ has quit IRC | 05:53 | |
| *** bobh has quit IRC | 05:54 | |
| openstackgerrit | Merged openstack/project-config master: Remove now obsolete publish jobs https://review.opendev.org/677804 | 05:55 |
| *** aluria has joined #openstack-infra | 05:56 | |
| *** bobh has joined #openstack-infra | 06:01 | |
| *** dchen has quit IRC | 06:01 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Make haproxy role more generic https://review.opendev.org/677903 | 06:01 |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: [dnm] lb idea https://review.opendev.org/678159 | 06:01 |
| *** dchen has joined #openstack-infra | 06:05 | |
| *** bobh has quit IRC | 06:05 | |
| *** xarses_ has quit IRC | 06:07 | |
| *** andreww has joined #openstack-infra | 06:07 | |
| *** eernst has joined #openstack-infra | 06:08 | |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: promote: Fix PDF path https://review.opendev.org/678463 | 06:10 |
| AJaeger | config-core, one more, please ^ | 06:10 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: promote: Fix PDF path https://review.opendev.org/678463 | 06:12 |
| *** bobh has joined #openstack-infra | 06:13 | |
| AJaeger | thanks, ianw for reviewing! | 06:13 |
| ianw | AJaeger: should just merge? what was the job failure? | 06:13 |
| AJaeger | "/var/lib/zuul/builds/488f93fca2be428bb538007a24814ca0/work/pdf/ was skipped as it does not seem to be a valid directory or it cannot be accessed" | 06:14 |
| AJaeger | see http://zuul.opendev.org/t/openstack/build/488f93fca2be428bb538007a24814ca0/console | 06:14 |
| AJaeger | Find PDF role | 06:14 |
| AJaeger | but we downloaded a PDF... | 06:15 |
| *** andreww has quit IRC | 06:16 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add PDF to api promote job https://review.opendev.org/678344 | 06:16 |
| AJaeger | ianw: could you review this again - and also https://review.opendev.org/678256, please? | 06:17 |
| *** xarses has joined #openstack-infra | 06:17 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote jobs for static site / releasenotes https://review.opendev.org/678351 | 06:17 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Remove now unused publish jobs https://review.opendev.org/678356 | 06:17 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add allowed-projects to static publish jobs https://review.opendev.org/678357 | 06:17 |
| *** bobh has quit IRC | 06:18 | |
| *** rcernin_ has quit IRC | 06:18 | |
| *** lpetrut has joined #openstack-infra | 06:18 | |
| *** ykarel is now known as ykarel|afk | 06:21 | |
| *** dpawlik has joined #openstack-infra | 06:23 | |
| *** ijw has joined #openstack-infra | 06:24 | |
| AJaeger | ianw: could you +2 678463 again, please? I just added an "/" | 06:27 |
| ianw | :) | 06:27 |
| AJaeger | thanks | 06:27 |
| openstackgerrit | Merged openstack/openstack-zuul-jobs master: Rebase api-guide/ref jobs https://review.opendev.org/678256 | 06:27 |
| *** bobh has joined #openstack-infra | 06:28 | |
| *** ijw has quit IRC | 06:29 | |
| *** ykarel|afk is now known as ykarel | 06:32 | |
| *** bobh has quit IRC | 06:33 | |
| *** takamatsu has joined #openstack-infra | 06:34 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Fix weak dependencies to work with child_jobs https://review.opendev.org/677936 | 06:39 |
| *** takamatsu has quit IRC | 06:42 | |
| AJaeger | frickler: could you review 678463, please? (and if possible rest of https://review.opendev.org/#/q/status:open+++topic:promote-docs at some time as well...) | 06:43 |
| *** bobh has joined #openstack-infra | 06:43 | |
| *** bobh has quit IRC | 06:48 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Fix weak dependencies to work with child_jobs https://review.opendev.org/677936 | 06:50 |
| *** zhurong has joined #openstack-infra | 06:55 | |
| *** ginopc has joined #openstack-infra | 06:57 | |
| *** ijw has joined #openstack-infra | 06:58 | |
| *** trident has quit IRC | 07:01 | |
| *** ykarel is now known as ykarel|afk | 07:01 | |
| *** ijw_ has joined #openstack-infra | 07:02 | |
| *** ijw has quit IRC | 07:04 | |
| *** jtomasek has joined #openstack-infra | 07:07 | |
| *** odicha has quit IRC | 07:09 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Fix weak dependencies to work with child_jobs https://review.opendev.org/677936 | 07:10 |
| *** trident has joined #openstack-infra | 07:10 | |
| *** jpena|off is now known as jpena | 07:11 | |
| *** bobh has joined #openstack-infra | 07:11 | |
| *** bobh has quit IRC | 07:16 | |
| *** sshnaidm|afk is now known as sshnaidm | 07:16 | |
| openstackgerrit | Ian Wienand proposed opendev/system-config master: Add a service load balancer https://review.opendev.org/678159 | 07:19 |
| *** odicha has joined #openstack-infra | 07:22 | |
| *** bobh has joined #openstack-infra | 07:23 | |
| ianw | clarkb: it seems the 2019-08-24T00:44:34,046144208+00:00 fedora run failed | 07:25 |
| ianw | Release failed: rxk: authentication expired | 07:25 |
| ianw | Could not end transaction on a ro volume: rxk: authentication expired | 07:25 |
| ianw | Could not update VLDB entry for volume 536871006 | 07:25 |
| ianw | it's been locked ever since | 07:25 |
| *** bobh has quit IRC | 07:28 | |
| ianw | unfortunately, no timestamps ... maybe network? perhaps we shoudl run this through the same filter we use for devstack that timestamps each line | 07:28 |
| ianw | i'm re-running manually in a screen on mirror-udpate.opendev.org now after manually unlocking | 07:29 |
| ianw | #status log unlocked mirror.fedora volume after afs release failure @ 2019-08-24T00:44:34 | 07:29 |
| openstackstatus | ianw: finished logging | 07:29 |
| *** ijw_ has quit IRC | 07:29 | |
| *** ijw has joined #openstack-infra | 07:30 | |
| *** roman_g has joined #openstack-infra | 07:35 | |
| *** bobh has joined #openstack-infra | 07:35 | |
| *** bobh has quit IRC | 07:39 | |
| *** ykarel|afk is now known as ykarel | 07:43 | |
| *** zhurong has quit IRC | 07:43 | |
| *** apetrich has joined #openstack-infra | 07:43 | |
| *** bobh has joined #openstack-infra | 07:45 | |
| openstackgerrit | Monty Taylor proposed opendev/git-review master: Install commit hook into submodules https://review.opendev.org/678428 | 07:48 |
| *** bobh has quit IRC | 07:49 | |
| *** markvoelker has joined #openstack-infra | 08:02 | |
| *** eernst has quit IRC | 08:04 | |
| *** bobh has joined #openstack-infra | 08:05 | |
| *** markvoelker has quit IRC | 08:07 | |
| *** ijw has quit IRC | 08:09 | |
| *** derekh has joined #openstack-infra | 08:09 | |
| *** bobh has quit IRC | 08:10 | |
| openstackgerrit | Monty Taylor proposed opendev/puppet-gerrit master: Add support for replicateOnStartup config option https://review.opendev.org/678486 | 08:13 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Don't run replication on gerrit startup https://review.opendev.org/678487 | 08:16 |
| *** dchen has quit IRC | 08:17 | |
| *** bobh has joined #openstack-infra | 08:18 | |
| *** gfidente has joined #openstack-infra | 08:18 | |
| AJaeger | mordred: you're up early ;) Could I bother you with reviewing https://review.opendev.org/678463 (and rest of topic:promote-docs if possible as well), please? | 08:21 |
| *** bobh has quit IRC | 08:22 | |
| mordred | AJaeger: I'm in the Europe! | 08:23 |
| mordred | AJaeger: corvus and I are in sweden at the Gerrit User Summit | 08:24 |
| mordred | and done | 08:24 |
| *** zerick has quit IRC | 08:25 | |
| *** ykarel is now known as ykarel|lunch | 08:26 | |
| *** pkopec has joined #openstack-infra | 08:27 | |
| *** janki has joined #openstack-infra | 08:29 | |
| *** tkajinam has quit IRC | 08:30 | |
| *** bobh has joined #openstack-infra | 08:32 | |
| *** janki has quit IRC | 08:32 | |
| *** janki has joined #openstack-infra | 08:34 | |
| openstackgerrit | Merged opendev/base-jobs master: promote: Fix PDF path https://review.opendev.org/678463 | 08:35 |
| *** bobh has quit IRC | 08:36 | |
| *** ralonsoh has joined #openstack-infra | 08:40 | |
| *** ramishra has quit IRC | 08:43 | |
| *** xenos76 has joined #openstack-infra | 08:44 | |
| *** ramishra has joined #openstack-infra | 08:46 | |
| *** dtantsur|afk is now known as dtantsur | 08:46 | |
| *** bobh has joined #openstack-infra | 08:48 | |
| *** bobh has quit IRC | 08:52 | |
| *** strigazi has joined #openstack-infra | 08:58 | |
| *** ykarel|lunch is now known as ykarel | 09:07 | |
| *** rcernin_ has joined #openstack-infra | 09:13 | |
| *** bobh has joined #openstack-infra | 09:18 | |
| *** bobh has quit IRC | 09:23 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove the extra bazel options https://review.opendev.org/678412 | 09:24 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Build gerrit with bazel 0.27.1 instead https://review.opendev.org/673141 | 09:24 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split out bazel builder and gerrit base image https://review.opendev.org/673142 | 09:24 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Put bazel options into bazelrc https://review.opendev.org/673143 | 09:24 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Put bazel options into bazelrc https://review.opendev.org/673143 | 09:24 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Remove the extra bazel options https://review.opendev.org/678412 | 09:26 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Build gerrit with bazel 0.27.1 instead https://review.opendev.org/673141 | 09:26 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Split out bazel builder and gerrit base image https://review.opendev.org/673142 | 09:27 |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Put bazel options into bazelrc https://review.opendev.org/673143 | 09:27 |
| mordred | corvus, clarkb: That stack should be ready for re-review, assuming it's all green. sorry for creating a file and then removing a file - I could squash - but I thought the churn was good for verifying the build was working :) | 09:29 |
| *** bobh has joined #openstack-infra | 09:35 | |
| *** bobh has quit IRC | 09:40 | |
| *** rcernin_ has quit IRC | 09:42 | |
| *** bhavikdbavishi has quit IRC | 09:45 | |
| *** apetrich has quit IRC | 09:49 | |
| *** bobh has joined #openstack-infra | 09:51 | |
| *** bobh has quit IRC | 09:55 | |
| *** bobh has joined #openstack-infra | 09:56 | |
| *** bobh has quit IRC | 10:00 | |
| AJaeger | mordred, thanks! Enjoy the summit! | 10:02 |
| *** markvoelker has joined #openstack-infra | 10:05 | |
| *** markvoelker has quit IRC | 10:10 | |
| *** xenos76 has quit IRC | 10:15 | |
| *** xenos76 has joined #openstack-infra | 10:21 | |
| *** jaosorior has quit IRC | 10:26 | |
| *** bobh has joined #openstack-infra | 10:28 | |
| *** bobh has quit IRC | 10:33 | |
| *** markvoelker has joined #openstack-infra | 10:35 | |
| *** markvoelker has quit IRC | 10:40 | |
| mnaser | infra-root: I submitted a patch to drop swift uploads from opendev/base-jobs but I'm still seeing a lot of uploads. Should there be somewhere else to remove it? | 10:49 |
| mnaser | It's making it hard to work on this maintenance | 10:50 |
| AJaeger | mnaser: remove it from playbooks/base/post-logs.yaml | 10:51 |
| AJaeger | you removed it from base-minimal which is unused... | 10:52 |
| mnaser | ahhh :( | 10:52 |
| AJaeger | mnaser: do you want to sent a patch? I would +2A... | 10:52 |
| mnaser | AJaeger: not near a computer right now | 10:52 |
| mnaser | I just got a message this morning that we still see a lot of requests | 10:53 |
| AJaeger | mnaser: let me do it... | 10:54 |
| mnaser | AJaeger: thanks | 10:55 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Remvoe vexxhost from swift upload https://review.opendev.org/678518 | 10:55 |
| AJaeger | infra-root, I'll self approve the change above ^ | 10:55 |
| openstackgerrit | Andreas Jaeger proposed opendev/base-jobs master: Revert "logs: stop uploading to mtl1" https://review.opendev.org/678440 | 10:57 |
| AJaeger | mnaser: updated the revert up as well ^ | 10:58 |
| AJaeger | mnaser: should merge in less than 5 minutes... | 10:59 |
| openstackgerrit | Merged opendev/base-jobs master: Remvoe vexxhost from swift upload https://review.opendev.org/678518 | 11:03 |
| *** udesale has quit IRC | 11:04 | |
| AJaeger | mnaser: merged, hope your team can do maintenance better now ^ | 11:05 |
| *** tesseract has joined #openstack-infra | 11:12 | |
| *** ccamacho has joined #openstack-infra | 11:21 | |
| *** jpena is now known as jpena|lunch | 11:25 | |
| *** rosmaita has joined #openstack-infra | 11:36 | |
| *** apetrich has joined #openstack-infra | 11:41 | |
| *** Garyx has quit IRC | 11:43 | |
| *** jaosorior has joined #openstack-infra | 11:44 | |
| *** jroll has quit IRC | 11:44 | |
| *** jroll has joined #openstack-infra | 11:45 | |
| noonedeadpunk | Hi everyone. We've got some problems with zuul manifest task on rocky. so we have this http://paste.openstack.org/show/763464/ for https://review.opendev.org/#/c/677716 and http://paste.openstack.org/show/764537/ for https://review.opendev.org/#/c/675886/ | 11:49 |
| noonedeadpunk | So I'd say that there's extra logs folder in the path, but not really sure... | 11:49 |
| *** rh-jelabarre has joined #openstack-infra | 11:53 | |
| *** rcernin_ has joined #openstack-infra | 11:53 | |
| *** rlandy has joined #openstack-infra | 11:59 | |
| *** rlandy is now known as rlandy|ruck | 11:59 | |
| *** derekh has quit IRC | 12:00 | |
| *** markvoelker has joined #openstack-infra | 12:00 | |
| *** e0ne has joined #openstack-infra | 12:01 | |
| *** guoqiao has quit IRC | 12:07 | |
| *** weshay_MOD is now known as weshay | 12:07 | |
| *** larainema has quit IRC | 12:08 | |
| noonedeadpunk | Actually, can you somehow provide access to that vm (hold?) so that we can check why these files were deiscovered by https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/generate-zuul-manifest/library/generate_manifest.py#L52 at the first place? | 12:15 |
| *** jpena|lunch is now known as jpena | 12:30 | |
| *** rcernin_ has quit IRC | 12:32 | |
| frickler | noonedeadpunk: I think there are two issues, the first one seems to be triggered by the "ip" command not being available here https://opendev.org/openstack/openstack-ansible-tests/src/branch/master/test-log-collect.sh#L283 | 12:38 |
| frickler | noonedeadpunk: for the second one, I'd guess the missing file to be a dangling symlink. I can set up a hold, but I'm not 100% sure whether it will be triggered on POST_FAILURE. which job would you like to check. | 12:39 |
| *** xek_ has joined #openstack-infra | 12:42 | |
| frickler | the post failures also only seem to occur for os_horizon and os_aodh | 12:43 |
| *** mgoddard has quit IRC | 12:47 | |
| *** xek_ has quit IRC | 12:47 | |
| *** jamesmcarthur has joined #openstack-infra | 12:47 | |
| *** mgoddard has joined #openstack-infra | 12:47 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Put bazel options into bazelrc https://review.opendev.org/673143 | 12:49 |
| *** slaweq has joined #openstack-infra | 12:58 | |
| AJaeger | fungi, frickler , could you review https://review.opendev.org/678344 and https://review.opendev.org/678351 for further docs promote job work, please? | 12:58 |
| *** e0ne has quit IRC | 12:59 | |
| *** Goneri has joined #openstack-infra | 13:02 | |
| *** sean-k-mooney has joined #openstack-infra | 13:03 | |
| sean-k-mooney | o/ | 13:03 |
| noonedeadpunk | frickler: actually not only, but also for ansible-role-systemd_networkd, os_neutron | 13:03 |
| sean-k-mooney | quick question is it me or have there been more POST_FAILURES lately. im wondering if this could be related to swaping to swirft for logs? | 13:04 |
| *** e0ne has joined #openstack-infra | 13:05 | |
| AJaeger | sean-k-mooney: I'm not aware of a current problem - why do you think this might be a problem? | 13:09 |
| jrosser | sean-k-mooney: we are seeing many in openstack-ansible, digging into it so far gives this http://paste.openstack.org/show/764537/ | 13:09 |
| jrosser | also http://zuul.openstack.org/builds?result=POST_FAILURE# | 13:09 |
| *** eharney has quit IRC | 13:09 | |
| sean-k-mooney | AJaeger: i have just seen a lot of random build fail with it over teh last few weeks while reviewing | 13:10 |
| sean-k-mooney | the reaon i remember it is in several cases all other jobs have passed and we have just rechecked due to post_failture | 13:11 |
| sean-k-mooney | and on the recheck it was fine | 13:11 |
| AJaeger | sean-k-mooney: the issue that jrosser and noonedeadpunk mentioned is indeed one on openstack-ansible... | 13:11 |
| AJaeger | please report them here and I hope somebody can look into them... | 13:11 |
| sean-k-mooney | well https://review.opendev.org/#/c/678497/ failed on bashate | 13:11 |
| jrosser | they are really hard to debug becasue no logs get collected | 13:12 |
| sean-k-mooney | and i have see it on other tox jobs too | 13:12 |
| jrosser | i only got that paste becasue i watched the job console to the bitter end and copied it out of my browser | 13:12 |
| AJaeger | jrosser: if you expect it to fail, tail the log in zuul.opendev.org | 13:12 |
| *** janki has quit IRC | 13:12 | |
| AJaeger | jrosser: yep, exactly... | 13:12 |
| sean-k-mooney | jrosser: yep how do i debug no logs.... look at the oh | 13:12 |
| eandersson | Anyone happen to have an example on how to set up ssl in devstack for a... non-wsgi ready service? | 13:12 |
| AJaeger | eandersson: you might better ask in #openstack-qa | 13:13 |
| eandersson | Will do - thanks | 13:13 |
| jrosser | AJaeger: for reasons not understood a lot of these POST_FAILURES affect the OSA rocky branch see the many failed jobs here https://review.opendev.org/#/q/topic:goodbye_42.3+(status:open+OR+status:merged) | 13:15 |
| AJaeger | jrosser: yeah, was surprised there as well... | 13:16 |
| jrosser | imho this is all linked to generate-zuul-manifext which seems to have been introduced at a similar time to these things started failing | 13:17 |
| AJaeger | let's ask mordred and corvus - but they are conferencing right now ^ | 13:17 |
| sean-k-mooney | is there a way in kibana to filer on post failures | 13:17 |
| sean-k-mooney | it looks like its not in build_status | 13:17 |
| donnyd | node_provider:"fortnebula-regionone" AND filename:job-output.txt AND message:"ON POST FAILURE" | 13:18 |
| donnyd | or something like that | 13:18 |
| sean-k-mooney | ha you have to use message | 13:18 |
| sean-k-mooney | actully that wont work becasue there will be non logs in logstace for kiban to filter on | 13:19 |
| donnyd | Its going to be something like that | 13:19 |
| sean-k-mooney | because it failed to upload the logs because its a post_failure | 13:20 |
| *** dave-mccowan has joined #openstack-infra | 13:20 | |
| jrosser | hmm yes i tried earlier and could find success & failure but not post_failure | 13:20 |
| sean-k-mooney | right but since post failure means no logs logstash would not know anything about that build right | 13:20 |
| *** haleyb|away is now known as haleyb | 13:21 | |
| AJaeger | jrosser, sean-k-mooney, sorry, can't help further myself, hope somebody else will be around later and has an idea | 13:21 |
| sean-k-mooney | its not really an issue at the moement just notice it more often then normal | 13:21 |
| sean-k-mooney | actully it looks liek we have som e logs for post failter | 13:22 |
| jrosser | for OSA it is a big issue becasue our rocky branch basically can't merge any code now for a couple of weeks | 13:22 |
| mnaser | fwiw: infra generated 19 million objects since swift was added | 13:23 |
| donnyd | I am looking through here, your right I only see RESULT_NORMAL results | 13:25 |
| donnyd | node_provider:"fortnebula-regionone" AND filename:"job-output.txt" AND message:"POST-RUN END" | 13:25 |
| sean-k-mooney | mnaser: do they get deleted eventurely | 13:25 |
| AJaeger | mnaser: how many objects were in your swift cluster before we used it? ;) | 13:25 |
| mnaser | AJaeger: that 19 million objects were only by infra, the issue is that infra uploads *so much* so quickly | 13:25 |
| mnaser | so ceph doesn't like it (esp using filestore) and struggles to keep up with file splitting | 13:25 |
| mnaser | sean-k-mooney: they should? | 13:25 |
| *** ykarel is now known as ykarel|away | 13:26 | |
| AJaeger | mnaser: wow ;( | 13:26 |
| mnaser | in one day, 2.4 million objects uploaded were just ara-report | 13:26 |
| *** aaronsheffield has joined #openstack-infra | 13:26 | |
| sean-k-mooney | mnaser: they should afte 3 months when they expire the old logs or however long the retention peiord is | 13:26 |
| AJaeger | mnaser , read your email, that's indeed something to optimize... | 13:27 |
| donnyd | mnaser i have around the same in FN | 13:27 |
| donnyd | good test of my swift cluster.. :) :) | 13:27 |
| donnyd | about 228 containers, 19M objects | 13:28 |
| mnaser | so for example, in a day, ~2.6m PUTs from infra, 1.5m of those were ara-report | 13:28 |
| mnaser | that's a significant overhead.. | 13:28 |
| sean-k-mooney | mnaser: clearly you need latest an gretest intel optaine ram hybrid stroage system. that will be 1 kidney or your first born to upgrade | 13:28 |
| mnaser | sean-k-mooney: lolol | 13:28 |
| mnaser | not wrong :) | 13:29 |
| AJaeger | mnaser: hope dmsimard has an idea... | 13:29 |
| AJaeger | jrosser: did you see frickler's comment above on missing "ip"? Please try getting it installed and see whether that fixes some of the issues | 13:34 |
| jrosser | That isn’t failing th job | 13:34 |
| jrosser | You see just below that the play succeeds, and then the post job is the one that returns failure | 13:35 |
| jrosser | That missing ip message has been there for ever in those jobs | 13:35 |
| *** slaweq has quit IRC | 13:36 | |
| mnaser | jrosser: i actually see the possible issue here in zuul | 13:38 |
| mnaser | jrosser, noonedeadpunk do you have a hold on an instance that has that sisue? | 13:41 |
| frickler | jrosser: ah, sorry, I didn't read that first paste to the end. but I confirmed locally that os.stat() gives me FileNotFoundError for a dangling symlink. so we need to add error handling for that | 13:41 |
| frickler | mnaser: ^^ | 13:41 |
| mnaser | frickler: yeah that's exactly my theory too | 13:41 |
| mnaser | i was trying to setup a test | 13:42 |
| *** jeliu_ has joined #openstack-infra | 13:42 | |
| noonedeadpunk | frickler: and is there at least /var/lib/zuul/builds/e5c0e0bf87b44decbe4ea3e307835b7a/work/logs/logs/etc/ folder? | 13:42 |
| *** jamesmcarthur has quit IRC | 13:44 | |
| noonedeadpunk | Yeah, that should be ok | 13:44 |
| frickler | corvus: that is the line appearing in the backtrace. maybe we add a "try:" around it and just ignore those that fail? https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/generate-zuul-manifest/library/generate_manifest.py#L70 | 13:45 |
| *** jamesmcarthur has joined #openstack-infra | 13:47 | |
| AJaeger | frickler: a try with continue - so that the next file is tried? Yeah, sounds like right approach... | 13:47 |
| mnaser | hmm | 13:49 |
| mnaser | i cant replicate this here locally | 13:50 |
| *** e0ne has quit IRC | 13:51 | |
| frickler | mnaser: do a "ln -s x y" and then os.stat("y"). assuming x doesn't exist, I get that error | 13:51 |
| openstackgerrit | Jens Harbott (frickler) proposed zuul/zuul-jobs master: Fix handling of dangling symlink on manifest generation https://review.opendev.org/678552 | 13:52 |
| *** jamesmcarthur has quit IRC | 13:52 | |
| AJaeger | corvus, mordred ^ | 13:52 |
| frickler | oh, I should actually handle the exception | 13:52 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Check path for existance in generate_manisfest.py https://review.opendev.org/678553 | 13:52 |
| noonedeadpunk | frickler: mnaser shouldn't that handle this? ^ | 13:53 |
| mnaser | i think frickler approach should be cleaner because *technically* an os.walk should give us all 'clean' files | 13:54 |
| mnaser | and existing ones | 13:54 |
| noonedeadpunk | at least it needs except: pass | 13:55 |
| noonedeadpunk | but, I'd say that non-existent files shouldn't be present in the file list | 13:55 |
| fungi | it's a eafp vs lbyl distinction | 13:56 |
| *** mgoddard has quit IRC | 13:56 | |
| openstackgerrit | Jens Harbott (frickler) proposed zuul/zuul-jobs master: Fix handling of dangling symlink on manifest generation https://review.opendev.org/678552 | 13:57 |
| noonedeadpunk | yep, you're right | 13:57 |
| frickler | listdir() lists files, dirs and symlinks. they all exist in a certain sense, it's just that stat() errors out on dangling symlinks | 13:58 |
| AJaeger | thanks, frickler. | 13:59 |
| noonedeadpunk | yeah, just os.path.exists checks if symlink is not borken | 13:59 |
| *** mriedem has joined #openstack-infra | 13:59 | |
| AJaeger | jrosser: see 678552 | 13:59 |
| noonedeadpunk | and thanks jrosser:) | 14:00 |
| *** apetrich has quit IRC | 14:00 | |
| mnaser | frickler: im working on tests for your change | 14:00 |
| *** eharney has joined #openstack-infra | 14:01 | |
| *** odicha has quit IRC | 14:01 | |
| *** apetrich has joined #openstack-infra | 14:03 | |
| corvus | mnaser, frickler: tests would be great -- there's a unit test suite for that role, which we normally would expect to add a test to for this before merging | 14:07 |
| mnaser | yes, i am working on those :) i might need a tiny lil refactor just to be avoid mocking out os.stat | 14:07 |
| corvus | (but obvs if it's an emergency, that's fine -- the current tests should at least prevent a regression) | 14:07 |
| corvus | mnaser: ++ | 14:07 |
| corvus | mnaser, frickler: also, there is very similar code in the upload-logs-swift role -- it's possible that it may hit the same thing | 14:08 |
| openstackgerrit | Merged zuul/zuul-jobs master: Fix handling of dangling symlink on manifest generation https://review.opendev.org/678552 | 14:14 |
| *** shilpasd has joined #openstack-infra | 14:19 | |
| shilpasd | Hi All: need discussion related to python3-train patch for masakari-monitors https://review.opendev.org/#/c/669387/ for py27/py36 and py37, here observed 'import libvirt' is an issue, seems libvirt-python package is an issue | 14:19 |
| shilpasd | pl throw few pointers to resolve this | 14:19 |
| openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: Add tests for manifest generation for missing files https://review.opendev.org/678573 | 14:19 |
| *** eharney has quit IRC | 14:20 | |
| AJaeger | shilpasd: "ImportError: libvirt.so.0: cannot open shared object file: No such file or directory", ensure that bindep.txt installs the proper packages | 14:20 |
| *** jamesmcarthur has joined #openstack-infra | 14:21 | |
| *** michael-beaver has joined #openstack-infra | 14:21 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Update keystoneauth exception name https://review.opendev.org/678575 | 14:21 |
| shilpasd | AJaeger: as of now we don't have bindep.txt for masakari-monitors, will add so, but what exactly package we need to list here | 14:21 |
| AJaeger | shilpasd: we installed these in the past: https://opendev.org/openstack/project-config/src/branch/master/nodepool/elements/bindep-fallback.txt#L51 but don't do for a few months anymore. So, I suggest to add bindep.txt with the libvirt lines | 14:22 |
| *** e0ne has joined #openstack-infra | 14:22 | |
| shilpasd | AJaeger:okay, tnx, for masakari we did the same https://review.opendev.org/#/c/678495/1 | 14:23 |
| *** e0ne has quit IRC | 14:23 | |
| shilpasd | AJaeger: but for monitors error was total different, so can't correlated | 14:23 |
| *** eharney has joined #openstack-infra | 14:23 | |
| *** e0ne has joined #openstack-infra | 14:24 | |
| mordred | infra-root: I've been pushing up some patches related to things that ahve come up in the room here at the gerrit user summit with the topic "gus2019" - we've also learned that we should almost certainly be tuning the H2 database that used for caching as running a large gerrit with default h2 settings is likely to be a performance issue - but I have not yet dug in to in which ways we should be tuning | 14:25 |
| mordred | it | 14:25 |
| mordred | also - in case anyone winds up needing to do gerrit patches, I just got .gitreview files added to the repo :) | 14:26 |
| AJaeger | mordred: thanks for fixing those quickly ;) | 14:26 |
| clarkb | mordred: wedont use an h2 db | 14:26 |
| clarkb | unless its a different h2 db than the twoo we have already replaced | 14:26 |
| AJaeger | clarkb, could you review https://review.opendev.org/678344 and https://review.opendev.org/678351 for further docs promote docs work, please? | 14:27 |
| mordred | clarkb: ah but we do | 14:28 |
| *** jamesmcarthur has quit IRC | 14:28 | |
| *** bdodd has joined #openstack-infra | 14:29 | |
| mordred | clarkb: this is the thing that we have discovered is both true and not clear to many people | 14:29 |
| mordred | clarkb: if you look in /home/gerrit2/cache you'll see a lovely collection of h2 cache databases - and apparently they're important to gerrit performance | 14:29 |
| clarkb | re swiftuploads 150k against 19 million isnt a really big cost? seems like we need to be looking at other improvements | 14:30 |
| clarkb | that is lessthan 1% of uploads | 14:31 |
| clarkb | mnaser: ^ | 14:31 |
| clarkb | while it might be nice to improve that I doubt it fixes anything | 14:31 |
| mnaser | i think we're comparing different numbers here | 14:31 |
| mnaser | on a busy day, ~2.5m requests come in from infra to swift, ~1.6m of them are ara-report files | 14:32 |
| mnaser | the 19m figure was just a 'hey this is the total' | 14:32 |
| mnaser | so my guess that maybe roughly 11.4 million of those 19 million is just ara | 14:32 |
| clarkb | gotcha | 14:32 |
| *** zigo has joined #openstack-infra | 14:32 | |
| clarkb | so daily is 2.5 million with ~150k being ara static files | 14:33 |
| fungi | so 60% | 14:33 |
| clarkb | the other 1.4 million will be the playbook/task/etc files that differ between runs | 14:33 |
| clarkb | which is 10% range | 14:34 |
| clarkb | still not sure reduving 9% of the uploads will fix anything? though that is a betterimprovement | 14:34 |
| corvus | note that there are two kinds of ara reports -- zuul's report and reports from any nested ansible runs (which are configured in the jobs themselves, eg, devstack) | 14:35 |
| mordred | clarkb, fungi: https://review.opendev.org/Documentation/config-gerrit.html#cache is the relevant config section. for instance, we should almost certainly be increasing our diff-cache size which defaults to 10M but our on-disk H2 db size is 3.9G | 14:35 |
| fungi | i don't see a /home/gerrit2/cache on our server | 14:36 |
| clarkb | mordred: diff-cache is the in memory cache and that h2 file is on disk cache? | 14:36 |
| *** jbadiapa has quit IRC | 14:36 | |
| clarkb | fungi: review_site/cache maybe? | 14:36 |
| mordred | clarkb: yeah. | 14:36 |
| mordred | sorry - review_site/cache | 14:36 |
| fungi | aha, yep | 14:37 |
| fungi | and i do indeed see 6 h2 files with current modification timestamps | 14:37 |
| mordred | clarkb: so cache.diff.memoryLimit would tune the amount of memory used for in-memory caching of diffs that are otherwise cached on disk in review_site/cache/diff.h2.db | 14:37 |
| openstackgerrit | Merged zuul/zuul-jobs master: Update keystoneauth exception name https://review.opendev.org/678575 | 14:38 |
| corvus | if the ara static reports are a problem, we could stop generating them for zuul's ansible -- i think the javascript summary covers most of what ara provides there (but it still has a couple of bugs and doesn't supply all the info ara does). it isn't in a place to substitute for nested ansible runs though. | 14:40 |
| *** markvoelker has quit IRC | 14:41 | |
| clarkb | mordred: worth noting we have occasional memory pressure issues with gerrit still | 14:41 |
| clarkb | so bumping up memory caches could make things worse if garbage collecting cost rises (we'll need to watch changes basically) | 14:41 |
| fungi | i wonder if we can classify how much of what we're uploading is ara reports for nested ansible in jobs vs the ones zuul provides. i gather some of the nested ansible jobs average many orders of magnitude more tasks so could easily account for the bulk of them | 14:42 |
| fungi | (since ara uses several files per task invocation) | 14:43 |
| *** jbadiapa has joined #openstack-infra | 14:43 | |
| *** markvoelker has joined #openstack-infra | 14:44 | |
| mordred | clarkb: yah. I think the main lesson is "there are h2 databases that are worth paying attention to and potentially tuning at some point" | 14:44 |
| mordred | clarkb: also - https://review.opendev.org/#/c/678487 | 14:45 |
| mordred | and https://review.opendev.org/#/c/678413 should help some of our users behind bad networking from hitting the 64 connection per-user limit | 14:46 |
| clarkb | those docs dont reflect the default we experience. Any idea if that option is available on older gerrit replication plugins? | 14:49 |
| clarkb | and the timeout thing must've changed | 14:51 |
| clarkb | beause I remember zuul having trouble with review-dev once upon a time as its ssh connection would go idle as there were infrequent events there | 14:52 |
| corvus | clarkb: mordred has a change up to link to version-appropriate plugin docs | 14:53 |
| mordred | https://gerrit-review.googlesource.com/c/gerrit/+/234719 | 14:54 |
| mordred | https://gerrit.googlesource.com/plugins/replication/+doc/stable-2.13/src/main/resources/Documentation/config.md is the appropriate config | 14:55 |
| clarkb | mordred: oh if you are pushing to gerrit now you may want to push your db fixes that we had to add to our fork (for mysql on that one table) | 14:55 |
| mordred | clarkb: good point. | 14:55 |
| *** lpetrut has quit IRC | 14:55 | |
| AJaeger | mnaser: want to WIP your swift revert change for vexxhost? https://review.opendev.org/678440 | 14:56 |
| mnaser | ah yes | 14:56 |
| mordred | clarkb: any chance you know if we ever filed a bug about that? | 14:57 |
| clarkb | mordred: I want to say there was a bug already because someone else had hit the lack of mysql support | 14:57 |
| *** josephrsandoval has joined #openstack-infra | 15:00 | |
| mordred | nod | 15:02 |
| mordred | clarkb: it was you! https://bugs.chromium.org/p/gerrit/issues/detail?id=6603 | 15:03 |
| *** armax has joined #openstack-infra | 15:04 | |
| clarkb | oh good I filed a bug like a good user then :) | 15:06 |
| mordred | clarkb: https://gerrit-review.googlesource.com/c/gerrit/+/234795 | 15:09 |
| clarkb | mordred: ++ | 15:09 |
| *** pkopec has quit IRC | 15:10 | |
| *** pkopec has joined #openstack-infra | 15:10 | |
| *** e0ne has quit IRC | 15:11 | |
| mordred | clarkb: oh - also - the docker image buikld for gerrit is green now | 15:11 |
| mordred | clarkb: we learned a few things about bazel | 15:11 |
| mordred | clarkb: https://review.opendev.org/#/c/678411 makes things work better, and allows us to do the next one in the chain (removing the crazy memory flags) | 15:12 |
| mordred | paladox: ^^ FYI | 15:12 |
| *** ykarel|away has quit IRC | 15:12 | |
| clarkb | great, I'll try to get to reviews after morning day startup things | 15:12 |
| paladox | thanks! | 15:13 |
| *** rajinir has joined #openstack-infra | 15:14 | |
| mordred | paladox: I do not know *why* that version of "use the java from the system" works better - but it seems to work much more consistently, and also without needing to reduce memory/cpu usage | 15:14 |
| paladox | Ah, that's what i was wondering how to do. | 15:14 |
| mordred | also - in case anyone is interested, there's a dude here who has a local patch he's about to push up to gerrit to improve reindex performance by a large margin. something about letting lucene use more than one CPU | 15:15 |
| mordred | his local patch decresed a gerrit reindex from 4 weeks to 3 hours | 15:15 |
| *** factor has joined #openstack-infra | 15:15 | |
| mordred | so I'm looking forward to it | 15:15 |
| paladox | \o/ | 15:16 |
| clarkb | wow | 15:16 |
| fungi | i guess the reindex threads are spread across processors (by project) but there's one lucene process they're all communicating with which is single-threaded? | 15:17 |
| mordred | fungi: yup | 15:17 |
| mordred | and no way exposed to tune that | 15:17 |
| mordred | also - apparently lucene is detecting disks incorrectly in many cases (including virtual) as spinning rust and throttling | 15:17 |
| paladox | i've just fixed an issue with PolyGerrit where it was not adding a base url to a url that is derived from commentlink. | 15:18 |
| paladox | https://gerrit-review.googlesource.com/c/gerrit/+/234732 | 15:18 |
| paladox | that issue affected us, so we worked around it with https://github.com/wikimedia/puppet/commit/cb61b2f6bb4b69ebda4fcdda7f59be0c482efaca | 15:18 |
| *** jamesmcarthur has joined #openstack-infra | 15:18 | |
| mordred | woot | 15:19 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Add PerconaXDB Cluster to Zuul-Operator https://review.opendev.org/677315 | 15:19 |
| *** jamesmcarthur has quit IRC | 15:22 | |
| *** jamesmcarthur has joined #openstack-infra | 15:22 | |
| *** e0ne has joined #openstack-infra | 15:23 | |
| noonedeadpunk | folks, do you have some ansible lint check that can be easily re-used (like included as a zuul template?) | 15:24 |
| evrardjp | noonedeadpunk: it would be better than each project copying ansible-lint calls inside tox.ini :) | 15:26 |
| *** xenos76 has quit IRC | 15:28 | |
| clarkb | we exclude a ton of rules | 15:30 |
| *** hamzy_ is now known as hamzy | 15:30 | |
| clarkb | I doubt our invocation of ansible-lint is widely applicable unless you subscribe to our opinions of which rules get in the way | 15:31 |
| fungi | i'd personally prefer to just exclude ansible-lint entirely, but there are people who insist it catches actual errors which functional testing does not | 15:31 |
| AJaeger | seems the post_failures for ansible are not fixed yet by frickler's change I3e6982c53a08f57ac0e592b8a0041bbb39812d1f - I'll tail logs again... | 15:32 |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Apply changes to command module from ansible 2.6 https://review.opendev.org/678594 | 15:33 |
| noonedeadpunk | fungi: yeah, I guess functional tests may skip bunch of scenarios (which are not included by default), but may contain some lint errors | 15:34 |
| noonedeadpunk | clarkb ok, got it, thanks. But it worth asking:) | 15:34 |
| *** e0ne has quit IRC | 15:38 | |
| *** mattw4 has joined #openstack-infra | 15:38 | |
| *** mattw4 has quit IRC | 15:39 | |
| *** mattw4 has joined #openstack-infra | 15:39 | |
| *** e0ne has joined #openstack-infra | 15:40 | |
| *** mattw4 has quit IRC | 15:42 | |
| *** mattw4 has joined #openstack-infra | 15:42 | |
| *** efried has quit IRC | 15:42 | |
| *** gyee has joined #openstack-infra | 15:45 | |
| mriedem | clarkb: so i thought https://review.opendev.org/#/c/677265/ was going to break e-r queries that did things like filename:"job-output.txt" but i'm still seeing hits on those http://status.openstack.org/elastic-recheck/#1793364 | 15:46 |
| mriedem | just wondering if i need to be fixing something in e-r | 15:46 |
| openstackgerrit | Merged zuul/nodepool master: openstack: handle safely invalid network name https://review.opendev.org/677501 | 15:47 |
| *** efried has joined #openstack-infra | 15:51 | |
| *** mattw4 has quit IRC | 15:52 | |
| *** factor has quit IRC | 15:53 | |
| *** factor has joined #openstack-infra | 15:54 | |
| clarkb | mordred: bazel stack lgtm | 15:54 |
| clarkb | mriedem: job-output.txt is special beacuse its new real canonical name is job-output.txt not job-output.txt.gz | 15:55 |
| clarkb | mriedem: it should only be a problem if the canonical name is .gz suffixed (things like devstack screen logs) | 15:55 |
| *** slaweq has joined #openstack-infra | 15:56 | |
| mriedem | and only with the filename attribute? | 15:57 |
| mriedem | b/c you added the proxy for tags | 15:57 |
| clarkb | yes | 15:57 |
| *** xenos76 has joined #openstack-infra | 16:02 | |
| *** e0ne has quit IRC | 16:02 | |
| *** jpena is now known as jpena|off | 16:02 | |
| openstackgerrit | Eric Harney proposed openstack/hacking master: Fix deprecated escape chars in strings https://review.opendev.org/678605 | 16:03 |
| shilpasd | AJaeger: hi, still further issues for for 'python3-train' after adding bindep.txt, https://zuul.opendev.org/t/openstack/build/9974591dbf0443caa454a96b1b5356b2 | 16:04 |
| shilpasd | AJaeger: any further suggestion? | 16:05 |
| shilpasd | AJaeger: FYI https://review.opendev.org/#/c/669387/5 | 16:06 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Add PerconaXDB Cluster to Zuul-Operator https://review.opendev.org/677315 | 16:07 |
| *** josephrsandoval has quit IRC | 16:08 | |
| dtantsur | folks, is it possible for an artifact publishing job to publish to tarballs.o.o/$project, where $project is a constant (not necessary the current project)? | 16:09 |
| AJaeger | shilpasd: Check https://zuul.opendev.org/t/openstack/build/9974591dbf0443caa454a96b1b5356b2, it explains the issue at the bottom, doesn't it? Install the missing package... | 16:09 |
| AJaeger | dtantsur: we don't have such a job right now - but sure, it could be done. Why? | 16:10 |
| openstackgerrit | Eric Harney proposed openstack/hacking master: Clean up "is" conditionals https://review.opendev.org/678607 | 16:10 |
| shilpasd | AJaeger: in chrome, its not listing anything at bottom, will check another browser for missing packages, tnx | 16:11 |
| dtantsur | AJaeger: we publish ironic-python-agent images in a post job. now that we've split the built scripts to another repository, we would like to also publish the images in a post job from it. but to the same location as before. | 16:11 |
| * dtantsur hopes he's clear enough, blames the weather otherwise | 16:11 | |
| *** ramishra has quit IRC | 16:11 | |
| AJaeger | shilpasd: check also https://zuul.opendev.org/t/openstack/build/9974591dbf0443caa454a96b1b5356b2/console | 16:11 |
| AJaeger | which should link to https://zuul.opendev.org/t/openstack/build/9974591dbf0443caa454a96b1b5356b2/console#3/0/7/ubuntu-bionic | 16:12 |
| AJaeger | and that one says "Exception: pkg-config binary is required to compile libvirt-python" | 16:12 |
| AJaeger | shilpasd: so, add pkg-config - the bindep-fallback has the lines to copy ;) | 16:12 |
| * dtantsur recalls adding pkg-config explicitly to bindep.txt | 16:12 | |
| AJaeger | jrosser, noonedeadpunk , frickler, mnaser , this is what I got from openstack-ansible-tests on CentoOS7: | 16:13 |
| AJaeger | http://paste.openstack.org/show/764709/ | 16:13 |
| AJaeger | "FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/zuul/builds/15aca570c15b45c19fb8df29f6c9b3a6/work/logs/logs/etc/host/network-scripts/ifdown-isdn'" | 16:13 |
| AJaeger | so, seems the fix was not enough | 16:13 |
| openstackgerrit | Merged zuul/zuul master: Document js tool installation in scratch doc https://review.opendev.org/678286 | 16:14 |
| noonedeadpunk | yeah, it seems the same... I've launched recheck for https://review.opendev.org/#/c/675886/ | 16:15 |
| AJaeger | clarkb, fungi, could you review https://review.opendev.org/678344 and https://review.opendev.org/678351 for further docs promote docs work, please? | 16:16 |
| shilpasd | AJaeger: tnx for details, FYI said links displays nothing in chrome, pl share your browser details, so that will get it installed ASAP | 16:16 |
| AJaeger | philroche: firefox - but let me check chromium as well... | 16:17 |
| clarkb | shilpasd: AJaeger it works in chrome for me | 16:17 |
| clarkb | Under the Results heading | 16:17 |
| jrosser | AJaeger: is the logs/logs part of the path suspicious? | 16:18 |
| AJaeger | philroche: sorry, wrong tab completion | 16:18 |
| shilpasd | clarkb: okay will upgrade chrome | 16:18 |
| fungi | shilpasd: check that you don't have a browser plugin blocking content from that page too | 16:18 |
| AJaeger | shilpasd: works in chromium for me as well. It takes in firefox a second or two to load... | 16:18 |
| AJaeger | so, wait - and then scroll to bottome - and follow fungi's advice on blocking content | 16:19 |
| shilpasd | AJaeger: Network Error (Unable to fetch URL, check your network connectivity, browser plugins, and ad-blockers) https://openstack.fortnebula.com:13808/v1/AUTH_e8fd161dc34c421a979a9e6421f823e9/logs_87/669387/5/check/openstack-tox-py37/9974591/zuul-manifest.json | 16:21 |
| shilpasd | AJaeger: might be connectivity issue, will check at my end, tnx | 16:22 |
| openstackgerrit | Matt Riedemann proposed opendev/elastic-recheck master: Include "Invalid" bugs in cleanup CLI https://review.opendev.org/678608 | 16:22 |
| openstackgerrit | Matt Riedemann proposed opendev/elastic-recheck master: Delete old queries: 2019-08-26 https://review.opendev.org/678609 | 16:22 |
| shilpasd | fungi: sure will check that also | 16:22 |
| clarkb | AJaeger: can you check my comment on https://review.opendev.org/#/c/678351/4 | 16:27 |
| shilpasd | AJaeger: thanks, will keep you posted for same, leaving for now | 16:27 |
| AJaeger | clarkb: good catch - let me double check... | 16:29 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add promote jobs for static site / releasenotes https://review.opendev.org/678351 | 16:31 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Remove now unused publish jobs https://review.opendev.org/678356 | 16:31 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add allowed-projects to static publish jobs https://review.opendev.org/678357 | 16:31 |
| AJaeger | clarkb: thanks, updated ^ | 16:32 |
| *** nicolasbock has joined #openstack-infra | 16:32 | |
| openstackgerrit | Merged zuul/zuul master: Set git user config in from scratch document https://review.opendev.org/678312 | 16:34 |
| AJaeger | clarkb: thanks, could you review https://review.opendev.org/#/c/678344 as well, please? That's below in the stack... | 16:37 |
| *** igordc has joined #openstack-infra | 16:37 | |
| clarkb | yup I was sorting out the pathing for where download_artifact puts things | 16:38 |
| clarkb | Approved now | 16:38 |
| AJaeger | ah - thanks | 16:38 |
| AJaeger | fungi, could you review https://review.opendev.org/678351 , please? Then I can use the jobs in other repos | 16:38 |
| *** factor has quit IRC | 16:39 | |
| *** jamesmcarthur has quit IRC | 16:40 | |
| *** ociuhandu has joined #openstack-infra | 16:44 | |
| noonedeadpunk | AJaeger: I guess I know what;'s wrong with https://review.opendev.org/#/c/678552/2/roles/generate-zuul-manifest/library/generate_manifest.py | 16:45 |
| AJaeger | please tell - or even better: patch ;) | 16:46 |
| noonedeadpunk | nah, I was testing against python2 - it just don't have FileNotFoundError exception (while python3 does) | 16:47 |
| *** josephrsandoval has joined #openstack-infra | 16:49 | |
| *** josephrsandoval has quit IRC | 16:49 | |
| noonedeadpunk | but stack trace differs | 16:49 |
| noonedeadpunk | and actually it's different task:) | 16:50 |
| noonedeadpunk | and role | 16:50 |
| *** ociuhandu has quit IRC | 16:51 | |
| openstackgerrit | Merged openstack/project-config master: Add PDF to api promote job https://review.opendev.org/678344 | 16:54 |
| jrosser | noonedeadpunk: looks like the same sort of thing again https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/upload-logs-swift/library/zuul_swift_upload.py#L197 | 16:54 |
| clarkb | is there a tl'dr of the issue ^ around files nto found and upload errors? | 16:55 |
| noonedeadpunk | jrosser: Yep, already patching | 16:55 |
| *** slaweq has quit IRC | 16:55 | |
| *** tesseract has quit IRC | 16:57 | |
| *** Lucas_Gray has joined #openstack-infra | 16:58 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Fix handling of dangling symlink https://review.opendev.org/678619 | 16:58 |
| openstackgerrit | Merged zuul/zuul master: bindep: add unzip and bzip2 for rpm platform https://review.opendev.org/678433 | 16:58 |
| AJaeger | ~. | 16:58 |
| AJaeger | ¨~~. | 16:58 |
| AJaeger | argh... | 16:59 |
| noonedeadpunk | ? | 16:59 |
| AJaeger | wrong window - tried to terminate session | 16:59 |
| AJaeger | clarkb: http://paste.openstack.org/show/764709/ - theory is that we get an error for a dangling symlink | 17:00 |
| *** eharney has quit IRC | 17:00 | |
| AJaeger | clarkb: frickler's fix was http://paste.openstack.org/show/764709/ but that was not enough | 17:00 |
| *** slaweq has joined #openstack-infra | 17:00 | |
| *** bhavikdbavishi has joined #openstack-infra | 17:01 | |
| noonedeadpunk | Yeah, since there's the same trap inside zuul_swift_upload | 17:01 |
| noonedeadpunk | but that part is pretty solid... | 17:02 |
| clarkb | I +2'd https://review.opendev.org/#/c/678619/1 with some notes on my own local testing. | 17:03 |
| clarkb | feel free to chime in if we need to test some additional cases | 17:03 |
| *** jamesmcarthur has joined #openstack-infra | 17:04 | |
| openstackgerrit | Andreas Jaeger proposed zuul/zuul-jobs master: Add tests for manifest generation for missing files https://review.opendev.org/678573 | 17:05 |
| *** slaweq_ has joined #openstack-infra | 17:05 | |
| *** slaweq has quit IRC | 17:05 | |
| noonedeadpunk | AJaeger: I can update zuul_swift_upload to the same pattern | 17:07 |
| noonedeadpunk | at least will add tests... | 17:08 |
| AJaeger | noonedeadpunk: did you see 678573 ? Yes, happy to see same pattern everywhere and tests | 17:09 |
| noonedeadpunk | yep | 17:09 |
| *** ociuhandu has joined #openstack-infra | 17:09 | |
| *** ociuhandu has quit IRC | 17:13 | |
| *** shilpasd has quit IRC | 17:16 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Apply changes to command module from ansible 2.6 https://review.opendev.org/678594 | 17:18 |
| AJaeger | noonedeadpunk: your change fails py27, with "NameError: global name 'FileNotFoundError' is not defined" | 17:19 |
| *** mattw4 has joined #openstack-infra | 17:19 | |
| *** chkumar|rover is now known as raukadah | 17:21 | |
| noonedeadpunk | AJaeger: wondering how https://review.opendev.org/#/c/678552/2 has pased | 17:22 |
| noonedeadpunk | As FileNotFoundError exception is not present for 2.7 | 17:22 |
| clarkb | the tests are probably running under different python versions? | 17:23 |
| AJaeger | seems our testing is not complete ;/ | 17:23 |
| clarkb | we likely want to update the code to handle both python2 and 3 | 17:23 |
| clarkb | (and then fix tests if we can sort that out) | 17:23 |
| AJaeger | that file has "#!/usr/bin/env python3" | 17:24 |
| AJaeger | noonedeadpunk: ^ | 17:24 |
| *** Lucas_Gray has quit IRC | 17:24 | |
| noonedeadpunk | zuul_swift_upload too :) | 17:24 |
| clarkb | OSError is what we get on python2 | 17:24 |
| clarkb | AJaeger: noonedeadpunk note the shebang doesn't force the script to run under that interpreter | 17:24 |
| clarkb | only uses that interpreter if executed directly as an executable file | 17:25 |
| AJaeger | yes | 17:25 |
| *** dtantsur is now known as dtantsur|afk | 17:25 | |
| clarkb | on centos7 in particular python3 is likely not going to be present | 17:25 |
| fungi | however i think we were using that as a convention to indicate it was a python3 script instead of python2.7 | 17:25 |
| fungi | but yeah, i guess we do still have platforms in opendev with no python3 | 17:26 |
| *** ralonsoh has quit IRC | 17:26 | |
| fungi | is manifest generation happening on test nodes? | 17:26 |
| fungi | not on the executor? | 17:27 |
| *** jamesmcarthur has quit IRC | 17:29 | |
| clarkb | noonedeadpunk: can we alias OSError to filenotfounderror if filenot found is undefined? | 17:29 |
| AJaeger | fungi, could I trouble you with https://review.opendev.org/678351 , please? That allows to move forward with promote docs... | 17:31 |
| fungi | AJaeger: yep, sorry, been juggling too many things today but will take a look at that one now | 17:31 |
| AJaeger | thanks | 17:32 |
| noonedeadpunk | clarkb: I guess it's possible to http://paste.openstack.org/show/764865/ but not sure about best practises | 17:35 |
| clarkb | noonedeadpunk: I think it is oserror not io error | 17:38 |
| clarkb | they are both children of environment error | 17:38 |
| noonedeadpunk | yeah, os, sure | 17:38 |
| noonedeadpunk | I mean the idea;) | 17:39 |
| clarkb | I think the other alternative is to use six, but then you have to manage package dependencies so I think doing the check ourselves here like that is a good idea | 17:39 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Fix handling of dangling symlink https://review.opendev.org/678619 | 17:45 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Add tests for manifest generation for missing files https://review.opendev.org/678573 | 17:46 |
| *** jamesmcarthur has joined #openstack-infra | 17:49 | |
| *** gfidente is now known as gfidente|sad_sum | 17:50 | |
| *** xenos76 has quit IRC | 17:55 | |
| *** xenos76 has joined #openstack-infra | 17:58 | |
| *** jamesmcarthur has quit IRC | 17:58 | |
| *** e0ne has joined #openstack-infra | 17:59 | |
| clarkb | noonedeadpunk: commented on 678619 about why tests failed | 18:01 |
| *** eharney has joined #openstack-infra | 18:03 | |
| *** whoami-rajat has quit IRC | 18:03 | |
| *** e0ne has quit IRC | 18:04 | |
| openstackgerrit | Merged openstack/project-config master: Add promote jobs for static site / releasenotes https://review.opendev.org/678351 | 18:08 |
| AJaeger | thanks, fungi. AS followup, here's a change for election repo: https://review.opendev.org/#/c/678352/ | 18:10 |
| *** jamesmcarthur has joined #openstack-infra | 18:12 | |
| fungi | saw that one earlier... we've got some time-sensitive publishing coming up over the next few weeks so i'm on the fence about this being a good time to change election site publishing processs | 18:12 |
| fungi | on the other hand, we go ~6 months without changing anything at all, so if we don't test it now we probably won't know it's broken for a very long time | 18:12 |
| *** pkopec has quit IRC | 18:16 | |
| mnaser | hmm. it seems like i cant have an openstack project listed as a required-project in opendev? | 18:19 |
| fungi | we can, we just need to include it in the projects list | 18:22 |
| fungi | are you trying to use it for a configuration dependency, or just to get the source code and be able to depends-on? | 18:23 |
| fungi | for configuration, we might be trying to keep the opendev tenant more self-contained and wouldn't want to wind up transitively dragging config from dozens of openstack tenant projects into the opendev tenant config | 18:24 |
| fungi | would probably need a bit more discussion | 18:24 |
| mnaser | fungi: nope, its to get source code and do depends-on | 18:27 |
| mnaser | and have tox use the checked out version in gate | 18:27 |
| fungi | i think we just need to add a - include: [] sublist in https://opendev.org/openstack/project-config/src/branch/master/zuul/main.yaml#L10 | 18:28 |
| fungi | similar to what you see in the kata-containers tenant below that | 18:28 |
| fungi | or this in the zuul tenant might be a better example: https://opendev.org/openstack/project-config/src/branch/master/zuul/main.yaml#L1619-L1626 | 18:30 |
| fungi | but yeah, since it's the first time doing this in the opendev tenant, probably merits a discussion to make sure this is something we want to be able to do in that tenant | 18:30 |
| *** bhavikdbavishi has quit IRC | 18:32 | |
| *** radez has joined #openstack-infra | 18:32 | |
| radez | hey folks, I'm finding an inconsistency with a manula devstack install and my CI run. I'm trying to access files that land in the system python site packages directory. They show up when I clone my patch to a local devstack run but they don't seem to be there when my ci job runs. | 18:35 |
| fungi | radez: hey! long time no see | 18:36 |
| radez | is there a way to inspect the file system on a check job to figure out where the files are being put? or doesn't somone know | 18:36 |
| radez | hey fungi | 18:36 |
| radez | files are being installed via pip | 18:36 |
| fungi | probably the best place to start is with a link to the build log | 18:36 |
| radez | hope things are well for you fungi? | 18:36 |
| radez | sure, lemme grab it | 18:37 |
| fungi | oh, yep, can't complain | 18:37 |
| radez | https://zuul.opendev.org/t/openstack/build/6ac9456ebb734de3b96518d9c2899fce | 18:37 |
| fungi | life in the obx is much slower than raleigh ;) | 18:37 |
| radez | hehe, hopfully that's a good thing? | 18:37 |
| fungi | absolutely. it teaches patience | 18:37 |
| radez | oh.. man that's a rough lesson sometimes | 18:38 |
| radez | you can see in that log that my cp fails that my plugin executes when it tries to put some ansible roles into place for my test | 18:38 |
| AJaeger | fungi: ossa has a patch as well if you find later time for reviews - https://review.opendev.org/678354 (topic:promote-static) | 18:38 |
| fungi | radez: which package are you trying to track down the install location for? | 18:39 |
| fungi | (and is it on the compute1 node or the controller node, or maybe both?) | 18:39 |
| AJaeger | fungi, merge the election repo change, either it works - then all is fine, or revert ;) | 18:39 |
| fungi | AJaeger: yeah, i think that's the plan | 18:39 |
| radez | fungi: the package is network-runner. when pip installs it lands the ansible roles in /usr/lib/python2.7/site-packages/etc/ansible/roles/ | 18:39 |
| radez | along side the network_runner directory | 18:40 |
| *** trident has quit IRC | 18:40 | |
| *** trident has joined #openstack-infra | 18:40 | |
| fungi | radez: so here's where it was installed for python2.7 on the controller node: https://zuul.opendev.org/t/openstack/build/6ac9456ebb734de3b96518d9c2899fce/log/job-output.txt#13296-13297 | 18:45 |
| radez | fungi: right, I'm expecting that the path that cp can't find is in that package | 18:46 |
| fungi | and pip2 freeze on the controller confirms it's present: https://zuul.opendev.org/t/openstack/build/6ac9456ebb734de3b96518d9c2899fce/log/controller/logs/pip2-freeze.txt.gz#84 | 18:46 |
| radez | I also havd downloaded that whl and I see the files in the package. | 18:47 |
| radez | so the correct version is being installed and the package shows the files I'm looking for. but the cp command can't seem to find them which is what's stumping me | 18:48 |
| fungi | ahh, okay, so the install_ansible_roles function in networking-ansible/devstack/plugin.sh on line 109... | 18:48 |
| radez | yup, it's copying something out of a git repo that's been cloned to /opt/stack successfullt | 18:49 |
| clarkb | does the double / break globbing? | 18:49 |
| radez | but doesn't seem to find the site-packages/etc/ansibe/roles dir | 18:49 |
| radez | clarkb: hm good question, lemme test that on my dev box | 18:50 |
| fungi | i guess it's this here? https://opendev.org/x/networking-ansible/src/branch/master/devstack/plugin.sh#L106 | 18:50 |
| radez | $ cp -r /usr/lib/python2.7/site-packages/etc/ansible/roles//* /tmp | 18:51 |
| clarkb | its because /usr/lib isn't where pip installs stuff on ubuntu | 18:51 |
| *** rmcall has joined #openstack-infra | 18:51 | |
| radez | this works from a termain | 18:51 |
| clarkb | you are using a centos/rhel path on ubuntu and it is breaking | 18:51 |
| clarkb | should be /usr/local/something | 18:51 |
| * radez facepalms | 18:51 | |
| fungi | yeah, it'll be /usr/local/lib/... on debian/ubuntu | 18:51 |
| radez | I knew it had to be something like that | 18:51 |
| fungi | on the other hand, i think you can use importlib to tell you the on-disk path for any module | 18:52 |
| fungi | (maybe it's not importlib, but there is something in stdlib) | 18:52 |
| radez | something that could give me the site-packages path? | 18:53 |
| radez | I'll poke around and see if I can find that too | 18:53 |
| fungi | the module search path can cover multiple locations, so if you really want to know which one got used in a portable way for any given module that's how i'd go about it | 18:53 |
| clarkb | isn't this what ansible galaxy is for? | 18:53 |
| radez | fungi: good call. good catch clarkb. Thanks both for your help! | 18:53 |
| radez | clarkb: yes we're moving that way, just haven't gotten there yet | 18:54 |
| *** pkopec has joined #openstack-infra | 18:54 | |
| *** markvoelker has quit IRC | 18:57 | |
| *** xek_ has joined #openstack-infra | 18:58 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Fix handling of dangling symlink https://review.opendev.org/678619 | 19:01 |
| *** slaweq_ has quit IRC | 19:01 | |
| *** markvoelker has joined #openstack-infra | 19:11 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Add tests for manifest generation for missing files https://review.opendev.org/678573 | 19:11 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Fix handling of dangling symlink https://review.opendev.org/678619 | 19:19 |
| noonedeadpunk | clarkb: are you sure about noqa comment? Since from what I've read, you can either disable for entire file or by placing it in front of line, which shouldn't be checked... | 19:20 |
| clarkb | noonedeadpunk: './roles/upload-logs-swift/library/zuul_swift_upload.py:182:13: F823 local variable 'FileNotFoundError' defined as a builtin referenced before assignment' is the error and that seems to complain about the line I mentioned | 19:21 |
| clarkb | I think it is annoyed that we are using the value before assigning it under python2 | 19:21 |
| clarkb | if we ran tox under python3 it would work | 19:21 |
| noonedeadpunk | Hm, I thought that it's violating because of reassignment of built-in variable "FileNotFoundError = OSError" | 19:22 |
| clarkb | that is line 184 | 19:22 |
| clarkb | but maybe I guess we'll know shortly | 19:22 |
| noonedeadpunk | Yeah... You're right probably. Just my local IDE (which uses pylint) complains about "Redefining built-in 'FileNotFoundError'pylint(redefined-builtin)" which led me the wrong way... | 19:23 |
| fungi | if memory serves, older flake8 disabled all checking for a file if it found "# noqa" anywhere and people were sticking it at the end of lines they wanted ignored thinking it was only ignoring those specific lines. then there was a transition where flake8 started making it do the thing people thought it did all along, but that exposed tons of previously-hidden errors they didn't know about | 19:24 |
| *** efried has quit IRC | 19:24 | |
| noonedeadpunk | Yeah, I can recall smth like this. But now docs says that this should be ok http://flake8.pycqa.org/en/3.1.1/user/ignoring-errors.html#in-line-ignoring-errors | 19:25 |
| fungi | yep | 19:25 |
| fungi | ahh, yeah, it was 3.6.0 last year: http://flake8.pycqa.org/en/3.7.8/release-notes/3.6.0.html | 19:28 |
| fungi | "Only skip a file if # flake8: noqa is on a line by itself" | 19:28 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Fix handling of dangling symlink https://review.opendev.org/678619 | 19:28 |
| fungi | yeah, i guess it was "# flake8: noqa" not simply "# noqa" | 19:30 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Add tests for manifest generation for missing files https://review.opendev.org/678573 | 19:31 |
| noonedeadpunk | yeah | 19:32 |
| *** gfidente|sad_sum has quit IRC | 19:35 | |
| AJaeger | yeah, noonedeadpunk changes pass tests \o/ | 19:42 |
| noonedeadpunk | yep, cool:) | 19:43 |
| AJaeger | clarkb: want to +2A https://review.opendev.org/#/c/678619 and https://review.opendev.org/678573 ? | 19:44 |
| clarkb | ya looking now | 19:46 |
| clarkb | noonedeadpunk: in 678619 we return 0 for the size but in 678573 we return None for the size when things fail | 19:52 |
| clarkb | noonedeadpunk: AJaeger ^ I think we should be consistent on that. My first thought is zero makes sense since we can't find it | 19:53 |
| *** jamesmcarthur has quit IRC | 19:53 | |
| clarkb | whereas None may not render properly in the index/manifest consumers? | 19:53 |
| clarkb | is there a reason to not be consistent? | 19:53 |
| AJaeger | mnaser: wrote the code in 678573 - I agree, let's be consistent | 19:55 |
| *** mattw4 has quit IRC | 19:58 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Fix handling of dangling symlink https://review.opendev.org/678619 | 19:59 |
| *** xek_ has quit IRC | 20:00 | |
| clarkb | I approved ^ before I noticed the delta | 20:00 |
| openstackgerrit | Jeff Liu proposed zuul/zuul-operator master: Add PerconaXDB Cluster to Zuul-Operator https://review.opendev.org/677315 | 20:00 |
| clarkb | That said https://review.opendev.org/678573 adds a behavior change too, it doesn't continue it instead records the not found files | 20:01 |
| clarkb | the other one didn't change any behavior iirc | 20:01 |
| *** efried has joined #openstack-infra | 20:02 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed zuul/zuul-jobs master: Add tests for manifest generation for missing files https://review.opendev.org/678573 | 20:02 |
| noonedeadpunk | AJaeger: clarkb I've updated 678573 ^ | 20:03 |
| clarkb | noonedeadpunk: AJaeger left a note about the other thing I noticed on ps5 | 20:03 |
| clarkb | can you take a quick look at that and decide if we want to chagne it or not? | 20:03 |
| AJaeger | clarkb: I left that behaviour change note there already ages ok - so agree ;) | 20:04 |
| clarkb | ah ya I see that now | 20:05 |
| * AJaeger waves good night | 20:05 | |
| *** efried has quit IRC | 20:07 | |
| *** efried has joined #openstack-infra | 20:13 | |
| *** mattw4 has joined #openstack-infra | 20:17 | |
| *** david-lyle has quit IRC | 20:25 | |
| *** mattw4 has quit IRC | 20:28 | |
| *** slaweq has joined #openstack-infra | 20:29 | |
| noonedeadpunk | clarkb: I'm basically already out but I don't see direct change of the behavior - we return 0 instead of None, which shouldn't change basic conditional passing | 20:34 |
| noonedeadpunk | and it's already changed from None to 0 for consistency | 20:35 |
| *** mattw4 has joined #openstack-infra | 20:39 | |
| *** dklyle has joined #openstack-infra | 20:39 | |
| clarkb | noonedeadpunk: the change if behavior is with the original change too in removing the continue | 20:41 |
| clarkb | previously we wanted to skip the file | 20:41 |
| clarkb | mnaser: ^ do you know if that was intentional? | 20:41 |
| clarkb | fungi: can we approve https://review.opendev.org/#/c/675733/1 and its parent? | 20:44 |
| *** kjackal has joined #openstack-infra | 20:57 | |
| *** Goneri has quit IRC | 20:57 | |
| *** slaweq has quit IRC | 21:01 | |
| *** pkopec has quit IRC | 21:03 | |
| *** kjackal has quit IRC | 21:06 | |
| *** lpetrut has joined #openstack-infra | 21:07 | |
| *** trident has quit IRC | 21:14 | |
| *** trident has joined #openstack-infra | 21:20 | |
| *** markvoelker has quit IRC | 21:21 | |
| *** lpetrut has quit IRC | 21:22 | |
| *** trident has quit IRC | 21:25 | |
| *** igordc has quit IRC | 21:26 | |
| fungi | lookin' | 21:27 |
| fungi | yeah, we have nothing in production relying on puppet-mediawiki | 21:27 |
| fungi | it's only a dev server | 21:28 |
| clarkb | done | 21:28 |
| *** mattw4 has quit IRC | 21:28 | |
| fungi | once those are merged i'll sync in the file content and maybe do a full data refresh from production while i'm at it | 21:28 |
| *** jeliu_ has quit IRC | 21:28 | |
| fungi | since that way i won't have to worry about a 1.27->1.28 upgrade of the dev copy on top of it all | 21:29 |
| *** trident has joined #openstack-infra | 21:33 | |
| *** tonyb[m] has quit IRC | 21:35 | |
| *** dosaboy has joined #openstack-infra | 21:35 | |
| *** mattw4 has joined #openstack-infra | 21:36 | |
| *** dosaboy has quit IRC | 21:39 | |
| *** dosaboy has joined #openstack-infra | 21:39 | |
| *** tonyb has quit IRC | 21:39 | |
| *** dosaboy has quit IRC | 21:40 | |
| *** dosaboy has joined #openstack-infra | 21:41 | |
| noonedeadpunk | clarkb: oh, yep, I've got what you're talking about | 21:44 |
| openstackgerrit | Merged opendev/puppet-mediawiki master: Put image data in a parallel path to source code https://review.opendev.org/675713 | 21:54 |
| openstackgerrit | Merged opendev/puppet-mediawiki master: Update to 1.28.x branch https://review.opendev.org/675733 | 21:54 |
| *** markvoelker has joined #openstack-infra | 22:05 | |
| *** trident has quit IRC | 22:05 | |
| *** markvoelker has quit IRC | 22:10 | |
| *** trident has joined #openstack-infra | 22:14 | |
| noonedeadpunk | Hurray - that worked! https://review.opendev.org/#/c/675886/ passed CI | 22:16 |
| *** eharney has quit IRC | 22:26 | |
| *** mriedem has quit IRC | 22:36 | |
| *** dklyle has quit IRC | 22:40 | |
| *** harlowja has joined #openstack-infra | 22:44 | |
| *** rcernin has joined #openstack-infra | 22:45 | |
| *** tkajinam has joined #openstack-infra | 23:02 | |
| *** mattw4 has quit IRC | 23:02 | |
| *** slaweq has joined #openstack-infra | 23:11 | |
| *** dave-mccowan has quit IRC | 23:11 | |
| *** slaweq has quit IRC | 23:16 | |
| clarkb | fungi: have time for https://review.opendev.org/#/c/678304/ ? hoping that reduces future confusion | 23:26 |
| clarkb | noonedeadpunk: yay | 23:26 |
| *** dklyle has joined #openstack-infra | 23:26 | |
| *** dchen has joined #openstack-infra | 23:31 | |
| *** dchen has joined #openstack-infra | 23:33 | |
| *** dave-mccowan has joined #openstack-infra | 23:34 | |
| *** markvoelker has joined #openstack-infra | 23:41 | |
| *** markvoelker has quit IRC | 23:46 | |
| *** sthussey has quit IRC | 23:55 | |
| ianw | clarkb: it looks like fedora is stuck on a vos release, again? (seemed to start @ 16:44 ... now 23:58) | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!