| -@gerrit:opendev.org- Zuul merged on behalf of Michal Nasiadka: [zuul/zuul] 955373: github-reference-pipeline: Fix error in state: https://review.opendev.org/c/zuul/zuul/+/955373 | 07:09 | |
| -@gerrit:opendev.org- Jan Gutter proposed: [zuul/zuul] 957215: Update ZooKeeper autopurge recommendation https://review.opendev.org/c/zuul/zuul/+/957215 | 10:01 | |
| -@gerrit:opendev.org- Jan Gutter proposed: [zuul/zuul] 957215: Update ZooKeeper autopurge recommendation https://review.opendev.org/c/zuul/zuul/+/957215 | 10:18 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Benjamin Schanzel: [zuul/zuul] 956660: web: Fix filtering on PipelineDetails page https://review.opendev.org/c/zuul/zuul/+/956660 | 10:20 | |
| @bennetefx:matrix.org | Hi Zuul community, today I was building the scheduler image, and it was building fine up until last hour, but now I think a release has been made in the python pbr package, and now when I try to build the image of zuul-scheduler, it is failing. | 10:41 |
|---|---|---|
| `50.32 Requirement already satisfied: pip in /tmp/venv/lib/python3.11/site-packages (24.0) | ||
| 50.51 Collecting pip | ||
| 50.61 Downloading pip-25.2-py3-none-any.whl.metadata (4.7 kB) | ||
| 50.70 Collecting wheel | ||
| 50.73 Downloading wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB) | ||
| 50.81 Collecting build | ||
| 50.84 Downloading build-1.3.0-py3-none-any.whl.metadata (5.6 kB) | ||
| 50.86 Requirement already satisfied: setuptools in /tmp/venv/lib/python3.11/site-packages (65.5.0) | ||
| 51.08 Collecting setuptools | ||
| 51.11 Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB) | ||
| 51.21 Collecting packaging>=19.1 (from build) | ||
| 51.24 Downloading packaging-25.0-py3-none-any.whl.metadata (3.3 kB) | ||
| 51.31 Collecting pyproject_hooks (from build) | ||
| 51.35 Downloading pyproject_hooks-1.2.0-py3-none-any.whl.metadata (1.3 kB) | ||
| 51.42 Downloading pip-25.2-py3-none-any.whl (1.8 MB) | ||
| 51.62 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 9.2 MB/s eta 0:00:00 | ||
| 51.64 Downloading wheel-0.45.1-py3-none-any.whl (72 kB) | ||
| 51.66 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.5/72.5 kB 4.3 MB/s eta 0:00:00 | ||
| 51.70 Downloading build-1.3.0-py3-none-any.whl (23 kB) | ||
| 51.74 Downloading setuptools-80.9.0-py3-none-any.whl (1.2 MB) | ||
| 51.81 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 18.7 MB/s eta 0:00:00 | ||
| 51.84 Downloading packaging-25.0-py3-none-any.whl (66 kB) | ||
| 51.85 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.5/66.5 kB 3.8 MB/s eta 0:00:00 | ||
| 51.88 Downloading pyproject_hooks-1.2.0-py3-none-any.whl (10 kB) | ||
| 51.99 Installing collected packages: wheel, setuptools, pyproject_hooks, pip, packaging, build | ||
| 52.04 Attempting uninstall: setuptools | ||
| 52.04 Found existing installation: setuptools 65.5.0 | ||
| 52.06 Uninstalling setuptools-65.5.0: | ||
| 52.07 Successfully uninstalled setuptools-65.5.0 | ||
| 52.64 Attempting uninstall: pip | ||
| 52.65 Found existing installation: pip 24.0 | ||
| 52.67 Uninstalling pip-24.0: | ||
| 52.67 Successfully uninstalled pip-24.0 | ||
| 53.34 Successfully installed build-1.3.0 packaging-25.0 pip-25.2 pyproject_hooks-1.2.0 setuptools-80.9.0 wheel-0.45.1 | ||
| 53.61 + /tmp/venv/bin/pip --version | ||
| 53.73 pip 25.2 from /tmp/venv/lib/python3.11/site-packages/pip (python 3.11) | ||
| 53.75 + '[' -f /tmp/src/upper-constraints.txt ']' | ||
| 53.75 + [[ -n '' ]] | ||
| 53.75 + apt-get install -y git | ||
| 53.76 Reading package lists... | ||
| 54.21 Building dependency tree... | ||
| 54.32 Reading state information... | ||
| 54.45 git is already the newest version (1:2.39.5-0+deb12u2). | ||
| 54.45 0 upgraded, 0 newly installed, 0 to remove and 10 not upgraded. | ||
| 54.45 + install_wheels | ||
| 54.45 + /tmp/venv/bin/python3 -m build -o /output/toplevel_wheels ./ | ||
| 54.51 * Creating isolated environment: venv+pip... | ||
| 54.52 * Installing packages in isolated environment: | ||
| 55.23 - setuptools >= 40.8.0 | ||
| 55.23 * Getting build dependencies for sdist... | ||
| 55.43 /tmp/build-env-j59kd_4l/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. | ||
| 55.43 !! | ||
| 55.43 | ||
| 55.43 ******************************************************************************** | ||
| 55.43 Please consider removing the following classifiers in favor of a SPDX license expression: | ||
| 55.43 | ||
| 55.43 License :: OSI Approved :: Apache Software License | ||
| 55.43 | ||
| 55.43 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. | ||
| 55.43 ******************************************************************************** | ||
| 55.43 | ||
| 55.43 !! | ||
| 55.43 self._finalize_license_expression() | ||
| 55.45 * Installing packages in isolated environment: | ||
| 56.30 - pbr | ||
| 56.30 * Building sdist... | ||
| 56.44 Error parsing | ||
| 56.44 Traceback (most recent call last): | ||
| 56.44 File "/tmp/build-env-j59kd_4l/lib/python3.11/site-packages/pbr/core.py", line 103, in pbr | ||
| 56.44 attrs = util.cfg_to_args(path, dist.script_args) | ||
| 56.44 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 56.44 File "/tmp/build-env-j59kd_4l/lib/python3.11/site-packages/pbr/util.py", line 345, in cfg_to_args | ||
| 56.44 hook_fn = resolve_name(hook) | ||
| 56.44 ^^^^^^^^^^^^^^^^^^ | ||
| 56.44 File "/tmp/build-env-j59kd_4l/lib/python3.11/site-packages/pbr/util.py", line 276, in resolve_name | ||
| 56.44 ret = __import__('.'.join(module_name), fromlist=[attr_name]) | ||
| 56.44 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 56.44 File "/tmp/src/zuul/_setup_hook.py", line 21, in <module> | ||
| 56.44 _old_from_git = pbr.packaging._from_git | ||
| 56.44 ^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 56.44 AttributeError: module 'pbr.packaging' has no attribute '_from_git' | ||
| 56.44 error in setup command: Error parsing /tmp/src/setup.cfg: AttributeError: module 'pbr.packaging' has no attribute '_from_git' | ||
| 56.47 | ||
| 56.47 ERROR Backend subprocess exited when trying to invoke build_sdist | ||
| ------ | ||
| Dockerfile:75 | ||
| -------------------- | ||
| 73 | COPY . /tmp/src | ||
| 74 | COPY --from=js-builder /tmp/src/build /tmp/src/zuul/web/static | ||
| 75 | >>> RUN assemble | ||
| 76 | | ||
| 77 | # The wheel install method doesn't run the setup hooks as the source based | ||
| -------------------- | ||
| ERROR: failed to build: failed to solve: process "/bin/sh -c assemble" did not complete successfully: exit code: 1 | ||
| @bennetefx:matrix.org | * Hi Zuul community, today I was building the scheduler image, and it was building fine up until last hour, but now I think a release has been made in the python pbr package, and now when I try to build the image of zuul-scheduler, it is failing. | 10:42 |
| ```python | ||
| ## \`50.32 Requirement already satisfied: pip in /tmp/venv/lib/python3.11/site-packages (24.0) | ||
| 50.51 Collecting pip | ||
| 50.61 Downloading pip-25.2-py3-none-any.whl.metadata (4.7 kB) | ||
| 50.70 Collecting wheel | ||
| 50.73 Downloading wheel-0.45.1-py3-none-any.whl.metadata (2.3 kB) | ||
| 50.81 Collecting build | ||
| 50.84 Downloading build-1.3.0-py3-none-any.whl.metadata (5.6 kB) | ||
| 50.86 Requirement already satisfied: setuptools in /tmp/venv/lib/python3.11/site-packages (65.5.0) | ||
| 51.08 Collecting setuptools | ||
| 51.11 Downloading setuptools-80.9.0-py3-none-any.whl.metadata (6.6 kB) | ||
| 51.21 Collecting packaging>=19.1 (from build) | ||
| 51.24 Downloading packaging-25.0-py3-none-any.whl.metadata (3.3 kB) | ||
| 51.31 Collecting pyproject\_hooks (from build) | ||
| 51.35 Downloading pyproject\_hooks-1.2.0-py3-none-any.whl.metadata (1.3 kB) | ||
| 51.42 Downloading pip-25.2-py3-none-any.whl (1.8 MB) | ||
| 51.62 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 9.2 MB/s eta 0:00:00 | ||
| 51.64 Downloading wheel-0.45.1-py3-none-any.whl (72 kB) | ||
| 51.66 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 72.5/72.5 kB 4.3 MB/s eta 0:00:00 | ||
| 51.70 Downloading build-1.3.0-py3-none-any.whl (23 kB) | ||
| 51.74 Downloading setuptools-80.9.0-py3-none-any.whl (1.2 MB) | ||
| 51.81 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 18.7 MB/s eta 0:00:00 | ||
| 51.84 Downloading packaging-25.0-py3-none-any.whl (66 kB) | ||
| 51.85 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.5/66.5 kB 3.8 MB/s eta 0:00:00 | ||
| 51.88 Downloading pyproject\_hooks-1.2.0-py3-none-any.whl (10 kB) | ||
| 51.99 Installing collected packages: wheel, setuptools, pyproject\_hooks, pip, packaging, build | ||
| 52.04 Attempting uninstall: setuptools | ||
| 52.04 Found existing installation: setuptools 65.5.0 | ||
| 52.06 Uninstalling setuptools-65.5.0: | ||
| 52.07 Successfully uninstalled setuptools-65.5.0 | ||
| 52.64 Attempting uninstall: pip | ||
| 52.65 Found existing installation: pip 24.0 | ||
| 52.67 Uninstalling pip-24.0: | ||
| 52.67 Successfully uninstalled pip-24.0 | ||
| 53.34 Successfully installed build-1.3.0 packaging-25.0 pip-25.2 pyproject\_hooks-1.2.0 setuptools-80.9.0 wheel-0.45.1 | ||
| 53.61 + /tmp/venv/bin/pip --version | ||
| 53.73 pip 25.2 from /tmp/venv/lib/python3.11/site-packages/pip (python 3.11) | ||
| 53.75 + '\[' -f /tmp/src/upper-constraints.txt '\]' | ||
| 53.75 + \[\[ -n '' \]\] | ||
| 53.75 + apt-get install -y git | ||
| 53.76 Reading package lists... | ||
| 54.21 Building dependency tree... | ||
| 54.32 Reading state information... | ||
| 54.45 git is already the newest version (1:2.39.5-0+deb12u2). | ||
| 54.45 0 upgraded, 0 newly installed, 0 to remove and 10 not upgraded. | ||
| 54.45 + install\_wheels | ||
| 54.45 + /tmp/venv/bin/python3 -m build -o /output/toplevel\_wheels ./ | ||
| 54.51 \* Creating isolated environment: venv+pip... | ||
| 54.52 \* Installing packages in isolated environment: | ||
| 55.23 - setuptools >= 40.8.0 | ||
| 55.23 \* Getting build dependencies for sdist... | ||
| 55.43 /tmp/build-env-j59kd\_4l/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. | ||
| 55.43 !! | ||
| 55.43 | ||
| 55.43 \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* | ||
| 55.43 Please consider removing the following classifiers in favor of a SPDX license expression: | ||
| 55.43 | ||
| 55.43 License :: OSI Approved :: Apache Software License | ||
| 55.43 | ||
| 55.43 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. | ||
| 55.43 \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* | ||
| 55.43 | ||
| 55.43 !! | ||
| 55.43 self.\_finalize\_license\_expression() | ||
| 55.45 \* Installing packages in isolated environment: | ||
| 56.30 - pbr | ||
| 56.30 \* Building sdist... | ||
| 56.44 Error parsing | ||
| 56.44 Traceback (most recent call last): | ||
| 56.44 File "/tmp/build-env-j59kd\_4l/lib/python3.11/site-packages/pbr/core.py", line 103, in pbr | ||
| 56.44 attrs = util.cfg\_to\_args(path, dist.script\_args) | ||
| 56.44 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 56.44 File "/tmp/build-env-j59kd\_4l/lib/python3.11/site-packages/pbr/util.py", line 345, in cfg\_to\_args | ||
| 56.44 hook\_fn = resolve\_name(hook) | ||
| 56.44 ^^^^^^^^^^^^^^^^^^ | ||
| 56.44 File "/tmp/build-env-j59kd\_4l/lib/python3.11/site-packages/pbr/util.py", line 276, in resolve\_name | ||
| 56.44 ret = **import**('.'.join(module\_name), fromlist=\[attr\_name\]) | ||
| 56.44 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 56.44 File "/tmp/src/zuul/\_setup\_hook.py", line 21, in \<module> | ||
| 56.44 \_old\_from\_git = pbr.packaging.\_from\_git | ||
| 56.44 ^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 56.44 AttributeError: module 'pbr.packaging' has no attribute '\_from\_git' | ||
| 56.44 error in setup command: Error parsing /tmp/src/setup.cfg: AttributeError: module 'pbr.packaging' has no attribute '\_from\_git' | ||
| 56.47 | ||
| 56.47 ERROR Backend subprocess exited when trying to invoke build\_sdist | ||
| ## Dockerfile:75 | ||
| ## 73 | COPY . /tmp/src | ||
| | COPY --from=js-builder /tmp/src/build /tmp/src/zuul/web/static | 00:01 | |
| | >>> RUN assemble | 00:01 | |
| | | 00:01 | |
| | # The wheel install method doesn't run the setup hooks as the source based | 00:01 | |
| ERROR: failed to build: failed to solve: process "/bin/sh -c assemble" did not complete successfully: exit code: 1 | ||
| -@gerrit:opendev.org- Jan Gutter proposed: [zuul/zuul-jobs] 957218: Raise connection pool for boto3 in s3 upload role https://review.opendev.org/c/zuul/zuul-jobs/+/957218 | 10:48 | |
| -@gerrit:opendev.org- Damian Fajfer proposed: [zuul/zuul-jobs] 957219: Remove version defaults for nodejs jobs https://review.opendev.org/c/zuul/zuul-jobs/+/957219 | 11:00 | |
| @jangutter:matrix.org | bennetefx looks like the recent release of pbr 7.0.0 might have broken things | 11:03 |
| -@gerrit:opendev.org- Bennet Benny proposed: [zuul/zuul] 957220: setup: Build JavaScript via setuptools cmdclass instead of pbr internals https://review.opendev.org/c/zuul/zuul/+/957220 | 11:08 | |
| @bennetefx:matrix.org | Yes, I have created a patch fixing the issue without relying on pbr. | 11:09 |
| -@gerrit:opendev.org- Jan Gutter proposed: [zuul/zuul-jobs] 957218: Raise connection pool for boto3 in s3 upload role https://review.opendev.org/c/zuul/zuul-jobs/+/957218 | 11:14 | |
| @bennetefx:matrix.org | * I’ve created a patch to fix the issue without relying on pbr internals. The new logic should resolve the problem and maintain the existing behaviour, hopefully without introducing other issues. | 11:15 |
| -@gerrit:opendev.org- Damian Fajfer proposed wip: [zuul/zuul-jobs] 957219: Remove version defaults for nodejs jobs https://review.opendev.org/c/zuul/zuul-jobs/+/957219 | 11:16 | |
| -@gerrit:opendev.org- Damian Fajfer marked as active: [zuul/zuul-jobs] 957219: Remove version defaults for nodejs jobs https://review.opendev.org/c/zuul/zuul-jobs/+/957219 | 11:17 | |
| @fajfer:reszka.org | I think I managed it, I also got another change that funghi +2'd already as I think it's kinda related | 11:31 |
| @fajfer:reszka.org | * I think I managed it, I also got another change that fungi +2'd already as I think it's kinda related | 11:31 |
| @bennetefx:matrix.org | * I’ve created a patch to fix the issue without relying on pbr internals. The new logic should resolve the problem and maintain the existing behaviour, hopefully without introducing other issues. | 11:47 |
| The patch failed to build the image. | ||
| -@gerrit:opendev.org- Jan Gutter proposed: [zuul/zuul] 957235: Remove dependency on pbr internal function https://review.opendev.org/c/zuul/zuul/+/957235 | 12:58 | |
| @jangutter:matrix.org | bennetefx: I tested ^^ locally and it seems to work. Your patch helped in spotting it, thanks! | 13:14 |
| @jangutter:matrix.org | At best I think it's papering over some build code that needs re-evaluating though. | 13:34 |
| @fungicide:matrix.org | jangutter: i want to say that change to pbr internals was in order to accommodate a change to setuptools internals, but i'll bring the regression to the attention of other pbr maintainers | 13:36 |
| @jangutter:matrix.org | Yeah, I'm reasonably sure that my cut-and-paste "solution" ignores 99% of the reality and should at best be regarded as yet another jenga brick on a wonky tower. | 13:40 |
| @jangutter:matrix.org | And the gate still needs unwedging: https://zuul.opendev.org/t/zuul/build/3ec89e41ba2c4a88a672f54a3c418c99 | 13:41 |
| @fungicide:matrix.org | jangutter: the change you linked didn't remove the function, merely relocated it into a different module. i'll add a detailed comment | 13:45 |
| @jangutter:matrix.org | Yeah, sorry, that comment is definitely out of whack. | 13:46 |
| @jangutter:matrix.org | * Yeah, sorry, that commit msg is definitely out of whack. | 13:46 |
| @jangutter:matrix.org | Alternative is to pin pbr<7.0.0? Would that be cleaner? | 13:49 |
| @jangutter:matrix.org | The nox-upgrade job is going to be more "fun" to fix, since the previous versions pull in the new pbr. | 13:51 |
| -@gerrit:opendev.org- Damian Fajfer proposed wip: [zuul/zuul] 956167: Update React version to 17.0.0 https://review.opendev.org/c/zuul/zuul/+/956167 | 13:53 | |
| -@gerrit:opendev.org- Damian Fajfer proposed wip: [zuul/zuul] 956167: Update React version to 17.0.0 https://review.opendev.org/c/zuul/zuul/+/956167 | 13:53 | |
| @fungicide:matrix.org | jangutter: yeah, probably not pinning it will be easier for now. i think that pbr change was mainly prophylactic in preparing for upcoming changes to setuptools, so it's possible that the distribution.get_option_dict('pbr') might get broken in setuptools in the coming months | 13:56 |
| @jangutter:matrix.org | Hyrum's law! | 13:56 |
| @jangutter:matrix.org | Just waiting for the CI runs to complete before I send up a new version with the _compat function. | 13:58 |
| @fungicide:matrix.org | but also i agree we (or at least i) didn't really consider that consumers might be calling into leading-underscore methods and thought it was safe to move that without warning or a release note | 13:58 |
| @jangutter:matrix.org | I think, while dynamic languages are probably more prone to these kind of linkages, it's basically a fact of life of integrated systems. If you don't have these kinds of things, you are relying on unchanging interfaces - and that's a different, harder to solve problem. | 14:01 |
| @fungicide:matrix.org | the point of that pbr change was really that anything in _compat is stuff that we need for supporting older versions of setuptools for older (non-pyproject.toml) workflows, but has broken or is likely to break in the future with newer setuptools releases | 14:05 |
| @jangutter:matrix.org | Now, the other question - should I set `zuul-nox-upgrade` to non-voting? I guess it's possible to figure out a trick to install the older Zuul versions - but that will also be temporary. | 14:06 |
| @fungicide:matrix.org | oh, good point, can't really backport the fix to the old zuul release | 14:08 |
| @fungicide:matrix.org | so that job may indeed need to use a version of pbr that works with the old release | 14:08 |
| @jangutter:matrix.org | Heh, I'm going to test if just forking requirements.txt into upgrade-requirements.txt temporarily will work. Just waiting for the current run to finish. | 14:26 |
| -@gerrit:opendev.org- Jan Gutter proposed: [zuul/zuul] 957235: Fix reference to internal pbr function https://review.opendev.org/c/zuul/zuul/+/957235 | 14:29 | |
| @clarkb:matrix.org | I wonder if we can just setup _build_javascript as the setup_hook and not do the indirection to run it later | 15:44 |
| @jangutter:matrix.org | I'll be honest, until I found the setup hook I had entertained an idea of malicious code for a millisecond. It's quite scary how much a pip install can actually change on your system. | 15:58 |
| @jim:acmegating.com | a temporary upgrade-requirements is one solution; or you could pin; fix; unpin. it probably doesn't matter which. | 16:01 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 957188: Vendor openvswitch_bridge https://review.opendev.org/c/zuul/zuul-jobs/+/957188 | 16:03 | |
| @jangutter:matrix.org | corvus: It's going to end up getting reverted anyway, one of those nice quantum tunneling things. | 16:04 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 957188: Vendor openvswitch_bridge https://review.opendev.org/c/zuul/zuul-jobs/+/957188 | 16:08 | |
| @fungicide:matrix.org | jangutter: corvus: in #opendev irc we're talking about the possibility of adding a backward-compat alias at the old location and releasing pbr 7.0.1 as an option too | 16:14 |
| @fungicide:matrix.org | though turn-around time for that is probably on the order of days, so a more immediate fix in zuul is likely preferable | 16:15 |
| @jim:acmegating.com | yep, i think we can fix this in zuul | 16:15 |
| @jangutter:matrix.org | clark made a good comment, so I can take one more stab at a band-aid, but please feel free to take over / abandon / change anything there. I'm regarding this as a learning opportunity. | 16:17 |
| @jim:acmegating.com | that channel is #_oftc_#opendev:matrix.org btw for anyone that wants a clickable matrix link | 16:17 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 957188: Vendor openvswitch_bridge https://review.opendev.org/c/zuul/zuul-jobs/+/957188 | 16:26 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957267: Run build-js directly from PBR setup hook https://review.opendev.org/c/zuul/zuul/+/957267 | 16:29 | |
| @jim:acmegating.com | Clark: jangutter fungi ^ that's Clark's idea manifested. i feel like there must be some reason that wasn't done originally. but i can't find anything in the historical record to indicate why. so maybe that will tell us. | 16:30 |
| @jim:acmegating.com | maybe the setup_hook runs too often? like maybe it runs if you do "pbr freeze" ? | 16:31 |
| @jim:acmegating.com | (no one expects pbr freeze to run a js build) | 16:31 |
| @clarkb:matrix.org | ah that could be | 16:31 |
| @jangutter:matrix.org | would have been be much clearer to call it spanish_inquisition_hook... | 16:32 |
| @jim:acmegating.com | well, a quick local test with that doesn't show any problems with pbr freeze and friends | 16:34 |
| @jim:acmegating.com | i guess we'll see what the test results show | 16:34 |
| @fungicide:matrix.org | might help to compare job runtimes too, in case additional invocations aren't obvious | 16:38 |
| @jangutter:matrix.org | One of the nice things I just noticed was hovering over the status tells you "waiting for nodeset xxxx" | 16:39 |
| @jim:acmegating.com | i intend for that to eventually provide more information (whether in that popup, or as a link) | 16:43 |
| @jangutter:matrix.org | aaah, these nodes have been requested a while ago. Looks like it's unlikely they'll get fulfilled, I'll send up a new version. | 16:44 |
| -@gerrit:opendev.org- Jan Gutter proposed: [zuul/zuul] 957235: Fix reference to internal pbr function https://review.opendev.org/c/zuul/zuul/+/957235 | 16:45 | |
| @jangutter:matrix.org | Using the pbr internal function: https://zuul.opendev.org/t/zuul/build/01dee3e0f0cf4bf1aa6eadc1db50e1ec/console | 17:01 |
| Running build-js directly from the hook: | ||
| https://zuul.opendev.org/t/zuul/build/c27c683e133b4effba34a12d371df5aa/console | ||
| @jangutter:matrix.org | Also, it's not as simple as just adding a new update-requirements: the entire repo gets reverted back to the current tip, meaning that it's effectively using the nox files that are in the repo at that time. I'm really thinking that setting it to non-voting until the fix is merged, then reverting immediately after is the simplest. | 17:08 |
| @jim:acmegating.com | we can do that or force-merge, depending on the circumstances. identifying the fix is the priority. | 17:12 |
| @clarkb:matrix.org | https://review.opendev.org/c/zuul/zuul/+/957235 seems safest from a preserving old behavior standpoint | 17:14 |
| @clarkb:matrix.org | I think my inclination would be to land that with upgrade job set to non voting or force merge it. Then in followups look into moving the function to setup_hook | 17:14 |
| @clarkb:matrix.org | that way we can do the most dangerous thing without feeling rushed | 17:14 |
| @fungicide:matrix.org | other than the upgrade-requirements.txt part i guess? | 17:15 |
| @jim:acmegating.com | i mean... it's broken | 17:15 |
| @fungicide:matrix.org | since that's not going to help | 17:15 |
| @jangutter:matrix.org | Yeah, that's functionally useless, I'll send up one yanking it. | 17:15 |
| @jim:acmegating.com | i'm not worried about breaking it more | 17:15 |
| @clarkb:matrix.org | in other news https://review.opendev.org/c/zuul/zuul-jobs/+/957188 passes CI testing now. I think we can land this change ahead of opendev's Ansible 11 default switch Monday | 17:16 |
| @clarkb:matrix.org | corvus: ya and I guess with most people consuming zuul from container images as long as those build successfully then we probably don't need to worry too much about any new behaviors from setuptools/pbr using the setup_hook directly? | 17:16 |
| @fungicide:matrix.org | i'm okay with rolling in the more invasive fix and then fixing any new regressions that may crop up with that approach | 17:17 |
| @clarkb:matrix.org | I just like to stick with the smallest change possible when under the gun and then figure things out from there | 17:17 |
| @fungicide:matrix.org | i'm not sure we're under any gun, we just can't merge new changes at the moment until we decide on something | 17:17 |
| @jim:acmegating.com | Clark: yeah, that's kind of what i'm thinking... normally i'd agree about the smallest fix; but i do wonder what else we would do? | 17:17 |
| @clarkb:matrix.org | corvus: I did leave a comment on https://review.opendev.org/c/zuul/zuul/+/957267 for the reason why it failed linters | 17:18 |
| @jim:acmegating.com | like, what other testing do we want? do we want to see if it's a problem for people building packages from source in some other way? | 17:18 |
| @clarkb:matrix.org | I think we should fix that so that we don't force merge something that creates a new known problem | 17:18 |
| @jim:acmegating.com | yeah, we wouldn't force merge something that isn't otherwise passing tests :) | 17:19 |
| @clarkb:matrix.org | corvus: I think the main concern would be in setup.py commands that aren't sdist/wheel/install commands like you mentioned with freeze previously. But I also think those use cases are secondary to what we're already testing with nox installs and container image builds | 17:19 |
| @clarkb:matrix.org | so as long as the tests we have work (other than the upgrade test) I think we can just go with the setup_hook change and not rely on pbr internals that might change | 17:19 |
| -@gerrit:opendev.org- Jan Gutter proposed: [zuul/zuul] 957235: Fix reference to internal pbr function https://review.opendev.org/c/zuul/zuul/+/957235 | 17:19 | |
| @jangutter:matrix.org | I'm not emotionally attached to ^ at all, mostly just sending it up so you have logs of an alternate universe. | 17:21 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 17:21 | |
| - [zuul/zuul] 957267: Run build-js directly from PBR setup hook https://review.opendev.org/c/zuul/zuul/+/957267 | ||
| - [zuul/zuul] 957270: Re-enable upgrade test https://review.opendev.org/c/zuul/zuul/+/957270 | ||
| @jim:acmegating.com | I'm fine with either approach. i think if no one can think of an issue with the setup_hook version, or any more testing we should do with it, then let's do that in order to go ahead and better future-proof. if we find an issue, we can fall back to jan's change. | 17:22 |
| @clarkb:matrix.org | sounds like a plan | 17:22 |
| @clarkb:matrix.org | one idea for a test: after 957267 builds container images we fetch those artifacts and double check the js content is present in the installation. We test the container images with quickstart but I'm not sure if that job exercises the web ui? | 17:24 |
| @jim:acmegating.com | one other bit of testing we should do: try to build a release artifact like we do for tagged releases. | 17:24 |
| @jim:acmegating.com | maybe that's enough pending tests to tip the scales to jangutter 's change | 17:25 |
| @jim:acmegating.com | yeah, i think i talked myself into that. let's do that. | 17:26 |
| @jangutter:matrix.org | Can definitely confirm that js-build gets skipped using the internal function, and runs in the wheel build using the direct hook method: https://zuul.opendev.org/t/zuul/build/c27c683e133b4effba34a12d371df5aa/console#3/0/23/ubuntu-jammy | 17:27 |
| compared to the one using the private method: | ||
| https://zuul.opendev.org/t/zuul/build/01dee3e0f0cf4bf1aa6eadc1db50e1ec/console#3/0/23/ubuntu-jammy | ||
| @clarkb:matrix.org | I went ahead and approved jangutter's change | 17:29 |
| @fungicide:matrix.org | and i suppose we need a follow-up to make zuul-nox-upgrade voting again | 17:30 |
| @clarkb:matrix.org | re running the build_javascript function more often eg when building wheels that is probably fine if inefficient. | 17:30 |
| @clarkb:matrix.org | As I suspect we'll build javascript when building the sdist to build the wheel from. Then we'll build javascript again when creating the wheel | 17:31 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: | 17:31 | |
| - [zuul/zuul] 957270: Re-enable nox-upgrade job https://review.opendev.org/c/zuul/zuul/+/957270 | ||
| - [zuul/zuul] 957267: Run build-js directly from PBR setup hook https://review.opendev.org/c/zuul/zuul/+/957267 | ||
| @clarkb:matrix.org | the js stuff isn't zero cost but it isn't too bad right now especially if we have things precached from the first run | 17:31 |
| @fungicide:matrix.org | that hook could probably become more complex and check for the existence of the built files or something, i suppose, if it became a problem | 17:32 |
| @fungicide:matrix.org | though i could see that having a down-side of shipping outdated versions if run in a dirty context (e.g. with build isolation disabled or something)_ | 17:33 |
| @jangutter:matrix.org | Newbie question: those javascript built files are expected to be packaged in the zuul release wheels? | 17:35 |
| @jim:acmegating.com | yes | 17:36 |
| @jim:acmegating.com | zuul-web is intended to be entirely self-contained | 17:36 |
| @fungicide:matrix.org | it's also a prime candidate for sbom generation in the future, i think | 17:38 |
| @fungicide:matrix.org | since those packages are shipping embedded components that didn't originate solely from zuul development | 17:38 |
| @jangutter:matrix.org | yeah, the hashes in the yarn lock file helps.... | 17:39 |
| @fungicide:matrix.org | would be nice some day for downstream consumers to be able to easily determine what versions of various libs are in a wheel, tracking that back to yarn.lock in the git repository isn't necessarily trivial | 17:39 |
| @fungicide:matrix.org | at the very least it's a manual task, not something that automated scanners would do | 17:40 |
| @fungicide:matrix.org | and if the sbom pep(s) for python packaging every stabilize, it's likely pypi will start listing component versions for stuff listed in them some day too | 17:41 |
| @fungicide:matrix.org | s/every.ever/ | 17:41 |
| @jangutter:matrix.org | Yeah, and then you'll start the fun portion of arguing with sbom scanners that insist you used version X of some artifact when you have the build records you used Y. | 17:42 |
| @jim:acmegating.com | Clark: fungi would you please review 267 and 270? | 17:42 |
| @fungicide:matrix.org | corvus: minor correction for the commit message on the second one | 17:44 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957267: Run build-js directly from PBR setup hook https://review.opendev.org/c/zuul/zuul/+/957267 | 17:45 | |
| @fungicide:matrix.org | both lgtm now, thanks! | 17:45 |
| @clarkb:matrix.org | corvus: I approved 270. With the info we have now I'm happy with 267 but wondering if I should avoid +2'ing until we've done those additional checks or +2 under the assumption we'll check them before approving? | 17:47 |
| @jim:acmegating.com | we should not approve it till then... feel free to just +1 it | 17:48 |
| @clarkb:matrix.org | done | 17:49 |
| @jim:acmegating.com | i'm working on the launcher issue, so i won't have a chance to do those tests for a bit (if someone else can, that'd be great) | 17:49 |
| @clarkb:matrix.org | once image builds complete I'll fetch the containers and look for the js content on disk | 17:50 |
| @fajfer:reszka.org | I wonder if these pbr changes will help me with react upgrades, I think that after the upgrade the frontend stuff became too heavy and that is why I have random failures from backend tests | 18:06 |
| @clarkb:matrix.org | I wouldn't expect it to affect react much. We're just changing slightly where the js stuff is installed in the zuul package building process | 18:07 |
| @clarkb:matrix.org | the js itself doesn't change | 18:07 |
| -@gerrit:opendev.org- Damian Fajfer proposed wip: [zuul/zuul] 956167: Update React version to 17.0.0 https://review.opendev.org/c/zuul/zuul/+/956167 | 18:19 | |
| -@gerrit:opendev.org- Damian Fajfer proposed wip: [zuul/zuul] 956167: Update React version to 17.0.0 https://review.opendev.org/c/zuul/zuul/+/956167 | 18:21 | |
| -@gerrit:opendev.org- Damian Fajfer proposed: [zuul/zuul] 957273: Temporary set node_version var for nodejs jobs https://review.opendev.org/c/zuul/zuul/+/957273 | 18:23 | |
| -@gerrit:opendev.org- Damian Fajfer proposed wip: [zuul/zuul] 956167: Update React version to 17.0.0 https://review.opendev.org/c/zuul/zuul/+/956167 | 18:24 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957275: Launcher: improve overfull allocation with incomplete quota info https://review.opendev.org/c/zuul/zuul/+/957275 | 18:49 | |
| @clarkb:matrix.org | my pull of the zuul-web image is very slow | 18:52 |
| @clarkb:matrix.org | but it is in progress and as soon as I get that last image layer I will check the js content in the image build | 18:53 |
| -@gerrit:opendev.org- Zuul merged on behalf of Jan Gutter: [zuul/zuul] 957235: Fix reference to internal pbr function https://review.opendev.org/c/zuul/zuul/+/957235 | 19:04 | |
| @clarkb:matrix.org | corvus: the container image contents lgtm using this naive check: https://paste.opendev.org/show/bWy8FuwsviJRHbyKek80/ | 19:09 |
| @jim:acmegating.com | that certainly seems plausible, thanks! | 19:14 |
| @clarkb:matrix.org | looking at the release process zuul-build-python-release and zuul-release-python ultimately run https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/build-python-release/tasks/main.yaml#L16-L32 | 19:14 |
| @clarkb:matrix.org | this means we've got good coverage of the high level process. The only thing that I think differs is that when run in the release pipeline we'll be generating the image from the explicit tag and not inferred from the last tag | 19:15 |
| @clarkb:matrix.org | I don't think the code path we're changing affects taht. However, it may be possible that version info wouldn't be available to the js code if we're not generating the version in a way that it can consume anymore? I think the js may ask the server for the version though and we don't version the js separately? | 19:15 |
| @jim:acmegating.com | correct | 19:16 |
| @clarkb:matrix.org | I think the risk here is low. However, if someone wants to do a local tag and use pyproject-build to build the sdist and wheel just to confirm that may not be a bad idea | 19:16 |
| @clarkb:matrix.org | I need to grab lunch now though | 19:16 |
| @clarkb:matrix.org | I'm working on running pyproject-build against a locally tagged checkout of 267 | 20:34 |
| @clarkb:matrix.org | took a minute to get a venv set up with node and yarn etc but its seems to be running now. Just need to wait for results | 20:34 |
| @clarkb:matrix.org | `Successfully built zuul-12.24.48.tar.gz and zuul-12.24.48-py3-none-any.whl` is the report from pyproject-build. I'll look at the package contents too momentarily | 20:35 |
| @clarkb:matrix.org | both the .tar.gz sdist and the .whl (a zipfile) have the static/js/ chunk files so I think this works | 20:36 |
| @clarkb:matrix.org | I've changed my +1 on https://review.opendev.org/c/zuul/zuul/+/957267 to a +2 as both of these additional checks we called out lgtm | 20:38 |
| @fungicide:matrix.org | out of curiosity, how do you go about getting the js build toolchain on your system? i've had minimal luck, in part because debian dropped the old yarn which zuul uses in favor of the rewrite | 20:38 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957282: Add request_id to nodes REST endpoint https://review.opendev.org/c/zuul/zuul/+/957282 | 20:39 | |
| @jim:acmegating.com | fungi: tools/install-js-tools.sh will do it, but by installing the yarn repo | 20:40 |
| @fungicide:matrix.org | ah, so third-party distro packages needed in this case. got it | 20:42 |
| @clarkb:matrix.org | yes I manually used the steps in tools/yarn-build.sh. Created a virtualenv, installed nodeenv and build to it, installed node with nodeenv, activated the top level virtualenv, installed yarn with npm, then ran pyproject-build from the same top level venv | 20:42 |
| @clarkb:matrix.org | I'm on tumbleweed and probably could have installed node, npm, and yarn globally from the distro too. But I wanted to test things as close to what zuul's release builds would do and this way I avoid unnecessary updates every time I system update | 20:43 |
| @fungicide:matrix.org | yeah, in debian the available packaged yarn is simply too new to support zuul's yarn.lock format and have the expected yarnpkg executable command | 20:51 |
| @clarkb:matrix.org | oh right this is the yarn did a complete rewrite between versions 1 and 2 and now supports both (or did for a while) as basically two different tools with the same name problem | 20:52 |
| @clarkb:matrix.org | and debian didn't make a yarn1 and yarn2 package. They just have a yarn package for v2 | 20:52 |
| @clarkb:matrix.org | my distro yarn package is v1 | 20:53 |
| @fungicide:matrix.org | right, there's supposedly a tool for migrating yarn.lock files to the yarn v2 format (where yarn v2 is actually now like v4 or v5 but that's another story) | 20:54 |
| @fungicide:matrix.org | at least people have finally mostly stopped referring to zuul v1x as "v3" | 20:55 |
| @fungicide:matrix.org | guess we're up to v12 now | 20:56 |
| @fungicide:matrix.org | i started looking at moving zuul to new-style yarn because it has a cyclonedx sbom generator plugin available, but it was somewhat involved and then i got distracted by something shiny flying by and haven't gotten back to it | 20:58 |
| @fungicide:matrix.org | was also well outside my field of expertise, so the learning curve was fairly steep | 20:59 |
| @fungicide:matrix.org | i think where i left off was actually trying to get old yarn working locally so that i could do proper side-by-side comparisons of the build results | 21:00 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957283: Launcher: avoid duplicate delete jobs https://review.opendev.org/c/zuul/zuul/+/957283 | 21:05 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957283: Launcher: avoid duplicate delete jobs https://review.opendev.org/c/zuul/zuul/+/957283 | 21:07 | |
| @clarkb:matrix.org | corvus: not sure if you want to do additional checks at this point or just wait on landing the setup_hook update for a quieter day but I think I've convinced myself it is unlikely to cause problems for the primary use case of producing working pacakges with js content in them | 21:13 |
| @jim:acmegating.com | Clark: thanks! i'm happy for it to merge asap | 21:15 |
| @clarkb:matrix.org | did you want to approev it then or should I? | 21:17 |
| @jim:acmegating.com | done | 21:17 |
| @clarkb:matrix.org | the zuul-jobs vendoring of openvswitch_bridge has a readme update with a warning about the role effectively being deprecated and needing a rewrite now. It passes its own tests and the tempest canary change that caught the issue also passes with the latest version of the vendoring change. I think that is good to go if we're happy with vendoring that code with the warning | 21:22 |
| @jim:acmegating.com | sounds like a plan to me -- probably worth a note to zuul-announce (but we don't need to wait for that because we're not breaking it) | 21:23 |
| @jim:acmegating.com | (or could just be zuul-discuss if you don't think announce is right) | 21:23 |
| @jim:acmegating.com | (no strong opinion here) | 21:23 |
| @clarkb:matrix.org | ++ I can write that now. Something like "We want you to know that the role is on life support now and we're updating it to keep it working with the Ansible 11 switch but there is no guarantee it will keep working over time. We would be happy to accept a rewrite to use linux bridges instead" | 21:24 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957284: Launcher: add more debug messages to quota percentage calc https://review.opendev.org/c/zuul/zuul/+/957284 | 21:36 | |
| @clarkb:matrix.org | I just sent an email to -announce. Feel free to reject it in moderation if the content is not correct or punt it to -discuss | 21:36 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957284: Launcher: add more debug messages to quota percentage calc https://review.opendev.org/c/zuul/zuul/+/957284 | 21:42 | |
| @jim:acmegating.com | Clark: ^ oopsie on that if you don't mind a re-review | 21:43 |
| @clarkb:matrix.org | corvus: should the self.log.exception case be chagned to use log as well? | 21:44 |
| @jim:acmegating.com | sure why not | 21:45 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 957284: Launcher: add more debug messages to quota percentage calc https://review.opendev.org/c/zuul/zuul/+/957284 | 21:45 | |
| @jim:acmegating.com | we have a pretty big list of changes trying to run the gauntlet right now; if you have a sec to look at https://review.opendev.org/957171 -- that's a super simple test race fix | 21:57 |
| @jim:acmegating.com | * we have a pretty big list of changes trying to run the gauntlet right now; Clark if you have a sec to look at https://review.opendev.org/957171 -- that's a super simple test race fix | 21:57 |
| @clarkb:matrix.org | approved | 21:59 |
| @jim:acmegating.com | thx. might improve our chances :) | 21:59 |
| @jim:acmegating.com | oh there's 2 more fixes if you have time: https://review.opendev.org/956821 https://review.opendev.org/956822 | 21:59 |
| @clarkb:matrix.org | also approved | 22:03 |
| @clarkb:matrix.org | I'm noticing a larger number than expected of node failures on zuul and nodepool changes | 23:29 |
| @clarkb:matrix.org | I guess we're still waiting for these fixes to land so may be a bit chicken and egg? | 23:30 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: | 23:31 | |
| - [zuul/zuul] 957270: Re-enable nox-upgrade job https://review.opendev.org/c/zuul/zuul/+/957270 | ||
| - [zuul/zuul] 957275: Launcher: improve overfull allocation with incomplete quota info https://review.opendev.org/c/zuul/zuul/+/957275 | ||
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 957282: Add request_id to nodes REST endpoint https://review.opendev.org/c/zuul/zuul/+/957282 | 23:32 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 957283: Launcher: avoid duplicate delete jobs https://review.opendev.org/c/zuul/zuul/+/957283 | 23:32 | |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul] 957284: Launcher: add more debug messages to quota percentage calc https://review.opendev.org/c/zuul/zuul/+/957284 | 23:42 | |
| -@gerrit:opendev.org- Clark Boylan proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/nodepool] 909782: Add pod image username attribute https://review.opendev.org/c/zuul/nodepool/+/909782 | 23:46 | |
| @jim:acmegating.com | Clark: yeah, i don't want to dig into that until we've restabilized | 23:49 |
| -@gerrit:opendev.org- Clark Boylan proposed on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/nodepool] 933899: Slow processing of failed nodescan connections https://review.opendev.org/c/zuul/nodepool/+/933899 | 23:55 | |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!