Wednesday, 2025-04-09

-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 945907: Make Gerrit Connection Replication Aware https://review.opendev.org/c/zuul/zuul/+/94590700:23
-@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 945907: Make Gerrit Connection Replication Aware https://review.opendev.org/c/zuul/zuul/+/94590703:18
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 945423: WIP: Changes needed for run tests in container https://review.opendev.org/c/zuul/zuul/+/94542308:02
-@gerrit:opendev.org- Dong Zhang proposed: [zuul/zuul] 942886: Prepare oidc token for playbook execution in executor. https://review.opendev.org/c/zuul/zuul/+/94288608:21
@morucci:matrix.orgHi Folks, do you have a schedule to tag the next version of Zuul ? I'm asking because we hit quite a lot (120 times in the last 7 days) the issue that might be fixed by https://opendev.org/zuul/zuul/commit/4e07779c5411b711e990d5a96ce28d85d2189258 08:21
@apevec:matrix.orguhh PS1 July 2023 and merged March 202509:09
-@gerrit:opendev.org- Matthieu Huin https://matrix.to/#/@mhuin:matrix.org proposed: [zuul/zuul] 946748: zuul-web: fix traceback in "job" endpoint if a job doesn't have variants https://review.opendev.org/c/zuul/zuul/+/94674810:01
-@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 946754: Capitalize AWS image filter key/value arguments https://review.opendev.org/c/zuul/zuul/+/94675411:07
@clarkb:matrix.orgLooks like the updated base container image script did fix image builds for zuul13:20
@fungicide:matrix.orgironically, the "official" python container images on dockerhub were completely broken by the same problem. in their build process they relied on ensurepip which installed an older version of setuptools bundled in cpython, but since ensurepip doesn't install wheel they were separately installing an unconstrained version of that so ended up installing 0.46.1 which then broke the installed setuptools13:28
@jim:acmegating.comfbo: no schedule as always, but probably within the next couple of weeks.  there's still a bunch of stuff in flux.13:35
@morucci:matrix.orgThanks for the insights, then we will backport the change in our images in the meantime13:46
-@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 946754: Capitalize AWS image filter key/value arguments https://review.opendev.org/c/zuul/zuul/+/94675413:55
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 946792: Slow processing of failed nodescan connections https://review.opendev.org/c/zuul/zuul/+/94679217:40
@f2ked:matrix.orgtips on debugging? when using `zuul-web` trying to get project info and I get this exception18:13
```
ERROR cherrypy.error.139717006660944: [req: 5fd57bea0ddb482888ddfc8f851fd2d1] [09/Apr/2025:18:08:18] HTTP
ERROR cherrypy.error.139717006660944: Traceback (most recent call last):
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/cherrypy/_cprequest.py", line 659, in respond
ERROR cherrypy.error.139717006660944: self._do_respond(path_info)
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/cherrypy/_cprequest.py", line 718, in _do_respond
ERROR cherrypy.error.139717006660944: response.body = self.handler()
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/cherrypy/lib/encoding.py", line 223, in __call__
ERROR cherrypy.error.139717006660944: self.body = self.oldhandler(*args, **kwargs)
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/zuul/web/__init__.py", line 936, in json_handler
ERROR cherrypy.error.139717006660944: value = cherrypy.serving.request._json_inner_handler(*args, **kwargs)
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
ERROR cherrypy.error.139717006660944: return self.callable(*self.args, **self.kwargs)
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/zuul/web/__init__.py", line 1922, in project
ERROR cherrypy.error.139717006660944: md = tenant.layout.getProjectMetadata(project.canonical_name).toDict()
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: AttributeError: 'NoneType' object has no attribute 'toDict'
INFO cherrypy.access.139717006660944: [req: 5fd57bea0ddb482888ddfc8f851fd2d1] XXX - - [09/Apr/2025:18:08:18] "GET /api/tenant/XXX-tenant/project/opendev.org/zuul/zuul-jobs HTTP/1.1" 500 825 "http://XXX-zuul-web01.XXX.com:9000/t/XXX-tenant/project/opendev.org/zuul/zuul-jobs" "XXX"
```
@f2ked:matrix.org* tips on debugging? when using `zuul-web` trying to get project info and I get this exception (redacted `XXX`)18:14
```
ERROR cherrypy.error.139717006660944: [req: 5fd57bea0ddb482888ddfc8f851fd2d1] [09/Apr/2025:18:08:18] HTTP
ERROR cherrypy.error.139717006660944: Traceback (most recent call last):
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/cherrypy/_cprequest.py", line 659, in respond
ERROR cherrypy.error.139717006660944: self._do_respond(path_info)
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/cherrypy/_cprequest.py", line 718, in _do_respond
ERROR cherrypy.error.139717006660944: response.body = self.handler()
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/cherrypy/lib/encoding.py", line 223, in __call__
ERROR cherrypy.error.139717006660944: self.body = self.oldhandler(*args, **kwargs)
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/zuul/web/__init__.py", line 936, in json_handler
ERROR cherrypy.error.139717006660944: value = cherrypy.serving.request._json_inner_handler(*args, **kwargs)
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
ERROR cherrypy.error.139717006660944: return self.callable(*self.args, **self.kwargs)
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: File "/usr/local/lib/python3.11/site-packages/zuul/web/__init__.py", line 1922, in project
ERROR cherrypy.error.139717006660944: md = tenant.layout.getProjectMetadata(project.canonical_name).toDict()
ERROR cherrypy.error.139717006660944: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR cherrypy.error.139717006660944: AttributeError: 'NoneType' object has no attribute 'toDict'
INFO cherrypy.access.139717006660944: [req: 5fd57bea0ddb482888ddfc8f851fd2d1] XXX - - [09/Apr/2025:18:08:18] "GET /api/tenant/XXX-tenant/project/opendev.org/zuul/zuul-jobs HTTP/1.1" 500 825 "http://XXX-zuul-web01.XXX.com:9000/t/XXX-tenant/project/opendev.org/zuul/zuul-jobs" "XXX"
```
@jim:acmegating.comf2ked: it's probably not you, i think that's a bug in master18:21
@f2ked:matrix.orgthanks!18:31
@f2ked:matrix.orgI'm still stuck on [this pipeline check](https://matrix.to/#/!yuuvjJSOEGSfTzxOjK:opendev.org/$4j8IFalWIGw-Bbe8ENFsIePHuNdC1EUr0d1IC6m763U?via=opendev.org&via=matrix.org&via=fedora.im) from yesterday.. I do not see the message that fungi suggests.. any other tips..18:33
I'm concerned that I just have my references crossed
@clarkb:matrix.orgyou confirmed that you have debug logs in your log files? I would expect similar messages if you have debug enabled18:36
@clarkb:matrix.orgthen the other thing is to check that you are looking at the correct logfile18:37
@clarkb:matrix.orgwould be the scheduler log for the things you are looking at18:37
@f2ked:matrix.orgwe have instrumented some of this code and it appears we don't have any `pipeline_manager`s to match against even thought pipelines are being parsed18:39
@f2ked:matrix.orgso we don't event get a change to check the messahe18:39
@f2ked:matrix.org* so we don't event get a chance to check the messahe18:39
@f2ked:matrix.org* so we don't event get a chance to check the message18:40
@clarkb:matrix.orgI would expect there to be errors logged if you have errors in the config. But also pipelines may need to be in trusted repo config and maybe yours aren't so are being ignored?18:40
@f2ked:matrix.orgit is in a repo listed under `tenant.source.config-projects` 18:43
@f2ked:matrix.organd 18:44
```
INFO zuul.TenantParser: Loading configuration from XXX-zuul-config/zuul.d/pipelines.yaml@main
```
is the right one
@clarkb:matrix.orgya that looks correct to me18:45
@clarkb:matrix.organy errors after that message?18:45
@f2ked:matrix.orgnone :(18:45
@f2ked:matrix.orgam I right that the pipeline tr18:48
```
trigger:
XXX-gerrit:
```
should match the connection in `zuul.conf`
```
[connection "XXX-gerrit"]
driver=gerrit
```
@f2ked:matrix.org* am I right that the pipeline 18:48
```
trigger:
XXX-gerrit:
```
should match the connection in `zuul.conf`
```
[connection "XXX-gerrit"]
driver=gerrit
```
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 946818: Avoid 500 on /project with no project configs https://review.opendev.org/c/zuul/zuul/+/94681818:49
@clarkb:matrix.orgyes I believe so. Its saying look for these triggers on that connection18:49
@jim:acmegating.comf2ked: that particular bug you hit suggests that you have a project defined in a tenant with no project config stanzas, which could be a cause for items not being enqueued in the pipeline.18:51
@f2ked:matrix.orghere is my simple tenant18:57
```
- tenant:
name: XXX-tenant
source:
XXX-gerrit:
config-projects:
- XXX-zuul-config
untrusted-projects:
- my-proj:
include-branches:
- master
opendev.org:
untrusted-projects:
- zuul/zuul-jobs:
include:
- job
```
@clarkb:matrix.orgf2ked: does my-proj have any pipelines and jobs configured on its master branch?19:10
@f2ked:matrix.orgthere is no zuul anything in the master branch, we are trying to get it working before we merge it19:12
@f2ked:matrix.orgoohh also `XXX-zuul-config`'s "master" branch is called "main".. is that significant?19:14
@f2ked:matrix.orghmmm... https://zuul-ci.org/docs/zuul/latest/tenants.html#attr-tenant.config-projects.%3Cproject%3E.load-branch19:16
@clarkb:matrix.orgf2ked: the config you pasted above says include-branches master not main.19:33
@clarkb:matrix.orgf2ked: but also what corvus said earlier is until your project has any config there is nothing to do19:33
@clarkb:matrix.orgthis is why you got the zuul-web error that you pasted19:33
@clarkb:matrix.orgif you configured jobs that run when that trigger is triggered then you should see something happen hopefully19:33
@f2ked:matrix.org`my-proj` does have `master` since it is legacy. `XXX-zuul-config` uses `main`19:34
@f2ked:matrix.orgkk I guess I'll merge just this once without testing 😬19:35
@clarkb:matrix.orgyou can start with just the noop job19:35
@clarkb:matrix.orgit doesn't actually do anything zuul just says success and returns19:35
@clarkb:matrix.orgbut that should be sufficient to exercise the pipeline19:36
@f2ked:matrix.org> this is why you got the zuul-web error that you pasted19:41
even for unrelated projects?
@f2ked:matrix.org* > this is why you got the zuul-web error that you pasted19:41
even for unrelated projects?
@clarkb:matrix.orgI don't know19:47
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 946792: Slow processing of failed nodescan connections https://review.opendev.org/c/zuul/zuul/+/94679219:58
-@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 946852: WIP: Add niz cleanup https://review.opendev.org/c/zuul/zuul/+/94685222:39
@f2ked:matrix.orgFYI: circling back, merging into `my-project@main` and performing a `full-reconfigure` got us pipelines we needed.. thank all!22:43
@jim:acmegating.comf2ked: great!  there's some discussion about the nuances of the bootstrapping process in the quickstart tutorial; in retrospect it sounds like that may have been what you were dealing with: https://zuul-ci.org/docs/zuul/latest/tutorials/quick-start.html#configure-zuul-pipelines22:50

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!