| @lecris:matrix.org | Does zuul not have an ansible API reference documentation, can't seem to find one for stuff like `zuul_return` | 10:59 |
|---|---|---|
| @lecris:matrix.org | Was also looking for any cache capabilities or other ways to share artifacts across different job runs | 11:01 |
| @bennetefx:matrix.org | https://zuul-ci.org/docs/zuul/latest/job-content.html#return-values | 11:07 |
| @jangutter:matrix.org | Cristian Le: We use something like this https://opendev.org/zuul/zuul-jobs/src/commit/4555597c1f145d4986e195051350f78e3626bb1e/roles/promote-artifact/tasks/main.yaml | 12:14 |
| Zuul itself does not store the artifact, but a zuul job uploads it to a known external location, the location is then passed using zuul_return, and can be queried even if the job does not have a direct dependency. | ||
| @jangutter:matrix.org | If you can run the jobs in the same buildset though, that's pretty easy, and you can pass the values like normal ansible vars using zuul_return. The job that depends on will get the variables in the inventory. | 12:15 |
| @jangutter:matrix.org | Cristian Le: Finally, while the internals of the zuul_return plugin is quite simple - https://opendev.org/zuul/zuul/src/branch/master/zuul/ansible/base/action/zuul_return.py you have to dig a bit deeper to find out how Zuul passes on the results.json on to the dependent jobs. | 12:18 |
| @lecris:matrix.org | But that's not an API reference, e.g. it does not show up on search (sphinx api reference creates and index to assist with those) | 12:56 |
| @lecris:matrix.org | Ok, so would have to do it in an external location, I'll check what we have available, thanks for the feedback | 12:58 |
| @clarkb:matrix.org | zuul_return isn't exposed via python directly but rather through ansible. I'm not sure it makes sense to use sphinx's python apidoc tooling to generate documentation for it | 14:30 |
| @clarkb:matrix.org | the docs that we do have are far more relevant to how zuul_return is used | 14:30 |
| @mnaser:matrix.org | Just wanted to share this.. I managed to do something really cool with Molecule. We copied the Zuul inventory into the test host, and then leveraged https://ansible.readthedocs.io/projects/molecule/examples/ansible_native_inventory/ -- and.. it just worked! Also, any group-vars, vars, etc, defined in Zuul get passed through straight to Molecule | 14:31 |
| @mnaser:matrix.org | It's literally just this: https://github.com/vexxhost/zuul-jobs/blob/main/playbooks/molecule/run.yaml (can even skip the ansible_host part, we did that to speed things up) | 14:32 |
| @lecris:matrix.org | Not using sphinx apidoc. Ansible has tools to generate the sphinx documentation (I think it's bull or something along those lines?) | 14:38 |
| @lecris:matrix.org | You can then cross-link it to the relevant page for more info, but having that generated would help with searchability | 14:38 |
| @clarkb:matrix.org | do you have a link to the sphinx plugin to do that? | 14:39 |
| @lecris:matrix.org | I might have on my other laptop the precise one that I was looking into, but it might be [antsibull](https://ansible.readthedocs.io/projects/antsibull-docs/) | 14:40 |
| @lecris:matrix.org | Probably it, based on what [ansible itself is using](https://docs.ansible.com/ansible/latest/dev_guide/developing_collections_documenting.html) and I remember it generates the same structure | 14:41 |
| @clarkb:matrix.org | looks like it generates rst for sphinx and can be given specific ansible modules/plugins to generate docs for. I wonder if that generates the index info you are looking for or if that only works when you use it against an entire collection or ansible itself | 14:42 |
| @clarkb:matrix.org | and then the sphinx extension portion is simply there for managing the css for the output? | 14:43 |
| @lecris:matrix.org | Yes, but probably as page index. I did not look closely into what it provides | 14:43 |
| @lecris:matrix.org | Difference would be between a sphinx domain or page, but on the user side, it doesn't make much difference aside from intersphinx | 14:45 |
| @jim:acmegating.com | Cristian Le: i think that section is probably too small to warrant automatic generation of sphinx output via a separate program. if it had a module with a special domain for doing that, that would be different. but if you wanted to manually add some index records, that would be welcome. :) | 14:46 |
| @clarkb:matrix.org | is adding internal rst doc link targets enough to add things to the serach index? | 14:47 |
| @clarkb:matrix.org | if so then ya annotating important items may be a simple and easy solution here | 14:48 |
| @lecris:matrix.org | Doc links are already there, but they are not linked to indexable keywords | 14:48 |
| @lecris:matrix.org | Ah, wait, there are no links for `zuul_return` at all actually | 14:49 |
| @clarkb:matrix.org | right but there must be some method to tell sphinx put this in the search index | 14:49 |
| @lecris:matrix.org | I am not aware of ways to manually create an index aside from proper sphinx language domains | 14:50 |
| @clarkb:matrix.org | https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#index-generating-markup | 14:50 |
| @lecris:matrix.org | Let me double-check what exactly is in `sphinx_anstibull_ext`, maybe it already provides a language domain | 14:51 |
| @clarkb:matrix.org | I think what i linked is likely to work but that affects the navigtation index too | 14:51 |
| @lecris:matrix.org | Oh, interesting, never investigated those. Sounds relevant indeed | 14:53 |
| @lecris:matrix.org | (and, no antixbull does not provide or use proper sphinx language domains 🙁) | 14:53 |
| @lecris:matrix.org | * (and, no antsibull does not provide or use proper sphinx language domains 🙁) | 14:54 |
| @jim:acmegating.com | it would be possible to use the zuul domain to document zuul_return via zuul:attr directives, and they should be indexed. i honestly don't know if that's a good idea or if it will look like a mess. if someone wants to experiment with that, i think that would be fine -- under the understanding that it would be experimental, just so we could evaluate it, and we might not choose to merge it. | 14:59 |
| @jim:acmegating.com | i think there's a good chance that might look and work okay, and if it does, we'd probably prefer that to an external tool, for consistency. after all, the zuul vars are documented that way right above it (using different directives, but similar). | 15:01 |
| @jangutter:matrix.org | A quick question on zuul_return design that I've always wondered. Is it on purpose that it never prints anything? Would it be useful to extend it to print something like a list of keys returned? | 15:06 |
| @jim:acmegating.com | jangutter: nope, just no need at the time. i don't see why it couldn't return more data in the ansible json return structure. it couldn't add anything to the streaming console log though. | 15:10 |
| @lecris:matrix.org | Btw I should probably ask here for some design help for what I am primarily trying to do. I want to propagate a variable from a job to its post-run (to do cleanup in case the job was canceled). I am currently thinking of saving it to an artifact and searching through them, but can I do something simpler like, would it be available as a fact in the post-run | 15:12 |
| @jim:acmegating.com | Cristian Le: you should be able to cache the fact. some roles in zuul-jobs use that technique. | 15:13 |
| @jangutter:matrix.org | Cristian Le: for complicated stuff you can also save some vars to json on the executor. In post-run you can load them. | 15:14 |
| @lecris:matrix.org | Cache where/how? In an external service like jangutter mentioned? | 15:14 |
| @jim:acmegating.com | Cristian Le: i meant a cacheable ansible fact | 15:14 |
| @lecris:matrix.org | That I don't know how to do | 15:15 |
| @jim:acmegating.com | Cristian Le: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/set_fact_module.html#parameter-cacheable | 15:15 |
| @lecris:matrix.org | Ah, yes I have that, but would it survive when going to different playbooks? | 15:15 |
| @jim:acmegating.com | yep | 15:15 |
| @lecris:matrix.org | Ah great, so it was a bit over-complication what I was trying to do, thanks | 15:16 |
| @lecris:matrix.org | * Ah great, so it was "a bit" over-complication what I was trying to do, thanks | 15:16 |
| @jim:acmegating.com | Cristian Le: no problem :) | 15:16 |
| @jangutter:matrix.org | Cristian Le: Let me check if this is your scenario: you have a _single job_ where you have a playbook that creates an external resource, and you'd like to ensure that it's cleaned if the job fails? | 15:16 |
| @lecris:matrix.org | Yes, it creates a build job (Koji build for Fedora if curious) and I want to make sure I stop it | 15:17 |
| @lecris:matrix.org | Also is there a variable to check if the job was either canceled or timed-out | 15:17 |
| @jangutter:matrix.org | (the fact cache is new to me, thanks corvus !) So that technique can be used to handle 99% of failures. You should still have a periodic janitor though - something could go wrong in cleanup and you would still have slow leaks. | 15:20 |
| @jangutter:matrix.org | If possible make sure you can associate the zuul build_id with the koji build_id some way and you can see which ones have leaked. | 15:21 |
| @lecris:matrix.org | Right now it is all of them 😅. Backwards navigation is quite straightforward there though | 15:25 |
| @sdodsley:matrix.org | What would cause the error message `Will not fetch project branches as read-only is set` in the Zuul config? I am using SoftwareFactory's implementation and there doesn't seem to be anywhere to add `exclude-locked-branches: true`, which I assume would fix the issue. Anyone have any SF knowledge who can assist? | 15:30 |
| @clarkb:matrix.org | Simon Dodsley: is that the message that zuul-web emits because it isn't a scheduler? | 15:47 |
| @clarkb:matrix.org | if you see that in logs for a service other than the scheduler I'm not sure it is a problem | 15:47 |
| @clarkb:matrix.org | Simon Dodsley: I see your other message on IRC. I think the problem is that the novnc project is hosted on github and you need to add it to your tenant config. | 15:48 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 958553: Remove model backwards compat https://review.opendev.org/c/zuul/zuul/+/958553 | 16:09 | |
| @sdodsley:matrix.org | Clark: I'm seeing it in the executor log: | 16:12 |
| ``` | ||
| 2025-08-28 16:10:47,888 DEBUG zuul.Repo: Cloning from ssh://purestorage-thirdparty-ci@review.opendev.org:29418/novnc/novnc to /var/lib/zuul/executor/opendev.org/novnc/novnc%2Fnovnc | ||
| 2025-08-28 16:11:18,617 WARNING zuul.Repo: Retry 1: Clone /var/lib/zuul/executor/opendev.org/novnc/novnc%2Fnovnc | ||
| 2025-08-28 16:11:49,383 WARNING zuul.Repo: Retry 2: Clone /var/lib/zuul/executor/opendev.org/novnc/novnc%2Fnovnc | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: Unable to reset repo <Repo 0x7feee98f2650 /var/lib/zuul/executor/opendev.org/novnc/novnc%2Fnovnc> | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: Traceback (most recent call last): | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: File "/usr/local/lib/python3.11/site-packages/zuul/merger/merger.py", line 1343, in getRepoState | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: repo.update() | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: File "/usr/local/lib/python3.11/site-packages/zuul/merger/merger.py", line 732, in update | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: repo = self.createRepoObject(zuul_event_id, build=build) | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: File "/usr/local/lib/python3.11/site-packages/zuul/merger/merger.py", line 336, in createRepoObject | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: self._ensure_cloned(zuul_event_id, build=build) | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: File "/usr/local/lib/python3.11/site-packages/zuul/merger/merger.py", line 173, in _ensure_cloned | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: self._git_clone(clone_url, zuul_event_id, build=build) | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: File "/usr/local/lib/python3.11/site-packages/zuul/merger/merger.py", line 250, in _git_clone | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: mygit.clone(git.cmd.Git.polish_url(url), self.local_path, | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: File "/usr/local/lib/python3.11/site-packages/git/cmd.py", line 800, in <lambda> | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: return lambda *args, **kwargs: self._call_process(name, *args, **kwargs) | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: File "/usr/local/lib/python3.11/site-packages/git/cmd.py", line 1386, in _call_process | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: return self.execute(call, **exec_kwargs) | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: File "/usr/local/lib/python3.11/site-packages/git/cmd.py", line 1183, in execute | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: raise GitCommandError(redacted_command, status, stderr_value, stdout_value) | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: git.exc.GitCommandError: Cmd('git') failed due to: exit code(128) | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: cmdline: git clone ssh://*****@review.opendev.org:29418/novnc/novnc /var/lib/zuul/executor/opendev.org/novnc/novnc%2Fnovnc | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: stderr: 'Cloning into '/var/lib/zuul/executor/opendev.org/novnc/novnc%2Fnovnc'... | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: fatal: project novnc/novnc not found | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: fatal: Could not read from remote repository. | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: Please make sure you have the correct access rights | ||
| 2025-08-28 16:11:50,145 ERROR zuul.Merger: and the repository exists.' | ||
| ``` | ||
| @clarkb:matrix.org | Simon Dodsley: there is no opendev.org/novnc/novnc | 16:12 |
| @clarkb:matrix.org | note the message at the end "please make sure the repository exists" | 16:13 |
| @sdodsley:matrix.org | i know - i didn't tell it to use opendev.org | 16:13 |
| @sdodsley:matrix.org | so i don't know why it is adding that | 16:13 |
| @clarkb:matrix.org | did you add the repo to your tenant config? | 16:15 |
| @sdodsley:matrix.org | i don't know where that is in SoftwareFactory. It's a weird Zuul they have hacked | 16:15 |
| @jim:acmegating.com | Simon Dodsley: so the zuul answer is that you set that in your tenant config. the docs are here: https://zuul-ci.org/docs/zuul/latest/tenants.html#tenant | 16:20 |
| if software factory has some special way of configuring that, i'd ask them in one of their communication channels; this isn't the best place for s-f specific discussion. | ||
| @sdodsley:matrix.org | going to try them now | 16:20 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | 16:32 | |
| @clarkb:matrix.org | corvus: ^ that is the first bit of what we discussed yseterday around docker builds (no test chagnes yet) | 16:32 |
| @clarkb:matrix.org | corvus: I'm kinda thinking that if that works with our existing tests then the next step would be to depends-on from my opendev updates and sanity check things work there, then figure out how to enforce things with testing within zuul-jobs | 16:32 |
| @clarkb:matrix.org | mnaser: re molecule using the job inventory opendev does similar with our system-config test like production jobs with a nested ansible. | 16:33 |
| @jim:acmegating.com | Clark: that sounds like a plan. whatever those test changes are, we can also make a new change with those test changes to see that they fail (easier if we do that before merging anything to zuul-jobs) | 16:34 |
| -@gerrit:opendev.org- Zuul merged on behalf of James E. Blair https://matrix.to/#/@jim:acmegating.com: [zuul/zuul-jobs] 954885: Add OIDC support to upload-logs-s3 https://review.opendev.org/c/zuul/zuul-jobs/+/954885 | 16:48 | |
| @clarkb:matrix.org | cri-o moved their package repos and microk8s install failed too. I'll see about fixing those too I guess | 16:51 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 17:53 | |
| - [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | ||
| - [zuul/zuul-jobs] 958800: Update cri-o packge location https://review.opendev.org/c/zuul/zuul-jobs/+/958800 | ||
| -@gerrit:opendev.org- Clark Boylan proposed: | 18:25 | |
| - [zuul/zuul-jobs] 958800: Fix kubernetes install methods https://review.opendev.org/c/zuul/zuul-jobs/+/958800 | ||
| - [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | ||
| @clarkb:matrix.org | I think the initial microk8s install problem was a fluke (I even got an error from snapcraft in my browser trying to look up microk8s snaps). But fixing cri-o exposed a problem with bookworms snap installation when installing microk8s so I'm trying to fix that and cri-o now. The always use buildx change seems to be working though. I'll set up that depends on once I can get microk8s running reliably | 18:26 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 18:42 | |
| - [zuul/zuul-jobs] 958800: Fix kubernetes install methods https://review.opendev.org/c/zuul/zuul-jobs/+/958800 | ||
| - [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | ||
| @clarkb:matrix.org | https://review.opendev.org/c/zuul/zuul-jobs/+/958800 passes testing now and I think should be a quick and easy review | 19:38 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 20:18 | |
| - [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | ||
| - [zuul/zuul-jobs] 958809: Update registry tests to better cover speculative image builds https://review.opendev.org/c/zuul/zuul-jobs/+/958809 | ||
| -@gerrit:opendev.org- Clark Boylan proposed: | 20:42 | |
| - [zuul/zuul-jobs] 958809: Update registry tests to better cover speculative image builds https://review.opendev.org/c/zuul/zuul-jobs/+/958809 | ||
| - [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | ||
| -@gerrit:opendev.org- Clark Boylan proposed: | 22:06 | |
| - [zuul/zuul-jobs] 958809: Update registry tests to better cover speculative image builds https://review.opendev.org/c/zuul/zuul-jobs/+/958809 | ||
| - [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | ||
| -@gerrit:opendev.org- Clark Boylan proposed: | 22:35 | |
| - [zuul/zuul-jobs] 958809: Update registry tests to better cover speculative image builds https://review.opendev.org/c/zuul/zuul-jobs/+/958809 | ||
| - [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | ||
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul-jobs] 958783: Always build docker images with custom buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/958783 | 23:03 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 958553: Remove model backwards compat https://review.opendev.org/c/zuul/zuul/+/958553 | 23:07 | |
| @clarkb:matrix.org | I noticed on my changes that zuul was reporting that zuul-jobs still uses cleanup-run | 23:08 |
| @clarkb:matrix.org | I feel like we're trying to remove that maybe in 958553? In any case I wonder if we should try and do a cleanup for that stuff | 23:08 |
| @jim:acmegating.com | Clark: nope 553 doesn't touch cleanup-run, but we should remove it now from zuul-jobs. https://review.opendev.org/946718 or https://review.opendev.org/947041 will do that. | 23:20 |
| @jim:acmegating.com | feel free to +3 either of those :) | 23:21 |
| @clarkb:matrix.org | done | 23:22 |
| @jim:acmegating.com | Clark: an earlier patchset of 553 attempted to remove some internal cleanup_run stuff, but i had to put it back because the implementation is tied to the UX. so it's going to have to wait until we're ready to remove cleanup-run completely. which... probably shouldn't be too much longer. | 23:23 |
| @jim:acmegating.com | (that's probably what you were remembering) | 23:23 |
| @clarkb:matrix.org | that was likely it | 23:24 |
| -@gerrit:opendev.org- Zuul merged on behalf of Aurelio Jargas: [zuul/zuul-jobs] 947041: Remove Artifactory cleanup playbook https://review.opendev.org/c/zuul/zuul-jobs/+/947041 | 23:39 | |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!