| @jkt_:matrix.org | hi, I'm on a rather ancient Zuul (3.19.1), and I'm trying to come up with a proper job for building (and uploading) a tagged release. I *think* that I need a playbook which runs on a transient node for the actual build, and then a second playbook that runs on the executor only, and which will push a tarball or something to a persistent location, right? (Because I *think* that there's some magic re handling credentials for upload to object storage, isn't it) | 10:18 |
|---|---|---|
| @jkt_:matrix.org | I went through the "tag" pipeline on opendev/openstack in an attempt to shamelessly steal some ideas, but that wasn't the full picture, apparently | 10:19 |
| @fungicide:matrix.org | > <@jkt_:matrix.org> I went through the "tag" pipeline on opendev/openstack in an attempt to shamelessly steal some ideas, but that wasn't the full picture, apparently | 12:34 |
| you're looking for the release (and pre-release) pipeline jobs, but you'll likely need to rewind relevant repositories to a state contemporary with 3.19.1 (2020-07-22) | ||
| @jkt_:matrix.org | I'm looking at, e.g., https://zuul.opendev.org/t/zuul/build/d39ccac8d04146d996145835b6929752/console , and it seems that you aren't using artifacts for this, are you? It's the same job that builds stuff, and a `post-run` playbook which gathers what was built and performs a pypi upload | 13:01 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 929978: Fix console log stream delay on idempotent command https://review.opendev.org/c/zuul/zuul/+/929978 | 13:42 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 929979: Handle kubectl port-foward terminations https://review.opendev.org/c/zuul/zuul/+/929979 | 13:42 | |
| @fungicide:matrix.org | > <@jkt_:matrix.org> I'm looking at, e.g., https://zuul.opendev.org/t/zuul/build/d39ccac8d04146d996145835b6929752/console , and it seems that you aren't using artifacts for this, are you? It's the same job that builds stuff, and a `post-run` playbook which gathers what was built and performs a pypi upload | 13:56 |
| correct. the only reasons to do an artifact hand-off between separate jobs, that i can think of, is if you wanted the artifact build process to be defined in an untrusted project while keeping the upload/credentials in a trusted config project, or if you wanted an intermediate job to test the exact artifact that was produced before uploading it to pypi | ||
| @fungicide:matrix.org | in the case of the job you're looking at, the packages are built in an ephemeral virtual machine, then pulled onto the executor and signed and uploaded from the executor's workspace so that the credentials never leave the executor and won't be accessible to the system where the build process ran | 13:58 |
| @fungicide:matrix.org | > <@jkt_:matrix.org> I'm looking at, e.g., https://zuul.opendev.org/t/zuul/build/d39ccac8d04146d996145835b6929752/console , and it seems that you aren't using artifacts for this, are you? It's the same job that builds stuff, and a `post-run` playbook which gathers what was built and performs a pypi upload | 13:58 |
| * correct. the only reasons to do an artifact hand-off between separate jobs, that i can think of, is if you wanted the artifact build job to be defined in an untrusted project while keeping the upload/credentials in a trusted config project, or if you wanted an intermediate job to test the exact artifact that was produced before uploading it to pypi | ||
| @mnaser:matrix.org | Has anyone ever dabbled with the idea of letting Zuul do multiarch builds for Docker images using native builders across two different nodes in a nodeset | 14:38 |
| @mnaser:matrix.org | In my case I'm trying to save the long emulated build time, so I am thinking of adding and arm node to the nodeset, have it deploy docker, add it as a builder inside buildx, and then the rest should kinda transparently happen | 14:40 |
| @mnaser:matrix.org | since buildx/docker will just use to run the arm64 stuff, and then the rest will continue as is.. i just want to know if someone has explored this before i start working on it | 14:40 |
| @jim:acmegating.com | today that will only work if a single nodepool provider can provide both types. opendev doesn't have such a provider (to my knowledge... perhaps that's no longer the case?). so opendev hasn't done that. | 14:42 |
| @mnaser:matrix.org | Ah I see, in my case this would be for our own deployment that we use for Atmosphere (which uses GitHub) and talks to our public cloud that has both.. so I guess this might be something that would make sense upstream with caveat "make sure you have nodes to handle all that" | 14:43 |
| @jim:acmegating.com | what's the upstream component you're contemplating here? | 14:44 |
| @mnaser:matrix.org | sorry, upstream aka zuul-jobs (rather than a job we have in our own zuul-jobs) | 14:44 |
| @jim:acmegating.com | got it, yep, agree | 14:44 |
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 15:36 | |
| @mnaser:matrix.org | i'm confused, i have a PR with Depends-On on the change above, but it doesn't seem that Zuul is checking it out, it's somehow treating zuul-jobs as trusted somehow? https://logs.atmosphere.dev/31/1931/5b94fa235f54b610ebe4e5f10d0329785ce2e821/check/atmosphere-build-container-image-nova/0f71452/zuul-info/inventory.yaml | 16:34 |
| @mnaser:matrix.org | It's certainly not checked out here - https://logs.atmosphere.dev/31/1931/5b94fa235f54b610ebe4e5f10d0329785ce2e821/check/atmosphere-build-container-image-nova/0f71452/workspace-repos.json | 16:35 |
| @clarkb:matrix.org | mnaser: do you have zuul-jobs configured as a git or gerrit connection in your zuul? I suspect that you cannot depends on changes if using the git driver (but may be able to do so some other way?) | 16:36 |
| @mnaser:matrix.org | ah you're right, i am using the "git" driver pointing to https://opendev.org | 16:37 |
| @mnaser:matrix.org | I will flip it to Gerrit | 16:37 |
| @jkt_:matrix.org | fungi: thanks again | 16:50 |
| -@gerrit:opendev.org- Mohammed Naser proposed: | 17:46 | |
| - [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | ||
| - [zuul/zuul-jobs] 930939: Only update qemu-static container settings on x86_64 https://review.opendev.org/c/zuul/zuul-jobs/+/930939 | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 930942: Finish removing test_image_permissions https://review.opendev.org/c/zuul/zuul/+/930942 | 18:10 | |
| @mnaser:matrix.org | > <@clarkb:matrix.org> mnaser: do you have zuul-jobs configured as a git or gerrit connection in your zuul? I suspect that you cannot depends on changes if using the git driver (but may be able to do so some other way?) | 19:00 |
| I think my issue here is that Depends-On that I'm working on is a config repo (because it's work against the docker image build code), so since I'm "testing" it against a job that leverages the buildset registry, which comes from my config repo.. it's not accepting it | ||
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 19:12 | |
| @fungicide:matrix.org | > <@mnaser:matrix.org> I think my issue here is that Depends-On that I'm working on is a config repo (because it's work against the docker image build code), so since I'm "testing" it against a job that leverages the buildset registry, which comes from my config repo.. it's not accepting it | 19:30 |
| zuul safely refuses to use speculative states of trusted config repos, if that's what's going on | ||
| @fungicide:matrix.org | anything less could leak sensitive secrets | 19:31 |
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 19:51 | |
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 21:32 | |
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 21:39 | |
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 22:06 | |
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 22:12 | |
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: WIP: Append other builds from nodeset https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 22:34 | |
| -@gerrit:opendev.org- Mohammed Naser proposed: [zuul/zuul-jobs] 930927: Add other nodes to buildx builder https://review.opendev.org/c/zuul/zuul-jobs/+/930927 | 22:58 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!