| -@gerrit:opendev.org- Clark Boylan proposed: | 00:10 | |
| - [zuul/zuul] 873043: Fix ResourceWarnings in inventory testing https://review.opendev.org/c/zuul/zuul/+/873043 | ||
| - [zuul/zuul] 873044: Fix ResourceWarnings in fingergw https://review.opendev.org/c/zuul/zuul/+/873044 | ||
| - [zuul/zuul] 873045: Fix ResourceWarnings in the executor server https://review.opendev.org/c/zuul/zuul/+/873045 | ||
| @clarkb:matrix.org | These resource warnings are great. I think they've identified several bugs now (all minor to be fair but ya) | 00:10 |
|---|---|---|
| @clarkb:matrix.org | I'm really hoping we'll eventually reach a point where having warnings pop up in unittesting is something that we treat as an error. However, cherrypy and GitPython may make this difficult | 00:11 |
| -@gerrit:opendev.org- Clark Boylan proposed: | 00:17 | |
| - [zuul/zuul] 873043: Fix ResourceWarnings in inventory testing https://review.opendev.org/c/zuul/zuul/+/873043 | ||
| - [zuul/zuul] 873044: Fix ResourceWarnings in fingergw https://review.opendev.org/c/zuul/zuul/+/873044 | ||
| - [zuul/zuul] 873045: Fix ResourceWarnings in the executor server https://review.opendev.org/c/zuul/zuul/+/873045 | ||
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 873047: Fix DeprecationWarning: ssl.PROTOCOL_TLS is deprecated https://review.opendev.org/c/zuul/zuul/+/873047 | 00:40 | |
| @clarkb:matrix.org | corvus: https://review.opendev.org/c/zuul/zuul/+/873043 could use a rereview since I found an extra case for that one and updated teh change after your first +2 | 00:47 |
| @jim:acmegating.com | done thx! | 00:48 |
| @clarkb:matrix.org | The first change that fixes the statsd thing takes us from ~3958 to ~268 ResourceWarnings | 00:53 |
| @clarkb:matrix.org | * The first change that fixes the statsd thing takes us from ~3958 to ~268 ResourceWarnings during py311 testing | 00:53 |
| @clarkb:matrix.org | Some of this is actually racy depending on what cherrypy and gitpython do if I've read some of the warnings properly | 00:53 |
| @clarkb:matrix.org | so exact numbers may differ. | 00:54 |
| @iwienand:matrix.org | any thoughts on why zuul doesn't seem to want to run tests against https://review.opendev.org/c/zuul/zuul-jobs/+/872806? | 01:04 |
| @clarkb:matrix.org | its based on an old unmerged parent | 01:09 |
| @clarkb:matrix.org | parent has a newer merged ps | 01:09 |
| @iwienand:matrix.org | yeah, just realised that | 01:10 |
| @iwienand:matrix.org | ```2023-02-08 01:01:53,260 DEBUG zuul.Pipeline.openstack.check: [e: 0d00d5f349c14104bf069b1d0dd452e2] Change <Change 0x7fa9a7583010 zuul/zuul-jobs 872375,8> does not match pipeline requirement <GerritRefFilter connection_name: gerrit open: True current-patchset: True> because False``` | 01:10 |
| -@gerrit:opendev.org- Ian Wienand proposed: [zuul/zuul-jobs] 872806: build-docker-image: further cleanup buildx path https://review.opendev.org/c/zuul/zuul-jobs/+/872806 | 01:10 | |
| @iwienand:matrix.org | i'm really not sure what happened there | 01:11 |
| @iwienand:matrix.org | maybe the rebase failed in git review? | 01:12 |
| -@gerrit:opendev.org- Clark Boylan proposed: [zuul/zuul] 873049: Fix more file opening ResourceWarnings https://review.opendev.org/c/zuul/zuul/+/873049 | 01:13 | |
| @clarkb:matrix.org | ianw: git review only keeps the rebase if it would fail (and asks you to fix it and rerun) | 01:13 |
| @clarkb:matrix.org | ianw: this is probably a deficiency in git review's rebase checking where it you rebase cleaning against the target branch it rolls back and doesn't detaect the rebase is necessary due to gerrit parent/child relationships | 01:14 |
| @clarkb:matrix.org | Or maybe you rebased prior to the chnge merging and it kept the od parent in the rebase | 01:14 |
| @iwienand:matrix.org | i dunno what I did :) too many patches in that series | 01:21 |
| -@gerrit:opendev.org- Zuul merged on behalf of Matthieu Huin https://matrix.to/#/@mhuin:matrix.org: [zuul/zuul] 810955: GUI: Add tenant dropdown to top menu https://review.opendev.org/c/zuul/zuul/+/810955 | 03:54 | |
| @michael_kelly_anet:matrix.org | Hey folks, any chances of getting some reviews on https://review.opendev.org/q/owner:mkelly+project:zuul/zuul-jobs+status:open ? | 04:59 |
| @michael_kelly_anet:matrix.org | They've been sitting there for a few weeks | 04:59 |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 872519: Replace use of ProjectContext with SourceContext https://review.opendev.org/c/zuul/zuul/+/872519 | 06:22 | |
| -@gerrit:opendev.org- Simon Westphahl proposed: [zuul/zuul] 872519: Replace use of ProjectContext with SourceContext https://review.opendev.org/c/zuul/zuul/+/872519 | 06:47 | |
| -@gerrit:opendev.org- Benedikt Löffler proposed: [zuul/nodepool] 802255: Add option for an upload script https://review.opendev.org/c/zuul/nodepool/+/802255 | 10:12 | |
| -@gerrit:opendev.org- Benedikt Löffler proposed: [zuul/nodepool] 802255: Add option for an upload script https://review.opendev.org/c/zuul/nodepool/+/802255 | 11:56 | |
| @mhuin:matrix.org | For those of you who run unit tests locally, how long does it take to complete the run, out of curiosity? I'm using a 4 cores, 8GB RAM Ubuntu 22.04 VM and going through the test suite takes at least 2 hours | 13:19 |
| @mhuin:matrix.org | Also, nice to see https://github.com/python-zk/kazoo/pull/706 merged, I won't miss the SSL warnings | 13:26 |
| @shoragan:matrix.org | Does Zuul support something like bors' rollups? https://internals.rust-lang.org/t/batched-merge-rollup-feature-has-landed-on-bors/1019 | 13:43 |
| @jim:acmegating.com | shoragan: no, but it does test in parallel for rapid merging of changes while each is still tested individually: https://zuul-ci.org/docs/zuul/latest/gating.html#testing-in-parallel | 14:23 |
| @jim:acmegating.com | mhu: more cores are better. also, py 3.10+ is faster. i've seen 20 cores do it in 11 minutes. the opendev vms take a bit under an hour. | 14:26 |
| @jim:acmegating.com | shoragan: or https://zuul-ci.org/media/simulation.webm if you're more of a visual person (that's the 2nd video on https://zuul-ci.org/ ) | 14:32 |
| @jpew:matrix.org | corvus: The use case shoragan and I are looking at is patch mailing lists where the number of changes is high and the amount of time to test is long; it's not feasible to test every patch | 14:33 |
| @jpew:matrix.org | So we manually group patches into groups of probably 10-30(?) and manually have CI build them, verify, then fast-forward the branch.... looking at alternatives though | 14:34 |
| @mhuin:matrix.org | > <@jim:acmegating.com> mhu: more cores are better. also, py 3.10+ is faster. i've seen 20 cores do it in 11 minutes. the opendev vms take a bit under an hour. | 14:37 |
| alright good to know, gotta get a better workstation at next PC refresh then! | ||
| @shoragan:matrix.org | corvus jpew: the rust devs are using rollups only for "simple" commits, for which the change of failure is low. in that scenario, they are saving a lot of build/test CPU time without much risk. | 14:57 |
| @shoragan:matrix.org | The current Yocto/openembedded-core CI builds and tests take several hours on fast servers, so testing each change individually by running them in parallel would need significantly more resources. :) | 14:59 |
| @jim:acmegating.com | for historical context, the zuul authors have typically designed around the idea that cpu time is cheaper than developer time, and worth every penny. not saying that's true in all situations, but that's the idea. sorry i have to run now. | 15:00 |
| @shoragan:matrix.org | corvus: thanks :) | 15:00 |
| @jpew:matrix.org | I wonder if we could have 2 gate pipelines or something that shared the same queue (or maybe just a way of choosing which jobs run besides `job.files`) | 15:08 |
| @fungicide:matrix.org | jpew: part of it is that zuul was designed originally for open source projects which allow patches to be proposed by anyone, so the maintainers set rules for how to decide what amount of testing is performed on a patch. letting the proposer of the patch decide how much testing that patch requires would mean one more thing reviewers have to look out for. maybe a knob the approver can tweak to say this patch needs only minimal testing would be sufficient | 15:13 |
| @fungicide:matrix.org | another part is that zuul is designed to ensure that the branch tip is always in good shape, so batching up changes means you can get into temporarily broken states on the branch if one commit fixes a bug in another and they both are in the same batch | 15:14 |
| @jpew:matrix.org | fungi: Ya, it's the maintainer that decided the amount of testing, not the submitter | 15:14 |
| @jpew:matrix.org | fungi: Ya, that can happen too | 15:15 |
| @fungicide:matrix.org | oh, now i see in the bors rollup explanation it mentions pull request comments (i misread it as the pull request description). i guess you tell it to ignore comments from non-maintainers | 15:16 |
| @fungicide:matrix.org | anyway, the "skip most testing for these changes" model can be simulated by having one change which removes jobs from the pipeline and another which adds them back. obviously that would be a poor workflow because it means lots of churn for the zuul config, but the end result (you're turning off testing for lots of patches and then pushing back on whoever proposed the last patch in the set) is more or less the same | 15:18 |
| @fungicide:matrix.org | i wonder if the gerrit "submit whole topic" support is already partway there? we don't use that option in opendev so i haven't analyzed the implementation all that closely | 15:20 |
| @shoragan:matrix.org | fungi: bors is not skipping any tests for the rollup, but it's testing several PRs together in one CI run | 16:18 |
| @fungicide:matrix.org | right, it's skipping testing for some PRs, and then assuming that the state it tested for the last one is representative of the ones queued ahead of it (in zuul terms) | 16:19 |
| @shoragan:matrix.org | fungi: yes | 16:20 |
| @fungicide:matrix.org | so if it batches up 5 PRs and the second one introduces a regression which the fourth one solves, then it won't know that | 16:20 |
| @fungicide:matrix.org | some projects don't care that every commit has been tested to work as merged, so for them it's probably an acceptable tradeoff | 16:20 |
| @shoragan:matrix.org | for PRs consisting of multiple commits, does zuul check each individual commit? | 16:21 |
| @shoragan:matrix.org | > <@fungicide:matrix.org> some projects don't care that every commit has been tested to work as merged, so for them it's probably an acceptable tradeoff | 16:22 |
| Yocto/OpenEmbedded already does this manually (by testing a manually updated 'master-next' branch in CI) | ||
| @fungicide:matrix.org | zuul combines the commits because it considers a pull request from the github driver (or a merge request from the gitlab driver) to be one unit of change | 16:22 |
| @fungicide:matrix.org | for those of us using gerrit, a change is represented by a single commit (like squashing in githubland) | 16:22 |
| @fungicide:matrix.org | gerrit's workflow encourages revising patches rather than piling fixes on top of bad commits | 16:23 |
| @fungicide:matrix.org | so more like the lkml workflow | 16:23 |
| @shoragan:matrix.org | still in the kernel workflow, one patch series is usually reviewed as a group of semantically related changes | 16:23 |
| @shoragan:matrix.org | (same as in Yocto) | 16:24 |
| @shoragan:matrix.org | so a PR with fixes for problems introduced in earlier commits would be rejected during code review | 16:24 |
| @shoragan:matrix.org | at least that's the way it works now :) | 16:24 |
| @fungicide:matrix.org | yes, in gerrit you usually have a series of related changes with git parent/child relationships, though since zuul is also meant for cross-repo work that can include depends-on relationships between commits in different repos and shared gerrit topics too | 16:27 |
| @shoragan:matrix.org | > <@fungicide:matrix.org> right, it's skipping testing for some PRs, and then assuming that the state it tested for the last one is representative of the ones queued ahead of it (in zuul terms) | 16:27 |
| Is there a way to configure this behavior in Zuul already? | ||
| @fungicide:matrix.org | but the changes are still reviewed and revised individually, even if within the context of the larger series | 16:27 |
| @fungicide:matrix.org | > <@shoragan:matrix.org> Is there a way to configure this behavior in Zuul already? | 16:28 |
| other than what i mentioned earlier (removing/replacing jobs in the pipeline with noop in a change and then reverting that in another change), i don't believe so currently | ||
| @shoragan:matrix.org | ok | 16:28 |
| @clarkb:matrix.org | did same topic stuff get ruled out? | 16:34 |
| @clarkb:matrix.org | it dedups jobs | 16:34 |
| @clarkb:matrix.org | maybe not exactly what is being looked for, but does cut down on test resource needs | 16:34 |
| @jpew:matrix.org | Clark: It might work, but Gerrit is highly unlikely to be an option in this case :) | 16:35 |
| @clarkb:matrix.org | I thought the github connection can emulate the behavior | 16:36 |
| @clarkb:matrix.org | via depends on cycles? | 16:36 |
| @clarkb:matrix.org | but I don't use github, cycles, or dedup'd jobs so can't say for certain | 16:36 |
| @jpew:matrix.org | Ya, it might be worth looking into | 16:37 |
| @clarkb:matrix.org | this is just me talking out loud here: another option might be merge commits on branches zuul pays attention to and changes to other branches zuul ignores (but again I don't do anything like this so can't see definitively if it would work) | 16:47 |
| @jpew:matrix.org | Ya, that's what I was thinking, assemble commits on a branch zuul ignores, then gate a merge it to a branch it does care about | 17:07 |
| @morucci:matrix.org | Hi, we are using the aws nodepool driver and we were running the version 5.0.0 since a while. Today we bumped our nodepool to 8.0.0 but we got trouble with the cloud-images.username attribute which is now mandatory. It was not with 5.0.0 (https://opendev.org/zuul/nodepool/src/tag/5.0.0/nodepool/driver/aws/config.py#L180) and it worked fine. What is it needed with the new version ? | 18:00 |
| @morucci:matrix.org | https://zuul-ci.org/docs/nodepool/latest/aws.html#attr-providers.[aws].cloud-images.username here in the doc it is not set as mandatory | 18:01 |
| @morucci:matrix.org | But in the code it is https://opendev.org/zuul/nodepool/src/branch/master/nodepool/driver/aws/config.py#L35 | 18:02 |
| @clarkb:matrix.org | fbo: I think zuul will always default to the `zuul` username | 18:04 |
| @clarkb:matrix.org | but you want the zuul side and the nodepool side to match for logins | 18:04 |
| @morucci:matrix.org | Thanks Clark then we'll put the same value both side | 18:07 |
| @clarkb:matrix.org | I guess if you are using amazon provided images then your username might be `ubuntu`? | 18:08 |
| @clarkb:matrix.org | * I guess if you are using amazon provided images then your username might be `ubuntu` and so on? | 18:08 |
| @clarkb:matrix.org | and now that I think about it more that attribute is probably there so that nodepool can tell zuul what to connect as | 18:08 |
| @morucci:matrix.org | Clark: we are using fedora cloud images already provisioned on aws so we'll set "fedora" as username. | 18:19 |
| @jim:acmegating.com | the drivers have been inconsistent about this, but are converging on always requiring username for cloud images. i think that's the right way to go since it's almost always required and there is no sensible default. we should update the docs and this should have had a release note. it just slipped through, sorry. | 18:21 |
| @morucci:matrix.org | corvus: thanks for the info, we'll set the username | 18:42 |
| @morucci:matrix.org | corvus: however we are experiencing that issue with really last Zuul version https://softwarefactory-project.io/paste/show/bFnmhzm5EZ0IP8d3KGSL/ | 18:43 |
| @morucci:matrix.org | AttributeError: 'ProjectContext' object has no attribute 'serialize' | 18:43 |
| @morucci:matrix.org | Is it a known issue ? | 18:43 |
| @jim:acmegating.com | fbo yes, fixed in https://review.opendev.org/872519 | 18:51 |
| @morucci:matrix.org | thanks | 19:37 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 873169: Test job variants when override-checkout is a tag https://review.opendev.org/c/zuul/zuul/+/873169 | 19:56 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 873210: DNM: Verify the direction of time https://review.opendev.org/c/zuul/nodepool/+/873210 | 22:03 | |
| @mordred:inaugust.com | corvus: I don't know exactly what's going on ... but it's good to see you're using Zuul to validate the space-time continuum ^^ | 22:27 |
| @jim:acmegating.com | mordred: yeah, i mean, i'm working a problem where either python is returning an incorrect value, or space-time causality is broken! place your bets! :) | 22:31 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!