| -@gerrit:opendev.org- Christian Mueller proposed wip: [zuul/nodepool] 869015: adding a testing container, which makes it easier to run unit tests locally in a container https://review.opendev.org/c/zuul/nodepool/+/869015 | 11:58 | |
| -@gerrit:opendev.org- Christian Mueller proposed: [zuul/nodepool] 873552: WIP: adding a testing container, which makes it easier to run unit tests locally in a container https://review.opendev.org/c/zuul/nodepool/+/873552 | 11:58 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 873037: Add OpenStack volume quota https://review.opendev.org/c/zuul/nodepool/+/873037 | 14:56 | |
| @noonedeadpunk:matrix.org | Hey there. Having quite stupid question (basic one), but can't really understand what would be the best approach. Basically what I'm trying to do - upload articats with role common to that https://paste.openstack.org/show/bfarFFkDVXLjI1CJ8pEY/ | 15:24 |
|---|---|---|
| @noonedeadpunk:matrix.org | The thing is, that I want to role to be re-usable and have same swift credentials as used for logs upload, so I've placed it to base | 15:24 |
| @noonedeadpunk:matrix.org | Now what I can't understand - is how to define `zuul_artifact_file` except as for job? | 15:25 |
| @noonedeadpunk:matrix.org | As I wanted file to contain timestamp and also pass checksum | 15:25 |
| @noonedeadpunk:matrix.org | (in paste we're calculating sha but it's because I couldn't find a way to pass it) | 15:26 |
| @clarkb:matrix.org | Dmitriy Rabotyagov: I think you can make it a cacheable fact? | 16:19 |
| @clarkb:matrix.org | corvus: yesterday openstack users pushed changes to jobs that increased the job timeout to a value larger than our default max of 10800 seconds. Zuul reported this back as an unknown configuration error. I haven't pulled up tracebacks yet (I didn't have keys loaded yesterday), but is that a known issue with config error reporting? | 16:19 |
| @jim:acmegating.com | Clark: as of yesterday? shouldn't be. | 16:20 |
| @noonedeadpunk:matrix.org | Clark: I kind of wonder when it has become a thing... As seems it's not working for us for some reason - I was about to blame usage of trusted repo for defining publish artifact job | 16:21 |
| @noonedeadpunk:matrix.org | We're running 6.1.0 fwiw | 16:22 |
| @clarkb:matrix.org | corvus: https://review.opendev.org/c/openstack/tempest/+/873472/2/zuul.d/integrated-gate.yaml is the file that was a problem. Once I've had tea and can load ssh keys I'll try to pull up logs | 16:22 |
| @noonedeadpunk:matrix.org | So for now my only assumtion I have is that uplaod artifact should not be define in trusted repo, but then if we want to store artifacts and logs in same swift/s3 - this means we should manage credentials independently, as publish logs for sure should be in base... | 16:25 |
| @clarkb:matrix.org | Dmitriy Rabotyagov: have you looked at examples in opendev? | 16:26 |
| @noonedeadpunk:matrix.org | Well, I've tried to trace back how docs work, but maybe you have better example ? :) | 16:27 |
| @clarkb:matrix.org | not off the top of my head, but I know there are similar use cases in there so it can be made to work I think | 16:27 |
| @noonedeadpunk:matrix.org | I just suspect that docs job simply defined in trusted repo... At least publish job is there. | 16:33 |
| @noonedeadpunk:matrix.org | And what we wanted to achive is have artifact upload as part of base post-run, that is run conditionally whether {{ zuul_artifact_file }} is defined or not... But if job, that tries to define zuul_artifact_file is not in trusted repo (while artifact upload is) - then it seems it can't use cached facts? | 16:35 |
| @clarkb:matrix.org | that may be, I don't know off the top of my head. | 16:49 |
| @fungicide:matrix.org | i have a vague recollection of security measures around fact caching, though can't find any obvious references by searching the zuul docs | 16:59 |
| @fungicide:matrix.org | there is some fact filtering that goes on in zuul.executor.server | 17:02 |
| @clarkb:matrix.org | swest: if you have time for the sqla stack that would be great: starts with https://review.opendev.org/c/zuul/zuul/+/872034. The first two changes should be very low risk as its just small modifications to be sqla 2.0 safe. Then the swap potentially changes sqla behavior, but our testing indicates we seem to be fine with it too | 18:10 |
| @bridgefan:matrix.org | Would anyone here know if Zuul supports a way to send emails for failed jobs with an email list that is different per job rather than per pipeline (I know how to do the per pipeline version)? | 18:20 |
| @clarkb:matrix.org | > <@bridgefan:matrix.org> Would anyone here know if Zuul supports a way to send emails for failed jobs with an email list that is different per job rather than per pipeline (I know how to do the per pipeline version)? | 18:26 |
| I think the only way to do that today is to have the job send the email itself. | ||
| @bridgefan:matrix.org | @Clark Ok thanks. | 18:33 |
| @jpew:matrix.org | bridgefan: A little convoluted, but we have zuul setup to report to prometheus monitor and have alert rules based on that | 18:49 |
| @bridgefan:matrix.org | jpew: That's an interesting idea. We may have to do something similar. | 18:55 |
| @fungicide:matrix.org | we have jobs which send e-mail, but those are mainly automated release announcements for projects' tag jobs | 19:58 |
| @bridgefan:matrix.org | Are you saying there is a base zuul job for these emails? | 20:03 |
| @fungicide:matrix.org | no, the job just runs this shell script which creates the text of the message and pumps it through sendmail: https://opendev.org/openstack/releases/src/branch/master/tools/announce.sh | 20:10 |
| @jim:acmegating.com | a reusable role in zuul-jobs that sends email though is a great idea | 20:11 |
| @fungicide:matrix.org | i completely agree. happy to review if someone writes it, though in our case the nuance is that sending e-mail from test nodes mainly works because we control the mailserver it's sending to. typical users of zuul are likely to want a smarthost of some kind, i expect | 20:12 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 873645: Fix configuration error related to YAML anchors https://review.opendev.org/c/zuul/zuul/+/873645 | 20:14 | |
| @jim:acmegating.com | fungi: agree, that role should probably have a smarthost option | 20:15 |
| @jim:acmegating.com | Clark: 873645 | 20:15 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 873645: Fix configuration error related to YAML anchors https://review.opendev.org/c/zuul/zuul/+/873645 | 20:19 | |
| @clarkb:matrix.org | aha its the source side object being reused in the consumption side. When we freeze the source side the consumption side sees that as frozen too | 20:32 |
| @clarkb:matrix.org | corvus: https://review.opendev.org/c/zuul/zuul/+/873049 is an easy ResourceWarnings cleanup change | 21:11 |
| @aspiers:matrix.org | Hey old OpenStack friends, long time and nice to see a lot of familiar names on here! I'm quite far away from the OpenInfra world these days - working on using blockchain for climate impact, which despite many excellent qualities has dragged me back to the murky world of using GitHub's godawful code review and CI systems. I've been longing for the goodness of Gerrit plus Zuul for CI and policy-based merges, but ideally without having to rewrite our GitHub CI workflows from scratch or lose branch protection on GitHub. Is this realistic? | 21:25 |
| @aspiers:matrix.org | I find it incredible what a hugely negative impact GitHub's design decision of allowing multiple commits per review cycle has on the quality of the whole development process. One glaring error is that CI checks are only run on the tip of the PR branch, which allows devs to be sloppy in how they construct commits. When you combine that with the fact that there is no sensible way to stack PRs on top of each other, it actively encourages sloppiness. | 21:29 |
| @aspiers:matrix.org | In comparison, being able to tweak a commit buried deep in a series, then type `git review` and have everything update perfectly on the review server is pure magic which I almost can't believe I used to take for granted :-) | 21:31 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 873651: Fix race in metastatic allocation https://review.opendev.org/c/zuul/nodepool/+/873651 | 21:31 | |
| @aspiers:matrix.org | Even if this is achievable, we currently face the challenge of having not yet open sourced our repos, so I guess we'd need to host our own Gerrit and Zuul for now. | 21:32 |
| @clarkb:matrix.org | aspiers: Zuul can integrate with GitHub. Zuul can do the merging for you allowing you to gate things and keep branch protections if I've understood how oethers are using zuul and github correctly. | 21:33 |
| @aspiers:matrix.org | I did find https://zuul-ci.org/docs/zuul/latest/drivers/github.html#configure-github | 21:33 |
| @clarkb:matrix.org | additionally it can integrate with github and gerrit at the same time allowing you to migrate at your own pace if you go down that path | 21:33 |
| @aspiers:matrix.org | I noticed it said "GitHub Actions are still in Beta" but wasn't sure what the caveats were. | 21:34 |
| @clarkb:matrix.org | Also depends-on allows you to effectively stack PRs if you use zuul | 21:34 |
| @clarkb:matrix.org | > <@aspiers:matrix.org> I noticed it said "GitHub Actions are still in Beta" but wasn't sure what the caveats were. | 21:35 |
| The caveats are in the paragraph above that line. Its only an issue if you use github actions and would like zuul to merge changes that affect the workflows | ||
| @aspiers:matrix.org | Ah OK, we can live without that - changes to workflows could go through the normal PR process | 21:36 |
| @clarkb:matrix.org | But zuul can't change user habits and the ability to sneak junk commits in | 21:37 |
| @clarkb:matrix.org | you can write jobs that inspect each commit in a PR separately and error if they don't meet some criteria | 21:37 |
| @aspiers:matrix.org | Yes exactly. Honestly just having the CI run on every commit is already an order of magnitude improvement. | 21:38 |
| @aspiers:matrix.org | It's incredible how often devs submit PRs which break something in one commit and fix it later in the same PR. | 21:38 |
| @aspiers:matrix.org | Also being able to actually review and comment on git commit messages is another luxury I can't believe I used to take for granted. | 21:39 |
| @jim:acmegating.com | aspiers: note that zuul tests github PRs, not individual commits, so the only way to avoid that would be either policy or Clark's suggestion of having a job examine individual commits (possibly to error if a PR has more than one commit) if you want to go down that route | 21:40 |
| @jim:acmegating.com | mostly just wanted to clarify that the "test every commit" behavior is due to gerrit, not zuul -- zuul tests what the source system gives it. | 21:40 |
| @aspiers:matrix.org | Ahhh - to be clear, I *really* need Gerrit's code review workflow, not GitHub PRs. In an ideal world, there'd be a way of synchronising the two, so that every PR contained exactly one new commit to be reviewed, preceded by the contiguous sequence of commits it depends on (already contained in other commits). And in this ideal world, depends-on could enforce the ordering, so that a PR could only ever be merged if it had a single new commit on top of the mainline. But while waiting for that utopia, I'd gladly ditch PRs altogether in favour of Gerrit reviews. | 21:45 |
| @jim:acmegating.com | aspiers: then Clark's suggestion of starting to use zuul to help migrate to gerrit could help with that. also, look into http://gerrithub.io/ from https://www.gerritforge.com/ in case that's useful | 21:48 |
| @aspiers:matrix.org | I would definitely use gerrithub if we were already open-sourced. That may take a while to resolve though. | 21:48 |
| @jim:acmegating.com | they may have a private offering, i don't know. worth an ask. :) | 21:49 |
| @aspiers:matrix.org | I emailed Luca about it last May | 21:49 |
| @aspiers:matrix.org | No response, I could try again but their business is designed entirely for large-scale enterprises, not tiny startups. | 21:50 |
| @aspiers:matrix.org | (They do have a private offering - that's their main business) | 21:50 |
| @jim:acmegating.com | i'd try again; they're usually pretty responsive | 21:50 |
| @aspiers:matrix.org | I think gerrithub is a loss-leader / promo tool | 21:50 |
| @aspiers:matrix.org | OK, just pinged him again | 21:52 |
| @jim:acmegating.com | aspiers: i also DM'd you with some info about my commercial offerings | 21:53 |
| @aspiers:matrix.org | Oh cool thanks! | 21:53 |
| @aspiers:matrix.org | BTW you guys might be interested in a comparison spreadsheet I did on ethercalc during my OpenStack days - now resurrected here: https://docs.google.com/spreadsheets/d/16X2qgqlv_TVaA4wjhfwTyC_MatO_dB-doMpUaii97iA/edit?usp=sharing | 21:58 |
| @fungicide:matrix.org | aspiers: oh, great to see you again! i didn't really have anything to add other than that, but it really is. and zuul is as good an excuse as any to be back in touch with an openinfra project (the best excuse, if you ask me!) | 22:27 |
| @aspiers:matrix.org | 😁 Great to see you too! These days I realise how ridiculously lucky I was to work with such talented folks in Open{Stack,Infra,Dev} (not that my current world isn't also awesome and full of super smart folks, but in a different way!) | 22:29 |
| @aspiers:matrix.org | It would make me deliriously happy to get our project up and running with Zuul and Gerrit. If it also funded a bit of development making it easier for other folks to achieve the same, and a success story to boot - so much the better 😎 | 22:31 |
| @clarkb:matrix.org | I've just noticed the tenant selector drop down in the new web ui | 22:42 |
| @fungicide:matrix.org | corvus: consistent failures of three pragma tests in the py38 and py311 jobs on 873645, coincidence? | 22:44 |
| @jim:acmegating.com | Clark: it's handy | 22:44 |
| @jim:acmegating.com | fungi: very likely not! | 22:44 |
| @fungicide:matrix.org | now i'm starting to wonder if the deepcopy is exposing a new problem in configuration handling | 22:44 |
| @jim:acmegating.com | i just switched to nodepool testing, so i'll have to take a look in a few mins | 22:45 |
| @fungicide:matrix.org | not urgent, just happened to notice after i wastefully rechecked thinking it was unrelated to the change (still might be) | 22:45 |
| @fungicide:matrix.org | i'm wading through the logs from those but also distracted by a few other things i'm trying to finish | 22:50 |
| @jim:acmegating.com | fungi: yeah it's a flaw in the change | 22:54 |
| @fungicide:matrix.org | glad you can spot it, because i got a little lost ;) | 22:55 |
| @jim:acmegating.com | we do rely on mutating the source context... so we need to deepcopy *except* for that | 22:57 |
| @clarkb:matrix.org | When reviewing the change I was wodnering ifwe could convince the yaml layer to dedup instead | 22:57 |
| @clarkb:matrix.org | and flatten out all the anchor stuff then we wouldn't even need to worry in job parsing | 22:58 |
| @clarkb:matrix.org | but I spent a few minutes in pyyaml before deciding that wasn't already implemented and would require us to overload a loader probably | 22:58 |
| @fungicide:matrix.org | fwiw, loader subclassing isn't all that terrible. i've done it to catch duplicate dict keys, for example | 23:00 |
| @fungicide:matrix.org | but still better avoided, yes | 23:01 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 873645: Fix configuration error related to YAML anchors https://review.opendev.org/c/zuul/zuul/+/873645 | 23:01 | |
| @jim:acmegating.com | Clark: yeah i came to the same conclusion | 23:02 |
| @jim:acmegating.com | fungi: we actually do subclass the loader, in part to catch duplicate dict keys | 23:02 |
| @jim:acmegating.com | fungi: but afaict, the loader doesn't get called a second time when resolving an anchor | 23:02 |
| @jim:acmegating.com | fungi: https://opendev.org/zuul/zuul/src/branch/master/zuul/configloader.py#L377-L406 | 23:03 |
| @fungicide:matrix.org | ah, okay that makes sense | 23:03 |
| @fungicide:matrix.org | indeed, i'd forgotten it was even in there | 23:05 |
| @jim:acmegating.com | fungi: yeah me too! also, it's not in the same file as the *other* two subclassed loaders! https://opendev.org/zuul/zuul/src/branch/master/zuul/lib/yamlutil.py :) | 23:06 |
| @jim:acmegating.com | (it's possible zuul's yaml routines have grown a bit sophisticated) | 23:06 |
| @fungicide:matrix.org | does seem like there could be an opportunity to converge some of that | 23:07 |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 873655: Use importlib for versioning https://review.opendev.org/c/zuul/nodepool/+/873655 | 23:24 | |
| @iwienand:matrix.org | > <@aspiers:matrix.org> It would make me deliriously happy to get our project up and running with Zuul and Gerrit. If it also funded a bit of development making it easier for other folks to achieve the same, and a success story to boot - so much the better 😎 | 23:50 |
| also don't forget about https://www.softwarefactory-project.io/ if you need something a bit more turn-key (as much as an entire development pipeline can be turn-key, anyway :) | ||
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/zuul] 873656: Add PBR_VERSION argument to Dockerfile https://review.opendev.org/c/zuul/zuul/+/873656 | 23:52 | |
| -@gerrit:opendev.org- James E. Blair https://matrix.to/#/@jim:acmegating.com proposed: [zuul/nodepool] 873657: Add PBR_VERSION argument to Dockerfile https://review.opendev.org/c/zuul/nodepool/+/873657 | 23:52 | |
| @aspiers:matrix.org | > <@iwienand:matrix.org> also don't forget about https://www.softwarefactory-project.io/ if you need something a bit more turn-key (as much as an entire development pipeline can be turn-key, anyway :) | 23:52 |
| Thanks for the reminder, hadn't considered that. Does it have things like artifact caching available out of the box | ||
| @aspiers:matrix.org | > <@iwienand:matrix.org> also don't forget about https://www.softwarefactory-project.io/ if you need something a bit more turn-key (as much as an entire development pipeline can be turn-key, anyway :) | 23:52 |
| * Thanks for the reminder, hadn't considered that. Does it have things like artifact caching available out of the box? | ||
| @iwienand:matrix.org | aspiers: not quite sure what you mean by artifact caching -- you'd have to upload your artifacts from jobs to swift/s3/whatever, but then zuul knows how to pull them? | 23:56 |
| @aspiers:matrix.org | I mean like caching the results of a `yarn install` or similar during one CI run, and then retrieving it for the next | 23:56 |
| @aspiers:matrix.org | corvus told me that artifactory can support this, but it requires a separate install | 23:56 |
| @aspiers:matrix.org | basically looking for an equivalent of https://github.com/actions/cache (and possibly other popular GitHub Actions) | 23:57 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!