Tuesday, 2023-05-02

fungihas ansible-lint exploded on us? https://zuul.opendev.org/t/openstack/build/a9f991b05eac4a37aab3e98f5b01db8e01:25
opendevreviewIan Wienand proposed opendev/system-config master: tools/atc: remove  https://review.opendev.org/c/opendev/system-config/+/88193702:04
ianwfungi: yeah just looking at that.  it's a bit weird, as it doesn't look like much has changed02:07
ianw^^ is also fixed by https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/88193802:08
ianwi am now getting the same error as the gate locally, so that's something02:08
ianwit must be https://pypi.org/project/ansible-compat/#history02:10
ianw"This release has been yanked"02:10
ianwwhy does pypi even bother supporting this :/02:10
ianwyep, 3.0.2 v 4.0.102:11
ianwfiled https://github.com/ansible/ansible-compat/issues/258 but looking at versions02:19
ianwthat bit doesn't look safe -- ansible-lint calls get_cache_dir and is passing in a string, not Path object02:32
fungilove the pr description... "No description provided."02:34
fungithe commit message isn't a lot better02:35
ianwfungi: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/881938 should avoid it, and i've made a pull request which may or may not be right02:53
fungithanks! looks like it worked02:58
opendevreviewMarcin Juszkiewicz proposed opendev/system-config master: reprepro: add Debian Bookworm config  https://review.opendev.org/c/opendev/system-config/+/88195209:40
opendevreviewMarcin Juszkiewicz proposed opendev/system-config master: reprepro: add Debian Bookworm config  https://review.opendev.org/c/opendev/system-config/+/88195209:59
*** dviroel__ is now known as dviroel11:36
opendevreviewAshutosh Sarode proposed openstack/project-config master: Add Harbor app to starlingx  https://review.opendev.org/c/openstack/project-config/+/88196011:37
opendevreviewAshutosh Sarode proposed openstack/project-config master: Add Harbor app to StarlingX  https://review.opendev.org/c/openstack/project-config/+/88196212:01
opendevreviewAshutosh Sarode proposed openstack/project-config master: Add Harbor app to StarlingX  https://review.opendev.org/c/openstack/project-config/+/88196212:06
opendevreviewAshutosh Sarode proposed openstack/project-config master: Add Harbor app to StarlingX  https://review.opendev.org/c/openstack/project-config/+/88196212:37
opendevreviewJeremy Stanley proposed openstack/project-config master: linters: avoid broken ansible-compat  https://review.opendev.org/c/openstack/project-config/+/88196312:43
*** sfinucan is now known as stephenfin13:04
opendevreviewAshutosh Sarode proposed openstack/project-config master: Add Harbor app to StarlingX  https://review.opendev.org/c/openstack/project-config/+/88196213:14
opendevreviewMerged openstack/project-config master: linters: avoid broken ansible-compat  https://review.opendev.org/c/openstack/project-config/+/88196313:26
opendevreviewMerged openstack/project-config master: Retire puppet-tacker - Step 5: Remove Project  https://review.opendev.org/c/openstack/project-config/+/87529113:57
opendevreviewMerged openstack/project-config master: Retire puppet-rally - Step 1: End project Gating  https://review.opendev.org/c/openstack/project-config/+/87941914:00
opendevreviewAshutosh Sarode proposed openstack/project-config master: Add Harbor app to StarlingX  https://review.opendev.org/c/openstack/project-config/+/88196215:02
*** Guest74 is now known as atmark15:22
opendevreviewJeremy Stanley proposed openstack/project-config master: Fix the "all" transformation so it actually works  https://review.opendev.org/c/openstack/project-config/+/88207515:25
clarkbfungi: in the commit message for ^ I'm confused because isn't a dry run going to not affect writing to disk? In other words this should be a noop other than it maybe does the work twice throwing it away once?15:29
clarkbor does it short circuit when processing the 0 transformation?15:30
fungitransformation "0" switches on dry run mode15:30
clarkboh so it is on globally?15:30
fungias does not specifying any transformations (in which case all are applied)15:30
fungithe way to not dry run but apply all transformations is to supply the list 1 2 3 4 5 6 7 8 915:31
fungithe "all" transformation is meant to be a shortcut for that15:31
opendevreviewAshutosh Sarode proposed openstack/project-config master: Add Harbor app to StarlingX  https://review.opendev.org/c/openstack/project-config/+/88196215:31
fungibut since the range it generated erroneously included 0 that caused it to be a dry run when not intended to be15:32
fungithough going back over the comments now i'm not positive that was intentional15:38
fungier, wasn't15:38
clarkboh I approved it already15:52
fungii set it wip, it also failed check15:52
fungii have an alternative i'm about to push15:52
fungii think i misread the intent, and we never implemented the thing i wanted it to have, so i'm adding that instead15:53
opendevreviewJeremy Stanley proposed openstack/project-config master: Add an "apply" transformation which applies all  https://review.opendev.org/c/openstack/project-config/+/88207515:54
opendevreviewJeremy Stanley proposed openstack/project-config master: Make option indenting a selectable transformation  https://review.opendev.org/c/openstack/project-config/+/88208015:54
fungiclarkb: ^ that should make more sense, sorry15:54
opendevreviewClark Boylan proposed opendev/system-config master: Update user agent filter list  https://review.opendev.org/c/opendev/system-config/+/88208316:28
clarkbfungi: noonedeadpunk ^ fyi16:28
fungiemergency single-core approved to hopefully get things less overloaded16:30
clarkbI think there is minimal testing of 882083 so we should keep eyes open for any trouble17:12
opendevreviewClark Boylan proposed opendev/system-config master: Run gitea and static tests when update Apache UA filters  https://review.opendev.org/c/opendev/system-config/+/88208817:17
clarkbactually I'm going to flip the order of those changes17:18
opendevreviewClark Boylan proposed opendev/system-config master: Run gitea and static tests when update Apache UA filters  https://review.opendev.org/c/opendev/system-config/+/88208817:18
opendevreviewClark Boylan proposed opendev/system-config master: Update user agent filter list  https://review.opendev.org/c/opendev/system-config/+/88208317:18
clarkbthis short term slowdown should make things better longer term since we'll test our updates and deploy them automatically without humans needing to do things manually17:19
clarkbfungi: ^ rereview needed though17:19
opendevreviewMerged opendev/system-config master: Run gitea and static tests when update Apache UA filters  https://review.opendev.org/c/opendev/system-config/+/88208817:37
JayFDoes the opendev hosted zuul have a GITHUB_TOKEN available we can use to prevent API rate limiting against github?18:40
JayFJust checking to see if this has been solved at a high level before we consider something ironic-specific18:40
clarkbJayF: The zuul installation runs as a github application which is subject to api limits18:41
JayFWe have a new job (metal3-integration for Ironic \o/) we saw fail on github rate limits18:42
clarkbthese have largely only been an issue when doing multiple zuul restarts in a short period of time as zuul tries to figure out the state of the world for github repos from scratch18:42
JayFand it appeared as if we didn't set a GITHUB_TOKEN for that job at all18:42
clarkbJayF: can you link to the job?18:42
JayFyeah looking for it already18:42
JayF> Error: failed to read "cert-manager.yaml" from provider's repository "cert-manager": failed to get GitHub release v1.11.0: rate limit for github api has been reached. Please wait one hour or get a personal API token and assign it to the GITHUB_TOKEN environment variable18:43
JayFthis is a frankenjob of sorts; using metal3-dev-env to spin up the ironic sha under test18:43
JayFso it's a bit of a snowflake as far as OpenStack jobs go18:44
clarkbok your job failed making requests to github, Zuul didn't. That clarifies some confusion I had18:44
JayFYeah, I'm basically asking if you all hydrate a credential I can drop in or if I need to manage that on my own18:45
JayF(next question after that would involve how to manage that secret)18:45
clarkbwe do not. everything we do happens before your job starts running in a priveleged context using the github zuul application credentials18:45
clarkbif you express your metal3 dependencies as zuul repos then zuul can ensure that is on disk for you when the job starts18:45
clarkbbut I'm not sure what exactly is going on there based on the error so hard to make a recommendation18:46
JayFThat would essentially require us to do a hard fork of the metal3-dev-env (basically the development stack for metal3 devs), I think18:46
JayFI don't have a good answer to that. It's just my impression based on how it's laid out. rpittau is the expert on this job; I'm just trying to hydrate some context about our options18:47
clarkbright without knowing what requests are being made there it is hard for me to suggest anything the logging doesn't really say much about the actual operation18:48
JayFyou did help, whether you realized it or not18:48
clarkbit is weird to me that you would need to do api requests at all when you can just look at the git repo state (since those look like yaml files you want)18:48
JayFI had the bad assumption that the real fix for that was to just get an API key to github to make it happy18:48
JayFbut it sounds like we should try to arrange the job so it does less external fetches at runtime18:48
clarkbJayF: well it may still be depending on what the tool is doing. It is possible the infromation is only included in the github api18:49
JayFand instead lets zuul provide more of the deps it needs (?)18:49
clarkbJayF: yes I suspect that is one way around this18:49
clarkbessentially you would add this github repo to zuul like any other repo and then require that project in your jobs so it is on disk18:49
clarkbthen you can grab the contents of that yaml file18:49
JayFack; thank you, you helped me understand the scope of the problem better. I'll dig in further tomorrow18:49
JayFrpittau: ^ clarkb's suggestion for how to fix that metal3 github rate limit18:50
JayFrpittau: if you don't have time to dig tomorrow; I'll try to make time 18:50
clarkbit looks like you are grabbing ~9 yaml files from github. Which should be well under the api limits18:50
clarkbI think they are like 5k requests per hour or something18:51
clarkbit is possible something is happening in the background that is exploding the request count18:51
clarkblooks like `clusterctl` kicks this off and I have no idea if you cna point it to things on disk vs it going off and talking to github18:54
funginote that caching a local copy of the git repositories you're depending on would make for a more efficient experience for developers running locally too. this is how devstack works, for example18:54
fungiopenstackansible too18:55
fungicloning all your requirements from scratch every time you run a test is not very efficient, regardless of whether it's happening in ci18:55
clarkbits 60 requests per hour without auth. 5k per hour with auth18:57
fungiit's shocking to me that the kubernetes community struggles with very basic challenges like that18:58
JayFIt's shocking to almost every other open source developer when they see how well opendev handles basic challenges like that :D 18:59
JayFyour perspective is broken becauase you all are good at your jobs :D 18:59
* JayF &19:00
clarkbif I parse that correctly you can set up a goproxy somewhere (or use someone elses?)19:02
clarkband that might help19:02
fungithat bug report was opened 3 years ago, and closed about a year ago by adding the ability to provide a github token? yikes19:07
fungion the theme of things that i'm disappointed but not surprised by, people using text editors which don't differentiate between different kinds of whitespace characters (mainly spaces vs tabs) and not providing easy control over which to use19:34
clarkbI have vim configured to give visual indicators for tabs when used. Similar to what gerrit does in its diff rendering. Makes it super noticeable to me19:35
fungiyeah, me too19:38
opendevreviewMerged opendev/system-config master: Update user agent filter list  https://review.opendev.org/c/opendev/system-config/+/88208319:44
clarkbthe UA filter appears to have applied to gitea0919:59
clarkbI need to pop out now but I suspect that gitea14 will be much happier now19:59
ianw++ looks like that script, again :/20:00
fungii'm surprised it's only one script20:00
clarkbmaking a direct request to gitea14 responds quickly now too20:00
fungiianw: your ansible-compat pr seems to be missing an update to tests that assert the thing you're reverting21:16
fungior at least that's what i'm inferring from the ci failure21:16
ianwfungi: ohhh, hrm that could be right21:19
fungii think that test was literally intended as "make sure nobody reverts this"21:20
ianwi mean, that makes sense21:21
ianwnow i look at the original, runtime.py needs a revert too21:21
clarkbrequest for AAAA glue records has been made21:22
ianwmaybe i'll just make it a full revert as i don't want to try pulling apart the change any further.  the cl doesn't give any context to help21:22
fungiit sends a clear signal, but i have a feeling i know how that will be received21:24
fungistill, it's probably a way to make some progress there21:25
ianwi pushed a full revert, see if that works21:30
clarkbinfra-root tonyb https://etherpad.opendev.org/p/tk-kBybo141kvSiI_hCb here is a first draft of the fedora feedback email21:35
ianwlgtm at a first glance.  the fact that we've missed f37 in nodepool shows how it never gets to the top of anyones todo list21:37
fungiyeah, i take that as a strong sign21:39
fungicorrected a minor typo, but lgtm21:43
clarkbJayF: reading more on the clusterapi thing I think it is already using a goproxy but still hitting the limits. I'm honestly amazed that this suggestion hasn't been implemented yet as this would make the problem go away completely aiui: https://github.com/kubernetes-sigs/cluster-api/issues/3982#issuecomment-125545078721:49
clarkbbut this is webscale cloudnative you must provide authentication to github so they can track you :)21:49
clarkbJayF: I suspect that you will need to provide a token because this tool (clusterapi/clusterctl) isn't built to look at anything but upstream state and that requires at least some github api interaction21:51
clarkbJayF: https://zuul-ci.org/docs/zuul/latest/config/secret.html#secret is the way to deal with that. The use of a secret will make a job post review only21:58
clarkbtonyb: I've got to pop out now, but let me know what you think and if ou don't object I can get that sent either tonight or tomorrow22:48
tonybwill do.22:49
tonybclarkb: looks good to me.22:52

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!