| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 1001456: CI: Disable swift in Devstack https://review.opendev.org/c/openstack/diskimage-builder/+/1001456 | 06:15 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 1001456: CI: Disable swift in Devstack https://review.opendev.org/c/openstack/diskimage-builder/+/1001456 | 06:15 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed wip: [openstack/diskimage-builder] 1001456: CI: Disable swift in Devstack https://review.opendev.org/c/openstack/diskimage-builder/+/1001456 | 06:41 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed wip: [openstack/diskimage-builder] 1001456: CI: Disable swift in Devstack https://review.opendev.org/c/openstack/diskimage-builder/+/1001456 | 06:41 | |
| -@gerrit:opendev.org- Michal Nasiadka marked as active: [openstack/diskimage-builder] 1001456: CI: Disable swift in Devstack https://review.opendev.org/c/openstack/diskimage-builder/+/1001456 | 06:41 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 1001465: rocky-container: Add support for DIB_DISTRIBUTION_MIRROR https://review.opendev.org/c/openstack/diskimage-builder/+/1001465 | 08:19 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 1001465: rocky-container: Add support for DIB_DISTRIBUTION_MIRROR https://review.opendev.org/c/openstack/diskimage-builder/+/1001465 | 08:19 | |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [openstack/diskimage-builder] 1001395: Increase devstack Glance total image size limits https://review.opendev.org/c/openstack/diskimage-builder/+/1001395 | 09:37 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 1001465: rocky-container: Add support for DIB_DISTRIBUTION_MIRROR https://review.opendev.org/c/openstack/diskimage-builder/+/1001465 | 12:15 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 1001465: rocky-container: Add support for DIB_DISTRIBUTION_MIRROR https://review.opendev.org/c/openstack/diskimage-builder/+/1001465 | 12:16 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 1001483: CI: Bump dib flavor disk to 6GB https://review.opendev.org/c/openstack/diskimage-builder/+/1001483 | 12:17 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: [openstack/diskimage-builder] 1001483: CI: Bump dib flavor disk to 6GB https://review.opendev.org/c/openstack/diskimage-builder/+/1001483 | 12:17 | |
| -@gerrit:opendev.org- Michal Nasiadka proposed: | 13:42 | |
| - [openstack/diskimage-builder] 1001465: rocky-container: Add support for DIB_DISTRIBUTION_MIRROR https://review.opendev.org/c/openstack/diskimage-builder/+/1001465 | ||
| - [openstack/diskimage-builder] 1001483: CI: Bump dib flavor disk to 6GB https://review.opendev.org/c/openstack/diskimage-builder/+/1001483 | ||
| @clarkb:matrix.org | Looks like things may be quiet for the moment. I'm going to get a bike ride in first thing today before it gets hot. fungi: I'm up for landing anubis related updates today if you are | 14:18 |
|---|---|---|
| @fungicide:matrix.org | Clark: sounds good. keep in mind that your 1.27.0 upgrade change is redundant, we already upgraded to 1.27.2 via another change of yours last week? | 14:22 |
| @fungicide:matrix.org | i left a comment on it to that effect yesterday after the meeting | 14:23 |
| -@gerrit:opendev.org- Michal Nasiadka proposed: | 14:37 | |
| - [openstack/diskimage-builder] 1001465: rocky-container: Add support for DIB_DISTRIBUTION_MIRROR https://review.opendev.org/c/openstack/diskimage-builder/+/1001465 | ||
| - [openstack/diskimage-builder] 1001483: CI: Bump dib flavor disk to 6GB https://review.opendev.org/c/openstack/diskimage-builder/+/1001483 | ||
| @clarkb:matrix.org | fungi: 1.27.2 was the gitea upgrade. 1.27.0 is anubis. Overlapping version numbers with different software | 15:41 |
| @fungicide:matrix.org | er, oh | 16:20 |
| @clarkb:matrix.org | https://github.com/prometheus/node_exporter#enabled-by-default is the default node-exporter list of collectors | 16:21 |
| @fungicide:matrix.org | okay, well i'm around and ready to approve it at this point, seems to still be the latest anubis version | 16:21 |
| @clarkb:matrix.org | cool I'm back and at a computer too so ready if you are. My plan is to dig into prometheus data otherwise | 16:22 |
| @clarkb:matrix.org | the "easy" way to audit this may be to query the metrics endpoint directly then skim that data | 16:25 |
| @fungicide:matrix.org | okay, anubis 1.27.0 approved | 16:27 |
| @fungicide:matrix.org | i went ahead and manually upgraded the anubis package on the wiki server too | 16:30 |
| @fungicide:matrix.org | and restarted the daemon | 16:30 |
| @clarkb:matrix.org | and I can still reach the wiki | 16:32 |
| @fungicide:matrix.org | yeah, i reloaded and saw the anubis splash screen briefly and it reported the new version number | 16:33 |
| @fungicide:matrix.org | so seems at first pass to be functional at least | 16:33 |
| @clarkb:matrix.org | the anubis change is just about done in the gate | 17:55 |
| @clarkb:matrix.org | oh I guess it has to do the testinra tests still | 17:58 |
| -@gerrit:opendev.org- Zuul merged on behalf of Clark Boylan: [opendev/system-config] 1000348: Upgrade Anubis to 1.27.0 https://review.opendev.org/c/opendev/system-config/+/1000348 | 18:02 | |
| @clarkb:matrix.org | hourlies enqueued just ahead of ^ | 18:02 |
| @clarkb:matrix.org | lists has updated and seems to work for me | 18:12 |
| @clarkb:matrix.org | gitea is almost down. In a browser I'm getting the new anubis version for the backend I talk to and can browse around | 18:22 |
| @clarkb:matrix.org | * gitea is almost done. In a browser I'm getting the new anubis version for the backend I talk to and can browse around | 18:22 |
| @clarkb:matrix.org | once the cluster is done I'll test with git too | 18:22 |
| @clarkb:matrix.org | https://zuul.opendev.org/t/openstack/buildset/8def3b007ad940b18bbf8cd68ac43c43 buildset was successful and I can git clone system-config | 18:24 |
| @clarkb:matrix.org | so initial checks look good. We'll have to monitor in case it degrades like last time. But assuming it doesn't we can turn on the honeypot again later today or tomorrow? | 18:25 |
| @fungicide:matrix.org | yeah, i was pulling up something in gitea and saw the new version number | 18:35 |
| @fungicide:matrix.org | sounds good to me | 18:35 |
| @clarkb:matrix.org | cool I'm going to grab lunch now. Back in a bit | 18:51 |
| @clarkb:matrix.org | I've been looking at the warnings in https://zuul.opendev.org/t/openstack/buildset/8def3b007ad940b18bbf8cd68ac43c43 and trying to find the correct place to claen up the known_hosts.old file. I had assumed that maybe https://opendev.org/opendev/base-jobs/src/branch/master/playbooks/infra-prod/setup-keys.yaml#L21-L24 was doing it but reading through the code for the known_hosts module it seems to create a named tempfile using python's tempfile module which shouldn't result in a .old file and then it does an atomic move for that file. So I don't know that it is the source of the .old file. Then I thought maybe https://opendev.org/opendev/system-config/src/branch/master/playbooks/zuul/roles/add-bastion-host/tasks/main.yaml#L2-L13 was doing it based on no complaints from the pre.yaml playbook in that set of warnings (the first warning in the zuul playbook order appears to be from opendev.org/opendev/system-config/playbooks/zuul/run-production-bootstrap-bridge.yaml and run-production-bootstra-bridge.yaml calls add-bastion-host. But add_host doesn't appear to touch known_hosts. | 20:34 |
| corvus do you know if the pre.yaml playbook would warn too if setup-keys.yaml was the source of the .old file? | ||
| -@gerrit:opendev.org- Clark Boylan proposed: [opendev/base-jobs] 1001567: Stat known_hosts.old after updating known_hosts https://review.opendev.org/c/opendev/base-jobs/+/1001567 | 20:47 | |
| @clarkb:matrix.org | it does seem likely that the known_hosts module is side effecting this and I'm just not seeing it. ^ is an update to check that hypothesis and help me remember to keep debugging this | 20:47 |
| @jim:acmegating.com | Clark: the warnings come from the playbook after the file is created | 20:48 |
| @clarkb:matrix.org | corvus: ya so I thought it was weird that the run playbooks don't seem to complain all that much for lists and gitea | 20:48 |
| @clarkb:matrix.org | which is why I thought maybe it was the run playbooks or the post playbook in system-config doing it (and why I want to check in that change I just pushed) | 20:49 |
| @clarkb:matrix.org | but maybe its only the pre/post pairings not the run stuff that will complain? | 20:49 |
| @jim:acmegating.com | Clark: `man ssh-keygen` says that `ssh-keygen -H` will write a .old file | 20:53 |
| @jim:acmegating.com | it doesn't say if `-R` will do the same, but that's related... so i wouldn't be surprised... | 20:54 |
| @jim:acmegating.com | the known_hosts ansible module does run -R | 20:54 |
| @clarkb:matrix.org | ah ok so maybe we're tripping over undocumented ssh-keygen behavior | 20:55 |
| @clarkb:matrix.org | in which case having a followup task remove the .ssh/known_hosts.old file is probably appropriate? | 20:55 |
| @jim:acmegating.com | is the only known_hosts thing we do to add the bastion? https://zuul.opendev.org/t/openstack/build/68a84526e5534e469625da2b941a7530/console#1/0/1/localhost | 20:56 |
| @clarkb:matrix.org | corvus: as far as I can tell yes | 20:57 |
| @clarkb:matrix.org | we use that single known_hosts update. But then we have to add-bastion-host to add_host in every new playbook | 20:57 |
| @jim:acmegating.com | Clark: i share your puzzlement and agree that triangulating like 567 is a good next step | 21:05 |
| @clarkb:matrix.org | I've spot checked the anubis log on gitea09 and don't see anything immediately concerning | 21:42 |
| @clarkb:matrix.org | seems like lots of indicators that actual bots are running up against the challenges which is good | 21:42 |
| @mordred:waterwanders.com | btw - in case nobody has noticed - codesearch links to source locations don't understand repos with main vs master | 22:19 |
| @mordred:waterwanders.com | it indexes and displays them just fine - but the web links to gitea seem to hardcode master | 22:20 |
| @clarkb:matrix.org | mordred: I think that is a codesearch issue. It only understands a single branch (this is why no stable branch search either) | 22:20 |
| @clarkb:matrix.org | and we force it to look at main instead of master but I guess when it renders things it doesn't know to check the branch name? | 22:20 |
| @clarkb:matrix.org | er hound is the name not codesearch | 22:21 |
| @mordred:waterwanders.com | yeah. I mean - I'm guessing it just clones and gets default branch | 22:21 |
| @mordred:waterwanders.com | so it doesn't have to know anything about that branch name | 22:21 |
| @clarkb:matrix.org | ya and doesn't think it needs to check it either | 22:21 |
| @mordred:waterwanders.com | not a big deal - just happened to notice | 22:21 |
| @mordred:waterwanders.com | Clark: actually - I think it's a bug in jeepyb | 22:24 |
| @mordred:waterwanders.com | https://opendev.org/opendev/jeepyb/src/branch/master/jeepyb/cmd/create_hound_config.py#L56 | 22:24 |
| @fungicide:matrix.org | we keep hoping longer term gitea's search function may improve enough to axe hound | 22:28 |
| @clarkb:matrix.org | oh interesting I had no idea that we set the paths like that | 22:28 |
| @clarkb:matrix.org | fungi: ya but if anything they've made it worse over time :/ | 22:28 |
| @fungicide:matrix.org | poop | 22:29 |
| @clarkb:matrix.org | mordred: https://opendev.org/opendev/bindep/src/branch/HEAD/bindep.txt this appears to work | 22:30 |
| @clarkb:matrix.org | mordred: so maybe we can s/master/HEAD/ and let gitea figure it out on the fly? | 22:31 |
| @clarkb:matrix.org | that would mimic the source cloning behavior too | 22:31 |
| @mordred:waterwanders.com | Clark: ooh - that's even easier. we could also template out main vs master from the projects.yaml info ... but I like HEAD instead | 22:32 |
| @clarkb:matrix.org | Gemini says that won't work fwiw, but experimentally gemini is wrong | 22:32 |
| -@gerrit:opendev.org- Monty Taylor https://matrix.to/#/@mordred:inaugust.com proposed: [opendev/jeepyb] 1001580: Use HEAD instead of master https://review.opendev.org/c/opendev/jeepyb/+/1001580 | 22:37 | |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!