| dmsimard[m] | btw I hacked on a proof of concept of ara using databases from zuul throughout the weekend, I cleaned it up a bit tonight and it works | 01:14 |
|---|---|---|
| dmsimard[m] | for example: https://openstack.recordsansible.org?url=https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fbf/openstack/fbfcf80c91984f60bd54b57876accfec/logs/ara-report/ansible.sqlite | 01:14 |
| dmsimard[m] | it doesn't "stream" the database, it downloads it and caches it locally | 01:15 |
| dmsimard[m] | I have to run for now, but can talk about it later | 01:16 |
| *** mrunge_ is now known as mrunge | 06:17 | |
| opendevreview | yatin proposed openstack/project-config master: [neutron] update dashboard with new jobs https://review.opendev.org/c/openstack/project-config/+/971155 | 08:39 |
| *** ykarel_ is now known as ykarel | 14:00 | |
| fungi | borg backup prune in progress in a root screen session on backup02.ca-ymq-1.vexxhost | 14:02 |
| fungi | #status log Pruned /opt/backups-202010 on backup02.ca-ymq-1.vexxhost reducing utilization from 90% to 71% | 15:00 |
| opendevstatus | fungi: finished logging | 15:00 |
| clarkb | fungi: thanks. I think last time we trimmed to ~67%. I've got some changes up to purge additional backups if we feel we don't need them any longer which should free additional space | 15:45 |
| fungi | yes, i confirm the previoys prune got it down to 67% | 15:48 |
| clarkb | https://review.opendev.org/c/opendev/system-config/+/951041 this is the purging change | 15:50 |
| fungi | lunch, bbiab | 17:07 |
| clarkb | I've learned some more things about H2 v2 caches in Gerrit 3.12. If you don't run a conversion tool the new version of Gerrit simply creates new caches as if the old caches didn't exist. These files go alongside the old files (they don't delete the old files). So this is very similar functionally to our restart process that involves deleting v1 cache dbs forcing them to be rebuilt | 17:11 |
| clarkb | I'm leaning towards not bothering with a conversion at all. This will force everyone to log back into Gerrit with this upgrade but I think that is fine | 17:11 |
| clarkb | I still don't understand the risk of corruption on shutdown. We may need to incrase our docker compose timeout to some large value and then benchmark with a restart post upgrade to understand how slow that is | 17:12 |
| clarkb | in theory it is faster than the process with h2 v1, but also theoretically not zero cost | 17:12 |
| fungi | yeah, i'm fine with the idea of login sessions force-expiring on upgrade | 18:08 |
| mnaser | interesting to discuss and keep in mind, github has introduced a $0.002 "platform fee" per minute for github actions jobs... but the main thing is that it INCLUDES even self hosted agents. | 19:01 |
| mnaser | so if you're bringing your own infra for github actions, you'll still have to pay | 19:01 |
| fungi | neat | 19:02 |
| fungi | github's ci/cd tax, they get your money whether or not they provide the test infrastructure | 19:03 |
| mnaser | i believe they've done this bc some customers will use a third party github actions runner like depot.dev or https://namespace.so/ | 19:03 |
| mnaser | a bummer is that i know the cncf got a whole bunch of infra donated from oracle cloud for self hosted runners because the github bill was running high, and it was all self hosted runners, and now they've got to pay (again) | 19:06 |
| fungi | that's crazy | 19:08 |
| fungi | but from a zuul maintainer standpoint, serves as a reminder that tying your projects to proprietary platform-specific ci/cd solutions like github actions runs exactly that risk | 19:09 |
| mnaser | ..even if you're hosting the runners yourself, which is the wild bit to me | 19:36 |
| fungi | yeah, because github is still initiating the hook or whatever in that case | 19:39 |
| fungi | as opposed to having a separate system watching for events and acting on them | 19:39 |
| clarkb | right it is not zero cost to them | 19:40 |
| clarkb | they have to run the scheduler | 19:40 |
| mnaser | (saying this as a github mostly user) but i think the concern with this kind of thing is that down the line this can set a precedent to stuff like... $0.002 per gb of git traffic served as platform fees and so on | 19:41 |
| fungi | i mean, microsoft is a commercial enterprise. i wouldn't put it past them to start charging for git hosting eventually. the first hit is free, then you charge them | 19:42 |
| fungi | when the time comes that they can no longer profit indirectly off github, they will decide to profit directly from it | 19:42 |
| fungi | it's not as if they're a nonprofit/charity | 19:43 |
| fungi | regardless of what users seem to assume | 19:43 |
| mnasiadka | mnaser: I think the fee for self hosted runners is only for private repositories? | 21:23 |
| mnaser | Even for public :) | 21:30 |
| corvus | i have approved the registry and statusbot changes | 22:28 |
| clarkb | corvus: thanks. I can help keep an eye on it. We should be able to do something like #status log from the new opendev room to verify it without creating too much noise `#status log Statusbot now speaks Matrix` or similar | 22:29 |
| corvus | yep | 22:34 |
| corvus | and i also approved its parent. :) | 22:41 |
| clarkb | for some reason I thought that had already merged | 22:42 |
| clarkb | zuul-registry has updated. I'm going to recheck my stack of gerrit chagnse since we know they trip the pruning issue | 23:07 |
| clarkb | then ~thursday we can check if the images are still there | 23:08 |
| opendevreview | Merged opendev/system-config master: Move test IRC statusbot config into test specific vars file https://review.opendev.org/c/opendev/system-config/+/969330 | 23:16 |
| opendevreview | Merged opendev/system-config master: Add matrix config support to statusbot deployment https://review.opendev.org/c/opendev/system-config/+/969328 | 23:16 |
| corvus | https://zuul.opendev.org/t/openstack/status?change=969328%2C3&pipeline=deploy | 23:23 |
| clarkb | statusbot just restarted | 23:36 |
| clarkb | it joined this channel which is a good sign | 23:36 |
| clarkb | I don't see it in the opendev matrix room yet (does it have to send am essage to join first?) | 23:37 |
| clarkb | no we get `ERROR statusbot.matrix: LoginError: M_FORBIDDEN Invalid username/password` | 23:37 |
| clarkb | corvus: maybe the user_id value needs to be the whole @status:opendev.org string? | 23:38 |
| corvus | clarkb: yeah, looks like that might be the case. sorry about that. :) it's different with my homeserver, but i should have checked the matrix-eavesdrop config | 23:40 |
| corvus | i propose i edit the file in place, test that it works, then make a change | 23:40 |
| clarkb | corvus: ++ | 23:41 |
| clarkb | looks like success? | 23:42 |
| corvus | yep, wiki updated with the log entry i sent in the other room | 23:42 |
| clarkb | corvus: I think that value may be set in private vars | 23:43 |
| corvus | i think you're right. i'll update there. | 23:43 |
| clarkb | so we might not need a change just edit the private vars and then daily runs will sync up to your manual fix | 23:43 |
| corvus | +statusbot_matrix_user: '@status:opendev.org' | 23:44 |
| corvus | from the diff ^ | 23:44 |
| clarkb | perfect | 23:45 |
| clarkb | fwiw manage-projects is still running and a quick look at the log shows it is slowly working through the giteas. Then looking at the giteas system load is a bit elevated and we've seen that slow down manage projects previously | 23:45 |
| clarkb | so I don't think anything is really wrong just that gitea is busy right now (we've probably got another web crawler going crazy) | 23:45 |
| clarkb | and it completed. One fewer thing on my todo list thank you! | 23:52 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!