| opendevreview | Ian Wienand proposed opendev/system-config master: Add a playbook to grab collated info from hosts https://review.opendev.org/c/opendev/system-config/+/801671 | 01:42 |
|---|---|---|
| opendevreview | Ian Wienand proposed opendev/system-config master: Add a playbook to grab collated info from hosts https://review.opendev.org/c/opendev/system-config/+/801671 | 01:45 |
| *** marios is now known as marios|ruck | 05:51 | |
| *** ykarel|away is now known as ykarel | 06:12 | |
| *** amoralej|off is now known as amoralej | 07:00 | |
| opendevreview | Sorin Sbârnea proposed opendev/git-review master: Make default output less verbose https://review.opendev.org/c/opendev/git-review/+/772646 | 07:32 |
| opendevreview | Merged openstack/diskimage-builder master: Replace deprecated import of ABCs from collections https://review.opendev.org/c/openstack/diskimage-builder/+/801092 | 07:38 |
| *** rpittau|afk is now known as rpittau | 07:49 | |
| opendevreview | Merged openstack/diskimage-builder master: Auto find greatest Fedora cloud image sub-release https://review.opendev.org/c/openstack/diskimage-builder/+/755992 | 07:53 |
| *** bhagyashris_ is now known as bhagyashris | 08:05 | |
| *** bhagyashris_ is now known as bhagyashris | 09:35 | |
| opendevreview | Takashi Kajinami proposed openstack/diskimage-builder master: Update IRC networks https://review.opendev.org/c/openstack/diskimage-builder/+/801758 | 10:22 |
| *** ykarel is now known as ykarel|lunch | 10:25 | |
| opendevreview | Merged openstack/diskimage-builder master: Add a keylime-agent element and a tpm-emulator element https://review.opendev.org/c/openstack/diskimage-builder/+/789601 | 10:34 |
| *** ykarel|lunch is now known as ykarel | 11:52 | |
| opendevreview | Kartikeya Jain proposed openstack/diskimage-builder master: Add new element oracle-linux https://review.opendev.org/c/openstack/diskimage-builder/+/801774 | 12:13 |
| *** amoralej is now known as amoralej|lunch | 12:28 | |
| *** ykarel_ is now known as ykarel | 12:38 | |
| *** elodilles is now known as elodilles_afk | 12:48 | |
| *** amoralej|lunch is now known as amoralej | 13:29 | |
| *** elodilles_afk is now known as elodilles | 14:34 | |
| opendevreview | Ananya proposed opendev/elastic-recheck master: Run elastic-recheck container https://review.opendev.org/c/opendev/elastic-recheck/+/729623 | 14:43 |
| *** ykarel_ is now known as ykarel | 14:56 | |
| opendevreview | Ananya proposed opendev/elastic-recheck master: Run elastic-recheck container https://review.opendev.org/c/opendev/elastic-recheck/+/729623 | 15:02 |
| *** ykarel is now known as ykarel|away | 15:08 | |
| clarkb | slow start this morning but looking to approve the gitea upgrade shortly. Any reason to not do that at this point? | 15:45 |
| fungi | i know of none | 15:46 |
| clarkb | I have approved it. And while i wait for it to gate I should go water my garden | 15:47 |
| opendevreview | Ananya proposed opendev/elastic-recheck master: Run elastic-recheck container https://review.opendev.org/c/opendev/elastic-recheck/+/729623 | 15:52 |
| clarkb | fungi: ianw: if you can review the matrix spec today that would be great. | 15:56 |
| fungi | i'm in the middle of it actually, but so far no concerns | 15:56 |
| clarkb | great | 15:56 |
| *** rpittau is now known as rpittau|afk | 16:02 | |
| opendevreview | Clark Boylan proposed opendev/system-config master: Upgrade gitea to 1.14.5 https://review.opendev.org/c/opendev/system-config/+/801906 | 16:19 |
| clarkb | I just noticed that there is another release we can update too. That is pushed as a followon since it should be straightforward to approve that if 1.14.4 goes well | 16:19 |
| clarkb | fungi: if 1.14.4 rolls out smoothly I think we shoukld try and land ^ quickly behind it just to avoid any unnecessary lag there | 16:23 |
| fungi | yeah, i concur | 16:23 |
| *** marios|ruck is now known as marios|out | 16:24 | |
| *** amoralej is now known as amoralej|off | 16:32 | |
| opendevreview | James E. Blair proposed opendev/system-config master: Add DNSSEC configuration for gating.dev https://review.opendev.org/c/opendev/system-config/+/801907 | 16:36 |
| opendevreview | James E. Blair proposed opendev/system-config master: Serve gating.dev static website https://review.opendev.org/c/opendev/system-config/+/801908 | 16:37 |
| opendevreview | Merged opendev/system-config master: Update gitea to 1.14.4 https://review.opendev.org/c/opendev/system-config/+/800274 | 17:01 |
| clarkb | the job to deploy that should start momentarily | 17:03 |
| clarkb | I'm watching it | 17:03 |
| fungi | i'm still around | 17:03 |
| clarkb | interesting it did the next job in the hourly buildset instead of the gitea deploy job in the deploy buildset | 17:04 |
| clarkb | I wonder if we have to wait for that whole buildset now? change in behavior with semaphores in zk maybe? | 17:05 |
| corvus | i'm not sure there's a strict ordering of semaphores across pipelines; it may be semi-random? | 17:07 |
| clarkb | ah that could be. In any case the job won't run as soon as I expected as other jobs are getting the semaphore | 17:08 |
| fungi | that would make some sense, yeah | 17:10 |
| opendevreview | James E. Blair proposed opendev/system-config master: Serve gating.dev static website https://review.opendev.org/c/opendev/system-config/+/801908 | 17:24 |
| corvus | clarkb, fungi, mordred: if you could approve https://review.opendev.org/801907 i can go ahead and add the dns key to the registrar | 17:25 |
| clarkb | corvus: actual host/group vars have the secret info ready? | 17:27 |
| corvus | yep | 17:27 |
| fungi | approved it now | 17:31 |
| clarkb | alright the job is running now | 17:56 |
| opendevreview | Merged opendev/system-config master: Add DNSSEC configuration for gating.dev https://review.opendev.org/c/opendev/system-config/+/801907 | 17:56 |
| clarkb | https://gitea01.opendev.org:3081/opendev/system-config seems to be up | 18:01 |
| clarkb | it wasn't the fastest update, but after a couple of minutes the service finishes it upgrade stuff and the ssh daemon was started | 18:01 |
| fungi | yeah, lgtm | 18:08 |
| fungi | seems like they make minor changes to fonts on every update | 18:08 |
| clarkb | The first 3 are done now. 4th is in progress | 18:10 |
| clarkb | and now 4 is done. We are halfway through the set. | 18:12 |
| clarkb | fungi: I guess if all 8 checkout I should go ahead and approve the 1.14.5 update? | 18:12 |
| fungi | yep | 18:15 |
| clarkb | Then I need to remember to clean up my held node once that is all done | 18:16 |
| clarkb | all 8 look reachable and render the repo as expected. I'm approving 1.14.5 update now | 18:27 |
| fungi | thanks | 18:32 |
| clarkb | It feels good to catch back up to them again. Time to start looking at gerrit 3.3 upgrades soon as well I think | 18:36 |
| clarkb | we could go to 3.4 but luca mentioend there are a ton of bugfixes in the queue for 3.4 so might be better to step through 3.3 first? I also kno that you can downgrade from 3.3 to 3.2 with a manual notedb edit if necessary which is a nice option | 18:37 |
| fungi | interesting | 18:38 |
| clarkb | I added a release note to gerrit for the oepnid change I'm making and that release note goes against 3.4 I guess. The list of changes is quite long there. | 18:38 |
| fungi | good to know | 18:40 |
| clarkb | fungi: side note I didn't break the gerrit opendev theme with the gitea update :) | 18:44 |
| clarkb | at least it currently loads for me in what I would consider the correct manner | 18:45 |
| fungi | yeah, seems to still be intact | 18:45 |
| fungi | future improvement would be to centralize/generalize our logo storage in the repo and then bake copies into images without having them depend on one another | 18:46 |
| clarkb | I think right now its fetching it via https in the browser, but ya if we can stick that in a location on the gerrit image and the gitea image from a central source that would be best | 18:47 |
| clarkb | Then we don't have to remember a bunch of update locations etc | 18:47 |
| fungi | and also a gitea outage doesn't degrade gerrit performance that way | 18:48 |
| clarkb | mordred and others may find https://engineering.fb.com/2021/07/22/data-infrastructure/mysql/ to be interesting | 19:05 |
| clarkb | ianw: related to ^ do we have an explicit record old db state step before shutting down review01 and then also a shutdown the db step? | 19:07 |
| opendevreview | Merged opendev/system-config master: Upgrade gitea to 1.14.5 https://review.opendev.org/c/opendev/system-config/+/801906 | 19:41 |
| Clark[m] | I'm eating lunch but gitea01 has updated and looks good on a phone | 19:51 |
| *** sshnaidm is now known as sshnaidm|afk | 20:10 | |
| clarkb | #status log Upgraded gitea to version 1.14.5 | 20:59 |
| opendevstatus | clarkb: finished logging | 20:59 |
| clarkb | Forgot to do that earlier | 20:59 |
| fungi | well now you don't have to redo it | 21:00 |
| fungi | it does look like they may have adjusted some icons as well | 21:01 |
| fungi | also this looks kinda weird, do we have an incomplete sync or something? https://opendev.org/opendev/bindep/commit/64205044f52b08d55a4c4f994aecede6ce51428d | 21:04 |
| clarkb | that belongs to changes/20/742320/meta ? | 21:05 |
| clarkb | I think that may be complete. Its just the notedb content? | 21:05 |
| clarkb | I'm not sure how well that all renders in gitea | 21:06 |
| fungi | oh! that's a notes ref not the commit itself | 21:06 |
| fungi | strange | 21:06 |
| clarkb | yes its the metadata about the change if I'm reading it correctly | 21:07 |
| fungi | yep, the commit is https://opendev.org/opendev/bindep/commit/3bee5129d72497070de7ded454ae88c2ad7e2634 | 21:07 |
| fungi | which unfortunately doesn't display the note (i think there was a todo to change the displayed notes ref base or something in the config) | 21:08 |
| fungi | and because it treats the notes refs as independent commits it makes the commit graph fairly useless: https://opendev.org/opendev/bindep/graph | 21:09 |
| clarkb | oh wow ya that is pretty useless | 21:10 |
| fungi | aha, also it's including unmerged refs in that: https://opendev.org/opendev/bindep/commit/4ebb650b203c42e6e0d64aaee063470e01f9d410 | 21:10 |
| fungi | so yeah, how gerrit stores patchsets and git notes seems to be incompatible with the design of that commit graph feature | 21:12 |
| ianw | clarkb: we don't have a record old db step; are you interested in keeping the <3 gerrit state? | 21:49 |
| clarkb | ianw: ya I was thinking it might be a good idea to take a final snapshot of that entire db (the pre notedb stuff and accountpatchreviewdb) before we clean up that server? | 21:52 |
| clarkb | fungi: for the matrix spec I guess your rollcall +1 indicates you're ok with it landing despite your feedback? I'm not super clear on that since you didn't leave a code review vote | 21:55 |
| clarkb | I think the debian afs mirror may be at quota | 22:01 |
| clarkb | at least the grafana graph shows that it ran out of space | 22:01 |
| clarkb | we freed up 140GB of the yum-puppetlabs repo so we should be able to bump debian up. Maybe from 600GB to 640? | 22:02 |
| clarkb | fungi: ^ You might have the most context for that repo and how it might grow over time | 22:02 |
| ianw | clarkb: i can; the Marie Kondo in me says that nothing about it "sparks joy" so perhaps it is better left as a memory | 22:09 |
| clarkb | ianw: in my head it would mostly be used to help correct any potential issues we discover from the migration. Granted we haven't seen any of those that I know of other than the account problems. But having the ability to look back at the info as it was pre migration could be helpful if we run into similar issueswith changes? | 22:10 |
| ianw | clarkb / fungi : i feel like there was no real objection to removing stretch, so we have a way forward to reduce the overhead, so bumping it for now seems ok while we do the work | 22:11 |
| clarkb | ianw: ya I don't expect the quota bump to be necessary long term. Wondering what an appropriate bump might be? Is 40GB too little or is that sufficient? | 22:15 |
| clarkb | looks like there may be a bit of room there but I'll go ahead and bump it by 40GB to avoid failed releases and updates | 22:18 |
| ianw | clarkb: ok, i've made an "attic" directory on review02 for things of interest to archive. i'll run a dump against the trove db and save it in there | 22:18 |
| clarkb | ianw: thanks | 22:18 |
| ianw | i think that should be plenty | 22:20 |
| ianw | (40gb) | 22:20 |
| clarkb | ok its bumped on the RW volume and the next regular update should push the quota bump to the RO volume | 22:23 |
| clarkb | my biggest concern right now is that we might break the mirror because we stop updating it. This should avoid that problem and then we can trim back down again via stretch removal | 22:25 |
| clarkb | we might also consider dropping the fedora mirror volume quota down since it is set quite large | 22:26 |
| clarkb | btu that is less urgent and might be better to do that for all the volumes once we've trimmed a bit more | 22:26 |
| clarkb | Gerrit screenshots making it into the tech press for all the wrong reasons https://arstechnica.com/gadgets/2021/07/google-pushed-a-one-character-typo-to-production-bricking-chrome-os-devices/ | 22:52 |
| clarkb | also check out the change number. | 22:54 |
| fungi | clarkb: i'm in favor of merging the matrix spec, yes | 23:01 |
| fungi | i left feedback as it occurred to me | 23:01 |
| clarkb | fungi: thanks for confirming. ianw did you want to review that matrix spec before I approve it? I can approve it a bit later today if so | 23:02 |
| fungi | i figured code review for something with a rollcall was not especially relevant | 23:02 |
| fungi | but happy to add a cr +1 as well (there is no core reviewer permission for +2 on the specs repo i don't think) | 23:02 |
| clarkb | ya its a +1. I guess in my head I associate CR as this is ok to merge as is and Roll Call as I am comfortable with this plan | 23:03 |
| clarkb | they are slightly different things | 23:03 |
| fungi | as for the old gerrit db, that's in trove anyway right? | 23:04 |
| fungi | so deleting the server instance isn't going to impact it | 23:04 |
| clarkb | fungi: correct, but the next step is to delete the trove instance :) | 23:04 |
| fungi | awesome | 23:08 |
| ianw | ok, backups in ~gerrit2/attic/old-db-2021-07 along with a readme for future admins who wonder what the heck we were doing :) | 23:09 |
| fungi | or s/doing/smoking/ | 23:10 |
| clarkb | I'm trying to push an update to my upstream gerrit change and it is hanging. It is unfortunate we can't really compare our srever performance against the upstream one | 23:16 |
| clarkb | though I suppose it could be local network problems too. I've got some weird rtt aross the local network | 23:19 |
| clarkb | strace says it is stuck on a read | 23:20 |
| clarkb | if in doubt try again. went through on second attempt | 23:21 |
| fungi | as one does | 23:27 |
| opendevreview | Clark Boylan proposed opendev/system-config master: Push a patch to try and prevent gerrit openid deletion https://review.opendev.org/c/opendev/system-config/+/800832 | 23:43 |
| clarkb | I got my first +2 upstream so I've set up a new hold and pushed the current state of the diff into ^ | 23:44 |
| clarkb | I'll redo my functional testing tomorrow on the held node | 23:44 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!