| clarkb | cool that is what I expected | 00:00 |
|---|---|---|
| opendevreview | Ian Wienand proposed opendev/system-config master: make-tarball: add some extraction instructions https://review.opendev.org/c/opendev/system-config/+/875587 | 00:01 |
| ianw | one interesting this is in the 1-abc... 2-def 3-... output of the ssss key split, the X- part is important | 00:03 |
| ianw | we want to make sure we store the whole thing | 00:03 |
| ianw | (i'm glad i actually spent the time validating that so i realised that before it was too late :) | 00:07 |
| clarkb | ++ | 00:41 |
| ianw | ok, thoughts on gerrit @ https://etherpad.opendev.org/p/gerrit-upgrade-3.7 | 00:58 |
| Clark[m] | ianw left a note about the ssh command thing might affect zuul | 01:04 |
| Clark[m] | Otherwise lgtm | 01:04 |
| *** promethe- is now known as prometheanfire | 02:47 | |
| opendevreview | Ian Wienand proposed openstack/project-config master: gerrit/acl : remove deprecated NoOp function https://review.opendev.org/c/openstack/project-config/+/875804 | 05:16 |
| opendevreview | Jake Yip proposed opendev/irc-meetings master: Update Magnum meeting chair https://review.opendev.org/c/opendev/irc-meetings/+/875807 | 05:52 |
| opendevreview | Merged opendev/system-config master: install-docker: remove apt-key cleanup https://review.opendev.org/c/opendev/system-config/+/872809 | 06:31 |
| *** ralonsoh_ is now known as ralonsoh | 07:36 | |
| *** jpena|off is now known as jpena | 08:45 | |
| *** ralonsoh__ is now known as ralonsoh | 10:18 | |
| opendevreview | Espen Flage-Larsen proposed openstack/diskimage-builder master: Updated serial configuration of the bootloader element https://review.opendev.org/c/openstack/diskimage-builder/+/875883 | 11:24 |
| opendevreview | Espen Flage-Larsen proposed openstack/diskimage-builder master: Updated serial configuration of bootloader https://review.opendev.org/c/openstack/diskimage-builder/+/875884 | 11:24 |
| opendevreview | Espen Flage-Larsen proposed openstack/diskimage-builder master: Updated serial configuration of bootloader https://review.opendev.org/c/openstack/diskimage-builder/+/875884 | 11:32 |
| opendevreview | Espen Flage-Larsen proposed openstack/diskimage-builder master: Updated serial configuration of bootloader https://review.opendev.org/c/openstack/diskimage-builder/+/875884 | 11:32 |
| opendevreview | Merged opendev/irc-meetings master: Update Magnum meeting chair https://review.opendev.org/c/opendev/irc-meetings/+/875807 | 13:11 |
| clarkb | infra-root I've got two hours of meetings that just started now. Towards the end of that I'll approve https://review.opendev.org/c/opendev/system-config/+/875533 to update our nine gitea to 1.18.5 | 16:06 |
| clarkb | fungi: it looks like someone on gerrit discord has said you can still lose replication events even with the disk backing | 16:07 |
| clarkb | fungi: what happens when you reload the config is you stop the plugin and then start it again and there is a small window during that process where you can miss new refs showing up in gerrit that need replication | 16:07 |
| clarkb | I think that this risk is still low particularly since we understand that it can occur. And that autoreloading the config will greatly simply the process of replacing gitea backends compared to restarting gerrit everytime we need to add or remove a backend | 16:08 |
| clarkb | we just need to periodically run full syncs after the config updates | 16:08 |
| clarkb | and then when we've replaced all the giteas we can remove autoreload config again to avoid the footgun | 16:09 |
| fungi | clarkb: yeah, saw that (i also follow the gerrit channel via matrix). seems like a small window compared to the old behavior where it dropped all queued replication tasks on reload | 16:45 |
| clarkb | right it will preserve what it already knew about via the disk location which is a big improvement | 16:45 |
| clarkb | but still a small risk is present | 16:45 |
| fungi | how long does a reload of that plugin take? | 16:45 |
| clarkb | I don't know hard numbers ut if you look at the error log on startup I expect it is subsecond | 16:46 |
| clarkb | it should be a very small window. | 16:46 |
| clarkb | If that small window and active mitigation after auto reloads is ok with other infra-root my next task for gitea replacements is likely to land the gerrit config update and the bind bound update | 16:46 |
| clarkb | But I want to get gitea upgraded first since we're in a good spot to pause for that | 16:47 |
| fungi | yeah, i'm okay with it. odds of short-term impact are very low | 16:48 |
| clarkb | one concern with the bind mount change (that I've already checked but will probably triple check) is we have a data dir on review02 host side already. But it is empty so bind mounting it should be fine | 16:50 |
| clarkb | since we could possibly bind mount in ancient data and cause plugins to take bad actions. But its empty so that shouldn't happen | 16:51 |
| *** jpena is now known as jpena|off | 17:35 | |
| clarkb | I've approved the gitea 1.18.5 change | 18:08 |
| fungi | thanks! | 18:09 |
| clarkb | Thats about an hour away I think. I'll keep an eye on it as it deploys | 18:19 |
| fungi | i need to pop out and run some errands now that meetings have wrapped up, but should be back before then | 18:22 |
| fungi | okay, i'm back | 19:24 |
| clarkb | fungi: and its just about to merge. | 19:24 |
| clarkb | I took the opportunity to do expense reports and check on my passport renewal. | 19:25 |
| fungi | i still need to do expense reports, probably saving that for the weekend | 19:26 |
| opendevreview | Merged opendev/system-config master: Update gitea to 1.18.5 https://review.opendev.org/c/opendev/system-config/+/875533 | 19:35 |
| clarkb | the deployment job has started | 19:36 |
| fungi | yep | 19:37 |
| clarkb | I'm on gitea01 waiting for its services to get restarted | 19:38 |
| clarkb | ok it has restarted. experiencing the slow repo load at the beginning of startup. I expect that to last a minute or so before things are good | 19:39 |
| clarkb | yup first load of system-config was 67 seconds. Second was 95ms | 19:41 |
| clarkb | and git clone works too | 19:41 |
| clarkb | this looks good. Now we have to wait for the other 8 giteas | 19:41 |
| clarkb | one thing I like about the url warning in newer gitea is it makes it easy to ensure that I actually refreshed the page off of the gitea I expected to | 19:47 |
| clarkb | it shows the url in the page content which confirms the X value in gitea0X | 19:47 |
| clarkb | 2/3 are done | 19:48 |
| clarkb | now all of the original servers are done. Last one is the new one | 19:52 |
| clarkb | it looks happy too. I think this went well | 19:53 |
| fungi | yeah, all lgtm | 20:23 |
| ianw | another successful upgrade :) | 20:36 |
| opendevreview | Amy Marrich proposed opendev/irc-meetings master: Correction to the Diversity meeting https://review.opendev.org/c/opendev/irc-meetings/+/875985 | 20:38 |
| opendevreview | Merged opendev/irc-meetings master: Correction to the Diversity meeting https://review.opendev.org/c/opendev/irc-meetings/+/875985 | 20:59 |
| clarkb | I'm going to try and work on some zuul reviews this afternoon since this is a good pause point before picking up the gerrit/gitea work again | 21:03 |
| clarkb | fungi: re gerrit/gitea, did you want to review the replication autoload and data dir bindmount changes? | 21:07 |
| fungi | i'm about freed up and can look in a sec | 21:12 |
| fungi | clarkb: approved those and the jdk change too | 21:18 |
| fungi | thanks! | 21:18 |
| clarkb | thank you. I guess once all of those are rolled out we'll want to down then up gerrit to pick them up | 21:20 |
| clarkb | and ya triple checking /home/gerrit2/review_site/data exists and is empty which should make it safe to bind mount as is | 21:21 |
| opendevreview | Merged opendev/system-config master: Enable Gerrit replication autoreload https://review.opendev.org/c/opendev/system-config/+/874340 | 22:21 |
| clarkb | fungi: your virtualpdu email got results. | 22:36 |
| clarkb | I'm logging in to moderate the email through | 22:38 |
| clarkb | I discarded the one to -announce and accepted the one to -discuss | 22:39 |
| opendevreview | Merged opendev/system-config master: Bind mount Gerrit's review_site/data dir https://review.opendev.org/c/opendev/system-config/+/875570 | 22:40 |
| opendevreview | Merged opendev/system-config master: Switch gerrit container from jre to jdk packages https://review.opendev.org/c/opendev/system-config/+/875553 | 22:40 |
| clarkb | after ^ all three of those deploy I think we want to double check the docker compose file has the new bind mount and that the dir is still looking good in review_site. Then we should be ready to do a gerrit restart at any time | 22:41 |
| clarkb | note it needs to be a down up I believe to pick up the changes to the mounts and stuff | 22:41 |
| fungi | clarkb: thanks! and yeah, i set reply-to and mail-followup-to service-discuss on that one | 22:50 |
| clarkb | fungi: related, we end up with a bunch of rxwholesale spam. Are we able to just reject all email from them? Or are we concerne they might one day have a legit reason to email us? | 23:26 |
| clarkb | I know they just want to get a piece of the fungi hair, but I suspect we're happy as is :P | 23:27 |
| fungi | mm2 had a checkbox in the moderation view to automatically add a sender to a "drop from" list, but mm3 seems to not have that implemented. i think we can manually add that address in the list config though | 23:28 |
| clarkb | ok the bind mount change appears to have done what I expected in docker-compose.yaml and the replication autoreload chnage too. Both will need gerrit to be down'd them up'd to take effect which will also pull in the jdk packgae change | 23:33 |
| clarkb | the gate is pretty busy right now so I'm inclined to wait on that for a bit | 23:33 |
| clarkb | (release time for openstack so being careful to not create big impacts is a good thing) | 23:33 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!