Tuesday, 2023-04-11

clarkbTC meeting distracted me but it is now time for our meeting19:02
ianwo/19:02
clarkb#startmeeting infra19:02
opendevmeetMeeting started Tue Apr 11 19:02:26 2023 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.19:02
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.19:02
opendevmeetThe meeting name has been set to 'infra'19:02
frickler\o19:02
clarkbAs mentioned yesterday I didn't expect this to be well attended but did want to write down some of what has happened recently19:02
clarkb#link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/LC5RWSADKSROICHN7YZKPOZX4AV55N34/ Our Agenda19:02
clarkb#topic Announcements19:04
clarkb#link https://meetpad.opendev.org/opendev-contributor-bootstrap-202304 New Contributor Onboarding Wednesday April 12, 2023 at 14:00 UTC19:04
clarkbWe've had at least two people interested in contributing and I'll have a bootstrapping session on meetpad tomorrow morning (for me)19:04
clarkbOthers are welcome to join if interested. Definitely not limited to those who have already chimed in19:05
* frickler plans to join19:05
ianw++ i've mentioned to a few others too19:05
clarkb#link https://etherpad.opendev.org/p/opendev-contributor-bootstrap-202304 Add your thoughts and ideas here19:05
clarkbthats a link to the etherpad we'll use for ease of use if you have anything to add to the agenda19:05
fricklerdo we want to order that a bit? looks pretty overwhelming to me19:06
clarkbyes, my idea was to go over basics like gerrit and zuul and ensuring everyone knows how to communicate, push changes, join the meeting etc. Then dig into the ideas for contributions. Grouping the contribution ideas by interest is probably a good idea19:07
fricklerlike select some (<5 ?) most important topics?19:07
fricklerthe getting started section is fine, but some more structure to the other ideas19:07
ianwyes, sorry that's probably largely my fault dumping a lot of todo things 19:07
clarkbI hesitate to prioritize on our side as I think that might discourage people who are interested in other things. More like here is some CI related stuff, here is some container related stuff, here is some something else stuff?19:08
clarkbor maybe just reorder so that the first things you see are priorities and then the list can go on if you don't find something that interests you to start19:08
clarkbI'll have to look at it more closely since it got updated19:09
ianwyes i agree, i think the point is that doing any one thing is a way into contributing, where you'll start to see the bigger picture19:09
clarkb++19:09
clarkbI'll take a look at organizing it a bit more after lunch today19:10
ianweven zuul/zuul-jobs things expose you to a lot of the mechanics of ci, etc.19:10
clarkbI do think we've got a good set of tasks that don't require root involvement. And some that do as things progress19:10
ianwi think the question of where we keep these backlog items is a good one19:11
ianwi don't think we have a good answer atm19:11
clarkbianw: ya I was thinking a bit about that after the storyboard question yesterday19:11
clarkbI think there are options but nothing is used consistently today. (options include wiki, etherpad, less formal section of the specs repo, etc)19:12
clarkbok lets continue with the other items on the agenda. Looking forward to seeing people tomorrow19:12
clarkb#topic Migrating to Quay19:12
clarkbAt last check all of the outstanding changes to update jobs merged. However, I think corvus may still have one to implement the intermediate registry promotion path that isn't quite ready?19:13
corvusyep that's on my list19:13
clarkbI think ianw and corvus were also working on testing the updates that had landed using zuul/zuul-client.19:13
clarkbcorvus: ianw: is there anything the rest of us can do at this point to help?19:14
ianwthat does work now with zuul-client, so that path of pushing and re-tagging via the api is ok19:14
corvusalso, just a friendly reminder from the last topic... there's some zuul stuff on there which is great, but keep in mind that zuul is an independent project and opendev's task list is kind of a separate thing :)19:15
clarkbyes, I see those as tasks that would benefit opendev and we would work on anyway19:15
clarkbbut that would be contribution to zuul that aids opendev not direct contribution to opendev19:16
corvusand yep, zuul-client is looking good with that container build path19:16
corvusso i think the intermediate registry thing is the next step; i should have time for that this week.19:16
corvusclarkb: yep.  95% of what i see there matches that.  a few of those are... kind of zuul-project level work.19:17
clarkbexcellent re intermediate registry stuff.19:17
clarkbLet me know if I can help with reviews etc19:17
corvusi don't think we need to get into specifics, but if we're indoctrinating new contributors, it might be good to highlight the separation.19:17
clarkbcorvus: can do19:17
clarkbAlso worth noting that April 14 (Friday) was the original you can't push to these orgs day on docker hub. They have said they will not do that anymore but we should keep our eyes open for anything unexpected19:18
clarkb#topic Bastion Host Updates19:19
clarkb#link https://review.opendev.org/c/opendev/system-config/+/879388 Fix rax rdns19:19
clarkbThis change is related to launching new servers recently and could use a second review if anyone has time19:19
clarkbAlso ianw you reported the fixes to reinstalling the launch venv have rax volume listing working now?19:19
ianwyeah, it has now updated itself to the latest everything19:20
ianwi think that's probably appropriate for this venv; as it is a bit of a dogfood for communicating with all our various clouds19:21
ianwsorry, i mean to say, it updating itself periodically is probably appropriate ....19:21
clarkbyup and we've identified specific versions of things where necessary too19:22
clarkbwhich pip should respect19:22
clarkbAnything else bridge related?19:23
ianwnot really; the backup stuff still hasn't had a lot of review, but i don't expect movement on that for a bit19:24
clarkbeveryone has been taking vacation :)19:24
clarkb#topic Mailman 319:24
clarkbrelated to my last comment I don't think there is any movement on this either. I expect fungi should be able to get to it once he returns19:25
clarkb#topic Gerrit Upgrade and Project Renames19:25
clarkbWe upgraded Gerrit and renamed three projects last Thursday/Friday (depending on timezone). As far as I can tell this went well though we did need almost all of our alotted time19:25
clarkbOne thing Gerrit did with its notedb migration was update all labels to have a copyCondition including changekind:NO_CHANGE19:26
clarkbI feel like this was a bit over the top and filed a bug about it. But haven't seen any resposne from upstream on whether or not they think it is a bug19:26
clarkbfungi reverted this update in All-Projects but we had updates to all of the labels in project specific configs too that we need to evaluate19:27
clarkbAdditionally gerrit updated the project specific config files to use hard tabs in the config file that our normalize config script removes. I think we should actually match what gerrit wants as much as possible to keep diffs as small as possible in situations like this19:27
ianwyeah i need to finish that audit, sorry, got distracted19:27
corvuswhat's wrong with it? (why not leave it?)19:27
clarkb#link https://review.opendev.org/c/openstack/project-config/+/879906 Gerrit also normalized indentation of config files we should consider this to be in sync19:27
ianwi still don't understand fully how a series of client events leads to a NO_CHANGE event being matched19:28
clarkbcorvus: two things. The first is that many of our copyConditions already include changekind:TRIVIAL_REBASE which is a superset of NO_CHANGE so it is confusingly redundant19:28
corvusit=copycondition19:28
clarkbcorvus: the second is that for Verified and Workflow we explicitly had no copyConditions and I don't think it is appropriate for Gerrit to add any copy conditions in those cases.19:28
corvusgot it, thx.19:29
clarkbWe may decide to add copyConditions to Verified and/or Workflow but that should be done by users not automation19:29
clarkbianw: ya I don't think we need to rush on this as I suspect in projects specific configs it is rare to lack a copycondition19:29
ianwi think it may relate to merge changes19:30
corvusand yeah, that sounds potentially dangerous for project-specific "core approval" types of labels too.  :(19:30
corvusis there some kind of "copycondition:never" that we should look for that might prevent this?19:31
ianwthere isn't afaik19:31
ianw... but yeah, i think fully understanding the conditions that lead to NO_CHANGE might help evaluate the impact19:31
clarkb++19:31
clarkband if we do bulk updates we should consider bundling in a hard tab change along with that19:32
ianw"Matches when a new patch set is uploaded that has the same parent tree, code delta, and commit message as the previous patch set."19:32
ianwi'm not sure how you do that, because i feel like gerrit usually rejects that as "nothing changed"?19:32
clarkbianw: it has and hasn't depending on the version. Maybe it won't reject it today anymore? Might be worth testing that19:33
clarkbIn any case doing an audit of what changed and understanding the situation when NO_CHANGE matches is the first step then we can plan any updates19:33
ianw++19:33
corvuscopycondition: "before:1970-01-01"19:33
clarkb#link https://groups.google.com/g/repo-discuss/c/kkbyIKkzlAU/m/4t789yerBAAJ new usp query parameters19:34
corvus(^ amusing way to get a copycondition that never matches?)19:34
clarkbThis is something frickler called out today. Gerrit is adding query strings to change urls to annotate where the link originated from. I've sent email asking them to clarify how this is actually used19:34
clarkbAs far as positives go the web ui now supports bulk actions (something gertty did first). Comments support markdown syntax.19:35
corvusperhaps used by google's gerrit and they expect everyone else to ignore it / don't really care19:35
clarkbSome of the diff annotations are nice too giving you a better indication of what is changing if not just file content19:36
corvusthey should do threaded change relationships next :)19:36
clarkb++19:36
clarkbThe only other next step I'ev currently got in mind is cleaning up the 3.6 image and adding 3.8 images and upgrade testing when we are ready19:37
clarkbThank you to everyone who helped, test, review, upgrade, fix etc during this process19:38
clarkb#topic Upgrading Old Servers19:39
clarkb#link https://etherpad.opendev.org/p/opendev-bionic-server-upgrades Notes19:39
clarkbstatic02 should be serving all of the static.o.o and related content now. I used the LE cert config to determine what all needed updating and think I've got that all changed at this point19:39
clarkbyesterday I shutdown apache2 on static01 as a sanity check. corvus found that zuul-ci.org was struggling this morning possibly due to stale dns pointing at static01? So I'll leave static01 alone for another day or two before cleaning it up to be sure there aren't any gremlins in the works19:40
clarkbOne thing to note is that Jammy's openafs-client package is newer than our PPAs which means static02 is using the upstream distro package version19:40
clarkbI've also got an etherpad02 in the works which should be deployed now with an empty db. I'll need to sort out an etherpad downtime and db migration and dns update soon (hopefully tomorrow)19:41
clarkbplease don't put anything you want ot persist in etherpad02's db as the db migration from 01 will dump what is in 0219:42
clarkbianw: nameservers are the other push on this item. Anything new to report for them?19:42
ianwno but soon i hope!19:42
clarkbthanks, let me know if I can help. This generally has become relatively high priority for me now that docker doom is cancellled/delayed19:43
clarkb#topic AFS volume quotas and utilization19:43
clarkbI've been keeping an eye on this and there hasn't been any dramatic change but he trend is up slowly19:43
clarkbI know that with osme of the wheel mirror cleanup work ideas we will delete content from AFS19:44
clarkbAnd also that fedora 36 mirroring will go away soon19:44
clarkbI think that those are good next steps. But please if you see something going sideways more quickly say something as we might need to intervene in different ways19:44
ianwi also have about 20gb of wheels to remove when i get back to it19:45
clarkbyup I think the current status/plan is good I jsut want to keep this on the agenda in case anyone has specific concerns or wants to tackle this in a different way. I'm not hearing anything like that so we can probably move on19:46
clarkb#topic Gitea 1.1919:46
clarkb#link https://review.opendev.org/c/opendev/system-config/+/877541 Upgrade opendev.org to 1.19.019:46
clarkbThis change is ready I think. There is a held node for it too if anyone else wants to look. But it looks like 1.19.1 is going to happen very soon judging by their issue tracker's milestone for 1.19.119:47
clarkbI feel like we have enough going on and there isn't a presssing reason to upgrade quickly so waiting for the bugfix version is a good idea19:47
clarkbThat said I don't expect the change we land to upgrade to differ much between 1.19.0 and 1.19.1 if you have time to review the existing change that may be a good way to get ahead of things19:48
clarkb#topic Quo vadis Storyboard19:48
clarkbAs expected a number of projects decided to move to launchpad at the PTG. You may see openstack/project-config changes to update their projects.yaml metadata to reflect this19:49
clarkbI think we can also continue to encourage projects to work together if they end up building tools to migrate content or similar.19:49
clarkbOh also we can set projects read only in storyboard via admin config stuff in the storyboard UI (and likely API) should we get requests for that19:50
clarkb#topic Open Discussion19:50
clarkbanything else?19:50
clarkbAs noted previously my priorities this week are largely static02, etherpad02, and then supporting the work everyone else has going on.19:51
clarkbI'm going to attempt to take Friday off as my kids are out of school Thursday and Friday. We'll see how that goes19:51
clarkbSounds like that may be everything. Thank you everyone! We'll be back here same time and location next week.19:54
clarkb#endmeeting19:54
opendevmeetMeeting ended Tue Apr 11 19:54:47 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)19:54
opendevmeetMinutes:        https://meetings.opendev.org/meetings/infra/2023/infra.2023-04-11-19.02.html19:54
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/infra/2023/infra.2023-04-11-19.02.txt19:54
opendevmeetLog:            https://meetings.opendev.org/meetings/infra/2023/infra.2023-04-11-19.02.log.html19:54

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!