19:01:22 #startmeeting infra 19:01:22 Meeting started Tue Dec 13 19:01:22 2022 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:22 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:22 The meeting name has been set to 'infra' 19:01:44 #link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/6DVJVHEPMASFL4PQWIQZFUT3H6T5BXP5/ Our Agenda 19:01:56 #topic Announcements 19:02:36 I didn't have anything to announce other than what ended up in our agenda 19:02:40 anything else to make note of? 19:03:06 this meeting does conflict with a world cup semifinal game 19:03:34 will you be able to announce the winner at the end of the meeting? 19:03:54 no the game is at least 90 minutes long and we only have 60 minutes here 19:04:05 ok lets dive right in then 19:04:11 #topic Bastion Host Updates 19:04:39 #link https://review.opendev.org/q/topic:prod-bastion-group this is the lower priority stack for parallel runs. Ideally landed whe nwe can observe carefully 19:04:54 #link https://review.opendev.org/q/topic:bridge-backups tooling to backup content in encrypted tarball 19:05:02 #link https://review.opendev.org/c/opendev/zone-opendev.org/+/867540 bridge CNAME 19:05:17 and this last change came out of a reminder to corvus today not to run the zuul restart playbook from old bridge 19:05:38 I think we are really close to being able to shutdown old bridge but suspected that the backup tooling might want to be used on old bridge as a final wrapup? 19:06:07 I'll try to find time to revie wthose but anytime gpg and encryption is involved I like to take my time and do my best to actually understand it 19:06:41 ianw any other thoughts on what our next steps here should be? 19:07:01 sure -- yes i can probably shutdown the old bridge at this point, it's just been waiting on finalising the other changes 19:07:15 if infra-roots would like to confirm that they have everything they want from their ~ we can shut it down 19:07:51 I think I did look at that and then left notes on the etherpad about things like the dns stuff and as far as I know all of that has been addressed. But I'll double check 19:08:00 and I guess we can shutdown before deleting too 19:08:18 i don't need anything from ~corvus; "delete at will" from me 19:08:23 yep no need to delete straight away, but also probably good to not have it lying around once we've moved on 19:08:55 ++ 19:09:21 * clarkb makes a note to take a look (probably after the afternoon school run) 19:09:27 anything else? 19:09:53 i've cleaned out my homedir on old bridge 19:10:31 no that's it, thanks 19:10:52 #topic Mailman 3 19:11:11 I think all of the oustanding change srelated to the mm3 migration have landed at this point? 19:11:33 yes 19:11:33 I was curious if we know if the site_owner update has been applied. I believe that would require a container restart and I'm not sure if that happened 19:11:44 we haven't restarted the container yet, i don't thinkl 19:11:53 fungi: ^ you might know or be able to check? I think you look in /etc/mailman/mailman.conf in the core container 19:12:17 also i noticed that the top-left corner of https://lists.zuul-ci.org/archives/ says lists.opendev.org, not sure yet where to configure that 19:12:19 ok I think that is worth following up on and restarting if it hasn't happened yet just to ensure we don't get bitten by that later if something is wrong 19:12:32 fungi: the thing I noticed about that is the link itself has the correct target 19:12:43 right, it's just the name apparently 19:12:47 fungi: for this reason I suspect this is a bug in mailman itself where they apply the value to the target but not the rendered name 19:12:59 might just need to stick the same string in both locations of the template or something 19:13:51 apparently also the page says "Available lists - lists.opendev.org" 19:14:34 <fungi> but as far as that hyperlink goes, it's relative which i guess is why it doesn't go to the wrong site 19:14:38 <fungi> looks like this: 19:14:40 <fungi> <a class="navbar-brand" href="/archives/">lists.opendev.org</a> 19:14:42 <clarkb> ah 19:15:07 <clarkb> probably need to dig up the source for that and take it from there. It might be missing config or it may be buggy vhost handling 19:15:10 <fungi> so i expect there's a site-specific name we have to give hyperkitty somewhere 19:15:23 <fungi> i didn't notice postorius doing it 19:15:49 <clarkb> I will say as a user I find this much easier to use 19:15:59 <clarkb> on the whole I think it is a big improvement even with the small issues like that 19:16:06 <corvus> https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/UAHC7L2M3KCKQXRQY3QCZZEQFGK4L4PK/ 19:16:52 <fungi> oh, though i see that the page <title> for e.g. https://lists.zuul-ci.org/mailman3/lists/ says "List Index - lists.opendev.org" 19:17:02 <fungi> so i guess postorius needs it somehow too 19:17:12 <clarkb> sound slik ewe need to look at the django site admin 19:17:20 <fungi> seems likely 19:17:25 <clarkb> (that should work, we interacted with it on the test node) 19:17:38 <corvus> postorius also uses site_name for the title https://gitlab.com/mailman/postorius/blob/master/src/postorius/templates/postorius/base.html#L10 19:17:50 <corvus> so at least, that's a thread to pull on :) 19:17:55 <clarkb> ++ good find 19:18:19 <clarkb> fungi: there should be a django admin user set as part of the ansible vars and you can use that to login via the admin url iirc 19:19:11 <fungi> yeah, wondering if this is something we should be setting via an api on creation or just accept that we'll have to manually edit it for each site 19:19:25 <clarkb> unfortunately the docs aren't so great for details like this 19:19:39 <clarkb> but we can usually work backward fro mthe expected process and figure it out 19:20:31 <clarkb> anything else? 19:20:39 <corvus> (or is it something we can set with proxy headers?) 19:22:03 <clarkb> We can dig into this more after the meeting. There are a few other topics I want to cover so we'll continue and can swing back to this later if necessary 19:22:13 <clarkb> #topic Quo vadis Storyboard 19:22:26 <clarkb> #link https://lists.opendev.org/pipermail/service-discuss/2022-October/000370.html 19:22:29 <clarkb> no new movement on this 19:22:43 <corvus> status quo vadis 19:23:20 <clarkb> I've got a number of writing exercises on my todo list and am thinking maybe I/we should just start drafting a plan in etherpad since feedback on the list isn't happening? 19:23:34 <clarkb> (that might actually mean drafting a couple of possible proposals and then choosing one) 19:24:36 <clarkb> there are no good options so we should maybe write them down in enough dtdail to make the least painful decision 19:25:53 <clarkb> no one is screaming, I'll take that as the next step here and pin th etodo to my list 19:26:57 <clarkb> #topic Gerrit 3.6 19:27:12 <clarkb> ianw upgraded Gerrit from 3.5.4 to 3.6.3 yesterday 19:27:43 <ianw> well i pushed the buttons but everyone, mostly zuul, helped get it there :) 19:27:44 <clarkb> seems to have gone well. The only issues I've seen so far are mostly around the UI changing 19:28:14 <clarkb> The main thing tripping people up seems to be how Gerrit renders votes 19:28:35 <clarkb> in particular it summarizes votes and doesn't put names next to them in the requirements section (it does put them in the reviewers section so you have to look elsewhere) 19:28:54 <clarkb> it also seems to only show required labels for merging in the change listing pages 19:29:13 <clarkb> this means review-priority and similar type sof labels that openstack use don't show up in the list summaries 19:29:30 <corvus> because they're not submit-requirements, yeah? 19:29:41 <clarkb> corvus: yes they are purely informational/trigger requirements 19:29:51 <clarkb> that seems to be the criteria for including them in the UI listings 19:30:22 <clarkb> it also looks like change owners might be able to delete positive votes on their changes. This doesn't seem harmful but may create confusion at some point 19:30:40 <corvus> is something like a submit requirement that accepts any or no value a possible workaround? 19:30:51 <ianw> the summaries is maybe worth raising? although the answer might be "make a dashboard, it works better anyway"? 19:31:09 <clarkb> corvus: yes that was one suggestion from fungi. I think we have options 19:31:20 <clarkb> ianw: the problem with dashborad is they don't show up thre either so it can be confusing 19:31:31 <clarkb> so you need a dashboard with lists for each possible vote value 19:31:52 <clarkb> I don't know how to make a submit requirement that accepts any value but if we can express that that may be the best option 19:31:57 <ianw> yeah, i meant a separate section like "changes with backport-candidate" or whatever. but yeah, it's not mixed in 19:32:28 <corvus> looks like a non-submit-requirement label is called a "trigger vote"? 19:32:36 <clarkb> corvus: yes these are trigger votes currently 19:32:42 <ianw> i wonder with that approach though, will we ever see anything on the overview page that isn't a grey, crossed circle? 19:33:23 <ianw> because our submit requirements are really not the same as the idea of "things that need to be done before you can push the merge button", if that makes sense 19:33:47 <ianw> because we never push the merge button, zuul does. so we're "ready to submit" when we have a +2 vote and +1 verified 19:34:04 <ianw> the +w is our "merge button" 19:34:18 <clarkb> ya ithink a followup thing might be to have a conversation upstrema about making this directly configurable somehow? like an annotation in the label definition? 19:34:39 <clarkb> then we can more directly express how our labels are used without assumptions based on upstrea mgerrit workflows 19:35:41 <ianw> does anyone happen to have a screenshot of the old layout, perchance? 19:35:49 <corvus> if the suggestion is that we would like to avoid having Verified+2 be a submit requirement, i don't agree with that 19:35:50 <clarkb> I don't, but our CI will take them iirc 19:36:02 <clarkb> corvus: no i think it is about how we render the requirements 19:36:18 <clarkb> you would still need a +2 but we can treat a +1 as "ready for zuul" 19:36:23 <corvus> okay. i may not be understanding where ianw was going with the merge button thing then. 19:36:27 <fungi> but we might like being able to avoid having "review-priority" as a submit requirement 19:37:09 <clarkb> corvus: the way they render submit requirements today is built around trying to show you what oyu need to click the submit button. But in our system we would ideally render it to show what is necessary for zuul to gate it 19:37:10 <ianw> coruvs: i'm just saying that i think the idea of the new column is to show you "things that are ready to merge". but it will never really show anything but a grey circle for us 19:37:39 <clarkb> ianw: and that specifically workflow 19:37:47 <corvus> fungi: yes. my only suggestion there was maybe a submit requirement can have a "satisfied by no vote" to work around the fact that it's not displayed there. that's all. no idea if that's possible, just thought it worth looking into. 19:37:49 <clarkb> because its usually set to 0 until set to +1 to trigger gating 19:37:58 <corvus> ianw: ah you're talking about the status column 19:38:10 <corvus> ianw: that is interesting and unfortunate 19:39:49 <ianw> perhaps to move it on, i can come up with a draft email, or maybe two really -- one about the status column -- one about the other flags -- and we can revise it between us and send to gerrit for feedback? 19:39:58 <clarkb> ++ 19:40:08 <fungi> sounds great 19:40:42 <ianw> ok, will do something up in an etherpad and let people know 19:40:48 <corvus> (we could, conceivably, lift the verified+2 requirement and rely entirely on zuul, but i think it's better to express the requirements in gerrit itself) 19:40:57 <corvus> ++ 19:41:14 <clarkb> thanks. And thank you to everyon ewho made the upgrade happen and happen so smoothly. Now we need to look at 3.7 which might be more complicated :? 19:41:24 <clarkb> and ya lets move on we have 19 minutes left and a couple more things I'd like to cover 19:41:30 <clarkb> #topic Tox v4 19:42:00 <clarkb> Last week tox released 4.0.0 and that very quickly broke all the things. This version is a rewrite and isn't backward compatible 19:42:17 <clarkb> to stop the bleeding we pinned the tox install to <4 in zuul-job' ensure-tox 19:42:24 <clarkb> but the plan is to remove that cap next week 19:42:55 <clarkb> What this means is we should be prepared to update our tox.ini files to be compatible with v4 and v3. 19:42:57 <clarkb> #link https://review.opendev.org/c/opendev/system-config/+/867269 system-config tox v4 compat 19:43:09 <clarkb> A change for openstack/project-config has already landed 19:43:38 <clarkb> Most of the issues seem to be related to whitelist_externals becoming allowlist_externals and unlisted commands becoming errors instead of warnings 19:43:51 <clarkb> neutron is running into a new issue where the project isn't installed to the venv for some reason that will need to be sorted out 19:44:10 <clarkb> so far though with zuul-jobs, project-config, and system-config it hasn't been too bad 19:44:12 <fungi> seems to be ignoring the usedevelop option 19:46:03 <clarkb> so ya reviews and changes welcome to help get ahead of that. But I'm sure we'll find problems once we uncap too 19:46:16 <clarkb> #topic Winding down iweb 19:46:48 <clarkb> Earlier this year we were told the iweb cloud would be going away. We thought it might initially go away mid year but they kept it running until the end of the year 19:47:06 <clarkb> we are at the end of the year now and need to shut it down. Ideally in a graceful way for nodepool to avoid manual cleanup 19:47:14 <clarkb> link https://review.opendev.org/q/topic:wind-down-iweb 19:47:28 <clarkb> I've got a stack of changes that should do the bulk of this work. We will need to manually delete the mirror node too 19:47:52 <clarkb> I'm happy for those to land anytime between now and about this time next week. After that my ability to pay attention will be decreased 19:48:09 <clarkb> Once we've done the cleanup I'll send them an email saying we've moved out 19:48:28 <ianw> ++ 19:48:50 <ianw> hopefully avoid end of lease cleaning fees :) 19:48:55 <clarkb> so ya reviews welcome and I'll do my best to get that cleared out gracefully 19:49:24 <clarkb> Due to time constraints I'm going to skip the vexxhost rescue instance and bionic upgrades topics. There isn't much new for them as far as I know 19:49:30 <clarkb> #topic End of year Holiday party 19:50:00 <clarkb> corvus: suggested last week that our December 20 meeting coul dbe treated more like a holiday party. I like this idea. We've spent a ton of time this year running opendev we can take an hour to have some fun :) 19:50:26 <clarkb> One idea I've got is we can play games on board game arena if people are interested. You do need to create a free account but a paid account can host games that include free tier players 19:50:38 <clarkb> I'm happy to upgrade to a paid subscription for this and do the hosting 19:50:43 <corvus> ☃️ 19:50:45 <clarkb> But I'm also open to other ideas 19:51:06 <clarkb> https://en.boardgamearena.com/ sorry board game erna 19:51:07 <corvus> i'm in 19:51:29 <clarkb> we can use meetpad for voice and play ticket to ride or carcassone etc 19:51:43 <ianw> ok, wow i'd never heard of that site! i'm surprised they have licensed so many games 19:52:16 <fungi> it's quite an amazing collection of games they've implemented, yes 19:52:28 <fungi> excellent for when you want to remotely play board or card games with friends 19:52:29 <clarkb> the ui can be clunky at times but a lot of the games are actually really well done 19:52:46 <ianw> it looks like my games cupboard online, haha 19:52:53 <fungi> i was thinking of upgrading my account to paid as well 19:54:06 <clarkb> I haven't started on the annual report content Ineed to write for 2022 yet, but it was a busy year just off the top of my head. So ya thank you everyone and lets have som efun next week :) 19:54:21 <ianw> ++ 19:54:40 <corvus> clarkb: thanks! 19:54:46 <clarkb> I'll send out an agenda email but give details on where to sign up and what meetpad room we'll use 19:55:01 <clarkb> #topic Open Discussion 19:55:12 <clarkb> Anything else before our hour is up? 19:56:42 <clarkb> I'll probably be floating around after the 25th but attempting to focus on writing those annual report things 19:56:52 <clarkb> I expect things will be quiet as they usually are 19:57:24 <clarkb> I hope everyone else is able to enjoy some down time too 19:57:44 <clarkb> It might snow here next week too. I'll have to get the sleds down 20:00:03 <clarkb> sounds like that is it. Thank you everyone! 20:00:06 <clarkb> #endmeeting