| opendevreview | Radosław Piliszek proposed openstack/project-config master: Drop NebulOuS Zuul tenant https://review.opendev.org/c/openstack/project-config/+/921725 | 06:26 |
|---|---|---|
| opendevreview | Radosław Piliszek proposed openstack/project-config master: Drop NebulOuS Zuul tenant https://review.opendev.org/c/openstack/project-config/+/921725 | 07:04 |
| *** elodilles_pto is now known as elodilles | 07:26 | |
| frickler | infra-root: while doing the pypi maintainer cleanup, I found https://pypi.org/project/glean/ and am wondering whether we also should drop sdirector==mordred from the maintainer list while at it, opinions? | 08:34 |
| frickler | a similar question would seem to hold for the tact sig repos like dib, which the TC seems to have ignored so far, too (cc gouthamr) | 08:57 |
| amorin | hello! Quick question about bindep: should a CI job run bindep before any other command (like tox) to figure out which packages are missing and eventually pipe the result to apt-get install? | 10:00 |
| *** benj_4 is now known as benj_ | 11:35 | |
| frickler | amorin: depends on the specific job definition I'd say. for unittest jobs (since you mention tox), the role is called here https://opendev.org/zuul/zuul-jobs/src/branch/master/playbooks/unittests/pre.yaml | 11:46 |
| fungi | amorin: but generally yes, using bindep to determine what additional system packages to install on the test node means that any commands after it can be supplied by packages installed as part of the job definition rather than relying only on preinstalled tools (of which there aren't many) | 12:32 |
| fungi | but at least doing it before any packages are installed from pypi means you can install system libraries and headers/toolchain needed to support installing python sdists that include c extensions which have to be built during installation | 12:33 |
| Guest8493 | frickler: no reason for me to be in the glean pypi list. That should be all zuul published | 12:39 |
| tonyb | Looks like infra-prod-run-cloud-launcher ran successfully overnight \o/ | 12:59 |
| amorin | ack, thanks fungi | 13:05 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Initial dump or mediawiki role and config https://review.opendev.org/c/opendev/system-config/+/921322 | 13:05 |
| fungi | frickler: glean shouldn't be delegated to the tact sig, it moved under control of the opendev sysadmins when we created the new namespace. dib is still in the openstack namespace and under the tact sig though, yes | 13:06 |
| *** ykarel_ is now known as ykarel | 13:10 | |
| fungi | just a heads up, login.ubuntu.com is down right now (503 service unavailable), so logins to our systems relying on it aren't working for the moment (launchpad either), not sure how long it's been down | 13:28 |
| fungi | i've already given the on-duty contact in #launchpad a heads up | 13:28 |
| fungi | and got redirected to #canonical-sysadmins where i then also reported it | 13:37 |
| fungi | if the situation persists much longer, i'll #status notice about it | 13:37 |
| fungi | sajoupa says "there's an issue currently, we're working on it" | 13:40 |
| opendevreview | Merged zuul/zuul-jobs master: Add a job for publishing a site to netlify https://review.opendev.org/c/zuul/zuul-jobs/+/739047 | 13:45 |
| fungi | awesome to see that progressing | 13:46 |
| fungi | Guest8493: any chance you've tried out the netlify cms integration for gerrit? | 13:49 |
| fungi | possible it's bitrotted in the 5 years since the poc was built though | 13:53 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Testing if stop_grace_period is valid with compose v2 https://review.opendev.org/c/opendev/system-config/+/921764 | 13:57 |
| fungi | and login.ubuntu.com seems to be back in working condition again | 13:59 |
| frickler | fungi: I didn't claim that glean belongs to the tact sig. the question rather is whether we as opendev admins should apply the same cleanup to pypi collaborators that openstack is doing | 14:05 |
| frickler | and then the second question is whether the cleanup should also apply to repos under the tact or other sig's governance | 14:06 |
| fungi | i suspect the intent of the tc decision was to apply to all packages of releases for official openstack repositories whether they're under project teams or sigs. as for things outside openstack, the tc doesn't really have jurisdiction there but as opendev sysadmins we could decide to do the same for our own repos, yes | 14:08 |
| fungi | it probably warrants some discussion first, of course | 14:08 |
| fungi | since when it came up in the opendev meeting that openstack was going to do that, there was some concern raised about lack of fallback accounts | 14:08 |
| frickler | well that might get easier once organizations are fully enabled for pypi | 14:11 |
| fungi | perhaps, but that effort has stalled for... i don't know... two years so far? | 14:16 |
| fungi | i think psf is hoping to hire someone to pick up that work | 14:18 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add configuration for new OpenMetal cloud https://review.opendev.org/c/opendev/system-config/+/921765 | 14:18 |
| opendevreview | Riccardo Pittau proposed openstack/diskimage-builder master: Fix uninstall gentoo packages https://review.opendev.org/c/openstack/diskimage-builder/+/921769 | 14:41 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add noble repo files https://review.opendev.org/c/opendev/system-config/+/921770 | 14:42 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Test mirror services on noble https://review.opendev.org/c/opendev/system-config/+/921771 | 14:42 |
| clarkb | tonyb: I see yuo've checked off the openmetal account creation and secrets stuff. thank you for doing that. Next up is adding the cloud to clouds.yaml files in system-config (and removing inmotion content) and then running cloud launcher against it. I can work on that after meetings and the dentist today if no one beats me to it | 14:43 |
| tonyb | clarkb: 921765 should be the first part of that | 14:44 |
| tonyb | I expect that merging that will also run the cloud-launcher | 14:45 |
| clarkb | no cloud launcher needs content updates in the bastion.yaml group vars file | 14:46 |
| clarkb | frickler's comments all look correct to me except I haven't verified the region name yet. And yes regions are case sensitive iirc | 14:46 |
| clarkb | I've ammened the names used in the etherpad for nodepool resources and the mirror to use iad3 instead of us-east. I think this is required for the mirror to match the exact (however case insensitive here because dns is case insensitive) region name | 14:48 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add configuration for new OpenMetal cloud https://review.opendev.org/c/opendev/system-config/+/921765 | 14:55 |
| tonyb | Okay | 14:56 |
| clarkb | I think that will fail ci (and the parent patchset did too for the same reason?) because we have a set of fake passwwords and things for all those clouds.yaml entries that need openmetal entries too? a git grep on some of the other variable names should pop them up | 14:58 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Test mirror services on noble https://review.opendev.org/c/opendev/system-config/+/921771 | 14:58 |
| Guest8493 | <fungi> "mordred: any chance you've tried..." <- Not since they did the poc back in the day. I haven't touched netlify cms in about 4 years... Tried it for a brief time to see if Sandy and I could collaborate on the website that way, but ended up just giving her visual studio code directly | 15:05 |
| fungi | makes sense | 15:06 |
| tonyb | rmgilmour: Hello :) | 15:12 |
| rmgilmour | hello :) | 15:12 |
| Guest8493 | I've been very pleased with them as a static site hosting platform though | 15:14 |
| roslyn | hello | 15:16 |
| roslyn | hello Ethan | 15:18 |
| ethan | Hello | 15:18 |
| Guest8493 | what are we using these days for the gerrit messages in zuul matrix? I've forgotten that piece of the puzzle | 15:22 |
| Guest8493 | ah - matrix-gerritbot. Found it :) | 15:23 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Testing if stop_grace_period is valid with compose v2 https://review.opendev.org/c/opendev/system-config/+/921764 | 15:27 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Testing if stop_grace_period is valid with compose v2 https://review.opendev.org/c/opendev/system-config/+/921764 | 15:28 |
| opendevreview | Monty Taylor proposed opendev/system-config master: Report Zuul events for WanderTracks repos https://review.opendev.org/c/opendev/system-config/+/921776 | 15:30 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add golang based docker compose tool. https://review.opendev.org/c/opendev/system-config/+/920760 | 15:31 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add an opendev specific build of mediawiki https://review.opendev.org/c/opendev/system-config/+/921321 | 15:31 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Initial dump or mediawiki role and config https://review.opendev.org/c/opendev/system-config/+/921322 | 15:31 |
| tonyb | Okay, how do I add a noble build to the openafs PPA? | 15:46 |
| clarkb | tonyb: I think you have to download the source package, update it's metadata as appropriate then push it back. IIRC ianw wrote docs on this in system-cofnig somewhere. That said you may not need to if the openafs version in noble is a reasonable version. iirc we use the distro openafs version on jammy | 15:49 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add configuration for new OpenMetal cloud https://review.opendev.org/c/opendev/system-config/+/921765 | 15:54 |
| clarkb | tonyb: that looks better, but also need to update the nodepool clouds.yaml in a similar way. I left notes for that | 15:57 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Testing if stop_grace_period is valid with compose v2 https://review.opendev.org/c/opendev/system-config/+/921764 | 16:01 |
| clarkb | as a reminder I'm popping out to visit the dentist in about half an hour. I should be back to run the meeting. Will try to ping via my matrix bridge client if that ends up not being the case | 16:08 |
| Guest8493 | do we have jobs anymore that propose changes back to gerrit? | 16:11 |
| clarkb | Guest8493: yes, openstack's translation jobs still do. | 16:11 |
| fungi | clarkb: thanks, happy to chair the meeting if necessary | 16:11 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add configuration for new OpenMetal cloud https://review.opendev.org/c/opendev/system-config/+/921765 | 16:13 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add configuration for new OpenMetal cloud https://review.opendev.org/c/opendev/system-config/+/921765 | 16:40 |
| tonyb | clarkb: I guess I/we really only need an empty ppa for noble? Basically I need to resolve: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_986/921771/2/check/system-config-run-mirror-x86/986b1dd/bridge99.opendev.org/ara-report/results/333.html#msg | 16:44 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Initial dump or mediawiki role and config https://review.opendev.org/c/opendev/system-config/+/921322 | 16:51 |
| * tonyb relocates to coffee shop | 16:57 | |
| opendevreview | Albin Vass proposed zuul/zuul-jobs master: DNM: Debug fetch-subunit-output https://review.opendev.org/c/zuul/zuul-jobs/+/921781 | 17:01 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Testing if stop_grace_period is valid with compose v2 https://review.opendev.org/c/opendev/system-config/+/921764 | 17:56 |
| Clark[m] | tonyb: or maybe we make noble skip setting up the ppa | 17:57 |
| tonyb | Clark[m]: sure. I can do that too. | 17:58 |
| tonyb | 921765: Add configuration for new OpenMetal cloud | https://review.opendev.org/c/opendev/system-config/+/921765 got a +1 from Zuul :) | 17:58 |
| Clark[m] | Cool I may end up being a little late after all unfortunately but I'll rereview once settled back at home and the meeting is sorted out | 18:07 |
| opendevreview | Albin Vass proposed zuul/zuul-jobs master: DNM: Debug fetch-subunit-output https://review.opendev.org/c/zuul/zuul-jobs/+/921781 | 18:12 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Test mirror services on noble https://review.opendev.org/c/opendev/system-config/+/921771 | 18:15 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Don't use the openstack-ci-core/openafs PPA on noble and later https://review.opendev.org/c/opendev/system-config/+/921786 | 18:15 |
| opendevreview | Albin Vass proposed zuul/zuul-jobs master: DNM: Debug fetch-subunit-output https://review.opendev.org/c/zuul/zuul-jobs/+/921781 | 18:25 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Don't use the openstack-ci-core/openafs PPA on noble and later https://review.opendev.org/c/opendev/system-config/+/921786 | 18:27 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Test mirror services on noble https://review.opendev.org/c/opendev/system-config/+/921771 | 18:27 |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Initial dump or mediawiki role and config https://review.opendev.org/c/opendev/system-config/+/921322 | 18:39 |
| clarkb | ok made it back in time | 18:51 |
| fungi | awesome! | 18:51 |
| frickler | clarkb: let's manually pause the c8 builds then to avoid useles builder work? | 20:03 |
| opendevreview | Clark Boylan proposed openstack/project-config master: Pause centos-8-stream image builds https://review.opendev.org/c/openstack/project-config/+/921799 | 20:03 |
| frickler | ah, great ;) | 20:03 |
| clarkb | frickler: oh heh ^ I just pushed the config update. I think that is better as it persists restarts and all that | 20:03 |
| clarkb | (though maybe pausing does now too due to zk?) | 20:03 |
| frickler | I think it should persist, but config is nicer for sure. though it may be "true" instead of "True"? | 20:04 |
| clarkb | tonyb: for asking centos-9-stream to rebuild the first thing to check is if our mirrors are up to date and have the missing pacakges. If so then you can log into nl01-nl04 or nb01/nb02/nb04 and issue commands like `sudo docker exec nodepool-docker_nodepool-launcher_1 nodepool list` one of the commands is `nodepool image-build $name` and --help should work there too | 20:04 |
| clarkb | frickler: let me check really quickly | 20:04 |
| frickler | "nodepool image-pause" or -unpause could also be done the same way if needed, just ftr | 20:06 |
| clarkb | frickler: ya looks like it uses yaml boolean vars and doesn't normalize them | 20:06 |
| opendevreview | Albin Vass proposed zuul/zuul-jobs master: prepare-workspace-git: urlencode src_dir https://review.opendev.org/c/zuul/zuul-jobs/+/839225 | 20:06 |
| opendevreview | Clark Boylan proposed openstack/project-config master: Pause centos-8-stream image builds https://review.opendev.org/c/openstack/project-config/+/921799 | 20:06 |
| clarkb | fixed in that patchset | 20:06 |
| tonyb | clarkb: Awesome thanks | 20:07 |
| frickler | thx, +2. and with that I'm out for today, cu | 20:08 |
| tonyb | Looks like the OpenAFS issue on Noble will be fixed once https://bugs.launchpad.net/ubuntu/+source/openafs/+bug/2064628 makes it out of proposed into release | 20:21 |
| * tonyb will pick a different trivial service to test on Noble | 20:22 | |
| opendevreview | Tony Breeds proposed opendev/system-config master: DNM: Initial dump or mediawiki role and config https://review.opendev.org/c/opendev/system-config/+/921322 | 20:28 |
| fungi | kinda frustrating that ubuntu is taking the debian package and then backporting upstream commits into ubuntu rather than getting it fixed in debian. but also frustrating that the maintainer for the package in debian seems to be fairly unresponsive | 20:29 |
| opendevreview | Merged openstack/project-config master: Pause centos-8-stream image builds https://review.opendev.org/c/openstack/project-config/+/921799 | 20:31 |
| clarkb | fungi: the package maintainer in debian is in the openafs irc channel iirc. I want to say they were one of the people who chimed in when I brought up the arm64 thing. Unfortunately no backports for that either | 20:53 |
| fungi | yeah, the main response on the bug i've been tracking was "this has been fixed upstream" implying that they were waiting for there to be a new official release and packaging that. but i guess it's only impacting users of unstable and testing right now, so we know what we're signing up for | 21:02 |
| fungi | the kernel in stable isn't new enough to expose that problem just yet | 21:03 |
| clarkb | if anyone else has time for https://review.opendev.org/c/opendev/system-config/+/921765 it lgtm now and addresses frickler's concerns. I checked on bridge and the vars all seem to be in place too | 21:38 |
| clarkb | tonyb: should I write a change on top of 921765 that adds the cloud to the cloud launcher ruleset or do you want to do that? https://opendev.org/opendev/system-config/src/branch/master/inventory/service/group_vars/bastion.yaml is the file that needs updating | 21:39 |
| fungi | 921765 lgtm too, approved | 21:41 |
| opendevreview | Merged opendev/system-config master: Add configuration for new OpenMetal cloud https://review.opendev.org/c/opendev/system-config/+/921765 | 22:12 |
| opendevreview | Tony Breeds proposed opendev/system-config master: Add OpenMetal to cloud-launcher https://review.opendev.org/c/opendev/system-config/+/921804 | 22:45 |
| clarkb | before we merge ^ we should test that basic openstack client commands on bridge work once 921765 is deployed | 22:49 |
| clarkb | looks like maybe it is deployed? | 22:50 |
| tonyb | It has deployed but they don't work ATM | 22:50 |
| clarkb | ya I'm seeing 401 errors implying something is not right with the auth | 22:52 |
| tonyb | Yup. I'm double checking the usernames and passwords | 22:52 |
| clarkb | its interesting that the error you get back from posting credentials to get a token is that the request you have made requires authentication rather than "invalid authentication" | 22:53 |
| clarkb | tonyb: we can also theoretically check the keystone logs on the cloud side for more info too | 22:57 |
| tonyb | Yeah it doesn't make sense to me | 22:58 |
| tonyb | I've double checked the appropriate vars and the match what I recorded | 22:59 |
| clarkb | tonyb: could it be that the domain is wrong? | 22:59 |
| tonyb | Possibly but changing it didn't alter the behaviour | 23:00 |
| clarkb | might try logging into horizon as those users to see if that works (that would confirm that it isn't the creds but some other auth details) | 23:00 |
| tonyb | Ahh good idea | 23:02 |
| tonyb | "You are not authorized for any projects or domains." | 23:03 |
| clarkb | I almost read that to say "you've authenticated but we can't associated you to anything that makes sense for real work" | 23:05 |
| tonyb | I'm pretty sure that's what it means | 23:06 |
| tonyb | I must have missed a step when creating the users and projects | 23:06 |
| tonyb | Fixed | 23:11 |
| tonyb | I can 'server list' in both clouds and get empty results | 23:12 |
| clarkb | same here | 23:12 |
| clarkb | tonyb: what was the issue? | 23:12 |
| tonyb | huzzah | 23:12 |
| tonyb | When I created the users and the projects I didn't grant the user the 'member' role | 23:13 |
| tonyb | I admit I assumed that was granted by default | 23:13 |
| clarkb | the cloud launcher change looked fine to me fwiw | 23:15 |
| clarkb | fungi: if you are still around that would be the next thing to review now that clouds.yaml is working | 23:15 |
| tonyb | clarkb: so once https://review.opendev.org/c/opendev/system-config/+/921804 merges and is deployed we can use launcher-venv to create the server that will be the mirror. correct? | 23:20 |
| ianw | tonyb: the deb build process is in openstack-zuul-jobs because ... i forget why :) start at https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/zuul.d/jobs.yaml#L1347 and you can follow along :) pushing and building in the ppa should be driven by zuul | 23:27 |
| ianw | as clarkb said, distro packages are fine, until they are not. it's an escape hatch we've needed several times, so worth keeping in order :) | 23:27 |
| tonyb | ianw: Thanks | 23:28 |
| tonyb | At this point the distro packages don't work on noble but there is a proposed-update that *should*. So I'll wait a few days for that if not I'll try adding to the PPA | 23:29 |
| Clark[m] | tonyb: yup. It may be possible now but will depend on the default security group rules. The cloud launcher sets them fairly open allowing us to ssh in and configure the VM once booted | 23:33 |
| tonyb | Clark[m]: cool. Feels like we're making reasonable progress | 23:51 |
| tonyb | and with that I'm moving away from the laptop | 23:52 |
| fungi | looking | 23:53 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!