Tuesday, 2022-02-01

opendevreviewIan Wienand proposed openstack/diskimage-builder master: Remove CentOS-8 jobs  https://review.opendev.org/c/openstack/diskimage-builder/+/82719600:01
ianwargh, clean still has it00:02
ianwglean even00:02
*** dviroel|brb is now known as dviroel00:03
fungioh, yep00:04
opendevreviewIan Wienand proposed opendev/glean master: testing: drop centos-8, add 8-stream and 9-stream  https://review.opendev.org/c/opendev/glean/+/82719700:04
opendevreviewIan Wienand proposed openstack/diskimage-builder master: Remove CentOS-8 jobs  https://review.opendev.org/c/openstack/diskimage-builder/+/82719600:05
*** dviroel is now known as dviroel_00:19
*** dviroel_ is now known as dviroel00:19
ianwhrm one of the nodepool container builds failed with WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fae54241fa0>: Failed to establish a new connection: [Errno 113] No route to host')': /simple/yappi/00:29
ianwi'm guessing this came from the mirror00:29
ianwcloud: inmotion00:30
ianwi'll keep an eye ... it's going to be one of those days00:33
*** rlandy|ruck is now known as rlandy|out00:44
clarkbbefore we restart gerrit we should preserve the dump file that corvus generated earlier today01:02
clarkbluca is asking for it on the issue I filed. I don't think we should upload it immediately as I'm not sure what is included in that (would it have anything private?) but if we restart on new container it will be lost01:02
clarkbI think the file was at /tmp/dump in the container itself01:03
clarkbmy ssh keys have expired tehmselves for the day and I need toget back to dinner01:04
clarkbbut maybe someone can copy that off the container really quickly and stash it somewhere on the host01:04
opendevreviewMerged openstack/project-config master: Stop building CentOS 8  https://review.opendev.org/c/openstack/project-config/+/82719501:06
clarkbhrm I went ahead and loaded a key let see fi I can do this real quick01:07
*** cloudnull2 is now known as cloudnull01:07
clarkbok thats done review02:~clarkb/dump has the file01:10
clarkbI think we can restart whenever is convenient now01:10
fungi(now get back to dinner)01:10
clarkbyes I'm told I need to eat now. :)01:11
fungii'm familiar with this01:11
ianwexcellent.  i'm happy to restart in about 2-3 hours, that seems a nice quiet time01:18
opendevreviewMerged opendev/system-config master: grafana: update docs and make an import script  https://review.opendev.org/c/opendev/system-config/+/82624101:25
fungiianw: i may still be around at that time if problems arise01:33
ianw++ heh, hopefully neither :)01:35
*** ysandeep is now known as ysandeep|away01:44
*** dviroel is now known as dviroel|out01:47
opendevreviewMerged opendev/glean master: testing: drop centos-8, add 8-stream and 9-stream  https://review.opendev.org/c/opendev/glean/+/82719702:28
ianwplanning to do gerrit restart in about 10 minutes03:53
ianw"RepoDigests": [04:06
ianw            "opendevorg/gerrit@sha256:8776f6539f47917683d4e6763fa4cf3aee8da476166f44434def9bfb6940e611"04:06
ianwis what i just pulled, which matches https://hub.docker.com/layers/opendevorg/gerrit/3.4/images/sha256-8776f6539f47917683d4e6763fa4cf3aee8da476166f44434def9bfb6940e611?context=explore04:06
ianwwhich was pushed by 82715304:06
ianw#status log restarted gerrit to get changes from 82715304:11
opendevstatusianw: finished logging04:11
opendevreviewMerged openstack/diskimage-builder master: Remove CentOS-8 jobs  https://review.opendev.org/c/openstack/diskimage-builder/+/82719604:34
opendevreviewIan Wienand proposed openstack/diskimage-builder master: Cleanup more CentOS 8 bits  https://review.opendev.org/c/openstack/diskimage-builder/+/82721004:44
opendevreviewIan Wienand proposed openstack/diskimage-builder master: Remove contrib/setup-gate-mirrors.sh  https://review.opendev.org/c/openstack/diskimage-builder/+/82721104:44
prometheanfirenew ssh?05:20
prometheanfirekex mismatch05:20
ianwprometheanfire: umm, none of that should have chagned05:37
ianwi have pushed changes via ssh since the update and didn't notice anything out of order05:39
ianwhttps://zuul.opendev.org/t/zuul/build/1ddea9a38a6641dbbc7c809bd4f1b008/log/zuul-info/inventory.yaml also failed in inmotion, a few hours ago though05:44
ianwi can't see anything wrong with the mirror, i just logged in an it's active and serving responses05:50
ianwmight just be a glitch in the matrix05:50
* prometheanfire shrugs05:51
*** marios is now known as marios|ruck06:04
*** ysandeep|away is now known as ysandeep07:04
*** mrunge_ is now known as mrunge07:35
opendevreviewMerged openstack/diskimage-builder master: centos: do not use $releasever in .repo files  https://review.opendev.org/c/openstack/diskimage-builder/+/82624407:43
opendevreviewMerged openstack/diskimage-builder master: Switch 9-stream testing to use opendev mirrors  https://review.opendev.org/c/openstack/diskimage-builder/+/82165107:43
opendevreviewMerged openstack/diskimage-builder master: Add 9-stream ARM64 testing  https://review.opendev.org/c/openstack/diskimage-builder/+/82165307:44
*** amoralej|off is now known as amoralej07:52
*** jpena|off is now known as jpena08:36
amoralejaccording to https://static.opendev.org/mirror/logs/rsync-mirrors/ rsync for centos-8-stream is not longer running since yesterday 06:47?08:55
ianwamoralej: ahh, thanks, i think i accidentaly still had a lock held from when i bumped its quota yesterday09:24
opendevreviewMerged openstack/project-config master: Set centos-8 min-ready to 0  https://review.opendev.org/c/openstack/project-config/+/82718309:28
rpittauhello everyone, we found a little issue with dib and debian bullseye, after building an image with the debian element it seems the network interfaces are not configured during boot :/10:27
rpittaufrom what I can see the /usr/local/sbin/dhcp-all-interfaces.sh script is not executed at startup10:27
*** marios|ruck is now known as marios10:41
*** marios is now known as marios|ruck10:41
amoralejianw, good, so you are removing the lock?10:41
opendevreviewAlfredo Moralejo proposed zuul/zuul-jobs master: Change RDO train repository for Centos 8 stream  https://review.opendev.org/c/zuul/zuul-jobs/+/82706710:42
*** rlandy is now known as rlandy|ruck11:14
*** dviroel|out is now known as dviroel11:27
dviroelmarios|ruck: ysandeep: please see https://review.rdoproject.org/r/c/rdo-jobs/+/38634/2#message-d942642017ca1572f4e0c4c6eb154d9edef7eb0c 11:45
dviroelmarios|ruck: wrong channel :P11:47
marios|ruckdviroel: ;)11:53
*** ysandeep is now known as ysandeep|break12:04
*** ysandeep|break is now known as ysandeep12:39
*** rlandy|ruck is now known as rlandy|ruck|mtg13:04
*** sshnaidm is now known as sshnaidm|afk13:06
*** amoralej is now known as amoralej|lunch13:13
fungirpittau: is this one you built yourself, or you mean our debian-bullseye job nodes are no longer working?13:35
rpittaufungi: hi! This is one ipa ramdisk built using the debian DIB module13:35
rpittauwe change to bullseye in december and since then we were not able to use dhcp on the images built with DIB13:37
fungiis that with or without glean?13:38
fungialso, you're aware there's an #openstack-dib channel, right? you might get better help there13:38
rpittaufungi: I know about that channel, I sent a message there too more than 1 month ago but I didn't get any answer13:39
fungiahh, okay13:39
fungiso... the bullseye images we're booting use glean, maybe that's why we're able to use dhcp?13:40
rpittauwe're building the image in the same way as we did with buster, the precise command is: disk-image-create -o /var/lib/ironic/httpboot/ipa debian ironic-python-agent-ramdisk 13:41
rpittaufungi: that could be, are they built using a different DIB element ?13:42
fungiwe build on the debian-minimal element, setting release: bullseye and then we add these elements: vm, simple-init, growroot (and some others which probably aren't relevant in this case)13:49
fungirpittau: are you using the simple-init element?13:49
fungii think that's the one responsible for our network setup13:49
fungiyeah, looking at its readme, that's the one that handles findinging a configdrive, calling glean, and so on13:50
fungiit's possible that the debian-minimal element on its own isn't complete with regards to network setup and nobody's noticed until now13:51
rpittaufungi: we use debian element that includes debian-minimal, according to the docs, but I don't see simple-init13:51
rpittauI could give that a try13:51
fungiworth a shot, but i have no idea whether it will be well-suited to a ramdisk boot since that's going to be different from a cloud vm in a few ways13:52
rpittaufungi: thanks, I'll work on that and see :)13:53
fungianyway, if the buster images were booting on their own and bullseye aren't, then it's probably a bug which will need solving in the debian(-minimal) element in dib 13:53
rpittauactually, now that I see the full command, we add simple-int in the end, so it doesn't look like it works :/13:54
*** sshnaidm|afk is now known as sshnaidm13:54
fungiinteresting. in that case if you can get the boot logs from glean on a working buster and broken bullseye image, maybe we could try to spot the difference13:55
rpittauone thing that I noticed is the network interfaces naming difference, on bullseye we don't have the old naming anymore (e.g. eth0)13:55
rpittauand DIB_NETWORK_INTERFACE_NAMES is set to eth0 eth113:56
fungiyeah, debian switched to bus-based interface numbering a while back. i mostly use sid though so am not sure which release it happened in13:57
rpittauseems bullseye, in buster I can still see the old naming13:57
fungiit may be possible to get them back with net.ifnames=013:59
rpittauthat's maybe worth a try, can I add that option when building an image with DIB ?14:00
fungiDIB_BOOTLOADER_DEFAULT_CMDLINE envvar maybe?14:02
rpittaufungi: thanks, I have some to work on at least :)14:03
fungicodesearch for net.ifnames turns up a number of projects fiddling with it in one way or another14:03
rpittauI'll have a look there too14:04
fungialso often coupled with biosdevname=014:04
*** rlandy|ruck|mtg is now known as rlandy|ruck14:05
rpittauI've also enabled DIB debugging message, let's see if we get some more info14:08
*** amoralej|lunch is now known as amoralej14:13
opendevreviewdasm proposed zuul/zuul-jobs master: Pull correct upstream pip version  https://review.opendev.org/c/zuul/zuul-jobs/+/82729714:22
louroto/ has anyone got time to land https://review.opendev.org/c/openstack/project-config/+/825089 ? thanks!14:38
lourotthanks fungi!14:40
*** bhagyashris_ is now known as bhagyashris14:42
opendevreviewMerged openstack/project-config master: Mirror charm-nova-compute-nvidia-vgpu to GitHub  https://review.opendev.org/c/openstack/project-config/+/82508914:47
clarkbfungi: rpittau: glean really expects to have a config drive to be able to do its job. I want to say the way ironic does things with its images is does dhcp all interfaces (which also requires you to know the interface names iirc) and then also configures glean. This works beacuse dhcp all interfaces writes out /e/n/i configs at image build time then if you haev a config drive present14:55
clarkbglean will write over the top of those on boot14:55
clarkbBut glean won't do much of anything without a config drive iirc14:55
rpittauclarkb: thanks, the problem happened when swithcing from buster to bullseye and one main differnece is that the network interfaces names changed, so I'm trying now with what fungi suggested to switch to the old names15:00
*** artom__ is now known as artom15:16
*** ysandeep is now known as ysandeep|dinner15:19
opendevreviewAurelien Lourot proposed openstack/project-config master: Add dummy charm project  https://review.opendev.org/c/openstack/project-config/+/82730715:31
clarkbfungi: any reason to not recheck (and potentially land) https://review.opendev.org/c/opendev/system-config/+/825338 today? Then we can test the gitea links change further down the stack15:32
clarkbI mean it should be a complete noop but it udpates gerrit config minorly and is worth monitoring I suppose15:32
opendevreviewTristan Cacqueray proposed zuul/zuul-jobs master: Add packages support to ensure-cargo  https://review.opendev.org/c/zuul/zuul-jobs/+/82713015:39
*** dviroel is now known as dviroel|lunch15:40
*** ysandeep|dinner is now known as ysandeep15:45
opendevreviewAurelien Lourot proposed openstack/project-config master: Add dummy charm project  https://review.opendev.org/c/openstack/project-config/+/82730715:47
lourotclarkb, fungi ^ how exactly should I specify that the default branch is `main` instead of `master`? Should it be done directly below the `upstream: ...` line? 15:48
clarkblourot: its a dictionary so order of the keys doesn't matter. Just needs to be in there15:48
clarkblooks like the key is default-branch15:49
lourotnice thanks, giving a try at `default-branch: main`15:49
opendevreviewAurelien Lourot proposed openstack/project-config master: Add dummy charm project  https://review.opendev.org/c/openstack/project-config/+/82730715:50
fungiclarkb: no objection, i've rechecked 825338 now15:50
clarkbhttps://opendev.org/opendev/jeepyb/src/branch/master/jeepyb/cmd/manage_projects.py#L488 and https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/gitea-git-repos/library/gitea_create_repos.py#L132 are the main points where we handle that15:51
clarkbyou'll also need to set a zuul setting somewhere that I don't remember to have zuul treat main as the fallback branch for branch resolution stuff15:51
fungiyeah, we can work that out once the repo creation is done, i expect15:55
clarkbthis just occured to me, but we may need to test renaming of a gerrit project whose branch isn't master15:55
clarkbNot necessary to create a new project that way. But if we get to renaming projects that aren't default master I wonder if renaming them in gerrit will try to switch HEAD to master15:56
clarkbbut we can take this one step at a time and learn as we go. No need to solve that before we do a new project15:56
fungiyep. some of that we might be able to just do in testinfra too15:58
clarkbfungi: ianw: so now that gerrit updated we can test the signed tag pushing fix?16:13
clarkbAnd once we are happy with it here I'll push the merge of that fix into gerrit 3.516:14
fungiyeah. we can simply approve it and if any new uses of createTag have snuck in the change will fail on removing it from the allowed list in the linter script16:14
clarkband warn the openstack release team to do a single release first?16:15
fungiright, once it merges16:15
fungiand deploys16:15
clarkbhttps://review.opendev.org/c/openstack/project-config/+/826335 has the necessary +2's do you want to +A or should I?16:15
fungii can16:16
fungiapproving will be faster than rechecking in that tenant16:17
clarkblourot: fungi: looks like we need to update our ci checks to allow for no master branch in the upstream repo if the default branch exist and accept default branch as a key16:24
clarkblourot: is that something you'd be willing to look at and update as part of your change? The nice thing with that is it will be self testing16:24
opendevreviewDariusz Smigiel proposed zuul/zuul-jobs master: Pull correct upstream pip version  https://review.opendev.org/c/zuul/zuul-jobs/+/82729716:34
*** dviroel|lunch is now known as dviroel16:35
*** ysandeep is now known as ysandeep|out16:43
opendevreviewMerged openstack/project-config master: Remove unsigned tagging permission from projects  https://review.opendev.org/c/openstack/project-config/+/82633516:46
*** marios|ruck is now known as marios|out16:46
opendevreviewMerged zuul/zuul-jobs master: Pull correct upstream pip version  https://review.opendev.org/c/zuul/zuul-jobs/+/82729716:57
fungideploy for 826335 has finished. i'll let #openstack-release know17:07
*** jpena is now known as jpena|off17:31
opendevreviewMerged opendev/system-config master: Fix mixed spaces and hard tabs in Gerrit config  https://review.opendev.org/c/opendev/system-config/+/82533817:48
fungiclarkb: ^ i guess we're clear to resume testing the patch now?17:52
clarkbfungi: ya I think we can recheck the other changes and tehn depends on should work17:53
fungidoing now17:53
fungiclarkb: should i set a new autohold for 825396 before rechecking it?17:54
fungiyeah, i guess i should, otherwise there's not much point to it17:54
clarkbfungi: I had an autohold but maybe it caught a different run? We should check and ya17:55
fungii'll clean up the previous on17:55
*** dviroel is now known as dviroel|biab17:55
clarkb0000000008 that one is the one I had before. not sure if it caught anything17:55
clarkbI've also got my hold of the gerrit node to test the tag acl which we can remoev as soon as someone pushes a tag successfully17:58
fungithere was also an earlier 0000000004 i'd set18:00
fungiany idea what we need to do to get zuul-client autohold to support the --change option correctly?18:00
clarkbfungi: I think you need to pass refs/changes/xy/abcxy and then it would work? So probably have the tool convert a lone number to that format?18:01
clarkbbut I'm not completely sure of that18:01
fungialready tried --change=refs/changes/96/825396 but in the ref filter column autohold-list just says .*18:01
fungisame as when i did --change=82539618:02
fungimaybe we need a newer zuul-client version?18:02
fungiZuul-client version: 0.0.5.dev1918:02
fungilooks like it may be up to dev21 now in master18:03
clarkblookign at the log the 2 latest changes don't seem related to that18:04
clarkbI'm guessing it is just broken18:04
clarkbfungi: the client appears to send chagne through unmodified to the zuul api currently18:06
fungiyeah, i'm not seeing any proposed patches which might address it either18:06
clarkband then the zuul api does ref_filter = project.source.getRefForChange(jbody['change'])18:07
fungido i need to add ,rev?18:07
clarkboh except the .* is set on an else case for ^ so either getRefForChange is returning .* or jbody(['change']) is none18:08
clarkbfungi: no it explicitly errors if you have a , in it18:08
fungior /rev?18:08
*** dviroel|biab is now known as dviroel18:08
fungino dice18:09
clarkbok gerrit's getRefForChange can never return just .* which means we're hitting the else case for change being unset18:10
clarkbthat implies to me the issue is entirely on the zuul-client side18:10
clarkbI'm not seeing how that could happen though18:12
fungiyeah, it literally passes "change": change in the json=args18:12
clarkbmaybe something with argparse?18:13
fungido we need to recast it as str?18:13
fungioh, or int18:13
fungiwhat does the rest api want in that json blob? int or str?18:14
fungiyou could be right and argparse is coercing it to the wrong thing18:14
clarkblooks like the zuul api coerces to string so either should be fine18:14
fungiso not that i guess18:15
fungi--ref=refs/changes/96/825396/3 seems to have put something other than .* there18:17
clarkbya I'm stumped. Might need to add debugging in the lcient18:17
fungi--ref=825396 also18:17
fungilooks like --ref will just pass anything through18:17
fungii wonder if that's a viable workaround18:17
fungiwhich of those would you expect to work? either? both?18:18
fungii'll try with the simple one first18:19
fungiif this works, we might be able to just deprecate --change and alias it to --ref18:22
clarkbmy keyboard is suddenly missing keys and doubing some entries. I need to fiurethat ut18:22
fungihave you tried turning it off and on again?18:22
clarkbathat is step 118:23
clarkbI semy numlo ight              flapping too18:24
clarkbuplgging th eybard and plgig i back n dnt hlp18:25
clarkbtis i fun :)18:25
clarkbI am on a new usb bus and now it seems to work18:25
clarkbugh I think that means my monitor or whatever bus the monitor is plugged into is going bad18:26
fungiyeah, i don't trust external hubs. my keyboard is important enough to win one of the few internal ports connecting directly to my motherboard18:30
fungii've had too many bad experiences where a file transfer to a usb mass storage device caused me to no longer be able to type18:30
clarkbI think I've isolated it to a single port on the main device18:31
clarkbanother port which I believe is on the same bus works fine18:31
clarkboh I wonder. I've got the usb dac on possibly the same hub as the monitor hub and maybe that bus can't provide power to all the thigns at the same time18:32
clarkblet me try again after removing the dac18:32
clarkbya quick testing seems ot indicate that is a likely cause18:33
clarkbanyway sorry to distract I just had a really weird experience with that is all :)18:34
fungiyeah, in fact this workstation has two stacks of usb ports on opposite sides of the motherboard. the stack where my keyboard is plugged in has nothing else connected, so i expect it's the only device on that bus at all other than probably an inbuilt hub which connects those ports18:34
fungithe other stack is where my pointer device and a powered hub are attached18:34
fungisince shuffling them around to this configuration i've had zero problems with the keyboard18:35
*** amoralej is now known as amoralej|off18:35
clarkbTIL this is something to be careful of18:38
opendevreviewTristan Cacqueray proposed zuul/zuul-jobs master: Update cargo-test job to support system packages  https://review.opendev.org/c/zuul/zuul-jobs/+/82713118:38
clarkbthe power cable for the dac is now on its own bus I believe18:39
fungiyep. for me, there's no peripheral so important that it should get to preempt my keyboard. like, ever18:41
fungineat, i missed that we're now getting warnings in comments from zuul. that's awesome18:44
fungiWarning: Job system-config-run-review-3.4 requires artifact(s) gerrit-3.4-container-image provided by build 7a2820c9b3934619a761a7a5092e0f5a (triggered by change 825337 on project opendev/system-config), but that build failed with result "FAILURE"18:45
fungithat one was from a few days ago, just hadn't spotted it until now18:46
clarkbya that was what caued the previous buidls to not do what we needed tehm to do18:46
fungii knew the feature was in the works but couldn't recall if it had been implemented. i guess it has18:46
clarkbI'm going to do a reboot now just to clear any power state sillyness gremlins that may be in my usb stack. Hopefully I come back :)18:46
fungigood call18:46
clarkband back. Seems like everything is fine after reorganizing cables18:54
fungino luck with --ref=825396 (it doesn't seem to have held the node). i'll try again now with --ref=refs/changes/96/825396/319:33
ianwfungi: it is making it into the hold request right?  when you check with autohold-list, the values look right?19:56
ianwjust we had some issues with quoting etc. when passing through arguments using the wrapper command19:56
fungiyeah, it makes the hold but sets the change filter to .* instead of whatever value you supply19:57
fungiand github becomes increasingly more gerritlike... if you click the "force-pushed" on a pr which had a push --force update, it shows you a diff of the changes between the old and new revisions20:01
*** dviroel is now known as dviroel|afk20:35
fungioh hey, --ref=refs/changes/96/825396/3 worked!20:40
fungiclarkb: ianw: ^20:40
clarkbgood to know20:40
fungimaybe that can give us some clue as to how to fix --change in the client20:40
clarkbat least until we fix it20:41
fungianyway, the held gerrit is at now20:41
clarkblooks like there is a hash in the file diff gitea links. Also super cool we can depends on upstream like that20:42
fungiand the "gitea" link at looks right, yeah20:42
clarkbcorvus: ^ fyi thats a really fun use of depends on in opendev20:48
opendevreviewAde Lee proposed zuul/zuul-jobs master: WIP/DNM - Test new version of python3  https://review.opendev.org/c/zuul/zuul-jobs/+/82736620:49
fungiokay, announcement followup about the signed tags situation has been sent20:56
corvusclarkb: you should be sure to tell the upstream gerrit folks you did that :)20:57
fungiyeah, a comment on the change in their gerrit to that effect would be great20:58
opendevreviewTristan Cacqueray proposed zuul/zuul-jobs master: Remove the periodic-weekly pipeline  https://review.opendev.org/c/zuul/zuul-jobs/+/82736921:00
clarkbcan do21:10
clarkbcommented on https://gerrit-review.googlesource.com/c/gerrit/+/32927921:15
clarkbok I looked at forward porting the gerrit 3.4 fix for acl pushing to 3.5 but 3.5 already has at least 2 outstanding forward merge chagnes and I don't want to deal with stacking that properly. I'll just keep an eye on it to ensure it gets forward ported21:27
fungiprobably good enough21:27
fungias long as it happens before we upgrade, should be fine21:27
clarkbthis is one reason why I'm a little concerned about updating to latest release in gerrit it seems to lag all the bug fixes. I guess we would just have to get aggressive about forward porting things ourselves if/when we get to the latest release21:27
opendevreviewJames E. Blair proposed opendev/system-config master: Remove gearman from Zuul  https://review.opendev.org/c/opendev/system-config/+/82679121:52
opendevreviewAde Lee proposed zuul/zuul-jobs master: WIP/DNM - Test new version of python3  https://review.opendev.org/c/zuul/zuul-jobs/+/82736622:15
ianwwhat's the feeling on what will happen if we merge the centos-8 node removal https://review.opendev.org/c/opendev/base-jobs/+/827181 with zuul reporting twice on it?22:45
opendevreviewIan Wienand proposed openstack/project-config master: Remove centos-8  https://review.opendev.org/c/openstack/project-config/+/82718422:48
fungiianw: i think openstack/openstack-ansible-tests is reporting from the openstack tenant with a configuration error because it can't evaluate the configuration there, but the change is being gated by the opendev tenant where it's passing, so it should merge modulo some other problem22:52
fungii expect the end result to be that configuration for openstack/openstack-ansible-tests will be broken once that merges, but presumably we're resigned to that22:53
fungiodds are it was broken anyway since we mirrored red hat's removal of the packages22:53
ianwyep, the jobs are now all broken irrespective of the node being there or not, the only path out of that would be to somehow start pointing things to vault.centos.org22:54
ianwthat seems like pointless work though22:54
fungiyes, probably more work than fixing the remaining tests22:55
opendevreviewMerged opendev/base-jobs master: Remove centos-8 as it is EOL  https://review.opendev.org/c/opendev/base-jobs/+/82718122:59
fungiand there we go22:59
ianwhttps://review.opendev.org/c/openstack/project-config/+/827184 was unhappy until that merged, but is verifying itself now after a recheck23:07
opendevreviewMerged opendev/system-config master: Remove gearman from Zuul  https://review.opendev.org/c/opendev/system-config/+/82679123:11
fungiremove all the things23:12
ianwfungi: https://review.opendev.org/c/zuul/nodepool/+/826773 and https://review.opendev.org/c/zuul/zuul/+/826772 are two more if you're in a removal mood :)23:16
fungioh, right, we talked about that already. in they go!23:19
opendevreviewMerged openstack/project-config master: Remove centos-8  https://review.opendev.org/c/openstack/project-config/+/82718423:25
ianwwhen that applies i'll dig into fedora, that should work now23:41
ianwf35 even23:41
ianwand we can hopefully turf f3423:41
fungimore things to destroy! er, i mean delete!23:42
*** dviroel|afk is now known as dviroel23:44

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!