| *** dtantsur_ is now known as dtantsur | 02:13 | |
| frickler | infra-root: not sure if we can (easily) fix this, but there is an issue with the gitea links on https://codesearch.openstack.org/?q=skip_kpartx&i=nope&literal=nope&files=&excludeFiles=&repos= because the charms repo uses "main" as branch, but the URLs still point to "master" | 08:40 |
|---|---|---|
| opendevreview | Merged openstack/project-config master: Add Nodepool images to Rackspace Flex https://review.opendev.org/c/openstack/project-config/+/927882 | 08:51 |
| frickler | ^^ seems to work fine, 3 images already ready, next one uploading | 09:30 |
| *** Guest113 is now known as paralla | 11:42 | |
| *** paralla is now known as parallax | 11:42 | |
| *** parallax is now known as Guest2594 | 11:42 | |
| *** Guest2594 is now known as parallax | 11:44 | |
| opendevreview | Jeremy Stanley proposed openstack/project-config master: Try booting nodes in Rackspace Flex https://review.opendev.org/c/openstack/project-config/+/928130 | 12:58 |
| opendevreview | Merged openstack/diskimage-builder master: Add tox-py312 job for testing https://review.opendev.org/c/openstack/diskimage-builder/+/923890 | 13:58 |
| frickler | seems people are either busy (travelling) or not in reviewing mood, so I'm single-approving these next raxflex things now | 14:04 |
| fungi | frickler: thanks, and yes some folks are travelling for the summit (on their way back now). i was dealing with a family emergency but finally got home last night after more than a week on the mainland, so i'm back to my usual hours for a bit | 14:08 |
| fungi | i can monitor the deployment for the subnet size change and see if it updates on its own or needs manual cleanup of the existing subnet first in order to take effect | 14:12 |
| opendevreview | Merged opendev/system-config master: Increase opendevzuul-subnet1 from /24 to /20 https://review.opendev.org/c/opendev/system-config/+/927813 | 14:22 |
| fungi | as suspected, infra-prod-run-cloud-launcher was not happy. checking the log now | 14:52 |
| fungi | TASK [cloud-launcher : Processing subnet opendevzuul-subnet1 for opendevzuul-raxflex SJC3] *** fatal: [bridge01.opendev.org]: FAILED! => { "changed": false } MSG: Cannot update cidr in existing subnet | 14:55 |
| fungi | and indeed, opendevzuul-subnet1 still has cidr 10.0.1.0/24 according to openstack subnet show | 14:56 |
| frickler | fungi: yes, that matches what I expected, I'd delete the subnet manually (which may require deleting it from the router first) and then wait for the next hourly run | 14:58 |
| fungi | yeah, i may need to delete the router and network as well in order to be able to delete the subnet | 14:59 |
| fungi | that did it. router remove port, router delete, network delete, subnet delete | 15:02 |
| fungi | i've reenqueued 927813,1 into deploy | 15:02 |
| fungi | it's waiting behind the hourly jobs which kicked off a few moments ago | 15:03 |
| fungi | i'm going to disappear for a quick lunch in about 15 minutes, but can check it again when i get back, should be no later than 16:30 utc | 15:04 |
| frickler | hmm, I don't think that one can delete the network without deleting the subnet automatically with it, but anyway that should work | 15:04 |
| fungi | frickler: oh! you're right, i missed the error... "Failed to delete subnet with name or ID 'opendevzuul-subnet1': No Subnet found for opendevzuul-subnet1" | 15:05 |
| fungi | so yes, deleting opendevzuul-network1 took care of it already | 15:05 |
| fungi | it was then really just router remove port, router delete, network delete | 15:06 |
| fungi | headed to lunch, bbiab | 15:22 |
| NeilHanlon | fungi: when you return, could we touch base about rocky linux mirrors for opendev? not urgent, just making sure I follow up before I forget : | 15:35 |
| NeilHanlon | :) | 15:35 |
| frickler | | dca1b0b2-644e-4b09-9afb-94d2ae82bf0d | opendevzuul-subnet1 | a1f7c5c4-cd2e-4cea-a3a2-8bc88bc2cd3d | 10.0.16.0/20 | | 16:09 |
| fungi | lgtm too, thanks! | 16:32 |
| fungi | NeilHanlon: yep, i think we want clarkb around too, he's probably still in transit home from the summit | 16:32 |
| NeilHanlon | ack. is there a regular meeting? I could just hop into the next one | 16:33 |
| fungi | NeilHanlon: the opendev sysadmins meet at 19:00 utc on tuesdays in the #opendev-meeting channel; clarkb (our elected service coordinator) typically sends the agenda out to the service-discuss@lists.opendev.org mailing list on mondays, based on whatever anyone adds to https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting | 16:36 |
| NeilHanlon | awesome! thanks for the info. I will signup for the list and also add a bit in the wiki :) | 16:37 |
| fungi | if you want an ics file, the link for it can be found at https://meetings.opendev.org/#OpenDev_Meeting | 16:37 |
| fungi | NeilHanlon: if you're signing up for opendev collaboratory mailing lists, opendev-announce is also highly recommended, probably averages a post a month but is where we send important updates | 16:38 |
| fungi | er, sorry, service-announce | 16:38 |
| opendevreview | Merged openstack/project-config master: Try booting nodes in Rackspace Flex https://review.opendev.org/c/openstack/project-config/+/928130 | 16:55 |
| fungi | next problem is that eaxflex-sjc3 is declining node requests due to consecutive boot/ready failures | 17:40 |
| fungi | er, raxflex-sjc3 | 17:40 |
| fungi | openstack.exceptions.ConflictException: ConflictException: 409: Client Error for url: https://nova.api.sjc3.rackspacecloud.com/v2.1/f063ac0bb70c486db47bcf2105eebcbd/servers, Multiple possible networks found, use a Network ID to be more specific. | 17:43 |
| fungi | i guess we need to specify the connected network in the nodepool launcher's configuration? | 17:46 |
| opendevreview | Jeremy Stanley proposed opendev/system-config master: Add default network for Rackspace Flex in nodepool https://review.opendev.org/c/opendev/system-config/+/928275 | 17:52 |
| fungi | that's ^ what we need, i think? | 17:52 |
| fungi | infra-root: ^ | 17:52 |
| * frickler looks | 17:54 | |
| frickler | fungi: downstream I'm using https://zuul-ci.org/docs/nodepool/latest/openstack.html#attr-providers.[openstack].pools.networks instead, which I'd tend to prefer here, too, since I think it is more explicit. that would get added right after the max-servers here https://opendev.org/openstack/project-config/src/branch/master/nodepool/nl01.opendev.org.yaml#L186 | 18:00 |
| frickler | that's not saying your approach couldn't work, too, I'd also agree to trying it if you prefer | 18:00 |
| opendevreview | Merged openstack/diskimage-builder master: Add block device rollback for lvm volumes and groups https://review.opendev.org/c/openstack/diskimage-builder/+/927703 | 18:04 |
| fungi | frickler: thanks for the hint, mine was a shot in the dark, i'll see if i can adapt your suggestion instead | 18:08 |
| opendevreview | Jeremy Stanley proposed openstack/project-config master: Add default network for Rackspace Flex in nodepool https://review.opendev.org/c/openstack/project-config/+/928277 | 18:14 |
| fungi | frickler: like that ^ ? | 18:14 |
| opendevreview | Merged openstack/diskimage-builder master: New elements replace-partition(-redhat) https://review.opendev.org/c/openstack/diskimage-builder/+/927561 | 18:26 |
| frickler | ack, approved, I'll try to stick around until it deploys | 19:27 |
| opendevreview | Merged openstack/project-config master: Add default network for Rackspace Flex in nodepool https://review.opendev.org/c/openstack/project-config/+/928277 | 19:34 |
| frickler | fungi: nodepool still ends up with a server in an error state :( but I don't see details in the log. I can dig more tomorrow, afk now | 19:50 |
| frickler | we are one step further now though I think. (couldn't leave it like that) | 20:02 |
| frickler | created a test instance that ends up in error, we likely need help from cardoe to find out why nova cannot build it, though. instance 459ebaf7-0801-44b4-9180-6cc545c60d03 | 20:03 |
| frickler | hmm, must be something about our image, if I do the same "server create" command with the public 24.04 image, it succeeds immediately | 20:06 |
| frickler | so let's wait for feedback from rackspace, might be that some of the extra hw_* flags are required or something similar | 20:09 |
| cardoe | Sorry been a day. | 21:43 |
| fungi | cardoe: it's in no way urgent! have your day, we're here tomorrow too ;) | 21:49 |
| cardoe | So looks like it was a blip out of glance. | 21:59 |
| cardoe | 500 error | 21:59 |
| fungi | oh, so maybe a transient error? | 22:00 |
| fungi | looks like now nodepool is calculating insufficient available quota | 22:02 |
| fungi | though that might just be how it represents the max-servers value when it's reached | 22:06 |
| fungi | i'm watching node 0038405085 which seems to be getting created | 22:07 |
| fungi | went into a ready state at 22:06:32 utc | 22:08 |
| fungi | looks like it's in use for https://zuul.opendev.org/t/openstack/build/f3a71d03ddb84c03b82de123c33feb60 | 22:12 |
| fungi | https://zuul.opendev.org/t/openstack/stream/f3a71d03ddb84c03b82de123c33feb60?logfile=console.log indicates it's doing things on that node | 22:14 |
| fungi | setting up devstack currently, so if it's gotten that far it's probably fine | 22:14 |
| fungi | this is a good test, because it's rebooting centos into fips mode | 22:15 |
| fungi | probably one of the most complex things we do with server instances cloud-wise | 22:16 |
| fungi | and it came back up and services are starting under devstack now | 22:16 |
| *** jonher_ is now known as jonher | 22:34 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!