Wednesday, 2023-07-19

opendevreviewMerged openstack/ironic master: Revert "Fix IRONIC_IMAGE_NAME=non-existent-image"  https://review.opendev.org/c/openstack/ironic/+/88696001:31
opendevreviewMerged openstack/ironic master: Retry SQLite DB write failures due to locks  https://review.opendev.org/c/openstack/ironic/+/88850004:13
TheJuliaWoot04:13
opendevreviewMerged openstack/ironic master: Firmware Interface  https://review.opendev.org/c/openstack/ironic/+/88527606:00
rpittaugood morning ironic! o/06:34
arne_wiebalckGood morning rpittau and Ironic!07:28
rpittauhey arne_wiebalck :)07:44
mohammed__TheJulia, I see the sqlite fixes merged :) Can anyone confirm that this fixed the "database is locked" error? 08:17
rpittaumohammed__: it should fix that :)08:58
rpittaujust found out the real issue with tinycore 14.x, and it's hilarious: "chroot: failed to run command '/bin/sh': No such file or directory"09:23
rpittau(┛ಠ_ಠ)┛彡┻━┻09:24
opendevreviewMahnoor Asghar proposed openstack/ironic master: WIP: Add inspection (processing) hooks  https://review.opendev.org/c/openstack/ironic/+/88755409:46
mohammed__rpittau, Thanks ! I will build the ironic image and push it to Quay 10:00
opendevreviewLucas Alvares Gomes proposed openstack/ironic master: DNM Enable OVN  https://review.opendev.org/c/openstack/ironic/+/88508710:13
iurygregorygood morning Ironic11:47
iurygregoryrpittau, tinycore hate us that much?!  O.o11:47
rpittauI have not idea what's happening, didn't have the time to investigate more, I'll check again tomorrow12:59
TheJuliagood morning13:15
opendevreviewRiccardo Pittau proposed openstack/bifrost master: Use Launchpad for bug reports  https://review.opendev.org/c/openstack/bifrost/+/88892513:15
rpittaugood morning TheJulia :)13:15
TheJuliaHow is everyone this fine morning?13:34
iurygregorydoing good13:34
iurygregorygood morning TheJulia =) 13:34
opendevreviewJulia Kreger proposed openstack/ironic master: Add wait step  https://review.opendev.org/c/openstack/ironic/+/87906113:41
opendevreviewJulia Kreger proposed openstack/ironic master: WIP/DNM: CI: SNMP job to Ubuntu Jammy  https://review.opendev.org/c/openstack/ironic/+/88892613:49
TheJuliagot to start ^ process at some point13:49
JayFo/13:56
JayFdo we have a grenade skip level yet?13:56
JayFwe need that, too13:56
JayF> ironic-grenade-skip-level https://zuul.opendev.org/t/openstack/build/f36b9d1c1ddc49ce8432fb7598c07dd6 : SUCCESS in 56m 30s (non-voting)13:57
JayFyes, and it's passing13:57
JayFgood stuff13:57
iurygregoryyou already answered =)13:57
iurygregoryI was too slow lol13:58
JayFdoes it make sense if I say, often I'm thinkking out loud with a questionmark on the end lol13:58
JayFI don't ever expect someone to curate public info that I can go find that easy13:58
iurygregoryoh ok13:58
JayFnot that I woulda complained if you beat me :D 13:58
iurygregorylol13:58
TheJuliaBrraaaaiiinnnsss14:01
rpittauTheJulia: thanks for the snmp job test, there's also mine open in case you want to check logs, but I couldn't really find anything from CI14:04
JayFwould be interesting to see it with the older dnsmasq14:05
JayFnow that we at least have a known issue with that ubu versino14:05
opendevreviewJulia Kreger proposed openstack/ironic-python-agent master: WIP: preserve/handle config drives on 4k block devices  https://review.opendev.org/c/openstack/ironic-python-agent/+/88879414:09
TheJuliarpittau: I didn't realize you had an open change14:10
TheJuliastevebaker[m]: reved based on review feedback, thanks!14:11
rpittauTheJulia: no worries, I was not doing any real steps forward :/14:16
TheJuliaI'll give it a try, maybe dnsmasq will help us shine some more light on it14:18
arne_wiebalckInitial assessment after MySQL update from 5.7 to 8.0; the upgrade seems to reduce Ironic API response time by ~15% (for the Ironic DB optimization aficionados ... looking at TheJulia ;-).15:00
TheJulianice!15:01
* TheJulia notes a release note might be good, just to communicate that to operators.15:01
TheJulia;)15:01
arne_wiebalck5.7 is out of support soon, so operators will need to move, but this may be an additional carrot :)15:04
TheJuliaTrue15:05
TheJuliaiurygregory: you missed a couple doc-ish items in with firmware interfaces, I'm going to just add them in my  service steps patch since I'm adding the items there, if you want to extract that is fine, but I'm kind of focused in trying to get it hammered out as fast as possible (the rest of the way)15:08
iurygregoryTheJulia, ack15:10
iurygregorysorry for that, but I can do a follow-up since is doc items =)15:11
TheJuliaiurygregory: I didn't see if you did api ref, I'm not going to do that, since there is not a user facing difference in the data with my patch, it is "new enough client to toggle the verb usage"15:11
TheJuliaalso, could be taken as more of an informational increment too15:12
TheJulia... which saves us headaches at times too15:12
iurygregoryapi-ref I added15:12
arne_wiebalckbye everyone o/15:19
JayFo/15:20
iurygregorybye arne_wiebalck o/15:20
opendevreviewMerged openstack/sushy stable/zed: Retry on ilo state error  https://review.opendev.org/c/openstack/sushy/+/88274615:47
TheJuliaerrr16:04
TheJuliahmmm16:04
TheJuliawhy do we have multiple files saying they ahve the same downrev16:04
TheJuliaokay, my mistake16:08
* TheJulia lets tests run16:08
opendevreviewJay Faulkner proposed openstack/ironic-python-agent master: Log the number of bytes downloaded  https://review.opendev.org/c/openstack/ironic-python-agent/+/88772916:20
JayFtrivial spelling fix there, figured I'd save the round trip16:21
JayFI still +2'd it if someone else wants to land it16:21
opendevreviewJulia Kreger proposed openstack/ironic master: WIP: service steps  https://review.opendev.org/c/openstack/ironic/+/88478316:26
TheJuliamuch appreciated16:27
TheJuliawow, it is actually kind of boiler plate at this point to add another steps model16:32
TheJuliaI need to double check the ipa code, but I don't think that change set will grow much more, especially if I split off docs.16:35
JayFI'm giving you an early review on that now btw16:35
TheJuliaerr, I need to going to walk the code and check tests16:36
rpittaugood night! o/16:36
TheJuliathanks, some refactoring to handle the third state is a common thing in there16:36
TheJuliaI'm mobile at the moment, going to be back at the house shortly, I have a call, and then back to the change set16:37
JayFnot a big deal, I'm just getting reviews in where they are owed16:37
TheJuliahopefully existing clean/service don't explide16:37
JayFI'll be going for an early lunch in about 30m16:37
TheJuliaexplode16:37
TheJuliacool cool16:37
TheJuliaI'll likely be heads down on it this evening, should worst comes to worst16:37
JayFI'm finishing up fixes to my homelab this afternoon after I'm done reviewing; getting my new VM host setup permanently16:38
TheJuliafun!16:39
JayFmore like annoying :) the fun was building the new computer yesterday evening16:41
JayFconverting the old one to a VM host involves lots of "do I need this data?" questions which are a pain to answer as a data pakrat lol16:42
TheJuliaThere is a reason I have a NAS now... and a tool ready to roll to do deduping16:43
JayFguess what's going to happen with the litany of disks in the new hypervisor :D 16:43
JayFlol16:43
TheJulia"oh, I have 9 copies of this file, sweet! Lets let the filesystem hide the duplication, kthxbai"16:44
TheJuliaJayF: any thoughts on backporting the bytes downloaded logging change?16:46
JayFcan't think of a reason why not16:46
TheJuliaokay, I think it would at least make my life, and the life of operators easier when troubleshooting fun networking issues16:46
JayFI always approach backports from a 'why not' standpoint16:47
JayFsomeone wouldn't have put it in master if it wasn't useful16:47
TheJuliaAt least once a year I get a variation of "spanning tree blocks traffic and deployment fails" and it is a PITA to troubleshoot when people don't want to share switch configs and just claim "we swapped the cable"16:47
TheJuliawhich reminds me, I need to go check the internal bug/case for that, to see if they actually, finally did :(16:50
opendevreviewVerification of a change to openstack/ironic master failed: add default conductor group capability  https://review.opendev.org/c/openstack/ironic/+/85570516:58
JayFuh oh, ugly failures down the list on tha tone16:59
JayFhttps://zuul.opendev.org/t/openstack/build/cefaa503719942099d1c9408c20bacec/log/job-output.txt#2189317:00
JayFlooks like devstack is busted building a cirros image?17:00
TheJuliacurl: (60) SSL: no alternative certificate subject name matches target host name 'objects.githubusercontent.com'17:00
opendevreviewVerification of a change to openstack/bifrost master failed: remove setuptools system packages requirement  https://review.opendev.org/c/openstack/bifrost/+/87452217:51
opendevreviewVerification of a change to openstack/bifrost master failed: Refactor the use of include_vars  https://review.opendev.org/c/openstack/bifrost/+/87452317:52
opendevreviewVerification of a change to openstack/ironic master failed: Add job to test with SQLAlchemy master (2.x)  https://review.opendev.org/c/openstack/ironic/+/88602018:14
TheJuliaso it is not the RC_DIR (no idea where that is coming from), it is purely the download18:25
TheJuliaand it looks like we get redirected18:25
TheJuliaI'm sort of wondering if it has resolved already18:31
TheJuliarechecked grenade18:32
TheJuliaerr18:32
TheJuliathe sqlalchemy change18:32
opendevreviewVerification of a change to openstack/ironic master failed: Add job to test with SQLAlchemy master (2.x)  https://review.opendev.org/c/openstack/ironic/+/88602018:40
TheJuliaso the issue is basically the cert has no subject alternative name, and is wildcarded to *.github.io19:12
JayFthis isn't unique to us, is it?19:14
JayFif not, we should engage qa/devstack folks?19:14
TheJuliait is unique to us19:18
TheJuliawe download a stock iamge and modify it19:18
TheJuliaso it will actually local boot properly19:18
TheJuliabecuase cirros is designed for VMs19:18
TheJuliaand thus is geared to that specifically19:18
TheJuliaAll we can really do, short of begin an campaign to excise/re-evaluate is to just go "its okay curl, don't error"19:19
JayFcan we tell curl "Verify but no SAN is OK"19:19
JayFprobably not : /19:19
TheJuliaafaik no19:19
TheJulialooks like the certs are updating19:20
TheJuliabut openssl s_client only gives so much data19:20
JayFyeah I'd +2 a temporary fix for telling curl to not verify19:20
JayFbut we should make a note to revisit19:20
TheJuliaall I can see is there is just no subject alternative name19:20
JayFI can setup a reminder to check on Monday19:20
TheJuliak19:20
JayFyeah, you wanna push it up, I'll +2A19:20
JayFand setting a slack reminder now to follow up on monday19:20
opendevreviewJulia Kreger proposed openstack/ironic master: CI: Handle unhappiness with github cert  https://review.opendev.org/c/openstack/ironic/+/88895119:23
JayFOpenStack Caracal :| 20:05
JayFthat's a lot of c's and a's to type for the next several years20:06
TheJulia... indeed20:20
iurygregorycaracal is the new release name? ouch >.<21:09
opendevreviewMerged openstack/bifrost master: remove setuptools system packages requirement  https://review.opendev.org/c/openstack/bifrost/+/87452221:42
JayFTheJulia: ... would CI fail in grenade because stable/2023.1 needs the curl fix, too?22:18
JayFI think so, and they might have to merge in inverse order, or we may have to drop the job and re-enable22:18
opendevreviewJay Faulkner proposed openstack/ironic stable/2023.1: CI: Handle unhappiness with github cert  https://review.opendev.org/c/openstack/ironic/+/88865422:18
JayFI give it a 50/50 chance of failing because zed will need the same change (lol)22:19
TheJuliai'd disable grenade on master, backport the change22:25
TheJuliaand then re-enable22:25
TheJuliaeach branch will have the same issue, so might as well do it alongside22:25
TheJuliaJayF: ^22:32
JayFIf you edit your change to do that, I can land it again23:04
JayFIf you need me to make the change, it'll be later this evening23:04
JayFQA23:05
JayFWhoops, somehow that's the string that came out when I fat finger exiting the mobile app23:06
JayFwhatever, I walked to a computer, lets see if I can swing this from windows :)23:07
opendevreviewJay Faulkner proposed openstack/ironic master: CI: Handle unhappiness with github cert  https://review.opendev.org/c/openstack/ironic/+/88895123:19
opendevreviewJay Faulkner proposed openstack/ironic master: CI: Re-enable grenade job as voting  https://review.opendev.org/c/openstack/ironic/+/88896023:19
JayFthat should do the trick23:19
* JayF disappears back into the inky shadows23:19
TheJuliaJust don’t make deals with them23:25
TheJulia:)23:25
opendevreviewJay Faulkner proposed openstack/ironic master: CI: Handle unhappiness with github cert  https://review.opendev.org/c/openstack/ironic/+/88895123:25
opendevreviewJay Faulkner proposed openstack/ironic master: CI: Re-enable grenade job as voting  https://review.opendev.org/c/openstack/ironic/+/88896023:26
opendevreviewJay Faulkner proposed openstack/ironic master: CI: Handle unhappiness with github cert  https://review.opendev.org/c/openstack/ironic/+/88895123:28
opendevreviewJay Faulkner proposed openstack/ironic master: CI: Handle unhappiness with github cert  https://review.opendev.org/c/openstack/ironic/+/88895123:30
opendevreviewJay Faulkner proposed openstack/ironic master: CI: Re-enable grenade job as voting  https://review.opendev.org/c/openstack/ironic/+/88896023:30
JayFand it was today I learned zuul configs really, really hate tabs23:31
JayFoh, and I'll note -- on my windows machine; raw.githubusercontent.com is serving up a cert for *.github.io; but listed under "Extensions -> Certificate Subject Alternative Name", it has *.github.com, and *.githubusercontent.com (among others)23:54
JayFI'm tending to think this might be a curl bug...23:54
JayFor github is serving up a different/wrong cert based on CDN location(?)23:55
TheJuliaJayF: I wasn't getting a alternative name earlier23:58
JayFThat is from microsoft edge23:58
TheJuliacould be they noticed a major mistake23:58
JayFso I don't know what that actually means lol23:58
TheJuliatabs == great unhappiness23:58
opendevreviewJulia Kreger proposed openstack/ironic master: WIP: service steps  https://review.opendev.org/c/openstack/ironic/+/88478323:59

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!