| ianw | "The warning appears both in the host and guest kernel logs." -- it's possible we're flooding the logs on the ovh side with this too | 00:01 |
|---|---|---|
| ianw | For reference, 0x4 is SSBD mitigation, and 0x2 is the STIBP mitigation, and in the original bug report which was related to SSBD only | 00:06 |
| ianw | the write was 0x4, but here it is 0x6 indicating that guest wants to enable both. | 00:06 |
| opendevreview | Merged opendev/system-config master: Add testing for jammy openafs https://review.opendev.org/c/opendev/system-config/+/841525 | 00:07 |
| ianw | Ign:4 https://mirror.dfw.rax.opendev.org/ubuntu jammy-backports InRelease | 00:28 |
| ianw | this might be something else to look into. as we figured out, bionic images don't have backports enabled. i'm not sure what our policy is on it, but we should probably make the images ~ the same | 00:29 |
| ianw | model name: Intel Core Processor (Haswell, no TSX) | 00:30 |
| ianw | this is what ovh is reporting. so it doesn't seem like amd is involved here | 00:31 |
| ianw | [ 2.290166] kernel: unchecked MSR access error: WRMSR to 0x48 (tried to write 0x0000000000000004) at rIP: 0xffffffffabc90af4 (native_write_msr+0x4/0x20) | 00:32 |
| ianw | interesting, this time is was 0x4 | 00:32 |
| ianw | oh, wait, it was before too | 00:32 |
| ianw | SSBD mitigation | 00:32 |
| Clark[m] | Re backports I'm not concerned about enabling them but iirc we do configure out sources lists explicitly with Ansible so someone likely decided not to include them on our servers | 00:33 |
| ianw | SSBD: speculative store bypass disable = false | 00:34 |
| ianw | virtualized SSBD = false | 00:34 |
| ianw | SSBD fixed in hardware = false | 00:34 |
| ianw | and another weird thing -- this is an OVH vm, but it has rax mirrors? | 01:11 |
| fungi | are you...sure? | 01:14 |
| fungi | if so, is it the rax mirror we use when building images? | 01:15 |
| fungi | maybe we're not resetting it correctly with the base job | 01:15 |
| fungi | yeah, dfw is what we have in our images, so that's likely | 01:16 |
| ianw | fungi: as sure as i ever am, which means there is a large possibility i have something dramatically wrong :) | 01:17 |
| ianw | so using a mainline 5.17 kernel on this ovh jammy image the problem doesn't happen | 01:18 |
| ianw | ergo there is something that can be backported to fix it | 01:18 |
| fungi | as long as it doesn't also need to support openafs ;) | 01:20 |
| fungi | er, with 5.17 i mean | 01:20 |
| fungi | whatever might get backported probably wouldn't have anything to do with my openafs build problem | 01:21 |
| ianw | oh, there's usually an upstream patch to openafs for more recent kernels pretty quickly after release? but it may not have made it into openafs releases yet | 01:24 |
| fungi | or debian may need to update its openafs package | 01:36 |
| fungi | the lkm for 1.8.8.1 won't build on debian with linux 5.17 kernel headers: https://bugs.debian.org/1010764 | 01:39 |
| fungi | error: implicit declaration of function ‘complete_and_exit’ | 01:40 |
| ianw | yeah it is fixed https://gerrit.openafs.org/#/c/14882/ | 01:42 |
| fungi | oh, awesome | 01:42 |
| ianw | not that i know much, but i've filed https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1973839 | 01:56 |
| ianw | i've got an ovh vm and am trying to bisect it there; we'll see if that works ... | 01:57 |
| Clark[m] | You are bisecting between latest mailine and 5.15? That might take a while | 02:28 |
| ianw | 5.17 and 5.15; yeah it says ~10 steps | 02:31 |
| ianw | i haven't actually managed to build a kernel yet, so ... yeah :) | 02:32 |
| fungi | maybe you'll get lucky and it'll be near an early bisection point | 02:44 |
| ianw | oh of course, i haven't run configure-mirrors on this manually setup node, doh. so that's why that is pointing at rax | 02:51 |
| *** pojadhav- is now known as pojadhav | 05:39 | |
| *** ysandeep|out is now known as ysandeep|rover | 06:04 | |
| *** ysandeep|rover is now known as ysandeep|rover|lunch | 07:24 | |
| *** ysandeep|rover|lunch is now known as ysandeep|rover | 08:38 | |
| ianw | 2f46993d83ff4abb310ef7b4beced56ba96f0d9d is the first fixed commit | 09:11 |
| ianw | https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2f46993d83ff4abb310ef7b4beced56ba96f0d9d | 09:12 |
| ianw | amorin: ^ | 09:16 |
| ianw | if we can understand why this changes fixes things, then hopefully we can request a backport | 09:19 |
| ianw | s/this changes/this change/ | 09:19 |
| *** jpena|off is now known as jpena | 09:46 | |
| frickler | ianw: nice find. according to that commit it looks like we could have its effect by setting proper kernel cmdline options? | 10:01 |
| *** rlandy|out is now known as rlandy | 10:22 | |
| ianw | yeah, that might be a workaround, i'll have to really parse what's going on tomorrow. when the changelog is a couple of orders of magnitude bigger than the change there's something going on :) | 10:27 |
| *** dviroel_ is now known as dviroel | 11:19 | |
| *** sfinucan is now known as stephenfin | 12:04 | |
| *** arxcruz_ is now known as arxcruz | 12:34 | |
| mnaser | infra-root: could someone have a look at the zuul logs to see why this is getting +2'dby zuul but not merged? https://review.opendev.org/c/openstack/neutron-vpnaas/+/840309/2 | 12:44 |
| mnaser | (with no reports as to why) | 12:44 |
| frickler | mnaser: it needs a rebase, note the red "merged" entries in the relation chaing | 12:52 |
| mnaser | frickler: but the parent change was in the same state also but that merged | 12:53 |
| mnaser | oh nvm, i see it was rebased | 12:53 |
| frickler | mnaser: also, if you expanded the full change info on the left with "show all", you could see the parent info with a circled "I" that shows a popup note "Not current - rebase possible" | 12:54 |
| *** pojadhav is now known as pojadhav|afk | 14:05 | |
| *** ysandeep|rover is now known as ysandeep|dinner | 15:17 | |
| *** rlandy is now known as rlandy|mtg | 15:31 | |
| *** dviroel is now known as dviroel|lunch | 15:39 | |
| *** ysandeep|dinner is now known as ysandeep | 15:52 | |
| *** ysandeep is now known as ysandeep|out | 15:57 | |
| johnsom | Follow up on the fips/reboot/unbound issue from yesterday: We have a patch for devstack that works: https://review.opendev.org/c/openstack/devstack/+/842217 | 16:12 |
| johnsom | However frickler feels this should not be in devstack, but in the zuul level. | 16:13 |
| johnsom | Personally I think it should be in devstack as I think it's good to stop devstack early with a clear error rather than have it run down to error out with missing packages. | 16:13 |
| johnsom | Given the DNS issues we have had in the past, I don't think this only applies to the FIPS jobs. | 16:14 |
| johnsom | Wondering if anyone here has additional thoughts to add to that patch. | 16:14 |
| *** dviroel|lunch is now known as dviroel | 16:31 | |
| *** rlandy|mtg is now known as rlandy | 16:35 | |
| *** jpena is now known as jpena|off | 16:52 | |
| *** rlandy is now known as rlandy|mtg | 18:28 | |
| *** rlandy|mtg is now known as rlandy | 19:08 | |
| *** dviroel is now known as dviroel|out | 20:28 | |
| *** rlandy is now known as rlandy|bbl | 22:11 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!