| kata-irc-bot | <feng.wang> @linux Sorry for the late reply. We had to implement our own image pruner to clean up the unused images in the device mapper case. I’m not aware of any other alternatives to use virtio-blk for rootfs in Kata. | 05:03 |
|---|---|---|
| kata-irc-bot | <anastassios.nanos> Hi there! Since kata 2.2 IIRC jailer is enabled by default in the FC case | 06:18 |
| kata-irc-bot | <anastassios.nanos> can you try disabling the jailer in the config just to test if that works? just comment out the ‘jailer=‘ line in the config | 06:19 |
| kata-irc-bot | <norbj.d_kata> thanks for your answer :pray: i've tried it but unfortunately it did not work :( | 08:11 |
| kata-irc-bot | <anastassios.nanos> so the paths are related to the jailer case -- if you see the config with the stripped paths then the jailer is enabled | 08:12 |
| kata-irc-bot | <norbj.d_kata> maybe i made a mistake, i'll try again and see :eyes: | 08:13 |
| kata-irc-bot | <zaripovbulat> Thanks, it should be the case, totally forgot about it! Will try it out soon. Can you comment on pvc experience with kata/firecracker? Am I missing some pieces of the puzzle or is it not possible to enable users to have the sameish workflow with regard to pvcs? | 08:19 |
| kata-irc-bot | <norbj.d_kata> my bad, with the `jailer` lines commented, I can see the whole path: ``` "vsock": { "guest_cid": 3, "uds_path": "/run/vc/firecracker/661494a7797a292c762a45ef894c7add/root/kata.hvsock", "vsock_id": "root" },``` but I got the same error: ```FATA[0030] failed to create shim task: Failed to check if grpc server is working: rpc error: code = DeadlineExceeded desc = timed | 08:20 |
| kata-irc-bot | out connecting to hybrid vsocket hvsock:/run/vc/firecracker/661494a7797a292c762a45ef894c7add/root/kata.hvsock: unknown``` | 08:20 |
| kata-irc-bot | <anastassios.nanos> can you try checking the version of the firecracker binary? should be `firecracker --version` | 08:24 |
| kata-irc-bot | <anastassios.nanos> IIRC somewhere around this release there was an upgrade to FC v0.23.1 | 08:25 |
| kata-irc-bot | <norbj.d_kata> `Firecracker v0.23.1` (same for jailer) | 08:25 |
| kata-irc-bot | <norbj.d_kata> yep i saw that in the commits :slightly_smiling_face: i have checked with kata 2.1.1 and the firecracker embedded is also 0.23.1 :thinking_face: | 08:27 |
| kata-irc-bot | <anastassios.nanos> ok, so its not the version -- could it be that there was a regression the tests couldn't catch? can you try another (more recent) release? or is it the case you would like to run on 2.2 ? | 08:28 |
| kata-irc-bot | <norbj.d_kata> in fact, i've tried with 2.4.1 and since it didn't work, i've tried with downgrading until it worked. That's why I found the culprit was 2.1.1 :slightly_smiling_face: i've tried with 2.4.1 (same error), I can try to build kata with `main` | 08:30 |
| kata-irc-bot | <anastassios.nanos> hmm, that's weird, we should have caught this regression with tests -- we will need a bit of time to repro and we'll report back -- could you open an issue to track this? | 08:31 |
| kata-irc-bot | <norbj.d_kata> already opened :grinning: https://github.com/kata-containers/kata-containers/issues/4412 | 08:32 |
| kata-irc-bot | <anastassios.nanos> sorry I missed that ;) OK, we'll report back here and on the issue | 08:32 |
| kata-irc-bot | <norbj.d_kata> thanks! i'll try with `main` version and keep you informed | 08:33 |
| kata-irc-bot | <norbj.d_kata> update: same error with `main` (last commit https://github.com/kata-containers/kata-containers/commit/5bd81ba232cd8ead900af0626b98c0aacd87b737) (firecracker and jailer are in version 0.23.4) | 09:05 |
| kata-irc-bot | <linux> Sweet, thanks Feng :thumbsup: How do you guys handle the Kubernetes scheduler? If the devmapper disk usage is 100% Kubernetes does not set the `DiskPressure` condition on node and still tries to schedule pods there. | 09:44 |
| kata-irc-bot | <anastassios.nanos> so, we tried that on 2.4.1 & 2.2.1 & 2.1.1 -- seems to boot fine -- can you share more details about your setup? what we've tried is on a bare-metal machine, but I suspect there should be no issue as you mentioned that 2.1.1 works fine | 12:28 |
| kata-irc-bot | <anastassios.nanos> can you verify that 2.1.1 boots on that same machine 2.4.1 does not? | 12:28 |
| kata-irc-bot | <norbj.d_kata> yes i've never changed my VM :slightly_smiling_face: | 12:37 |
| kata-irc-bot | <norbj.d_kata> but it's not a bare metal machine, rather a vm from a cloud provider (scaleway) | 12:38 |
| kata-irc-bot | <norbj.d_kata> do you need more info than the `Show kata-collect-data.sh details` in my issue? https://github.com/kata-containers/kata-containers/issues/4412 | 12:39 |
| kata-irc-bot | <anastassios.nanos> maybe (just to be sure) `ctr plugins ls |grep devmapper` -- but I suspect it's there | 15:49 |
| kata-irc-bot | <anastassios.nanos> tried with nerdctl also, on another node we have, also works fine | 15:50 |
| kata-irc-bot | <anastassios.nanos> both jailed and non-jailed, firecracker v0.23.1-4, kata 2.4.1, the latest main branch and 2.1.1 | 15:53 |
| kata-irc-bot | <norbj.d_kata> ```$ ctr plugins ls | grep devmapper io.containerd.snapshotter.v1 devmapper linux/amd64 ok``` weird, it seems ok i've followed this gist to create devmapper: https://gist.github.com/norbjd/4142e1f2345d2b9d81e73ac815b21ddf | 15:54 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!