| kata-irc-bot | <gmaglion> @simon.kaegi Hi, could you give me some example on how you use overlayfs?, because I thought cap_sys_admin was only required to mount overlayfs | 16:12 |
|---|---|---|
| kata-irc-bot | <gmaglion> So I can reproduce your configuration for testing, thanks | 16:13 |
| kata-irc-bot | <simon.kaegi> Sure... ```apiVersion: v1 kind: Pod metadata: name: untrusted-dind annotations: io.kubernetes.cri.untrusted-workload: "true" spec: containers: - name: docker image: docker:dind imagePullPolicy: IfNotPresent command: - dockerd-entrypoint.sh - dockerd - --host=unix:///var/run/docker.sock - --host=tcp://localhost:2375 - --mtu=1400 securityContext: privileged: | 16:14 |
| kata-irc-bot | true ports: - containerPort: 2375 nodeSelector: katacontainers.io/kata-runtime: "true"``` | 16:14 |
| kata-irc-bot | <simon.kaegi> That's using "untrusted" so you might use `runtimeClassName: kata` in your set-up | 16:15 |
| kata-irc-bot | <gmaglion> Thanks | 16:16 |
| kata-irc-bot | <simon.kaegi> but basically... if you `kubectl exec -it untrusted-dind -- sh` and `docker info` you should get a good idea | 16:16 |
| kata-irc-bot | <gmaglion> do you need sys_admin because trusted xattr? | 16:17 |
| kata-irc-bot | <gmaglion> I just want to try "user" xattr instead an see if works | 16:17 |
| kata-irc-bot | <simon.kaegi> I think that was the original reasoning .... more than two years ago now ;) | 16:18 |
| kata-irc-bot | <simon.kaegi> FWIW this is what we use currently... ```apiVersion: v1 kind: Pod metadata: name: untrusted-dind spec: runtimeClassName: kata-clh containers: - name: docker image: docker:dind command: ["sh", "-c"] args: - if [[ $(df -PT /var/lib/docker | awk 'NR==2 {print $2}') == virtiofs ]]; then apk add e2fsprogs && truncate -s 20G /tmp/disk.img && mkfs.ext4 /tmp/disk.img && | 16:18 |
| kata-irc-bot | mount /tmp/disk.img /var/lib/docker; fi && dockerd-entrypoint.sh --mtu=1400; securityContext: privileged: true nodeSelector: katacontainers.io/kata-runtime: "true"``` | 16:18 |
| kata-irc-bot | <gmaglion> ok, I'll let you know if I found something :slightly_smiling_face: | 16:18 |
| kata-irc-bot | <simon.kaegi> Yep I'll try too. Noticed the binary a fair bit bigger... is that because everything is statically linked? | 16:19 |
| kata-irc-bot | <gmaglion> everything but libc, libseccom, libcapng, etc.. | 16:21 |
| kata-irc-bot | <gmaglion> well, not technically statically linked, the deps are like source libraries | 16:22 |
| kata-irc-bot | <fidencio> @simon.kaegi, how much bigger? Are you building with `--release`? | 17:20 |
| kata-irc-bot | <fidencio> The binary is debug mode is around 40M, but the "release" binary should be around 3~4M, at most. That's 1M (~25%) bigger than what you'd get from the C version. | 17:21 |
| kata-irc-bot | <simon.kaegi> I'm not building yet -- just looking and waiting for now. | 17:21 |
| kata-irc-bot | <fidencio> Yeah, that's because it's the "debug" / "development" version. | 17:24 |
| kata-irc-bot | <simon.kaegi> Also seriously wondering if we have been hitting the "--inode-file-handles" problem in production... | 17:24 |
| kata-irc-bot | <fidencio> Once you build it with `cargo build --release` it goes down to 3.5~4M | 17:24 |
| kata-irc-bot | <gmaglion> @simon.kaegi what "--inode-file-handles" problem? | 19:21 |
| kata-irc-bot | <simon.kaegi> we sometime run into issues without nfs when the file handle counts get high. Certainly a flag I think would be reasonable to use is all I meant. | 20:25 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!