| *** gmmaha has quit IRC | 03:38 | |
| *** gmmaha has joined #kata-dev | 03:40 | |
| *** gmmaha has quit IRC | 03:45 | |
| *** gmmaha has joined #kata-dev | 03:49 | |
| *** sameo has joined #kata-dev | 05:09 | |
| *** igordc has quit IRC | 06:10 | |
| *** pcaruana has joined #kata-dev | 06:30 | |
| *** sgarzare has joined #kata-dev | 06:50 | |
| *** sameo has quit IRC | 06:56 | |
| *** jodh has joined #kata-dev | 06:58 | |
| *** sameo has joined #kata-dev | 07:33 | |
| *** davidgiluk has joined #kata-dev | 08:03 | |
| *** jugs1 is now known as jugs | 11:43 | |
| brtknr | any kata devs around? the invite behind this link is no longer active: bit.ly/KataSlack | 12:18 |
|---|---|---|
| stefanha | brtknr: Apparently I can invite people. Privmsg me an email address and I'll give it a try. | 12:42 |
| brtknr | bharat@stackhpc.com :) | 12:43 |
| stefanha | brtknr: Done | 12:43 |
| brtknr | stefanha: thanks, got it | 12:43 |
| kata-irc-bot | <xu> http://bit.ly/katacontainersslack | 12:43 |
| kata-irc-bot | <xu> could change to this one | 12:44 |
| *** devimc has joined #kata-dev | 13:10 | |
| brtknr | thanks xu | 13:20 |
| *** lpetrut has joined #kata-dev | 13:21 | |
| brtknr | anyone else attempted qemu-virtiofsd with kata-deploy? | 13:23 |
| *** fuentess has joined #kata-dev | 13:29 | |
| *** pcaruana has quit IRC | 13:32 | |
| *** canyounot has joined #kata-dev | 13:32 | |
| brtknr | Failed create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox runtime: no runtime for "kata-qemu-virtiofs" is configured | 13:33 |
| *** pcaruana has joined #kata-dev | 14:04 | |
| *** dklyle has quit IRC | 14:18 | |
| *** dklyle has joined #kata-dev | 14:43 | |
| brtknr | stefanha: could you shed light on this: https://seashells.io/v/354m9xEX | 14:55 |
| brtknr | I've managed to get a bit further by inserting qemu-virtiofs as an additional shim in kata deploy | 14:56 |
| stefanha | brtknr: It suggests that qemu-virtiofsd closed the vhost-user UNIX domain socket. It probably means qemu-virtiofsd terminated with an error. | 14:57 |
| stefanha | brtknr: virtiofsd should be launched with --syslog -d so you get details in the journal | 14:58 |
| stefanha | brtknr: Sometimes it's useful to replace binaries that kata-runtime executes with wrapper scripts so you can change the command-line options, strace the process, etc. | 14:59 |
| brtknr | https://seashells.io/p/QpZGvq2N | 15:00 |
| stefanha | brtknr: Nothing obvious in that output. | 15:00 |
| brtknr | maybe this one: https://seashells.io/p/9Anf6HZY | 15:00 |
| *** devimc has quit IRC | 15:02 | |
| stefanha | brtknr: I think debugging isn't enabled in virtiofsd. There should be more output like "virtio_loop: Waiting for VU event" | 15:02 |
| stefanha | brtknr: Are you sure virtiofsd is running with the -d option? The -d option should be set by kata-runtime if hypervisor.qemu enable_debug = true. | 15:03 |
| stefanha | (in /etc/kata-containers/configuration.toml) | 15:03 |
| brtknr | that is set to true | 15:03 |
| stefanha | brtknr: In that case you could move the binary aside and put a wrapper script there: | 15:04 |
| stefanha | #!/bin/bash | 15:04 |
| stefanha | exec strace -f -o /tmp/virtiofsd.log path/to/virtiofsd.orig "$@" | 15:04 |
| stefanha | The strace output will show what the virtiofsd process is doing. | 15:05 |
| stefanha | I use this trick with kata-runtime when I want to debug what QEMU, virtiofsd, etc are doing. This way you can interpose between kata-runtime and the binaries it launches. | 15:06 |
| *** jodh has quit IRC | 15:06 | |
| kata-irc-bot | <greg.bock> I had trouble getting args to parse right when I tried this a few months ago. Does using exec help with that? | 15:06 |
| kata-irc-bot | <greg.bock> hmm, actually i was trying to add args in my wrapper maybe that was the issue. | 15:08 |
| stefanha | greg.bock: exec doesn't help but using exactly "$@" helps with shell word splitting. | 15:10 |
| brtknr | stefanha: https://seashells.io/p/eYFfGRtq | 15:11 |
| stefanha | brtknr: Ah, a seccomp error | 15:14 |
| kata-irc-bot | <greg.bock> Yeah I think I had tried all manner of dereferencing and quoting and eventually just patched the code | 15:15 |
| stefanha | brtknr: I wonder if the environment in which your virtiofsd binary is being launched does not allow seccomp for some reason. | 15:16 |
| *** canyounot is now known as theinternet | 15:16 | |
| *** theinternet is now known as canyounot | 15:16 | |
| stefanha | brtknr: I think your host has an older kernel version. | 15:18 |
| stefanha | brtknr: There is a patch to fix this: | 15:18 |
| stefanha | brtknr: virtio-fs-v0.3 requires a host kernel that supports the seccomp SCMP_ACT_KILL_PROCESS feature. | 15:19 |
| stefanha | brtknr: Newer virtio-fs code has a fallback for older kernels. | 15:20 |
| brtknr | stefanha: aha | 15:21 |
| stefanha | brtknr: Which host kernel version are you running? uname -r | 15:21 |
| brtknr | 3.10.0-1062.1.2.el7.x86_64 | 15:21 |
| stefanha | brtknr: Upstream added the SCMP_ACT_KILL_PROCESS feature in 4.14 | 15:21 |
| brtknr | its centos 7.7 | 15:21 |
| stefanha | ok | 15:21 |
| stefanha | brtknr: Are you already in a position where you can recompile virtiofsd? | 15:22 |
| stefanha | If yes, then I can link a patch. | 15:22 |
| brtknr | i can definitely try :) | 15:22 |
| stefanha | brtknr: Are you currently using a virtiofsd binary distributed by Kata Containers? | 15:23 |
| brtknr | stefanha: i am | 15:23 |
| brtknr | version 7.29 | 15:23 |
| stefanha | Okay, that explains it. Because if you recompile virtiofsd on your CentOS 7.7 host then seccomp will succeed, | 15:23 |
| stefanha | it's just that your binary was probably compiled on a build host with a newer kernel. | 15:24 |
| stefanha | So no virtiofsd patch is even required. | 15:24 |
| brtknr | stefanha: oh, so i just need to recompile and thats all? | 15:24 |
| stefanha | BTW the latest virtio-fs code falls back at runtime so even older kernels are supported. It's just virtio-fs-v0.3 that doesn't fall back. | 15:24 |
| stefanha | brtknr: Yep | 15:24 |
| brtknr | fatal error: sys/capability.h: No such file or directory | 15:28 |
| brtknr | stefanha: | 15:28 |
| stefanha | brtknr: yum install libcap-devel | 15:30 |
| *** igordc has joined #kata-dev | 15:37 | |
| brtknr | stefanha: aaaaah it seems to be working | 15:38 |
| stefanha | brtknr: \o/ | 15:38 |
| brtknr | even the dd if=/dev/zero of=path/to/beegfs/foo.dat oflag=direct bs=4k count=1000 doesnt collapse | 15:39 |
| *** sameo has quit IRC | 15:52 | |
| *** devimc has joined #kata-dev | 16:13 | |
| *** igordc has quit IRC | 16:31 | |
| *** sameo has joined #kata-dev | 16:50 | |
| *** sgarzare has quit IRC | 16:56 | |
| *** lpetrut has quit IRC | 17:01 | |
| *** igordc has joined #kata-dev | 18:36 | |
| *** canyounot has quit IRC | 19:05 | |
| *** canyounot has joined #kata-dev | 19:07 | |
| *** canyounot has quit IRC | 19:07 | |
| *** canyounot has joined #kata-dev | 19:09 | |
| *** davidgiluk has quit IRC | 19:11 | |
| *** canyounot has quit IRC | 19:13 | |
| *** canyounot has joined #kata-dev | 19:14 | |
| *** canyounot has joined #kata-dev | 19:15 | |
| *** sameo has quit IRC | 19:28 | |
| *** eernst has joined #kata-dev | 20:00 | |
| *** eernst has quit IRC | 20:04 | |
| *** pcaruana has quit IRC | 20:25 | |
| *** pcaruana has joined #kata-dev | 20:34 | |
| *** canyounot is now known as canyounot|gone | 20:46 | |
| *** canyounot|gone has quit IRC | 20:46 | |
| *** pcaruana has quit IRC | 20:51 | |
| *** devimc has quit IRC | 22:35 | |
| *** fuentess has quit IRC | 22:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!