Tuesday, 2020-09-29

*** sameo has quit IRC01:07
*** sgarzare has joined #kata-general07:19
*** Rene__ has quit IRC07:22
*** sameo has joined #kata-general07:30
*** pvdp66556 has quit IRC11:48
*** devimc has joined #kata-general12:08
*** fuentess has joined #kata-general12:24
*** devimc has quit IRC13:10
*** devimc has joined #kata-general13:11
kata-irc-bot<gerrit.schwerthelm> Thanks already for your help, Jakob. In the meantime I compiled multiple Kernels (using the kernel config from your `build_kernel.sh` script and enabling Virtualization modules), but the VM does not start up. :(  Error in the kernel log is: ```... EXT4-fs (pmem0p1): DAX unsupported by block device. ...  No filesystem could mount root, tried: ext4 ...``` Do you know this problem maybe or point me into a direction on how to13:11
kata-irc-botsolve this? I would be very thankful indeed. ,:)13:11
devimc@gerrit.schwerthelm I guess your image doesn't have the DAX metadata https://github.com/kata-containers/osbuilder/blob/master/image-builder/image_builder.sh#L9713:45
*** fuentess has quit IRC14:52
*** fuentess1 has joined #kata-general14:52
*** devimc has quit IRC14:56
*** devimc has joined #kata-general14:58
kata-irc-bot<gerrit.schwerthelm> Hey! I did not build an own OS image, I am still using the one that starts with the original kernel: ```... # /usr/share/defaults/kata-containers/configuration.toml [hypervisor.qemu] image = "/usr/share/kata-containers/kata-containers.img" ...```15:10
devimcsorry I need more context15:12
kata-irc-bot<gerrit.schwerthelm> No problem, thanks for willing to help. :slightly_smiling_face: I am trying to virtualize _again_ in a kata-container (nested virtualization with qemu hypervisor). For this, I attempted to build a kernel which has all the virtualization modules enabled. But the kernel that I built does not start due to the kernel panic error you saw. Only thing I altered is the path to the kernel in the `configuration.toml` , which points15:24
kata-irc-botto my custom-built kernel. Was not touching anything else.15:24
devimc@gerrit.schwerthelm thanks, did you use fragments to build your kernel ?15:29
devimchttps://github.com/kata-containers/packaging/tree/master/kernel/configs/fragments15:29
devimcto run kata containers you will need those CONFIGs15:29
devimcdoes your host kernel support nested virtualization ?15:30
kata-irc-bot<gerrit.schwerthelm> Host kernel supports nested virtualization and is a bare metal server. I think, I used the fragments. I've tried to build a 4.14.199 kernel config for x86 with the command: `./build-kernel.sh -v 4.14.199 -g intel -c $(pwd)/configs/x86_64_kata_kvm_4.14.x -f -d setup` In the end, the script asks me a lot of questions, which is why I aborted the script with a keyboard interrupt. Then, I changed into the newly created15:38
kata-irc-bot`kata-linux-4.14.199-89` directory and ran `make menuconfig` , enabled the Virtualization modules and saved everything to `.config`. Then built the kernel... and the error came up. :(15:39
devimc@gerrit.schwerthelm that's an old kernel - afaik we don't support fragments for that version15:40
devimccopy this config in your kernel directory15:41
devimchttps://github.com/kata-containers/packaging/blob/master/kernel/configs/x86_64_kata_kvm_4.14.x15:41
devimccp x86_64_kata_kvm_4.14.x .config15:41
devimcrun: make oldconfig15:41
devimcthem: make menuconfig  -  and enable nested virtualization15:42
devimc*then15:42
*** sgarzare has quit IRC16:06
kata-irc-bot<gerrit.schwerthelm> On `make oldconfig` ... how do I answer all those questions?16:20
devimc@gerrit.schwerthelm yes16:31
devimcfew questions I think16:31
devimcbecause you are using linux 4.14.19916:32
devimcand that config is for 4.14.6716:32
kata-irc-bot<gerrit.schwerthelm> You were right, it were just a few questions this time. Also the error changed with this kernel. It's now coming from user space, the kernel logs look clean now. :slightly_smiling_face:  This is now left: ```ERRO[0001] rpc error: code = Internal desc = Could not run process: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"proc\\\" to17:02
kata-irc-botrootfs \\\"/run/kata-containers/shared/containers/foo/rootfs\\\" at \\\"/proc\\\" caused \\\"mkdir /run/kata-containers/shared/containers/foo/rootfs/proc: read-only file system\\\"\""  arch=amd64 command=run container=foo name=kata-runtime pid=1640143 source=runtime rpc error: code = Internal desc = Could not run process: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused17:02
kata-irc-bot\"rootfs_linux.go:58: mounting \\\"proc\\\" to rootfs \\\"/run/kata-containers/shared/containers/foo/rootfs\\\" at \\\"/proc\\\" caused \\\"mkdir /run/kata-containers/shared/containers/foo/rootfs/proc: read-only file system\\\"\""```17:02
kata-irc-bot<gerrit.schwerthelm> Oh. And this is only happening if I run the standalone thing. The pod is starting now! Let me check. :slightly_smiling_face:17:03
devimc@gerrit.schwerthelm ok - at least the agent is running17:04
devimcthis could be a configuration problem17:04
*** stackedsax has quit IRC17:36
*** CeeMac has quit IRC17:38
*** stackedsax has joined #kata-general17:49
*** CeeMac has joined #kata-general17:50
kata-irc-bot<parthasl> Hello, Trying to use sysctl inside kata-container and getting this error. same with DIND. (kata 1.10.2 / containerd 1.13 / kubernetes) STDERR: sysctl: setting key "kernel.shmmax": Read-only file system ---- End output of sysctl -w "kernel.shmmax=17179869184" ---- Ran sysctl -w "kernel.shmmax=17179869184" returned 25518:46
devimc@parthasl using debugging console?18:51
devimcanyway - try adding `rw` to the kernel cmdline18:52
kata-irc-bot<parthasl> no, this is the output from the container and I see /proc/sys is RO18:53
devimc2.0 ?18:53
devimcahh no18:53
devimc1.x18:53
kata-irc-bot<parthasl> yes 1.10.218:54
devimcyeah try with the rw option18:55
kata-irc-bot<eric.ernst> :eyes:  I was going to test this later today too. Let me know what you find @parthasl18:59
kata-irc-bot<eric.ernst> Curious if you need to run privileged. You using CRI or just docker cli?19:01
kata-irc-bot<parthasl> using CRI, yes running in privileged mode without host devices option19:02
kata-irc-bot<eric.ernst> But that doesn’t work?19:02
kata-irc-bot<parthasl> no that doesnt work, returns << Read-only file system >>19:04
kata-irc-bot<eric.ernst> One thing you could do is run a prestart hook.19:09
kata-irc-bot<eric.ernst> Or,... maybe agent isn’t able to write either. Can you attempt via debug console?19:09
kata-irc-bot<eric.ernst> I think you may need remount rw19:10
kata-irc-bot<parthasl> sure will try19:11
kata-irc-bot<eric.ernst> AFAIU it should work from dev console and/or from a prestart hook19:27
*** devimc has quit IRC21:05
kata-irc-bot<archana.m.shinde> @parthasl How are you setting the sysctl ?21:28
kata-irc-bot<archana.m.shinde> Take a look at this: https://github.com/kata-containers/documentation/blob/8d5f2f0783fc2f848514c5d2fd1991fd7b755be7/how-to/how-to-use-sysctls-with-kata.md21:29
kata-irc-bot<archana.m.shinde> ```kernel.shmmax = 18446744073692774399 / # sysctl -w "kernel.shmmax=17179869184" kernel.shmmax = 17179869184 / #  / # sysctl kernel.shmmax kernel.shmmax = 17179869184```21:40
kata-irc-bot<archana.m.shinde> works21:40
kata-irc-bot<archana.m.shinde> @chen.bo @jose.carlos.venegas.m I was just trying out the sysctl integration test for cloud-hypervisor, it works21:41
kata-irc-bot<archana.m.shinde> I think we should enable that21:41
kata-irc-bot<chen.bo> @archana.m.shinde Thanks a lot. I just sent a PR to enable it here: https://github.com/kata-containers/tests/pull/2901.21:41
*** CeeMac has quit IRC21:52
*** stackedsax has quit IRC21:53
*** stackedsax has joined #kata-general21:56
*** CeeMac has joined #kata-general21:57
*** sameo has quit IRC22:42
kata-irc-bot<parthasl> Thank you @archana.m.shinde, able to set it inside kata container. DIND (Kata) scenario with CAP_SYS_ADMIN privilege, getting Read only file system error23:37
kata-irc-bot<archana.m.shinde> you mean your nested container has CAP_SYS_ADMIN?23:48
kata-irc-bot<archana.m.shinde> @parthasl ^23:48
kata-irc-bot<parthasl> yes23:49

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!