| kata-irc-bot | <zkaiser> If you say match the semantics of hooks for other container runtimes, do not get that, the hooks are meant to be applied to the container running in the VM. Why would I apply a prestart hook outside of the VM> | 08:14 |
|---|---|---|
| kata-irc-bot | <zkaiser> In the case of GPUs I need to inject special files into the container that are only available in the VM not outside of the VM> | 08:14 |
| kata-irc-bot | <zkaiser> Running the contianer outside of Kata and with Kata should "feel" the same. | 08:16 |
| kata-irc-bot | <zkaiser> Adding a prestart hook into the ocispec it would just fail because the outer runtime has no clue about the drivers and aux files that are inside the VM> | 08:16 |
| kata-irc-bot | <zkaiser> What is special about other runtimes that we need to run the hooks outside of the VM? | 08:17 |
| kata-irc-bot | <dgibson> well, so it actually depends on the hook | 09:37 |
| kata-irc-bot | <dgibson> but the oci spec says the context that each hook should run in | 09:37 |
| kata-irc-bot | <dgibson> for most of those that's the *host* context, *not* the container context | 09:37 |
| kata-irc-bot | <dgibson> hooks frequently access - or even change - things on the host side, rather than the container side | 09:38 |
| kata-irc-bot | <zkaiser> Ok. but doesn't the VM become the *host* for the Kata container? I want to change lets say sysctls for the Kata container, why would I change them on the host. I would like to see them in the guest. I am still not seeing a relevent use-case for running the hooks outside of the guest. | 09:40 |
| kata-irc-bot | <dgibson> heh.. well | 09:41 |
| kata-irc-bot | <dgibson> here you run into the fundamental problem with Kata - the OCI spec really doesn't anticipate a VM runtime, and lots of things don't really work that way | 09:41 |
| kata-irc-bot | <dgibson> so it all depends what the hook is doing | 09:42 |
| kata-irc-bot | <dgibson> in your case, sure, you want to be running in the VM | 09:42 |
| kata-irc-bot | <dgibson> but if you're trying to access tools that are deployed on the host, then running it in the VM clearly won't work | 09:42 |
| kata-irc-bot | <zkaiser> Ok which tools would that be? I mean what would you do on the host to influence the VM and the container running in it? | 09:44 |
| kata-irc-bot | <zkaiser> If you need tools than they should be packaged into the guest fs. | 09:45 |
| kata-irc-bot | <dgibson> that doesn't really make sense | 09:45 |
| kata-irc-bot | <dgibson> the person creating the pod usually doesn't have control of the guest fs | 09:45 |
| kata-irc-bot | <dgibson> "which tools would that be"? could be anything. the whole thing about hooks is they're used for handling weird situations that aren't handled by the built in container mechanisms | 09:46 |
| kata-irc-bot | <zkaiser> The user running the Pod would have control over what the guest-fs. A kube runtimeclass points to a containerd runtime configuration that points to a kata configuration which has a specific guest-fs, kernel combination. | 09:50 |
| kata-irc-bot | <zkaiser> That is what I am doing for GPU, vGPU use-cases. | 09:50 |
| kata-irc-bot | <dgibson> sure in *your* use case the same person has control | 09:50 |
| kata-irc-bot | <dgibson> but that doesn't generalize | 09:51 |
| kata-irc-bot | <zkaiser> It makes no sensed to run e.g. the GPU prestart hook on geust-fs that are not having the right bits. | 09:53 |
| kata-irc-bot | <dgibson> but doesn't e.g. for the case of a k8s deployment which has a bunch of pre-prepared hooks which assume the *real* host environment | 09:54 |
| kata-irc-bot | <dgibson> sure, but your GPU prestart hook is not the only hook out there | 09:54 |
| kata-irc-bot | <dgibson> this is not helped, of course, by the fact that there are kind of two sets of hooks | 09:54 |
| kata-irc-bot | <dgibson> there's the "legacy" ones, which are very vaguely defined in terms of semantics | 09:55 |
| kata-irc-bot | <dgibson> then there are the newer ones defined by OCI which are at least somewhat clearer (though still doesn't handle the case of a VM in the middle) | 09:55 |
| kata-irc-bot | <dgibson> unfortunately, which OCI calls the older ones deprecated, they appear to me the main ones used in practice (including "prestart" IIRC) | 09:56 |
| kata-irc-bot | <zkaiser> I am not saying to disable the prestart hooks on the outside runtime, my point is/was that prestart hooks should be executed "automatically" also in the guest, without specifying the "weird' config. If my OCI spec has a prestart hook defined than I would expect that this is run inside the VM. | 09:57 |
| kata-irc-bot | <dgibson> you expect that | 09:58 |
| kata-irc-bot | <dgibson> someone else does not | 09:58 |
| kata-irc-bot | <dgibson> Kata (and the rest of the stack) has no way to know which case is which | 09:58 |
| kata-irc-bot | <dgibson> and running both outside and inside is dangerous, because it doesn't have to be idempotent | 09:59 |
| kata-irc-bot | <dgibson> not to mention that if it fails in either one, that will stop the container starting, I believe | 09:59 |
| kata-irc-bot | <zkaiser> Right, currently we do not know if outside or inside. | 10:00 |
| kata-irc-bot | <liuj97> Call for competition:$ | 12:02 |
| kata-irc-bot | <liuj97> A justified gvisor：https://github.com/QuarkContainer/Quark | 12:02 |
| kata-irc-bot | <liuj97> | 12:02 |
| kata-irc-bot | <zkaiser> Folks, per default a PCI bridge is added which consumes a slot on the root bus. Is there still need for a pcie-pci-bridge? `````` | 15:58 |
| kata-irc-bot | <zkaiser> Per default nothing is attached ```-[0000:00]-+-00.0 Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller +-01.0 Red Hat, Inc. Virtio console +-02.0-[01]-- +-03.0 Red Hat, Inc. Virtio SCSI +-04.0 Red Hat, Inc. Virtio RNG +-05.0-[02]----00.0 NVIDIA Corporation Device 20b8``` | 15:58 |
| kata-irc-bot | <zkaiser> The `02.0-[01]` is the pcie-pci-bridge. | 15:59 |
| kata-irc-bot | <zkaiser> If needed we can hotplug a pcie-pci-bridge on a pcie-root-port. | 17:29 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!