| kata-irc-bot | <itskumaresan> Hi Team, Good day to you. I am in the process of building the guest image for SUSE. The process I followed mentioned below, On a running SUSE Linux machine: git clone the https://github.com/kata-containers/kata-containers.git cd kata-containers/tools/osbuilder sudo -E PATH=$PATH make BUILD_METHOD=dracut image The image generated is working as expected. Question: Would like to understand, does the image that gets | 02:00 |
|---|---|---|
| kata-irc-bot | generated is of SUSE Linux? I tried kata-runtime exec <ID> however, I was not able to see the SUSE Linux or not. Note: Tested Debian distro and working as expected. Link: https://github.com/kata-containers/kata-containers/tree/main/tools/osbuilder#dracut-based-image | 02:00 |
| kata-irc-bot | <dgibson> not really looking at this particular aspect | 03:16 |
| kata-irc-bot | <dgibson> I suspect this is quite hard to address | 03:17 |
| kata-irc-bot | <dgibson> we probably need enough memory space allocated to the bridge, but the bridge must be created before we know the large bar device is going to be attached | 03:17 |
| kata-irc-bot | <zkaiser> Is there any trade-off allocating enough memory ignoring if it is going to be a small or large BAR device? | 08:43 |
| kata-irc-bot | <dgibson> sorry, I'm on another call now, I'll talk later | 08:47 |
| kata-irc-bot | <dgibson> hello, free now, looking at your questions | 09:42 |
| kata-irc-bot | <dgibson> it's not really a question of memory allocation | 09:43 |
| kata-irc-bot | <dgibson> "memory space" here means allocations from the PCI memory address space | 09:43 |
| kata-irc-bot | <dgibson> not actual RAM | 09:44 |
| kata-irc-bot | <dgibson> there's a single memory address space for an entire PCI domain (== PCI host bridge) | 09:44 |
| kata-irc-bot | <dgibson> that has to be subdivided between all the bridges / busses in the domain | 09:44 |
| kata-irc-bot | <dgibson> and the problem is that we need to create the bridges before we know what devices are going to go under them, so we don't know in advance how much address space the bridge will need | 09:45 |
| kata-irc-bot | <dgibson> roughly speaking | 09:45 |
| kata-irc-bot | <dgibson> I don't know all the details | 09:45 |
| kata-irc-bot | <dgibson> it's pretty complicated, because technically the allocation is up to the guest kernel, but in practice it often expects the guest firmware to have done the allocation for it (depends on platform), and the firmware in turn may vary how it does things based on hints from the hypervisor | 09:47 |
| kata-irc-bot | <dgibson> it's not impossible to reallocate everything at runtime... but that will generally require a complete rescan of the PCI bus, so it may interrupt access to drivers for *all* devices on the bus, which on x86 generally means every device | 09:48 |
| kata-irc-bot | <dgibson> TBH, the current PCI code is barely held together with chewing gum and string at the moment... figuring out how to get it to handle large BARs somewhat reliably is a long way off from my radar | 09:49 |
| kata-irc-bot | <dgibson> maybe someone else is looking at it, but I'm not aware of them | 09:49 |
| kata-irc-bot | <zkaiser> Thanks, I remember there was an PR to remove the rescan for PCI so essentially this was the behavior some time ago but this led to other errors. | 10:20 |
| kata-irc-bot | <zkaiser> One thing that also puzzled me how do the sleep(2sec) hacks have cirucmvented the situation with large BARs. | 10:21 |
| kata-irc-bot | <james.o.hunt> Hi All - please consider helping review PRs in these lesser known Kata repos: https://github.com/kata-containers/ci/pulls, https://github.com/kata-containers/.github/pulls. | 11:12 |
| kata-irc-bot | <zkaiser> With `hotplug_vfio_on_root_bus = false` and `pcie_root_port = 0` i have: ```root@982a634cc5fb:/# lspci -tv -[0000:00]-+-00.0 Device 8086:29c0 +-01.0 Device 1af4:1003 +-02.0-[01]----01.0 Device 10de:20b9``` Behind a pci-bridge, with `hotplug_vfio_on_root_bus = true` and `pcie_root_port = 1` I have: ```root@982a634cc5fb:/# lspci -tv -[0000:00]-+-00.0 Device 8086:29c0 +-01.0 Device 1af4:1003 | 14:50 |
| kata-irc-bot | +-02.0-[01]-- +-03.0 Device 1af4:1004 +-04.0 Device 1af4:1005 +-05.0-[02]----00.0 Device 10de:20b9``` The very same hiearchy, the 0x10de should be at a root port. | 14:50 |
| kata-irc-bot | <zkaiser> @james.o.hunt Is anyone else looking into ^^ this, besides David? | 15:14 |
| kata-irc-bot | <eric.ernst> PTAL ya'll 0 version bump for rc0 release: https://github.com/kata-containers/kata-containers/pull/3845 https://github.com/kata-containers/tests/pull/4554 | 15:21 |
| kata-irc-bot | <james.o.hunt> @zkaiser - Are you referring specifically to https://github.com/kata-containers/kata-containers/issues/835 ? | 15:26 |
| kata-irc-bot | <steven> Just to round this off for anyone catching up. Under https://github.com/kata-containers/kata-containers/issues/3830 containerd was bumped to Wang Lei’s 1.6.1 rebased fork in `CCv0` and we’ve done a merge of `main` into `CCv0` since main updated to 1.6.1 too, so I think we are all good. Big thanks to Fidencio for co-ordinating this. | 15:28 |
| kata-irc-bot | <zkaiser> Yes, I am hitting the very same. as David explained before the whole PCI thing "is barely held together with chewing gum and string at the moment.." | 15:28 |
| kata-irc-bot | <zkaiser> I am updating the doc on NVIDIA GPUs but hitting this issue. | 15:29 |
| kata-irc-bot | <zkaiser> I found also some older PRs and Issues that have similiar problems, looks like a long-standing issue. | 15:29 |
| kata-irc-bot | <zkaiser> I am also interested to see some more "complicated" topologies, especially for use-cases like GPUDirect NIC+GPU. | 15:30 |
| kata-irc-bot | <zkaiser> Rather then plugging everything to the root ports, add a PCIe swtich e.g. underlying userspace stack may not work if it detects that both devices are plugged into the root ports. | 15:31 |
| kata-irc-bot | <zkaiser> Of course keeping a very simple topology for the "normal" use-case. | 15:34 |
| kata-irc-bot | <samuel.ortiz> Thanks so much @steven @wllenyj | 16:39 |
| kata-irc-bot | <james.o.hunt> I'm not aware of anyone else working on it. If you'd like to get involved, that would be great. You could always ask for further help from the community on the mailing list or bring the issue up at one of the Architecture Committee meetings (https://github.com/kata-containers/community#architecture-committee-meetings). | 17:48 |
| kata-irc-bot | <eric.ernst> Can use reack for test PR please/thanks! See https://github.com/kata-containers/tests/pull/4554 | 21:57 |
| kata-irc-bot | <dgibson> I think your heirarchy above is not the same in the two cases | 22:57 |
| kata-irc-bot | <dgibson> it looks the same but in one case the device is plugged into a pcie to pci bridge, in the other into a pcie root port | 22:57 |
| kata-irc-bot | <dgibson> they'll look the same to `lspci -t` since a root port acts as a logical bride | 22:58 |
| kata-irc-bot | <eric.ernst> 2.4 release notes :thread: | 22:58 |
| kata-irc-bot | <eric.ernst> @feng.wang -- direct assign work is _awesome_ - thanks for getting it in. Want to give a small blurb (1-2 sentence/bullets) for release notes? | 22:59 |
| kata-irc-bot | <eric.ernst> @fidencio @julio.montes -- there've been improvements with respect to TDX support that's introduced in 2.4 -- can you give a small blurb (or correct me)? | 22:59 |
| kata-irc-bot | <fidencio> Sure thing, I will go through those Tomorrow and suggest changes. | 23:00 |
| kata-irc-bot | <eric.ernst> @bergwolf -- can you share release notes (or delegate) for nydus? What's new / latest user facing status? | 23:01 |
| kata-irc-bot | <eric.ernst> @dgibson -- anything new from 2.3 --> 2.4 w.r.t PCI hotplug/VFIO that we can highlight for release notes? | 23:04 |
| kata-irc-bot | <eric.ernst> @liubin0329 -- what about a summary for huge pages? | 23:05 |
| kata-irc-bot | <dgibson> well, `vfio_mode = vfio` is new in 2.4 IIRC | 23:05 |
| kata-irc-bot | <eric.ernst> @steven - does it make sense to have any release notes re: agent config file support (is this something an end user may see / use)? | 23:16 |
| kata-irc-bot | <feng.wang> @eric.ernst Sounds good. I’ll give it to you later today. | 23:19 |
| kata-irc-bot | <eric.ernst> @jianyong.wu - any release notes you think make sense re: vCPU hotplug support for ARM (on QEMU)? | 23:20 |
| kata-irc-bot | <eric.ernst> @fgiudici - anything you want to highlight for changes in kata-monitor? Anything user facing? Key improvements? | 23:26 |
| kata-irc-bot | <eric.ernst> (all, see https://github.com/kata-containers/kata-containers/releases/tag/2.4.0-rc0) | 23:45 |
| kata-irc-bot | <eric.ernst> 2.4 is going to be a great release -- if I "at'd" you above, please take a look and either provide feedback to me for release notes, or take a look and improve the draft posted above. Thanks! | 23:46 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!