[kata-dev] VFIO hotplug in Q35

Adrian Moreno amorenoz at redhat.com
Thu May 28 09:09:44 UTC 2020


Hi,

Currently VFIO passthrough on q35 is broken because of two connected issues:

- pci-rescan race condition: https://github.com/kata-containers/runtime/issues/2664
- Pcie-root-port failing: https://github.com/kata-containers/runtime/issues/2678

So far, my understanding is that:

- The PCI hotplug/rescan race condition affects both bridge and root-port
hotplugging and regardless of the device's properties
- The rescan is kept because of the 5s delay shpc introduces when devices are
hotplugged on a pcie-to-pci bridge

Am i missing something?

Based on that I would suggest:

1 - Disable the rescan for pcie-root-port hotplugging as that kind does not
incur in the 5s delay that justifies the rescan AFAIK.

2 - Study a way of systematically avoiding the race on bridge hotplugging. For
this we need further investigation. I personally have many questions:
   - Can the 5s delay be avoided in some other way?
   - If we are scanning the bus anyway, can we make shcp be manualy triggered? I
see it has a "polling mode". Has that been investigated?
   - What is the cost (in startup time, memory, etc) of having a high number of
pcie-root-ports (to avoid the cost of an exact estimation)

3 - Investigate whether we can fix this in the kernel. I have not looked into
this but maybe shpc could detect the situation and "fail" gracefully not messing
up the pci device.

What do you think?

Thanks,
Adrian




More information about the kata-dev mailing list