Open Stack

Hi,

Currently VFIO passthrough on q35 is broken because of two connected issues:

- pci-rescan race condition: https://github.com/kata-containers/runtime/issues/2664
- Pcie-root-port failing: https://github.com/kata-containers/runtime/issues/2678

So far, my understanding is that:

- The PCI hotplug/rescan race condition affects both bridge and root-port
hotplugging and regardless of the device's properties
- The rescan is kept because of the 5s delay shpc introduces when devices are
hotplugged on a pcie-to-pci bridge

Am i missing something?

Based on that I would suggest:

1 - Disable the rescan for pcie-root-port hotplugging as that kind does not
incur in the 5s delay that justifies the rescan AFAIK.

2 - Study a way of systematically avoiding the race on bridge hotplugging. For
this we need further investigation. I personally have many questions:
   - Can the 5s delay be avoided in some other way?
   - If we are scanning the bus anyway, can we make shcp be manualy triggered? I
see it has a "polling mode". Has that been investigated?
   - What is the cost (in startup time, memory, etc) of having a high number of
pcie-root-ports (to avoid the cost of an exact estimation)

3 - Investigate whether we can fix this in the kernel. I have not looked into
this but maybe shpc could detect the situation and "fail" gracefully not messing
up the pci device.

What do you think?

Thanks,
Adrian