On 2020/5/28 17:09, Adrian Moreno wrote:
Hi,
Currently VFIO passthrough on q35 is broken because of two connected issues:
- pci-rescan race condition: https://github.com/kata-containers/runtime/issues/2664
- Pcie-root-port failing: https://github.com/kata-containers/runtime/issues/2678
So far, my understanding is that:
- The PCI hotplug/rescan race condition affects both bridge and root-port
hotplugging and regardless of the device's properties
- The rescan is kept because of the 5s delay shpc introduces when devices are
hotplugged on a pcie-to-pci bridge
Am i missing something?
Based on that I would suggest:
1 - Disable the rescan for pcie-root-port hotplugging as that kind does not incur in the 5s delay that justifies the rescan AFAIK.
2 - Study a way of systematically avoiding the race on bridge hotplugging. For this we need further investigation. I personally have many questions: - Can the 5s delay be avoided in some other way? - If we are scanning the bus anyway, can we make shcp be manualy triggered? I see it has a "polling mode". Has that been investigated? - What is the cost (in startup time, memory, etc) of having a high number of pcie-root-ports (to avoid the cost of an exact estimation)
3 - Investigate whether we can fix this in the kernel. I have not looked into this but maybe shpc could detect the situation and "fail" gracefully not messing up the pci device.
Is it a known issue to the kernel community? If not, we should report to kernel maintainers first.
Thanks, Tao