Open Stack

> On 26 Jul 2021, at 11:44, David Gibson <kata-dev at gibson.dropbear.id.au> wrote:
> 
> Hello all,
> 
> I recently discovered that the kernel config generated by
> tools/packaging/kernel has CONFIG_PCI_MMCONFIG disabled.

I'm curious how much difference it makes in memory size (if at all).
I suspect things that were configured out were most likely selected to
reduce the kernel size more than for any other reason.


>  That's probably not a good idea, since that's a very atypical config for
> modern systems, and for complicated reasons it completely stops SHPC
> hotplug from working.

That's weird. Shouldn't that be abstracted behind pci_read_config_byte() and
the like? Now you make me curious why this breaks SHPC hotplug. If
you have the time to explain ;-) Does this seem like a kernel bug or some
hardware / architectural limitation?


> 
> So, I've filed
> https://github.com/kata-containers/kata-containers/pull/2311 to change
> it.
> 
> But..
> 
> I have reason to suspect that anyone using Kata's VFIO support is
> actually *relying* on this bogus guest behaviour to make things work.
> In all my experiements when I try to use VFIO devices with a more
> normal kernel, the device is detected in the guest after the agent's
> rescan and works, but only briefly.  When the SHPC hotplug completes
> 5s later, this interferes somehow and puts the device in a badly
> broken state (config space accesses fail completely)[0].
> 
> To make things harder, seeing how this affects the CI is also non-trivial:
>  1) Changing the config in the kata-containers repo will not (AFAICT)
>     trigger a rebuild of the cached guest kernel in the CI, and so it
>     will be tested effectively without the change.
>  2) The vfio CI testcase is pretty rudimentary.  I think it's quite
>     likely it could complete fast enough to succeed and shut down
>     before the SHPC hotplug completes and breaks the device
> 
> At this point I'm inclined to just push the change anyway and deal
> with the fallout as and when it occurs.  But does any one have a
> better idea?
> 
> 
> [0] In fact, if disable_idle_d3=N for the vfio-pci module, it even
>    puts the device into a broken state in the *host*, requiring a
>    host reboot to restore.
> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson
> _______________________________________________
> kata-dev mailing list
> kata-dev at lists.katacontainers.io
> http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev