[Kata-hypervisor] kata requirements for image formats and devices

Paolo Bonzini pbonzini at redhat.com
Mon Aug 6 16:53:59 UTC 2018


On 06/08/2018 16:22, Allison Randal wrote:
> This is partly for the practical reason that
> it's time-consuming and expensive to update a large codebase, but it's
> also partly because the "improvements" often come with performance
> trade-offs that make virtual machines unusable.

This is actually not true of most papers you cite; in fact Kata
Containers itself is an example of a performance/functionality/security
trade-off, since VM-based containers will usually be somewhat slower
than OS-level containers.

An important point is that often containers actually run as virtual
machines on a hypervisor that you don't control.  So you can assess the
impact of the various precautions/measures from the papers you cite, on
both nested-VMs-as-containers and OS-containers-in-a-VM.  For example:

> https://doi.org/10.1145/3193111.3193118
"Hardening Hypervisors against Vulnerabilities in Instruction Emulators"

The paper says that if you can assume a new-enough processor so that you
do not have to emulate big real mode, you can cut a good amount of code
from the instruction emulator.  I think Google is already doing this in
GCE because they don't care about old processors.

In this case, if you run your container on a vulnerable hypervisor, you
could use the vulnerabilities to get privilege escalation for the entire
guest.  If you run your container as a virtual machine, privilege
escalation would only extend to the container.

> https://doi.org/10.1145/3134600.3134648

"Nioh: Hardening The Hypervisor by Filtering Illegal I/O Requests to
Virtual Devices"

Nice.  This is a bit more tricky to assess; it's an (important) QoI
issue but it's not a fundamental problem with virtual machines in
general or KVM in particular.  In a perfect world, the device emulator
would be confined using OS-level security features such as cgroups,
capabilities, seccomp, etc.  However, this is not always the case and in
particular it's not for Kata containers (though its predecessor Clear
Containers did some OS-level isolation of the device emulator).  In that
case, indeed you could exploit the device emulator to run code outside
the container.

> https://doi.org/10.1109/TDSC.2015.2429132
> https://doi.org/10.1109/TIFS.2015.2505680

"Using Virtual Machine Allocation Policies to Defend against Co-Resident
Attacks in Cloud Computing"
"Preventing Co-Residence Using Semi-Supervised Learning"

The same issue and solution applies to virtual machine and
container-based PaaS.

> https://doi.org/10.1145/2517488.2517489

"An Architecture for Concurrent Execution of Secure
Environments in Clouds"

Heh, one of the authors was a student of mine during his bachelor. :)
Basically a virtualized TXT, which is not available to containers.  If
you don't have the functionality in the paper, you can still use virtual
TPM with virtual machines, but you only have one TPM shared by all
containers.

> https://www.usenix.org/conference/atc17/technical-sessions/presentation/gopalan

"Multi-Hypervisor Virtual Machines: Enabling an Ecosystem of
Hypervisor-level Services"

Using nested VMs to isolate the various introspectors from the guest.
Very interesting (KVM doesn't have introspection but it's coming) but
it's difficult to assess it because it's more of an architecture than a
security feature.

> While containers aren't currently considered very secure either (e.g.
> https://doi.org/10.1109/i-Society.2016.7854163,
> https://doi.org/10.1145/3058060.3058085), the isolation that containers
> do provide is a collection of features added to more modern kernels
> (namespaces, cgroups, capabilities, seccomp, etc), in a modular approach
> that has allowed containers to more readily adopt existing advances in
> security, and also means containers can more easily pivot to incorporate
> new low-level security features that become available in the future.

Note that these same modular features can be applied to virtual machines
simply by running the device emulation in a container.  This provides
isolation for the parts that are most complex and deal the most with
untrusted data from the guest.

I think it would be disingenuous, and it somewhat reminds me to Theo de
Raadt's famous virtualization quote[1], to say that OS-level containers
can be more secure than virtual machines.  Capabilities and the like are
nice, but software has bugs and that's why you defend in depth.  Virtual
machines are just an additional layer in the defend-in-depth onion.

[1] https://marc.info/?l=openbsd-misc&m=119318909016582

Paolo



More information about the Kata-hypervisor mailing list