On 06/08/2018 16:22, Allison Randal wrote:
This is partly for the practical reason that it's time-consuming and expensive to update a large codebase, but it's also partly because the "improvements" often come with performance trade-offs that make virtual machines unusable.
This is actually not true of most papers you cite; in fact Kata Containers itself is an example of a performance/functionality/security trade-off, since VM-based containers will usually be somewhat slower than OS-level containers. An important point is that often containers actually run as virtual machines on a hypervisor that you don't control. So you can assess the impact of the various precautions/measures from the papers you cite, on both nested-VMs-as-containers and OS-containers-in-a-VM. For example:
https://doi.org/10.1145/3193111.3193118 "Hardening Hypervisors against Vulnerabilities in Instruction Emulators"
The paper says that if you can assume a new-enough processor so that you do not have to emulate big real mode, you can cut a good amount of code from the instruction emulator. I think Google is already doing this in GCE because they don't care about old processors. In this case, if you run your container on a vulnerable hypervisor, you could use the vulnerabilities to get privilege escalation for the entire guest. If you run your container as a virtual machine, privilege escalation would only extend to the container.
"Nioh: Hardening The Hypervisor by Filtering Illegal I/O Requests to Virtual Devices" Nice. This is a bit more tricky to assess; it's an (important) QoI issue but it's not a fundamental problem with virtual machines in general or KVM in particular. In a perfect world, the device emulator would be confined using OS-level security features such as cgroups, capabilities, seccomp, etc. However, this is not always the case and in particular it's not for Kata containers (though its predecessor Clear Containers did some OS-level isolation of the device emulator). In that case, indeed you could exploit the device emulator to run code outside the container.
https://doi.org/10.1109/TDSC.2015.2429132 https://doi.org/10.1109/TIFS.2015.2505680
"Using Virtual Machine Allocation Policies to Defend against Co-Resident Attacks in Cloud Computing" "Preventing Co-Residence Using Semi-Supervised Learning" The same issue and solution applies to virtual machine and container-based PaaS.
"An Architecture for Concurrent Execution of Secure Environments in Clouds" Heh, one of the authors was a student of mine during his bachelor. :) Basically a virtualized TXT, which is not available to containers. If you don't have the functionality in the paper, you can still use virtual TPM with virtual machines, but you only have one TPM shared by all containers.
https://www.usenix.org/conference/atc17/technical-sessions/presentation/gopa...
"Multi-Hypervisor Virtual Machines: Enabling an Ecosystem of Hypervisor-level Services" Using nested VMs to isolate the various introspectors from the guest. Very interesting (KVM doesn't have introspection but it's coming) but it's difficult to assess it because it's more of an architecture than a security feature.
While containers aren't currently considered very secure either (e.g. https://doi.org/10.1109/i-Society.2016.7854163, https://doi.org/10.1145/3058060.3058085), the isolation that containers do provide is a collection of features added to more modern kernels (namespaces, cgroups, capabilities, seccomp, etc), in a modular approach that has allowed containers to more readily adopt existing advances in security, and also means containers can more easily pivot to incorporate new low-level security features that become available in the future.
Note that these same modular features can be applied to virtual machines simply by running the device emulation in a container. This provides isolation for the parts that are most complex and deal the most with untrusted data from the guest. I think it would be disingenuous, and it somewhat reminds me to Theo de Raadt's famous virtualization quote[1], to say that OS-level containers can be more secure than virtual machines. Capabilities and the like are nice, but software has bugs and that's why you defend in depth. Virtual machines are just an additional layer in the defend-in-depth onion. [1] https://marc.info/?l=openbsd-misc&m=119318909016582 Paolo