On Tue, Jul 17, 2018 at 6:32 AM, Boeuf, Sebastien sebastien.boeuf@intel.com wrote:
Hi Stefan,
Thanks for bringing up this topic.
About better isolation, we have 2 issues opened here: https://github.com/kata-containers/runtime/issues/344 https://github.com/kata-containers/runtime/issues/345 They're about cgroups and namespaces for the qemu process itself. You've mentioned it's not worth isolating the process through namespaces but I think it is since the kernel running inside the VM is very likely to be different from the kernel running on the host, meaning the exploit might be harder on the host itself even if it happened inside the guest.
Now, I agree that running the VM as root is also a real breach regarding security and we also have an issue to address this concern here: https://github.com/kata-containers/runtime/issues/389 I think you summarized the blockers pretty well right now. If qemu is not root, we have some issues accessing the files on the host through 9p, but you also seem to provide a solution here and it'd be great if you could comment on the Github issue. One small thing that needs to happen too is to make sure we run kata-runtime with a user that at least can have access to /dev/kvm, but other than that, nothing else I can think of.
Last thing, about libvirt-go, is there all the support needed by Kata (hotplug of a bunch of different device types) ?
libvirt-go does have that and runv is using libvirt-go (https://github.com/hyperhq/runv/tree/master/hypervisor/libvirt) for a long time. Last I checked, there are still a few missing pieces of libvirt though: 1. vsock -- runv only supports vsock through the qemu hypervisor 2. vm templating -- runv works around it by using a customized qemu wrapper
IIUC, libvirt is set to be hypervisor agnostic so I'm not sure if these qemu specific features can get in there. However, if we only consider basic usage of Kata, libvirt-go should work w/o issues.
Thanks, Sebastien
From: Stefan Hajnoczi [stefanha@redhat.com] Sent: Monday, July 16, 2018 5:31 AM To: kata-dev@lists.katacontainers.io Cc: Daniel Berrange Subject: [kata-dev] Improving isolation in Kata
Kata currently launches QEMU as user 'root' and I haven't found isolation mechanisms that would confine the QEMU process in the event of a guest escape.
Inevitably there will be a security bug that allows a malicious guest to escape to QEMU, so it's important that the QEMU process is unprivileged and isolated.
Running QEMU as 'root' simplifies 9p configuration by allowing the QEMU process to access files with any uid/gid. QEMU does support other 9p configurations where the QEMU process does not run as root, including the virtfs-proxy-helper privilege separation approach.
Are there any other reasons for running the QEMU process as root?
Creation of new namespaces, and handling of process cgroups on the host side, would require CAP_SYS_ADMIN. Though we do not have them right now (only netns is enforced), we need to be aware of the limitation since there are people working on adding these abilities at host side.
For comparison, libvirt launches QEMU as the unprivileged 'qemu' user with a unique SELinux context. This makes it much harder to access other guests or escape to the host if the guest compromises QEMU.
Is there any work underway to improve isolation in Kata?
Two directions that come to mind are:
- Design isolation from scratch for Kata, including unprivileged uid/gid, namespaces, and SELinux. This requires changes to Kata because the QEMU process will not be able to open arbitrary files (e.g. device hotplug, 9p configuration).
Does it prevent general device hotplug, or just hotplugging of local devices (e.g., `/dev/sdb` etc.)? I'm mostly considering two cases: 1. remote devices like rbd/iscsi/nbd 2. CPU and memory devices
FWIW, relying on namespaces doesn't add a lot of security because if the guest is malicious, then that may well be because of a Linux namespaces security hole - the attacker could just use the same exploit again to escape from QEMU to the host.
- Use libvirt-go (API) or libvirt-go-xml (XML) instead of govmm to get strong isolation. This would also slim down virtcontainers and remove the dependency on govmm (no more command-line munging and defining object models for QEMU command-line concepts).
Even if we support libvirt, I would think it still reasonable to keep both qemu and libvirt-go implementations due to the feature differences. OTOH, libvirt-go can be a good place to support other hypervisors like xen.
Cheers, Tao