Open Stack

On Tue, Jul 17, 2018 at 6:32 AM, Boeuf, Sebastien
<sebastien.boeuf at intel.com> wrote:
> Hi Stefan,
>
> Thanks for bringing up this topic.
>
> About better isolation, we have 2 issues opened here:
> https://github.com/kata-containers/runtime/issues/344
> https://github.com/kata-containers/runtime/issues/345
> They're about cgroups and namespaces for the qemu process itself. You've mentioned it's not worth isolating the process through namespaces but I think it is since the kernel running inside the VM is very likely to be different from the kernel running on the host, meaning the exploit might be harder on the host itself even if it happened inside the guest.
>
> Now, I agree that running the VM as root is also a real breach regarding security and we also have an issue to address this concern here: https://github.com/kata-containers/runtime/issues/389
> I think you summarized the blockers pretty well right now. If qemu is not root, we have some issues accessing the files on the host through 9p, but you also seem to provide a solution here and it'd be great if you could comment on the Github issue.
> One small thing that needs to happen too is to make sure we run kata-runtime with a user that at least can have access to /dev/kvm, but other than that, nothing else I can think of.
>
> Last thing, about libvirt-go, is there all the support needed by Kata (hotplug of a bunch of different device types) ?
>
libvirt-go does have that and runv is using libvirt-go
(https://github.com/hyperhq/runv/tree/master/hypervisor/libvirt) for a
long time. Last I checked, there are still a few missing pieces of
libvirt though:
1. vsock -- runv only supports vsock through the qemu hypervisor
2. vm templating -- runv works around it by using a customized qemu wrapper

IIUC, libvirt is set to be hypervisor agnostic so I'm not sure if
these qemu specific features can get in there. However, if we only
consider basic usage of Kata, libvirt-go should work w/o issues.

> Thanks,
> Sebastien
>
> ________________________________________
> From: Stefan Hajnoczi [stefanha at redhat.com]
> Sent: Monday, July 16, 2018 5:31 AM
> To: kata-dev at lists.katacontainers.io
> Cc: Daniel Berrange
> Subject: [kata-dev] Improving isolation in Kata
>
> Kata currently launches QEMU as user 'root' and I haven't found
> isolation mechanisms that would confine the QEMU process in the event of
> a guest escape.
>
> Inevitably there will be a security bug that allows a malicious guest to
> escape to QEMU, so it's important that the QEMU process is unprivileged
> and isolated.
>
> Running QEMU as 'root' simplifies 9p configuration by allowing the QEMU
> process to access files with any uid/gid.  QEMU does support other 9p
> configurations where the QEMU process does not run as root, including
> the virtfs-proxy-helper privilege separation approach.
>
> Are there any other reasons for running the QEMU process as root?
>
Creation of new namespaces, and handling of process cgroups on the
host side, would require CAP_SYS_ADMIN. Though we do not have them
right now (only netns is enforced), we need to be aware of the
limitation since there are people working on adding these abilities at
host side.

> For comparison, libvirt launches QEMU as the unprivileged 'qemu' user
> with a unique SELinux context.  This makes it much harder to access
> other guests or escape to the host if the guest compromises QEMU.
>
> Is there any work underway to improve isolation in Kata?
>
> Two directions that come to mind are:
>
> 1. Design isolation from scratch for Kata, including unprivileged
>    uid/gid, namespaces, and SELinux.  This requires changes to Kata
>    because the QEMU process will not be able to open arbitrary files
>    (e.g. device hotplug, 9p configuration).
>
Does it prevent general device hotplug, or just hotplugging of local
devices (e.g., `/dev/sdb` etc.)? I'm mostly considering two cases:
1. remote devices like rbd/iscsi/nbd
2. CPU and memory devices

>    FWIW, relying on namespaces doesn't add a lot of security because if
>    the guest is malicious, then that may well be because of a Linux
>    namespaces security hole - the attacker could just use the same
>    exploit again to escape from QEMU to the host.
>
> 2. Use libvirt-go (API) or libvirt-go-xml (XML) instead of govmm to get
>    strong isolation.  This would also slim down virtcontainers and
>    remove the dependency on govmm (no more command-line munging and
>    defining object models for QEMU command-line concepts).
>
Even if we support libvirt, I would think it still reasonable to keep
both qemu and libvirt-go implementations due to the feature
differences. OTOH, libvirt-go can be a good place to support other
hypervisors like xen.

Cheers,
Tao
-- 
bergwolf at hyper.sh