Hi Stefan, Daniel, Thanks for the feedback here. I have just tried the virtfs-proxy-helper and it works like a charm, allowing me to run Qemu from a non root user. Now, I haven't tried this yet, but I wanted to know if you thought about any reason using virtio-net-pci would not work by using the vhost backend like this: -netdev tap,id=network-0,vhost=on,vhostfds=3:4:5:6:7:8:9:10,fds=11:12:13:14:15:16:17:18 -device driver=virtio-net-pci,netdev=network-0,mac=02:42:ac:11:00:02,mq=on,vectors=18 Same question about block devices. We're doing heavy uses of virtio-scsi in order to share a block device (our container rootfs) with the guest. Is it possible to run Qemu non root in this case ? Should I put the user running Qemu in the disk group so that Qemu can read/write to this block device ? Last use case is VFIO for device passthrough. After we unbind the device from its driver on the host, should we change the ownership of the device created (something like /dev/vfio/X) after it has been bound to vfio-pci driver ? Thanks Sebastien ________________________________________ From: Daniel P. Berrangé [berrange@redhat.com] Sent: Tuesday, July 17, 2018 1:59 AM To: Boeuf, Sebastien Cc: Stefan Hajnoczi; kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] Improving isolation in Kata On Mon, Jul 16, 2018 at 10:32:22PM +0000, Boeuf, Sebastien wrote:
Hi Stefan,
Thanks for bringing up this topic.
About better isolation, we have 2 issues opened here: https://github.com/kata-containers/runtime/issues/344 https://github.com/kata-containers/runtime/issues/345 They're about cgroups and namespaces for the qemu process itself. You've mentioned it's not worth isolating the process through namespaces but I think it is since the kernel running inside the VM is very likely to be different from the kernel running on the host, meaning the exploit might be harder on the host itself even if it happened inside the guest.
I think perhaps stefan oversimplified. The better point is that namespaces alone are not sufficient security protection against a malicious attacker, especially if the guest is relying on the same namespace separation for the containers it is running. When libvirt runs QEMU it will use multiple layers of security protection. The most important is sVirt/SELinux which applies mandatory access control, whitelisting resources that the guest is permitted to touch. Second, it will apply regular discretionary access control by running QEMU as an unprivileged user ID, with the privileged libvirtd setting file permissions on the few resources it is allowed to use. Third it will use namespaces, setting up a custom mount namespace which hides most of /dev, only allowing the few device nodes the guest is permitted to use. Fourth it will use cgroups to apply resource controls via a number of the controllers the kernel exposes. Finally it can use iptables+ebtables to setup filtering of guest traffic to provide protection against MAC & IP address spooofing and other arbitrary rules. All of these layers are configurable so you can decide how much protection you get, but by default you'll get all of them except the firewalling.
Now, I agree that running the VM as root is also a real breach regarding security and we also have an issue to address this concern here: https://github.com/kata-containers/runtime/issues/389 I think you summarized the blockers pretty well right now. If qemu is not root, we have some issues accessing the files on the host through 9p, but you also seem to provide a solution here and it'd be great if you could comment on the Github issue.
Yes, 9p is the biggest problem with running non-root, and would require use of some kind of proxied solution. You could perhaps use capabilities to grant the CAP_DAC_OVERRIDE bit but this does reduce the level of protection you have quite dramatically, so proxy is probably better if 9p is a must have.
One small thing that needs to happen too is to make sure we run kata-runtime with a user that at least can have access to /dev/kvm, but other than that, nothing else I can think of.
Last thing, about libvirt-go, is there all the support needed by Kata (hotplug of a bunch of different device types) ?
libvirt-go is just a shim to the main native libvirt.so library API, which in turn talks to libvirtd. The libvirtd QEMU driver supports hotplug for a wide array of device types (disks, disk controllers, leases, PCI & USB host devices, USB redirection devices, character devices, RNG devices, memory DIMMS, shmem devices, watchdogs, input devices, and vsock devices). Libvirt would be happy to see more devices support as people raise a need for it. Given the wide & diverse usage of libvirt I'd expect we would support the vast majority of functionality you currently need, and any feature gaps would be fairly small & straightforward to address. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|