* Tao Peng (bergwolf@hyper.sh) wrote:
On Mon, Sep 24, 2018 at 10:32 PM, Dr. David Alan Gilbert <dgilbert@redhat.com> wrote:
* Qixuan Wu (qixuan.wu@linux.alibaba.com) wrote:
Hi Stefan,
Hi, Stefan is out at the moment, so I thought I'd reply.
Now we found 9pfs performance is poor, we are thinking of other solution for the rootfs of kata container.
As per the link, https://github.com/kata-containers/runtime/issues/279. Seem you are doing about nfs/vsock optimization, and nfs/vsock seems that can possible be the future default rootfs of kata container.
And I heard about qcow2+snapshot+virtio_scsi also. And how do you think about it as the rootfs of kata container? And do you know how the kata commumity think about this solution also and compare it with nfs/vsock ?
We're currently experimenting with something a bit different; we've got a setup that uses a modified version of the FUSE protocol running over vhost-user; it's: a) Got the filesystem access split out of qemu into a separate daemon - that's just a modified version of a normal FUSE filesystem daemon with the nice bit being that since it's a separate process you can do whatever isolation on it you want. b) But the latency is low because vhost-user means the daemon can read the request queue straight out of the guest memory c) We've got a setup with DAX so that the files are mapped straight into guest address space, so the overhead is very low for large files. d) We've got a caching scheme for metadata, which again removes a lot of latency. e) We've got some patches to use it in KATA; I can start a basic KATA guest with it.
This is the first public mention of it because I didn't want you waiting for a reply; but our code is still rather messy and experimental; give us a few weeks and as soon as it survives some smoke tests we'll make the code public.
Because we're reusing both FUSE and vhost-user the kernel changes are quite small, as are the qemu changes.
Hi Dave,
Thanks for sharing and sorry to chime in late. IIUC this is pretty like a vhost-user-fuse design. On the guest side, it uses a virtio-fuse frontend that takes any fs IO and encode it in fuse wire protocol and send through virtio. And the host daemon is a vhost-user-fuse process that just need to talk in fuse wire protocol with vhost-user fd rather than /dev/fuse. Am I understanding correctly?
Yes! That's exactly what it is. Thanks for confirming! It's a brilliant idea IMO and a truly native solution to the host fs sharing problem. Since it's based on FUSE, I suppose we can get better POSIX compliance than 9pfs. And I don't
On Wed, Sep 26, 2018 at 4:35 PM, Dr. David Alan Gilbert <dgilbert@redhat.com> wrote: think it requires any host kernel change, right?
Does it require any modification to the fuse wire protocol (e.g., include/uapi/linux/fuse.h)?
We've got a couple of extra opcodes for a performance trick, but otherwise it's just the same.
Great, I'm really looking forward to it! Thanks, Tao -- bergwolf@hyper.sh