Open Stack

* Tao Peng (bergwolf at hyper.sh) wrote:
> On Mon, Sep 24, 2018 at 10:32 PM, Dr. David Alan Gilbert
> <dgilbert at redhat.com> wrote:
> > * Qixuan Wu (qixuan.wu at linux.alibaba.com) wrote:
> >> Hi Stefan,
> >
> > Hi,
> >   Stefan is out at the moment, so I thought I'd reply.
> >
> >> Now we found 9pfs performance is poor, we are thinking of other solution
> >> for the rootfs of kata container.
> >>
> >> As per the link, https://github.com/kata-containers/runtime/issues/279.
> >> Seem you are doing about nfs/vsock optimization, and nfs/vsock seems
> >> that can possible be the future default rootfs of kata container.
> >>
> >> And I heard about qcow2+snapshot+virtio_scsi also. And how do you think
> >> about it as the rootfs of kata container? And do you know how the kata
> >> commumity think about this solution also and compare it with nfs/vsock ?
> >
> > We're currently experimenting with something a bit different;
> > we've got a setup that uses a modified version of the FUSE protocol
> > running over vhost-user;  it's:
> >   a) Got the filesystem access split out of qemu into a separate daemon
> >       - that's just a modified version of a normal FUSE filesystem daemon
> >       with the nice bit being that since it's a separate process you
> >       can do whatever isolation on it you want.
> >   b) But the latency is low because vhost-user means the daemon can read
> >      the request queue straight out of the guest memory
> >   c) We've got a setup with DAX so that the files are mapped straight
> >      into guest address space, so the overhead is very low for large
> >      files.
> >   d) We've got a caching scheme for metadata, which again removes a lot
> >      of latency.
> >   e) We've got some patches to use it in KATA; I can start a basic KATA
> >      guest with it.
> >
> > This is the first public mention of it because I didn't want you waiting
> > for a reply; but our code is still rather messy and experimental; give
> > us a few weeks and as soon as it survives some smoke tests we'll make
> > the code public.
> >
> > Because we're reusing both FUSE and vhost-user the kernel changes are
> > quite small, as are the qemu changes.
> >
> Hi Dave,
> 
> Thanks for sharing and sorry to chime in late. IIUC this is pretty
> like a vhost-user-fuse design. On the guest side, it uses a
> virtio-fuse frontend that takes any fs IO and encode it in fuse wire
> protocol and send through virtio. And the host daemon is a
> vhost-user-fuse process that just need to talk in fuse wire protocol
> with vhost-user fd rather than /dev/fuse. Am I understanding
> correctly?


Yes! That's exactly what it is.

> Does it require any modification to the fuse wire protocol
> (e.g., include/uapi/linux/fuse.h)?

We've got a couple of extra opcodes for a performance trick, but
otherwise it's just the same.

Dave

> 
> Cheers,
> Tao
> 
> -- 
> bergwolf at hyper.sh
--
Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK