Open Stack

On Wed, Sep 26, 2018 at 4:35 PM, Dr. David Alan Gilbert
<dgilbert at redhat.com> wrote:
> * Tao Peng (bergwolf at hyper.sh) wrote:
>> On Mon, Sep 24, 2018 at 10:32 PM, Dr. David Alan Gilbert
>> <dgilbert at redhat.com> wrote:
>> > * Qixuan Wu (qixuan.wu at linux.alibaba.com) wrote:
>> >> Hi Stefan,
>> >
>> > Hi,
>> >   Stefan is out at the moment, so I thought I'd reply.
>> >
>> >> Now we found 9pfs performance is poor, we are thinking of other solution
>> >> for the rootfs of kata container.
>> >>
>> >> As per the link, https://github.com/kata-containers/runtime/issues/279.
>> >> Seem you are doing about nfs/vsock optimization, and nfs/vsock seems
>> >> that can possible be the future default rootfs of kata container.
>> >>
>> >> And I heard about qcow2+snapshot+virtio_scsi also. And how do you think
>> >> about it as the rootfs of kata container? And do you know how the kata
>> >> commumity think about this solution also and compare it with nfs/vsock ?
>> >
>> > We're currently experimenting with something a bit different;
>> > we've got a setup that uses a modified version of the FUSE protocol
>> > running over vhost-user;  it's:
>> >   a) Got the filesystem access split out of qemu into a separate daemon
>> >       - that's just a modified version of a normal FUSE filesystem daemon
>> >       with the nice bit being that since it's a separate process you
>> >       can do whatever isolation on it you want.
>> >   b) But the latency is low because vhost-user means the daemon can read
>> >      the request queue straight out of the guest memory
>> >   c) We've got a setup with DAX so that the files are mapped straight
>> >      into guest address space, so the overhead is very low for large
>> >      files.
>> >   d) We've got a caching scheme for metadata, which again removes a lot
>> >      of latency.
>> >   e) We've got some patches to use it in KATA; I can start a basic KATA
>> >      guest with it.
>> >
>> > This is the first public mention of it because I didn't want you waiting
>> > for a reply; but our code is still rather messy and experimental; give
>> > us a few weeks and as soon as it survives some smoke tests we'll make
>> > the code public.
>> >
>> > Because we're reusing both FUSE and vhost-user the kernel changes are
>> > quite small, as are the qemu changes.
>> >
>> Hi Dave,
>>
>> Thanks for sharing and sorry to chime in late. IIUC this is pretty
>> like a vhost-user-fuse design. On the guest side, it uses a
>> virtio-fuse frontend that takes any fs IO and encode it in fuse wire
>> protocol and send through virtio. And the host daemon is a
>> vhost-user-fuse process that just need to talk in fuse wire protocol
>> with vhost-user fd rather than /dev/fuse. Am I understanding
>> correctly?
>
>
> Yes! That's exactly what it is.
Thanks for confirming! It's a brilliant idea IMO and a truly native
solution to the host fs sharing problem. Since it's based on FUSE, I
suppose we can get better POSIX compliance than 9pfs. And I don't
think it requires any host kernel change, right?

>
>> Does it require any modification to the fuse wire protocol
>> (e.g., include/uapi/linux/fuse.h)?
>
> We've got a couple of extra opcodes for a performance trick, but
> otherwise it's just the same.
Great, I'm really looking forward to it!

Thanks,
Tao

-- 
bergwolf at hyper.sh