On Wed, Dec 26, 2018 at 10:32:06AM +0800, jiangyiwen wrote:
On 2018/12/11 3:25, Stefan Hajnoczi wrote:
Dear Kata Containers Community, I'm delighted to announce the first release of virtio-fs, a new shared file system for virtual machines that is designed for container use cases, including shared volumes.
Unlike virtio-9p and NFS over AF_VSOCK, virtio-fs aims to take advantage of the co-location between the virtual machine and the hypervisor in order to achieve local file system semantics and improve performance.
For example, it can use Linux Direct Access (DAX) to access file contents directly from the host page cache. This reduces communication with the file server and avoids duplicating data into each sandbox VM. It also means that mmap MAP_SHARED on a shared volume is coherent between sandbox VMs.
The Linux kernel code (including performance numbers) has been posted here: https://marc.info/?l=linux-fsdevel&m=154446243324255&w=2
Kata Containers integration is already available so you can benchmark and test virtio-fs. The project is under active development and we still expect to make significant changes based on feedback and collaboration.
We hope virtio-fs is interesting as a next step in overcoming virtio-9p's performance and limitations. Let us know how it performs!
You can read more about virtio-fs here: https://virtio-fs.gitlab.io/
The Kata HowTo is here: https://virtio-fs.gitlab.io/howto-kata.html
The Kata runtime and agent changes are fairly straightforward and comparable to virtio-9p. There are several other code changes due to using a Fedora initramfs, systemd, and modular kernel. These are not essential to virtio-fs but are simply how I preferred to develop and test.
The FAQ on the virtio-fs website explains the main technical features. Please let me know if you have any questions or need help getting it running! I'm also on #kata-dev IRC if you need a hand.
Stefan
kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
Hi Stefan,
It is amazing to see such a great job. I setup a virtio-fs environment using qemu, virtio-fs using "cache=none, dax" mode, and I found some problems as follows:
- Now it don't support write file with direct_io, then I filter
O_DIRECT flag in libfuse-daemon when opening file, and success. So I think it may be related to dio alignment and mmap(). Like 9p also filter the O_DIRECT flag in qemu.
Can you give some more details. What's the error you face and what do you mean by "I filter O_DIRECT flag in libfuse-daemon".
- Cache-size can't be hot increased, I mean after the memory layout
is like "Data cache->Metadata Cache->Journal Cache, once data cache size is changed, the metadata cache and journal cache offset will be changed, this memory data need to be moved. I don't know if there are any plans to support the dynamic increase. Because I understand user can have the demand of hot increase cache.
I think one should be able to increase cache size. Right now that feature is not there but can't think why it can't be added. For example, for data cache, we just need to some kind of notification to virtio-fs driver and it will allocate more free ranges internally and start using these for next allocation onwards.
- Why real mmap operation is executed in qemu process instead of libfuse
daemon? I hope the data plane doesn't not need to pass through qemu again.
If libfuse daemon does mmap() in its address space, then how guest kernel will see those mappings. So, IIUC, qemu needs to call mmap().
- When I use fio with psync, write/read only 20% improve than 9p, I think
psync can be more used than mmap. This user-case may need to improved.
Can you send me your fio job. I want to try it out.
Thanks Vivek