Hi, guys I have Some doubts about virtiofs cache and dax . In the past two days, I have asked a few questions on WeChat and slack：https://katacontainers.slack.com/archives/C86U7NZND/p1621580960033100 , and some experts have answered some of them, which is very useful. But I still hope to get more detailed knowledge and information. @fidencio suggested that I post the question to the mailing list, so I posted it Question 1 On the official website of virtio-fs, in this article https://virtio-fs.gitlab.io/howto-qemu.html, it seems that $TESTDIR can set an existing directory on the host , but if I set the source parameter in the /usr/share/defaults/kata-containers/configuration.toml configuration file and specify a directory like /opt/kata-instance on the host # Format example: # ["-o", "arg1=xxx,arg2", "-o", "hello world", "--arg3=yyy"] # # see `virtiofsd -h` for possible options. virtio_fs_extra_args = ["-o","--thread-pool-size=1","-o","/opt/kata-instance"] kubelet will report error Warning FailedCreatePodSandBox 1s (x14 over 15s) kubelet, k8s05 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to launch qemu: exit status 1, error messages from qemu log: qemu-system- x86_64: cannot create PID file: Cannot open pid file: No such file or directory Question 2 I saw a picture that said that DAX uses the nvdimm device simulated by qemu. It seems to be a device for each VM, which is mapped to the host filesystem buffer cache. For example, there are 100 kata VMs on a single host. I set virtio_fs_cache_size=1024M, Then the host will use up to 1024M memory, or 1024M*100 memory?Also, can this shmem be writable or read-only? Is it only for kata pod images or all data read and write, including kubernetes persistence volume? From the official website of qemu, I found that nvdimm is used like this -machine pc, nvdimm -m $RAM_SIZE, slots=$N,maxmem=$MAX_SIZE -object memory-backend-file,id=mem1,share=on,mem-path=$PATH,size=$NVDIMM_SIZE -device nvdimm,id=nvdimm1,memdev=mem1 When the kata instance is created, the qemu process can see that it contains a parameter memory-backend-file, -object memory-backend-file,id=dimm1,size=3072M,mem-path=/dev/shm,share=on -numa node,memdev=dimm1 Is this cache dedicated to virtiofs? Where does dimm1 come from? It has nothing to do with nvdimm, right? Qeustion 3 Now my understanding is that kata has two caches 3.1 Data transmission between guest and virtiofsd requires a cache. The corresponding qemu parameter is -object memory-backend-file,id=dimm1,size=XXX. I guess the function of this cache may be the memory space of the simulated PCI device? It seems unlikely that it is the vring queue size used by the vhost-user protocol. Sorry, I'm not very proficient in qemu There is another question. Is the virtiofs cache specially allocated for each VM? If a single host starts many VMs, won't the memory consumption be very large? 3.2 dax, the corresponding parameter is virtio_fs_cache_size, there is no cache policy control in the kata configuration file. # Cache mode: virtio_fs_cache It controls the virtiofs cache, not DAX windows, right? Sorry，asked a lot of questions, it looks a bit scary ：） ryo from shanghai 2021-05-21