[kata-dev] virtio-fs + VM Templating
Dr. David Alan Gilbert
dgilbert at redhat.com
Thu Jun 13 11:09:26 UTC 2019
* Peng Tao (bergwolf at hyper.sh) wrote:
> On Thu, Jun 6, 2019 at 1:33 AM Dr. David Alan Gilbert
> <dgilbert at redhat.com> wrote:
> > * Castelino, Manohar R (manohar.r.castelino at intel.com) wrote:
> > > > It's very similar to the trick that NEMU uses for templating.
> > > > With the x-ignore-shared-ram migration capability enabled, migration will
> > > > not write to the migration stream any RAM block that had the
> > > > shared=on flag on the qemu commandline. So you should then be able
> > > > to restart from the migration stream and existing RAM image.
> > >
> > > So does it mean we can drop our vm-templating patches and move to using " x-ignore-shared-ram" on QEMU 4.0.
> > >
> > > Today we only need two patches, which will come down to a single patch which would bring up closer to upstream QEMU 4.0 which would be ideal.
> > > https://github.com/kata-containers/packaging/blob/master/qemu/patches/4.0.x/0001-9p-removing-coroutines-of-9p-to-increase-the-I-O-per.patch
> > > https://github.com/kata-containers/packaging/blob/master/qemu/patches/4.0.x/0002-migration-add-capability-to-bypass-the-shared-memory.patch
> > Yes, I'm hoping that with 4.0 you can avoid 0002 - but I've not tried
> > it; I'd be interested in your results.
> Hi Dave,
> I gave it a try and failed to do vm templating with x-ignore-shared.
> One key difference between x-ignore-shared and Lai's
> migrate-bypass-shared patch is that Lai doesn't verify shared memory
> block upon ram load. OTOH x-ignore-shared rely on ram share property
> to reconstruct the destination guest memory. In kata, we make use of
> the fact that destination ram can be private to implement vm
> templating feature so that multiple new guests can share map the same
> template VM memory privately (memory-backend-file share=off).
(Copying in Yury who wrote those patches).
> The way we implement vm templating in kata is:
> 1. Start the template VM:
> qemu-system-x86 -m 2G \
> -object memory-backend-file,id=mem0,size=2G,share=on,mem-path=/tmpfs/template-memory
> -numa node,memdev=mem0
> 2. Stop the template VM, set migration bypass-shared-memory
> capability, migrate exec:cat>/tmpfs/state, quit it
> 3. Start target VM:
> qemu-system-x86 -m 2G \
> -object memory-backend-file,id=mem0,size=2G,share=off,mem-path=/tmpfs/template-memory
> -numa node,memdev=mem0 \
> -incoming "exec:cat /tmpfs/template-state"
> 4. Start more target VMs like 3
> There are some major differences between above and the example given
> by x-ignore-shared patchset:
> 1. on target vm, memory-backend-file is mapped privately (share=off)
> 2. no migration capability is set on the target vm
> 3. it is possible to create multiple target VMs based on the same template VM
> I have made some hacky change to make x-ignore-shared work for vm
> templating, mostly by reverting the commit "migration: Add
> capabilities validation" and reconstructing destination VM ram without
> using migration share capability or ram share property.
> So I want to ask about your opinion on how to make vm templating work.
> Shall we change qemu to load ram without the x-ignore-shared
> capability? Or add another capability to implement similar feature
> along side x-ignore-shared?
Can we first try a different hack; if you set the ignore shared on the
destination as well as the source, what happens? I guess it'll hit one
of the errors in ram_load? Which one?
>  https://lists.gnu.org/archive/html/qemu-devel/2019-02/msg04332.html
>  https://github.com/bergwolf/qemu/commits/vm-templating
> Into something rich and strange.
Dr. David Alan Gilbert / dgilbert at redhat.com / Manchester, UK
More information about the kata-dev