Open Stack

On Thu, Jun 13, 2019 at 7:09 PM Dr. David Alan Gilbert
<dgilbert at redhat.com> wrote:
>
> * Peng Tao (bergwolf at hyper.sh) wrote:
> > On Thu, Jun 6, 2019 at 1:33 AM Dr. David Alan Gilbert
> > <dgilbert at redhat.com> wrote:
> > >
> > > * Castelino, Manohar R (manohar.r.castelino at intel.com) wrote:
> > > > > It's very similar to the trick that NEMU uses for templating.
> > > > > With the x-ignore-shared-ram migration capability enabled, migration will
> > > > > not write to the migration stream any RAM block that had the
> > > > > shared=on flag on the qemu commandline.   So you should then be able
> > > > > to restart from the migration stream and existing RAM image.
> > > >
> > > > So does it mean we can drop our vm-templating patches and move to using " x-ignore-shared-ram" on QEMU 4.0.
> > > >
> > > > Today we only need two patches, which will come down to a single patch which would bring up closer to upstream QEMU 4.0 which would be ideal.
> > > > https://github.com/kata-containers/packaging/blob/master/qemu/patches/4.0.x/0001-9p-removing-coroutines-of-9p-to-increase-the-I-O-per.patch
> > > > https://github.com/kata-containers/packaging/blob/master/qemu/patches/4.0.x/0002-migration-add-capability-to-bypass-the-shared-memory.patch
> > >
> > > Yes, I'm hoping that with 4.0 you can avoid 0002 - but I've not tried
> > > it; I'd be interested in your results.
> > >
> > Hi Dave,
> >
> > I gave it a try and failed to do vm templating with x-ignore-shared.
> > One key difference between x-ignore-shared and Lai's
> > migrate-bypass-shared patch is that Lai doesn't verify shared memory
> > block upon ram load. OTOH x-ignore-shared rely on ram share property
> > to reconstruct the destination guest memory. In kata, we make use of
> > the fact that destination ram can be private to implement vm
> > templating feature so that multiple new guests can share map the same
> > template VM memory privately (memory-backend-file share=off).
>
> (Copying in Yury who wrote those patches).
>
> > The way we implement vm templating in kata is:
> > 1. Start the template VM:
> >    qemu-system-x86 -m 2G \
> >      -object memory-backend-file,id=mem0,size=2G,share=on,mem-path=/tmpfs/template-memory
> > \
> >      -numa node,memdev=mem0
> > 2. Stop the template VM, set migration bypass-shared-memory
> > capability, migrate exec:cat>/tmpfs/state, quit it
> > 3. Start target VM:
> >    qemu-system-x86 -m 2G \
> >      -object memory-backend-file,id=mem0,size=2G,share=off,mem-path=/tmpfs/template-memory
> > \
> >      -numa node,memdev=mem0 \
> >      -incoming "exec:cat /tmpfs/template-state"
> > 4. Start more target VMs like 3
> >
> > There are some major differences between above and the example given
> > by x-ignore-shared patchset[1]:
> > 1. on target vm, memory-backend-file is mapped privately (share=off)
> > 2. no migration capability is set on the target vm
> > 3. it is possible to create multiple target VMs based on the same template VM
> > I have made some hacky change to make x-ignore-shared work for vm
> > templating, mostly by reverting the commit "migration: Add
> > capabilities validation" and reconstructing destination VM ram without
> > using migration share capability or ram share property[2].
> >
> > So I want to ask about your opinion on how to make vm templating work.
> > Shall we change qemu to load ram without the x-ignore-shared
> > capability? Or add another capability to implement similar feature
> > along side x-ignore-shared?
>
> Can we first try a different hack; if you set the ignore shared on the
> destination as well as the source, what happens?  I guess it'll hit one
> of the errors in  ram_load?  Which one?
>
With vanilla 4.0? I got:
 qemu-system-x86_64: RAM block mem should  be migrated
 qemu-system-x86_64: error while loading state for instance 0x0 of device 'ram'
 qemu-system-x86_64: load of migration failed: Invalid argument

which matches:
4341                     if (migrate_ignore_shared()) {
4342                         hwaddr addr = qemu_get_be64(f);
4343                         bool ignored = qemu_get_byte(f);
4344                         if (ignored != ramblock_is_ignored(block)) {
4345                             error_report("RAM block %s should %s
be migrated",
4346                                          id, ignored ? "" : "not");
4347                             ret = -EINVAL;

If I remove the check it succeeds. If we go this way, there is no need
to pass shared ram states during migration since this is the only
place they are used.

Cheers,
Tao
-- 
Into something rich and strange.