Open Stack

Hi EJ and Wei,

First of all, I don't think there is a strong relation between bare-metal
and live-migration.

In Kubernetes, once a Pod is assigned to a node, it won't go anywhere else
in its entire life. And the pod itself should be stateless and
even stateful workload should rely on permanent volumes or other global
resources. Under the above assumption, any migration-like
requirements should be defined by deployment or other high-level tasks
definition instead of sandbox-level migration. This is the philosophy
of Kubernetes.

However, I understand Wei, who is at the role of an operator just like me,
he may be very happy if he could be able to do some tweak to
avoid unnecessary re-schedule operations, which will reduce service
interruption at least. The only problem here is how much is the cost
of the live migration.

Personally, I don't think I need live migration in the current stage but I
won't reject the live-migration feature if it does not have
significant side effects.

-Xu


On Tue, Dec 18, 2018 at 3:30 PM zhangwei (CR) <zhangwei555 at huawei.com>
wrote:

>
>
> Hi EJ,
>
>
>
> Kata Container is one tiny VM per POD, you can choose to run k8s+kata in
> another VM provisioned by OpenStack, in this case, we run into the nested
> virtualization scenario. It works well but brings some unnecessary overhead
> from nested VM usage.
>
>
>
> To avoid this kind of overhead, we can also choose to run K8S cluster on
> bare metal, which means every worker/node of K8S is bare metal machine
> instead of VM. By doing this, K8S becomes the lowest level of
> infrastructure and should provide live migration capability similar to
> OpenStack VM scenarios.
>
>
>
> Live migration could be useful in this scene as every POD is a VM running
> on bare metal now, we need a reliable way to keep workload alive when host
> machine needs reboot(let’s say, fixing some CVEs).
>
>
>
> -Wei
>
>
>
> *发件人**:* EJ Campbell [mailto:ejc3 at oath.com <ejc3 at oath.com>]
> *发送时间:* 2018年12月18日 14:11
> *收件人:* zhangwei (CR) <zhangwei555 at huawei.com>
> *抄送:* Ernst, Eric <eric.ernst at intel.com>; Stefan Hajnoczi <
> stefanha at redhat.com>; sweil at redhat.com; Qixuan Wu <
> qixuan.wu at linux.alibaba.com>; Graham Whaley <graham.whaley at gmail.com>;
> miklos at szeredi.hu; kata-dev at lists.katacontainers.io; swhiteho at redhat.com;
> vgoyal at redhat.com
> *主题:* Re: [kata-dev] 答复: [Announce] virtio-fs released with Kata
> Containers support
>
>
>
> Hi Zhangwei,
>
>
>
> Could you go into more detail on the use case for live migration on bare
> metal?
>
>
>
> Thanks,
>
> EJ
>
>
>
>
>
> On Mon, Dec 17, 2018 at 6:20 PM zhangwei (CR) <zhangwei555 at huawei.com>
> wrote:
>
> Stefan, Eric,
>
> There isn't live migration requirement so far, but, I will take this as a
> potential requirement when Kata is growing more powerful.
> In my mind, this is rational as some of us are running kata on bare metal,
> in this scene, we don't have an infrastructure software such as OpenStack
> to guarantee the lifecycle of workload.
>
> Virtio-fs is in RFC state, it could be OK as long as it doesn't have
> native gap for supporting live migration,  and I will be glad to see it
> being listed in some roadmap.
>
> By the way, really nice work! We finally get a better option against 9pfs
> :-) Thanks!
>
> -----邮件原件-----
> 发件人: Ernst, Eric [mailto:eric.ernst at intel.com]
> 发送时间: 2018年12月17日 22:30
> 收件人: Stefan Hajnoczi <stefanha at redhat.com>
> 抄送: sweil at redhat.com; Qixuan Wu <qixuan.wu at linux.alibaba.com>; Graham
> Whaley <graham.whaley at gmail.com>; miklos at szeredi.hu;
> kata-dev at lists.katacontainers.io; swhiteho at redhat.com; vgoyal at redhat.com
> 主题: Re: [kata-dev] [Announce] virtio-fs released with Kata Containers
> support
>
> Stefan -
>
> No, there isn’t a live migration requirement for Kata (sorry for the top
> post).
>
> Eric
>
> > On Dec 17, 2018, at 6:27 AM, Stefan Hajnoczi <stefanha at redhat.com>
> wrote:
> >
> >> On Sun, Dec 16, 2018 at 09:09:48AM +0800, Qixuan Wu wrote:
> >> 1) Did you compare the performance to the virtio-blk raw or qcow2
> >> solution of normal virtual machine?
> >
> > Not yet.
> >
> > Guest and host page cache performance can dominate benchmarks, so we
> > typically use fio direct=1 with QEMU -drive cache=none (O_DIRECT) to
> > focus purely on disk I/O performance and not page cache.  The same
> > thing can be done with virtio-fs so that every I/O operation requires
> > communication with the host.  In theory virtio-fs should be comparable
> > to virtio-blk on raw.
> >
> > In real-world scenarios the page cache will be enabled, especially for
> > the virtio-fs DAX feature.  So I need to think carefully about what to
> > benchmark, but it will probably include both configurations.
> >
> >> 2) Whether does it impact live-migration of guest os ?
> >
> > Virtio-fs currently does not support live migration.  Is there a
> > requirement for live migration with Kata Containers use cases?
> >
> > Stefan
>
> _______________________________________________
> kata-dev mailing list
> kata-dev at lists.katacontainers.io
> http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
> _______________________________________________
> kata-dev mailing list
> kata-dev at lists.katacontainers.io
> http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
>
> _______________________________________________
> kata-dev mailing list
> kata-dev at lists.katacontainers.io
> http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
>


-- 
--

Xu Wang
CTO & Cofounder, Hyper
github/twitter/wechat: @gnawux
http://hyper.sh

Hyper_: Make VM run like container
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.katacontainers.io/pipermail/kata-dev/attachments/20181218/15ebff98/attachment.html>