Hi Wilson,
It is great to know you are working on some POC based in kata and K8s. The new features Bo mentioned will land in Kata 1.11 (about to be released this week) for that release Kata will use the latest release for Cloud-hypervisor 0.7.0.
Today for Kata in master we already have testing for K8s and containerd as CRI runtime + Cloud-hypervisor.
For our current CI testing we are just missing support for hot plug of storage devices. We hope in next release of Ka ta alpha 1.12-alpha0 this could be ready for testing. Additionally we are working to enable End-to-End Kubernetes testing to increase the coverage of use cases. There are still a few API options that Cloud-hypervisor support and we have not evaluated. But I’d say that after blk/pmem hotplug enabling , we will have parity with Qemu implementation and the use-cases we have in our CI today.
Please open any issue you find, we want to make sure cloud-hypervisor and kata has the best experience.
You can get some useful information about future fixes, issues, and missing items in the following github project:
https://github.com/orgs/kata-
containers/projects/21
A summary of the current state also is documented here:
https://github.com/kata-
containers/runtime/issues/2545
Summary of used cloud-hypervisor APIs (and comments why some are not used)
https://github.com/kata-
containers/runtime/issues/2541
To summarize the more relevant missing items are related with hotplug are:
- Pmem/Block device hotplug
https://github.com/kata-
containers/runtime/issues/2541 https://github.com/kata-
containers/runtime/issues/2575
- Net hotplug : First we need to add some testing (at least with qemu) to know where could be used in the k8s workflow, then we could enable it.
You can track the CI stability of master and the current set of jobs that we use for test cloud-hypervisor integration.
https://github.com/kata-
containers/ci/wiki/cloud- hypervisor
Cheers!
-
Carlos
From: "wilsonny371@gmail.com" <wilsonny371@gmail.com>
Date: Wednesday, 6 May 2020, 18:25
To: "Chen, Bo2" <chen.bo@intel.com>
Cc: "Venegas Munoz, Jose Carlos" <jose.carlos.venegas.munoz@intel.com >, "kata-dev@lists.katacontainers.io " <kata-dev@lists.katacontainers.io >
Subject: Re: [kata-dev] hot plug/unplug for cloud hypervisor support
Bo,
Thanks for your reply.
My colleagues and I are working on some POC and it is based on Kata & K8s. The new features you mentioned will definitely help us. However, we are still at the very early stage and trying to prove that cloud hypervisor is truly what we are looking for compared to firecracker which does not have these features yet.
Regards
Wilson
On Wed, May 6, 2020 at 4:00 PM Chen, Bo2 <chen.bo@intel.com> wrote:
Hi Wilson,
Thanks for promoting this question. Yes, that is our plan. Along with the supported device hot-plug we have in Kata with cloud-hypervisor (e.g. vcpu/memory/vfio), we are enabling block (‘disk ‘and ‘pmem’) device hot-plug very soon. For few other HTTP APIs related to device hotplug that cloud-hypervisor recently added (e.g. ‘/vm.add-fs’, ‘/vm.add-net’, and ‘/vm.add-vsock’), we will need to look into them and add support if there are use cases in Kata.
Carlos, would you please comment on the device hot-unplug side? Base on my limited understanding, we don’t have much use-case for device hot-unplug in Kata, right? Also, please correct me and add more for the device hot-plug side. Thanks a lot.
I hope it helps and makes sense to you. Any more questions or comments, please don’t hesitate to share.
Best,
Bo
From: wilsonny371@gmail.com <wilsonny371@gmail.com>
Sent: Wednesday, May 6, 2020 3:35 PM
To: kata-dev@lists.katacontainers.io
Subject: [kata-dev] hot plug/unplug for cloud hypervisor support
Hi guys,
I have a question regarding Kata Containers hotplug support for cloud hypervisor.
I saw recently cloud hypervisor added support for several different virtio devices hotplug. But the support for this is not yet there in Kata Containers.
Do we have a plan to add this support soon?
Thanks
Wilson