how much does kata depend on kernel features?
I'd like to use kata to test applications on different kernels, ideally I'd like to be able to import kernels from most modern distributions and test on those kernels without any changes.
So far I've not been able to boot kata VMs successfully with Ubuntu 18,04 and CentOS 8 kernels.
I have built a custom image and been able to boot that with the original kata kernel from 1.11.0-alpha1.
My question is - should it be fundamentally possible to make kata work with any kernel, or all of its functionality as a k8s runtime is highly dependent on kernel it uses?
When I look at qemu flags, I see a lot of things going on, that make me think there is a set of very specific features that kata depends on, is that correct? Or can one reduce this set of features to something more basic?
Also, if I am to use firecracker, would my options be any different or worse actually?
Ilya
Here is another idea: what if I booted using kata's official kernel and a custom rootfs image, and then pivot to a desired kernel with kexec. Any thoughts on that kind of approach?
Ilya
On Wed, 1 Apr 2020 at 20:19, Ilya Dmitrichenko errordeveloper@gmail.com wrote:
I'd like to use kata to test applications on different kernels, ideally I'd like to be able to import kernels from most modern distributions and test on those kernels without any changes.
So far I've not been able to boot kata VMs successfully with Ubuntu 18,04 and CentOS 8 kernels.
I have built a custom image and been able to boot that with the original kata kernel from 1.11.0-alpha1.
My question is - should it be fundamentally possible to make kata work with any kernel, or all of its functionality as a k8s runtime is highly dependent on kernel it uses?
When I look at qemu flags, I see a lot of things going on, that make me think there is a set of very specific features that kata depends on, is that correct? Or can one reduce this set of features to something more basic?
Also, if I am to use firecracker, would my options be any different or worse actually?
Ilya
* Ilya Dmitrichenko (errordeveloper@gmail.com) wrote:
I'd like to use kata to test applications on different kernels, ideally I'd like to be able to import kernels from most modern distributions and test on those kernels without any changes.
So far I've not been able to boot kata VMs successfully with Ubuntu 18,04 and CentOS 8 kernels.
I have built a custom image and been able to boot that with the original kata kernel from 1.11.0-alpha1.
My question is - should it be fundamentally possible to make kata work with any kernel, or all of its functionality as a k8s runtime is highly dependent on kernel it uses?
It should if it has the features needed. You need a shared filesystem, so that's either 9p or virtiofs. 9p is turned off in CentOS/RHEL; virtiofs was added but only recently. I think you should find it in the current (March 2020) CentOS Stream kernel.
I think Ubuntu 18.04 probably has 9p turned on, but I'm less sure.
There are some vsock features that make life easier on Kata as well; I'm less clear as to exactly when those landed.
Dave
When I look at qemu flags, I see a lot of things going on, that make me think there is a set of very specific features that kata depends on, is that correct? Or can one reduce this set of features to something more basic?
Also, if I am to use firecracker, would my options be any different or worse actually?
Ilya
kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
-- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
-----Original Message----- From: Dr. David Alan Gilbert dgilbert@redhat.com Sent: Thursday, April 2, 2020 10:11 AM To: Ilya Dmitrichenko errordeveloper@gmail.com Cc: kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] how much does kata depend on kernel features?
- Ilya Dmitrichenko (errordeveloper@gmail.com) wrote:
I'd like to use kata to test applications on different kernels, ideally I'd like to be able to import kernels from most modern distributions and test on those kernels without any changes.
OOI Ilya, is there a particular reason you are using Kata for this, and not using just vanilla VMs? I think adding Kata to the mix is going to make your life significantly harder - it is/was not designed to 'just work' on any kernel without specific features etc., as you are finding. If there is a specific reason to use Kata, fine, great - but we'd probably be interested to hear it 😉 If not, then your life might get significantly easier if you switch to using plain QEMU (and if your apps are in containers, then having docker installed on the client you run under vanilla QEMU etc.).
Graham
--------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
On Thu, 2 Apr 2020 at 11:11, Whaley, Graham graham.whaley@intel.com wrote:
OOI Ilya, is there a particular reason you are using Kata for this, and not using just vanilla VMs? I think adding Kata to the mix is going to make your life significantly harder - it is/was not designed to 'just work' on any kernel without specific features etc., as you are finding. If there is a specific reason to use Kata, fine, great - but we'd probably be interested to hear it 😉 If not, then your life might get significantly easier if you switch to using plain QEMU (and if your apps are in containers, then having docker installed on the client you run under vanilla QEMU etc.).
Graham, as I mentioned on IRC, I am pretty certain of what I'm after here.
Conceptually, this is about running Kubernetes-in-Kubernetes, which is a know general patter, and this is just a variation of that. I have a requirement on having kernel isolation as well as a strong desire to test different kernels also. I actually have some of this working already.
I do think it's plausible to consider not using kata and e.g. have qemu or firecracker inside container image and just run it from the pod, as I don't see a major need for storage after VM startup, it could be quite simple, yet I would still need to make networking work resource management would have to be take care of - whatever pod needs to run the VMM and whatever the VM resources should be set to... I guess for some use-cases this might just work, but ultimately one will eventually desire a more native pod-as-VM setup, something very much like kata.
KubeVirt could be another option, but from what I gathered it's much less native to Kubenernetes, and you cannot map a KubeVirt VM to a service like you can with Kata.
participants (3)
-
Dr. David Alan Gilbert
-
Ilya Dmitrichenko
-
Whaley, Graham