On Wed, Aug 12, 2020 at 9:06 AM Samuel Ortiz <sameo@linux.intel.com> wrote:
Hi Fabio,
Sorry for missing yesterday's call as well, and thanks for starting that discussion. See my answers below:
On Tue, Aug 11, 2020 at 11:09:16PM +0200, Fabiano Fidêncio wrote:
Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros.
In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers.
Given the specific configurations that we require/advise for from our VMMs, and given that out of all the VMMs that we support the only one that's actually distro shipped is QEMU (obviously very generically configured builds of QEMU...), I'd disagree with that.
Hmm. I see a huge mistake on my assumption. Maybe I should rename this to: "Stop building & distributing our own QEMU".
With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves.
Keep building the kernel =================== Differently than the suggestion about the VMM, the kernel bits, IMHO, should still be built. The kernels we build can be used upstream, but we must ensure those are under no circumstances ready for production.
I agree although I am not sure how we can verify that our kernel is not production ready. We can only document the fact that this is mostly a testing kernel that comes with a kernel config that can be used as a reference.
Why? Because we have no power to track and fix CVEs as the distros do.
Certainly.
So, we can keep shipping our own kernel and testing using it, but we should also incentive distros to use (and test) their own kernels with kata. The reason I say this is because it'd help the project to *not* require a distro to ship a non-tested kernel,
I agree here as well. I am not sure if we actually _require_ distros to ship a non tested kernel. I think it should be the other way around: If a distro kernel can boot as a Kata guest, we should work with the distro to fix that.
AFAIR (but here I have a problem to trust my memory) we would at least strongly recommend to use the kata assets. But I agree with you on your point. :-)
Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back.
For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro.
I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves.
I like that approach. Could you ellaborate on what kata-osinfo would looks like?
So, let's travel back in time when deploying a VM was a normal thing to do and when virt-manager / GNOME Boxes had to know about, for instance, kernel and initrd location of all the distros in order to extract those from the ISO and boot up the VM. Back on those times, it was a mess to have this kind of knowledge hardcoded on the apps themselves and we ended up creating libosinfo, which has some APIs that the virtual machine managers could call, and a database with all the info they could use. I see something similar happening for our use case, to be used together with the kata-operator. A simple Go package that would get info from a simple database that could look like: $ cat debian.yaml apiVersion: v1 distro: debian version: [testing] package-installer: apt arch: x86_64 vmm: name: qemu packages: qemu-kvm path: /usr/bin/qemu-system-x86_64 fs: name: virtio-fs path: /usr/lib/qemu/virtiofsd name: virtio-9p $ cat centos.yaml apiVersion: v1 distro: centos version: [8.2.2004] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: qemu-kvm-core path: /usr/bin/qemu-kvm fs: name: virtio-fs path: /usr/libexec/virtiofsd $ cat fedora.yaml distro: fedora version: [32, rawhide] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: [qemu-kvm-core] path: /usr/bin/qemu-kvm fs: preferred: virtio-fs name: virtio-fs path: /usr/libexec/virtiofsd name: 9p The way it could function would be: - Operator checks which distro they're installing on (checking /etc/os-release is enough); - Operator asks kata-osinfo about the distro (getDistro()); - Operator checks whether the package is installed (isVMMInstalled()) - Operator could then do the specific distro package installation (installVMMPackage(), based on getPackageInstallername() and getVMMPackage)) - Operator could tweak the kata config according to the info we provide (I see here the VMMs / virtiofsd paths being the main focus here, but I can also imagine we may want to know and tweak the vsock / kata-proxy usage); Keep in mind that this is a really simple example without much effort put into it, but I guess we could follow that path. What do you think?