[...]
Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back.
For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro.
I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves.
I like that approach. Could you ellaborate on what kata-osinfo would looks like?
So, let's travel back in time when deploying a VM was a normal thing to do and when virt-manager / GNOME Boxes had to know about, for instance, kernel and initrd location of all the distros in order to extract those from the ISO and boot up the VM. Back on those times, it was a mess to have this kind of knowledge hardcoded on the apps themselves and we ended up creating libosinfo, which has some APIs that the virtual machine managers could call, and a database with all the info they could use.
I see something similar happening for our use case, to be used together with the kata-operator. A simple Go package that would get info from a simple database that could look like:
$ cat debian.yaml apiVersion: v1 distro: debian version: [testing] package-installer: apt arch: x86_64 vmm: name: qemu packages: qemu-kvm path: /usr/bin/qemu-system-x86_64 fs: name: virtio-fs path: /usr/lib/qemu/virtiofsd name: virtio-9p
$ cat centos.yaml apiVersion: v1 distro: centos version: [8.2.2004] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: qemu-kvm-core path: /usr/bin/qemu-kvm fs: name: virtio-fs path: /usr/libexec/virtiofsd
$ cat fedora.yaml distro: fedora version: [32, rawhide] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: [qemu-kvm-core] path: /usr/bin/qemu-kvm fs: preferred: virtio-fs name: virtio-fs path: /usr/libexec/virtiofsd name: 9p
The way it could function would be: - Operator checks which distro they're installing on (checking /etc/os-release is enough); - Operator asks kata-osinfo about the distro (getDistro()); - Operator checks whether the package is installed (isVMMInstalled()) - Operator could then do the specific distro package installation (installVMMPackage(), based on getPackageInstallername() and getVMMPackage)) - Operator could tweak the kata config according to the info we provide (I see here the VMMs / virtiofsd paths being the main focus here, but I can also imagine we may want to know and tweak the vsock / kata-proxy usage);
Keep in mind that this is a really simple example without much effort put into it, but I guess we could follow that path. What do you think?
Do we know of distros that don't have suitable bits, i.e. where the existing qemu or kernel is not good enough?
Let's get some old debian / ubuntu / fedora. They most likely have the bits to support kata, but for sure not to support 9p. That's the reason I think a fs field would be needed, and maybe a preferred one. But those are technicalities that we can have a really detailed discussion *if* we decide to take this approach.
Can we design some fall-back mechanism in that case?
Hmm. I was pretty sure I mentioned a fallback mechanism but I didn't. I'll blame the (lack of) coffee. :-/ Imagine we drop those database files on /usr/share/kata-osinfo/ and make that the default location. What I see happening is us providing the admins a way to augment that config by adding content to /etc/kata-osinfo/, which would then take precedence over the "default" ones. So, if I want to use my own build, I create another entry in /etc/kata-osinfo/ with the modified content that will have the paths to the binaries I built, what they support or not, and the kata-operator can use that.
What about other VMMs? How would you deal with, say, firecracker?
If they're packaged (and I wish distros would start packaging those), same as qemu. If not, same as the fallback. Best Regards, -- Fabiano Fidêncio