Supported OSes, packaging plans for the future & some ideas
Folks! We have discussed during this week's Architecture's meeting what could be a plan for the future with regards to packaging and I'd like to drop a suggestion here and then have it discussed in the August 25th meeting. The current situation is that the kata project builds kata pieces (which may differ considering whether you're using 2.x or 1.x branches) and distributes those together with our own qemu built statically and the kernel / initrd / image to be used. And, sincerely, this is *a* *lot* to take care of. My suggestion would be to reduce the amount of work we do, the amount of things we build, and start relying more on what the distros can provide us. Let me explain. For instance, we could ... Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros. In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers. With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves. Keep building the kernel =================== Differently than the suggestion about the VMM, the kernel bits, IMHO, should still be built. The kernels we build can be used upstream, but we must ensure those are under no circumstances ready for production.Why? Because we have no power to track and fix CVEs as the distros do. So, we can keep shipping our own kernel and testing using it, but we should also incentive distros to use (and test) their own kernels with kata. The reason I say this is because it'd help the project to *not* require a distro to ship a non-tested kernel, which would drop a little bit the barrier of accepting kata as their official packages. Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back. For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro. I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves. Doing this, we can keep it simple on our side, while relying more on the distros. Those things would already reduce the maintenance burden and, at the same time, show to our consumers what's supported and tested from our side. Of course, it goes without saying, but here I'm hoping our tests will follow the same approach and will use the distro packages, ensuring the basics would be working there and also reporting issues against the distro packages when needed. Please, take some time to think about the proposal. I'd love to hear feedback here in the Mailing List, but if you have something to add, please, also consider joining the Architecture's meeting on August 25th so we can take our final decision on this topic. Best Regards, -- Fabiano Fidêncio
Sorry I missed the call this morning. My takes below. --Eric On Tue, Aug 11, 2020 at 2:09 PM Fabiano Fidêncio <fidencio@redhat.com> wrote:
Folks!
We have discussed during this week's Architecture's meeting what could be a plan for the future with regards to packaging and I'd like to drop a suggestion here and then have it discussed in the August 25th meeting.
The current situation is that the kata project builds kata pieces (which may differ considering whether you're using 2.x or 1.x branches) and distributes those together with our own qemu built statically and the kernel / initrd / image to be used. And, sincerely, this is *a* *lot* to take care of.
My suggestion would be to reduce the amount of work we do, the amount of things we build, and start relying more on what the distros can provide us. Let me explain.
For instance, we could ...
Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros.
In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers.
With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves.
I disagree on this. We are often relying on very recent features in each VMM, and we are opinionated on how we build it (see Kconfig options, configure flags). We want a very specific set of devices and features in each of our VMMs; we don't care about running Windows 95, etc.
Keep building the kernel =================== Differently than the suggestion about the VMM, the kernel bits, IMHO, should still be built. The kernels we build can be used upstream, but we must ensure those are under no circumstances ready for production.Why? Because we have no power to track and fix CVEs as the distros do.
So, we can keep shipping our own kernel and testing using it, but we should also incentive distros to use (and test) their own kernels with kata. The reason I say this is because it'd help the project to *not* require a distro to ship a non-tested kernel, which would drop a little bit the barrier of accepting kata as their official packages.
Agreed that we should keep building the kernel (again, the config is important, and this can be at least used as a reference).
Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back.
For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro.
I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves.
Doing this, we can keep it simple on our side, while relying more on the distros.
I want input from users of Kata in production on this. My guess is that most organizations are rolling their own packages or are building their own binaries and deploying via an operator/daemonset like kata-deploy. But, that's my guess. Let's hope that users chime in on this thread. More specifically: as a user, do you use Kata provided packages? Do you use distro packages? Do you build your own packages/deployment? I'd also want to get an ACK from the SUSE folks, who worked to add package support in the past. Thanks for bringing this up Fabiano. --Eric
Those things would already reduce the maintenance burden and, at the same time, show to our consumers what's supported and tested from our side.
Of course, it goes without saying, but here I'm hoping our tests will
follow the same approach and will use the distro packages, ensuring the basics would be working there and also reporting issues against the distro packages when needed.
Please, take some time to think about the proposal. I'd love to hear feedback here in the Mailing List, but if you have something to add, please, also consider joining the Architecture's meeting on August 25th so we can take our final decision on this topic.
Best Regards, -- Fabiano Fidêncio
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
Eric, On Wed, Aug 12, 2020 at 2:42 AM Eric Ernst <eric.g.ernst@gmail.com> wrote:
Sorry I missed the call this morning. My takes below.
--Eric
On Tue, Aug 11, 2020 at 2:09 PM Fabiano Fidêncio <fidencio@redhat.com> wrote:
Folks!
We have discussed during this week's Architecture's meeting what could be a plan for the future with regards to packaging and I'd like to drop a suggestion here and then have it discussed in the August 25th meeting.
The current situation is that the kata project builds kata pieces (which may differ considering whether you're using 2.x or 1.x branches) and distributes those together with our own qemu built statically and the kernel / initrd / image to be used. And, sincerely, this is *a* *lot* to take care of.
My suggestion would be to reduce the amount of work we do, the amount of things we build, and start relying more on what the distros can provide us. Let me explain.
For instance, we could ...
Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros.
In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers.
With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves.
I disagree on this. We are often relying on very recent features in each VMM, and we are opinionated on how we build it (see Kconfig options, configure flags). We want a very specific set of devices and features in each of our VMMs; we don't care about running Windows 95, etc.
I understand your point. But do we really have to build the VMMs ourselves, distribute those ourselves, patch those ourselves, receive bug reports of those ourselves? IMHO, we should keep documenting what's the optimal way, what's the recommended way. However, we should not take the bullet to distribute those as we currently do. Do you want a very very minimal QEMU that fits your needs? Cool, build it following our documentation. You don't think you need that? Better, just use whatever your distro provides and be happy. [...] Best Regards, -- Fabiano Fidêncio
On 12 Aug 2020, at 02:42, Eric Ernst <eric.g.ernst@gmail.com> wrote:
Sorry I missed the call this morning. My takes below.
--Eric
On Tue, Aug 11, 2020 at 2:09 PM Fabiano Fidêncio <fidencio@redhat.com <mailto:fidencio@redhat.com>> wrote: Folks!
We have discussed during this week's Architecture's meeting what could be a plan for the future with regards to packaging and I'd like to drop a suggestion here and then have it discussed in the August 25th meeting.
The current situation is that the kata project builds kata pieces (which may differ considering whether you're using 2.x or 1.x branches) and distributes those together with our own qemu built statically and the kernel / initrd / image to be used. And, sincerely, this is *a* *lot* to take care of.
My suggestion would be to reduce the amount of work we do, the amount of things we build, and start relying more on what the distros can provide us. Let me explain.
For instance, we could ...
Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros.
In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers.
With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves.
I disagree on this. We are often relying on very recent features in each VMM, and we are opinionated on how we build it (see Kconfig options, configure flags). We want a very specific set of devices and features in each of our VMMs; we don't care about running Windows 95, etc.
This is a valid point, but at least in the case of Fedora, there is some discussion on building a minimalist qemu for that purpose, from the same sources as the upstream qemu, and with the same patches (therefore addressing the same CVEs). See https://src.fedoraproject.org/fork/ddd/rpms/qemu/tree/mini.
Hi Fabio, Sorry for missing yesterday's call as well, and thanks for starting that discussion. See my answers below: On Tue, Aug 11, 2020 at 11:09:16PM +0200, Fabiano Fidêncio wrote:
Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros.
In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers.
Given the specific configurations that we require/advise for from our VMMs, and given that out of all the VMMs that we support the only one that's actually distro shipped is QEMU (obviously very generically configured builds of QEMU...), I'd disagree with that.
With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves.
Keep building the kernel =================== Differently than the suggestion about the VMM, the kernel bits, IMHO, should still be built. The kernels we build can be used upstream, but we must ensure those are under no circumstances ready for production.
I agree although I am not sure how we can verify that our kernel is not production ready. We can only document the fact that this is mostly a testing kernel that comes with a kernel config that can be used as a reference.
Why? Because we have no power to track and fix CVEs as the distros do.
Certainly.
So, we can keep shipping our own kernel and testing using it, but we should also incentive distros to use (and test) their own kernels with kata. The reason I say this is because it'd help the project to *not* require a distro to ship a non-tested kernel,
I agree here as well. I am not sure if we actually _require_ distros to ship a non tested kernel. I think it should be the other way around: If a distro kernel can boot as a Kata guest, we should work with the distro to fix that.
Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back.
For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro.
I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves.
I like that approach. Could you ellaborate on what kata-osinfo would looks like? Cheers, Samuel.
On Wed, Aug 12, 2020 at 9:06 AM Samuel Ortiz <sameo@linux.intel.com> wrote:
Hi Fabio,
Sorry for missing yesterday's call as well, and thanks for starting that discussion. See my answers below:
On Tue, Aug 11, 2020 at 11:09:16PM +0200, Fabiano Fidêncio wrote:
Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros.
In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers.
Given the specific configurations that we require/advise for from our VMMs, and given that out of all the VMMs that we support the only one that's actually distro shipped is QEMU (obviously very generically configured builds of QEMU...), I'd disagree with that.
Hmm. I see a huge mistake on my assumption. Maybe I should rename this to: "Stop building & distributing our own QEMU".
With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves.
Keep building the kernel =================== Differently than the suggestion about the VMM, the kernel bits, IMHO, should still be built. The kernels we build can be used upstream, but we must ensure those are under no circumstances ready for production.
I agree although I am not sure how we can verify that our kernel is not production ready. We can only document the fact that this is mostly a testing kernel that comes with a kernel config that can be used as a reference.
Why? Because we have no power to track and fix CVEs as the distros do.
Certainly.
So, we can keep shipping our own kernel and testing using it, but we should also incentive distros to use (and test) their own kernels with kata. The reason I say this is because it'd help the project to *not* require a distro to ship a non-tested kernel,
I agree here as well. I am not sure if we actually _require_ distros to ship a non tested kernel. I think it should be the other way around: If a distro kernel can boot as a Kata guest, we should work with the distro to fix that.
AFAIR (but here I have a problem to trust my memory) we would at least strongly recommend to use the kata assets. But I agree with you on your point. :-)
Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back.
For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro.
I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves.
I like that approach. Could you ellaborate on what kata-osinfo would looks like?
So, let's travel back in time when deploying a VM was a normal thing to do and when virt-manager / GNOME Boxes had to know about, for instance, kernel and initrd location of all the distros in order to extract those from the ISO and boot up the VM. Back on those times, it was a mess to have this kind of knowledge hardcoded on the apps themselves and we ended up creating libosinfo, which has some APIs that the virtual machine managers could call, and a database with all the info they could use. I see something similar happening for our use case, to be used together with the kata-operator. A simple Go package that would get info from a simple database that could look like: $ cat debian.yaml apiVersion: v1 distro: debian version: [testing] package-installer: apt arch: x86_64 vmm: name: qemu packages: qemu-kvm path: /usr/bin/qemu-system-x86_64 fs: name: virtio-fs path: /usr/lib/qemu/virtiofsd name: virtio-9p $ cat centos.yaml apiVersion: v1 distro: centos version: [8.2.2004] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: qemu-kvm-core path: /usr/bin/qemu-kvm fs: name: virtio-fs path: /usr/libexec/virtiofsd $ cat fedora.yaml distro: fedora version: [32, rawhide] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: [qemu-kvm-core] path: /usr/bin/qemu-kvm fs: preferred: virtio-fs name: virtio-fs path: /usr/libexec/virtiofsd name: 9p The way it could function would be: - Operator checks which distro they're installing on (checking /etc/os-release is enough); - Operator asks kata-osinfo about the distro (getDistro()); - Operator checks whether the package is installed (isVMMInstalled()) - Operator could then do the specific distro package installation (installVMMPackage(), based on getPackageInstallername() and getVMMPackage)) - Operator could tweak the kata config according to the info we provide (I see here the VMMs / virtiofsd paths being the main focus here, but I can also imagine we may want to know and tweak the vsock / kata-proxy usage); Keep in mind that this is a really simple example without much effort put into it, but I guess we could follow that path. What do you think?
On 12 Aug 2020, at 09:58, Fabiano Fidêncio <fidencio@redhat.com> wrote:
On Wed, Aug 12, 2020 at 9:06 AM Samuel Ortiz <sameo@linux.intel.com> wrote:
Hi Fabio,
Sorry for missing yesterday's call as well, and thanks for starting that discussion. See my answers below:
On Tue, Aug 11, 2020 at 11:09:16PM +0200, Fabiano Fidêncio wrote:
Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros.
In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers.
Given the specific configurations that we require/advise for from our VMMs, and given that out of all the VMMs that we support the only one that's actually distro shipped is QEMU (obviously very generically configured builds of QEMU...), I'd disagree with that.
Hmm. I see a huge mistake on my assumption. Maybe I should rename this to: "Stop building & distributing our own QEMU".
With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves.
Keep building the kernel =================== Differently than the suggestion about the VMM, the kernel bits, IMHO, should still be built. The kernels we build can be used upstream, but we must ensure those are under no circumstances ready for production.
I agree although I am not sure how we can verify that our kernel is not production ready. We can only document the fact that this is mostly a testing kernel that comes with a kernel config that can be used as a reference.
Why? Because we have no power to track and fix CVEs as the distros do.
Certainly.
So, we can keep shipping our own kernel and testing using it, but we should also incentive distros to use (and test) their own kernels with kata. The reason I say this is because it'd help the project to *not* require a distro to ship a non-tested kernel,
I agree here as well. I am not sure if we actually _require_ distros to ship a non tested kernel. I think it should be the other way around: If a distro kernel can boot as a Kata guest, we should work with the distro to fix that.
AFAIR (but here I have a problem to trust my memory) we would at least strongly recommend to use the kata assets. But I agree with you on your point. :-)
Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back.
For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro.
I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves.
I like that approach. Could you ellaborate on what kata-osinfo would looks like?
So, let's travel back in time when deploying a VM was a normal thing to do and when virt-manager / GNOME Boxes had to know about, for instance, kernel and initrd location of all the distros in order to extract those from the ISO and boot up the VM. Back on those times, it was a mess to have this kind of knowledge hardcoded on the apps themselves and we ended up creating libosinfo, which has some APIs that the virtual machine managers could call, and a database with all the info they could use.
I see something similar happening for our use case, to be used together with the kata-operator. A simple Go package that would get info from a simple database that could look like:
$ cat debian.yaml apiVersion: v1 distro: debian version: [testing] package-installer: apt arch: x86_64 vmm: name: qemu packages: qemu-kvm path: /usr/bin/qemu-system-x86_64 fs: name: virtio-fs path: /usr/lib/qemu/virtiofsd name: virtio-9p
$ cat centos.yaml apiVersion: v1 distro: centos version: [8.2.2004] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: qemu-kvm-core path: /usr/bin/qemu-kvm fs: name: virtio-fs path: /usr/libexec/virtiofsd
$ cat fedora.yaml distro: fedora version: [32, rawhide] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: [qemu-kvm-core] path: /usr/bin/qemu-kvm fs: preferred: virtio-fs name: virtio-fs path: /usr/libexec/virtiofsd name: 9p
The way it could function would be: - Operator checks which distro they're installing on (checking /etc/os-release is enough); - Operator asks kata-osinfo about the distro (getDistro()); - Operator checks whether the package is installed (isVMMInstalled()) - Operator could then do the specific distro package installation (installVMMPackage(), based on getPackageInstallername() and getVMMPackage)) - Operator could tweak the kata config according to the info we provide (I see here the VMMs / virtiofsd paths being the main focus here, but I can also imagine we may want to know and tweak the vsock / kata-proxy usage);
Keep in mind that this is a really simple example without much effort put into it, but I guess we could follow that path. What do you think?
Do we know of distros that don't have suitable bits, i.e. where the existing qemu or kernel is not good enough? Can we design some fall-back mechanism in that case? What about other VMMs? How would you deal with, say, firecracker?
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
[...]
Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back.
For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro.
I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves.
I like that approach. Could you ellaborate on what kata-osinfo would looks like?
So, let's travel back in time when deploying a VM was a normal thing to do and when virt-manager / GNOME Boxes had to know about, for instance, kernel and initrd location of all the distros in order to extract those from the ISO and boot up the VM. Back on those times, it was a mess to have this kind of knowledge hardcoded on the apps themselves and we ended up creating libosinfo, which has some APIs that the virtual machine managers could call, and a database with all the info they could use.
I see something similar happening for our use case, to be used together with the kata-operator. A simple Go package that would get info from a simple database that could look like:
$ cat debian.yaml apiVersion: v1 distro: debian version: [testing] package-installer: apt arch: x86_64 vmm: name: qemu packages: qemu-kvm path: /usr/bin/qemu-system-x86_64 fs: name: virtio-fs path: /usr/lib/qemu/virtiofsd name: virtio-9p
$ cat centos.yaml apiVersion: v1 distro: centos version: [8.2.2004] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: qemu-kvm-core path: /usr/bin/qemu-kvm fs: name: virtio-fs path: /usr/libexec/virtiofsd
$ cat fedora.yaml distro: fedora version: [32, rawhide] package-installer: dnf arch: [x86_64, ppc64le] vmm: name: qemu packages: [qemu-kvm-core] path: /usr/bin/qemu-kvm fs: preferred: virtio-fs name: virtio-fs path: /usr/libexec/virtiofsd name: 9p
The way it could function would be: - Operator checks which distro they're installing on (checking /etc/os-release is enough); - Operator asks kata-osinfo about the distro (getDistro()); - Operator checks whether the package is installed (isVMMInstalled()) - Operator could then do the specific distro package installation (installVMMPackage(), based on getPackageInstallername() and getVMMPackage)) - Operator could tweak the kata config according to the info we provide (I see here the VMMs / virtiofsd paths being the main focus here, but I can also imagine we may want to know and tweak the vsock / kata-proxy usage);
Keep in mind that this is a really simple example without much effort put into it, but I guess we could follow that path. What do you think?
Do we know of distros that don't have suitable bits, i.e. where the existing qemu or kernel is not good enough?
Let's get some old debian / ubuntu / fedora. They most likely have the bits to support kata, but for sure not to support 9p. That's the reason I think a fs field would be needed, and maybe a preferred one. But those are technicalities that we can have a really detailed discussion *if* we decide to take this approach.
Can we design some fall-back mechanism in that case?
Hmm. I was pretty sure I mentioned a fallback mechanism but I didn't. I'll blame the (lack of) coffee. :-/ Imagine we drop those database files on /usr/share/kata-osinfo/ and make that the default location. What I see happening is us providing the admins a way to augment that config by adding content to /etc/kata-osinfo/, which would then take precedence over the "default" ones. So, if I want to use my own build, I create another entry in /etc/kata-osinfo/ with the modified content that will have the paths to the binaries I built, what they support or not, and the kata-operator can use that.
What about other VMMs? How would you deal with, say, firecracker?
If they're packaged (and I wish distros would start packaging those), same as qemu. If not, same as the fallback. Best Regards, -- Fabiano Fidêncio
[...]
Let's get some old debian / ubuntu / fedora. They most likely have the bits to support kata, but for sure not to support 9p.
Oops, my bad. Here I meant virtio-fs instead of 9p. Old systems have the bits to support kata, but for sure not to support virtio-fs. Thanks Stefano for pointing this out! [...]
On 2020/8/12 05:09, Fabiano Fidêncio wrote:
Folks!
We have discussed during this week's Architecture's meeting what could be a plan for the future with regards to packaging and I'd like to drop a suggestion here and then have it discussed in the August 25th meeting.
The current situation is that the kata project builds kata pieces (which may differ considering whether you're using 2.x or 1.x branches) and distributes those together with our own qemu built statically and the kernel / initrd / image to be used. And, sincerely, this is *a* *lot* to take care of.
My suggestion would be to reduce the amount of work we do, the amount of things we build, and start relying more on what the distros can provide us. Let me explain.
For instance, we could ...
Stop building & distributing our own VMMs ================================= While I understand the reason why it's done, it'd be better to rely on the VMMs shipped by the distros.
In order to do this, we'd have to either approach the distros, or to do the tests ourselves, to ensure that their latest release has a VMM that's capable of running kata. If the distro has that, we document we support the distro. If the distro does not have that, we can provide instructions on how to build the VMM, but we take out of our back the work of building it statically and shipping to the consumers.
With this, we can do better testing on specific distros and ensure to possible consumers of kata where they can run the project and, with that, also forward the non kata specific issues to the distros themselves.
Keep building the kernel =================== Differently than the suggestion about the VMM, the kernel bits, IMHO, should still be built. The kernels we build can be used upstream, but we must ensure those are under no circumstances ready for production.Why? Because we have no power to track and fix CVEs as the distros do.
So, we can keep shipping our own kernel and testing using it, but we should also incentive distros to use (and test) their own kernels with kata. The reason I say this is because it'd help the project to *not* require a distro to ship a non-tested kernel, which would drop a little bit the barrier of accepting kata as their official packages.
Stop building & distributing our own distro packages ======================================== While I also understand that building the packages for several distros may help the users, I don't think this is something that should be on the kata developer's back.
For this, I'd suggest we take advantage of the kata-operator, being worked by Harshal and Jens Freimann, and consolidate on that as the way to install kata in your distro.
I foresee that we have to write some kind of "kata-osinfo" package (as in, a module, not as in an rpm or deb), where we can store information about the distro, which are the VMMs supported by the distro, which are the virt filesystems supported by that distro, etc; By doing this we can rely on a good documentation, or even on the operator to install the packages based on the info of each distro, but the only bit we'll distribute will be the kata binaries themselves.
Doing this, we can keep it simple on our side, while relying more on the distros.
Thanks for bringing it up and sorry for chiming in late. IMO the purpose of building/shipping packages is to make sure users get to install and use Kata Containers as easy as possible. We started building and distributing distro packages for the exact purpose.
Those things would already reduce the maintenance burden and, at the same time, show to our consumers what's supported and tested from our side.
Yes, we have been doing a lot of distribution work and it becomes a burden. If there are other ways to do it, it would be very good for us to shift the burden. Could we do a round of query, and see which distributions agrees to ship Kata Containers packages? And we can play an upstream role for those distributions. Take Fedora for example, if someone can stand up and become a package maintainer for Kata Containers (starting from rawhide of course), it would be pretty reasonable for Kata Community to stop building/shipping Kata Containers Fedora in OBS. My concern is that we might not be able to find enough package maintainers for each distribution. Then a possible solution is that we rely on package maintainers for those distributions that we can find a package maintainer, and continue building distribution packages for those we cannot. Cheers, Tao -- Into something rich and strange.
On Fri, 14 Aug 2020 at 05:09, Peng Tao <tao.peng@linux.alibaba.com> wrote:
Could we do a round of query, and see which distributions agrees to ship Kata Containers packages? And we can play an upstream role for those distributions.
Take Fedora for example, if someone can stand up and become a package maintainer for Kata Containers (starting from rawhide of course), it would be pretty reasonable for Kata Community to stop building/shipping Kata Containers Fedora in OBS.
My concern is that we might not be able to find enough package maintainers for each distribution.
This page is a summary of our understanding of the status of upstream Kata packaging: https://github.com/kata-containers/packaging/wiki/Distro-packaging-status Please can folks update if necessary. Cheers, James --- https://katacontainers.io/ | https://github.com/kata-containers <https://github.com/clearcontainers> Open Source Technology Center Intel Corporation (UK) Ltd. - Co. Reg. #1134945 - Pipers Way, Swindon SN3 1RJ.
participants (7)
-
Christophe de Dinechin
-
Christophe de Dinechin
-
Eric Ernst
-
Fabiano Fidêncio
-
Hunt, James O
-
Peng Tao
-
Samuel Ortiz