To summarize:
Use dracut to improve linux distribution adoption. - This can work for Feodra/RHEL - Allow to use the host kernel (required by fedora/Red Hat) - Waiting if some other distro provide feedback - If we implement it should we remove fedora support from osbuilder? as this was done to allow Fedora/Red Hat/CentOS adoption but are going with the dracut approach
Keep using osbuilder for the rest of use cases dracut not cover. Spacially for DAX images And not systemd based images (sorry I did not get if possible to not ship systemd).
Based in this summary, this lgtm.
On Thu, May 09, 2019 at 05:23:58PM +0100, Stefan Hajnoczi wrote:
On Tue, May 07, 2019 at 03:38:48PM -0500, Jose Carlos Venegas Munoz wrote:
Thanks to bring this to the ML. This is an interesting approach I have not play with Dracut a lot in the past but it definitely is a good thing look at.
When we started osbuilder the idea was to show adopters that Kata was able to boot with any OS as base (as long follow some requirements) and probably they adapt as needed or create their own tooling get Kata working for them. After all this time I see that it is more value to have just one tool that anyone can use and is flexible for developers and linux distributions adoption.
So I think would need more feedback from all the community and also from linux distributions in general so we can have a solution that works better for all.
This dracut approach is only aimed at distro package maintainers wishing to ship official Kata Containers packages. Users building Kata from source or customizing should continue to use osbuilder.
dracut does not cover all cases that osbuilder covers. It's also not used by all Linux distros, so they will have to use their equivalent tool.
However, I think dracut or equivalent distro tools are the right fit for official Kata Containers packages in Fedora, Debian, etc because:
- The same tool is already used to build the host initramfs and is familiar to package maintainers.
- It produces a smaller initramfs than osbuilder.
- It is fast because files are copied from the host instead of installed using a package manager.
- It eliminates packaging policy headaches because all files come from the host and therefore don't raise questions like "can we redistribute a ClearLinux rootfs and where do we mirror/publish the sources?".
Due to dracut's speed it would be appropriate to rebuild kata-containers-initrd.img every time the host kernel or kata-agent package is updated. Previously this problem was discussed because building a rootfs with osbuilder is slow and shouldn't hold up software updates, but I think it becomes a non-issue with dracut.
It make sense your points, and if this works for your cases (Fedora/Red Hat) I am happy to go in this path.
Some things to consider are :
Where this could leave? - agent code -osbuilder code
The best of this proposed case could be something that works well for your policy. Also from Fedora/Redhat perspective, do you think that keep Fedora in osbuilder supported rootfs has some value or we should remove it and go with this path.
As starting point we could have a job that uses this proposal using Fedora.
As extra note for the rest of the community: If someone is part of canonical, debian, or other distro let us know if this could simplify your work.
Here some question related with dracut usage:
- Is it possible to build a disk image?
I think the answer is not. This is a feature that is still difficult to be compatible with packaging but helps in memory usage if used with DAX. So probably a nice feature to keep
AFAIK dracut does not build disk images.
I've been booting from virtio-fs (with DAX) recently and am now wondering about skipping the cpio + compression step in dracut. That could achieve the same thing although virtiofsd has more memory overhead than a disk image NVDIMM.
- Is it possible to have a custom rootfs with packages I want?
Today we require system + agent + iptables + chrony. But also we support not having system but the agent as init. Also, users may want to add additional binaries in the future (to be part of the official image or just for hacking).
Additional files and binaries can be added from the host system. dracut does not use a package manager to install them, instead it copies them and library dependencies.
Nice I was open an issue to implement something similar for osbuilder https://github.com/kata-containers/osbuilder/issues/259 but if this work out of the box lets use it.
In the example I posted there is an initramfs-overlay/ directory. Files dropped in there will be built into the initramfs.
Stefan