Open Stack

Hi,
I wanted to share some thoughts after running Kata on Fedora.  I built
Kata from source and used the Fedora rootfs together with a vanilla
modular kernel.  This configuration is similar to what I imagine
official Fedora RPMs would eventually use, because they probably want to
use a stock Fedora kernel instead of building a special monolithic
kernel for Kata.

The Fedora rootfs was built like this:

  # KERNEL_MODULES_DIR=/lib/modules/$(uname -r) OS_VERSION=28 ./rootfs.sh fedora

My understanding is Kata wants *either* an initramfs or a disk image.
Modular kernels typically need to load drivers before they can access
the disk, so I opted for the initramfs even though it is several hundred
megabytes.  The initramfs could probably be shrunk significantly with
some work on the Fedora rootfs script.

This configuration required several tweaks to get Kata running.  I'm not
going to create a patch series or send a pull request, because at this
stage these are dirty hacks that cannot be merged.  But it's likely that
other people will encounter these issues, so I'm including them below.

Stefan

---

 * kata-agent assumes initramfs is equivalent to running the agent as
   init.  In our case kata-agent *is* in an initramfs but *not* init, so
   it calculates the wrong value for noPivotRoot:

diff --git a/agent.go b/agent.go
index ec01250..182c2de 100644
--- a/agent.go
+++ b/agent.go
@@ -830,7 +832,7 @@ func realMain() {
                running:    false,
                // pivot_root won't work for init, see
                // Documention/filesystem/ramfs-rootfs-initramfs.txt
-               noPivotRoot:    os.Getpid() == 1,
+               noPivotRoot:    true, // TODO check initramfs, not pid=1!  kata-agent might have been started by systemd form an initramfs os.Getpid() == 1,
                subreaper:      r,
                pciDeviceMap:   make(map[string]string),
                deviceWatchers: make(map[string](chan string)),

 * kata-agent assumes AF_VSOCK should be used if the guest kernel has
   vsock support.  A generic kernel like the Fedora kernel will have
   AF_VSOCK but that doesn't mean the virtio-serial device should be
   ignored.  Perhaps change the check to see if
   /dev/virtio-ports/agent.channel.0 is present?

diff --git a/channel.go b/channel.go
index b4f8977..34ef292 100644
--- a/channel.go
+++ b/channel.go
@@ -29,25 +29,29 @@ type channel interface {
 }
 
 func newChannel() (channel, error) {
+       // This is broken, it ignores the virtio-serial agent.channel.0 device
+       // in favor of AF_VSOCK support, even on systems that are not going to
+       // use AF_VSOCK but happen to have the drivers installed.
+
        // Check for vsock support.
-       vSockSupported, err := isAFVSockSupported()
-       if err != nil {
-               return nil, err
-       }
-
-       if vSockSupported {
-               // Check if vsock socket exists. We want to cover the case
-               // where the guest OS can support vsock, but the runtime is
-               // still using virtio serial connection.
-               exist, err := vSockPathExist()
-               if err != nil {
-                       return nil, err
-               }
-
-               if exist {
-                       return &vSockChannel{}, nil
-               }
-       }
+//     vSockSupported, err := isAFVSockSupported()
+//     if err != nil {
+//             return nil, err
+//     }
+//
+//     if vSockSupported {
+//             // Check if vsock socket exists. We want to cover the case
+//             // where the guest OS can support vsock, but the runtime is
+//             // still using virtio serial connection.
+//             exist, err := vSockPathExist()
+//             if err != nil {
+//                     return nil, err
+//             }
+//
+//             if exist {
+//                     return &vSockChannel{}, nil
+//             }
+//     }
 
        return &serialChannel{}, nil
 }

 * The Fedora initramfs and modular kernel rely on udev to load the
   virtio_console.ko driver.  Make sure to start kata-agent *after* the
   agent device appears.  Note this change will break AF_VSOCK if you
   choose to use that instead of virtio-serial!

diff --git a/kata-agent.service.in b/kata-agent.service.in
index 861757d..5395047 100644
--- a/kata-agent.service.in
+++ b/kata-agent.service.in
@@ -2,6 +2,8 @@
 Description=Kata Containers Agent
 Documentation=https://github.com/kata-containers/agent
 Wants=kata-containers.target
+BindsTo=dev-virtio\x2dports-agent.channel.0.device
+After=dev-virtio\x2dports-agent.channel.0.device
 
 [Service]
 # Send agent output to tty to allow capture debug logs

 * Make sure the rootfs has modprobe and udev.  Needed by the modular
   kernel.

diff --git a/rootfs-builder/fedora/config.sh b/rootfs-builder/fedora/config.sh
index a0ca15a..19fbc7b 100644
--- a/rootfs-builder/fedora/config.sh
+++ b/rootfs-builder/fedora/config.sh
@@ -14,4 +14,4 @@ PACKAGES="iptables"
 #Optional packages:
 # systemd: An init system that will start kata-agent if kata-agent
 #          itself is not configured as init process.
-[ "$AGENT_INIT" == "no" ] && PACKAGES+=" systemd" || true
+[ "$AGENT_INIT" == "no" ] && PACKAGES+=" systemd systemd-udev" || true
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: not available
URL: <http://lists.katacontainers.io/pipermail/kata-dev/attachments/20180629/6c339b21/attachment.sig>