On Mon, Dec 16, 2019 at 07:30:40AM +0000, Boeuf, Sebastien wrote:
> Hi Stefano,
>
> On Fri, 2019-12-13 at 09:35 +0100, Stefano Garzarella wrote:
> > Hi Sebastien,
> >
> > On Thu, Dec 12, 2019 at 09:32:18PM +0000, Boeuf, Sebastien wrote:
> > > On Wed, 2019-12-11 at 18:27 +0100, Stefano Garzarella wrote:
> > > > On Thu, Dec 5, 2019 at 9:07 AM Boeuf, Sebastien <
> > > > sebastien.boeuf@intel.com> wrote:
> > > > > On Wed, 2019-12-04 at 17:36 +0000, Montes, Julio wrote:
> > > > > > Do you think could be an easy change in the Kata runtime?
> > > > > > I need to look better, but if you already know the answer
> > > > > > you'll
> > > > > > save me some time :)
> > > > >
> > > > > ok, this means all kata components must be in the same network
> > > > > namespace.
> > > > >
> > > > > fortunately, kata-shim and qemu run in the same network
> > > > > namespace,
> > > > > so the change should be easy, just move kata-proxy and kata-
> > > > > runtime
> > > > > into the same net namespace
> > > > >
> > > > >
> > > > > Well in case of vsock, we don't need kata-proxy, the shim being
> > > > > directly connected to the VM through vsock. And the kata-shim
> > > > > runs
> > > > > in the same netns as the VMM, so we're good from this
> > > > > perspective.
> > > > >
> > > > > In case of containerd-shimv2, we don't have kata-shim anymore,
> > > > > the
> > > > > shim being a simple thread part of the kata-runtime process,
> > > > > which
> > > > > means kata-runtime needs to enter the netns, and that's where
> > > > > it
> > > > > might be pretty complex. IIRC, Julio did some experiments to
> > > > > run
> > > > > the kata-runtime in a set of namespaces, but I don't think we
> > > > > merged it, did we?
> > > >
> > > > Looking in the code seems that kata-runtime uses the "vhostfd"
> > > > parameter
> > > > of the vsock device, this means that the vhost device
> > > > (/dev/vhost-
> > > > vsock)
> > > > is opened by the runtime and then the file descriptor is passed
> > > > to
> > > > QEMU.
> > > > This is great for this case because the netns assigned to the
> > > > device
> > > > is
> > > > the one of the process that opened the /dev/vhost-vsock.
> > > >
> > > > I tried containerd-shimv2 with the vhost-vsock modified to
> > > > support
> > > > netns
> > > > and seems to work well.
> > > >
> > > > Instead with v1 it doesn't work, maybe because kata-runtime and
> > > > kata-shim run in two different netns. (or maybe I did something
> > > > wrong,
> > > > but it works disabling vsock or using the vhost-vsock without
> > > > netns)
> > > >
> > > > Does that make sense?
> > >
> > > Yes I think your experiment and the result makes sense.
> >
> > ;-)
> >
> > > In case of v1, kata-shim runs in the same netns as QEMU, but kata-
> > > runtime runs in a separate one (the default netns). Later when
> > > kata-
> > > shim tries to access the fd, because it runs in a different netns
> > > than
> > > kata-runtime, the permission is denied.
> >
> > Do you think can easly changed?
> >
> > As Julio suggested, maybe we can move kata-runtime in the same netns.
>
> As I said earlier, this is something that could work, and I think Julio
> had a POC on this. The point is to make sure we run the entire process
> into the netns.

Cool!
@Julio, let me know if you have a POC working. I can try with my
changes.

>
> A possible hack would be to enter the netns when we open /dev/vhost-
> vsock, but that also means we need to enter the netns anytime the kata-
> runtime tries to communicate with the agent. Not sure this is a viable
> solution.

Perhaps it might be enough to enter into the netns only during the open.
If the fd is not closed, it remains associated with the netns, so there
is no need to re-enter for communication.

>
> >
> > Otherwise, is it possible to open the /dev/vhost-vsock in kata-
> > runtime,
> > then pass it to kata-shim that pass it to QEMU?
>
> From a pure technical standpoint, I'd say yes, but problem is, we would
> modify the shim and the kata-runtime/kata-shim interaction quite a lot.
> The kata-shim has been designed to be as simple as possible and act as
> a thin wrapper between containerd/crio and the container running in the
> VM.
>

Got it. If the hack above works, there's no point in complicating the
architecture.

> >
> > So if we can solve this easily, it seems that there are no big issues
> > for Kata with this new feature that I would like to introduce on
> > Linux.
>
> I agree, I think that eventually if Kata wants this feature to be
> supported, it will work out of the box for v2, and depending on the
> need for v1, the changes might be introduced.
>

Great! Thank you very much for your collaboration,
Stefano