Namespaces in the host
Hi All Currently Kata Containers only supports network namespaces in the host, other namespaces are not supported. This week I started to work on this but I faced some issues with the mount namespace in Go, the main issue is that a multithread process can't be reassociated with a new mount namespace [1] as it's reported here [2]. A solution for this might be to write a new binary in C to create and join namespaces and start the kata components (shim, qemu, etc) inside it, also this binary would be used to clean up (unmount, etc) the containers. Another solution might be to use a C constructor in Go (cgo) to create and join the namespace, but that means kata-runtime will run inside the namespace. If you are already working on this or know how to deal with this issue, please let me know. In the meantime I'll try to figure out how. Thank you! -- Julio [1] - http://man7.org/linux/man-pages/man2/setns.2.html [2] - https://github.com/golang/go/issues/8676
Hi Montes, As the issue described, Golang `setns` won't work with mount namespace, if you want to setns to a mount namespace, I think `cgo` should be a good way(at least I don't know a better one:-) ) Another way to put shim or qemu inside a new mount ns in Golang code is that you can set `Cmd.SysProcAttr.Cloneflags=syscall.CLONE_NEWNS` for the `shim` and `qemu` command, a quick example: ``` package main import ( "fmt" "os/exec" "syscall" ) func main() { cmd := exec.Cmd{ Path: "/bin/sleep", Args: []string{"sleep", "300"}, Dir: "/", SysProcAttr: &syscall.SysProcAttr{ Cloneflags: syscall.CLONE_NEWNET | syscall.CLONE_NEWNS, }, } out, err := cmd.CombinedOutput() if err != nil { fmt.Printf("error: %#v\n", err) return } fmt.Printf("output: %s\n", out) } ``` You can see the `sleep 300` process has a standalone mount ns after the programe started and hung, replace the "sleep" with `qemu`, I think it would work. Hope this can help :-) -Wei -----邮件原件----- 发件人: Montes, Julio [mailto:julio.montes@intel.com] 发送时间: 2018年9月28日 1:28 收件人: kata-dev@lists.katacontainers.io 主题: [kata-dev] Namespaces in the host Hi All Currently Kata Containers only supports network namespaces in the host, other namespaces are not supported. This week I started to work on this but I faced some issues with the mount namespace in Go, the main issue is that a multithread process can't be reassociated with a new mount namespace [1] as it's reported here [2]. A solution for this might be to write a new binary in C to create and join namespaces and start the kata components (shim, qemu, etc) inside it, also this binary would be used to clean up (unmount, etc) the containers. Another solution might be to use a C constructor in Go (cgo) to create and join the namespace, but that means kata-runtime will run inside the namespace. If you are already working on this or know how to deal with this issue, please let me know. In the meantime I'll try to figure out how. Thank you! -- Julio [1] - http://man7.org/linux/man-pages/man2/setns.2.html [2] - https://github.com/golang/go/issues/8676 _______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
Hi Wei, Thanks for your input. Regarding `cgo`, it's simply a way to compile some code written in C to be used by Go, but if we call a C function entering a mount namespace from our Go multithreaded process, this won't work. About the snippet of code that you provided here, it's not solving the whole problem unfortunately. With the fork and exec provided by Go, you can make a child process create and enter a new mount namespace, but you won't be able to enter an existing one. And the problem we're facing here is that we need to enter the mount when the container rootfs needs to be unmounted. And to add one more thing, it's not simply about Qemu process, but also about executing mount syscalls from a different mount namespace. This way, any remount that "docker cp" would perform, would not impact what's mounted through the shared directory that is passed through 9p. Thanks, Sebastien ________________________________________ From: zhangwei (CR) [zhangwei555@huawei.com] Sent: Thursday, September 27, 2018 7:43 PM To: Montes, Julio; kata-dev@lists.katacontainers.io Subject: [kata-dev] 答复: Namespaces in the host Hi Montes, As the issue described, Golang `setns` won't work with mount namespace, if you want to setns to a mount namespace, I think `cgo` should be a good way(at least I don't know a better one:-) ) Another way to put shim or qemu inside a new mount ns in Golang code is that you can set `Cmd.SysProcAttr.Cloneflags=syscall.CLONE_NEWNS` for the `shim` and `qemu` command, a quick example: ``` package main import ( "fmt" "os/exec" "syscall" ) func main() { cmd := exec.Cmd{ Path: "/bin/sleep", Args: []string{"sleep", "300"}, Dir: "/", SysProcAttr: &syscall.SysProcAttr{ Cloneflags: syscall.CLONE_NEWNET | syscall.CLONE_NEWNS, }, } out, err := cmd.CombinedOutput() if err != nil { fmt.Printf("error: %#v\n", err) return } fmt.Printf("output: %s\n", out) } ``` You can see the `sleep 300` process has a standalone mount ns after the programe started and hung, replace the "sleep" with `qemu`, I think it would work. Hope this can help :-) -Wei -----邮件原件----- 发件人: Montes, Julio [mailto:julio.montes@intel.com] 发送时间: 2018年9月28日 1:28 收件人: kata-dev@lists.katacontainers.io 主题: [kata-dev] Namespaces in the host Hi All Currently Kata Containers only supports network namespaces in the host, other namespaces are not supported. This week I started to work on this but I faced some issues with the mount namespace in Go, the main issue is that a multithread process can't be reassociated with a new mount namespace [1] as it's reported here [2]. A solution for this might be to write a new binary in C to create and join namespaces and start the kata components (shim, qemu, etc) inside it, also this binary would be used to clean up (unmount, etc) the containers. Another solution might be to use a C constructor in Go (cgo) to create and join the namespace, but that means kata-runtime will run inside the namespace. If you are already working on this or know how to deal with this issue, please let me know. In the meantime I'll try to figure out how. Thank you! -- Julio [1] - http://man7.org/linux/man-pages/man2/setns.2.html [2] - https://github.com/golang/go/issues/8676 _______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev _______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
Hi Julio, if you haven't already, you can take a look at what runc is doing. It forks a child process that re-execute itself (exec /proc/self/exe). Only the child process joins the namespaces with `setns` called from C code using cgo [1]. The child process is controlled from the parent with commands over pipes. Marco [1]: https://github.com/opencontainers/runc/tree/master/libcontainer/nsenter On Thu, 2018-09-27 at 17:27 +0000, Montes, Julio wrote:
Hi All
Currently Kata Containers only supports network namespaces in the host, other namespaces are not supported. This week I started to work on this but I faced some issues with the mount namespace in Go, the main issue is that a multithread process can't be reassociated with a new mount namespace [1] as it's reported here [2]. A solution for this might be to write a new binary in C to create and join namespaces and start the kata components (shim, qemu, etc) inside it, also this binary would be used to clean up (unmount, etc) the containers. Another solution might be to use a C constructor in Go (cgo) to create and join the namespace, but that means kata-runtime will run inside the namespace.
If you are already working on this or know how to deal with this issue, please let me know. In the meantime I'll try to figure out how.
Thank you!
-- Julio
[1] - http://man7.org/linux/man-pages/man2/setns.2.html [2] - https://github.com/golang/go/issues/8676 _______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev
participants (4)
-
Boeuf, Sebastien
-
Marco Vedovati
-
Montes, Julio
-
zhangwei (CR)