[AMD Official Use Only - Internal Distribution Only]
Hi Developers,
1. Trying version-1 syntax: (for /usr/local/bin/containerd -config /etc/containerd/config.toml),
# cat /etc/containerd/config.toml
[plugins.cri.containerd]
no_pivot = false
[plugins.cri.containerd.runtimes]
[plugins.cri.containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v1"
[plugins.cri.containerd.runtimes.runc.options]
NoPivotRoot = false
NoNewKeyring = false
ShimCgroup = ""
IoUid = 0
IoGid = 0
BinaryName = "runc"
Root = ""
CriuPath = ""
SystemdCgroup = false
[plugins.cri.containerd.runtimes.kata]
runtime_type = "io.containerd.kata.v2"
[plugins.cri.containerd.runtimes.kata.options]
ConfigPath = "/etc/kata-containers/configuration.toml"
[plugins.cri.containerd.runtimes.katacli]
runtime_type = "io.containerd.runc.v1"
[plugins.cri.containerd.runtimes.katacli.options]
NoPivotRoot = false
NoNewKeyring = false
ShimCgroup = ""
IoUid = 0
IoGid = 0
BinaryName = "/usr/local/bin/kata-runtime"
Root = ""
CriuPath = ""
SystemdCgroup = false
1. kata-runtime version 1.1.0 is successful as follows :
/usr/local/bin/kata-runtime --version
kata-runtime : 1.1.0
commit : 292786890bd4792226c4fbf9b5268a92c27fe0f1-dirty
OCI specs: 1.0.1
1. kata-runtime ver 1.1.0 works, as follows :
# ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest sh
/ #
/ # ps -eaf
PID USER TIME COMMAND
1 root 0:00 sh
6 root 0:00 ps -eaf
/ #
/ # uname -a
Linux kata-container 5.6.0-sev-sev-es-5.6-v3.container #3 SMP Tue Nov 3 06:38:40 UTC 2020 x86_64 GNU/Linux
/ #
1. For kata-runtime, ver 1.1.0 :
# ps -eaf | grep -i container
root 98257 1 0 03:52 ? 00:00:19 /usr/local/bin/containerd
root 98329 1 0 03:52 ? 00:00:33 /usr/local/bin/containerd-shim-kata-v2 -namespace default -address /run/containerd/containerd.sock -publish-binary /usr/local/bin/containerd -id bash
root 98347 1 0 03:52 ? 00:00:51 /usr/local/bin/qemu-system-x86_64 -name sandbox-bash -uuid 27d16f68-1fc9-483f-a33b-e6224772273e -machine q35,accel=kvm,kernel_irqchip -cpu host, -qmp unix:/run/vc/vm/bash/qmp.sock,server,nowait -m 512M,slots=10,maxmem=129755M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2,romfile= -device virtio-serial-pci,disable-modern=false,id=serial0,romfile= -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/bash/console.sock,server,nowait -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0,romfile= -device virtserialport,chardev=charch0,id=channel0,name=agent.channel.0 -chardev socket,id=charch0,path=/run/vc/vm/bash/kata.sock,server,nowait -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared,romfile= -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/bash/shared,security_model=none,multidevs=remap -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-ram,id=dimm1,size=512M -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinuz-sev-es-5.6-v3.container -initrd /usr/share/kata-containers/kata-containers-initrd_alpine_1.12.0-rc0_agent_5cfb8ec960.initrd -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 iommu=off debug panic=1 nr_cpus=240 agent.use_vsock=false root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -pidfile /run/vc/vm/bash/pid -D /run/vc/vm/bash/qemu.log -smp 1,cores=1,threads=1,sockets=240,maxcpus=240
root 99602 99593 0 07:55 pts/1 00:00:00 grep --color=auto -i container
1. Exactly the same (above) environment, kata-runtime ver 2.0.0 is failing as follows:
# /usr/local/bin/kata-runtime --version
: 2.0.0
commit : <<unknown>>
OCI specs: 1.0.1-dev
/# ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest sh
ctr: Failed to check if grpc server is working: unexpected EOF: unknown
1. I can discuss this Issue in, Kata AC call in 2021 on January 5th.
--Thanking You --Best Regards, --Sandeep Gupta
Sandeep,
[...]
Trying version-1 syntax: (for /usr/local/bin/containerd –config /etc/containerd/config.toml),
# cat /etc/containerd/config.toml [plugins.cri.containerd] no_pivot = false
[plugins.cri.containerd.runtimes] [plugins.cri.containerd.runtimes.runc] runtime_type = "io.containerd.runc.v1" [plugins.cri.containerd.runtimes.runc.options] NoPivotRoot = false NoNewKeyring = false ShimCgroup = "" IoUid = 0 IoGid = 0 BinaryName = "runc" Root = "" CriuPath = "" SystemdCgroup = false [plugins.cri.containerd.runtimes.kata] runtime_type = "io.containerd.kata.v2" [plugins.cri.containerd.runtimes.kata.options] ConfigPath = "/etc/kata-containers/configuration.toml" [plugins.cri.containerd.runtimes.katacli] runtime_type = "io.containerd.runc.v1" [plugins.cri.containerd.runtimes.katacli.options] NoPivotRoot = false NoNewKeyring = false ShimCgroup = "" IoUid = 0 IoGid = 0 BinaryName = "/usr/local/bin/kata-runtime" Root = "" CriuPath = "" SystemdCgroup = false
[...]
/# ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest sh
ctr: Failed to check if grpc server is working: unexpected EOF: unknown
I can discuss this Issue in, Kata AC call in 2021 on January 5th.
We can, but I think it may be easier to debug this via e-mail or Slack. So, I'd like to ask you a few questions that will help me (and hopefully others) to understand where the error is coming from. * When you're using 2.x, you're only replacing the runtime binary, but also the image and the agent inside it, right? - The reason I'm asking is because 2.x runtime doesn't speak the same language as the 1.x agent. * Is your agent built using musl or gnu libc?
If you want to give it a quick try just to ensure kata 2.0 works for you before you start hacking the code, I'd suggest using https://github.com/kata-containers/packaging/tree/master/kata-deploy
I hope that helps!
Best Regards,
[AMD Official Use Only - Internal Distribution Only]
Thank you Fabiano, Will try the deployed version first, One more to check please - In my understanding - Docker doesn't yet support the containerd shimv2 architecture directly, Kata 2.0.0 won't work with Docker ? Is this correct ? Any workarounds.
--Best Regards --Sandeep
-----Original Message----- From: Fabiano Fidêncio fidencio@redhat.com Sent: Sunday, January 3, 2021 11:38 PM To: Gupta, Sandeep Sandeep.Gupta@amd.com Cc: kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime
[CAUTION: External Email]
Sandeep,
[...]
Trying version-1 syntax: (for /usr/local/bin/containerd -config /etc/containerd/config.toml),
# cat /etc/containerd/config.toml [plugins.cri.containerd] no_pivot = false
[plugins.cri.containerd.runtimes] [plugins.cri.containerd.runtimes.runc] runtime_type = "io.containerd.runc.v1" [plugins.cri.containerd.runtimes.runc.options] NoPivotRoot = false NoNewKeyring = false ShimCgroup = "" IoUid = 0 IoGid = 0 BinaryName = "runc" Root = "" CriuPath = "" SystemdCgroup = false [plugins.cri.containerd.runtimes.kata] runtime_type = "io.containerd.kata.v2" [plugins.cri.containerd.runtimes.kata.options] ConfigPath = "/etc/kata-containers/configuration.toml" [plugins.cri.containerd.runtimes.katacli] runtime_type = "io.containerd.runc.v1" [plugins.cri.containerd.runtimes.katacli.options] NoPivotRoot = false NoNewKeyring = false ShimCgroup = "" IoUid = 0 IoGid = 0 BinaryName = "/usr/local/bin/kata-runtime" Root = "" CriuPath = "" SystemdCgroup = false
[...]
/# ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest sh
ctr: Failed to check if grpc server is working: unexpected EOF: unknown
I can discuss this Issue in, Kata AC call in 2021 on January 5th.
We can, but I think it may be easier to debug this via e-mail or Slack. So, I'd like to ask you a few questions that will help me (and hopefully others) to understand where the error is coming from. * When you're using 2.x, you're only replacing the runtime binary, but also the image and the agent inside it, right? - The reason I'm asking is because 2.x runtime doesn't speak the same language as the 1.x agent. * Is your agent built using musl or gnu libc?
If you want to give it a quick try just to ensure kata 2.0 works for you before you start hacking the code, I'd suggest using https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com...
I hope that helps!
Best Regards, -- Fabiano Fidêncio
Sandeep,
On Mon, Jan 4, 2021 at 12:09 PM Gupta, Sandeep Sandeep.Gupta@amd.com wrote:
[AMD Official Use Only - Internal Distribution Only]
Thank you Fabiano, Will try the deployed version first, One more to check please - In my understanding - Docker doesn't yet support the containerd shimv2 architecture directly, Kata 2.0.0 won't work with Docker ? Is this correct ?
That's correct. CLI support has been dropped as part of 2.x so Kata 2.x won't work with docker.
Any workarounds.
Not sure if I can call it a workaround, but you need to stick to using ctr or crictl.
[...]
Best Regards,
Hi Sandeep,
Correct - Docker doesn't support shimv2 yet. However, getting Kata 2.x running with containerd is very simple and the CLI for containerd is reasonably similar to docker:
# Kata 1.x
$ docker run --runtime=kata-runtime ...
# Kata 2.x
$ ctr run --runtime io.containerd.kata.v2 ...
See [1] for further details.
Cheers,
James
[1] - https://github.com/kata-containers/kata-containers/blob/2.0-dev/docs/install... --- https://katacontainers.io/ | https://github.com/kata-containers Open Source Technology Center Intel Corporation (UK) Ltd. - Co. Reg. #1134945 - Pipers Way, Swindon SN3 1RJ.
________________________________________ From: Gupta, Sandeep Sandeep.Gupta@amd.com Sent: Monday, 4 January 2021 11:09 To: Fabiano Fidêncio; Gupta, Sandeep Cc: kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime
[AMD Official Use Only - Internal Distribution Only]
Thank you Fabiano, Will try the deployed version first, One more to check please - In my understanding - Docker doesn't yet support the containerd shimv2 architecture directly, Kata 2.0.0 won't work with Docker ? Is this correct ? Any workarounds.
--Best Regards --Sandeep
-----Original Message----- From: Fabiano Fidêncio fidencio@redhat.com Sent: Sunday, January 3, 2021 11:38 PM To: Gupta, Sandeep Sandeep.Gupta@amd.com Cc: kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime
[CAUTION: External Email]
Sandeep,
[...]
Trying version-1 syntax: (for /usr/local/bin/containerd -config /etc/containerd/config.toml),
# cat /etc/containerd/config.toml [plugins.cri.containerd] no_pivot = false
[plugins.cri.containerd.runtimes] [plugins.cri.containerd.runtimes.runc] runtime_type = "io.containerd.runc.v1" [plugins.cri.containerd.runtimes.runc.options] NoPivotRoot = false NoNewKeyring = false ShimCgroup = "" IoUid = 0 IoGid = 0 BinaryName = "runc" Root = "" CriuPath = "" SystemdCgroup = false [plugins.cri.containerd.runtimes.kata] runtime_type = "io.containerd.kata.v2" [plugins.cri.containerd.runtimes.kata.options] ConfigPath = "/etc/kata-containers/configuration.toml" [plugins.cri.containerd.runtimes.katacli] runtime_type = "io.containerd.runc.v1" [plugins.cri.containerd.runtimes.katacli.options] NoPivotRoot = false NoNewKeyring = false ShimCgroup = "" IoUid = 0 IoGid = 0 BinaryName = "/usr/local/bin/kata-runtime" Root = "" CriuPath = "" SystemdCgroup = false
[...]
/# ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest sh
ctr: Failed to check if grpc server is working: unexpected EOF: unknown
I can discuss this Issue in, Kata AC call in 2021 on January 5th.
We can, but I think it may be easier to debug this via e-mail or Slack. So, I'd like to ask you a few questions that will help me (and hopefully others) to understand where the error is coming from. * When you're using 2.x, you're only replacing the runtime binary, but also the image and the agent inside it, right? - The reason I'm asking is because 2.x runtime doesn't speak the same language as the 1.x agent. * Is your agent built using musl or gnu libc?
If you want to give it a quick try just to ensure kata 2.0 works for you before you start hacking the code, I'd suggest using https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com...
I hope that helps!
Best Regards, -- Fabiano Fidêncio
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
[AMD Official Use Only - Internal Distribution Only]
Thank you Fabiano, Thank you James,
I built 'kata-containers-stable-2.0.0/src/agent' and included it in the rootfs ---- "sudo install -o root -g root -m 0550 -T ../../agent/kata-agent ${ROOTFS_DIR}/sbin/init", And rebuilt 'tools/osbuilder/initrd-builder/kata-containers-initrd.img' ---- script -fec 'sudo -E AGENT_INIT=yes ./initrd_builder.sh ${ROOTFS_DIR}'
Now I can run -
# ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest sh / # / # / # ps PID USER TIME COMMAND 1 root 0:00 sh 2 root 0:00 ps / # exit
# /usr/local/bin/kata-runtime --version : 2.0.0 commit : <<unknown>> OCI specs: 1.0.1-dev
--Thanking You --Sandeep
-----Original Message----- From: Hunt, James O james.o.hunt@intel.com Sent: Monday, January 4, 2021 7:36 AM To: Gupta, Sandeep Sandeep.Gupta@amd.com; Fabiano Fidêncio fidencio@redhat.com Cc: kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime
[CAUTION: External Email]
Hi Sandeep,
Correct - Docker doesn't support shimv2 yet. However, getting Kata 2.x running with containerd is very simple and the CLI for containerd is reasonably similar to docker:
# Kata 1.x
$ docker run --runtime=kata-runtime ...
# Kata 2.x
$ ctr run --runtime io.containerd.kata.v2 ...
See [1] for further details.
Cheers,
James
[1] - https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com... --- https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fkatacontai... | https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com... Open Source Technology Center Intel Corporation (UK) Ltd. - Co. Reg. #1134945 - Pipers Way, Swindon SN3 1RJ.
________________________________________ From: Gupta, Sandeep Sandeep.Gupta@amd.com Sent: Monday, 4 January 2021 11:09 To: Fabiano Fidêncio; Gupta, Sandeep Cc: kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime
[AMD Official Use Only - Internal Distribution Only]
Thank you Fabiano, Will try the deployed version first, One more to check please - In my understanding - Docker doesn't yet support the containerd shimv2 architecture directly, Kata 2.0.0 won't work with Docker ? Is this correct ? Any workarounds.
--Best Regards --Sandeep
-----Original Message----- From: Fabiano Fidêncio fidencio@redhat.com Sent: Sunday, January 3, 2021 11:38 PM To: Gupta, Sandeep Sandeep.Gupta@amd.com Cc: kata-dev@lists.katacontainers.io Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime
[CAUTION: External Email]
Sandeep,
[...]
Trying version-1 syntax: (for /usr/local/bin/containerd -config /etc/containerd/config.toml),
# cat /etc/containerd/config.toml [plugins.cri.containerd] no_pivot = false
[plugins.cri.containerd.runtimes] [plugins.cri.containerd.runtimes.runc] runtime_type = "io.containerd.runc.v1" [plugins.cri.containerd.runtimes.runc.options] NoPivotRoot = false NoNewKeyring = false ShimCgroup = "" IoUid = 0 IoGid = 0 BinaryName = "runc" Root = "" CriuPath = "" SystemdCgroup = false [plugins.cri.containerd.runtimes.kata] runtime_type = "io.containerd.kata.v2" [plugins.cri.containerd.runtimes.kata.options] ConfigPath = "/etc/kata-containers/configuration.toml" [plugins.cri.containerd.runtimes.katacli] runtime_type = "io.containerd.runc.v1" [plugins.cri.containerd.runtimes.katacli.options] NoPivotRoot = false NoNewKeyring = false ShimCgroup = "" IoUid = 0 IoGid = 0 BinaryName = "/usr/local/bin/kata-runtime" Root = "" CriuPath = "" SystemdCgroup = false
[...]
/# ctr run --runtime io.containerd.run.kata.v2 -t --rm docker.io/library/busybox:latest sh
ctr: Failed to check if grpc server is working: unexpected EOF: unknown
I can discuss this Issue in, Kata AC call in 2021 on January 5th.
We can, but I think it may be easier to debug this via e-mail or Slack. So, I'd like to ask you a few questions that will help me (and hopefully others) to understand where the error is coming from. * When you're using 2.x, you're only replacing the runtime binary, but also the image and the agent inside it, right? - The reason I'm asking is because 2.x runtime doesn't speak the same language as the 1.x agent. * Is your agent built using musl or gnu libc?
If you want to give it a quick try just to ensure kata 2.0 works for you before you start hacking the code, I'd suggest using https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com...
I hope that helps!
Best Regards, -- Fabiano Fidêncio
_______________________________________________ kata-dev mailing list kata-dev@lists.katacontainers.io https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.katac... --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
participants (3)
-
Fabiano Fidêncio
-
Gupta, Sandeep
-
Hunt, James O