kata-dev
  Threads by month 
                
            - ----- 2025 -----
 - November
 - October
 - September
 - August
 - July
 - June
 - May
 - April
 - March
 - February
 - January
 - ----- 2024 -----
 - December
 - November
 - October
 - September
 - August
 - July
 - June
 - May
 - April
 - March
 - February
 - January
 - ----- 2023 -----
 - December
 - November
 - October
 - September
 - August
 - July
 - June
 - May
 - April
 - March
 - February
 - January
 - ----- 2022 -----
 - December
 - November
 - October
 - September
 - August
 - July
 - June
 - May
 - April
 - March
 - February
 - January
 - ----- 2021 -----
 - December
 - November
 - October
 - September
 - August
 - July
 - June
 - May
 - April
 - March
 - February
 - January
 - ----- 2020 -----
 - December
 - November
 - October
 - September
 - August
 - July
 - June
 - May
 - April
 - March
 - February
 - January
 - ----- 2019 -----
 - December
 - November
 - October
 - September
 - August
 - July
 - June
 - May
 - April
 - March
 - February
 - January
 - ----- 2018 -----
 - December
 - November
 - October
 - September
 - August
 - July
 - June
 - May
 - April
 - March
 - February
 - January
 - ----- 2017 -----
 - December
 
August 2021
- 22 participants
 - 31 discussions
 
                        
                            
                                
                            
                            Re: [kata-dev] , "kata-containers-2.2.0-alpha1" with confidential_guest = true (SEV Guest)
                        
                        
by Christophe de Dinechin 04 Aug '21
                    by Christophe de Dinechin 04 Aug '21
04 Aug '21
                    
                        
> On 2 Aug 2021, at 23:07, Gupta, Sandeep <Sandeep.Gupta(a)amd.com> wrote:
> 
> [AMD Official Use Only]
>  
> Hi,
>  
> I switched to “virtio-fs”,
> But now I see –
>  
> time="2021-08-02T20:54:02.621074148Z" level=info msg="virtiofsd quits" name=containerd-shim-v2 pid=180910 sandbox=ksh source=virtcontainers subsystem=virtiofsd
> time="2021-08-02T20:54:02.621252859Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=180910 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T20:54:02.621410880Z" level=debug msg="qemu-system-x86_64: -device vhost-user-fs-pci,chardev=char-2478d93684006754,tag=kataShared: iommu_platform=true is not supported by the device" name=containerd-shim-v2 pid=180910 sandbox=ksh source=virtcontainers subsystem=qemu
What version of qemu are you using? You may want to rebase it on something more recent I believe.
>  
>  
> qemu-system-x86_64:
> ============================
> time="2021-08-02T21:01:49.140554897Z" level=info msg="launching /usr/local/bin/qemu-system-x86_64 with: [-name sandbox-ksh -uuid d7bc6db6-03a6-4900-9aff-0cf40d0f27ea -machine q35,accel=kvm,kernel_irqchip=split,confidential-guest-support=sev -cpu host,pmu=off -qmp unix:/run/vc/vm/ksh/qmp.sock,server=on,wait=off -m 2048M,slots=10,maxmem=129755M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2 -device virtio-serial-pci,disable-modern=false,id=serial0 -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/ksh/console.sock,server=on,wait=off -device virtio-scsi-pci,id=scsi0,disable-modern=false -object sev-guest,id=sev,cbitpos=51,reduced-phys-bits=5 -drive if=pflash,format=raw,readonly=on,file=/usr/local/share/qemu/OVMF_CODE.fd.kata -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-1358627234,guest-cid=1358627234 -chardev socket,id=char-5492eaf755dce6c5,path=/run/vc/vm/ksh/vhost-fs.sock -device vhost-user-fs-pci,chardev=char-5492eaf755dce6c5,tag=kataShared -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-file,id=dimm1,size=2048M,mem-path=/dev/shm,share=on -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug panic=1 nr_cpus=240 scsi_mod.scan=none agent.log=debug root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -pidfile /run/vc/vm/ksh/pid -D /run/vc/vm/ksh/qemu.log -smp 1,cores=1,threads=1,sockets=240,maxcpus=240]" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qmp
>  
>  
> FULL LOG:
> ============
> DEBU[2021-08-02T21:01:48.966766727Z] stat snapshot                                 key="sha256:7444ea29e45e927abea1f923bf24cac20deaddea603c4bb1c7f2f5819773d453"
> DEBU[2021-08-02T21:01:48.972845851Z] prepare snapshot                              key=ksh parent="sha256:7444ea29e45e927abea1f923bf24cac20deaddea603c4bb1c7f2f5819773d453"
> DEBU[2021-08-02T21:01:48.978950864Z] event published                               ns=default topic=/snapshot/prepare type=containerd.events.SnapshotPrepare
> DEBU[2021-08-02T21:01:48.985793529Z] get snapshot mounts                           key=ksh
> DEBU[2021-08-02T21:01:49.002191295Z] event published                               ns=default topic=/containers/create type=containerd.events.ContainerCreate
> DEBU[2021-08-02T21:01:49.006646863Z] get snapshot mounts                           key=ksh
> time="2021-08-02T21:01:49.094225293Z" level=debug msg="registering ttrpc server"
> time="2021-08-02T21:01:49.094330423Z" level=debug msg="serving api on socket" socket="[inherited from parent]"
> time="2021-08-02T21:01:49.094348687Z" level=info msg="starting signal loop" namespace=default path=/run/containerd/io.containerd.runtime.v2.task/default/ksh pid=181864
> time="2021-08-02T21:01:49.094796228Z" level=debug msg="converting /run/containerd/io.containerd.runtime.v2.task/default/ksh/config.json" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=compatoci
> time="2021-08-02T21:01:49.098307832Z" level=info msg="loaded configuration" file=/etc/kata-containers/configuration.toml format=TOML name=containerd-shim-v2 pid=181864 sandbox=ksh source=katautils
> time="2021-08-02T21:01:49.098468848Z" level=info msg="IOMMUPlatform is enabled by default." name=containerd-shim-v2 pid=181864 sandbox=ksh source=katautils
> time="2021-08-02T21:01:49.098516779Z" level=debug default-kernel-parameters= name=containerd-shim-v2 pid=181864 sandbox=ksh source=katautils
> time="2021-08-02T21:01:49.099545003Z" level=debug msg="container rootfs: /run/containerd/io.containerd.runtime.v2.task/default/ksh/rootfs" source=virtcontainers subsystem=oci
> time="2021-08-02T21:01:49.099611590Z" level=info msg="shm-size detected: 67108864" source=virtcontainers subsystem=oci
> time="2021-08-02T21:01:49.104516813Z" level=info msg="create netns" name=containerd-shim-v2 netns=/var/run/netns/cnitest-8cca1b82-4956-c050-40ff-9d300bb4e199 pid=181864 sandbox=ksh source=katautils
> time="2021-08-02T21:01:49.104849866Z" level=debug msg="restore sandbox failed" error="open /run/vc/sbs/ksh/persist.json: no such file or directory" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=sandbox
> time="2021-08-02T21:01:49.129147606Z" level=info msg="Enabling SEV guest protection" kernel-params="[{tsc reliable} {no_timer_check } {rcupdate.rcu_expedited 1} {i8042.direct 1} {i8042.dumbkbd 1} {i8042.nopnp 1} {i8042.noaux 1} {noreplace-smp } {reboot k} {console hvc0} {console hvc1} {cryptomgr.notests } {net.ifnames 0} {pci lastbus=0}]" kernel-params-debug="[{debug }]" kernel-params-non-debug="[{quiet }]" machine="{q35  accel=kvm,kernel_irqchip=split}" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemuAmd64
> time="2021-08-02T21:01:49.129350371Z" level=debug msg="Creating bridges" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:49.129395657Z" level=debug msg="Creating UUID" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:49.136527482Z" level=debug msg="Disable nesting environment checks" inside-vm=false name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:49.137406503Z" level=info msg="adding volume" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu volume-type=virtio-fs
> time="2021-08-02T21:01:49.138303958Z" level=info msg="Endpoints found after scan" endpoints="[]" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=network
> time="2021-08-02T21:01:49.138475655Z" level=debug msg="Network added" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=network
> time="2021-08-02T21:01:49.138523075Z" level=info msg="Starting VM" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=sandbox
> time="2021-08-02T21:01:49.138628215Z" level=debug default-kernel-parameters="tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:49.139581246Z" level=info name=containerd-shim-v2 path=/usr/local/libexec/qemu/virtiofsd pid=181864 sandbox=ksh source=virtcontainers subsystem=virtiofsd
> time="2021-08-02T21:01:49.139680605Z" level=info args="--syslog -o cache=auto -o no_posix_lock -o source=/run/kata-containers/shared/sandboxes/ksh/shared --fd=3 -d --thread-pool-size=1" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=virtiofsd
> time="2021-08-02T21:01:49.140476457Z" level=info msg="Adding extra file [0xc000011468]" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qmp
> time="2021-08-02T21:01:49.140554897Z" level=info msg="launching /usr/local/bin/qemu-system-x86_64 with: [-name sandbox-ksh -uuid d7bc6db6-03a6-4900-9aff-0cf40d0f27ea -machine q35,accel=kvm,kernel_irqchip=split,confidential-guest-support=sev -cpu host,pmu=off -qmp unix:/run/vc/vm/ksh/qmp.sock,server=on,wait=off -m 2048M,slots=10,maxmem=129755M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2 -device virtio-serial-pci,disable-modern=false,id=serial0 -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/ksh/console.sock,server=on,wait=off -device virtio-scsi-pci,id=scsi0,disable-modern=false -object sev-guest,id=sev,cbitpos=51,reduced-phys-bits=5 -drive if=pflash,format=raw,readonly=on,file=/usr/local/share/qemu/OVMF_CODE.fd.kata -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-1358627234,guest-cid=1358627234 -chardev socket,id=char-5492eaf755dce6c5,path=/run/vc/vm/ksh/vhost-fs.sock -device vhost-user-fs-pci,chardev=char-5492eaf755dce6c5,tag=kataShared -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-file,id=dimm1,size=2048M,mem-path=/dev/shm,share=on -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug panic=1 nr_cpus=240 scsi_mod.scan=none agent.log=debug root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -pidfile /run/vc/vm/ksh/pid -D /run/vc/vm/ksh/qemu.log -smp 1,cores=1,threads=1,sockets=240,maxcpus=240]" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qmp
> DEBU[2021-08-02T21:01:49.177999243Z] garbage collected                             d=3.573111ms
> time="2021-08-02T21:01:50.609796180Z" level=info msg="virtiofsd quits" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=virtiofsd
> time="2021-08-02T21:01:50.609974409Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:50.610074960Z" level=debug msg="qemu-system-x86_64: -device vhost-user-fs-pci,chardev=char-5492eaf755dce6c5,tag=kataShared: iommu_platform=true is not supported by the device" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:50.610218803Z" level=warning msg="Unable to connect to unix socket (/run/vc/vm/ksh/qmp.sock): dial unix /run/vc/vm/ksh/qmp.sock: connect: connection refused" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qmp
> time="2021-08-02T21:01:50.610258178Z" level=error msg="Failed to connect to QEMU instance" error="dial unix /run/vc/vm/ksh/qmp.sock: connect: connection refused" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:50.610313303Z" level=info msg="cleanup vm path" dir=/run/vc/vm/ksh link=/run/vc/vm/ksh name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:50.611649432Z" level=error msg="Unable to launch /usr/local/bin/qemu-system-x86_64: exit status 1" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qmp
> time="2021-08-02T21:01:50.611754672Z" level=error name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qmp
> time="2021-08-02T21:01:50.611826689Z" level=error msg="failed to launch qemu: " error="exit status 1" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-08-02T21:01:50.936215508Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qemu
>  
>  
>  
> From: Christophe de Dinechin <cdupontd(a)redhat.com> 
> Sent: Monday, August 2, 2021 1:53 AM
> To: Gupta, Sandeep <Sandeep.Gupta(a)amd.com>
> Cc: kata-dev <kata-dev(a)lists.katacontainers.io>; James Cadden <jcadden(a)ibm.com>; Hollingsworth, Brent <brent.hollingsworth(a)amd.com>
> Subject: Re: [kata-dev] , "kata-containers-2.2.0-alpha1" with confidential_guest = true (SEV Guest)
>  
> [CAUTION: External Email] 
>  
>  
> 
> On 31 Jul 2021, at 19:42, Gupta, Sandeep <Sandeep.Gupta(a)amd.com> wrote:
>  
> [AMD Official Use Only]
> 
> Hi Christophe,
> 
> The 'qemu logs' are empty.
> 
> root@sandeepg-ethanolx-milan:/var/run/vc/vm/ksh# ls -al
> total 4
> drwxr-x--- 2 root root 120 Jul 31 17:37 .
> drwxr-x--- 3 root root  60 Jul 31 17:37 ..
> srwxr-x--- 1 root root   0 Jul 31 17:37 console.sock
> -rw------- 1 root root   8 Jul 31 17:37 pid
> -rw-r--r-- 1 root root   0 Jul 31 17:37 qemu.log
> srwxr-x--- 1 root root   0 Jul 31 17:37 qmp.sock
> 
> time="2021-07-31T17:37:05.007627568Z" level=info msg="VM started" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=sandbox
> time="2021-07-31T17:37:05.007708140Z" level=debug msg="console watcher starts" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=sandbox
> time="2021-07-31T17:37:05.007841804Z" level=debug msg="DNS file not present in ociMounts. Sandbox DNS will not be set." name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=kata_agent
> time="2021-07-31T17:37:05.007915363Z" level=info msg="New client" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=kata_agent url="vsock://3018373783:1024"
> time="2021-07-31T17:37:35.251396370Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-07-31T17:37:35.253486712Z" level=warning msg="The virtiofsd had stopped" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=qemu
> time="2021-07-31T17:37:35.253569129Z" level=debug msg="got QMP event" event="{SHUTDOWN map[guest:false reason:host-qmp-quit] 2021-07-31 17:37:35.000253386 +0000 UTC}" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=qemu
> 
> 
> I am using "virtio-9p" --
> 
> # Shared file system type:
> #   - virtio-fs (default)
> #   - virtio-9p
> shared_fs = "virtio-9p"
> 
> Then why this warning message - warning msg="The virtiofsd had stopped" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=qemu   ??
>  
> Oh, I had not seen that you had configured 9pfs.
>  
> Is that still supported?
>  
> In my limited experience, you can get the "virtiofsd has stopped" message when there is some abrupt crash of one of the components, and the socket is closed.
>  
> 
> 
> 
> --Thanking You
> --Best
> --Sandeep
> 
> 
> 
> -----Original Message-----
> From: Christophe de Dinechin <cdupontd(a)redhat.com> 
> Sent: Friday, July 30, 2021 1:54 AM
> To: Gupta, Sandeep <Sandeep.Gupta(a)amd.com>
> Cc: kata-dev <kata-dev(a)lists.katacontainers.io>
> Subject: Re: [kata-dev] , "kata-containers-2.2.0-alpha1" with confidential_guest = true (SEV Guest)
> 
> [CAUTION: External Email]
> 
> 
> On 30 Jul 2021, at 10:21, Gupta, Sandeep via kata-dev <kata-dev(a)lists.katacontainers.io> wrote:
> 
> [AMD Official Use Only]
> 
> Hi All,
> 
> (1) Using kata-containers-2.2.0-alpha1 -  configuration.toml --
> 
> # Copyright (c) 2017-2019 Intel Corporation # # SPDX-License-Identifier: Apache-2.0 #
> 
> # XXX: WARNING: this file is auto-generated.
> # XXX:
> # XXX: Source file: "cli/config/configuration-qemu.toml.in"
> # XXX: Project:
> # XXX:   Name: Kata Containers
> # XXX:   Type: kata
> 
> [hypervisor.qemu]
> path = "/usr/local/bin/qemu-system-x86_64"
> #kernel = "/usr/share/kata-containers/vmlinux.container"
> kernel = "/usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container"
> #image = "/usr/share/kata-containers/kata-containers.img"
> initrd = "/usr/share/kata-containers/kata-containers-initrd.img"
> machine_type = "q35"
> 
> # Enable confidential guest support.
> # Toggling that setting may trigger different hardware features, ranging # from memory encryption to both memory and CPU-state encryption and integrity.
> # The Kata Containers runtime dynamically detects the available feature set and # aims at enabling the largest possible one.
> # Default false
> confidential_guest = true
> 
> # List of valid annotation names for the hypervisor # Each member of the list is a regular expression, which is the base name # of the annotation, e.g. "path" for io.katacontainers.config.hypervisor.path"
> enable_annotations = []
> 
> # List of valid annotations values for the hypervisor # Each member of the list is a path pattern as described by glob(3).
> # The default if not set is empty (all annotations rejected.) # Your distribution recommends: ["/usr/bin/qemu-system-x86_64"] valid_hypervisor_paths = ["/usr/local/bin/qemu-system-x86_64"]
> 
> # Optional space-separated list of options to pass to the guest kernel.
> # For example, use `kernel_params = "vsyscall=emulate"` if you are having # trouble running pre-2.15 glibc.
> #
> # WARNING: - any parameter specified here will take priority over the default # parameter value of the same name used to start the virtual machine.
> # Do not set values here unless you understand the impact of doing so as you # may stop the virtual machine from booting.
> # To see the list of default parameters, enable hypervisor debug, create a # container and look for 'default-kernel-parameters' log entries.
> #kernel_params = ""
> kernel_params = " root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug"
> 
> # Path to the firmware.
> # If you want that qemu uses the default firmware leave this option empty #firmware = ""
> firmware = "/usr/local/share/qemu/OVMF_CODE.fd.kata"
> ...........
> ..............
> # Shared file system type:
> #   - virtio-fs (default)
> #   - virtio-9p
> shared_fs = "virtio-9p"
> ..............
> ................
> 
> 
> (2) containerd version 1.5.2
> /usr/local/bin/containerd --version
> containerd github.com/containerd/containerd 1.5.2-0ubuntu1~20.04.2
> 
> /usr/local/bin/containerd --config /etc/containerd/config.toml
> 
> # cat /etc/containerd/config.toml
> #   Copyright 2018-2020 Docker Inc.
> 
> #   Licensed under the Apache License, Version 2.0 (the "License");
> #   you may not use this file except in compliance with the License.
> #   You may obtain a copy of the License at
> 
> #       https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.apache…
> 
> #   Unless required by applicable law or agreed to in writing, software
> #   distributed under the License is distributed on an "AS IS" BASIS,
> #   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> #   See the License for the specific language governing permissions and
> #   limitations under the License.
> [debug]
>       level = "debug"
> 
> [plugins.cri.containerd]
>     no_pivot = false
>   [plugins.cri.containerd.runtimes]
>     [plugins.cri.containerd.runtimes.runc]
>        runtime_type = "io.containerd.runc.v1"
>        [plugins.cri.containerd.runtimes.runc.options]
>          NoPivotRoot = false
>          NoNewKeyring = false
>          ShimCgroup = ""
>          IoUid = 0
>          IoGid = 0
>          BinaryName = "runc"
>          Root = ""
>          CriuPath = ""
>          SystemdCgroup = false
>     [plugins.cri.containerd.runtimes.kata]
>        runtime_type = "io.containerd.kata.v2"
>         [plugins.cri.containerd.runtimes.kata.options]
>          ConfigPath = "/etc/kata-containers/configuration.toml"
>          NoPivotRoot = false
>          NoNewKeyring = false
>          ShimCgroup = ""
>          IoUid = 0
>          IoGid = 0
>          BinaryName = "/usr/local/bin/kata-runtime"
>          Root = ""
>          CriuPath = ""
>          SystemdCgroup = false
>     [plugins.cri.containerd.runtimes.katacli]
>        runtime_type = "io.containerd.runc.v1"
>        [plugins.cri.containerd.runtimes.katacli.options]
>          NoPivotRoot = false
>          NoNewKeyring = false
>          ShimCgroup = ""
>          IoUid = 0
>          IoGid = 0
>          BinaryName = "/usr/local/bin/kata-runtime"
>          Root = ""
>          CriuPath = ""
>          SystemdCgroup = false
> 
> 
> (3) # ctr run --runtime io.containerd.kata.v2 -t --rm docker.io/library/alpine_linux:latest bash
> ctr: failed to create shim: Failed to check if grpc server is working: rpc error: code = DeadlineExceeded desc = timed out connecting to vsock 1025935578:1024: unknown
> 
> 
> (4) containerd LOG:
> 
> DEBU[2021-07-30T07:53:27.915940335Z] stat snapshot                                 key="sha256:7444ea29e45e927abea1f923bf24cac20deaddea603c4bb1c7f2f5819773d453"
> DEBU[2021-07-30T07:53:27.921636473Z] prepare snapshot                              key=bash parent="sha256:7444ea29e45e927abea1f923bf24cac20deaddea603c4bb1c7f2f5819773d453"
> DEBU[2021-07-30T07:53:27.930411184Z] event published                               ns=default topic=/snapshot/prepare type=containerd.events.SnapshotPrepare
> DEBU[2021-07-30T07:53:27.939032985Z] get snapshot mounts                           key=bash
> DEBU[2021-07-30T07:53:27.955103540Z] event published                               ns=default topic=/containers/create type=containerd.events.ContainerCreate
> DEBU[2021-07-30T07:53:27.962015595Z] get snapshot mounts                           key=bash
> DEBU[2021-07-30T07:53:28.039779478Z] garbage collected                             d=2.588852ms
> time="2021-07-30T07:53:28.043806590Z" level=debug msg="registering ttrpc server"
> time="2021-07-30T07:53:28.043934212Z" level=debug msg="serving api on socket" socket="[inherited from parent]"
> time="2021-07-30T07:53:28.043956084Z" level=info msg="starting signal loop" namespace=default path=/run/containerd/io.containerd.runtime.v2.task/default/bash pid=1590317 time="2021-07-30T07:53:28.044534952Z" level=debug msg="converting /run/containerd/io.containerd.runtime.v2.task/default/bash/config.json" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=compatoci time="2021-07-30T07:53:28.049788100Z" level=info msg="loaded configuration" file=/etc/kata-containers/configuration.toml format=TOML name=containerd-shim-v2 pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.049991937Z" level=info msg="IOMMUPlatform is enabled by default." name=containerd-shim-v2 pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.051163250Z" level=info msg="shm-size detected: 67108864" source=virtcontainers subsystem=oci time="2021-07-30T07:53:28.054669132Z" level=info msg="create netns" name=containerd-shim-v2 netns=/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.071513285Z" level=info msg="Enabling SEV guest protection" kernel-params="[{tsc reliable} {no_timer_check } {rcupdate.rcu_expedited 1} {i8042.direct 1} {i8042.dumbkbd 1} {i8042.nopnp 1} {i8042.noaux 1} {noreplace-smp } {reboot k} {console hvc0} {console hvc1} {cryptomgr.notests } {net.ifnames 0} {pci lastbus=0}]" kernel-params-debug="[{debug }]" kernel-params-non-debug="[{quiet }]" machine="{q35  accel=kvm,kernel_irqchip=split}" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemuAmd64 time="2021-07-30T07:53:28.092119026Z" level=info msg="adding volume" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu volume-type=virtio-9p time="2021-07-30T07:53:28.093015598Z" level=info msg="Endpoints found after scan" endpoints="[]" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=network time="2021-07-30T07:53:28.093157978Z" level=info msg="Starting VM" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:28.093562065Z" level=info msg="Adding extra file [0xc000010058]" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:28.093652406Z" level=info msg="launching /usr/local/bin/qemu-system-x86_64 with: [-name sandbox-bash -uuid b1ff0af6-de06-4025-ab63-9aad959900d5 -machine q35,accel=kvm,kernel_irqchip=split,confidential-guest-support=sev -cpu host,pmu=off -qmp unix:/run/vc/vm/bash/qmp.sock,server=on,wait=off -m 2048M,slots=10,maxmem=129746M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2 -device virtio-serial-pci,disable-modern=false,id=serial0 -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/bash/console.sock,server=on,wait=off -device virtio-scsi-pci,id=scsi0,disable-modern=false -object sev-guest,id=sev,cbitpos=51,reduced-phys-bits=5 -drive if=pflash,format=raw,readonly=on,file=/usr/local/share/qemu/OVMF_CODE.fd.kata -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-1025935578,guest-cid=1025935578 -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/bash/shared,security_model=none,multidevs=remap -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-ram,id=dimm1,size=2048M -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug panic=1 nr_cpus=240 scsi_mod.scan=none root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -pidfile /run/vc/vm/bash/pid -D /run/vc/vm/bash/qemu.log -smp 1,cores=1,threads=1,sockets=240,maxcpus=240]"
> 
> 
> 
> name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:29.231939187Z" level=info msg="QMP details" name=containerd-shim-v2 pid=1590317 qmp-capabilities=oob qmp-major-version=6 qmp-micro-version=91 qmp-minor-version=0 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:29.232726561Z" level=info msg="scanner return error: read unix @->/run/vc/vm/bash/qmp.sock: use of closed network connection" name=containerd-shim-v2 pid=1590317 sandbox=bash
> 
> It looks like your qemu stopped right after being started (T07:53:29.231939187Z vs T07:53:28.093652406Z). I think that the qemu logs would help us.
> 
> 
> source=virtcontainers subsystem=qmp time="2021-07-30T07:53:29.232914117Z" level=info msg="VM started" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:29.233044113Z" level=info msg="New client" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=kata_agent url="vsock://1025935578:1024"
> time="2021-07-30T07:53:59.475362081Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477063059Z" level=warning msg="The virtiofsd had stopped" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477246988Z" level=info msg="cleanup vm path" dir=/run/vc/vm/bash link=/run/vc/vm/bash name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477447608Z" level=info msg="Network namespace \"/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b\" deleted" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=network time="2021-07-30T07:53:59.478504534Z" level=warning msg="sandbox cgroups path is empty" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:59.478574457Z" level=info msg="cleanup agent" name=containerd-shim-v2 path=/run/kata-containers/shared/sandboxes/bash/shared pid=1590317 sandbox=bash source=virtcontainers subsystem=kata_agent time="2021-07-30T07:53:59.479262623Z" level=warning msg="failed to cleanup netns" error="failed to get netns /var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b: failed to Statfs \"/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b\": no such file or directory" name=containerd-shim-v2 path=/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:59.479464936Z" level=info msg="scanner return error: <nil>" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:59.479683231Z" level=info msg="QMP event channel closed" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu
> DEBU[2021-07-30T07:53:59.480447321Z] failed to delete task                         error="rpc error: code = NotFound desc = container does not exist bash: not found" id=bash
> INFO[2021-07-30T07:53:59.484752059Z] shim disconnected                             id=bash
> WARN[2021-07-30T07:53:59.484860645Z] cleaning up after shim disconnected           id=bash namespace=default
> INFO[2021-07-30T07:53:59.484882296Z] cleaning up dead shim
> ERRO[2021-07-30T07:53:59.545695272Z] failed to delete                              cmd="/usr/local/bin/containerd-shim-kata-v2 -namespace default -address /run/containerd/containerd.sock -publish-binary /usr/local/bin/containerd -id bash -bundle /run/containerd/io.containerd.runtime.v2.task/default/bash delete" error="exit status 1"
> WARN[2021-07-30T07:53:59.545862980Z] failed to clean up after shim disconnected    error="time=\"2021-07-30T07:53:59Z\" level=warning msg=\"failed to cleanup container\" container=bash error=\"open /run/vc/sbs/bash: no such file or directory\" name=containerd-shim-v2 pid=1590410 sandbox=bash source=containerd-kata-shim-v2\nio.containerd.kata.v2: open /run/vc/sbs/bash: no such file or directory\n: exit status 1" id=bash namespace=default
> ERRO[2021-07-30T07:53:59.545951698Z] copy shim log                                 error="read /proc/self/fd/15: file already closed"
> DEBU[2021-07-30T07:53:59.557980504Z] remove snapshot                               key=bash
> DEBU[2021-07-30T07:53:59.560442896Z] event published                               ns=default topic=/snapshot/remove type=containerd.events.SnapshotRemove
> DEBU[2021-07-30T07:53:59.563483766Z] event published                               ns=default topic=/containers/delete type=containerd.events.ContainerDelete
> DEBU[2021-07-30T07:53:59.610327214Z] schedule snapshotter cleanup                  snapshotter=overlayfs
> DEBU[2021-07-30T07:53:59.612570831Z] removed snapshot                              key=default/23/bash snapshotter=overlayfs
> DEBU[2021-07-30T07:53:59.613058306Z] snapshot garbage collected                    d=2.633938ms snapshotter=overlayfs
> DEBU[2021-07-30T07:53:59.613107950Z] garbage collected                             d=2.265409ms
> 
> 
> Any suggestions please.
> 
> 
> --Thanking You
> --Sandeep
> 
> _______________________________________________
> kata-dev mailing list
> kata-dev(a)lists.katacontainers.io
> https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.kata…
                    
                  
                  
                          
                            
                            1
                            
                          
                          
                            
                            1
                            
                          
                          
                            
    
                          
                        
                    
                    
                        Hi everyone.
Samuels and I worked today on mapping the current tasks we have on the V0
plan spreadsheet
<https://docs.google.com/spreadsheets/d/1M_MijAutym4hMg8KtIye1jIDAUMUWsFCri9…>
to
the proposed V0 milestones
<https://docs.google.com/presentation/d/1SIqLogbauLf6lG53cIBPMOFadRT23aXuTGC…>
from last week (September, October, November).
For each task we added the milestone we think it fits into and also added a
tab to the spreadsheet with the details on each of the milestones content.
It would be great if the relevant task owners could review this mapping and
comment if it makes sense or if it should be changed. We are also trying to
identify any internal/external dependencies we have which we also need to
take into account.
Thanks.
                    
                  
                  
                          
                            
                            1
                            
                          
                          
                            
                            0
                            
                          
                          
                            
    
                          
                        
                    
                        
                            
                                
                            
                            Re: [kata-dev] , "kata-containers-2.2.0-alpha1" with confidential_guest = true (SEV Guest)
                        
                        
by Gupta, Sandeep 02 Aug '21
                    by Gupta, Sandeep 02 Aug '21
02 Aug '21
                    
                        [AMD Official Use Only]
Hi,
I switched to "virtio-fs",
But now I see -
time="2021-08-02T20:54:02.621074148Z" level=info msg="virtiofsd quits" name=containerd-shim-v2 pid=180910 sandbox=ksh source=virtcontainers subsystem=virtiofsd
time="2021-08-02T20:54:02.621252859Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=180910 sandbox=ksh source=virtcontainers subsystem=qemu
time="2021-08-02T20:54:02.621410880Z" level=debug msg="qemu-system-x86_64: -device vhost-user-fs-pci,chardev=char-2478d93684006754,tag=kataShared: iommu_platform=true is not supported by the device" name=containerd-shim-v2 pid=180910 sandbox=ksh source=virtcontainers subsystem=qemu
qemu-system-x86_64:
============================
time="2021-08-02T21:01:49.140554897Z" level=info msg="launching /usr/local/bin/qemu-system-x86_64 with: [-name sandbox-ksh -uuid d7bc6db6-03a6-4900-9aff-0cf40d0f27ea -machine q35,accel=kvm,kernel_irqchip=split,confidential-guest-support=sev -cpu host,pmu=off -qmp unix:/run/vc/vm/ksh/qmp.sock,server=on,wait=off -m 2048M,slots=10,maxmem=129755M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2 -device virtio-serial-pci,disable-modern=false,id=serial0 -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/ksh/console.sock,server=on,wait=off -device virtio-scsi-pci,id=scsi0,disable-modern=false -object sev-guest,id=sev,cbitpos=51,reduced-phys-bits=5 -drive if=pflash,format=raw,readonly=on,file=/usr/local/share/qemu/OVMF_CODE.fd.kata -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-1358627234,guest-cid=1358627234 -chardev socket,id=char-5492eaf755dce6c5,path=/run/vc/vm/ksh/vhost-fs.sock -device vhost-user-fs-pci,chardev=char-5492eaf755dce6c5,tag=kataShared -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-file,id=dimm1,size=2048M,mem-path=/dev/shm,share=on -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug panic=1 nr_cpus=240 scsi_mod.scan=none agent.log=debug root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -pidfile /run/vc/vm/ksh/pid -D /run/vc/vm/ksh/qemu.log -smp 1,cores=1,threads=1,sockets=240,maxcpus=240]" name=containerd-shim-v2 pid=181864 sandbox=ksh source=virtcontainers subsystem=qmp
From: Christophe de Dinechin <cdupontd(a)redhat.com<mailto:cdupontd@redhat.com>>
Sent: Monday, August 2, 2021 1:53 AM
To: Gupta, Sandeep <Sandeep.Gupta(a)amd.com<mailto:Sandeep.Gupta@amd.com>>
Cc: kata-dev <kata-dev(a)lists.katacontainers.io<mailto:kata-dev@lists.katacontainers.io>>; James Cadden <jcadden(a)ibm.com<mailto:jcadden@ibm.com>>; Hollingsworth, Brent <brent.hollingsworth(a)amd.com<mailto:brent.hollingsworth@amd.com>>
Subject: Re: [kata-dev] , "kata-containers-2.2.0-alpha1" with confidential_guest = true (SEV Guest)
[CAUTION: External Email]
On 31 Jul 2021, at 19:42, Gupta, Sandeep <Sandeep.Gupta(a)amd.com<mailto:Sandeep.Gupta@amd.com>> wrote:
[AMD Official Use Only]
Hi Christophe,
The 'qemu logs' are empty.
root@sandeepg-ethanolx-milan:/var/run/vc/vm/ksh# ls -al
total 4
drwxr-x--- 2 root root 120 Jul 31 17:37 .
drwxr-x--- 3 root root  60 Jul 31 17:37 ..
srwxr-x--- 1 root root   0 Jul 31 17:37 console.sock
-rw------- 1 root root   8 Jul 31 17:37 pid
-rw-r--r-- 1 root root   0 Jul 31 17:37 qemu.log
srwxr-x--- 1 root root   0 Jul 31 17:37 qmp.sock
time="2021-07-31T17:37:05.007627568Z" level=info msg="VM started" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=sandbox
time="2021-07-31T17:37:05.007708140Z" level=debug msg="console watcher starts" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=sandbox
time="2021-07-31T17:37:05.007841804Z" level=debug msg="DNS file not present in ociMounts. Sandbox DNS will not be set." name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=kata_agent
time="2021-07-31T17:37:05.007915363Z" level=info msg="New client" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=kata_agent url="vsock://3018373783:1024"
time="2021-07-31T17:37:35.251396370Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=qemu
time="2021-07-31T17:37:35.253486712Z" level=warning msg="The virtiofsd had stopped" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=qemu
time="2021-07-31T17:37:35.253569129Z" level=debug msg="got QMP event" event="{SHUTDOWN map[guest:false reason:host-qmp-quit] 2021-07-31 17:37:35.000253386 +0000 UTC}" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=qemu
I am using "virtio-9p" --
# Shared file system type:
#   - virtio-fs (default)
#   - virtio-9p
shared_fs = "virtio-9p"
Then why this warning message - warning msg="The virtiofsd had stopped" name=containerd-shim-v2 pid=1897831 sandbox=ksh source=virtcontainers subsystem=qemu   ??
Oh, I had not seen that you had configured 9pfs.
Is that still supported?
In my limited experience, you can get the "virtiofsd has stopped" message when there is some abrupt crash of one of the components, and the socket is closed.
--Thanking You
--Best
--Sandeep
-----Original Message-----
From: Christophe de Dinechin <cdupontd(a)redhat.com<mailto:cdupontd@redhat.com>>
Sent: Friday, July 30, 2021 1:54 AM
To: Gupta, Sandeep <Sandeep.Gupta(a)amd.com<mailto:Sandeep.Gupta@amd.com>>
Cc: kata-dev <kata-dev(a)lists.katacontainers.io<mailto:kata-dev@lists.katacontainers.io>>
Subject: Re: [kata-dev] , "kata-containers-2.2.0-alpha1" with confidential_guest = true (SEV Guest)
[CAUTION: External Email]
On 30 Jul 2021, at 10:21, Gupta, Sandeep via kata-dev <kata-dev(a)lists.katacontainers.io<mailto:kata-dev@lists.katacontainers.io>> wrote:
[AMD Official Use Only]
Hi All,
(1) Using kata-containers-2.2.0-alpha1 -  configuration.toml --
# Copyright (c) 2017-2019 Intel Corporation # # SPDX-License-Identifier: Apache-2.0 #
# XXX: WARNING: this file is auto-generated.
# XXX:
# XXX: Source file: "cli/config/configuration-qemu.toml.in"
# XXX: Project:
# XXX:   Name: Kata Containers
# XXX:   Type: kata
[hypervisor.qemu]
path = "/usr/local/bin/qemu-system-x86_64"
#kernel = "/usr/share/kata-containers/vmlinux.container"
kernel = "/usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container"
#image = "/usr/share/kata-containers/kata-containers.img"
initrd = "/usr/share/kata-containers/kata-containers-initrd.img"
machine_type = "q35"
# Enable confidential guest support.
# Toggling that setting may trigger different hardware features, ranging # from memory encryption to both memory and CPU-state encryption and integrity.
# The Kata Containers runtime dynamically detects the available feature set and # aims at enabling the largest possible one.
# Default false
confidential_guest = true
# List of valid annotation names for the hypervisor # Each member of the list is a regular expression, which is the base name # of the annotation, e.g. "path" for io.katacontainers.config.hypervisor.path"
enable_annotations = []
# List of valid annotations values for the hypervisor # Each member of the list is a path pattern as described by glob(3).
# The default if not set is empty (all annotations rejected.) # Your distribution recommends: ["/usr/bin/qemu-system-x86_64"] valid_hypervisor_paths = ["/usr/local/bin/qemu-system-x86_64"]
# Optional space-separated list of options to pass to the guest kernel.
# For example, use `kernel_params = "vsyscall=emulate"` if you are having # trouble running pre-2.15 glibc.
#
# WARNING: - any parameter specified here will take priority over the default # parameter value of the same name used to start the virtual machine.
# Do not set values here unless you understand the impact of doing so as you # may stop the virtual machine from booting.
# To see the list of default parameters, enable hypervisor debug, create a # container and look for 'default-kernel-parameters' log entries.
#kernel_params = ""
kernel_params = " root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug"
# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty #firmware = ""
firmware = "/usr/local/share/qemu/OVMF_CODE.fd.kata"
...........
..............
# Shared file system type:
#   - virtio-fs (default)
#   - virtio-9p
shared_fs = "virtio-9p"
..............
................
(2) containerd version 1.5.2
/usr/local/bin/containerd --version
containerd github.com/containerd/containerd<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgithub.com…> 1.5.2-0ubuntu1~20.04.2
/usr/local/bin/containerd --config /etc/containerd/config.toml
# cat /etc/containerd/config.toml
#   Copyright 2018-2020 Docker Inc.
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#       https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.apache…<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.apache…>
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.
[debug]
      level = "debug"
[plugins.cri.containerd]
    no_pivot = false
  [plugins.cri.containerd.runtimes]
    [plugins.cri.containerd.runtimes.runc]
       runtime_type = "io.containerd.runc.v1"
       [plugins.cri.containerd.runtimes.runc.options]
         NoPivotRoot = false
         NoNewKeyring = false
         ShimCgroup = ""
         IoUid = 0
         IoGid = 0
         BinaryName = "runc"
         Root = ""
         CriuPath = ""
         SystemdCgroup = false
    [plugins.cri.containerd.runtimes.kata]
       runtime_type = "io.containerd.kata.v2"
        [plugins.cri.containerd.runtimes.kata.options]
         ConfigPath = "/etc/kata-containers/configuration.toml"
         NoPivotRoot = false
         NoNewKeyring = false
         ShimCgroup = ""
         IoUid = 0
         IoGid = 0
         BinaryName = "/usr/local/bin/kata-runtime"
         Root = ""
         CriuPath = ""
         SystemdCgroup = false
    [plugins.cri.containerd.runtimes.katacli]
       runtime_type = "io.containerd.runc.v1"
       [plugins.cri.containerd.runtimes.katacli.options]
         NoPivotRoot = false
         NoNewKeyring = false
         ShimCgroup = ""
         IoUid = 0
         IoGid = 0
         BinaryName = "/usr/local/bin/kata-runtime"
         Root = ""
         CriuPath = ""
         SystemdCgroup = false
(3) # ctr run --runtime io.containerd.kata.v2 -t --rm docker.io/library/alpine_linux:latest<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocker.io%…> bash
ctr: failed to create shim: Failed to check if grpc server is working: rpc error: code = DeadlineExceeded desc = timed out connecting to vsock 1025935578:1024: unknown
(4) containerd LOG:
DEBU[2021-07-30T07:53:27.915940335Z] stat snapshot                                 key="sha256:7444ea29e45e927abea1f923bf24cac20deaddea603c4bb1c7f2f5819773d453"
DEBU[2021-07-30T07:53:27.921636473Z] prepare snapshot                              key=bash parent="sha256:7444ea29e45e927abea1f923bf24cac20deaddea603c4bb1c7f2f5819773d453"
DEBU[2021-07-30T07:53:27.930411184Z] event published                               ns=default topic=/snapshot/prepare type=containerd.events.SnapshotPrepare
DEBU[2021-07-30T07:53:27.939032985Z] get snapshot mounts                           key=bash
DEBU[2021-07-30T07:53:27.955103540Z] event published                               ns=default topic=/containers/create type=containerd.events.ContainerCreate
DEBU[2021-07-30T07:53:27.962015595Z] get snapshot mounts                           key=bash
DEBU[2021-07-30T07:53:28.039779478Z] garbage collected                             d=2.588852ms
time="2021-07-30T07:53:28.043806590Z" level=debug msg="registering ttrpc server"
time="2021-07-30T07:53:28.043934212Z" level=debug msg="serving api on socket" socket="[inherited from parent]"
time="2021-07-30T07:53:28.043956084Z" level=info msg="starting signal loop" namespace=default path=/run/containerd/io.containerd.runtime.v2.task/default/bash pid=1590317 time="2021-07-30T07:53:28.044534952Z" level=debug msg="converting /run/containerd/io.containerd.runtime.v2.task/default/bash/config.json" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=compatoci time="2021-07-30T07:53:28.049788100Z" level=info msg="loaded configuration" file=/etc/kata-containers/configuration.toml format=TOML name=containerd-shim-v2 pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.049991937Z" level=info msg="IOMMUPlatform is enabled by default." name=containerd-shim-v2 pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.051163250Z" level=info msg="shm-size detected: 67108864" source=virtcontainers subsystem=oci time="2021-07-30T07:53:28.054669132Z" level=info msg="create netns" name=containerd-shim-v2 netns=/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.071513285Z" level=info msg="Enabling SEV guest protection" kernel-params="[{tsc reliable} {no_timer_check } {rcupdate.rcu_expedited 1} {i8042.direct 1} {i8042.dumbkbd 1} {i8042.nopnp 1} {i8042.noaux 1} {noreplace-smp } {reboot k} {console hvc0} {console hvc1} {cryptomgr.notests } {net.ifnames 0} {pci lastbus=0}]" kernel-params-debug="[{debug }]" kernel-params-non-debug="[{quiet }]" machine="{q35  accel=kvm,kernel_irqchip=split}" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemuAmd64 time="2021-07-30T07:53:28.092119026Z" level=info msg="adding volume" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu volume-type=virtio-9p time="2021-07-30T07:53:28.093015598Z" level=info msg="Endpoints found after scan" endpoints="[]" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=network time="2021-07-30T07:53:28.093157978Z" level=info msg="Starting VM" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:28.093562065Z" level=info msg="Adding extra file [0xc000010058]" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:28.093652406Z" level=info msg="launching /usr/local/bin/qemu-system-x86_64 with: [-name sandbox-bash -uuid b1ff0af6-de06-4025-ab63-9aad959900d5 -machine q35,accel=kvm,kernel_irqchip=split,confidential-guest-support=sev -cpu host,pmu=off -qmp unix:/run/vc/vm/bash/qmp.sock,server=on,wait=off -m 2048M,slots=10,maxmem=129746M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2 -device virtio-serial-pci,disable-modern=false,id=serial0 -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/bash/console.sock,server=on,wait=off -device virtio-scsi-pci,id=scsi0,disable-modern=false -object sev-guest,id=sev,cbitpos=51,reduced-phys-bits=5 -drive if=pflash,format=raw,readonly=on,file=/usr/local/share/qemu/OVMF_CODE.fd.kata -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-1025935578,guest-cid=1025935578 -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/bash/shared,security_model=none,multidevs=remap -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-ram,id=dimm1,size=2048M -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug panic=1 nr_cpus=240 scsi_mod.scan=none root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -pidfile /run/vc/vm/bash/pid -D /run/vc/vm/bash/qemu.log -smp 1,cores=1,threads=1,sockets=240,maxcpus=240]"
name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:29.231939187Z" level=info msg="QMP details" name=containerd-shim-v2 pid=1590317 qmp-capabilities=oob qmp-major-version=6 qmp-micro-version=91 qmp-minor-version=0 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:29.232726561Z" level=info msg="scanner return error: read unix @->/run/vc/vm/bash/qmp.sock: use of closed network connection" name=containerd-shim-v2 pid=1590317 sandbox=bash
It looks like your qemu stopped right after being started (T07:53:29.231939187Z vs T07:53:28.093652406Z). I think that the qemu logs would help us.
source=virtcontainers subsystem=qmp time="2021-07-30T07:53:29.232914117Z" level=info msg="VM started" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:29.233044113Z" level=info msg="New client" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=kata_agent url="vsock://1025935578:1024"
time="2021-07-30T07:53:59.475362081Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477063059Z" level=warning msg="The virtiofsd had stopped" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477246988Z" level=info msg="cleanup vm path" dir=/run/vc/vm/bash link=/run/vc/vm/bash name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477447608Z" level=info msg="Network namespace \"/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b\" deleted" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=network time="2021-07-30T07:53:59.478504534Z" level=warning msg="sandbox cgroups path is empty" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:59.478574457Z" level=info msg="cleanup agent" name=containerd-shim-v2 path=/run/kata-containers/shared/sandboxes/bash/shared pid=1590317 sandbox=bash source=virtcontainers subsystem=kata_agent time="2021-07-30T07:53:59.479262623Z" level=warning msg="failed to cleanup netns" error="failed to get netns /var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b: failed to Statfs \"/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b\": no such file or directory" name=containerd-shim-v2 path=/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:59.479464936Z" level=info msg="scanner return error: <nil>" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:59.479683231Z" level=info msg="QMP event channel closed" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu
DEBU[2021-07-30T07:53:59.480447321Z] failed to delete task                         error="rpc error: code = NotFound desc = container does not exist bash: not found" id=bash
INFO[2021-07-30T07:53:59.484752059Z] shim disconnected                             id=bash
WARN[2021-07-30T07:53:59.484860645Z] cleaning up after shim disconnected           id=bash namespace=default
INFO[2021-07-30T07:53:59.484882296Z] cleaning up dead shim
ERRO[2021-07-30T07:53:59.545695272Z] failed to delete                              cmd="/usr/local/bin/containerd-shim-kata-v2 -namespace default -address /run/containerd/containerd.sock -publish-binary /usr/local/bin/containerd -id bash -bundle /run/containerd/io.containerd.runtime.v2.task/default/bash delete" error="exit status 1"
WARN[2021-07-30T07:53:59.545862980Z] failed to clean up after shim disconnected    error="time=\"2021-07-30T07:53:59Z\" level=warning msg=\"failed to cleanup container\" container=bash error=\"open /run/vc/sbs/bash: no such file or directory\" name=containerd-shim-v2 pid=1590410 sandbox=bash source=containerd-kata-shim-v2\nio.containerd.kata.v2: open /run/vc/sbs/bash: no such file or directory\n: exit status 1" id=bash namespace=default
ERRO[2021-07-30T07:53:59.545951698Z] copy shim log                                 error="read /proc/self/fd/15: file already closed"
DEBU[2021-07-30T07:53:59.557980504Z] remove snapshot                               key=bash
DEBU[2021-07-30T07:53:59.560442896Z] event published                               ns=default topic=/snapshot/remove type=containerd.events.SnapshotRemove
DEBU[2021-07-30T07:53:59.563483766Z] event published                               ns=default topic=/containers/delete type=containerd.events.ContainerDelete
DEBU[2021-07-30T07:53:59.610327214Z] schedule snapshotter cleanup                  snapshotter=overlayfs
DEBU[2021-07-30T07:53:59.612570831Z] removed snapshot                              key=default/23/bash snapshotter=overlayfs
DEBU[2021-07-30T07:53:59.613058306Z] snapshot garbage collected                    d=2.633938ms snapshotter=overlayfs
DEBU[2021-07-30T07:53:59.613107950Z] garbage collected                             d=2.265409ms
Any suggestions please.
--Thanking You
--Sandeep
_______________________________________________
kata-dev mailing list
kata-dev(a)lists.katacontainers.io<mailto:kata-dev@lists.katacontainers.io>
https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.kata…<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.kata…>
                    
                  
                  
                          
                            
                            2
                            
                          
                          
                            
                            1
                            
                          
                          
                            
    
                          
                        
                    
                    
                        Hi,
I’ve read through contributing.md[1] and the developer guide[2].
It looks like the recommended path might be for me to setup a linux box and use a packaged install. I typically develop on my mac, though, and I'm wondering if there's some best practices / tricks for working within darwin?
Also, I work on a variety of projects and there is often a way to encapsulate the dev environment for each using vagrant or docker so the system isn't exposed to dependency cross-contamination :)
Has anyone worked out something like that for their Kata work?
[1] https://github.com/kata-containers/community/blob/master/CONTRIBUTING.md
[2] https://github.com/kata-containers/kata-containers/blob/main/docs/Developer…
Thanks,
Dan Middleton
Principal Engineer
Intel <https://www.intel.com/content/www/us/en/diversity/diversity-at-intel.html> 
                    
                  
                  
                          
                            
                            2
                            
                          
                          
                            
                            2
                            
                          
                          
                            
    
                          
                        
                    
                        
                            
                                
                            
                            Re: [kata-dev] , "kata-containers-2.2.0-alpha1" with confidential_guest = true (SEV Guest)
                        
                        
by Gupta, Sandeep 02 Aug '21
                    by Gupta, Sandeep 02 Aug '21
02 Aug '21
                    
                        [AMD Official Use Only]
Hi All,
(1) Using kata-containers-2.2.0-alpha1 -  configuration.toml --
# Copyright (c) 2017-2019 Intel Corporation # # SPDX-License-Identifier: Apache-2.0 #
# XXX: WARNING: this file is auto-generated.
# XXX:
# XXX: Source file: "cli/config/configuration-qemu.toml.in"
# XXX: Project:
# XXX:   Name: Kata Containers
# XXX:   Type: kata
[hypervisor.qemu]
path = "/usr/local/bin/qemu-system-x86_64"
#kernel = "/usr/share/kata-containers/vmlinux.container"
kernel = "/usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container"
#image = "/usr/share/kata-containers/kata-containers.img"
initrd = "/usr/share/kata-containers/kata-containers-initrd.img"
machine_type = "q35"
# Enable confidential guest support.
# Toggling that setting may trigger different hardware features, ranging # from memory encryption to both memory and CPU-state encryption and integrity.
# The Kata Containers runtime dynamically detects the available feature set and # aims at enabling the largest possible one.
# Default false
confidential_guest = true
# List of valid annotation names for the hypervisor # Each member of the list is a regular expression, which is the base name # of the annotation, e.g. "path" for io.katacontainers.config.hypervisor.path"
enable_annotations = []
# List of valid annotations values for the hypervisor # Each member of the list is a path pattern as described by glob(3).
# The default if not set is empty (all annotations rejected.) # Your distribution recommends: ["/usr/bin/qemu-system-x86_64"] valid_hypervisor_paths = ["/usr/local/bin/qemu-system-x86_64"]
# Optional space-separated list of options to pass to the guest kernel.
# For example, use `kernel_params = "vsyscall=emulate"` if you are having # trouble running pre-2.15 glibc.
#
# WARNING: - any parameter specified here will take priority over the default # parameter value of the same name used to start the virtual machine.
# Do not set values here unless you understand the impact of doing so as you # may stop the virtual machine from booting.
# To see the list of default parameters, enable hypervisor debug, create a # container and look for 'default-kernel-parameters' log entries.
#kernel_params = ""
kernel_params = " root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug"
# Path to the firmware.
# If you want that qemu uses the default firmware leave this option empty #firmware = ""
firmware = "/usr/local/share/qemu/OVMF_CODE.fd.kata"
...........
..............
# Shared file system type:
#   - virtio-fs (default)
#   - virtio-9p
shared_fs = "virtio-9p"
..............
................
(2) containerd version 1.5.2
/usr/local/bin/containerd --version
containerd github.com/containerd/containerd 1.5.2-0ubuntu1~20.04.2
/usr/local/bin/containerd --config /etc/containerd/config.toml
# cat /etc/containerd/config.toml
#   Copyright 2018-2020 Docker Inc.
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#       http://www.apache.org/licenses/LICENSE-2.0
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.
[debug]
        level = "debug"
[plugins.cri.containerd]
      no_pivot = false
    [plugins.cri.containerd.runtimes]
      [plugins.cri.containerd.runtimes.runc]
         runtime_type = "io.containerd.runc.v1"
         [plugins.cri.containerd.runtimes.runc.options]
           NoPivotRoot = false
           NoNewKeyring = false
           ShimCgroup = ""
           IoUid = 0
           IoGid = 0
           BinaryName = "runc"
           Root = ""
           CriuPath = ""
           SystemdCgroup = false
      [plugins.cri.containerd.runtimes.kata]
         runtime_type = "io.containerd.kata.v2"
          [plugins.cri.containerd.runtimes.kata.options]
           ConfigPath = "/etc/kata-containers/configuration.toml"
           NoPivotRoot = false
           NoNewKeyring = false
           ShimCgroup = ""
           IoUid = 0
           IoGid = 0
           BinaryName = "/usr/local/bin/kata-runtime"
           Root = ""
           CriuPath = ""
           SystemdCgroup = false
      [plugins.cri.containerd.runtimes.katacli]
         runtime_type = "io.containerd.runc.v1"
         [plugins.cri.containerd.runtimes.katacli.options]
           NoPivotRoot = false
           NoNewKeyring = false
           ShimCgroup = ""
           IoUid = 0
           IoGid = 0
           BinaryName = "/usr/local/bin/kata-runtime"
           Root = ""
           CriuPath = ""
           SystemdCgroup = false
(3) # ctr run --runtime io.containerd.kata.v2 -t --rm docker.io/library/alpine_linux:latest bash
ctr: failed to create shim: Failed to check if grpc server is working: rpc error: code = DeadlineExceeded desc = timed out connecting to vsock 1025935578:1024: unknown
(4) containerd LOG:
DEBU[2021-07-30T07:53:27.915940335Z] stat snapshot                                 key="sha256:7444ea29e45e927abea1f923bf24cac20deaddea603c4bb1c7f2f5819773d453"
DEBU[2021-07-30T07:53:27.921636473Z] prepare snapshot                              key=bash parent="sha256:7444ea29e45e927abea1f923bf24cac20deaddea603c4bb1c7f2f5819773d453"
DEBU[2021-07-30T07:53:27.930411184Z] event published                               ns=default topic=/snapshot/prepare type=containerd.events.SnapshotPrepare
DEBU[2021-07-30T07:53:27.939032985Z] get snapshot mounts                           key=bash
DEBU[2021-07-30T07:53:27.955103540Z] event published                               ns=default topic=/containers/create type=containerd.events.ContainerCreate
DEBU[2021-07-30T07:53:27.962015595Z] get snapshot mounts                           key=bash
DEBU[2021-07-30T07:53:28.039779478Z] garbage collected                             d=2.588852ms
time="2021-07-30T07:53:28.043806590Z" level=debug msg="registering ttrpc server"
time="2021-07-30T07:53:28.043934212Z" level=debug msg="serving api on socket" socket="[inherited from parent]"
time="2021-07-30T07:53:28.043956084Z" level=info msg="starting signal loop" namespace=default path=/run/containerd/io.containerd.runtime.v2.task/default/bash pid=1590317 time="2021-07-30T07:53:28.044534952Z" level=debug msg="converting /run/containerd/io.containerd.runtime.v2.task/default/bash/config.json" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=compatoci time="2021-07-30T07:53:28.049788100Z" level=info msg="loaded configuration" file=/etc/kata-containers/configuration.toml format=TOML name=containerd-shim-v2 pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.049991937Z" level=info msg="IOMMUPlatform is enabled by default." name=containerd-shim-v2 pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.051163250Z" level=info msg="shm-size detected: 67108864" source=virtcontainers subsystem=oci time="2021-07-30T07:53:28.054669132Z" level=info msg="create netns" name=containerd-shim-v2 netns=/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:28.071513285Z" level=info msg="Enabling SEV guest protection" kernel-params="[{tsc reliable} {no_timer_check } {rcupdate.rcu_expedited 1} {i8042.direct 1} {i8042.dumbkbd 1} {i8042.nopnp 1} {i8042.noaux 1} {noreplace-smp } {reboot k} {console hvc0} {console hvc1} {cryptomgr.notests } {net.ifnames 0} {pci lastbus=0}]" kernel-params-debug="[{debug }]" kernel-params-non-debug="[{quiet }]" machine="{q35  accel=kvm,kernel_irqchip=split}" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemuAmd64 time="2021-07-30T07:53:28.092119026Z" level=info msg="adding volume" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu volume-type=virtio-9p time="2021-07-30T07:53:28.093015598Z" level=info msg="Endpoints found after scan" endpoints="[]" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=network time="2021-07-30T07:53:28.093157978Z" level=info msg="Starting VM" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:28.093562065Z" level=info msg="Adding extra file [0xc000010058]" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:28.093652406Z" level=info msg="launching /usr/local/bin/qemu-system-x86_64 with: [-name sandbox-bash -uuid b1ff0af6-de06-4025-ab63-9aad959900d5 -machine q35,accel=kvm,kernel_irqchip=split,confidential-guest-support=sev -cpu host,pmu=off -qmp unix:/run/vc/vm/bash/qmp.sock,server=on,wait=off -m 2048M,slots=10,maxmem=129746M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2 -device virtio-serial-pci,disable-modern=false,id=serial0 -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/bash/console.sock,server=on,wait=off -device virtio-scsi-pci,id=scsi0,disable-modern=false -object sev-guest,id=sev,cbitpos=51,reduced-phys-bits=5 -drive if=pflash,format=raw,readonly=on,file=/usr/local/share/qemu/OVMF_CODE.fd.kata -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0 -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-1025935578,guest-cid=1025935578 -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/bash/shared,security_model=none,multidevs=remap -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-ram,id=dimm1,size=2048M -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinux-sev-es-5.9-v1.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug panic=1 nr_cpus=240 scsi_mod.scan=none root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -pidfile /run/vc/vm/bash/pid -D /run/vc/vm/bash/qemu.log -smp 1,cores=1,threads=1,sockets=240,maxcpus=240]" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:29.231939187Z" level=info msg="QMP details" name=containerd-shim-v2 pid=1590317 qmp-capabilities=oob qmp-major-version=6 qmp-micro-version=91 qmp-minor-version=0 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:29.232726561Z" level=info msg="scanner return error: read unix @->/run/vc/vm/bash/qmp.sock: use of closed network connection" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:29.232914117Z" level=info msg="VM started" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:29.233044113Z" level=info msg="New client" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=kata_agent url="vsock://1025935578:1024"
time="2021-07-30T07:53:59.475362081Z" level=info msg="Stopping Sandbox" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477063059Z" level=warning msg="The virtiofsd had stopped" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477246988Z" level=info msg="cleanup vm path" dir=/run/vc/vm/bash link=/run/vc/vm/bash name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu time="2021-07-30T07:53:59.477447608Z" level=info msg="Network namespace \"/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b\" deleted" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=network time="2021-07-30T07:53:59.478504534Z" level=warning msg="sandbox cgroups path is empty" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=sandbox time="2021-07-30T07:53:59.478574457Z" level=info msg="cleanup agent" name=containerd-shim-v2 path=/run/kata-containers/shared/sandboxes/bash/shared pid=1590317 sandbox=bash source=virtcontainers subsystem=kata_agent time="2021-07-30T07:53:59.479262623Z" level=warning msg="failed to cleanup netns" error="failed to get netns /var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b: failed to Statfs \"/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b\": no such file or directory" name=containerd-shim-v2 path=/var/run/netns/cnitest-6747183b-4b3c-3ec6-81c4-d517d32b120b pid=1590317 sandbox=bash source=katautils time="2021-07-30T07:53:59.479464936Z" level=info msg="scanner return error: <nil>" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qmp time="2021-07-30T07:53:59.479683231Z" level=info msg="QMP event channel closed" name=containerd-shim-v2 pid=1590317 sandbox=bash source=virtcontainers subsystem=qemu
DEBU[2021-07-30T07:53:59.480447321Z] failed to delete task                         error="rpc error: code = NotFound desc = container does not exist bash: not found" id=bash
INFO[2021-07-30T07:53:59.484752059Z] shim disconnected                             id=bash
WARN[2021-07-30T07:53:59.484860645Z] cleaning up after shim disconnected           id=bash namespace=default
INFO[2021-07-30T07:53:59.484882296Z] cleaning up dead shim
ERRO[2021-07-30T07:53:59.545695272Z] failed to delete                              cmd="/usr/local/bin/containerd-shim-kata-v2 -namespace default -address /run/containerd/containerd.sock -publish-binary /usr/local/bin/containerd -id bash -bundle /run/containerd/io.containerd.runtime.v2.task/default/bash delete" error="exit status 1"
WARN[2021-07-30T07:53:59.545862980Z] failed to clean up after shim disconnected    error="time=\"2021-07-30T07:53:59Z\" level=warning msg=\"failed to cleanup container\" container=bash error=\"open /run/vc/sbs/bash: no such file or directory\" name=containerd-shim-v2 pid=1590410 sandbox=bash source=containerd-kata-shim-v2\nio.containerd.kata.v2: open /run/vc/sbs/bash: no such file or directory\n: exit status 1" id=bash namespace=default
ERRO[2021-07-30T07:53:59.545951698Z] copy shim log                                 error="read /proc/self/fd/15: file already closed"
DEBU[2021-07-30T07:53:59.557980504Z] remove snapshot                               key=bash
DEBU[2021-07-30T07:53:59.560442896Z] event published                               ns=default topic=/snapshot/remove type=containerd.events.SnapshotRemove
DEBU[2021-07-30T07:53:59.563483766Z] event published                               ns=default topic=/containers/delete type=containerd.events.ContainerDelete
DEBU[2021-07-30T07:53:59.610327214Z] schedule snapshotter cleanup                  snapshotter=overlayfs
DEBU[2021-07-30T07:53:59.612570831Z] removed snapshot                              key=default/23/bash snapshotter=overlayfs
DEBU[2021-07-30T07:53:59.613058306Z] snapshot garbage collected                    d=2.633938ms snapshotter=overlayfs
DEBU[2021-07-30T07:53:59.613107950Z] garbage collected                             d=2.265409ms
Any suggestions please.
--Thanking You
--Sandeep
                    
                  
                  
                          
                            
                            2
                            
                          
                          
                            
                            3
                            
                          
                          
                            
    
                          
                        
                    
                    
                        The Diversity & Inclusion WG invites members of all OIF projects to attend
our next meeting Monday Auguat 2nd, at 17:00 UTC in the
#openinfra-diversity channel
on OFTC. The agenda can be found at https://etherpad.openstack.org/p/
diversity-wg-agenda.
 Please feel free to add any topics you wish to discuss at the meeting.
Thanks,
Amy (apotz)
                    
                  
                  
                          
                            
                            1
                            
                          
                          
                            
                            0