<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 8 Jan 2021, at 21:16, Gupta, Sandeep <<a href="mailto:Sandeep.Gupta@amd.com" class="">Sandeep.Gupta@amd.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">[AMD Official Use Only - Internal Distribution Only]</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Thank you Christophe,</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Yes - We change .../vendor/src/</span><a href="http://github.com/intel/govmm/qemu/qemu.go" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">github.com/intel/govmm/qemu/qemu.go</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">, "appendBios()"</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">func (config *Config) appendBios() {</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> if config.Bios != "" {</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> if config.Knobs.MemEncrypt == true {</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> config.qemuParams = append(config.qemuParams, "-drive")</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> config.qemuParams = append(config.qemuParams, fmt.Sprintf("if=pflash,format=raw,unit=0,file=%s,readonly", config.Bios))</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> } else {</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> config.qemuParams = append(config.qemuParams, "-bios")</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> config.qemuParams = append(config.qemuParams, config.Bios)</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> }</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class=""> }</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">}</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""></div></blockquote><div><br class=""></div><div>So I see two problems with that change:</div><div><br class=""></div><div>1/ I think it depends on using OVMF, not on encrypted memory.</div><div><br class=""></div><div>2/ You need the VARS file as discussed earlier.</div><div><br class=""></div><div>I believe that a more robust solution would be:</div><div><br class=""></div><div>1/ Add an OVMF option alongside "firmware", as well as OVMF_VARS</div><div><br class=""></div><div>2/ When OVMF is specified, create a copy of the VARS file, and pass two -drive options, one for OVMF_CODE and one for the copied OVMF_VARS.</div><div><br class=""></div><div>3/ When MemEncrypt is true, check that you ahve an OVMF given and no bios.</div><div><br class=""></div><div><br class=""></div><blockquote type="cite" class=""><div class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">But then "containerd" does not use this method ?</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">--Thanking You</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">--Sandeep</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">-----Original Message-----</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">From: Christophe de Dinechin <</span><a href="mailto:cdupontd@redhat.com" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">cdupontd@redhat.com</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">><span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Sent: Thursday, January 7, 2021 12:43 AM</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">To: Gupta, Sandeep <</span><a href="mailto:Sandeep.Gupta@amd.com" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">Sandeep.Gupta@amd.com</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Cc: Fabiano Fidêncio <</span><a href="mailto:fidencio@redhat.com" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">fidencio@redhat.com</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">>;<span class="Apple-converted-space"> </span></span><a href="mailto:kata-dev@lists.katacontainers.io" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">kata-dev@lists.katacontainers.io</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">; Hollingsworth, Brent <</span><a href="mailto:brent.hollingsworth@amd.com" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">brent.hollingsworth@amd.com</a><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime, qemu OVMF</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Importance: High</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">[CAUTION: External Email]</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">On 7 Jan 2021, at 09:01, Gupta, Sandeep <<a href="mailto:Sandeep.Gupta@amd.com" class="">Sandeep.Gupta@amd.com</a>> wrote:<br class=""><br class="">[AMD Official Use Only - Internal Distribution Only]<br class=""><br class="">Hi All,<br class=""><br class="">I may have mixed up my working and not working cases - sorry about<span class="Apple-converted-space"> </span><br class="">this, I have finally isolated my working and not-working cases, (for sure) as follows :<br class=""></blockquote><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">This seems consistent with what we deduced from the earlier exchanges.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">In short:</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class=""> . Not working :<br class=""><br class="">msg="launching /usr/local/bin/qemu-system-x86_64 with: [-name sandbox-csh -uuid 8b5369f5-1439-48d7-9b67-a32f6ef8ff7f -machine q35,accel=kvm,kernel_irqchip -cpu host, -qmp unix:/run/vc/vm/csh/qmp.sock,server,nowait -m 2048M,slots=10,maxmem=129755M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2,romfile= -device virtio-serial-pci,disable-modern=false,id=serial0,romfile= -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/csh/console.sock,server,nowait -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0,romfile= -device virtserialport,chardev=charch0,id=channel0,name=agent.channel.0 -chardev socket,id=charch0,path=/run/vc/vm/csh/kata.sock,server,nowait -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared,romfile= -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/csh/shared,security_model=none,multidevs=remap -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-ram,id=dimm1,size=2048M -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinuz-sev-es-5.6-v3.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 iommu=off debug panic=1 nr_cpus=240 agent.use_vsock=false root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -bios /usr/local/share/qemu/OVMF_CODE.fd.kata -pidfile /run/vc/vm/csh/pid -D /run/vc/vm/csh/qemu.log -smp 1,cores=1,threads=1,sockets=240,maxcpus=240]"<br class=""></blockquote><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">So this one uses the -bios option.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">-bios /usr/local/share/qemu/OVMF_CODE.fd.kata</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">I believe you told us this is what 2.0 does, right?</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class=""><br class=""> . Working :<br class=""><br class="">/usr/local/bin/qemu-system-x86_64 -name<span class="Apple-converted-space"> </span><br class="">sandbox-91d9f660fe1338d724c0ccd5c755a618d44100e697c82f8753d1051d2fdb30<br class="">c1 -uuid e7888d8f-2e65-43d5-bc5a-3a3e853b3e13 -machine<span class="Apple-converted-space"> </span><br class="">q35,accel=kvm,kernel_irqchip,nvdimm -cpu host -qmp<span class="Apple-converted-space"> </span><br class="">unix:/run/vc/sbs/91d9f660fe1338d724c0ccd5c755a618d44100e697c82f8753d10<br class="">51d2fdb30c1/qmp.sock,server,nowait -m 512M,slots=2,maxmem=258779M<span class="Apple-converted-space"> </span><br class="">-device<span class="Apple-converted-space"> </span><br class="">pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2<span class="Apple-converted-space"> </span><br class="">-device<span class="Apple-converted-space"> </span><br class="">virtio-serial-pci,disable-legacy=on,id=serial0,iommu_platform=true<span class="Apple-converted-space"> </span><br class="">-device virtconsole,chardev=charconsole0,id=console0 -chardev<span class="Apple-converted-space"> </span><br class="">socket,id=charconsole0,path=/run/vc/sbs/91d9f660fe1338d724c0ccd5c755a6<br class="">18d44100e697c82f8753d1051d2fdb30c1/console.sock,server,nowait -object<span class="Apple-converted-space"> </span><br class="">sev-guest,id=sev0,cbitpos=51,reduced-phys-bits=5 -machine<span class="Apple-converted-space"> </span><br class="">memory-encryption=sev0 -device<span class="Apple-converted-space"> </span><br class="">virtserialport,chardev=charch0,id=channel0,name=agent.channel.0<span class="Apple-converted-space"> </span><br class="">-chardev<span class="Apple-converted-space"> </span><br class="">socket,id=charch0,path=/run/vc/sbs/91d9f660fe1338d724c0ccd5c755a618d44<br class="">100e697c82f8753d1051d2fdb30c1/kata.sock,server,nowait -device<span class="Apple-converted-space"> </span><br class="">virtio-9p-pci,disable-legacy=on,iommu_platform=true,fsdev=extra-9p-kat<br class="">aShared,mount_tag=kataShared -fsdev<span class="Apple-converted-space"> </span><br class="">local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxe<br class="">s/91d9f660fe1338d724c0ccd5c755a618d44100e697c82f8753d1051d2fdb30c1,sec<br class="">urity_model=none -netdev<span class="Apple-converted-space"> </span><br class="">tap,id=network-0,vhost=on,vhostfds=3:4:5:6:7:8:9:10,fds=11:12:13:14:15<br class="">:16:17:18 -device<span class="Apple-converted-space"> </span><br class="">driver=virtio-net-pci,netdev=network-0,mac=02:42:ac:11:00:02,disable-l<br class="">egacy=on,iommu_platform=true,mq=on,vectors=18 -rtc<span class="Apple-converted-space"> </span><br class="">base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -vga<span class="Apple-converted-space"> </span><br class="">none -no-user-config -nodefaults -nographic -daemonize -kernel<span class="Apple-converted-space"> </span><br class="">/usr/share/kata-containers/vmlinuz-sev-es-5.6-v3.container -initrd<span class="Apple-converted-space"> </span><br class="">/usr/share/kata-containers/kata-containers-initrd.img -append<span class="Apple-converted-space"> </span><br class="">tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1<span class="Apple-converted-space"> </span><br class="">i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k<span class="Apple-converted-space"> </span><br class="">console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0<span class="Apple-converted-space"> </span><br class="">pci=lastbus=0 debug panic=1 initcall_debug nr_cpus=240<span class="Apple-converted-space"> </span><br class="">ip=::::::91d9f660fe1338d724c0ccd5c755a618d44100e697c82f8753d1051d2fdb3<br class="">0c1::off:: root=/dev/vda1 rootflags=data=ordered,errors=remount-ro<span class="Apple-converted-space"> </span><br class="">agent.log=debug initcall_debug -drive<span class="Apple-converted-space"> </span><br class="">if=pflash,format=raw,unit=0,file=/usr/local/share/qemu/OVMF_CODE.fd.ka<br class="">ta,readonly -smp<br class="">1,cores=1,threads=1,sockets=1,maxcpus=240<br class=""><br class=""></blockquote><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">This one uses -drive</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">-drive if=pflash,format=raw,unit=0,file=/usr/local/share/qemu/OVMF_CODE.fd.kata,readonly</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">I believe this was your 1.x demo, right?</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Now, this -drive is good, but it is unexpected, because I see nowhere in the normal Kata code that would emit that. This suggests you use a version I don't know about, or maybe that there is for example a patch that was sent to a release but dropped from the 2.0 branch.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">Further, please note that Laszlo Ersek seems to imply in a parallel thread that -drive is no longer the right way to do it either. We may need to implement qemu-version dependent options.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><blockquote type="cite" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class="">--Best Regards,<br class="">--Sandeep Gupta<br class=""><br class=""><br class="">From: Christophe de Dinechin <<a href="mailto:cdupontd@redhat.com" class="">cdupontd@redhat.com</a>><br class="">Sent: Wednesday, January 6, 2021 7:00 AM<br class="">To: Gupta, Sandeep <<a href="mailto:Sandeep.Gupta@amd.com" class="">Sandeep.Gupta@amd.com</a>><br class="">Cc: Fabiano Fidêncio <<a href="mailto:fidencio@redhat.com" class="">fidencio@redhat.com</a>>;<span class="Apple-converted-space"> </span><br class=""><a href="mailto:kata-dev@lists.katacontainers.io" class="">kata-dev@lists.katacontainers.io</a>; Hollingsworth, Brent<span class="Apple-converted-space"> </span><br class=""><<a href="mailto:brent.hollingsworth@amd.com" class="">brent.hollingsworth@amd.com</a>><br class="">Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime, qemu OVMF<br class=""><br class="">[CAUTION: External Email]<br class="">Hi Sandeep,<br class=""><br class=""><br class=""><br class=""><br class="">On 6 Jan 2021, at 15:19, Christophe de Dinechin <<a href="mailto:cdupontd@redhat.com" class="">cdupontd@redhat.com</a>> wrote:<br class=""><br class="">Hi Sandeep,<br class=""><br class=""><br class=""><br class="">On 6 Jan 2021, at 09:51, Christophe de Dinechin <<a href="mailto:cdupontd@redhat.com" class="">cdupontd@redhat.com</a>> wrote:<br class=""><br class="">Hi Sandeep,<br class=""><br class=""><br class="">On 5 Jan 2021, at 21:19, Gupta, Sandeep <<a href="mailto:Sandeep.Gupta@amd.com" class="">Sandeep.Gupta@amd.com</a>> wrote:<br class=""><br class="">[AMD Official Use Only - Internal Distribution Only]<br class=""><br class="">Thanks again Christophe,<br class=""><br class="">From my kata-runtime 1.x, setup :<br class=""><br class="">root 2972741 2845780 0 20:13 pts/2 00:00:00 docker run --runtime kata-sev-runtime --cap-add=NET_ADMIN -it apache2-sev<br class=""><br class=""><br class="">root 2018662 2018416 0 15:01 ? 00:00:16 /usr/local/bin/qemu-system-x86_64 -name sandbox-4a28f6d66e95a08be407b10e1fbfd6d0bd41d4048b5823029f02a17b7200adbc -uuid fce8220b-780b-4da6-8f5d-77ace1e06ed6 -machine q35,accel=kvm,kernel_irqchip,nvdimm -cpu host -qmp unix:/run/vc/sbs/4a28f6d66e95a08be407b10e1fbfd6d0bd41d4048b5823029f02a17b7200adbc/qmp.sock,server,nowait -m 512M,slots=2,maxmem=258779M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2 -device virtio-serial-pci,disable-legacy=on,id=serial0,iommu_platform=true -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/sbs/4a28f6d66e95a08be407b10e1fbfd6d0bd41d4048b5823029f02a17b7200adbc/console.sock,server,nowait -object sev-guest,id=sev0,cbitpos=51,reduced-phys-bits=5 -machine memory-encryption=sev0 -device virtserialport,chardev=charch0,id=channel0,name=agent.channel.0 -chardev socket,id=charch0,path=/run/vc/sbs/4a28f6d66e95a08be407b10e1fbfd6d0bd41d4048b5823029f02a17b7200adbc/kata.sock,server,nowait -device virtio-9p-pci,disable-legacy=on,iommu_platform=true,fsdev=extra-9p-kataShared,mount_tag=kataShared -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/4a28f6d66e95a08be407b10e1fbfd6d0bd41d4048b5823029f02a17b7200adbc,security_model=none -netdev tap,id=network-0,vhost=on,vhostfds=3:4:5:6:7:8:9:10,fds=11:12:13:14:15:16:17:18 -device driver=virtio-net-pci,netdev=network-0,mac=02:42:ac:11:00:02,disable-legacy=on,iommu_platform=true,mq=on,vectors=18 -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic -daemonize -kernel /usr/share/kata-containers/vmlinuz-sev-es-5.6-v3.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 iommu=off cryptomgr.notests net.ifnames=0 pci=lastbus=0 debug panic=1 initcall_debug nr_cpus=240 ip=::::::4a28f6d66e95a08be407b10e1fbfd6d0bd41d4048b5823029f02a17b7200adbc::off:: root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -drive if=pflash,format=raw,unit=0,file=/usr/local/share/qemu/OVMF_CODE.fd.kata,readonly -smp 1,cores=1,threads=1,sockets=1,maxcpus=240<br class=""><br class="">As Fabiano pointed out (indirectly), there is no "-bios" option in the<span class="Apple-converted-space"> </span><br class="">command line above. So either<br class=""><br class="">I'm sorry, Fabiano pointed out that I somehow had managed to miss the -drive if=pflash,format=raw,unit=0,file=/usr/local/share/qemu/OVMF_CODE.fd.kata,readonly option. Not enough coffee, I guess.<br class=""><br class="">There is something mysterious about this, as I pointed out in<span class="Apple-converted-space"> </span><a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkata-containers%2Fkata-containers%2Fissues%2F1231%23issuecomment-755333403&data=04%7C01%7Csandeep.gupta%40amd.com" class="">https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkata-containers%2Fkata-containers%2Fissues%2F1231%23issuecomment-755333403&data=04%7C01%7Csandeep.gupta%40amd.com</a>%7C3d884bd4e4904f858e6308d8b2e83de9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637456057849018749%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6Gm0u0gxkCbpYwD6eSnpi99BtnGszhvyWVV%2Bzfs2dMc%3D&reserved=0. Basically, govmm does not seem to emit that option.<br class=""><br class="">func (config *Config) appendBios() {<br class="">if config.Bios != "" {<br class="">config.qemuParams = append(config.qemuParams, "-bios")<span class="Apple-converted-space"> </span><br class="">config.qemuParams = append(config.qemuParams, config.Bios) } }<br class=""><br class="">It apparently has always been like this. Do you remember either patching that file, or maybe finding some other way to pass the OVMF arguments to qemu (like using a qemu wrapper)?<br class=""><br class=""><br class=""><br class="">Disregard what is below. Instead, now we have an observable command-line difference between 1.x and 2.0. My guess is that this explains why you have trouble with 2.0. Even if my own quick experiments today showed that you can boot an OVMF with -bios, I'm not sure you can get SecureBoot that way, because of the missing vars partition.<br class=""><br class="">I've opened<span class="Apple-converted-space"> </span><a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkata-containers%2Fkata-containers%2Fissues%2F1231&data=04%7C01%7Csandeep.gupta%40amd.com" class="">https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkata-containers%2Fkata-containers%2Fissues%2F1231&data=04%7C01%7Csandeep.gupta%40amd.com</a>%7C3d884bd4e4904f858e6308d8b2e83de9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637456057849018749%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tUAcbzpVNvm8iiSX1UuOcLAjwcLTNBNQY1MUEAu2%2Bf0%3D&reserved=0. I will add a comment highlighting the difference.<br class=""><br class=""><br class="">1) You did that run without having the "firmware = " option. If that<span class="Apple-converted-space"> </span><br class="">was the case, please send the command-line with the firmware option<br class=""><br class="">(Hypothesis: maybe you are looking at the wrong configuration file,<span class="Apple-converted-space"> </span><br class="">e.g. you have a local /etc/ override; that might explain why it<span class="Apple-converted-space"> </span><br class="">"worked" with 1.x, although it would then require investigation to<span class="Apple-converted-space"> </span><br class="">understand why SEV was enabled without your firmware)<br class=""><br class="">2) Something else removes the firmware setting or causes qemu to not<span class="Apple-converted-space"> </span><br class="">receive the -bios option<br class=""><br class="">When "firmware" is specified, qemu should be invoked with the -bios option, and the corresponding commit dates back to early 2018, so I don't expect any difference between 1.x and 2.0 there.<br class=""><br class=""><br class="">Cheers,<br class="">Christophe<br class=""><br class=""><br class=""><br class=""><br class="">root 2972841 2970388 0 20:13 ? 00:00:00 containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8 -address /var/run/docker/containerd/containerd.sock -containerd-binary /snap/docker/471/bin/containerd -runtime-root /var/run/docker/runtime-kata-sev-runtime -debug<br class=""><br class=""><br class="">root 2973736 2972841 0 20:13 ? 00:00:00 /usr/libexec/kata-containers/kata-proxy -listen-socket <a href="unix:///run/vc/sbs/54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8/proxy.sock" class="">unix:///run/vc/sbs/54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8/proxy.sock</a> -mux-socket /run/vc/sbs/54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8/kata.sock -log debug -agent-logs-socket /run/vc/sbs/54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8/console.sock<br class=""><br class=""><br class="">root 2973856 2972841 0 20:13 pts/4 00:00:00 /usr/libexec/kata-containers/kata-shim -agent <a href="unix:///run/vc/sbs/54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8/proxy.sock" class="">unix:///run/vc/sbs/54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8/proxy.sock</a> -container 54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8 -exec-id 54db6810ae4ed8ab3e7a378c4e95d73e334797a5a5234978b4f976750f4f64d8 -terminal -log debug<br class=""><br class=""><br class="">--Best Regards,<br class="">--Sandeep Gupta<br class=""><br class=""><br class="">-----Original Message-----<br class="">From: Christophe de Dinechin <<a href="mailto:cdupontd@redhat.com" class="">cdupontd@redhat.com</a>><br class="">Sent: Tuesday, January 5, 2021 11:25 AM<br class="">To: Gupta, Sandeep <<a href="mailto:Sandeep.Gupta@amd.com" class="">Sandeep.Gupta@amd.com</a>><br class="">Cc: Fabiano Fidêncio <<a href="mailto:fidencio@redhat.com" class="">fidencio@redhat.com</a>>;<span class="Apple-converted-space"> </span><br class=""><a href="mailto:kata-dev@lists.katacontainers.io" class="">kata-dev@lists.katacontainers.io</a>; Hollingsworth, Brent<span class="Apple-converted-space"> </span><br class=""><<a href="mailto:brent.hollingsworth@amd.com" class="">brent.hollingsworth@amd.com</a>><br class="">Subject: Re: [kata-dev] Regarding, Kata 2.0.0, kata-runtime, qemu OVMF<br class=""><br class="">[CAUTION: External Email]<br class=""><br class=""><br class="">On 5 Jan 2021, at 17:21, Gupta, Sandeep <<a href="mailto:Sandeep.Gupta@amd.com" class="">Sandeep.Gupta@amd.com</a>> wrote:<br class=""><br class=""><br class="">Hi Developers,<br class="">One more please - for Kata 2.0.0, kata-runtime, qemu OVMF<br class=""><br class="">In /etc/kata-containers/configuration.toml --<br class=""><br class="">======<br class=""># WARNING: - any parameter specified here will take priority over the<span class="Apple-converted-space"> </span><br class="">default # parameter value of the same name used to start the virtual machine.<br class=""># Do not set values here unless you understand the impact of doing so<span class="Apple-converted-space"> </span><br class="">as you # may stop the virtual machine from booting.<br class=""># To see the list of default parameters, enable hypervisor debug,<span class="Apple-converted-space"> </span><br class="">create a # container and look for 'default-kernel-parameters' log entries.<br class="">kernel_params = " root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug"<br class=""><br class=""># Path to the firmware.<br class=""># If you want that qemu uses the default firmware leave this option<span class="Apple-converted-space"> </span><br class="">empty #firmware = "/usr/local/share/qemu/OVMF_CODE.fd.kata"<br class="">firmware = ""<br class="">==========<br class=""><br class="">If I use the default firmware - I get correct results :<br class=""><br class=""># ctr run --runtime io.containerd.run.kata.v2 -t --rm<span class="Apple-converted-space"> </span><br class=""><a href="http://docker.io/library/busybox:latest" class="">docker.io/library/busybox:latest</a> sh / # / # uname -a Linux<span class="Apple-converted-space"> </span><br class="">sandeepg-ubuntu 5.6.0-sev-sev-es-5.6-v3.container #3 SMP Tue Nov 3<br class="">06:38:40 UTC 2020 x86_64 GNU/Linux / #<br class=""><br class=""># ps -eaf | grep -i qemu<br class=""><br class="">root 360867 1 0 15:56 ? 00:00:01 /usr/local/bin/qemu-system-x86_64 -name sandbox-sh -uuid 72820914-dcce-4cea-a01f-3cf9851d136d -machine q35,accel=kvm,kernel_irqchip -cpu host,pmu=off -qmp unix:/run/vc/vm/sh/qmp.sock,server,nowait -m 2048M,slots=10,maxmem=129755M -device pci-bridge,bus=pcie.0,id=pci-bridge-0,chassis_nr=1,shpc=on,addr=2,romfile= -device virtio-serial-pci,disable-modern=false,id=serial0,romfile= -device virtconsole,chardev=charconsole0,id=console0 -chardev socket,id=charconsole0,path=/run/vc/vm/sh/console.sock,server,nowait -device virtio-scsi-pci,id=scsi0,disable-modern=false,romfile= -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0,romfile= -device vhost-vsock-pci,disable-modern=false,vhostfd=3,id=vsock-1189071429,guest-cid=1189071429,romfile= -device virtio-9p-pci,disable-modern=false,fsdev=extra-9p-kataShared,mount_tag=kataShared,romfile= -fsdev local,id=extra-9p-kataShared,path=/run/kata-containers/shared/sandboxes/sh/shared,security_model=none,multidevs=remap -rtc base=utc,driftfix=slew,clock=host -global kvm-pit.lost_tick_policy=discard -vga none -no-user-config -nodefaults -nographic --no-reboot -daemonize -object memory-backend-ram,id=dimm1,size=2048M -numa node,memdev=dimm1 -kernel /usr/share/kata-containers/vmlinuz-sev-es-5.6-v3.container -initrd /usr/share/kata-containers/kata-containers-initrd.img -append tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k console=hvc0 console=hvc1 cryptomgr.notests net.ifnames=0 pci=lastbus=0 quiet panic=1 nr_cpus=240 scsi_mod.scan=none root=/dev/vda1 rootflags=data=ordered,errors=remount-ro agent.log=debug initcall_debug -pidfile /run/vc/vm/sh/pid -smp 1,cores=1,threads=1,sockets=240,maxcpus=240<br class="">root 360966 360958 0 16:05 pts/2 00:00:00 grep --color=auto -i qemu<br class=""><br class=""><br class="">But if I use : firmware = "/usr/local/share/qemu/OVMF_CODE.fd.kata"<br class="">I get Error:<br class=""><br class=""># ctr run --runtime io.containerd.run.kata.v2 -t --rm<span class="Apple-converted-space"> </span><br class=""><a href="http://docker.io/library/busybox:latest" class="">docker.io/library/busybox:latest</a> s<br class="">ctr: failed to launch qemu: exit status 1, error messages from qemu log: qemu: could not load PC BIOS '/usr/local/share/qemu/OVMF_CODE.fd.kata'<br class="">: unknown<br class=""><br class="">Any suggestions please.<br class=""><br class="">I did a quick look. The runtime apparently uses the -bios option when the firmware option is passed, not the -drive option as I thought it would. I'm not entirely sure you can boot OVMF that way. I don't have any idea at the moment why it worked with version 1.x. If you could share with me the 1.x qemu command line option with the firmware option, I'm curious to see what it is.<br class=""><br class="">Thanks<br class="">Christophe<br class=""><br class=""><br class=""><br class="">--Thanking You<br class="">--Sandeep<br class=""><br class="">_______________________________________________<br class="">kata-dev mailing list<br class=""><a href="mailto:kata-dev@lists.katacontainers.io" class="">kata-dev@lists.katacontainers.io</a><br class="">https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists<br class="">.katacontainers.io%2Fcgi-bin%2Fmailman%2Flistinfo%2Fkata-dev&data=<br class="">04%7C01%7Csandeep.gupta%40amd.com%7C5764f6fb89fd4809aafc08d8b1af94df%7<br class="">C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637454714974608155%7CUnkno<br class="">wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL<br class="">CJXVCI6Mn0%3D%7C1000&sdata=U85zx8%2FDKniPytZoOsm%2F7eR%2FNq5e8aHFm<br class="">fJKMKlX%2B84%3D&reserved=0</blockquote></div></blockquote></div><br class=""></body></html>