<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 9 Apr 2021, at 17:38, Eric Ernst <<a href="mailto:eric.g.ernst@gmail.com" class="">eric.g.ernst@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><br class="Apple-interchange-newline"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div dir="ltr" class="gmail_attr">On Fri, Apr 9, 2021 at 8:08 AM Christophe de Dinechin <<a href="mailto:cdupontd@redhat.com" class="">cdupontd@redhat.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On 8 Apr 2021, at 02:07, Eric Ernst <<a href="mailto:eric.g.ernst@gmail.com" target="_blank" class="">eric.g.ernst@gmail.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class=""><div dir="ltr" class=""><br class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 1, 2021 at 7:58 AM Stefan Hajnoczi <<a href="mailto:stefanha@redhat.com" target="_blank" class="">stefanha@redhat.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">On Mon, Mar 29, 2021 at 04:49:31PM -0700, Eric Ernst wrote:<br class="">> I'd like to see if we can create a pattern in upstream Kata Containers to<br class="">> help facilitate directly assigning volumes to the VM via virtio-blk,<br class="">> skipping any mounts on the host, and avoiding needing to use a shared<br class="">> file-system for the particular volume. Some of the benefits in doing this:<br class="">> - we can better isolate the host (no mounted filesystem),<br class=""><br class="">It would be nice to make the mechanism extensible so other types of<br class="">volumes can be attached in the future.<br class=""><br class="">It might be desirable to perform an NFS mount inside the sandbox VM<br class="">instead of on the host, for example. The downside is that the sandbox VM<br class="">needs access to the storage network, but the host kernel is no longer<br class="">involved.<br class=""></blockquote><div class=""><br class=""></div><div class="">Stefan,</div><div class=""><br class=""></div><div class="">Sorry I missed this initial reply. Can you help with identifying how you think we should augment the DiskMountInfo structure to accomodate? </div></div></div></div></div></blockquote><div class=""><br class=""></div><div class=""><div class="">At the last arch committee meeting, Peng Tao presented the work they did</div><div class="">CSI and CNI side to expose a port that the runtime can talk to to get device</div><div class="">information.</div><div class=""><br class=""></div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Ah, I did not see that on the AC agenda. Tao, can you share link to that presentation?</div></div></div></blockquote><div><br class=""></div><div>Sorry, my bad, not the AC meeting, I think it was the use case meeting.</div><div><br class=""></div><div>ETOOMANYMEETINGS</div><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 18px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><div class=""><div class=""></div><div class="">We will keep discussing about it next week, but I think there are very</div><div class="">interesting options for network-attached storage if we can somehow</div><div class="">make sure we can combine CSI + CNI (I'm not discounting the effort to</div><div class="">translate the network topology from host to guest for such cases, what</div><div class="">Sfefan described as "access to the storage network).</div><div class=""><br class=""></div><div class="">Consider an NFS volume, or iSCSI, or something like that. Now,</div><div class="">let's imagine that we also have some "fast" network e.g. a NIC directly</div><div class="">mapped in the guest, a VF, etc. Then it would make sense to do all</div><div class="">the networking from within the guest e.g. for performance reasons.</div><div class=""><br class=""></div><div class="">But the interaction becomes quite complex in that case:</div><div class="">- CSI has the original storage definition</div><div class="">- The runtime needs to detect that this is a network volume</div><div class=""> <span class="Apple-converted-space"> </span>(I believe your DiskMountInfo has what we need, not 100% sure)</div><div class="">- CNI will have the information on where to access that network</div><div class=""> <span class="Apple-converted-space"> </span>(and generally speaking, we know how to expose that in-guest)</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">For what you describe would you then expect a second network interface being added to the sandbox, or are you reusing the existing? While this may be extra work needed on the infra operator, I'm not sure if this would necessarily be tied to the direct-assigned part, but more work if you are intending to do direct assigned (ie, chained CNI). AFAICT, we'd still want to communicate the volume information (where source would now be a network location) from CSI to the runtime?</div><div class=""><br class=""></div><div class="">Still open in my mind:</div><div class="">- how are credentials handled for being able to access the remote storage (ie, mounted into the rootfs at specific location that agent is aware of, ala [1]). Or, other suggestions?</div><div class="">- I assume that the remote URL:path can still be communicated via Device field in proposed DirectMount structure (perhaps Device should be renamed to Source).</div><div class=""> </div><div class="">Thanks,</div><div class=""><br class=""></div><div class="">[1] - <a href="https://github.com/kata-containers/kata-containers/blob/main/src/runtime/cli/config/configuration-clh.toml.in#L255-L259" class="">https://github.com/kata-containers/kata-containers/blob/main/src/runtime/cli/config/configuration-clh.toml.in#L255-L259</a></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""> <br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="word-wrap: break-word; line-break: after-white-space;" class=""><div class=""><div class=""><div class="">- That turns into an in-guest mount, e.g. NFS</div><div class="">- … plus some routing to make sure we can reach that storage</div><div class=""><br class=""></div><div class="">I am not sure this is exactly what Stefan had in mind, but that is what</div><div class="">popped in my own brain when Peng Tao showed up his slides.</div><div class=""><br class=""></div><div class="">That does not directly answer your question. I believe that as you</div><div class="">propose it, we'd have all the information we need, since we have</div><div class="">an FsType (where presumably we could have "nfs") and a volume</div><div class="">type (where we presumably could see "iscsi"). I have not considered</div><div class="">all the combinations, but at first sight, it looks sane as is ;-)</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Thanks</div><div class="">Christophe</div></div></div></div></blockquote></div></div></blockquote></div><br class=""></body></html>