<div dir="ltr">Hi Julio,<div>I really appreciate your help.</div><div><br></div><div>The "mems" section in the .json file looks like this.</div><div><br></div><div><table class="gmail-highlight gmail-tab-size gmail-js-file-line-container" style="box-sizing:border-box;border-spacing:0px;border-collapse:collapse;color:rgb(36,41,46);font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Helvetica,Arial,sans-serif,"Apple Color Emoji","Segoe UI Emoji";font-size:14px"><tbody style="box-sizing:border-box"><tr style="box-sizing:border-box"></tr><tr style="box-sizing:border-box"><td id="gmail-LC95" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line gmail-annotated" style="box-sizing:border-box;padding:0px 10px;line-height:20px;vertical-align:top;overflow:visible;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;white-space:pre"><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">cpu</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span style="box-sizing:border-box"><span style="box-sizing:border-box">{</span></span></td></tr><tr style="box-sizing:border-box"><td id="gmail-L96" class="gmail-blob-num gmail-js-line-number" style="box-sizing:border-box;padding:0px 10px;width:50px;min-width:50px;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top"></td><td id="gmail-LC96" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line gmail-annotated" style="box-sizing:border-box;padding:0px 10px;line-height:20px;vertical-align:top;overflow:visible;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;white-space:pre"><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">type</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">object</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">,</span></td></tr><tr style="box-sizing:border-box"><td id="gmail-L97" class="gmail-blob-num gmail-js-line-number" style="box-sizing:border-box;padding:0px 10px;width:50px;min-width:50px;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top"></td><td id="gmail-LC97" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line gmail-annotated" style="box-sizing:border-box;padding:0px 10px;line-height:20px;vertical-align:top;overflow:visible;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;white-space:pre"><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">properties</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span style="box-sizing:border-box"><span style="box-sizing:border-box">{</span></span></td></tr><tr style="box-sizing:border-box"><td id="gmail-L98" class="gmail-blob-num gmail-js-line-number" style="box-sizing:border-box;padding:0px 10px;width:50px;min-width:50px;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top"></td><td id="gmail-LC98" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line gmail-annotated" style="box-sizing:border-box;padding:0px 10px;line-height:20px;vertical-align:top;overflow:visible;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;white-space:pre"><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">cpus</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span style="box-sizing:border-box"><span style="box-sizing:border-box">{</span></span></td></tr><tr style="box-sizing:border-box"><td id="gmail-L99" class="gmail-blob-num gmail-js-line-number" style="box-sizing:border-box;padding:0px 10px;width:50px;min-width:50px;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top"></td><td id="gmail-LC99" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line gmail-annotated" style="box-sizing:border-box;padding:0px 10px;line-height:20px;vertical-align:top;overflow:visible;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;white-space:pre"><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">type</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">string</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span></td></tr><tr style="box-sizing:border-box"><td id="gmail-L100" class="gmail-blob-num gmail-js-line-number" style="box-sizing:border-box;padding:0px 10px;width:50px;min-width:50px;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top"></td><td id="gmail-LC100" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line gmail-annotated" style="box-sizing:border-box;padding:0px 10px;line-height:20px;vertical-align:top;overflow:visible;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;white-space:pre"><span style="box-sizing:border-box"> </span><span style="box-sizing:border-box"><span style="box-sizing:border-box">}</span></span><span style="box-sizing:border-box"><span style="box-sizing:border-box">,</span></span></td></tr><tr style="box-sizing:border-box"><td id="gmail-L101" class="gmail-blob-num gmail-js-line-number" style="box-sizing:border-box;padding:0px 10px;width:50px;min-width:50px;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top"></td><td id="gmail-LC101" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line gmail-annotated" style="box-sizing:border-box;padding:0px 10px;line-height:20px;vertical-align:top;overflow:visible;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;white-space:pre"><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">mems</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span style="box-sizing:border-box"><span style="box-sizing:border-box">{</span></span></td></tr><tr style="box-sizing:border-box"><td id="gmail-L102" class="gmail-blob-num gmail-js-line-number" style="box-sizing:border-box;padding:0px 10px;width:50px;min-width:50px;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top"></td><td id="gmail-LC102" class="gmail-blob-code gmail-blob-code-inner gmail-js-file-line gmail-highlighted gmail-annotated" style="box-sizing:border-box;padding:0px 10px;line-height:20px;vertical-align:top;overflow:visible;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;white-space:pre;background-color:rgb(255,251,221)"><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">type</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">string</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span></td></tr><tr style="box-sizing:border-box"><td id="gmail-L103" class="gmail-blob-num gmail-js-line-number" style="box-sizing:border-box;padding:0px 10px;width:50px;min-width:50px;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,monospace;font-size:12px;line-height:20px;color:rgba(27,31,35,0.3);text-align:right;white-space:nowrap;vertical-align:top"></td></tr></tbody></table>I need to change the mems section to this?</div><div><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span style="box-sizing:border-box"><br></span></span></div><div><font face="monospace"><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span style="box-sizing:border-box">mems</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span style="box-sizing:border-box"><span style="box-sizing:border-box">{</span></span><span style="box-sizing:border-box"> </span></font></div><div><font face="monospace"><span style="box-sizing:border-box"></span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box"> "</span></span><span style="box-sizing:border-box">type</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span><span style="box-sizing:border-box">:</span><span style="box-sizing:border-box"> </span><span class="gmail-pl-s" style="box-sizing:border-box;color:rgb(3,47,98)"><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span><span style="box-sizing:border-box">1</span><span class="gmail-pl-pds" style="box-sizing:border-box"><span style="box-sizing:border-box">"</span></span></span> </font></div><div><br></div><div>Thanks,</div><div>Al</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 11, 2019 at 8:53 AM Montes, Julio <<a href="mailto:julio.montes@intel.com">julio.montes@intel.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="text-align:left;direction:ltr">
<div>On Mon, 2019-11-11 at 08:45 -0800, Al Lau wrote:</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div dir="ltr">Hi Julio,
<div>I'm using /usr/bin/ctr (containerd) with kata. Would setting the --cpuset-mems through docker help in my case?</div>
</div>
</blockquote>
<div><br>
</div>
<div>If you have access to the config.json, you can change the number of nodes, </div>
<div>see <a href="https://github.com/opencontainers/runtime-spec/blob/master/schema/config-linux.json#L102" target="_blank">
https://github.com/opencontainers/runtime-spec/blob/master/schema/config-linux.json#L102</a></div>
<div><br>
</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div dir="ltr">
<div><br>
</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Al</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Nov 11, 2019 at 8:42 AM Montes, Julio <<a href="mailto:julio.montes@intel.com" target="_blank">julio.montes@intel.com</a>> wrote:<br>
</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div style="text-align:left;direction:ltr">
<div>Thanks,</div>
<div><br>
</div>
<div>seems like by default the container engine that you're using tries to assign both nodes (0,1) to the container, unfortunately inside the VM (where the container is created)</div>
<div>there is only one node (0), as a temporal workaround, can you change this behavior? (i.e docker run --cpuset-mems="0" ....)</div>
<div><br>
</div>
<div><br>
</div>
<div>On Mon, 2019-11-11 at 08:36 -0800, Al Lau wrote:</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div dir="ltr"><font face="monospace">$ ls -l /sys/devices/system/node<br>
total 0<br>
-r--r--r-- 1 root root 4096 Nov 11 08:32 has_cpu<br>
-r--r--r-- 1 root root 4096 Nov 11 08:32 has_memory<br>
-r--r--r-- 1 root root 4096 Nov 11 08:32 has_normal_memory<br>
drwxr-xr-x 4 root root 0 Nov 8 01:45 node0<br>
drwxr-xr-x 4 root root 0 Nov 8 01:45 node1<br>
-r--r--r-- 1 root root 4096 Nov 11 08:32 online<br>
-r--r--r-- 1 root root 4096 Nov 11 08:32 possible<br>
drwxr-xr-x 2 root root 0 Nov 11 08:32 power<br>
-rw-r--r-- 1 root root 4096 Nov 8 01:45 uevent<br>
$ ls -l /sys/devices/system/node | grep node | wc -l <br>
2</font><br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Nov 11, 2019 at 6:28 AM Montes, Julio <<a href="mailto:julio.montes@intel.com" target="_blank">julio.montes@intel.com</a>> wrote:<br>
</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div style="text-align:left;direction:ltr">
<div>On Mon, 2019-11-11 at 08:23 -0600, Julio Montes wrote:</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div>Hi Al</div>
<div><br>
</div>
<div>how many memory nodes have your system? ( ls /sys/devices/system/ | grep node | wc -l )</div>
</blockquote>
<div><br>
</div>
<div>* ls /sys/devices/system/node | grep node | wc -l </div>
<div><br>
</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div><br>
</div>
<div>can you change the number of cpuset-mems to 0? for example in docker: docker run --cpuset-mems="0" ....</div>
<div><br>
</div>
<div>-</div>
<div>Julio</div>
<div><br>
</div>
<div>On Fri, 2019-11-08 at 14:56 -0800, Al Lau wrote: </div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div dir="ltr">Hi Julio,
<div>Attached is the configuration.toml file. I hope this is what you are looking. for.</div>
<div>Thanks,</div>
<div>Al</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Nov 8, 2019 at 2:11 PM Montes, Julio <<a href="mailto:julio.montes@intel.com" target="_blank">julio.montes@intel.com</a>> wrote:<br>
</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div style="text-align:left;direction:ltr">
<div>I'd like to see the kata configuration and pod configuration (yaml)</div>
<div><br>
</div>
<div>running custom kernel?</div>
<div><br>
</div>
<div>On Fri, 2019-11-08 at 14:05 -0800, Al Lau wrote:</div>
<blockquote type="cite" style="margin:0px 0px 0px 0.8ex;border-left:2px solid rgb(114,159,207);padding-left:1ex">
<div dir="ltr">My environment configured containerd with kata-runtime. I followed the setup instructions on <a href="https://github.com/kata-containers/documentation/blob/master/how-to/containerd-kata.md#configuration" target="_blank">https://github.com/kata-containers/documentation/blob/master/how-to/containerd-kata.md#configuration</a>
<div><br>
</div>
<div>When I attempt to create a kata container, I got this error message. Anyone know how I can resolve this?<br>
</div>
<div>
<div><br>
</div>
<div><font face="monospace">rpc error: code = Unknown desc = failed to create containerd task: Could not run process: container_linux.go:346: starting container process caused "process_linux.go:297: applying cgroup configuration for process caused \"failed
to write 0,1 to cpuset.mems: write /sys/fs/cgroup/cpuset/<a href="http://k8s.io/f1fdd8d5be9030388e4e04dd7d42df9e77a3d46ecb4922ecb2936bf92f74595d/cpuset.mems" target="_blank">k8s.io/f1fdd8d5be9030388e4e04dd7d42df9e77a3d46ecb4922ecb2936bf92f74595d/cpuset.mems</a>:
numerical result out of range\"": unknown </font><br>
</div>
<div><br>
</div>
<div><font face="monospace">kata-runtime : 1.8.2<br>
commit : cdc30296c370414d8ef445abd06378199775d076<br>
OCI specs: 1.0.1-dev<br>
</font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace">containerd <a href="http://github.com/containerd/containerd" target="_blank">
github.com/containerd/containerd</a> 1.2.4 e6b3f5632f50dbc4e9cb6288d911bf4f5e95b18e<br>
</font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace">$ cat /etc/containerd/config.toml</font></div>
<div><font face="monospace">. . .</font></div>
<div><font face="monospace"> [plugins.cri.containerd.runtimes.kata]<br>
runtime_type = "io.containerd.kata.v2"<br>
[plugins.cri.containerd.runtimes.kata.options]<br>
ConfigPath = "/etc/kata-containers/configuration.toml"<br>
</font></div>
<div><font face="monospace"> [plugins.cri.cni]<br>
bin_dir = "/opt/cni/bin"<br>
conf_dir = "/etc/cni/net.d"<br>
conf_template = ""</font><br>
</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Al</div>
<div><br>
</div>
</div>
</div>
<pre>_______________________________________________</pre>
<pre>kata-dev mailing list</pre>
<a href="mailto:kata-dev@lists.katacontainers.io" target="_blank">
<pre>kata-dev@lists.katacontainers.io</pre>
</a><br>
<a href="http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev" target="_blank">
<pre>http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev</pre>
</a><br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote></div>