Open Stack

On 2021-07-07 at 22:46 +02, Jens Freimann <jfreiman at redhat.com> wrote...
> On Wed, Jul 7, 2021 at 12:48 PM Christophe de Dinechin
> <dinechin at redhat.com> wrote:
>> On 2021-07-07 at 01:31 UTC, "Adams, Eric" <eric.adams at intel.com> wrote...
>> > Christophe,
>> > 2) I also observed that Pod Overhead isn't used in the calculation for
>> > hotplugged CPU's.
>>
>> That seems correct. The pod overhead accounts for the overhead "outside" the
>> VM, i.e. virtiofsd, qemu's own memory needs, the extra cost of doing I/Os,
>> etc. So this is additional resources the host needs, not the VM.
>
> This is very counter-intuitive and also doesn't fit the 'Motivation'
> section in the original design[1]. Wouldn't this lead to wrong results for
> the resource quota and scheduler calculations?

In addition to Eric's answer, let me answer your specific question: on the
contrary, I believe that this fits the motivation section and ensures we get
the right results from resource scheduling.

Consider a container where you request 5 CPUs and 17G of memory. We need
5VCPUs to run it, and 17G of memory in the guest. Since the guest starts
with 1CPU and its default 2G, and adding a limit to workaround the #2071
issue, we now get 6 VCPUs and 19G in the guest. So the "VM overhead" is
already covered by the default values and the hotplugging mechanism.

Now, in practice, when we measure things, we observe that the VM actually
consumes something like 110M (minimal kernel) - 300M (standard kernel with
file buffers after an install). In any case, that's way below the 2G
allocated by default. On top of this, we need memory for virtiofs, qemu or
the shum (~50M-80M).

So when we tell kubernetes "how much extra memory do you need", we need to
report that amount that k8s does not know about from the container requests,
which is the 160M-350M I just described above. That is what will allow the
scheduler to do its job properly. Same with CPU.

Hope this makes sense.

>
> [1] https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/688-pod-overhead
>
> regards
> Jens


--
Cheers,
Christophe de Dinechin (IRC c3d)