Open Stack

Hi, Kata-team,

 

To address the objective of how to shape up Kata performance metrics testing
and execute it well last week, and we happen to have an open source
hackathon event in Shenzhen this week, gathering most of Kata
developers/user delegates together, here is a brief note of the initial
discussion and also proposal of the meeting set up as a follow-up. If there
is any comments or feedback or suggestions, please kindly share here.

 

Two main areas are majorly addressed on the discussion,

1.      Performance metrics testing environment

2.      Performance metrics testing requirement

 

And some of the initial conclusion in these two areas are below,

Base Guideline

1.      To measure performance of each major/minor change and prevent
regression, light weight testing for each PR, full functional test on daily
or weekly basis.

2.      To measure Kata performance, comparing with default container
runtime (e.g. runc).

Testing Environment

3.      Physical server nodes required, virtualized environment or nested
virtualization is not accepted (as it is dynamically allocated and cannot be
used as baseline)

4.      Several testing environment options, Openlab, Alibaba server.
https://openlabtesting.org/

5.      It may not require large scale hardware resource environment,
typically like 2 server nodes should be efficient.

Testing Requirement

6.      Base metrics testing tools are:
https://github.com/kata-containers/tests/tree/master/metrics

7.      Network testing - latency testing, pps testing, throughput testing
are critical.

8.      Also needs to measure CPU utilization when reaching best network
performance

9.      Performance comparing with runc, 10% overhead tolerance maximally.

10.   I/O latency, throughput, CPU utilization when reaching best I/O
performance

11.   Typical container workload performance measurement, one proposal is to
fetch top 10 list from famous docker registry, and to measure how these
workload performed in Kata containers. For example, how long it will take to
complete a tensorflow job or finish a kernel building.

 

To further follow up the discussion, and push the effort. A biweekly meeting
was discussed as well to be set up for people to join and share
progress/feedback. Please kindly share feedback if biweekly works or not for
you. It could be a zoom call or IRC meeting.

 

 

Thanks,

Horace

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.katacontainers.io/pipermail/kata-dev/attachments/20190419/e7f0e939/attachment-0001.html>