Hi, Kata-team, To address the objective of how to shape up Kata performance metrics testing and execute it well last week, and we happen to have an open source hackathon event in Shenzhen this week, gathering most of Kata developers/user delegates together, here is a brief note of the initial discussion and also proposal of the meeting set up as a follow-up. If there is any comments or feedback or suggestions, please kindly share here. Two main areas are majorly addressed on the discussion, 1. Performance metrics testing environment 2. Performance metrics testing requirement And some of the initial conclusion in these two areas are below, Base Guideline 1. To measure performance of each major/minor change and prevent regression, light weight testing for each PR, full functional test on daily or weekly basis. 2. To measure Kata performance, comparing with default container runtime (e.g. runc). Testing Environment 3. Physical server nodes required, virtualized environment or nested virtualization is not accepted (as it is dynamically allocated and cannot be used as baseline) 4. Several testing environment options, Openlab, Alibaba server. https://openlabtesting.org/ 5. It may not require large scale hardware resource environment, typically like 2 server nodes should be efficient. Testing Requirement 6. Base metrics testing tools are: https://github.com/kata-containers/tests/tree/master/metrics 7. Network testing - latency testing, pps testing, throughput testing are critical. 8. Also needs to measure CPU utilization when reaching best network performance 9. Performance comparing with runc, 10% overhead tolerance maximally. 10. I/O latency, throughput, CPU utilization when reaching best I/O performance 11. Typical container workload performance measurement, one proposal is to fetch top 10 list from famous docker registry, and to measure how these workload performed in Kata containers. For example, how long it will take to complete a tensorflow job or finish a kernel building. To further follow up the discussion, and push the effort. A biweekly meeting was discussed as well to be set up for people to join and share progress/feedback. Please kindly share feedback if biweekly works or not for you. It could be a zoom call or IRC meeting. Thanks, Horace