Open Stack

Thanks! This is good news, thanks for the heads up. +1 to improve  documentation, this will help new reviewers. 

-
Carlos 


-----Original Message-----
From: Whaley, Graham [mailto:graham.whaley at intel.com] 
Sent: Tuesday, September 11, 2018 10:44 AM
To: kata-dev at lists.katacontainers.io
Subject: [kata-dev] metrics CI tracking proxy/agent/shim PRs

Hi all,
 Thanks to the generosity of Packet.net and the CNCF, we have gained some hardware resource to enable metrics CI tracking.
 I've configured and hooked up an instance to track PRs on the proxy/agent/shim repos. You may/will start to notice an extra 'jenkins-metrics-ubuntu-16-04' item appear in the CI checks list for those repos. This should be *non-blocking* whilst we settle the system - but, be aware, even non-blocking CI failure items stop the big merge button going green - it will be grey, and still allow you to merge, if a non-blocking item fails.

As you may now need to check metrics CI failures, the 'quick' route is to go to the Jenkins console log output of the relevant build (via the link off the github status), and scroll to near the bottom of the console. You should find (given the CI built and ran OK), a summary table that looks something like this:

Report Summary:
+-----+----------------------+-------+--------+--------+-------+--------+--------+-------+------+-----+
| P/F |         NAME         |  FLR  |  MEAN  |  CEIL  |  GAP  |  MIN   |  MAX   |  RNG  | COV  | ITS |
+-----+----------------------+-------+--------+--------+-------+--------+--------+-------+------+-----+
| P   | boot-times           | 95.0% | 98.4%  | 105.0% | 10.0% | 94.1%  | 103.7% | 10.2% | 2.3% |  20 |
| P   | memory-footprint     | 95.0% | 100.4% | 105.0% | 10.0% | 100.4% | 100.4% | 0.0%  | 0.0% |   1 |
| P   | memory-footprint-ksm | 95.0% | 97.0%  | 105.0% | 10.0% | 97.0%  | 97.0%  | 0.0%  | 0.0% |   1 |
+-----+----------------------+-------+--------+--------+-------+--------+--------+-------+------+-----+
Fails: 0, Passes 3

Check the P/F column - anything with a '*F*' will be a failure case, and cause the CI to fail. And, yes, improving the documentation about how to set up, configure, and read the results is on 'the list' :-) You are basically looking for the MEAN to fall between the FLR(floor) and CEIL(celing) of expected results.

Once we are happy the checks are stable we will enable them across the other repos (runtime, tests etc.), and make them blocking.

 Graham


---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.


_______________________________________________
kata-dev mailing list
kata-dev at lists.katacontainers.io
http://lists.katacontainers.io/cgi-bin/mailman/listinfo/kata-dev