Open Stack

Hello Kata Community,

My understanding is that initially Prow [1] grew out of a need for more
reliable GitHub integration just handled triggering Jenkins jobs;
eventually though someone realized that most of our tasks ran in docker
containers and that Kubernetes Clusters are a great way to run containers,
so we started scheduling some tests as Kubernetes Pods instead.
Since then we've also realized that we were wasting a lot of time dealing
with Jenkins cluster outages, nodes running out of disk space, etc. and
that we didn't run into those sort of issues very often with Kubernetes.
We now run all of our builds / tests for Kubernetes (10k+ jobs a day [2])
and generally only have outages when say, GitHub is unreachable or we've
broken something in Prow, we rarely actually need to do any maintenance
tasks on the cluster itself.
Prow has also expanded to handle a lot of our other project automation
tasks [3] in a scale-able manner with plugins that handle GitHub events
(via webhook) through the same tooling used for triggering presubmits.

There's definitely some bias in the preference for building on Kubernetes,
but I think this system has generally worked really well and we've been
able to make Prow do a lot for us. A few other communities (OpenShift,
Istio...) are also using Prow and seem to be doing pretty well with it.

[1] https://github.com/kubernetes/test-infra/tree/master/prow
[2] http://velodrome.k8s.io/dashboard/db/bigquery-metrics?orgId=1
[3] https://prow.k8s.io/plugins


On Thu, Mar 1, 2018 at 6:11 PM Tim Allclair <tallclair at google.com> wrote:

> ​Several folks from our Kubernetes test-infra team have agreed to share
> some of their experience with running Kubernetes CI, which was originally
> based on jenkins but now uses a custom built tool called prow
> <https://github.com/kubernetes/test-infra/blob/master/prow/README.md>.
>
> @Kata community - Feel free to respond with questions.
> @Ben, Sen & Cole - Thanks for sharing! Would you like to give a brief
> summary of the issues we had with Jenkins, and why we decided to build prow?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.katacontainers.io/pipermail/kata-dev/attachments/20180302/df21e186/attachment.html>