Open Stack

On Fri, Jan 15, 2021 at 10:49:34AM +0100, Christophe de Dinechin wrote:
> 
> 
> > On 15 Jan 2021, at 04:29, David Gibson <david at gibson.dropbear.id.au> wrote:
> > 
> > Hi all,
> > 
> > I'm a pretty recently started Kata contributor.  One of many things I
> > found frustrating when I started working on Kata was the way the code
> > was split into a whole batch of repositories.  I'm relieved that
> > they're now merged together for Kata 2.x.
> > 
> > However, the tests repository is still separate, and AIUI there are no
> > plans to integrate it.  I still find that really annoying, so I want
> > to make the case for integrating it into the kata-containers
> > repository as well.
> > 
> > 1. It removes delays fixing test bugs
> > 
> > On several occasions when working on code changes, I've hit CI
> > failures due to what turn out to be bugs in the tests code.  These
> > have often been problems with static checks essentially unrelated to
> > what I'm actually trying to accomplish.
> > 
> > To deal with this, I've had to file a separate issue and PR against
> > the tests repo, wait for that to be tested and reviewed, and only then
> > can I resume work on what I was actually trying to do.  Especially
> > since I'm a different timezone from most of the Kata team, that can
> > easily be several days of delay.
> > 
> > If the tests were integrated, I could include a fix for the test as a
> > commit in the same PR, it would automatically fix the CI for that PR
> > and could be reviewed with the same batch.
> > 
> > 2. Allows tests to be commited along with new features
> > 
> > Obviously it's desirable for new features to come with tests for those
> > features.  Integrating the tests repo means that can be done in the
> > same PR, reducing the amount of github busywork involved in doing so.
> > It also means the code and tests can be reviewed and discussed in the
> > same place.
> > 
> > In addition to being more convenient, that means we don't need to
> > rerun basically the same tests for both the code PR and the tests PR.
> > That means less demand on the CI infrastructure, which helps everyone.
> > 
> > 3. Testing scripts can be simplified
> > 
> > AFAICT the current scripts in the tests code have a bunch of
> > conditionals for various different configurations: Kata1 vs. Kata 2.x,
> > Rust agent vs. Go agent amongst others.  If the tests repo is
> > integrated we can cut down the test scripts to just the cases that are
> > relevant for the current version of the code.
> > 
> > Understanding how the test scripts work is a real barrier to debugging
> > Kata at the moment, so simplifying them is valuable.
> > 
> > 3. Removes a barrier to running CI tests locally
> > 
> > At the moment debugging CI failures is really painful, because
> > duplicating the problem in a debuggable environment is extremely
> > awkward.  One (certainly not the only) reason for that is that the
> > code repo and tests repo need to be both assembled together in a
> > suitable environment.  The CI scripts do that, but this often involves
> > downloading one or the other repo from a global URL.  If you want to
> > do frequently repeated tests on a local repo while debugging, that has
> > to be worked around.
> > 
> > If tests and code are in the same repo, it removes the need to
> > download either one from a global URL, making it easier to replicate
> > the CI test in a local environment and with an ad-hoc tree.  It also
> > allows some further simplifications of the CI scripts.
> > 
> > 
> > I could probably come up with more reasons, but those 3 seem like a
> > solid start.
> 
> Solid +1 from me ;-)
> 
> Now, in the interest of exhaustivity, can you think of reasons not to?

I really can't, but I'm kind of assuming someone will tell me some.

> I can think of a couple, but I'm interested in your viewpoint here.
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.katacontainers.io/pipermail/kata-dev/attachments/20210115/f6895877/attachment.sig>