Open Stack

> On 15 Jan 2021, at 04:29, David Gibson <david at gibson.dropbear.id.au> wrote:
> 
> Hi all,
> 
> I'm a pretty recently started Kata contributor.  One of many things I
> found frustrating when I started working on Kata was the way the code
> was split into a whole batch of repositories.  I'm relieved that
> they're now merged together for Kata 2.x.
> 
> However, the tests repository is still separate, and AIUI there are no
> plans to integrate it.  I still find that really annoying, so I want
> to make the case for integrating it into the kata-containers
> repository as well.
> 
> 1. It removes delays fixing test bugs
> 
> On several occasions when working on code changes, I've hit CI
> failures due to what turn out to be bugs in the tests code.  These
> have often been problems with static checks essentially unrelated to
> what I'm actually trying to accomplish.
> 
> To deal with this, I've had to file a separate issue and PR against
> the tests repo, wait for that to be tested and reviewed, and only then
> can I resume work on what I was actually trying to do.  Especially
> since I'm a different timezone from most of the Kata team, that can
> easily be several days of delay.
> 
> If the tests were integrated, I could include a fix for the test as a
> commit in the same PR, it would automatically fix the CI for that PR
> and could be reviewed with the same batch.
> 
> 2. Allows tests to be commited along with new features
> 
> Obviously it's desirable for new features to come with tests for those
> features.  Integrating the tests repo means that can be done in the
> same PR, reducing the amount of github busywork involved in doing so.
> It also means the code and tests can be reviewed and discussed in the
> same place.
> 
> In addition to being more convenient, that means we don't need to
> rerun basically the same tests for both the code PR and the tests PR.
> That means less demand on the CI infrastructure, which helps everyone.
> 
> 3. Testing scripts can be simplified
> 
> AFAICT the current scripts in the tests code have a bunch of
> conditionals for various different configurations: Kata1 vs. Kata 2.x,
> Rust agent vs. Go agent amongst others.  If the tests repo is
> integrated we can cut down the test scripts to just the cases that are
> relevant for the current version of the code.
> 
> Understanding how the test scripts work is a real barrier to debugging
> Kata at the moment, so simplifying them is valuable.
> 
> 3. Removes a barrier to running CI tests locally
> 
> At the moment debugging CI failures is really painful, because
> duplicating the problem in a debuggable environment is extremely
> awkward.  One (certainly not the only) reason for that is that the
> code repo and tests repo need to be both assembled together in a
> suitable environment.  The CI scripts do that, but this often involves
> downloading one or the other repo from a global URL.  If you want to
> do frequently repeated tests on a local repo while debugging, that has
> to be worked around.
> 
> If tests and code are in the same repo, it removes the need to
> download either one from a global URL, making it easier to replicate
> the CI test in a local environment and with an ad-hoc tree.  It also
> allows some further simplifications of the CI scripts.
> 
> 
> I could probably come up with more reasons, but those 3 seem like a
> solid start.

Solid +1 from me ;-)

Now, in the interest of exhaustivity, can you think of reasons not to?

I can think of a couple, but I'm interested in your viewpoint here.