March 2019 - Kata-hypervisor - lists.katacontainers.io

Any known conflicts if I run Kata containers and regular VMs on a same host?
by XiaoNing Ding 28 Mar '19

28 Mar '19

Hi folks, I'm trying to run Kata containers and regular QEMU-KVM VMs side by side on one same host. While I'm setting up the test environment, I'd like to check if there are any known conflicts between them? I guess the question is mostly about possible conflicts between QEMU-lite and the vanilla QEMU, when they are running side by side. It seems the community is discussing moving to vanilla QEMU (https://github.com/kata-containers/runtime/issues/1397) but before that happens QEMU-lite is still the default option in releases. Besides hypervisor, could there be other conflicts when we run Kata and full VMs together? Thanks, Xiaoning

2 2

Fwd: State of QEMU CI as we enter 4.0
by Paolo Bonzini 14 Mar '19

14 Mar '19

Guys, you are probably interested in this. It provides a good summary of what QEMU is doing for CI, and I think you shouldn't be afraid to ask about even potentially controversial stuff (moving or automating Peter's pre-pull-request testing; requiring a GitLab repo for maintainers; steps to improve acceptance testing and how to implement it). This is the right time for that. Paolo -------- Forwarded Message -------- Subject: State of QEMU CI as we enter 4.0 Date: Thu, 14 Mar 2019 15:57:06 +0000 From: Alex Bennée <alex.bennee(a)linaro.org> To: qemu-devel(a)nongnu.org <qemu-devel(a)nongnu.org> CC: Paolo Bonzini <pbonzini(a)redhat.com>, Peter Maydell <peter.maydell(a)linaro.org>, Daniel P. Berrangé <berrange(a)redhat.com>, Philippe Mathieu-Daudé <philmd(a)redhat.com>, thuth(a)redhat.com, emaste(a)freebsd.org, lwhsu(a)freebsd.org, fam(a)euphon.net, Cleber Rosa <crosa(a)redhat.com>, ymankad(a)redhat.com Hi, As we approach stabilisation for 4.0 I thought it would be worth doing a review of the current state of CI and stimulate some discussion of where it is working for us and what could be improved. Testing in Build System ======================= Things seem to be progressing well in this respect. More and more tests have been added into the main source tree and they are only a make invocation away. These include: check (includes unit, qapi-schema, qtest and decodetree) check-tcg (now with system mode tests!) check-softfloat check-block check-acceptance Personally check-acceptance is the area I've looked at the least but this seems to be the best place for "full life cycle" tests like booting kernels and running stress and performance tests. I'm still a little unsure how we deal with prebuilt kernels and images here though. Are they basically provided by 3rd parties from their websites? Do we mirror any of the artefacts we use for these tests? One area of concern is how well this all sits with KVM (and other HW accelerators) and how that gets tested. With my ARM hat on I don't really see any integration between testing kernel and QEMU changes together to catch any problems as the core OS support for KVM gets updated. Another area I would like to improve is how we expand testing with existing test suites. I'm thinking things like LTP and kvm-unit-tests which can exercise a bunch of QEMU code but are maybe a bit to big to be included in the source tree. Although given we included TestFloat (via a git submodule) maybe we shouldn't dismiss that approach? Or is this something that could be done via Avocado? Generally though I think we are doing pretty well at increasing our test coverage while making the tests more directly available to developers without having to rely on someones personal collection of random binaries. I wanted to know if we should encode this somewhere in our developer documentation: There is a strong preference for new QEMU tests to be integrated with the build system. Developers should be able to (build and) run the new tests locally directly from make. ? Testing in the Cloud ==================== After BuildBot went out-of-service we have been relying heavily on Travis as our primary CI platform. This has been creaking somewhat under the strain and while we have a large test matrix its coverage is fairly Ubuntu/x86 centric. However in recent months we've expanded and we now have: - Shippable, cross compilers - catches a lot of 32/64 bit isms - Cirrus, FreeBSD and MacOS builds - GitLab, Alternative x86/Debian - iotests Currently they don't a whole lot to the diversity of our testing although Shippable is pretty quick and does catch cross-compile missteps quite nicely. I think there is a good argument for removing some of the testing from Travis and trying to get its long run time down to something a bit more useful and balancing that with more tests in on the other services. I'm currently looking at how easy it is to expand the build farm with GitLab. It holds a promise of making it easy to add external runners to the build farm with a fairly simple installation of a runner client on the machine. We'd just need to beg and borrow for more non-x86 machines. Cloud CI Feedback ------------------ Currently only Travis reports the current status to our IRC channel but with the stability of the testing being up and down it sometimes feels like unnecessary noise. I've put together a wiki template that allows tracking of the current CI status using badges: https://wiki.qemu.org/Template:CIStatus Which includes patchew and Coverity status. This can be included on any personal pages you wish but it is also fairly prominent on the testing page: https://wiki.qemu.org/Testing For those wishing to have a central point of reference for other branches there is: https://wiki.qemu.org/Template:CustomCIStatus which is parameterised so you can enable for different branches, for example my fpu/next branch is linked from: https://wiki.qemu.org/Features/Softfloat Of course for these to be useful people need to a) look at them and b) be confident enough that non-green is worth looking at. I'm wary of adding a bunch more notifications onto the IRC channel. What I really would like is a cibot which could sit on our channel and aggregate the status from the various services and also be queried for the state of various branches. However this would be a chunk of non-QEMU related work to get up and running. Test Stability -------------- We have a slowly growing number of tests who seem to fail on a fairly regular basis in our CI tests. Sometimes it has been possible to replicate the failure but often it seems to be a feature of running in the CI system that is hard to replicate on developer machines. I've had some success in replicating some and getting the appropriate developers to debug on a tmate session. However we really we need more eyes on these troublesome tests helping with the effort. For now I've created a table to track them so the details are not lost on the ephemeral IRC: https://wiki.qemu.org/Testing/CI/TroublesomeTests If we start collecting data on this page we can maybe spot some patterns or gain some clues on how to replicate the failures locally. As you may notices I've created the a Testing/CI page and am slowly populating the sub pages with more information. Please feel free to help in the effort ;-) Docker Images ============= The addition of docker has unlocked the ability to build a lot more tests as well as compile testing on a much wider range of distros. I think there are two outstanding areas that need improvement Daniel has been looking at building and hosting the images somewhere. This would be useful as it would stop us slamming the distros repositories constantly rebuilding the same images and also help reduce the time to test. The other area that needs some work is better supporting non-x86 hosts. While Docker's multi-arch story is much better (docker run debian:stable will DTRT on any main architecture) we get stumped by things like Debian's uneven support of cross compilers. For 4.1 I'd like to reorganise the dockerfiles subdirectory into multiarch and arch specific directories so we approach this is a less ad-hoc way. It would also be nice to have the ability to gracefully fallback to linux-user powered images where the host architecture doesn't have what we need. Conclusion ========== I think generally the state of QEMU's CI has improved over the last few years but we still have a number of challenges caused by its distributed nature and test stability. We are still re-active to failures rather than having something fast and reliable enough to gate changes going into the code base. This results in fairly long periods when one or more parts of the testing mosaic are stuck on red waiting for fixes to finally get merged back into master. So what do people think? Have I missed anything out? What else can we do to improve the situation? Let the discussion begin ;-) -- Alex Bennée

1 0

kconfig is in for QEMU 4.0!
by Paolo Bonzini 10 Mar '19

10 Mar '19

This should make it easier to take a minimal QEMU configuration and carry it across releases. In particular NEMU probably wants to make --without-default-devices the default and adjust their default-configs/{aarch64,x86_64}-softmmu.mak file. Thanks Yang from Intel and everyone from Red Hat that worked on this. It's nice to see this completed after, ehm, 5.5 years. Paolo

4 3