test: drop nested virt required on integration test #1249
Conversation
henrywang
force-pushed
the
tmt_runner
branch
4 times, most recently
from
98b26df to
8eccc27
Compare
|
Yes, all green now! |
Use TF runner as test machine and run all test on TF runner itself Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>
|
|
||
| FROM $TIER1_IMAGE_URL | ||
|
|
||
| COPY hack/provision-derived.sh /tmp |
There was a problem hiding this comment.
It'd be nice in the future to not duplicate this...I think we need to tease apart "run tmt test using bootc install to-existing-root" in this repo with using the same flow in others.
There was a problem hiding this comment.
Actually I was looking at this more and I think we really want a flow where the tmt tests reference an image in containers-storage instead of building it on the target again.
To me a key flow here is that it's possible to both podman build and podman run the image under test - then I should be able to boot a VM from that image, without rebuilding it.
This flow is currently breaking that.
What I still find endlessly confusing with tmt is the use case of "inject code under test"; it seems like every case needs a tmt wrapper to do this.
There was a problem hiding this comment.
To me a key flow here is that it's possible to both podman build and podman run the image under test - then I should be able to boot a VM from that image, without rebuilding it.
And this one strongly relates to virt-manager/virt-manager#739 which talks about setting up a virtiofs mount from the target VM that can fetch the container image.
I think that's the kind of thing we want to plumb through tmt...actually I filed something related to this a while ago teemtee/tmt#2348
There was a problem hiding this comment.
I agree that this process should be simplified somehow :) I'm trying to copy this for DNF: rpm-software-management/ci-dnf-stack#1670.
Currently, the container is getting rebuilt for each the test-XX-* tests, correct? Since each test has /tmt/tests/test-00-bootc-install listed as a step?
If so, as a temporary hack, could we cache the image in some directory that is persisted between tests? Maybe $TMT_PLAN_DATA?: https://tmt.readthedocs.io/en/stable/overview.html#step-variables.
There was a problem hiding this comment.
Currently, TF image mode runner does not work with bootc integration test and TF images don't include Fedora 42 and rawhide. So we use package mode TF runner. The good thing is no any changes required in TF side. We can re-install package mode TF runner to image mode TF runner with bootc install to-existing-root running by /tmt/tests/test-00-bootc-install.
Another benefit we can get is Containerfile can be defined by us according to our test requirements. Then we can get our own image mode TF runner.
The last benefit is with this method, image mode test can be run on Fedora CI and OSCI smoothly without changes. e.g. https://src.fedoraproject.org/rpms/bootc/pull-request/18.
The plans listed in tmt/plans/integrations.fmf are running in parallel. Different plan has its own TF runner. So the test /tmt/tests/test-00-bootc-install has to be run first in each plan. If the test is not running in parallel, the /tmt/tests/test-00-bootc-install can be run once first.
There was a problem hiding this comment.
The plans listed in tmt/plans/integrations.fmf are running in parallel.
Ah, I see, then in that case it makes less sense to build the image up front and then reuse it.
There was a problem hiding this comment.
No it definitely still makes sense to build the image up front, parallelization can be solved here by having multiple test jobs wait on the build job.
In a simple software project it's the same problem as building the binary-under-test and running unit tests in parallel against it, which e.g. meson and cargo and many systems know how to do.
The huge complication here is that really Testing Farm is just not designed with this type of thing in mind as far as I can tell at all - even with RPMs it's really packit that helps wrap TF and ensure that RPMs-under-test get injected into the target environment. But packit is also just about RPMs, not containers.
The more I think about this the more I keep coming back to the idea that we do actually need a new thing that wraps TF.
| # local image: file:///home/foobar/image.qcow2 | ||
| provision: | ||
| how: virtual | ||
| image: https://composes.stream.centos.org/stream-10/production/CentOS-Stream-10-20250331.0/compose/BaseOS/x86_64/images/CentOS-Stream-GenericCloud-10-20250331.0.x86_64.qcow2 |
There was a problem hiding this comment.
Just circling back to the details here...this is going to get garbage collected at some point, we need to change to use a floating image or bump via renovate (noisy)
There was a problem hiding this comment.
Yeah, this one is for local run only, testing farm has its own image to deploy on TF runner. I left this option here for easy running locally.
Use TF runner as test machine and run all test on TF runner itself. Integration test does not need nested virtualization TF runner any more. This change helps speed up deployment time and running time.
After this change:
tmt/tests/tmt run. It's easy to run test locally for debugging.