Reuse CAPI e2e test

[mboukhalfa]

User Story

As a developer, I would like to reuse CAPI (Cluster API) e2e tests such as kcp_remediation test in CAPM3 (Cluster API Provider Metal3) however CAPI randomly generates the namespaces hosting the cluster in the BeforeEach of the test, and in CAPM3, the cluster expect the BMH resources to be pre-created that namespace. I want to find a solution that allows us to make BMH (BareMetalHost) resources available in the namespace that hosts the cluster.

Detailed Description

Currently, when reusing CAPI e2e test in CAPM3, we encounter difficulties because CAPI creates clusters in randomly generated namespaces. However, in CAPM3, we pre-create the BMH resources in a fixed namespace where we expect the cluster to be created. This misalignment prevents us from reusing CAPI e2e tests.

To overcome this challenge, we need to find a way to synchronize the creation of namespaces and BMH resources, ensuring that the required BMHs are available in the namespace hosting the cluster at the time of the creation of this cluster. This will enable us to reuse many CAPI tests in CAPM3 instead of reimplimenting them.

We have explored potential workarounds, the best option seems to include BMH resources in the cluster template in a new CI cluster template dedicated for testing only, but this approach is not be suitable for official cluster templates and could potentially confuse developers and users regarding the intended production usage of Metal3.

Additionally, creating BMHs for each test poses challenges in terms of time and resource consumption. We need to consider the trade-off between running tests in parallel, skipping inspection, or finding alternative solutions to maintain efficiency without compromising the integrity of tests.

Overall, we aim to find a solution that allows us to reuse CAPI tests in CAPM3, ensuring the availability of BMH resources in the namespace generated by CAPI tests when creating a cluster.

Anything else you would like to add:

I believe addressing this issue will improve the testability of CAPM3, reusing more e2e tests that are maintained by CAPI community. This enhancement will facilitate the adoption of new e2e test created on CAPI.

/kind feature

[Rozzii]

On the last community meeting there were not enough participants to arrive to a conclusion regarding this topic.

[Rozzii]

/triage accepted

[Rozzii]

On the community meeting on 19th of July 2023 the community has decided that this is a valid use-case.

[metal3-io-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues will close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[Rozzii]

/remove-lifecycle stale
/lifecycle frozen

[lentzi90]

This would be a good topic for roadmap and API design discussions. A slightly different but related issue is if you have one team managing the BareMetalHosts and another (or even multiple) team(s) that "consume" them. In this case you may want to have the BMHs in other namespaces than where the clusters are.

In my opinion, the solution is to decouple CAPM3 from BMO. What I mean is that CAPM3 should not directly interact with the BMHs. Instead we could have something similar to how PersistentVolumes and PersistentVolumeClaims work. CAPM3 can create a claim that BMO can then fill with a suitable BMH. The BMHs could even be cluster wide objects. We should investigate what that would mean first though.

Another part of the puzzle would be multi-tenancy and authentication. How can we ensure that user A does not hijack user B's BMHs? In the CAPI model, other providers take some credentials as input when creating a cluster. This is enough to authenticate against the providers "backend". We do not have something like this currently, but it sure would solve some issues around multi-tenancy. It would mean that CAPM3 needs to provide credentials in order to get a BMH, and the pool of BMHs that are available would depend on the credentials.

Food for thought 🙂 I would love to discuss this more and play with the design to see what we could come up with