Create dedicated namespaces per test case in reconcilers tests

[souleb]

Create dedicated namespaces per test case, then delete the whole NS at the end. This will help avoid the case where we fail to stop the kube-apiserver.

for example:

namedNS, err := testEnv.CreateNamespace(context.TODO(), mockReleaseNamespace)
			g.Expect(err).NotTo(HaveOccurred())
			t.Cleanup(func() {
				_ = testEnv.Delete(context.TODO(), namedNS)
			})

https://github.com/fluxcd/helm-controller/blob/5d28aa42505641652648e338afc9c1ec4e8dfb55/internal/reconcile/install_test.go#L183

[stefanprodan]

We also need to delete all test objects before we delete the namespace.

[snebel29]

I would like to take a look to this issue, can anyone assign it to me?

[snebel29]

@stefanprodan

When I try to make test in my computer from main (with no other local changes added) I am getting several failures, whose "non-obvious" source I believe might be

--- FAIL: TestHelmChartReconciler_buildFromTarballArtifact (4.25s)
    --- FAIL: TestHelmChartReconciler_buildFromTarballArtifact/Resolves_chart_dependencies_and_builds (4.05s)
        helmchart_controller_test.go:1300: 
            Expected
                <bool>: true
            to equal
                <bool>: false
        helmchart_controller_test.go:1172: 
            Expected success, but got an error:
                <*fs.PathError | 0xc000dafd10>: {
                    Op: "remove",
                    Path: "",
                    Err: <syscall.Errno>0x2,
                }
                remove : no such file or directory

Here is the detail of the error producing that test failure

dependency build error: failed to add remote dependency 'podinfo': failed to get chart 'podinfo' version '>=6.1.*' from 'oci://ghcr.io/stefanprodan/charts': could not get tags for "podinfo": could not fetch tags for "oci://ghcr.io/stefanprodan/charts/podinfo": GET "https://ghcr.io/v2/stefanprodan/charts/podinfo/tags/list": unable to retrieve credentials

It does feel the test is trying to download that podinfo chart dependency from ghcr.io and failing to authenticate, which later causes other test cases to fail with spec.url invalid errors, etc.

• Is that "online" download with authentication expected?
• Does this trigger anything in your head?

Cheers.

[pjbgf]

@snebel29 do you mind sharing more details about your development environment? Are you on Linux/Darwin?

Feel free to ask questions on the contributors channel.

[snebel29]

Hi @pjbgf
I am developing using Goland on top of a Mac, here are the versions of the dependencies listed in DEVELOPMENT.md.

$ go version
go version go1.18.2 darwin/amd64 

$ kustomize version
{Version:kustomize/v4.5.7 GitCommit:56d82a8378dfc8dc3b3b1085e5a6e67b82966bd7 BuildDate:2022-08-02T16:28:01Z GoOs:darwin GoArch:amd64}

$ pkg-config --version
0.29.2

$ docker --version
Docker version 20.10.15, build fd82621d35

It is worth to mention that under the hood, the docker runtime is implemented using colima project.

$ colima status
INFO[0000] colima is running                            
INFO[0000] arch: x86_64                                 
INFO[0000] runtime: docker                              
INFO[0000] mountType:                                   
INFO[0000] socket: unix:///Users/myuser/.colima/default/docker.sock 

Basically I just did checkout the repository then ran make test expecting everything to pass, instead I got different test failures, I started looking to the first kind and found that suspicious error message, see.

I'll open a thread in the slack channel that you have shared, to avoid polluting this issue. Thanks in advance.

[snebel29]

For the record the culprit of the error was found to be registry client falling back on docker ~/.docker/config.json and that file not having an auth section.

{
	"auths": {},
	"credsStore": "desktop",
	"currentContext": "colima"
}

Full details on slack.