Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e flake: starting test fails with panic: runtime error: index out of range [0] with length 0 #1562

Open
nirs opened this issue Sep 16, 2024 · 0 comments · May be fixed by #1608
Open

e2e flake: starting test fails with panic: runtime error: index out of range [0] with length 0 #1562

nirs opened this issue Sep 16, 2024 · 0 comments · May be fixed by #1608
Assignees
Labels
bug Something isn't working test Testing related issue

Comments

@nirs
Copy link
Member

nirs commented Sep 16, 2024

I see this failure many times when running the tests in the first time with a new environment. Typically the next tries work. Looks like incorrect setup that fixes itself on the next run since a resource was created on the cluster in the failed attempt.

We can add a retry for the entire test suite, it is faster to rerun the tests than to create a new environment.

=== RUN   TestSuites/Exhaustive/Deploy-rbd#01/Appset
2024-09-16T22:16:15.056+0530	INFO	deployers/applicationset.go:17	enter Deploy appset-deploy-cephfs-busybox
=== PAUSE TestSuites/Exhaustive/Deploy-rbd#01/Appset
=== CONT  TestSuites/Exhaustive/Deploy-rbd#01/Appset
=== RUN   TestSuites/Exhaustive/Deploy-rbd#01/Appset/Deploy
=== RUN   TestSuites/Exhaustive/Deploy-rbd/Subscr
2024-09-16T22:16:15.056+0530	INFO	deployers/applicationset.go:17	enter Deploy appset-deploy-rbd-busybox
=== PAUSE TestSuites/Exhaustive/Deploy-rbd/Subscr
=== RUN   TestSuites/Exhaustive/Deploy-cephfs/Subscr/Deploy
2024-09-16T22:16:15.056+0530	INFO	deployers/subscription.go:31	enter Deploy subscr-deploy-cephfs-busybox
=== PAUSE TestSuites/Exhaustive/Deploy-cephfs#02/Disapp
=== CONT  TestSuites/Exhaustive/Deploy-cephfs#02/Disapp
=== RUN   TestSuites/Exhaustive/Deploy-rbd#02/Disapp
=== CONT  TestSuites/Exhaustive/Deploy-rbd/Subscr
=== PAUSE TestSuites/Exhaustive/Deploy-rbd#02/Disapp
=== CONT  TestSuites/Exhaustive/Deploy-rbd#02/Disapp
=== RUN   TestSuites/Exhaustive/Deploy-rbd#02/Disapp/Deploy
2024-09-16T22:16:15.0[57](https://github.com/RamenDR/ramen/actions/runs/10887936359/job/30211267322#step:12:58)+0530	INFO	deployers/discoveredapps.go:24	enter Deploy disapp-deploy-rbd-busybox
=== RUN   TestSuites/Exhaustive/Deploy-rbd/Subscr/Deploy
=== NAME  TestSuites/Exhaustive/Deploy-cephfs#02/Disapp
    exhaustive_suite_test.go:93: Workload Deploy-cephfs not supported by deployer Disapp, skip test
2024-09-16T22:16:15.057+0530	INFO	deployers/subscription.go:31	enter Deploy subscr-deploy-rbd-busybox
=== RUN   TestSuites/Exhaustive/Deploy-cephfs#01/Appset/Enable
2024-09-16T22:16:16.920+0530	INFO	dractions/actions.go:38	enter EnableProtection appset-deploy-cephfs-busybox
=== RUN   TestSuites/Exhaustive/Deploy-rbd#01/Appset/Enable
2024-09-16T22:16:16.936+0530	INFO	dractions/actions.go:38	enter EnableProtection appset-deploy-rbd-busybox
2024-09-16T22:16:17.022+0530	INFO	util/crud.go:99	channel ramen-gitops is deleted
--- FAIL: TestSuites (2.99s)
    --- PASS: TestSuites/Validate (0.38s)
        --- PASS: TestSuites/Validate/CheckRamenHubOperatorStatus (0.32s)
        --- PASS: TestSuites/Validate/CheckRamenSpokeOperatorStatus (0.06s)
    --- FAIL: TestSuites/Exhaustive (2.35s)
        --- PASS: TestSuites/Exhaustive/Deploy-cephfs#02 (0.00s)
            --- SKIP: TestSuites/Exhaustive/Deploy-cephfs#02/Disapp (0.00s)
        --- FAIL: TestSuites/Exhaustive/Deploy-rbd#01 (1.92s)
            --- FAIL: TestSuites/Exhaustive/Deploy-rbd#01/Appset (1.92s)
                --- PASS: TestSuites/Exhaustive/Deploy-rbd#01/Appset/Deploy (1.88s)
                --- FAIL: TestSuites/Exhaustive/Deploy-rbd#01/Appset/Enable (0.04s)
panic: runtime error: index out of range [0] with length 0 [recovered]
	panic: runtime error: index out of range [0] with length 0
@nirs nirs added bug Something isn't working test Testing related issue labels Sep 23, 2024
@nirs nirs self-assigned this Sep 24, 2024
nirs added a commit to nirs/ramen that referenced this issue Oct 28, 2024
Using the same algorithm used by drenv/test.lookup_cluster:
- Wait for the placement Satisfied condition
- Find the placement decision using using the label selector

We know that this works. In some case we can optimize by getting the
placement decision name from the placement without listing, but this
does not work in all cases so lets keep the code simpler.

Fixes RamenDR#1562

Signed-off-by: Nir Soffer <nsoffer@redhat.com>
@nirs nirs linked a pull request Oct 28, 2024 that will close this issue
nirs added a commit to nirs/ramen that referenced this issue Oct 28, 2024
Using the same algorithm used by drenv/test.lookup_cluster[1]:
- Wait for the placement Satisfied condition
- Find the placement decision using using the label selector

We know that this works. In some case we can optimize by getting the
placement decision name from the placement without listing, but this
does not work in all cases so lets keep the code simpler.

[1] https://github.com/RamenDR/ramen/blob/2886c647e9c465f5fe7a8665e53c37de002b6326/test/drenv/test.py#L204

Fixes: RamenDR#1562
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
nirs added a commit to nirs/ramen that referenced this issue Oct 28, 2024
We cannot use the method we used in drenv/test.lookup_cluster[1] since
placement Satisfied condition is not set in discovered apps flows.

Simplify the code to wait until we have placement decisions status
contains non empty decisions list.

[1] https://github.com/RamenDR/ramen/blob/2886c647e9c465f5fe7a8665e53c37de002b6326/test/drenv/test.py#L204

Fixes: RamenDR#1562
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
nirs added a commit to nirs/ramen that referenced this issue Oct 28, 2024
We cannot use the method we used in drenv/test.lookup_cluster[1] since
placement Satisfied condition is not set in discovered apps flows.

Simplify the code to wait until we have placement decisions status
contains non empty decisions list.

[1] https://github.com/RamenDR/ramen/blob/2886c647e9c465f5fe7a8665e53c37de002b6326/test/drenv/test.py#L204

Fixes: RamenDR#1562
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
nirs added a commit to nirs/ramen that referenced this issue Oct 28, 2024
We cannot use the method we used in drenv/test.lookup_cluster[1] since
placement Satisfied condition is not set in discovered apps flows.

Simplify the code to wait until we have placement decisions status
contains non empty decisions list.

[1] https://github.com/RamenDR/ramen/blob/2886c647e9c465f5fe7a8665e53c37de002b6326/test/drenv/test.py#L204

Fixes: RamenDR#1562
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
nirs added a commit to nirs/ramen that referenced this issue Oct 28, 2024
We cannot use the method we used in drenv/test.lookup_cluster[1] since
placement Satisfied condition is not set in discovered apps flows.

Simplify the code to wait until we have placement decisions status
contains non empty decisions list.

[1] https://github.com/RamenDR/ramen/blob/2886c647e9c465f5fe7a8665e53c37de002b6326/test/drenv/test.py#L204

Fixes: RamenDR#1562
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working test Testing related issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant