Improve the backup e2e tests to allow multiple tests

[johscheuer]

Description

Before those changes only one backup and restore worked. Since we are adding more tests we have to improve the test suite for the backup and restore part of the operator.

Type of change

• Other

Discussion

Testing

Ran the e2e tests.

Documentation

Follow-up

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: ece79a6
• Duration 2:19:06
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 63e653d
• Duration 2:19:24
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: e5e9cf7
• Duration 2:56:37
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 51c7af4
• Duration 2:07:00
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 51c7af4
• Duration 2:45:56
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 51c7af4
• Duration 2:53:29
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[johscheuer]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 51c7af4
• Duration 2:53:29
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[Container] 2025/09/15 13:57:16.701721 Running command if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/logs/*.log; export fail_test=true; fi
TESTS FAILED SEE THESE LOGS:

/codebuild/output/src1840832526/src/github.com/FoundationDB/fdb-kubernetes-operator/logs/test_operator_ha_upgrades.log

[Container] 2025/09/15 13:57:17.836583 Running command for f in $(grep -L -- "SUCCESS\!" ${CODEBUILD_SRC_DIR}/logs/*.log); do if ! grep -q -- "--- FAIL:" $f; then echo "This test has timed out or had a panic: $f"; export fail_test=true; fi ;  done

Unrelated failure.

[akankshamahajan15]

I tested these changes with my changes (which is adding additional test - encryption backup test) and it passes successfully.

[johscheuer]

Waiting for the test result before merging.

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 6362134
• Duration 2:48:41
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[johscheuer]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 6362134
• Duration 2:48:41
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

  [FAILED] in [BeforeEach] - /codebuild/output/src1732650022/src/github.com/FoundationDB/fdb-kubernetes-operator/e2e/fixtures/fdb_restore.go:121 @ 09/17/25 17:15:58.948
• [FAILED] [1427.596 seconds]
Operator Backup when a cluster has backups enabled and then restored when the default backup system is used when using a restorable version [BeforeEach] should restore the cluster successfully with a restorable version [e2e, pr]
  [BeforeEach] /codebuild/output/src1732650022/src/github.com/FoundationDB/fdb-kubernetes-operator/e2e/test_operator_backups/operator_backup_test.go:133
  [It] /codebuild/output/src1732650022/src/github.com/FoundationDB/fdb-kubernetes-operator/e2e/test_operator_backups/operator_backup_test.go:137

  [FAILED] Timed out after 1200.001s.
  Expected
      <v1beta2.FoundationDBRestoreState>: Unknown
  to equal
      <v1beta2.FoundationDBRestoreState>: completed
  In [BeforeEach] at: /codebuild/output/src1732650022/src/github.com/FoundationDB/fdb-kubernetes-operator/e2e/fixtures/fdb_restore.go:121 @ 09/17/25 17:15:58.948

👀

I did a manual run and everything was fine:

-----------------------------
[ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report
autogenerated by Ginkgo
[ReportAfterSuite] PASSED [0.008 seconds]
------------------------------

Ran 2 of 3 Specs in 454.382 seconds
SUCCESS! -- 2 Passed | 0 Failed | 0 Pending | 1 Skipped
--- PASS: TestOperator (454.39s)
PASS
ok      github.com/FoundationDB/fdb-kubernetes-operator/v2/e2e/test_operator_backups    455.294s

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 6362134
• Duration 1:52:00
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 6362134
• Duration 1:53:55
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux RHEL 9

• Commit ID: 75f4fcb
• Duration 2:40:39
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)