OADP-4640: Downstream only to allow override kopia default algorithms #334

mpryc · 2024-08-02T13:12:25Z

Introduction of downstream only option to override Kopia default:

hashing algorithm
splitting algorithm
encryption algorithm

With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero:

KOPIA_HASHING_ALGORITHM
KOPIA_SPLITTER_ALGORITHM
KOPIA_ENCRYPTION_ALGORITHM

If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change.

Does your change fix a particular issue?

Fixes #OADP-4640

The change introduces new test file to easy rebases. Also the changes to the original file are minimal and split into one function to reduce risk of further rebase headaches.

Testing

$ pkg/repository/udmrepo/kopialib/backend/
$ go test -v .

openshift-ci-robot · 2024-08-02T13:12:30Z

@mpryc: This pull request references OADP-4640 which is a valid jira issue.

In response to this:

Introduction of downstream only option to override Kopia default:

hashing algorithm

splitting algorithm

encryption algorithm

With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero:

KOPIA_HASHING_ALGORITHM
KOPIA_SPLITTER_ALGORITHM
KOPIA_ENCRYPTION_ALGORITHM

If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change.

Does your change fix a particular issue?

Fixes #OADP-4640

The change introduces new test file to easy rebases. Also the changes to the original file are minimal and split into one function to reduce risk of further rebase headaches.

Testing
$ pkg/repository/udmrepo/kopialib/backend/
$ go test -v .

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2024-08-02T13:12:34Z

@mpryc: This pull request references OADP-4640 which is a valid jira issue.

In response to this:

Introduction of downstream only option to override Kopia default:

hashing algorithm

splitting algorithm

encryption algorithm

With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero:

KOPIA_HASHING_ALGORITHM
KOPIA_SPLITTER_ALGORITHM
KOPIA_ENCRYPTION_ALGORITHM

If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change.

Does your change fix a particular issue?

Fixes #OADP-4640

The change introduces new test file to easy rebases. Also the changes to the original file are minimal and split into one function to reduce risk of further rebase headaches.

Testing
$ pkg/repository/udmrepo/kopialib/backend/
$ go test -v .

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

pkg/repository/udmrepo/kopialib/backend/common.go

pkg/repository/udmrepo/kopialib/backend/common_kopia_algorithms_test.go

mateusoliveira43 · 2024-08-08T14:24:00Z

pkg/repository/udmrepo/kopialib/backend/common_kopia_algorithms_test.go

+			},
+		},
+		{
+			name:    "with hash algo in flags, env var ignored",


this test is not doing what is expected, no?

value from env var appears in expected

Yes but I wanted to test that udmrepo does not change the algorhtm note the SHA3-256 is different then SHA3-224 so different ones are set with env and different with udmrepo.

I didn't change this code as I think it's correct.

the test name makes no sense, the environment variable is not being ignored

Introduction of downstream only option to override Kopia default: - hashing algorithm - splitting algorithm - encryption algorithm With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero: KOPIA_HASHING_ALGORITHM KOPIA_SPLITTER_ALGORITHM KOPIA_ENCRYPTION_ALGORITHM If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change. Signed-off-by: Michal Pryc <mpryc@redhat.com>

openshift-ci · 2024-08-09T20:46:55Z

@mpryc: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

weshayutin · 2024-08-09T20:57:17Z

From https://github.com/openshift/velero
 * [new ref]             refs/pull/334/head -> issue-oadp-4640
Switched to branch 'issue-oadp-4640'
whayutin@thinkdoe:~/OPENSHIFT/git/OADP/velero$ cd pkg/repository/udmrepo/kopialib/backend/
whayutin@thinkdoe:~/OPENSHIFT/git/OADP/velero/pkg/repository/udmrepo/kopialib/backend$ ls
azure          backend.go                       common_test.go       gcs.go       s3.go       utils_test.go
azure.go       common.go                        file_system.go       gcs_test.go  s3_test.go
azure_test.go  common_kopia_algorithms_test.go  file_system_test.go  mocks        utils.go
whayutin@thinkdoe:~/OPENSHIFT/git/OADP/velero/pkg/repository/udmrepo/kopialib/backend$ go test -v .
=== RUN   TestAzureSetup
--- PASS: TestAzureSetup (0.00s)
=== RUN   TestSetupNewRepoAlgorithms
=== RUN   TestSetupNewRepoAlgorithms/with_valid_non-default_hash_algo_from_env
=== RUN   TestSetupNewRepoAlgorithms/with_valid_non-default_encryption_algo_from_env
=== RUN   TestSetupNewRepoAlgorithms/with_valid_non-default_splitter_algo_from_env
=== RUN   TestSetupNewRepoAlgorithms/with_valid_non-default_splitter_and_hashing_algo_from_env,_invalid_encryption_from_env
=== RUN   TestSetupNewRepoAlgorithms/with_unsupported_hash_algo_in_env,_fallback_to_default
=== RUN   TestSetupNewRepoAlgorithms/hash_in_StoreOptionGenHashAlgo_and_env,_env_wins
--- PASS: TestSetupNewRepoAlgorithms (0.00s)
    --- PASS: TestSetupNewRepoAlgorithms/with_valid_non-default_hash_algo_from_env (0.00s)
    --- PASS: TestSetupNewRepoAlgorithms/with_valid_non-default_encryption_algo_from_env (0.00s)
    --- PASS: TestSetupNewRepoAlgorithms/with_valid_non-default_splitter_algo_from_env (0.00s)
    --- PASS: TestSetupNewRepoAlgorithms/with_valid_non-default_splitter_and_hashing_algo_from_env,_invalid_encryption_from_env (0.00s)
    --- PASS: TestSetupNewRepoAlgorithms/with_unsupported_hash_algo_in_env,_fallback_to_default (0.00s)
    --- PASS: TestSetupNewRepoAlgorithms/hash_in_StoreOptionGenHashAlgo_and_env,_env_wins (0.00s)
=== RUN   TestSetupNewRepositoryOptions
=== RUN   TestSetupNewRepositoryOptions/with_hash_algo
=== RUN   TestSetupNewRepositoryOptions/with_encrypt_algo
=== RUN   TestSetupNewRepositoryOptions/with_splitter_algo
=== RUN   TestSetupNewRepositoryOptions/with_retention_algo
--- PASS: TestSetupNewRepositoryOptions (0.00s)
    --- PASS: TestSetupNewRepositoryOptions/with_hash_algo (0.00s)
    --- PASS: TestSetupNewRepositoryOptions/with_encrypt_algo (0.00s)
    --- PASS: TestSetupNewRepositoryOptions/with_splitter_algo (0.00s)
    --- PASS: TestSetupNewRepositoryOptions/with_retention_algo (0.00s)
=== RUN   TestSetupConnectOptions
=== RUN   TestSetupConnectOptions/with_domain
=== RUN   TestSetupConnectOptions/with_username
=== RUN   TestSetupConnectOptions/with_wrong_readonly
=== RUN   TestSetupConnectOptions/with_correct_readonly
=== RUN   TestSetupConnectOptions/with_description
--- PASS: TestSetupConnectOptions (0.00s)
    --- PASS: TestSetupConnectOptions/with_domain (0.00s)
    --- PASS: TestSetupConnectOptions/with_username (0.00s)
    --- PASS: TestSetupConnectOptions/with_wrong_readonly (0.00s)
    --- PASS: TestSetupConnectOptions/with_correct_readonly (0.00s)
    --- PASS: TestSetupConnectOptions/with_description (0.00s)
=== RUN   TestFSSetup
=== RUN   TestFSSetup/must_have_fs_path
=== RUN   TestFSSetup/with_fs_path_only
=== RUN   TestFSSetup/with_prefix
--- PASS: TestFSSetup (0.00s)
    --- PASS: TestFSSetup/must_have_fs_path (0.00s)
    --- PASS: TestFSSetup/with_fs_path_only (0.00s)
    --- PASS: TestFSSetup/with_prefix (0.00s)
=== RUN   TestGcsSetup
=== RUN   TestGcsSetup/must_have_bucket_name
=== RUN   TestGcsSetup/must_have_credential_file
=== RUN   TestGcsSetup/with_prefix
=== RUN   TestGcsSetup/with_wrong_readonly
=== RUN   TestGcsSetup/with_correct_readonly
--- PASS: TestGcsSetup (0.00s)
    --- PASS: TestGcsSetup/must_have_bucket_name (0.00s)
    --- PASS: TestGcsSetup/must_have_credential_file (0.00s)
    --- PASS: TestGcsSetup/with_prefix (0.00s)
    --- PASS: TestGcsSetup/with_wrong_readonly (0.00s)
    --- PASS: TestGcsSetup/with_correct_readonly (0.00s)
=== RUN   TestS3Setup
=== RUN   TestS3Setup/must_have_bucket_name
=== RUN   TestS3Setup/with_bucket_only
=== RUN   TestS3Setup/with_others
=== RUN   TestS3Setup/with_wrong_tls
=== RUN   TestS3Setup/with_correct_tls
=== RUN   TestS3Setup/with_wrong_ca
=== RUN   TestS3Setup/with_correct_ca
--- PASS: TestS3Setup (0.00s)
    --- PASS: TestS3Setup/must_have_bucket_name (0.00s)
    --- PASS: TestS3Setup/with_bucket_only (0.00s)
    --- PASS: TestS3Setup/with_others (0.00s)
    --- PASS: TestS3Setup/with_wrong_tls (0.00s)
    --- PASS: TestS3Setup/with_correct_tls (0.00s)
    --- PASS: TestS3Setup/with_wrong_ca (0.00s)
    --- PASS: TestS3Setup/with_correct_ca (0.00s)
=== RUN   TestOptionalHaveBool
=== RUN   TestOptionalHaveBool/key_not_exist
=== RUN   TestOptionalHaveBool/value_valid
=== RUN   TestOptionalHaveBool/value_invalid
--- PASS: TestOptionalHaveBool (0.00s)
    --- PASS: TestOptionalHaveBool/key_not_exist (0.00s)
    --- PASS: TestOptionalHaveBool/value_valid (0.00s)
    --- PASS: TestOptionalHaveBool/value_invalid (0.00s)
PASS
ok  	github.com/vmware-tanzu/velero/pkg/repository/udmrepo/kopialib/backend	0.008s

weshayutin

/LGTM

shubham-pampattiwar · 2024-08-13T06:26:53Z

pkg/repository/udmrepo/kopialib/backend/common_kopia_algorithms_test.go

+				},
+			},
+		},
+		{


@mpryc IMO we should add another testcase when StoreOptionGenHashAlgos has a valid algo value but the env var has an invalid algo value. Then in such a scenario default would not be used but the one present in StoreOptionGenHashAlgos would be used, right ?

@shubham-pampattiwar right. there are 2 levels of fallback here:

use env var if it's set and value is supported, if not, fall back to...

value in StoreOptionGenHashAlgo, if not fall back to...

default

shubham-pampattiwar · 2024-08-13T15:18:13Z

/approved

openshift-ci · 2024-08-13T15:20:40Z

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: mpryc, shubham-pampattiwar, sseago, weshayutin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

DOWNSTREAM_OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…openshift#334) add missing unit test for kopia hashing algo (openshift#337) Introduction of downstream only option to override Kopia default: - hashing algorithm - splitting algorithm - encryption algorithm With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero: KOPIA_HASHING_ALGORITHM KOPIA_SPLITTER_ALGORITHM KOPIA_ENCRYPTION_ALGORITHM If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change. Signed-off-by: Michal Pryc <mpryc@redhat.com> Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

…#334) (#338) add missing unit test for kopia hashing algo (#337) Introduction of downstream only option to override Kopia default: - hashing algorithm - splitting algorithm - encryption algorithm With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero: KOPIA_HASHING_ALGORITHM KOPIA_SPLITTER_ALGORITHM KOPIA_ENCRYPTION_ALGORITHM If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change. Signed-off-by: Michal Pryc <mpryc@redhat.com> Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

…openshift#334) Introduction of downstream only option to override Kopia default: - hashing algorithm - splitting algorithm - encryption algorithm With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero: KOPIA_HASHING_ALGORITHM KOPIA_SPLITTER_ALGORITHM KOPIA_ENCRYPTION_ALGORITHM If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change. Signed-off-by: Michal Pryc <mpryc@redhat.com>

…#334) (#338) add missing unit test for kopia hashing algo (#337) Introduction of downstream only option to override Kopia default: - hashing algorithm - splitting algorithm - encryption algorithm With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero: KOPIA_HASHING_ALGORITHM KOPIA_SPLITTER_ALGORITHM KOPIA_ENCRYPTION_ALGORITHM If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change. Signed-off-by: Michal Pryc <mpryc@redhat.com> Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

This fixes the PR openshift#334 where one additional line was in the code. This was not exposed previously as we did not had downstream CI Lint jobs. Signed-off-by: Michal Pryc <mpryc@redhat.com>

This fixes the PR #334 where one additional line was in the code. This was not exposed previously as we did not had downstream CI Lint jobs. Signed-off-by: Michal Pryc <mpryc@redhat.com>

This fixes the PR openshift#334 where one additional line was in the code. This was not exposed previously as we did not had downstream CI Lint jobs. Signed-off-by: Michal Pryc <mpryc@redhat.com>

This fixes the PR #334 where one additional line was in the code. This was not exposed previously as we did not had downstream CI Lint jobs. Signed-off-by: Michal Pryc <mpryc@redhat.com> Co-authored-by: Michal Pryc <mpryc@redhat.com>

…openshift#334) (openshift#338) add missing unit test for kopia hashing algo (openshift#337) Introduction of downstream only option to override Kopia default: - hashing algorithm - splitting algorithm - encryption algorithm With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero: KOPIA_HASHING_ALGORITHM KOPIA_SPLITTER_ALGORITHM KOPIA_ENCRYPTION_ALGORITHM If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change. Signed-off-by: Michal Pryc <mpryc@redhat.com> Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

This fixes the PR openshift#334 where one additional line was in the code. This was not exposed previously as we did not had downstream CI Lint jobs. Signed-off-by: Michal Pryc <mpryc@redhat.com>

…openshift#334) (openshift#338) add missing unit test for kopia hashing algo (openshift#337) Introduction of downstream only option to override Kopia default: - hashing algorithm - splitting algorithm - encryption algorithm With introduction of 3 environment variables it is possible to override Kopia algorithms used by Velero: KOPIA_HASHING_ALGORITHM KOPIA_SPLITTER_ALGORITHM KOPIA_ENCRYPTION_ALGORITHM If the env algorithms are not set or they are not within Kopia SupportedAlgorithms, the default algorithm will be used. This behavior is consistent with current behavior without this change. Signed-off-by: Michal Pryc <mpryc@redhat.com> Signed-off-by: Shubham Pampattiwar <shubhampampattiwar7@gmail.com>

This fixes the PR openshift#334 where one additional line was in the code. This was not exposed previously as we did not had downstream CI Lint jobs. Signed-off-by: Michal Pryc <mpryc@redhat.com>

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 2, 2024

mateusoliveira43 reviewed Aug 8, 2024

View reviewed changes

pkg/repository/udmrepo/kopialib/backend/common.go Outdated Show resolved Hide resolved

mateusoliveira43 reviewed Aug 8, 2024

View reviewed changes

pkg/repository/udmrepo/kopialib/backend/common_kopia_algorithms_test.go Outdated Show resolved Hide resolved

mateusoliveira43 reviewed Aug 8, 2024

View reviewed changes

mpryc force-pushed the issue-oadp-4640 branch from 4afd920 to 8480ac4 Compare August 8, 2024 18:43

mpryc requested review from kaovilai and mateusoliveira43 August 9, 2024 06:29

mpryc force-pushed the issue-oadp-4640 branch from 8480ac4 to 2fd82cf Compare August 9, 2024 20:09

weshayutin approved these changes Aug 9, 2024

View reviewed changes

openshift-ci bot assigned weshayutin Aug 9, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Aug 9, 2024

shubham-pampattiwar reviewed Aug 13, 2024

View reviewed changes

sseago approved these changes Aug 13, 2024

View reviewed changes

shubham-pampattiwar approved these changes Aug 13, 2024

View reviewed changes

shubham-pampattiwar added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 13, 2024

openshift-merge-bot bot merged commit 48b7ec0 into openshift:oadp-1.4 Aug 13, 2024
3 checks passed

This was referenced Aug 16, 2024

OADP-4640: Downstream only to allow override kopia default algorithms… #338

Merged

Hardcoded hash, split and encryption algorithms for testing. #329

Closed

mpryc mentioned this pull request Sep 5, 2024

Downstream lint fix error #343

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OADP-4640: Downstream only to allow override kopia default algorithms #334

OADP-4640: Downstream only to allow override kopia default algorithms #334

mpryc commented Aug 2, 2024 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Aug 2, 2024 •

edited by openshift-ci bot

Loading

Does your change fix a particular issue?

Testing

openshift-ci-robot commented Aug 2, 2024 •

edited by openshift-ci bot

Loading

Does your change fix a particular issue?

Testing

mateusoliveira43 Aug 8, 2024

mpryc Aug 8, 2024

mpryc Aug 8, 2024

mateusoliveira43 Aug 9, 2024

mpryc Aug 9, 2024

openshift-ci bot commented Aug 9, 2024

weshayutin commented Aug 9, 2024

weshayutin left a comment

shubham-pampattiwar Aug 13, 2024 •

edited

Loading

sseago Aug 13, 2024

shubham-pampattiwar commented Aug 13, 2024

openshift-ci bot commented Aug 13, 2024

OADP-4640: Downstream only to allow override kopia default algorithms #334

OADP-4640: Downstream only to allow override kopia default algorithms #334

Conversation

mpryc commented Aug 2, 2024 • edited by openshift-ci bot Loading

Does your change fix a particular issue?

Testing

openshift-ci-robot commented Aug 2, 2024 • edited by openshift-ci bot Loading

Does your change fix a particular issue?

Testing

openshift-ci-robot commented Aug 2, 2024 • edited by openshift-ci bot Loading

Does your change fix a particular issue?

Testing

mateusoliveira43 Aug 8, 2024

Choose a reason for hiding this comment

mpryc Aug 8, 2024

Choose a reason for hiding this comment

mpryc Aug 8, 2024

Choose a reason for hiding this comment

mateusoliveira43 Aug 9, 2024

Choose a reason for hiding this comment

mpryc Aug 9, 2024

Choose a reason for hiding this comment

openshift-ci bot commented Aug 9, 2024

weshayutin commented Aug 9, 2024

weshayutin left a comment

Choose a reason for hiding this comment

shubham-pampattiwar Aug 13, 2024 • edited Loading

Choose a reason for hiding this comment

sseago Aug 13, 2024

Choose a reason for hiding this comment

shubham-pampattiwar commented Aug 13, 2024

openshift-ci bot commented Aug 13, 2024

mpryc commented Aug 2, 2024 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Aug 2, 2024 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Aug 2, 2024 •

edited by openshift-ci bot

Loading

shubham-pampattiwar Aug 13, 2024 •

edited

Loading