Skip to content

auth: add dictionary storage #204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 18, 2022
Merged

auth: add dictionary storage #204

merged 1 commit into from
Jul 18, 2022

Conversation

efiop
Copy link
Contributor

@efiop efiop commented Jul 17, 2022

Allows not to save anything to disk, which is useful overall but especially
useful when working with env vars, which could be sensitive so saving their
value to disk is undesirable.

@efiop efiop changed the title auth: add memory storage [WIP] auth: add memory storage Jul 17, 2022
@efiop efiop changed the title [WIP] auth: add memory storage [WIP] auth: add dictionary storage Jul 17, 2022
@efiop efiop force-pushed the auth branch 2 times, most recently from b127769 to 0938c78 Compare July 17, 2022 23:55
@efiop efiop force-pushed the auth branch 10 times, most recently from f7068fa to 2fddaca Compare July 18, 2022 02:03
Comment on lines +201 to +203
storages, default = self._InitializeStoragesFromSettings()
self._storages = storages
self._default_storage = default
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Original names are a bit unwieldy, so useing temp short ones. Not the prettiest, but I'm trying to make minimal changes.

@efiop efiop force-pushed the auth branch 2 times, most recently from 1742f0c to 2495f54 Compare July 18, 2022 02:24
@efiop efiop temporarily deployed to internal July 18, 2022 09:50 Inactive
@efiop efiop changed the title [WIP] auth: add dictionary storage auth: add dictionary storage Jul 18, 2022
@efiop efiop changed the title auth: add dictionary storage [WIP] auth: add dictionary storage Jul 18, 2022
@efiop efiop temporarily deployed to internal July 18, 2022 10:00 Inactive
@efiop efiop temporarily deployed to internal July 18, 2022 10:04 Inactive
@efiop efiop temporarily deployed to internal July 18, 2022 10:05 Inactive
@efiop efiop temporarily deployed to internal July 18, 2022 10:13 Inactive
@efiop efiop changed the title [WIP] auth: add dictionary storage auth: add dictionary storage Jul 18, 2022
@efiop efiop temporarily deployed to internal July 18, 2022 11:23 Inactive
@efiop efiop requested a review from shcheklein July 18, 2022 11:34
@efiop
Copy link
Contributor Author

efiop commented Jul 18, 2022

@shcheklein Ready for review.

Allows not to save anything to disk, which is useful overall but especially
useful when working with env vars, which could be sensitive so saving their
value to disk is undesirable.
@efiop efiop temporarily deployed to internal July 18, 2022 11:57 Inactive
@efiop efiop temporarily deployed to internal July 18, 2022 13:05 Inactive
@@ -347,11 +348,16 @@ def _InitializeStoragesFromSettings(self):
"Please specify credentials file to read"
)
result[backend] = Storage(credentials_file)
elif backend == "dictionary":
result[backend] = DictionaryStorage(
self.settings["save_credentials_dict"],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the file case there is a check if credentials_file is None .... raise InvalidConfigError should we do the same? (probably should be part of ValidateSettings in a single place).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@efiop ^^

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good point, missed that. Will adjust.

:save_credentials_file (str): Destination of credentials file. **Required**: Yes, only if *save_credentials_backend* is 'file'.
:save_credentials_dict (dict): Dict to use for storing credentials. **Required**: Yes, only if *save_credentials_backend* is 'dictionary'.
:save_credentials_key (str): Key within the *save_credentials_dict* to store the credentials in. **Required**: Yes, only if *save_credentials_backend* is 'dictionary'.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feels redundant tbh ... use can do [save_credentials_key] on their own ... do you have some DVC case in mind?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or is it a default convention in the DictionaryStorage?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it is default convention in DictionaryStorage. I went with a simpler solution initially, but this is indeed more flexible and more consistent.

Copy link
Member

@shcheklein shcheklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @efiop ! a few comments, approved so that we don't block it

ga.ServiceAuth()
assert not ga.access_token_expired
assert creds_dict == first_creds_dict
time.sleep(1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that's it's a convention here ... no worries then, not clear why it's needed though :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is testing that we reused the creds and didn't generate new ones. A bit vague but it does the job.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, I meant the time.sleep(1) :)

@efiop efiop merged commit 90d1d8e into master Jul 18, 2022
@efiop efiop deleted the auth branch July 18, 2022 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants