Skip to content

Fix 163: make credentials save/load thread safe #164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 17, 2022
Merged

Fix 163: make credentials save/load thread safe #164

merged 4 commits into from
Apr 17, 2022

Conversation

shcheklein
Copy link
Member

@shcheklein shcheklein commented Apr 16, 2022

Fixes #163

The reason for the bug in #163 is here:

https://github.com/iterative/PyDrive2/blob/master/pydrive2/auth.py#L396-L398

The problem is that every time any thread runs SaveCredentialsFile when token is being refreshed it sets the new Storage object, each storage object defines the Lock that is being used later for put, get, and other operations that are being done with the storage (clearly we don't want multiple threads writing and / or reading the same file again).

The following situation was possible:

T1 - acquires lock A from the Storage A
T2 - waits to acquire the same lock A
T1 - saves credentials, releases lock A, and updates Storage A to Storage B and the same happens with lock, now it's lock B
(it's already not thread safe since T1 now can do some stuff in parallel with T2 since they use different locks)
T2 - acquires lock A
T2 - reads credentials
T2 - get's to the point where it needs to release lock, but credentials object now points to Storage B and it's trying to release lock B (thus "release unlocked lock" exception)

It's a critical MT bug and it's strange that it was working at all before.

TODO:

  • First Update flake and black #165 should be merged, this one will be rebased
  • run pre-commit
  • I still need to run some tests and may be even add some unit tests to safe guard this for the future.

@shcheklein shcheklein requested review from efiop and casperdcl April 16, 2022 02:33
@shcheklein shcheklein marked this pull request as draft April 16, 2022 04:33
@shcheklein shcheklein marked this pull request as ready for review April 16, 2022 04:43
storage.put(self.credentials)
self.credentials.set_store(storage)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record - removing this line here and moving storage = Storage(credentials_file) is the most important part of this change.

This way storage initialized once instead of every time we are refreshing the token.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

access_token refresh causes threads to hang
1 participant