gdrive: oauth authentication broken

[dtrifiro]

Bug Report

google drive remote support with oauth authentication is currently broken for dvc

Description

Reproduce

1. Follow oauth setup guide
2. Add the google drive remote dvc remote add gdrive://<id>
3. Run the following snippet:

echo foo > file
dvc add file
dvc push

Expected

  0% Checking cache in '<id> |                                                                                                                                                                                            |0/? [00:00<?,    ?files/s]Go to the following link in your browser:

    https://accounts.google.com/o/oauth2/auth?client_id=<client_id.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.appdata&access_type=offline&response_type=code&approval_prompt=force

Enter verification code:

Clicking on the URL would prompt the user with the OAuth consent screen.

Actual result:

Clicking on oauth2 authentication URL returns

You can't sign in to this app because it doesn't comply with Google's OAuth 2.0 policy for keeping apps secure.

You can let the app developer know that this app doesn't comply with one or more Google validation rules.
Request Details
The content in this section has been provided by the app developer. This content has not been reviewed or verified by Google.
If you’re the app developer, make sure that these request details comply with Google policies.
redirect_uri: urn:ietf:wg:oauth:2.0:oob

Environment information

---------------------------------
Platform: Python 3.10.4 on Linux-5.15
Supports:
	gdrive (pydrive2 = 1.10.1),
	gs (gcsfs = 2022.5.0),
	webhdfs (fsspec = 2022.5.0),
	http (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
	https (aiohttp = 3.8.1, aiohttp-retry = 2.4.6),
	ssh (sshfs = 2022.6.0)
Cache types: reflink, hardlink, symlink
Cache directory: btrfs on /dev/mapper/lvm-root
Caches: local
Remotes: gs, gdrive
Workspace directory: btrfs on /dev/mapper/lvm-root
Repo: dvc, git

Reported on discord

[daavoo]

For the record, dvc relies on our fork https://github.com/iterative/PyDrive2 -> https://github.com/iterative/PyDrive2/blob/master/pydrive2/auth.py

[shcheklein]

Hmm, I can't reproduce this on my end.

The first message is not about Gdrive - it's about Google Storage, right? The second one is also about GCP. not GDrive?

@daavoo @dtrifiro ?

[shcheklein]

For GDrive. It's broken when users are creating their custom applications. It's should be working with a https://dvc.org/doc/user-guide/setup-google-drive-remote#using-service-accounts as a workaround or with a DVC built-in application.

To mitigate this we need to more from a CommandLineAuth to the LocalWebServer auth. I hope that a matter of changing a few lines of code, updating docs a bit:

gauth.LocalWebserverAuth() instead of auth.CommandLineSomething() that we have now.

[dtrifiro]

By looking at the error (redirect_uri: urn:ietf:wg:oauth:2.0:oob) and doing some research, it seems it is related to the deprecation of the out-of-band OAuth flow (see https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html).

https://github.com/iterative/PyDrive2/blob/2e43e4561d965ce78dc158a02fbdb75ba6c38105/pydrive2/settings.py#L58-L64

[dtrifiro]

Related iterative/PyDrive2#180