Invalid JWT Token when using Service Account JSON

[dstufft]

1. OS type and version

macOS Sierra running a Debian Jessie Docker Container

2. Python version and virtual environment information python --version

CPython 3.5.0, no virtual environment

3. google-cloud-python version pip show google-cloud, pip show google-<service> or pip freeze

google-cloud-bigquery==0.23.0

4. Stacktrace if available

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/code.py", line 91, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.5/site-packages/celery/local.py", line 191, in __call__
    return self._get_current_object()(*a, **kw)
  File "/app/warehouse/celery.py", line 53, in __call__
    return super().__call__(pyramid_env["request"], *args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/celery/app/task.py", line 379, in __call__
    return self.run(*args, **kwargs)
  File "/app/warehouse/packaging/tasks.py", line 58, in compute_trending
    query.run()
  File "/usr/local/lib/python3.5/site-packages/google/cloud/bigquery/query.py", line 364, in run
    method='POST', path=path, data=self._build_resource())
  File "/usr/local/lib/python3.5/site-packages/google/cloud/_http.py", line 299, in api_request
    headers=headers, target_object=_target_object)
  File "/usr/local/lib/python3.5/site-packages/google/cloud/_http.py", line 193, in _make_request
    return self._do_request(method, url, headers, data, target_object)
  File "/usr/local/lib/python3.5/site-packages/google/cloud/_http.py", line 223, in _do_request
    body=data)
  File "/usr/local/lib/python3.5/site-packages/google_auth_httplib2.py", line 187, in request
    self._request, method, uri, request_headers)
  File "/usr/local/lib/python3.5/site-packages/google/auth/credentials.py", line 116, in before_request
    self.refresh(request)
  File "/usr/local/lib/python3.5/site-packages/google/oauth2/service_account.py", line 318, in refresh
    request, self._token_uri, assertion)
  File "/usr/local/lib/python3.5/site-packages/google/oauth2/_client.py", line 143, in jwt_grant
    response_data = _token_endpoint_request(request, token_uri, body)
  File "/usr/local/lib/python3.5/site-packages/google/oauth2/_client.py", line 109, in _token_endpoint_request
    _handle_error_response(response_body)
  File "/usr/local/lib/python3.5/site-packages/google/oauth2/_client.py", line 59, in _handle_error_response
    error_details, response_body)
google.auth.exceptions.RefreshError: ('invalid_grant: Invalid JWT Signature.', '{\n  "error" : "invalid_grant",\n  "error_description" : "Invalid JWT Signature."\n}')

5. Steps to reproduce

Try to query anything in BigQuery using a service account with "Viewer" permissions and GOOGLE_APPLICATION_CREDENTIALS pointed to a JSON file downloaded when creating the service account.

6. Code example

bq = bigquery.Client()
query = bq.run_sync_query(
        """ SELECT project,
                   IF(
                        STDDEV(downloads) > 0,
                        (todays_downloads - AVG(downloads))/STDDEV(downloads),
                        NULL
                    ) as zscore
            FROM (
                SELECT project,
                       date,
                       downloads,
                       FIRST_VALUE(downloads) OVER (
                            PARTITION BY project
                            ORDER BY DATE DESC
                            ROWS BETWEEN UNBOUNDED PRECEDING
                                AND UNBOUNDED FOLLOWING
                        ) as todays_downloads
                FROM (
                    SELECT file.project as project,
                           DATE(timestamp) AS date,
                           COUNT(*) as downloads
                    FROM `the-psf.pypi.downloads*`
                    WHERE _TABLE_SUFFIX BETWEEN
                        FORMAT_DATE(
                            "%Y%m%d",
                            DATE_ADD(CURRENT_DATE(), INTERVAL -31 day))
                        AND
                        FORMAT_DATE(
                            "%Y%m%d",
                            DATE_ADD(CURRENT_DATE(), INTERVAL -1 day))
                    GROUP BY file.project, date
                )
            )
            GROUP BY project, todays_downloads
            HAVING SUM(downloads) >= 5000
            ORDER BY zscore DESC
        """
    )
query.use_legacy_sql = False
query.run()

[dhermes]

First, thanks for all the work you do on packaging @dstufft!

@jonparrott Any ideas? @dstufft This probably belongs in https://github.com/GoogleCloudPlatform/google-auth-library-python/, but let's discuss here first.

I've seen an issue where an invalid grant was caused by the clock on the local machine being out of sync by 4+ hours with the real world, but even being wrong by 65 minutes (in either direction) would make one of the "issued at" and "expires" timestamps in the JWT be impossible.

[theacodes]

Yeah this is a weird one. I'll try to reproduce.

…

On Mon, Mar 6, 2017, 8:38 AM Danny Hermes ***@***.***> wrote: First, thanks for all the work you do on packaging @dstufft <https://github.com/dstufft>! @jonparrott <https://github.com/jonparrott> Any ideas? @dstufft <https://github.com/dstufft> This probably belongs in https://github.com/GoogleCloudPlatform/google-auth-library-python/, but let's discuss here first. I've seen an issue where an invalid grant was caused by the clock on the local machine being out of sync by 4+ hours with the real world, but even being wrong by 65 minutes (in either direction) would make one of the "issued at" and "expires" timestamps in the JWT be impossible. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3100 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAPUcwogK28lENp134BjxTWz6c0nW4czks5rjDaOgaJpZM4MTr5e> .

[theacodes]

@dstufft I can't reproduce. :(

I can't really debug any further without asking you for sensitive information. Do you mind reaching out to me via email or hangouts? I'm jonwayne at google.com.

[dstufft]

Sure.

[theacodes]

As per email conversation with @dstufft, closing for now.

[hakanak]

Hello,
I have the same problem. I am developing an application with C #.
How did you find a solution?
Thank you.

[liangjacky]

Having the same problem as well on the java library.

trying to do
credential.refreshToken();
String token = credential.getRefreshToken();
and it fails at the first line, giving the exact same error.

Running from the console with the command:
gcloud auth application-default print-access-token
gives the same error as well.

[theacodes]

@liangjacky can you check your system clock? This is a very strange error and is difficult to reproduce.

[liangjacky]

@jonparrott what about my system clock? Is there a specific command I should run? Sorry for being so layman, but I think the time on my macbook is correct?

[theacodes]

Basically just ensuring it's not too far off. Are you getting the exact same error? The code you posted in your comment appears to be java/javascript not Python.

[liangjacky]

I just synced my time with time.apple.com and and it didn't seem to change anything. Thanks for the suggestion though. To answer your question, I'm getting the exact same error.

invalid_grant + invalid JWT signature. Authenticating using the application-default, and a json key on disk.

[liangjacky]

Sorry, it was an issue with the key I was using. Please ignore my comments.