Google Cloud Service Accounts not working with Python Google Cloud Client Libraries

[obulpathi]

Information

OS: Mac OS X El Capitan 10.11.6

Python: Python 2.7.11

Google Cloud:

• Name: google-cloud
• Version: 0.22.0
• Summary: API Client library for Google Cloud
• Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
• Author: Google Cloud Platform
• Author-email: jjg+google-cloud-python@google.com
• License: Apache 2.0
• Location: /Users/ochall/.envs/env/lib/python2.7/site-packages
• Requires: google-cloud-pubsub, google-cloud-runtimeconfig, google-cloud-bigtable, google-cloud-bigquery, google-cloud-core, google-cloud-resource-manager, google-cloud-datastore, google-cloud-logging, google-cloud-translate, google-cloud-storage, google-cloud-happybase, google-cloud-language, google-cloud-dns, google-cloud-vision, google-cloud-error-reporting, google-cloud-monitoring

Google Cloud Datastore

• Name: google-cloud-datastore
• Version: 1.0.0
• Summary: Python Client for Google Cloud Datastore
• Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
• Author: Google Cloud Platform
• Author-email: jjg+google-cloud-python@google.com
• License: Apache 2.0
• Location: /Users/ochall/.envs/env/lib/python2.7/site-packages
• Requires: gapic-google-cloud-datastore-v1, google-cloud-core, google-gax

How to reproduce

My goal is to use service accounts running on-premise infrastructure (A Docker container on my laptop) for accessing Cloud Datastore. I created a Google Cloud service account following the instructions on the page https://cloud.google.com/docs/authentication. I downloaded the credentials file and pointed the GOOGLE_APPLICATION_CREDENTIALS environment variable to that file. Then tried to access Datastore using Python Datastore Library: google-cloud-datastore. I get an error saying that project or resource does not exist or insufficient permissions. My understanding is that Google Cloud Client Libraries does not support Service Accounts. Only Google Cloud API Client supports client libraries. This is a big pain to use Google Cloud Client Libraries, which are so awesome and idiomatic to use. Please enable or fix the usage of service accounts with Google Cloud Client Libraries. Thanks!

The instructions for creating service account

Service running on-premises

To run your code outside Google Cloud Platform, such as in your own private datacenter or in another public cloud, you should use Application Default Credentials with explicitly created service accounts:

Use the Google Cloud Platform Console to create a service account, and download its credentials JSON file to your servers. Make sure to keep the file secure.
Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of your downloaded credentials JSON file.

[dhermes]

@obulpathi We do support service accounts, in fact it's the "primary" auth type we recommend.

I get an error saying that project or resource does not exist or insufficient permissions

It's likely that you project has not yet enabled the Google Cloud Datastore API or that you have a mismatch between your project and your service account.

Have you read our auth doc yet?

[obulpathi]

@dhermes Thanks a ton! I figured out what the problem is. I was creating a service account but was not giving it the service actor role. Once I gave the service account the service actor role and datastore access role, auth is seamless and code is working perfectly. Thanks a ton!

Can you please make it a little bit more explicit in the documentation that users need to assign Service Actor Role to the service account in addition to giving it permissions for accessing services like datastore / bigquery. Or it may be that I am missed that in documentation. I thought that by merely clicking on the "Create service account" and creating an account from Cloud Console gives the permissions to get access tokens.

[dhermes]

I thought that by merely clicking on the "Create service account" and creating an account from Cloud Console gives the permissions to get access tokens.

This should work, it's possible you unclicked something by mistake?

[obulpathi]

You are right! I must have fat fingered something / somewhere. Just creating a service account works perfectly fine. Thanks for the help!