Authenticating with a google cloud service account

[sertys3]

Greetings,
thank you for the wonderful work thus far on the project. I have a case where I'd would like to authenticate with a service account for the users in a Google Workspace account. From what I am seeing such an account would not require an authorization on the web and the credentials come in a .json file. It has an embedded certificate instead of a secret.

{
  "type": "service_account",
  "project_id": "imap-proxy-*****",
  "private_key_id": "2e9bdef***********************",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQI*************************************************\n-----END PRIVATE KEY-----\n",
  "client_email": "domain-wide-service-account@imap-proxy-******.iam.gserviceaccount.com",
  "client_id": "11***************************",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/domain-wide-service-account%40imap-proxy-***********.iam.gserviceaccount.com",
  "universe_domain": "googleapis.com"
}

Has that been implemented as a use case already and I'm missing something or there would be a different OAuth2 path for authorizing the token to access the accounts in the Workspace?

Regards,
Daniel

[sertys3]

I have been able to produce a working solution with a token that was "sniffed" by GYB with a service account file and voiding the encryption/decryption of access_token config variables in the code. Thus I could place then generated token in plaintext in the config. And authentication works as expected.
Now I have 2 options - inspect the GYB way of taking the token and implementing this in your project or have a script that gets a valid token and populates the config file for email accounts of the Workspace.

EDIT:
That was fast. Using the google.oauth2 it is easy to generate a valid access token:

import os
import os.path
import json
import httplib2
import google.oauth2.service_account
import google_auth_oauthlib.flow
import google_auth_httplib2
import google.oauth2.id_token

def _createHttpObj(cache=None, timeout=600):
  http_args = {'cache': cache, 'timeout': timeout}

  http_args['tls_maximum_version'] = 'TLSv1_3'

  http_args['tls_minimum_version'] = 'TLSv1_2'
  return httplib2.Http(**http_args)

def readFile(filename, mode='r', continueOnError=False, displayError=True, encoding=None):
  try:
    if filename != '-':
      if not encoding:
        with open(os.path.expanduser(filename), mode) as f:
          return f.read()
      with codecs.open(os.path.expanduser(filename), mode, encoding) as f:
        content = f.read()
# codecs does not strip UTF-8 BOM (ef:bb:bf) so we must
        if not content.startswith(codecs.BOM_UTF8):
          return content
        return content[3:]
    return unicode(sys.stdin.read())
  except IOError as e:
    if continueOnError:
      if displayError:
        sys.stderr.write(str(e))
      return None
    systemErrorExit(6, e)
  except (LookupError, UnicodeDecodeError, UnicodeError) as e:
    systemErrorExit(2, str(e))

def getSvcAcctCredentials(scopes, act_as):
  try:
    json_string = readFile('./oauth2service.json', continueOnError=True, displayError=True)
    if not json_string:
      print(MESSAGE_INSTRUCTIONS_OAUTH2SERVICE_JSON)
      systemErrorExit(6, None)
    sa_info = json.loads(json_string)
    credentials = google.oauth2.service_account.Credentials.from_service_account_info(sa_info)
    credentials = credentials.with_scopes(scopes)
    credentials = credentials.with_subject(act_as)
    request = google_auth_httplib2.Request(_createHttpObj())
    credentials.refresh(request)
    return credentials
  except (ValueError, KeyError):
    print(MESSAGE_INSTRUCTIONS_OAUTH2SERVICE_JSON)
    systemErrorExit(6, 'oauth2service.json is invalid.')

API_SCOPE_MAPPING = {
  'email': ['https://www.googleapis.com/auth/userinfo.email'],
  'drive': ['https://www.googleapis.com/auth/drive.appdata',],
  'gmail': ['https://mail.google.com/',],
  'groupsmigration': ['https://www.googleapis.com/auth/apps.groups.migration',],
}
creds = getSvcAcctCredentials(['https://mail.google.com/'],'account@google-workspace.tld')
print(vars(creds))

I would think that would be trivial to integrate in the existing codebase.

Regards,
Daniel

[simonrob]

Thanks for this suggestion and your work so far. It looks like you've managed to do pretty much everything that would be required.

Regarding integration, are you suggesting that the proxy's configuration file is extended to be able to point to a separate service account JSON file which it uses to authenticate account access?

[sertys3]

Great to see such a fast response. Adding a config option for this would be awesome. The requirements-core.txt would also need to be updated or a different file created as to install the necessary modules.
Also in this case, I'm using domain catch-all configuration which makes the most sense for provisioning a service account. Hence it would be pretty straightforward to plug into the appropriate calls for generating/refreshing and saving the access token. I would be more than happy to assist with testing the case as well.

Kindest regards,
Daniel

[simonrob]

Thanks for the extra detail – I'll take a look at this when I get chance, though it may be a while depending on availability. I'm happy to look at adding this feature on a consultancy basis if it needs prioritising.

[sertys3]

That is a splendid suggestion. Even though we will not be having a direct application for this, I am sure my company would be willing to dole out a contribution to the wonderful work you are doing. Drop me an email at sertys[at]gmail.com so we arrange this. Regards

…

On Mon, Dec 18, 2023, 7:00 PM Simon Robinson ***@***.***> wrote: Thanks for the extra detail – I'll take a look at this when I get chance, though it may be a while depending on availability. I'm happy to look at adding this feature on a consultancy basis <https://github.com/sponsors/simonrob?frequency=one-time&sponsor=simonrob> if it needs prioritising. — Reply to this email directly, view it on GitHub <#212 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALSALWOFFWOASWFAPGAZELDYKBZERAVCNFSM6AAAAABAUUVD2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRRGA3DGMZWGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[simonrob]

This mode has now been merged into the proxy. Please carefully read the documentation, as just like the client credentials grant flow (CCG), this feature essentially means that there is no access control when using the proxy in certain environments, and it is very important to be aware of this before using it in a publicly accessible context.

The following script will create a pre-encrypted proxy configuration entry:

import base64
import getpass
import os

import prompt_toolkit
from cryptography.fernet import Fernet
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC

print('\nThis script creates a pre-encrypted Email OAuth 2.0 Proxy configuration file entry for accounts using a '
      'Google Cloud service account with Google Workspace Gmail.\nFor more information about configuring the service '
      'account itself, see: https://cloud.google.com/iam/docs/service-account-overview and (for domain-wide'
      'delegation) https://admin.google.com/u/4/ac/owl/domainwidedelegation\n')
username = input('Enter your email address: ')
password = getpass.getpass('Enter the password you will use for IMAP/POP/SMTP access: ')
client_id = None
while client_id not in ['file', 'key']:
    client_id = input('Will you use a path to a service account JSON key _file_ or paste the contents of the _key_? '
                      'Enter `file` or `key`: ')
if client_id == 'file':
    client_secret = input('Enter the full path to your service account JSON key file: ')
else:
    client_secret = prompt_toolkit.prompt('Paste the entire content of your service account JSON key file: ',
                                          is_password=True)

token_salt = base64.b64encode(os.urandom(16)).decode('utf-8')
token_iterations = 870_000
key_derivation_function = PBKDF2HMAC(algorithm=hashes.SHA256(), length=32,
                                     salt=base64.b64decode(token_salt.encode('utf-8')), iterations=token_iterations,
                                     backend=default_backend())
fernet = Fernet(base64.urlsafe_b64encode(key_derivation_function.derive(password.encode('utf-8'))))

print('\n[%s]' % username)
print('token_url = https://oauth2.googleapis.com/token')
print('oauth2_scope = https://mail.google.com/')
print('oauth2_flow = service_account')
print('redirect_uri = http://localhost')
print('client_id =', client_id)
print('client_secret_encrypted =', fernet.encrypt(client_secret.encode('utf-8')).decode('utf-8'))
print('token_salt =', token_salt)
print('token_iterations =', token_iterations, '\n')

[johspa933]

First off, thank you for the great work on the project.
I was in need of a Google Oauth2 method to use my older applications and found this would work perfectly for my needs. I wanted to have an "auto-auth" method so I created the Service_Account on my Google Workspace and generated the JSON file. Using the JSON file I generated the pre-generated keys for each user and It is working perfectly except after about an hour, it stops working and will no longer accept authentication. It says "Accepting new connections..." but fails... It is happening with all the services I have configured which is IMAP and POP with the same result. The current resolution is to stop the program, remove the cache-store file and restart the program.

I have each service on a different physical box and the command to start the app and configurations are listed below:

Command:

python3 emailproxy.py --no-gui --config-file emailproxy.config --cache-store cache-store --log-file emailproxy.log

PC A

[Server setup]

[IMAP-993]
server_address = imap.gmail.com
server_port = `993`
local_address = <ipaddr>

[Account setup]

[user@domain.aaa]
token_url = https://oauth2.googleapis.com/token
oauth2_scope = https://mail.google.com/
oauth2_flow = service_account
redirect_uri = http://localhost
client_id = file
client_secret_encrypted = <secret>
token_salt = <salt>
token_iterations = 870000 

[Advanced proxy configuration]

[emailproxy]
delete_account_token_on_password_error = False
encrypt_client_secret_on_first_use = True
allow_catch_all_accounts = False

PC B

[Server setup]

[POP-995]
server_address = pop.gmail.com
server_port = 995
local_address = <ipaddr>

[Account setup]

[user2@domain.aaa]
token_url = https://oauth2.googleapis.com/token
oauth2_scope = https://mail.google.com/
oauth2_flow = service_account
redirect_uri = http://localhost
client_id = file
client_secret_encrypted = <secret>
token_salt = <salt>
token_iterations = 870000 

[Advanced proxy configuration]

[emailproxy]
delete_account_token_on_password_error = False
encrypt_client_secret_on_first_use = True
allow_catch_all_accounts = False

Please let me know if there is something I may be missing, doing wrong, or additional steps that may be required to help understand the issue and help to solve the problem.

Regards,
Johspa

[sertys3]

You are using oauth2 wrong as you should let the flow renew your tokens whenever they expire. So point the config to your service token .json and let it handle the token generation. Daniel

…

On Wed, Jan 24, 2024, 5:25 AM johspa933 ***@***.***> wrote: First off, thank you for the great work on the project. I was in need of a Google Oauth2 method to use my older applications and found this would work perfectly for my needs. I wanted to have an "auto-auth" method so I created the Service_Account on my Google Workspace and generated the JSON file. Using the JSON file I generated the pre-generated keys for each user and It is working perfectly except after about an hour, it stops working and will no longer accept authentication. It says "Accepting new connections..." but fails... It is happening with all the services I have configured which is IMAP and POP with the same result. The current resolution is to stop the program, remove the cache-store file and restart the program. I have each service on a different physical box and the command to start the app and configurations are listed below: Command: python3 emailproxy.py --no-gui --config-file emailproxy.config --cache-store cache-store --log-file emailproxy.log PC A [Server setup] [IMAP-993] server_address = imap.gmail.com server_port = `993` local_address = <ipaddr> [Account setup] ***@***.*** token_url = https://oauth2.googleapis.com/token oauth2_scope <https://oauth2.googleapis.com/tokenoauth2_scope> = https://mail.google.com/ oauth2_flow <https://mail.google.com/oauth2_flow> = service_account redirect_uri = http://localhost client_id = file client_secret_encrypted = <secret> token_salt = <salt> token_iterations = 870000 [Advanced proxy configuration] [emailproxy] delete_account_token_on_password_error = False encrypt_client_secret_on_first_use = True allow_catch_all_accounts = False PC B [Server setup] [POP-995] server_address = pop.gmail.com server_port = 995 local_address = <ipaddr> [Account setup] ***@***.*** token_url = https://oauth2.googleapis.com/token oauth2_scope <https://oauth2.googleapis.com/tokenoauth2_scope> = https://mail.google.com/ oauth2_flow <https://mail.google.com/oauth2_flow> = service_account redirect_uri = http://localhost client_id = file client_secret_encrypted = <secret> token_salt = <salt> token_iterations = 870000 [Advanced proxy configuration] [emailproxy] delete_account_token_on_password_error = False encrypt_client_secret_on_first_use = True allow_catch_all_accounts = False Please let me know if there is something I may be missing, doing wrong, or additional steps that may be required to help understand the issue and help to solve the problem. Regards, Johspa — Reply to this email directly, view it on GitHub <#212 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALSALWIE4I255H3VORWVRW3YQB5LXAVCNFSM6AAAAABAUUVD2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBXGI4TENZXGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>