Skip to content

Commit

Permalink
Re-add SMTP email provider (lucyparsons#1047)
Browse files Browse the repository at this point in the history
  • Loading branch information
sea-kelp committed Oct 5, 2023
1 parent 08ee32c commit 2c0af64
Show file tree
Hide file tree
Showing 19 changed files with 521 additions and 75 deletions.
26 changes: 21 additions & 5 deletions CONTRIB.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,17 +49,33 @@ $ docker exec -it openoversight_web_1 /bin/bash

Once you're done, `make stop` and `make clean` to stop and remove the containers respectively.

## Gmail Requirements
**NOTE:** If you are running on dev and do not currently have a `service_account_key.json` file, create one and leave it empty. The email client will then default to an empty object and simulate emails in the logs.

For the application to work properly, you will need a [Google Cloud Platform service account](https://cloud.google.com/iam/docs/service-account-overview) that is attached to a GSuite email address. Here are some general tips for working with service accounts: [Link](https://support.google.com/a/answer/7378726?hl=en).
## Setting Up Email
OpenOversight tries to auto-detect which email implementation to use based on which of the following is configured (in this order):
* Google: `service_account_key.json` exists and is not empty
* SMTP: `MAIL_SERVER` and `MAIL_PORT` environment variables are set
* Simulated: If neither of the previous 2 implementations are configured, emails will only be logged

### GSuite
To send email using a GSuite email account, you will need a [Google Cloud Platform service account](https://cloud.google.com/iam/docs/service-account-overview) that is attached to that email address. Here are some general tips for working with service accounts: [Link](https://support.google.com/a/answer/7378726?hl=en).
We would suggest that you do not use a personal email address, but instead one that is used strictly for sending out OpenOversight emails.

You will need to do these two things for the service account to work as a Gmail bot:
1. Enable domain-wide delegation for the service account: [Link](https://support.google.com/a/answer/162106?hl=en)
2. Enable the `https://www.googleapis.com/auth/gmail.send` scope in the Gmail API for your service account: [Link](https://developers.google.com/gmail/api/auth/scopes#scopes)
3. Save the service account key file in OpenOversight's base folder as `service_account_key.json`. The file is in the `.gitignore` file GitHub will not allow you to save it, provided you've named it correctly.
4. For production, save the email address associated with your service account to a variable named `OO_SERVICE_EMAIL` in a `.env` file in the base directory of this repository. For development and testing, update the `OO_SERVICE_EMAIL` variable in the `docker-compose.yml` file.

### SMTP
To send email using SMTP, set the following environment variables in your docker-compose.yml file or .env file:
* `MAIL_SERVER`
* `MAIL_PORT`
* `MAIL_USE_TLS`
* `MAIL_USERNAME`
* `MAIL_PASSWORD`

For more information about these settings, please see the [Flask-Mail](https://flask-mail.readthedocs.io/en/latest/) documentation.

### Setting email aliases
Regardless of implementation, save the email address associated with your service account to a variable named `OO_SERVICE_EMAIL` in a `.env` file in the base directory of this repository. For development and testing, update the `OO_SERVICE_EMAIL` variable in the `docker-compose.yml` file.

Example `.env` variable:
```bash
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ assets:
docker-compose run --rm web yarn build

.PHONY: dev
dev: build start create_db populate
dev: create_empty_secret build start create_db populate

.PHONY: populate
populate: create_db ## Build and run containers
Expand Down
12 changes: 3 additions & 9 deletions OpenOversight/app/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from OpenOversight.app.email_client import EmailClient
from OpenOversight.app.models.config import config
from OpenOversight.app.models.database import db
from OpenOversight.app.utils.constants import MEGABYTE, SERVICE_ACCOUNT_FILE
from OpenOversight.app.utils.constants import MEGABYTE


bootstrap = Bootstrap()
Expand All @@ -41,14 +41,8 @@ def create_app(config_name="default"):
bootstrap.init_app(app)
csrf.init_app(app)
db.init_app(app)
# This allows the application to run without creating an email client if it is
# in testing or dev mode and the service account file is empty.
service_account_file_size = os.path.getsize(SERVICE_ACCOUNT_FILE)
EmailClient(
config=app.config,
dev=app.debug and service_account_file_size == 0,
testing=app.testing,
)
with app.app_context():
EmailClient()
limiter.init_app(app)
login_manager.init_app(app)
sitemap.init_app(app)
Expand Down
173 changes: 142 additions & 31 deletions OpenOversight/app/email_client.py
Original file line number Diff line number Diff line change
@@ -1,54 +1,165 @@
from apiclient import errors
import base64
import os.path
from abc import ABC, abstractmethod
from threading import Thread
from typing import Optional, Self

from flask import current_app
from flask_mail import Mail, Message
from google.oauth2 import service_account
from googleapiclient.discovery import build

from OpenOversight.app.models.emails import Email
from OpenOversight.app.utils.constants import SERVICE_ACCOUNT_FILE
from OpenOversight.app.utils.constants import (
KEY_MAIL_PORT,
KEY_MAIL_SERVER,
KEY_OO_HELP_EMAIL,
KEY_OO_SERVICE_EMAIL,
SERVICE_ACCOUNT_FILE,
)


class EmailClient(object):
"""
EmailClient is a Singleton class that is used for the Gmail client.
This can be fairly easily switched out with another email service, but it is
currently defaulted to Gmail.
"""
class EmailProvider(ABC):
"""Base class to define how emails are sent."""

@abstractmethod
def start(self):
"""Set up the email provider."""

@abstractmethod
def is_configured(self) -> bool:
"""Determine the required env variables for this provider are configured."""

@abstractmethod
def send_email(self, email: Email):
"""Send an email with this email provider."""


class GmailEmailProvider(EmailProvider):
"""Sends email through Gmail using the Google API client."""

SCOPES = ["https://www.googleapis.com/auth/gmail.send"]

_instance = None
def start(self):
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=self.SCOPES
)
delegated_credentials = credentials.with_subject(
current_app.config[KEY_OO_SERVICE_EMAIL]
)
self.service = build("gmail", "v1", credentials=delegated_credentials)

def is_configured(self) -> bool:
return (
os.path.isfile(SERVICE_ACCOUNT_FILE)
and os.path.getsize(SERVICE_ACCOUNT_FILE) > 0
)

def send_email(self, email: Email):
message = email.create_message()
resource = {"raw": base64.urlsafe_b64encode(message.as_bytes()).decode()}

(self.service.users().messages().send(userId="me", body=resource).execute())


class SMTPEmailProvider(EmailProvider):
"""Sends email with SMTP using Flask-Mail."""

def start(self):
self.mail = Mail(current_app)

def is_configured(self) -> bool:
return bool(
current_app.config.get(KEY_MAIL_SERVER)
and current_app.config.get(KEY_MAIL_PORT)
)

def send_email(self, email: Email):
app = current_app._get_current_object()
msg = Message(
email.subject,
sender=app.config[KEY_OO_SERVICE_EMAIL],
recipients=[email.receiver],
reply_to=app.config[KEY_OO_HELP_EMAIL],
)
msg.body = email.body
msg.html = email.html

thread = Thread(
target=SMTPEmailProvider.send_async_email,
args=[self.mail, app, msg],
)
current_app.logger.info("Sent email.")
thread.start()

@staticmethod
def send_async_email(mail: Mail, app, msg: Message):
with app.app_context():
mail.send(msg)

def __new__(cls, config=None, dev=False, testing=False):
if (testing or dev) and cls._instance is None:
cls._instance = {}

if cls._instance is None and config:
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=cls.SCOPES
class SimulatedEmailProvider(EmailProvider):
"""Writes messages sent with this provider to log for dev/test usage."""

def start(self):
if not current_app.debug and not current_app.testing:
current_app.logger.warning(
"Using simulated email provider in non-development environment. "
"Please see CONTRIB.md to set up a email provider."
)

def is_configured(self):
return True

def send_email(self, email: Email):
current_app.logger.info("simulated email:\n%s\n%s", email.subject, email.body)


class EmailClient:
"""
EmailClient is a Singleton class used for sending email. It auto-detects
the email provider implementation based on whether the required
configuration is provided for each implementation.
"""

DEFAULT_PROVIDER: EmailProvider = SimulatedEmailProvider()
PROVIDER_PRECEDENCE: list[EmailProvider] = [
GmailEmailProvider(),
SMTPEmailProvider(),
DEFAULT_PROVIDER,
]

_provider: Optional[EmailProvider] = None
_instance: Optional[Self] = None

def __new__(cls):
if cls._instance is None:
cls._provider = cls.auto_detect()
cls._provider.start()
current_app.logger.info(
f"Using email provider: {cls._provider.__class__.__name__}"
)
delegated_credentials = credentials.with_subject(config["OO_SERVICE_EMAIL"])
cls.service = build("gmail", "v1", credentials=delegated_credentials)
cls._instance = super(EmailClient, cls).__new__(cls)
return cls._instance

@classmethod
def auto_detect(cls):
"""Auto-detect the configured email provider to use for email sending."""
if current_app.testing:
return cls.DEFAULT_PROVIDER

for provider in cls.PROVIDER_PRECEDENCE:
if provider.is_configured():
return provider

raise ValueError("No configured email providers")

@classmethod
def send_email(cls, email: Email):
"""
Deliver the email from the parameter list using the Singleton client.
:param email: the specific email to be delivered
"""
if not cls._instance:
current_app.logger.info(
"simulated email:\n%s\n%s", email.subject, email.body
)
else:
try:
(
cls.service.users()
.messages()
.send(userId="me", body=email.create_message())
.execute()
)
except errors.HttpError as error:
print("An error occurred: %s" % error)
if cls._provider is not None:
cls._provider.send_email(email)
20 changes: 18 additions & 2 deletions OpenOversight/app/models/config.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
import os

from OpenOversight.app.utils.constants import MEGABYTE
from OpenOversight.app.utils.constants import (
KEY_MAIL_PASSWORD,
KEY_MAIL_PORT,
KEY_MAIL_SERVER,
KEY_MAIL_USE_TLS,
KEY_MAIL_USERNAME,
KEY_OO_HELP_EMAIL,
MEGABYTE,
)
from OpenOversight.app.utils.general import str_is_true


basedir = os.path.abspath(os.path.dirname(__file__))
Expand Down Expand Up @@ -39,7 +48,14 @@ def __init__(self):
self.OO_SERVICE_EMAIL = os.environ.get("OO_SERVICE_EMAIL")
# TODO: Remove the default once we are able to update the production .env file
# TODO: Once that is done, we can re-alpha sort these variables.
self.OO_HELP_EMAIL = os.environ.get("OO_HELP_EMAIL", self.OO_SERVICE_EMAIL)
self.OO_HELP_EMAIL = os.environ.get(KEY_OO_HELP_EMAIL, self.OO_SERVICE_EMAIL)

# Flask-Mail-related settings
setattr(self, KEY_MAIL_SERVER, os.environ.get(KEY_MAIL_SERVER))
setattr(self, KEY_MAIL_PORT, os.environ.get(KEY_MAIL_PORT))
setattr(self, KEY_MAIL_USE_TLS, str_is_true(os.environ.get(KEY_MAIL_USE_TLS)))
setattr(self, KEY_MAIL_USERNAME, os.environ.get(KEY_MAIL_USERNAME))
setattr(self, KEY_MAIL_PASSWORD, os.environ.get(KEY_MAIL_PASSWORD))

# AWS Settings
self.AWS_ACCESS_KEY_ID = os.environ.get("AWS_ACCESS_KEY_ID")
Expand Down
Loading

0 comments on commit 2c0af64

Please sign in to comment.