feat(services): remove fxa-auth-db-mysql, fxa-email-event-proxy, and …

…fxa-email-service

and fxa-metrics-processor

.circleci/README.md

@@ -2,13 +2,6 @@
 
 `config.yml` defines the jobs and workflows of our CircleCI deployment.
 
-## Special Cases
-
-`fxa-email-service` isn't tested for most PRs because it doesn't change
-often and is relatively resource intensive. In order to
-trigger these tests the PR branch should be prefixed with `email-service-`.
-PRs that change those packages on other branches will (intentionally) fail.
-
 ## Scripts
 
 This directory contains scripts used by `config.yml` to run jobs. More

.circleci/assert-branch.sh

@@ -8,10 +8,3 @@ if [[ $CIRCLE_BRANCH =~ ^train-.* ]]; then
   echo "Train branch, skipping package check"
   exit 0
 fi
-
-if grep -e 'fxa-email-service' ../packages/test.list; then
-  if [[ ! $CIRCLE_BRANCH =~ ^email-service-.* ]]; then
-    echo "Please create a new PR from a branch name starting with 'email-service-'"
-    exit 1
-  fi
-fi

.circleci/config.yml

@@ -137,14 +137,12 @@ jobs:
             'fxa-admin-panel' \
             'fxa-support-panel' \
             'fxa-event-broker' \
-            'fxa-auth-db-mysql' \
             'fxa-profile-server' \
             '123done' \
             'browserid-verifier' \
             'fortress' \
             'fxa-auth-client' \
             'fxa-geodb' \
-            'fxa-email-event-proxy' \
             'fxa-customs-server' \
             )
             for p in "${PACKAGES[@]}"; do
@@ -195,23 +193,6 @@ jobs:
       - test-content-server-part:
           index: 5
 
-  test-email-service:
-    resource_class: large
-    docker:
-      - image: circleci/rust:latest-node
-      - image: mysql:5.7.27
-        environment:
-          - MYSQL_DATABASE: fxa
-          - MYSQL_ALLOW_EMPTY_PASSWORD: yes
-          - MYSQL_ROOT_PASSWORD: ''
-      - image: redis
-    steps:
-      - base-install
-      - run: cargo install cargo-audit
-      - run: ./packages/fxa-email-service/scripts/test-ci.sh
-      - store_artifacts:
-          path: artifacts
-
   deploy-packages:
     resource_class: small
     docker:
@@ -364,17 +345,6 @@ workflows:
               ignore: main
             tags:
               ignore: /.*/
-      - test-email-service:
-          # since email-service is expensive
-          # to build and rarely changes
-          # we only run it on branches
-          # starting with "email-service-"
-          filters:
-            branches:
-              only:
-                - /^email-service-.*/
-            tags:
-              ignore: /.*/
       - build-and-deploy-storybooks:
           filters:
             branches:

.circleci/modules-to-test.js

@@ -68,9 +68,7 @@ async function modulesToSkip(org, repo, prNumber, branch) {
       `https://api.github.com/repos/${org}/${repo}/issues/${prNumber}/labels`
     );
     const labels = await labelRes.json();
-    const skipModules = branch.startsWith('email-service')
-      ? []
-      : ['fxa-email-service'];
+    const skipModules = [];
     if (Array.isArray(labels) && labels.find((l) => l.name === '🙈 skip ci')) {
       const skipLabels = labels
         .filter((l) => l.name.startsWith('fxa-'))

.dockerignore

@@ -12,6 +12,4 @@
 packages/*/build
 packages/fxa-auth-server/.mail_output
 packages/fxa-dev-launcher
-packages/fxa-email-event-proxy
-packages/fxa-email-service
 packages/fxa-content-server/dist

.github/.codecov.yaml

@@ -38,7 +38,6 @@ flags:
   many:
     paths:
       - packages/fxa-admin-panel
-      - packages/fxa-auth-db-mysql
       - packages/fxa-customs-server
       - packages/fxa-payments-server
       - packages/fxa-profile-server

.gitignore

@@ -75,12 +75,7 @@ Thumbs.db
 packages/browserid-verifier/loadtest/venv
 packages/browserid-verifier/loadtest/*.pyc
 
-# fxa-auth-db-mysql
-packages/fxa-auth-db-mysql/config/dev.js
-packages/fxa-auth-db-mysql/sandbox
-
 # fxa-auth-server
-packages/fxa-auth-server/fxa-auth-db-mysql
 packages/fxa-auth-server/sandbox
 packages/fxa-auth-server/config/key.json
 packages/fxa-auth-server/config/oldKey.json
@@ -114,16 +109,6 @@ packages/fxa-content-server/.tscompiled
 # fxa-customs-server
 packages/fxa-customs-server/test/mocks/temp.netset
 
-# fxa-email-event-proxy
-packages/fxa-email-event-proxy/*.zip
-
-# fxa-email-service
-packages/fxa-email-service/**/*.rs.bk
-packages/fxa-email-service/config/local.*
-packages/fxa-email-service/fxa-auth-db-mysql
-packages/fxa-email-service/target
-packages/fxa-email-service/.sourcehash
-
 # fxa-payments-server
 packages/fxa-payments-server/fxa-content-server-l10n/
 

_dev/docker/docker-compose.yml

@@ -20,21 +20,6 @@ services:
     init: true
     ports:
       - "5050:5050"
-  authdb:
-    image: fxa-auth-db-mysql:build
-    entrypoint: /bin/bash -c
-    command: ["/fxa/_scripts/check-mysql.sh 3306 mysql && node bin/db_patcher.js && node bin/server.js"]
-    environment:
-      - HOST=0.0.0.0
-      - PORT=8000
-      - NODE_ENV=dev
-      - MYSQL_HOST=mysql
-      - MYSQL_SLAVE_HOST=mysql
-    init: true
-    ports:
-      - "8000:8000"
-    depends_on:
-      - mysql
   auth:
     image: fxa-auth-server:build
     entrypoint: /bin/bash -c
@@ -156,11 +141,3 @@ services:
     image: jdlk7/firestore-emulator
     ports:
       - "9090:9090"
-  email:
-    image: mozilla/fxa-email-service
-    environment:
-      - NODE_ENV=dev
-      - FXA_EMAIL_ENV=dev
-      - FXA_EMAIL_LOG_LEVEL=debug
-    ports:
-      - "8001:8001"

_scripts/base-docker.sh

@@ -39,7 +39,6 @@ npx yarn workspaces focus --production \
   browserid-verifier \
   fxa-admin-panel \
   fxa-admin-server \
-  fxa-auth-db-mysql \
   fxa-auth-server \
   fxa-content-server \
   fxa-customs-server \

_scripts/check-ports.sh

@@ -8,7 +8,6 @@ PORTS=(
   8085 # google-pubsub-emulator
   9090 # google-firestore-emulator
   5000 # sync server
-  8001 # email-service
   8000 # auth-server db mysql
   9000 # auth-server key server
   3030 # content-server

_scripts/gh-pages.sh

@@ -9,9 +9,6 @@ fi
 
 echo "Building docs."
 
-cd packages/fxa-email-service
-cargo doc --no-deps
-
 cd ../../packages/fxa-payments-server
 yarn workspaces focus fxa-payments-server
 npm run build-storybook
@@ -21,9 +18,6 @@ git clone --branch gh-pages git@github.com:mozilla/fxa.git docs-build
 
 cd docs-build
 
-rm -rf ./fxa-email-service
-mv ../packages/fxa-email-service/target/doc fxa-email-service
-
 rm -rf ./fxa-payments-server
 mv ../packages/fxa-payments-server/storybook-static fxa-payments-server
 

_scripts/test-package.sh

@@ -6,7 +6,6 @@ Usage: npm test [all|<package-name>*]
 examples:
 
 npm test fxa-shared
-npm test fxa-auth-db-mysql fxa-auth-server
 npm test all
 "
 

docs/adr/0028-retiring-email-service.md

@@ -0,0 +1,55 @@
+# Retiring fxa-email-service
+
+- Deciders: Danny Coates, Vijay Budhram, Jon Buckley
+- Date: 2021-10-06
+
+## Context and Problem Statement
+
+The goal of fxa-email-service was to spin off the email sending responsibilities of FxA into a shared service that multiple Mozilla projects could use. From its readme:
+
+> The FxA team had an OKR for Q2 2018 about decoupling the auth server from SES and making it possible to send email via different providers. Subsequently, some other teams expressed an interest in depending on a standalone email service too.
+
+> This repo started as our experiment to see what a decoupled email-sending service would look like, written in Rust. It is now handling all FxA email traffic in production, and we are gradually separating it from the FxA stack to run as a standalone service in its own right.
+
+Had it achieved the goal of fully decoupling from FxA and being more widely used we'd likely continue using it. However, in the approximately 4 years of its existance it has only really been used as an intermediate step between auth-server and SES. Fortunately in that time it hasn't required much maintenance so its "weight", being a fairly large codebase for what it does and our only Rust service, has never been a concern. Recent work to eliminate fxa-auth-db-mysql meant we either needed make changes to it or rethink how FxA sends email. It turned out the work to update email-service was larger than eliminating it and replacing it by sending email directly from auth-server via SES or SMTP.
+
+## Decision Drivers
+
+- Future maintenance
+- Email provider flexability
+
+## Considered Options
+
+- Keep email-service and update db access
+- Eliminate email-service and send directly from auth-server
+
+## Decision Outcome
+
+We will eliminate email-service.
+
+## Pros and Cons of Options
+
+### Keep email-service
+
+Pros:
+
+- Change is scary
+
+Cons:
+
+- More code to maintain
+- Less team experience with Rust
+
+### Eliminate email-service
+
+Pros:
+
+- Significantly reduces our code footprint
+- No Rust simplifies our build process
+- Future email work will be easier to implement and maintain in JS/TS
+
+Cons:
+
+- Lose (at least temporarily) the ability to use multiple email providers simultaneously
+  - We've never used this beyond testing so far
+  - We could add it to auth-server if required