Google Cloud Platform (GCP) Machine Learning Engineering & MLOps Portfolio

A curated collection of production-grade, reproducible Machine Learning Engineering and MLOps projects built on Google Cloud Platform (GCP). Each project is designed as an independent, fully tested, and portfolio-ready system adhering to industry standards for reproducibility, Infrastructure as Code (IaC), quality gates, and data lineage.

---

🚀 Projects Overview

Project	Domain / Problem	GCP Services & Tools	Model / Tech Stack	Key MLOps Practices
speech-to-text-mlops	Audio transcription evaluation & quality assurance	Cloud Speech-to-Text API, Cloud Storage, Terraform	Managed STT, Python 3.11+, Pytest, Ruff, Mypy	Word Error Rate (WER), slice evaluation, executable quality gates, immutable run artifacts, least-privilege IAM
video-intelligence-metadata	Video segment annotation & catalog indexing	Cloud Video Intelligence API, Cloud Storage, Terraform	Managed Video Intelligence, Python CLI, Pydantic	Normalized schema contract, acceptance canaries, quality gate enforcement, automatic resource teardown
bracketology-bigquery-ml	NCAA tournament matchup outcome & probability estimation	BigQuery, BigQuery ML, Terraform	LOGISTIC_REG, SQL, Python CLI	Temporal holdout splits, score leakage prevention, dry-run SQL generation, calibrated probability scoring
bigquery-ml-taxi-fare-forecasting	NYC taxi fare regression & route analysis	BigQuery, BigQuery ML, Cloud Storage	BOOSTED_TREE_REGRESSOR, SQL, Python	Feature engineering (distance, temporal), deterministic sampling, offline SQL rendering, metrics tracking
bigquery-ml-visitor-purchases	E-commerce conversion & purchase intent classification	BigQuery, BigQuery ML, Google Analytics	LOGISTIC_REG (Class Weights), SQL, Python	Temporal validation split, auditable SQL pipeline, classification metrics & confusion matrix evaluation

---

🏛️ Engineering & Architecture Principles

Every project in this repository adheres to high-standard software and ML engineering patterns:

1. Infrastructure as Code (IaC):

   • Modular, reproducible cloud infrastructure defined with Terraform.
   • Automatic resource teardown policies and isolated environments.

2. Quality Gates & Benchmarks:

   • Explicit promotion rules comparing candidate models/APIs against established baselines.
   • Slice-based evaluations (e.g., locale, category, duration) to detect localized degradation.

3. Data Integrity & Lineage:

   • Strict avoidance of data leakage (temporal train/test splits, reference withholding).
   • Immutable artifact storage with full provenance (runtime configs, inputs, metrics, and decisions).

4. Code Quality & Developer Experience:

   • Strong static typing with Mypy.
   • High-performance linting and formatting with Ruff.
   • Comprehensive test suites with Pytest (unit tests, contract tests, and mocked cloud boundaries).

5. Responsible AI & Operational Safety:

   • Clear documentation of model limitations, failure modes, fairness considerations, and monitoring strategies.

---

📋 Prerequisites & Local Setup

General Requirements

• Python: 3.11+
• Terraform: 1.6+ / 1.7+
• Google Cloud SDK (gcloud CLI)
• A billing-enabled Google Cloud Project

Authentication & Setup

Authenticate your local environment with Application Default Credentials (ADC):

gcloud auth login
gcloud auth application-default login
gcloud config set project <YOUR_GCP_PROJECT_ID>

---

⚡ Quickstart Guide

Navigate to any subproject directory to set up the local environment and run checks:

Example: Running a BigQuery ML Project

cd bigquery-ml-taxi-fare-forecasting

# 1. Setup virtual environment & install dependencies
python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'

# 2. Run local tests and linters (no GCP billing incurred)
pytest tests/
ruff check src tests
mypy src

# 3. Render SQL queries offline
bqml-taxi-fare render --project-id <YOUR_GCP_PROJECT_ID>

# 4. (Optional) Run end-to-end pipeline in GCP
GCP_PROJECT_ID=<YOUR_GCP_PROJECT_ID> ./run.sh

Example: Running an MLOps Evaluation Project

cd speech-to-text-mlops

# 1. Setup environment
python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'

# 2. Run tests and Terraform validation
pytest tests/
terraform -chdir=terraform init -backend=false
terraform -chdir=terraform validate

# 3. Execute quality-gated evaluation
GCP_PROJECT_ID=<YOUR_GCP_PROJECT_ID> ./run.sh

---

📂 Repository Structure

.
├── .github/
│   └── machine-learning-engineering/   # Repository skill definitions & engineering blueprints
├── bigquery-ml-taxi-fare-forecasting/  # NYC Taxi Fare Regression with BigQuery ML
├── bigquery-ml-visitor-purchases/     # Visitor Purchase Classification with BigQuery ML
├── bracketology-bigquery-ml/          # NCAA Tournament Prediction with BigQuery ML
├── speech-to-text-mlops/              # Quality-Gated Speech-to-Text Evaluation Pipeline
├── video-intelligence-metadata/       # Video Metadata Extraction & Acceptance Pipeline
└── README.md                          # Root documentation (this file)

---

🛡️ License & Responsible Use

All datasets used across these projects are open/public documentation samples (Google Cloud public datasets, BigQuery public datasets). Refer to each subproject's README.md for specific source citations, terms of use, and operational constraints.