I like problems where the goal is not simply to train a model, but to build a reliable path from:

Raw Data → Analysis → Modeling → Evaluation → Decision

My work currently focuses on:

• Large-scale Data Science — turning millions of records into decision-ready analytics
• Statistical Experimentation — measuring treatment effects with uncertainty and practical significance
• ML Reliability — identifying data, leakage, imbalance, and modeling risks before training
• Applied AI — combining deterministic analytics with retrieval, reranking, and grounded generation

Evidence before claims · Baselines before complexity · Limitations documented, not hidden

Large-Scale Analytics · NLP · Forecasting

Built an end-to-end intelligence platform on the CFPB Consumer Complaint dataset to transform a very large raw dataset into analytical, forecasting, NLP, and risk signals for decision-making.

Key Evidence

• Processed 15.95M complaint records from an 8–9 GB raw CSV
• Built chunked preprocessing and reusable Parquet-based analytical storage
• Developed product, issue, company, geography, response, and narrative analytics
• Trained TF-IDF + Logistic Regression models for complaint routing
• Achieved 3.57% MAPE on a documented six-month forecasting holdout
• Added risk scoring, growth analysis, forecasting, and 14/14 core unit tests

Stack Python · Pandas · PyArrow · scikit-learn · Prophet · Streamlit · Docker

Repository · Demo · Evaluation

Statistics · Experimentation · Business Decision-Making

Analyzed a controlled marketing experiment to determine whether advertising produced a meaningful conversion improvement over a PSA control group.

Key Evidence

• Evaluated 588,101 users
• Advertisement conversion: 2.5547%
• PSA conversion: 1.7854%
• Absolute uplift: +0.7692 percentage points
• Relative uplift: +43.09%
• 95% uplift interval: +0.5951 to +0.9434 pp
• Approximately 130 users per additional conversion
• Used hypothesis testing, confidence intervals, effect sizes, simulation, logistic-regression consistency checks, and power analysis

Decision

The experiment provides strong evidence of higher conversion under advertising, while ROI is intentionally not claimed without campaign-cost and customer-value inputs.

Stack Python · Pandas · SciPy · Statsmodels · Matplotlib · Statistical Inference

Repository

ML Reliability · Human-in-the-Loop · MLOps

Built a pre-training audit system that evaluates whether tabular data is sufficiently reliable for baseline modeling before allowing the workflow to continue.

Workflow

Dataset → Profiling → Risk Checks → Human Review → Baselines → MLflow → SHAP → Report

Key Capabilities

• Detects data-quality, target-leakage, class-imbalance, and modeling risks
• Uses deterministic Python for ML calculations and audit decisions
• Pauses risky workflows at a human review gate
• Compares baseline models instead of pretending to be AutoML
• Tracks experiments with MLflow
• Provides explainability using SHAP
• Uses the LLM only for grounded explanations, reports, and Q&A
• Includes automated pytest coverage, API serving, and Docker packaging

Stack Python · scikit-learn · LangGraph · MLflow · SHAP · FastAPI · Streamlit · Docker

Repository · Live App · Walkthrough

Hybrid Retrieval · Exact Analytics · Grounded AI

Built an enterprise document assistant that separates exact structured analytics from semantic document retrieval instead of forcing every query through a single RAG path.

Structured Queries

CSV / Excel → Query Router → Pandas Analytics → Exact Result

Semantic Queries

Documents → BGE + BM25 → Fusion → CrossEncoder → Grounded Answer + Citations

Key Capabilities

• Supports PDF, DOCX, CSV, JSON, TXT, XLS, and XLSX
• Combines BGE dense retrieval with BM25 lexical retrieval
• Adds query expansion, fusion, deduplication, and CrossEncoder reranking
• Routes structured questions to deterministic Pandas analytics
• Preserves source evidence, citations, and fallback behavior
• Uses a FastAPI backend with React + Vite frontend
• Supports Docker workflows and feedback persistence

29-case strict hybrid RAGAS evaluation

• Composite: 0.947
• Faithfulness: 0.966
• Context Precision: 1.000
• Context Recall: 1.000
• Tier: PRODUCTION_STRONG

A separate 1,642-case production benchmark is documented independently. Final acceptance is intentionally not claimed while its latency gate remains open.

Stack Python · FastAPI · React/Vite · BGE · BM25 · CrossEncoder · ChromaDB · Pandas · Docker

Repository · Walkthrough · Evaluation

Data & Statistics Python · SQL · Pandas · NumPy · PyArrow · Parquet · EDA · A/B Testing · Confidence Intervals · Hypothesis Testing · Forecasting

Machine Learning & NLP scikit-learn · Classification · Regression · Cross-validation · Feature Engineering · Model Evaluation · TF-IDF · Text Classification · Topic Modeling · SHAP

Applied AI LangGraph · Embeddings · BM25 · Hybrid Retrieval · CrossEncoder Reranking · ChromaDB · Grounded Generation · Human-in-the-Loop Workflows

Engineering & MLOps FastAPI · REST APIs · Streamlit · Docker · MLflow · pytest · Ruff · Git · GitHub Actions

Start with the simplest defensible approach and add complexity only when evidence justifies it.

Use holdout performance, uncertainty, baselines, and failure analysis instead of relying on headline metrics alone.

Use code for calculations, validation, and business rules; use LLMs where language understanding or explanation genuinely adds value.

Clearly state what was measured, what remains unverified, and what the system cannot claim.

LeetCode: 511+ problems solved

Arrays · Strings · Hashing · Stack/Queue · Linked Lists · Trees · Heaps · Recursion · SQL

LeetCode Profile

Portfolio · LinkedIn · Kaggle · Docker Hub · YouTube

Email: shivam.x.rajput1308@gmail.com LinkedIn: Shivam Rajput