MNIST Dimensionality Reduction Benchmark

A comprehensive study of Principal Component Analysis (PCA), Kernel PCA, and t-SNE on the MNIST handwritten digit dataset.

This project investigates how dimensionality reduction affects:

• Feature Compression
• Classification Performance
• Image Reconstruction
• Nonlinear Visualization

---

Project Workflow

MNIST Dataset
      │
      ▼
Exploratory Data Analysis
      │
      ▼
Baseline Logistic Regression
      │
      ▼
Principal Component Analysis
      │
      ├──────────────► Explained Variance Analysis
      │
      ├──────────────► Compression Benchmark
      │
      ├──────────────► Image Reconstruction
      │
      ▼
Kernel PCA
      │
      ▼
t-SNE Visualization
      │
      ▼
Final Benchmark & Conclusions

---

Techniques Used

• Principal Component Analysis (PCA)
• Kernel PCA (RBF & Polynomial)
• t-SNE
• Logistic Regression
• Image Reconstruction
• Compression Benchmarking
• Data Visualization

---

Repository Structure

mnist-dimensionality-reduction/
│
├── notebook/
│     MNIST_Dimensionality_Reduction.ipynb
│
├── images/
│     explained_variance.png
│     cumulative_variance.png
│     image_reconstruction.png
│     pca_vs_tsne.png
│     kernel_pca_comparision.png
│
├── requirements.txt
├── LICENSE
└── README.md

---

Project Visualizations

Explained Variance

---

Cumulative Explained Variance

---

Image Reconstruction

---

PCA vs t-SNE

---

Kernel PCA Comparison

---

Key Findings

• PCA reduced the original 784-dimensional feature space while preserving 95% of the dataset variance.
• Significant dimensionality reduction resulted in only a minor change in classification accuracy.
• Image reconstruction demonstrated that PCA preserves most visual information even after aggressive compression.
• Kernel PCA captured nonlinear structures more effectively than linear PCA.
• t-SNE produced the clearest visualization of handwritten digit clusters.

---

Summary

Technique	Purpose	Linear	Reconstruction	Best Use
Logistic Regression	Baseline Classification	✔	—	Prediction
PCA	Compression	✔	✔	Feature Reduction
Kernel PCA (RBF)	Nonlinear Reduction	✘	Approximate	Curved Manifolds
Kernel PCA (Polynomial)	Nonlinear Reduction	✘	Approximate	Polynomial Relationships
t-SNE	Visualization	✘	✘	Cluster Visualization

---

Installation

Clone the repository

git clone https://github.com/yourusername/mnist-dimensionality-reduction.git

cd mnist-dimensionality-reduction

Install dependencies

pip install -r requirements.txt

---

Running the Notebook

Launch Jupyter Notebook

jupyter notebook

Open

notebook/MNIST_Dimensionality_Reduction.ipynb

Run all cells.

---

Future Improvements

• Compare PCA with UMAP
• Incremental PCA for large datasets
• Autoencoder-based dimensionality reduction
• Truncated SVD Benchmark
• Deep Feature Compression

---

Libraries

• NumPy
• Pandas
• Matplotlib
• Scikit-Learn
• Jupyter Notebook

---

If you found this project useful

Please consider giving this repository a ⭐.