Image Caption Generator using Deep Learning

An AI-powered Image Caption Generator that automatically generates meaningful natural language descriptions for images using Computer Vision and Deep Learning. The project combines a CNN encoder (VGG16) for image feature extraction with an LSTM decoder for caption generation.

---

📌 Project Overview

Image Captioning is a multimodal AI task that combines Computer Vision and Natural Language Processing (NLP). The model learns visual features from images and generates human-like captions describing the content of an image.

This project uses the Flickr8k Dataset for training and evaluation.

---

✨ Features

• 🖼️ Automatic image caption generation
• 🧠 Deep Learning-based encoder-decoder architecture
• 📷 VGG16 for image feature extraction
• 💬 LSTM for sequence generation
• 📝 Text preprocessing and tokenization
• 📊 BLEU Score evaluation
• 📈 Training using TensorFlow/Keras
• 🔍 Visualize generated captions on test images

---

🛠️ Tech Stack

Category	Technologies
Language	Python
Deep Learning	TensorFlow, Keras
Computer Vision	OpenCV, VGG16
NLP	Tokenizer, Padding Sequences
Data Processing	NumPy, Pickle
Visualization	Matplotlib
Dataset	Flickr8k

---

🧠 Model Architecture

Input Image
      │
      ▼
Pretrained VGG16
(Image Feature Extraction)
      │
      ▼
4096-D Feature Vector
      │
      ▼
Dense + Dropout
      │
      ▼
Embedding Layer
      │
      ▼
LSTM Decoder
      │
      ▼
Generated Caption

---

📂 Dataset

Dataset: Flickr8k

The dataset contains:

• 8,000 images
• 5 captions for each image
• Thousands of unique vocabulary words

---

⚙️ Workflow

1. Load Flickr8k dataset
2. Extract image features using VGG16
3. Clean and preprocess captions
4. Tokenize text
5. Create vocabulary
6. Generate training sequences
7. Build CNN-LSTM model
8. Train the model
9. Generate captions for unseen images
10. Evaluate using BLEU Score

---

📊 Evaluation

The model performance is evaluated using the BLEU (Bilingual Evaluation Understudy) Score, which compares generated captions with the ground truth captions.

Evaluation metrics include:

• BLEU-1
• BLEU-2
• BLEU-3
• BLEU-4

---

📁 Project Structure

Image-Caption-Generator/
│
├── Image_caption_generator.ipynb
├── Flickr8k_Dataset/
├── captions.txt
├── features.pkl
├── best_model.h5
├── README.md
└── requirements.txt

---

🚀 Installation

Clone the repository:

git clone https://github.com/yourusername/Image-Caption-Generator.git

Move into the project folder:

cd Image-Caption-Generator

Install the required packages:

pip install -r requirements.txt

Run the Jupyter Notebook:

jupyter notebook

---

📸 Sample Output

Input Image

(Add a sample image here)

Generated Caption

"A young boy is playing football on a grassy field."

---

🎯 Applications

• Image Understanding
• Accessibility for Visually Impaired Users
• Smart Photo Albums
• Image Search
• Digital Asset Management
• AI-powered Content Generation

---

🔮 Future Improvements

• Replace VGG16 with EfficientNet or Vision Transformer (ViT)
• Use Attention Mechanism
• Integrate Transformer-based Caption Generator
• Deploy using Streamlit or Flask
• Support real-time image uploads
• Improve caption quality using larger datasets

---

👨‍💻 Author

Sanjai K

• 🎓 MCA Graduate (2026)
• 🤖 AI / Machine Learning Engineer
• 📚 IEEE Conference Published Researcher

Portfolio: https://sanjaikmca.github.io/Sanjai_Portfolio/

LinkedIn: https://www.linkedin.com/in/sanjaikmca

GitHub: https://github.com/sanjaikmca

---

⭐ If you found this project useful, don't forget to Star the repository!