Image Processing From CSV

Table of Contents

• Introduction
• Features
• Tech Stack
• Installation
• Configuration
• Running the Application
• API Documentation
  • Upload API
  • Status API
  • Webhook API
• Database Schema
  • Request Collection
  • Product Collection
• Asynchronous Workers Documentation
• LLD Draw.io Diagram PDF

Introduction

This project is an image processing system designed to handle CSV files containing product information and associated image URLs. The system performs the following tasks:

1. Receives and validates CSV files.
2. Asynchronously processes images (compressing them by 50%).
3. Stores the processed image data and associated product information in a database.
4. Provides an API to check the processing status using a unique request ID.
5. Uses webhooks to notify the completion of image processing tasks.

Features

• Asynchronous image processing.
• Validation of CSV file format.
• Database storage of product and request data.
• Webhook notifications upon completion of processing.
• Clear API endpoints for uploading CSV files and checking processing status.

Tech Stack

• Backend: Node.js, Express.js
• Database: MongoDB
• Image Processing: Sharp
• HTTP Requests: Axios

Installation

1. Clone the repository:

   git clone https://github.com/Adithyasharma1162/image-processing-from-csv.git
   cd image-processing-system
   

Configuration

1. MongoDB: Set up your MongoDB connection string in the .env file.
2. Port: Configure the server port in the .env file.
3. Webhook URL: Specify the webhook URL to receive notifications upon completion of image processing.

Running the Application

1. Start the server:

   npm start

2. Run the worker script for processing images:

   node workers/processImages.js

API Documentation

1: API Documentation PDF Provided in Repository

Database Schema

1: Database Schema PDF provided in Repository

Asynchronous Workers Documentation

1: Asynchronous Workers Documentation PDF provided in Repository

LLD Draw IO

Click the image below to view LLD.pdf