Skip to content
ย 
ย 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

86 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

๐Ÿ‡ฎ๐Ÿ‡ณ Dev-Store: AI for Bharat

Empowering Indian developers with intelligent, RAG-powered resource discovery

An AI-first marketplace that transforms how developers discover, evaluate, and integrate APIs, models, and datasets through context-aware search with multilingual support for English, Hindi, and Hinglish.


๐Ÿš€ Tech Stack

Next.js FastAPI AWS Bedrock Pinecone Neon AWS Amplify

Frontend: Next.js 16 (App Router) โ€ข TypeScript โ€ข Tailwind CSS โ€ข NextAuth
Backend: FastAPI โ€ข Python 3.11 โ€ข Pydantic โ€ข Mangum (Lambda)
AI/ML: AWS Bedrock (Claude 3.5 Sonnet) โ€ข Titan Embeddings v2
Database: Neon (Serverless PostgreSQL) โ€ข Pinecone (Serverless Vector Store)
Infrastructure: AWS Amplify โ€ข EC2 โ€ข Redis โ€ข S3


โœจ Key Features

๐ŸŽฏ Dual-Layer Filtering

Advanced search combining SQL metadata filtering (category, pricing, popularity) with vector similarity ranking. SQL filters narrow the dataset, then vector search ranks by semantic relevance.

๐Ÿค– RAG-Powered Chat Assistant

Context-aware AI assistant using AWS Bedrock Claude 3.5 Sonnet that understands developer intent and provides personalized resource recommendations with code examples.

๐Ÿ“Š Multi-Source Ingestion Pipeline

Automated data pipeline ingesting from:

  • GitHub: 654+ repositories and developer tools
  • HuggingFace: 1,100+ ML models and datasets
  • Kaggle: 40+ structured datasets
  • OpenRouter: 346+ LLM models with pricing

โš–๏ธ Score Normalization Engine

Custom ranking algorithm normalizing scores across different platforms using composite metrics: relevance, popularity, optimization, and freshness with unified rank_score computation.

๐ŸŒ AI for Bharat Mission

Native support for English, Hindi, and Hinglish queries with intelligent language detection and culturally-aware response matching for India's 5M+ developers.


๐Ÿ—๏ธ Architecture

Serverless-First Approach with hybrid deployment strategy:

graph TB
    A[Next.js 16 Frontend<br/>AWS Amplify] --> B[FastAPI Gateway<br/>EC2 + Gunicorn]
    B --> C[Neon PostgreSQL<br/>Serverless Metadata]
    B --> D[Pinecone Vector DB<br/>Serverless Search]
    B --> E[AWS Bedrock<br/>Claude 3.5 + Titan]
    B --> F[Redis Cache<br/>ElastiCache]
    G[Ingestion Pipeline<br/>Python Orchestrator] --> C
    G --> D
    G --> E
Loading

Key Design Decisions:

  • Frontend: Amplify for global CDN and auto-scaling
  • Backend: EC2 for cost optimization with reserved instances
  • Database: Neon for serverless PostgreSQL with connection pooling
  • Vector Store: Pinecone for sub-100ms semantic search
  • AI: Bedrock for enterprise-grade LLM access
  • Cache: Redis for distributed locks and embedding cache

๐Ÿ“‚ Project Structure

dev-store-ai-bharat/
โ”œโ”€โ”€ backend/                    # FastAPI backend
โ”‚   โ”œโ”€โ”€ main.py                # Application entry point
โ”‚   โ”œโ”€โ”€ api_gateway.py         # EC2 deployment gateway
โ”‚   โ”œโ”€โ”€ config.py              # Configuration management
โ”‚   โ”œโ”€โ”€ clients/               # AWS Bedrock & Database clients
โ”‚   โ”œโ”€โ”€ routers/               # API route handlers
โ”‚   โ”‚   โ”œโ”€โ”€ search.py          # Dual-layer search endpoint
โ”‚   โ”‚   โ”œโ”€โ”€ rag.py             # RAG chat assistant
โ”‚   โ”‚   โ”œโ”€โ”€ resources.py       # Resource management
โ”‚   โ”‚   โ””โ”€โ”€ auth.py            # Authentication
โ”‚   โ”œโ”€โ”€ services/              # Business logic
โ”‚   โ”‚   โ”œโ”€โ”€ search.py          # Search orchestration
โ”‚   โ”‚   โ”œโ”€โ”€ embeddings.py      # Vector operations
โ”‚   โ”‚   โ””โ”€โ”€ ranking.py         # Score normalization
โ”‚   โ”œโ”€โ”€ models/                # Pydantic data models
โ”‚   โ”œโ”€โ”€ ingestion/             # Multi-source pipeline
โ”‚   โ”‚   โ”œโ”€โ”€ orchestrator_production.py  # Main pipeline
โ”‚   โ”‚   โ”œโ”€โ”€ fetchers/          # Source-specific harvesters
โ”‚   โ”‚   โ”œโ”€โ”€ services/          # Chunking, embedding, ranking
โ”‚   โ”‚   โ”œโ”€โ”€ stages/            # Atomic pipeline stages
โ”‚   โ”‚   โ””โ”€โ”€ INGESTION_GUIDE.md # Pipeline documentation
โ”‚   โ”œโ”€โ”€ migrations/            # Database schema
โ”‚   โ””โ”€โ”€ tests/                 # Test suite
โ”œโ”€โ”€ frontend/                  # Next.js 16 frontend
โ”‚   โ”œโ”€โ”€ app/                   # App Router pages
โ”‚   โ”œโ”€โ”€ components/            # React components
โ”‚   โ”‚   โ””โ”€โ”€ DevStoreDashboard.jsx  # Main dashboard
โ”‚   โ”œโ”€โ”€ lib/                   # API client utilities
โ”‚   โ””โ”€โ”€ auth.ts                # NextAuth configuration
โ”œโ”€โ”€ docs/                      # Documentation
โ””โ”€โ”€ .kiro/specs/               # Feature specifications

๐Ÿš€ Quick Start

1๏ธโƒฃ Clone & Setup

git clone https://github.com/your-org/dev-store-ai-bharat.git
cd dev-store-ai-bharat

2๏ธโƒฃ Environment Configuration

# Backend
cp backend/.env.example backend/.env
# Frontend  
cp frontend/.env.local.example frontend/.env.local
# Configure your API keys (see .env files for details)

3๏ธโƒฃ Start Development

# Terminal 1 - Backend
cd backend && pip install -r requirements.txt
uvicorn main:app --reload --port 8000

# Terminal 2 - Frontend
cd frontend && npm install && npm run dev

# Or use the convenience script
./start_all.sh

๐ŸŒ Access: Frontend at http://localhost:3000 โ€ข API docs at http://localhost:8000/docs


๐Ÿš€ Deployment

Production Architecture

  • Frontend: AWS Amplify with global CDN (configured via amplify.yml)
  • Backend: EC2 with Application Load Balancer (see backend/deploy.sh)
  • Database: Neon PostgreSQL with connection pooling
  • Monitoring: Custom health checks and systemd services

Deployment Commands

# Frontend (Amplify auto-deploys on push)
git push origin main

# Backend (EC2 deployment)
cd backend && ./deploy.sh

๐Ÿ“Š Performance Metrics

  • Search Latency: <200ms average response time
  • Vector Search: Sub-100ms with Pinecone serverless optimization
  • Pipeline Throughput: 10-15 resources/sec (Bedrock embedding limited)
  • Full Sync: 3-5 minutes for 2,500+ resources
  • Scale: Optimized for 100K+ resource tier

๐Ÿ› ๏ธ API Endpoints

Endpoint Method Description
/api/v1/search POST Dual-layer semantic search (SQL + Vector)
/api/v1/rag/chat POST RAG-powered AI assistant
/api/v1/resources GET Resource listing with filters
/api/v1/resources/{id} GET Resource details with metadata
/api/v1/health GET System health and metrics

๐Ÿ”„ Ingestion Pipeline

Run the automated multi-source ingestion:

cd backend

# Production pipeline (requires AWS credentials)
python ingestion/run_production.py

# Development pipeline (JSON output only)
python ingestion/run_ingestion.py

Pipeline Stages: Fetch โ†’ Normalize โ†’ Deduplicate โ†’ Upsert (Neon) โ†’ Embed (Bedrock) โ†’ Upsert (Pinecone) โ†’ Rank โ†’ Cache

See backend/ingestion/INGESTION_GUIDE.md for detailed documentation.


๐Ÿ† Built for AI4 Bharat Hackathon

Team Dev-Store-AI-Bharat:

  • Mohd Arsh - AI/ML Engineering (RAG, Embeddings, Multilingual)
  • Raunak - Data Engineering (Pipelines, Infrastructure, Monitoring)
  • Vansh - Frontend Engineering (UI/UX, Performance, Accessibility)
  • Aryan - Backend Engineering (APIs, DevOps, Architecture)

๐Ÿ“š Documentation


๐Ÿ“„ License

Apache License 2.0 - Built with โค๏ธ for the Indian developer community


About

A Google Play Store-inspired marketplace that enables developers to discover, evaluate, and integrate APIs, Models, and Datasets through intelligent.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages