Implement modern storage service with namespace isolation, multipart uploads, and audit logging

.env.example

@@ -1,6 +1,31 @@
+# Application
 PORT=4000
-S3_BUCKET=test
-S3_REGION=auto
-S3_ENDPOINT=https://storage.vdaily.app
-S3_ACCESS_KEY=UwdHMXbJkKmukM9xvxP7
-S3_SECRET_KEY=ZdCe3VVUKC8Ok3GCoY1H56zpH8McMuMgVP8lUgZo
+NODE_ENV=development
+
+# S3/MinIO Storage Configuration
+S3_BUCKET=uploads
+S3_REGION=us-east-1
+S3_ENDPOINT=http://localhost:9000
+S3_ACCESS_KEY=minioadmin
+S3_SECRET_KEY=minioadmin
+S3_FORCE_PATH_STYLE=true
+
+# PostgreSQL Database Configuration
+DATABASE_HOST=localhost
+DATABASE_PORT=5432
+DATABASE_USERNAME=storage
+DATABASE_PASSWORD=storage123
+DATABASE_NAME=storage
+DATABASE_SYNCHRONIZE=true
+DATABASE_LOGGING=false
+
+# Redis Configuration
+REDIS_HOST=localhost
+REDIS_PORT=6379
+REDIS_PASSWORD=
+REDIS_DB=0
+
+# Bull Queue Configuration
+BULL_REDIS_HOST=localhost
+BULL_REDIS_PORT=6379
+BULL_REDIS_PASSWORD=

README.md

@@ -104,3 +104,104 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 ## Support
 
 For support, please open an issue in the GitHub repository.
+
+## Modern Storage Service Features
+
+This service now includes the following capabilities:
+
+### Core Features
+- **Namespace Management**: Create and manage isolated namespaces with quotas
+- **Object Storage**: Store objects with metadata and generate signed URLs
+- **Multipart Uploads**: Resumable uploads for large files (up to 100MB)
+- **Audit Logging**: Track all CREATE, READ, and DELETE operations
+- **Health Checks**: Liveness and readiness endpoints
+
+### API Endpoints
+
+Base URL: `http://localhost:4000/api/v1`
+
+#### Namespaces
+- `POST /namespaces` - Create a namespace
+- `GET /namespaces/:name` - Get namespace details
+
+#### Objects
+- `POST /objects` - Create object and get upload URL
+- `GET /objects/:id` - Get object metadata
+- `POST /objects/:id/signed-url` - Generate download URL
+- `DELETE /objects/:id` - Delete object
+- `GET /namespaces/:namespace/objects` - List objects in namespace
+
+#### Multipart Uploads
+- `POST /uploads/initiate` - Start multipart upload
+- `PUT /uploads/:sessionId/parts/:partNumber` - Upload a part
+- `POST /uploads/:sessionId/complete` - Complete upload
+
+#### Health
+- `GET /health/live` - Liveness check
+- `GET /health/ready` - Readiness check (DB + S3)
+
+### Quick Start
+
+1. Install dependencies:
+```bash
+npm install
+```
+
+2. Start services with Docker Compose:
+```bash
+docker-compose up -d postgres redis minio
+```
+
+3. Copy environment file:
+```bash
+cp .env.example .env
+```
+
+4. Run the application:
+```bash
+npm run dev
+```
+
+5. Access Swagger documentation:
+```
+http://localhost:4000/api
+```
+
+### Example Usage
+
+#### Create a Namespace
+```bash
+curl -X POST http://localhost:4000/api/v1/namespaces \
+  -H "Content-Type: application/json" \
+  -d '{
+    "name": "org-demo",
+    "displayName": "Demo Organization",
+    "quotaBytes": 1073741824
+  }'
+```
+
+#### Create an Object
+```bash
+curl -X POST http://localhost:4000/api/v1/objects \
+  -H "Content-Type: application/json" \
+  -d '{
+    "namespace": "org-demo",
+    "name": "test-file.pdf",
+    "contentType": "application/pdf",
+    "sizeBytes": 1024000
+  }'
+```
+
+The response will include a pre-signed `uploadUrl` that you can use to upload the file directly to S3/MinIO storage using an HTTP PUT request.
+
+### Database Schema
+
+The service uses PostgreSQL with TypeORM. Tables are auto-created on startup:
+- `namespaces` - Namespace/tenant information
+- `objects` - Object metadata
+- `upload_sessions` - Multipart upload sessions
+- `audit_events` - Audit trail
+
+### Configuration
+
+See `.env.example` for all configuration options.

docker-compose.yml

@@ -1,5 +1,35 @@
 version: '3.8'
 services:
+  postgres:
+    image: postgres:16-alpine
+    restart: unless-stopped
+    ports:
+      - 5432:5432
+    environment:
+      - POSTGRES_USER=storage
+      - POSTGRES_PASSWORD=storage123
+      - POSTGRES_DB=storage
+    volumes:
+      - postgres-data:/var/lib/postgresql/data
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U storage"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+
+  redis:
+    image: redis:7-alpine
+    restart: unless-stopped
+    ports:
+      - 6379:6379
+    volumes:
+      - redis-data:/data
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 10s
+      timeout: 3s
+      retries: 5
+
   minio:
     image: quay.io/minio/minio
     restart: on-failure
@@ -9,22 +39,34 @@ services:
     command: server /data --console-address ":9090"
     volumes:
       - minio-data:/data
-    env_file:
-      - .env
     environment:
-      - MINIO_ROOT_USER=ADMIN
-      - MINIO_ROOT_PASSWORD=ADMIN@123
+      - MINIO_ROOT_USER=minioadmin
+      - MINIO_ROOT_PASSWORD=minioadmin
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
+      interval: 30s
+      timeout: 20s
+      retries: 3
+
   storage:
     image: ghcr.io/pnstack/nestjs-storage-service:release
     container_name: nestjs-storage-service
     depends_on:
-      - minio
+      postgres:
+        condition: service_healthy
+      redis:
+        condition: service_healthy
+      minio:
+        condition: service_healthy
     restart: on-failure
     ports:
       - 4006:4000
     volumes:
       - ./uploads:/app/uploads
     env_file:
-      - .env  
+      - .env
+
 volumes:
+  postgres-data:
+  redis-data:
   minio-data: